System and method for real-time control of autonomous devices

Abstract

The invention relates to a system and method for real-time control of autonomous devices. An autonomous vehicle has sensors that advantageously vary in capability, advantageously locate, and advantageously are unaffected by environmental conditions. A system executing on an autonomous vehicle is capable of receiving a map including, for example, substantially discontinuous surface features and data from sensors, creating an occupancy grid based on the map and the data, and changing a configuration of the autonomous vehicle based on a type of surface on which the autonomous vehicle navigates. The device is capable of safely navigating surfaces and surface features, including traversing discontinuous surfaces and other obstacles.

Description

[0001] This application is a divisional application, and the original application it is targeting is a Chinese invention patent with an application date of July 10, 2020, the applicant is "Deka Products Co., Ltd.", the invention name is "System and method for real-time control of autonomous equipment", and the application number is 202080058758.4.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This patent application is a continuation-in-part of U.S. Patent Application No. 16 / 800,497, filed on February 25, 2020, entitled System and Method for Surface Feature Detection and Traversal (Attorney Docket No. AA164), which is incorporated herein by reference in its entirety. This patent application claims the benefit of U.S. Provisional Patent Application Serial No. 62 / 872,396, filed on July 10, 2019, entitled Apparatus for Long and Short Range Sensors on an Autonomous Delivery Device (Attorney Docket No. AA028), U.S. Provisional Patent Application Serial No. 62 / 990,485, filed on March 17, 2020, entitled System and Method for Managing an Occupancy Grid (Attorney Docket No. AA037), and U.S. Provisional Patent Application Serial No. 62 / 872,320, filed on July 10, 2019, entitled System and Method for Real-Time Control of the Configuration of an Autonomous Device (Attorney Docket No. Z96).

[0004] This application is related to U.S. patent application serial number 16 / 035,205, filed on July 13, 2018, entitled MOBILITY DEVICE (Attorney Docket No. X80), U.S. patent application serial number 15 / 787,613, filed on October 18, 2017, entitled MOBILITY DEVICE (Attorney Docket No. W10), U.S. patent application serial number 15 / 600,703, filed on May 20, 2017, entitled MOBILITY DEVICE (Attorney Docket No. U22), U.S. patent application serial number 15 / 982,737, filed on May 17, 2018, entitled SYSTEM AND METHOD FOR SECURE REMOTE CONTROL OF A MEDICAL DEVICE (Attorney Docket No. X5 ... IMPROVEMENTS, U.S. Provisional Application Serial No. 62 / 532,993 (Attorney Docket No. U30), entitled MOBILITY DEVICE SEAT, filed on September 15, 2017 (Attorney Docket No. V85), and U.S. Provisional Application Serial No. 62 / 581,670 (Attorney Docket No. W07), entitled MOBILITY DEVICE SEAT, filed on November 4, 2017, which are incorporated herein by reference in their entirety. Technical Field

[0005] The present teachings relate generally to AVs, and more specifically to autonomous route planning, global occupancy grid management, onboard sensors, surface feature detection and traversal, and real-time vehicle configuration changes. Background Art

[0006] Navigation of AVs and semi-autonomous vehicles (AVs) typically relies on long-range sensors including, for example, but not limited to, LIDAR, cameras, stereo cameras, and radars. Long-range sensors are capable of sensing distances between 4 and 100 meters from the AV. In contrast, object avoidance and / or surface detection typically rely on short-range sensors including, for example, but not limited to, stereo cameras, short-range radars, and ultrasonic sensors. These short-range sensors typically observe an area or volume of about 5 meters outside the AV. Sensors enable, for example, directional AVs within their environment and navigate through streets, sidewalks, obstacles, and open spaces to reach a desired destination. Sensors can also enable imagining humans, signs, traffic lights, obstacles, and surface features.

[0007] Surface feature traversal can be challenging because surface features, such as, but not limited to, substantially discontinuous surface features (SDSFs), can be found in heterogeneous topologies, and the topology can be unique to a particular geography. However, SDSFs, such as, but not limited to, ramps, edges, curbs, steps, and curb-like geometries (referred to herein in a non-limiting manner as SDSFs or simply surface features), can include some typical characteristics that can aid in their identification. Surface / road conditions and surface types can be identified and classified, for example, by fusing multi-sensory data, which can be complex and expensive. Surface features and conditions can be used to control the physical reconfiguration of the AV in real time.

[0008] Sensors can be used to enable the creation of an occupancy grid that can represent the world for path planning purposes for an AV. Path planning requires a grid that identifies spaces as free, occupied, or unknown. However, the probability that a space is occupied can improve decisions about the space. The log-odds representation of the probability can be used to improve the accuracy at the numerical boundaries of the probabilities of 0 and 1. The probability that a cell is occupied may depend on at least new sensor information, previous sensor information, and prior occupancy information.

[0009] What is needed is a system that combines collected sensor data and real-time sensor data with changes in physical configuration changes of a vehicle to enable variable terrain traversal. What is needed is advantageous sensor placement for enabling physical configuration changes, variable terrain traversal, and object avoidance. What is needed is the ability to locate a SDSF based on a multi-part model associated with several criteria for SDSF identification. What is needed is the determination of candidate surface feature traversals based on criteria such as candidate traversal approach angles, candidate traversal driving surfaces on both sides of the candidate surface feature, and real-time determination of candidate traversal path obstacles. What is needed is a system and method for incorporating drivable surface and device mode information into occupancy grid determination. Summary of the invention

[0010] The AV of the present teachings can autonomously navigate to a desired location. In some configurations, the AV can include: a sensor; a device controller including a perception subsystem, an autonomous subsystem, and a drive subsystem; a powered base; four powered wheels; two casters and a cargo container. In some configurations, the perception and autonomous subsystems can receive and process sensor information (perception) and map information (perception and autonomy), and can provide directions to the drive subsystem. The map information can include surface classifications and related device modes. The movement of the AV controlled by the drive subsystem and achieved by the powered base can be sensed by the sensor subsystem, thereby providing a feedback loop. In some configurations, the SDSF can be accurately identified and memorized in a map based on point cloud data, for example, according to the process described herein. The portion of the map associated with the location of the AV can be provided to the AV during navigation. The perception subsystem can maintain an occupancy grid that can inform the AV of the probability that the path to be traversed is currently occupied. In some configurations, the AV can operate in a variety of fundamentally different modes. These modes can achieve complex terrain traversal and other benefits. A combination of maps (e.g., surface classification), sensor data (sensing features around the AV), occupancy grid (probability of upcoming waypoints being occupied), and mode (is it ready to traverse difficult terrain) can be used to identify the AV’s direction, configuration, and speed.

[0011] With respect to preparing a map, in some configurations, a method of the present teachings is used to create a map to navigate at least one SDSF encountered by an AV, wherein the AV travels a path on a surface, wherein the surface includes at least one SDSF, and wherein the path includes a start point and an end point, the method can include, but is not limited to: accessing point cloud data representing the surface; filtering the point cloud data; forming the filtered point cloud data into processable portions; and merging the processable portions into at least one concave polygon. The method can include locating and marking at least one SDSF in at least one concave polygon. Locating and marking can form the marked point cloud data. The method can include: creating a graphic polygon based at least on the at least one concave polygon; and selecting a path from the start point to the end point based at least on the graphic polygon. When navigating, the AV can traverse the at least one SDSF along the path.

[0012] Filtering the point cloud data can optionally include conditionally removing points representing transient objects and points representing outliers from the point cloud data, and replacing the removed points with a preselected height. Forming the processing portion can optionally include segmenting the point cloud data into processable portions, and removing points of preselected heights from the processable portions. Merging the processable portions can optionally include reducing the size of the processable portions by analyzing outliers, voxels, and normals, growing regions from the reduced-size processable portions, determining an initial drivable surface based on the grown regions, segmenting and meshing the initial drivable surface, locating polygons within the segmented and meshed initial drivable surface, and setting the drivable surface based on at least the polygons. Locating and marking at least one SDSF feature can optionally include classifying the point cloud data of the drivable surface according to an SDSF filter, the SDSF filter including at least three categories of points, and locating at least one SDSF point based at least on whether the points of the categories meet at least one first preselected criterion in combination. The method can optionally include creating at least one SDSF track based at least on whether the plurality of at least one SDSF points in combination satisfy at least one second preselected criterion. Creating the graphic polygon can further optionally include creating at least one polygon from at least one drivable surface. The at least one polygon can include edges. Creating the graphic polygon can include: smoothing the edges; forming a driving margin based on the smoothed edges; adding at least one SDSF track to at least one drivable surface; and removing edges from at least one drivable surface based on at least one third preselected criterion. Smoothing the edges can optionally include trimming the edges outwardly. Forming the driving margin of the smoothed edges can optionally include trimming the outward edges inwardly.

[0013] In some configurations, a system of the present teachings is used to create a map for navigating at least one SDSF encountered by an AV, wherein the AV travels a path on a surface, wherein the surface includes at least one SDSF, wherein the path includes a start point and a destination, the system can include, but is not limited to, including: a first processor that accesses point cloud data representing the surface; a first filter that filters the point cloud data; a second processor that forms a processable portion from the filtered point cloud data; a third processor that merges the processable portion into at least one concave polygon; a fourth processor that locates and marks at least one SDSF in the at least one concave polygon, the location and marking forming the marked point cloud data; a fifth processor that creates a graphic polygon; and a path selector that selects a path from the start point to the destination based at least on the graphic polygon. The AV can traverse the at least one SDSF along the path.

[0014] The first filter can optionally include executable code, which can include, but is not limited to conditionally removing points representing transient objects and points representing outliers from the point cloud data, and replacing the removed points with preselected heights. The segmenter can optionally include executable code, which can include, but is not limited to segmenting the point cloud data into processable portions, and removing points of preselected heights from the processable portions. The third processor can optionally include executable code, which can include, but is not limited to reducing the size of the processable portions by analyzing outliers, voxels, and normals, growing regions from the reduced-size processable portions, determining an initial drivable surface based on the grown regions, segmenting and meshing the initial drivable surface, locating polygons within the segmented and meshed initial drivable surface, and setting the drivable surface based on at least the polygons. The fourth processor can optionally include executable code, which can include, but is not limited to, classifying the point cloud data of the drivable surface according to an SDSF filter, the SDSF filter including at least three categories of points, and locating at least one SDSF point based at least on whether the points of the categories meet at least one first preselected criterion in combination. The system can optionally include executable code, which can include, but is not limited to, creating at least one SDSF track based at least on whether a plurality of at least one SDSF point meet at least one second preselected criterion in combination.

[0015] Creating a graphics polygon can optionally include executable code that can include, but is not limited to, including: creating at least one polygon from at least one drivable surface, the at least one polygon including edges; smoothing the edges; forming a driving margin based on the smoothed edges; adding at least one SDSF track to the at least one drivable surface; and removing edges from the at least one drivable surface according to at least one third preselected criterion. Smoothing the edges can optionally include executable code that can include, but is not limited to, including trimming the edges outward. Forming a driving margin for the smoothed edges can optionally include executable code that can include, but is not limited to, including trimming the outward edges inward.

[0016] In some configurations, a method of the present teachings is used to create a map for navigating at least one SDSF encountered by an AV, wherein the AV drives a path on a surface, wherein the surface includes at least one SDSF, wherein the path includes a start point and a destination point, the method can include, but is not limited to, accessing a route topology. The route topology can include at least one graphic polygon, and the at least one graphic polygon can include filtered point cloud data. The point cloud data can include marked features and drivable margins. The method can include: transforming the point cloud data into a global coordinate system; determining the boundary of at least one SDSF; creating an SDSF buffer of a preselected size around the boundary; determining which of the at least one SDSF can be traversed based on at least one SDSF traversal criterion; creating an edge / weight graph based on at least at least one SDSF traversal criterion, the transformed point cloud data, and the route topology; and selecting a path from the start point to the destination point based on at least the edge / weight graph.

[0017] The at least one SDSF crossing criteria can optionally include a preselected width of the at least one SDSF and a preselected smoothness of the at least one SDSF, a minimum entry distance and a minimum exit distance between the at least one SDSF and the AV including the drivable surface, and a minimum entry distance between the at least one SDSF and the AV that allows the AV to approach the at least one SDSF at approximately 90°.

[0018] In some configurations, a system of the present teachings is used to create a map for navigating at least one SDSF encountered by an AV, wherein the AV drives a path on a surface, wherein the surface includes at least one SDSF, and wherein the path includes a start point and a destination point, the system can include, but is not limited to, a sixth processor that accesses a route topology. The route topology can include at least one graphic polygon, and the at least one graphic polygon can include filtered point cloud data. The point cloud data can include marked features and drivable margins. The system can include: a seventh processor that transforms the point cloud data into a global coordinate system; and an eighth processor that determines the boundaries of at least one SDSF. The eighth processor can create an SDSF buffer of a preselected size around the boundary. The system can include: a ninth processor that determines which of the at least one SDSF can be traversed based at least on at least one SDSF traversal criterion; a tenth processor that creates an edge / weight graph based at least on at least one SDSF traversal criterion, the transformed point cloud data, and the route topology; and a base controller that selects a path from the start point to the destination point based at least on the edge / weight graph.

[0019] In some configurations, a method of the present teachings is used to create a map for navigating at least one SDSF encountered by an AV, wherein the AV traverses a path on a surface, wherein the surface includes at least one SDSF, and wherein the path includes a start point and an end point, the method can include, but is not limited to, accessing point cloud data representing the surface. The method can include: filtering the point cloud data; forming the filtered point cloud data into processable portions; and merging the processable portions into at least one concave polygon. The method can include locating and marking at least one SDSF in the at least one concave polygon. Locating and marking can form marked point cloud data. The method can include creating a graphic polygon based at least on the at least one concave polygon. The graphic polygon can form a route topology, and the point cloud data can include marked features and drivable margins. The method can include: transforming point cloud data into a global coordinate system; determining a boundary of at least one SDSF; creating an SDSF buffer of a preselected size around the boundary; determining which of the at least one SDSF can be traversed based at least on at least one SDSF traversal criterion; creating an edge / weight graph based at least on at least one SDSF traversal criterion, the transformed point cloud data, and a route topology; and selecting a path from a starting point to a destination point based at least on the edge / weight graph.

[0020] Filtering the point cloud data can optionally include conditionally removing points representing transient objects and points representing outliers from the point cloud data, and replacing the removed points with a preselected height. Forming the processing portion can optionally include segmenting the point cloud data into processable portions, and removing points of preselected heights from the processable portions. Merging the processable portions can optionally include reducing the size of the processable portions by analyzing outliers, voxels, and normals, growing regions from the reduced-size processable portions, determining an initial drivable surface based on the grown regions, segmenting and meshing the initial drivable surface, locating polygons within the segmented and meshed initial drivable surface, and setting the drivable surface based on at least the polygons. Locating and marking at least one SDSF feature can optionally include classifying the point cloud data of the drivable surface according to an SDSF filter, the SDSF filter including at least three categories of points, and locating at least one SDSF point based at least on whether the points of the categories meet at least one first preselected criterion in combination. The method can optionally include creating at least one SDSF track based at least on whether a plurality of at least one SDSF points in combination satisfy at least one second preselected criterion. Creating the graphic polygon can further optionally include creating at least one polygon from at least one drivable surface. At least one polygon can include edges. Creating the graphic polygon can include: smoothing the edges; forming a driving margin based on the smoothed edges; adding at least one SDSF track to at least one drivable surface; and removing edges from at least one drivable surface according to at least one third preselected criterion. Smoothing of the edges can optionally include trimming the edges outward. Forming the driving margin of the smoothed edges can optionally include trimming the outward edges inward. At least one SDSF crossing criterion can optionally include a preselected width of at least one SDSF and a preselected smoothness of at least one SDSF, a minimum entry distance and a minimum exit distance between at least one SDSF and an AV including a drivable surface, and a minimum entry distance between at least one SDSF and the AV that allows the AV to approach the at least one SDSF at approximately 90°.

[0021] In some configurations, a system of the present teachings is used to create a map for navigating at least one SDSF encountered by an AV, wherein the AV drives a path on a surface, wherein the surface includes at least one SDSF, wherein the path includes a start point and an end point, the system can include, but is not limited to: a point cloud accessor that accesses point cloud data representing the surface; a first filter that filters the point cloud data; a segmenter that forms processable portions from the filtered point cloud data; a third processor that merges the processable portions into at least one concave polygon; a fourth processor that locates and marks at least one SDSF in the at least one concave polygon, the location and marking forming marked point cloud data; a fifth processor that creates a graphic polygon. The route topology can include at least one graphic polygon, and the at least one graphic polygon can include the filtered point cloud data. The point cloud data can include marked features and drivable margins. The system can include: a seventh processor that transforms the point cloud data into a global coordinate system; and an eighth processor that determines a boundary of the at least one SDSF. The eighth processor can create a SDSF buffer of a preselected size around the boundary. The system may include: a ninth processor that determines which of the at least one SDSF can be traversed based at least on at least one SDSF traversal criterion; a tenth processor that creates an edge / weight graph based at least on at least one SDSF traversal criterion, the transformed point cloud data, and the route topology; and a base controller that selects a path from a starting point to a destination point based at least on the edge / weight graph.

[0022] The first filter can optionally include executable code, which can include, but is not limited to conditionally removing points representing transient objects and points representing outliers from the point cloud data, and replacing the removed points with preselected heights. The segmenter can optionally include executable code, which can include, but is not limited to segmenting the point cloud data into processable portions, and removing points of preselected heights from the processable portions. The third processor can optionally include executable code, which can include, but is not limited to reducing the size of the processable portions by analyzing outliers, voxels, and normals, growing regions from the reduced-size processable portions, determining an initial drivable surface based on the grown regions, segmenting and meshing the initial drivable surface, locating polygons within the segmented and meshed initial drivable surface, and setting the drivable surface based on at least the polygons. The fourth processor can optionally include executable code, which can include, but is not limited to, classifying the point cloud data of the drivable surface according to an SDSF filter, the SDSF filter including at least three categories of points, and locating at least one SDSF point based at least on whether the points of the categories meet at least one first preselected criterion in combination. The system can optionally include executable code, which can include, but is not limited to, creating at least one SDSF track based at least on whether a plurality of at least one SDSF point meet at least one second preselected criterion in combination.

[0023] Creating a graphics polygon can optionally include executable code that can include, but is not limited to, including: creating at least one polygon from at least one drivable surface, the at least one polygon including edges; smoothing the edges; forming a driving margin based on the smoothed edges; adding at least one SDSF track to the at least one drivable surface; and removing edges from the at least one drivable surface according to at least one third preselected criterion. Smoothing the edges can optionally include executable code that can include, but is not limited to, including trimming the edges outward. Forming a driving margin of the smoothed edges can optionally include executable code that can include, but is not limited to, including trimming the outward edges inward.

[0024] In some configurations, the SDSF can be identified by its size. For example, a curb can include, but is not limited to, a width of about 0.6-0.7 m. In some configurations, the point cloud data can be processed to locate the SDSF, and those data can be used to prepare a path from a starting point to a destination for the AV. In some configurations, the path can be included in a map and provided to the perception subsystem. When the AV is traversing the path, in some configurations, the SDSF can be allowed to traverse by sensor-based positioning of the AV partially enabled by the perception subsystem. The perception subsystem can be executed on at least one processor within the AV.

[0025] The AV can include, but is not limited to, a powered base including two powered front wheels, two powered rear wheels, an energy storage device, and at least one processor. The powered base can be configured to move at a command rate. The AV can include a cargo platform mechanically attached to the powered base, including a plurality of short-range sensors. The AV can include a cargo container mounted on top of the cargo platform in some configurations, thereby having a volume for receiving one or more objects to be delivered. The AV can include a long-range sensor suite mounted on top of the cargo container in some configurations, which can include, but is not limited to, LIDAR and one or more cameras. The AV can include a controller that can receive data from the long-range sensor suite and the short-range sensor suite.

[0026] The short-range sensor suite can optionally detect at least one characteristic of a drivable surface and can optionally include a stereo camera, an IR projector, two image sensors, an RGB sensor, and a radar sensor. The short-range sensor suite can optionally supply RGB-D data to the controller. The controller can optionally determine the geometry of the road surface based on the RGB-D data received from the short-range sensor suite. The short-range sensor suite can optionally detect objects within 4 meters of the AV, while the long-range sensor suite can optionally detect objects more than 4 meters from the AV.

[0027] The perception subsystem can fill in an occupancy grid using data collected by the sensors. The occupancy grid of the present teaching can be configured as a 3D grid of points around the AV, with the AV occupying the center point. In some configurations, the occupancy grid can extend 10m to the left, right, back, and front of the AV. The grid can approximately include the height of the AV, and can actually travel with the AV as it moves, thereby representing obstacles around the AV. The grid can be converted to two dimensions by reducing its vertical axis, and can be divided into polygons, such as but not limited to a size of approximately 5cm x 5cm. Obstacles that appear in the 3D space around the AV can be reduced to 2D shapes. If a 2D shape overlaps any segment of a polygon, the polygon can be assigned a value of 100, indicating that the space is occupied. Any polygon that is not filled can be assigned a value of 0, and can be referred to as a free space in which the AV can move.

[0028] As the AV navigates, it may encounter situations that may require a change in the AV configuration. The method taught herein is for real-time control of the configuration of an AV, comprising a chassis, at least four wheels, a first side of the chassis operably coupled to at least one of the at least four wheels, and an opposing second side of the chassis operably coupled to at least one of the at least four wheels, the method being capable of including, but not limited to, receiving environmental data; determining a surface type based at least on the environmental data; determining a mode based at least on the surface type and a first configuration; determining a second configuration based at least on the mode and the surface type; determining a movement command based at least on the second configuration; and controlling the configuration of the device by using the movement command to change the device from the first configuration to the second configuration.

[0029] The method can optionally include filling an occupancy grid based at least on a surface type and pattern. The environmental data can optionally include RGB-D image data and a topology of a road surface. The configuration can optionally include two pairs of clusters of at least four wheels. The first pair of the two pairs can be positioned on a first side, and the second pair of the two pairs can be positioned on a second side. The first pair can include a first front wheel and a first rear wheel, and the second pair can include a second front wheel and a second rear wheel. The control of the configuration can optionally include coordinated powering of the first pair and the second pair based at least on the environmental data. The control of the configuration can optionally include switching from driving at least four wheels and a pair of retracting casters to driving two wheels, wherein the first pair of the cluster and the second pair of the cluster rotate to raise the first front wheel and the second front wheel. The pair of casters can be operably connected to the chassis. The device can rest on the first rear wheel, the second rear wheel and the pair of casters. The control of the configuration can optionally include rotating a pair of clusters operably connected to two powered wheels on the first side and two powered wheels on the second side based at least on the environmental data.

[0030] The system for real-time control of the configuration of an AV of the present teachings can include, but is not limited to, including a device processor and a powered base processor. The AV can include a chassis, at least four wheels, a first side of the chassis, and an opposite second side of the chassis. The device processor can receive real-time environmental data surrounding the AV, determine a surface type based at least on the environmental data, determine a mode based at least on the surface type and a first configuration, and determine a second configuration based at least on the mode and the surface type. The powered base processor can enable the AV to move based at least on the second configuration, and can enable the AV to change from the first configuration to the second configuration. The device processor can optionally include filling an occupancy grid based at least on the surface type and the mode.

[0031] During navigation, the AV can encounter an SDSF that may require maneuvering the AV to successfully traverse. In some configurations, the method of the present teachings is for navigating the AV along a path line in a travel area toward a target point across at least one SDSF, the AV including a leading edge and a trailing edge can include, but is not limited to, receiving SDSF information and obstacle information of the travel area, detecting at least one candidate SDSF based on the SDSF information, and selecting an SDSF line from at least one candidate SDSF line based on at least one selection criterion. The method can include determining at least one traversable portion of the selected SDSF line based on at least one position of at least one obstacle found in the obstacle information near the selected SDSF line, advancing the AV operated at a first speed toward the at least one traversable portion by turning the AV to travel along a line perpendicular to the traversable portion, and continuously correcting the heading of the AV based on the relationship of the heading to the perpendicular line. The method can include driving the AV at a second speed by adjusting a first speed of the AV based at least on the heading and the distance between the AV and the traversable portion. If the SDSF associated with at least one traversable portion is elevated relative to the surface of the travel route, the method can include traversing the SDSF by elevating a leading edge relative to a trailing edge and driving the AV at a third increasing speed per degree of elevation, and driving the AV at a fourth speed until the AV has cleared the SDSF.

[0032] Detecting at least one candidate SDSF based on the SDSF information can optionally include (a) drawing a closed polygon containing the location of the AV and the location of the target point, (b) drawing a path line between the location of the target point and the location of the AV, (c) selecting two SDSF points based on the SDSF information, the SDSF points being within the polygon, and (d) drawing the SDSF line between the two points. Detecting at least one candidate SDSF can include (e) if there are fewer than a first preselected number of points within a first preselected distance of the SDSF line, and if there are fewer than a second preselected number of attempts in selecting SDSF points, drawing lines between them, and having fewer than a first preselected number of points around the SDSF line, repeating steps (c)-(e). Detecting at least one candidate SDSF can include (f) fitting a curve to SDSF points that fall within a first preselected distance of the SDSF line if there are a first preselected number of points or more, (g) identifying the curve as an SDSF line if the first number of SDSF points within the first preselected distance of the curve exceeds the second number of SDSF points within the first preselected distance of the SDSF line, and if the curve intersects the path line, and if there are no gaps between SDSF points on the curve that exceed the second preselected distance. Detecting at least one candidate SDSF can include (h) if the number of points within the first preselected distance of the curve does not exceed the number of points within the first preselected distance of the SDSF line, or if the curve does not intersect the path line, or if there are gaps between SDSF points on the curve that exceed the second preselected distance, and if the SDSF line does not remain stable, and if steps (f)-(h) have not been attempted for more than a second preselected number of attempts, repeating steps (f)-(h).

[0033] The closed polygon can optionally include a preselected width, and the preselected width can optionally include a width dimension of the AV. Selecting the SDSF points can optionally include random selection. At least one selection criterion can optionally include that a first number of SDSF points within a first preselected distance of the curve exceeds a second number of SDSF points within a first preselected distance of the SDSF line, the curve intersects the path line, and there are no gaps between SDSF points on the curve that exceed a second preselected distance.

[0034] Determining at least one traversable portion of the selected SDSF can optionally include selecting a plurality of obstacle points based on the obstacle information. Each of the plurality of obstacle points can include a probability that the obstacle point is associated with at least one obstacle. Determining at least one traversable portion can include projecting the plurality of obstacle points onto the SDSF line to form at least one projection if the probability is higher than a preselected percentage and any one of the plurality of obstacle points is between the SDSF line and the target point, and if any one of the plurality of obstacle points is less than a third preselected distance from the SDSF line. Determining at least one traversable portion can optionally include connecting at least two of the at least one projection to each other, locating endpoints of the connected at least two projections along the SDSF line, marking the connected at least two projections as a non-traversable SDSF portion, and marking the SDSF line outside the non-traversable portion as at least one traversable portion.

[0035] Traversing at least one traversable portion of the SDSF can optionally include advancing the AV, operating at a first speed, toward the traversable portion, turning the AV to travel along a line perpendicular to the traversable portion, continuously correcting the heading of the AV based on a relationship between the heading and the perpendicular line, and driving the AV at a second speed by adjusting the first speed of the AV based on at least the heading and a distance between the AV and the traversable portion. Traversing at least one traversable portion of the SDSF can optionally include traversing the SDSF by raising a leading edge relative to a trailing edge and driving the AV at a third increasing speed per degree of the elevation if the SDSF is elevated relative to a surface of the travel route, and driving the AV at a fourth speed until the AV has cleared the SDSF.

[0036] Traversing at least one traversable portion of the SDSF can optionally include (a) if the heading error relative to a line perpendicular to the SDSF line is less than a third preselected amount, ignoring updates to the SDSF information and driving the AV at a preselected speed, (b) if the elevation of the front of the AV relative to the rear of the AV is between a sixth preselected amount and a fifth preselected amount, driving the AV forward and increasing the speed of the AV per degree of elevation to an eighth preselected speed, (c) if the front is elevated relative to the rear by less than a sixth preselected amount, driving the AV forward at a seventh preselected speed, and (d) if the rear is less than or equal to a fifth preselected distance from the SDSF line, repeating steps (a)-(d).

[0037] In some configurations, the wheels of the SDSF and AV can be automatically aligned to avoid system instability. Automatic alignment can be achieved by, for example, but not limited to, continuously testing and correcting the heading of the AV as the AV approaches the SDSF. Another aspect of the SDSF traversal feature of this teaching is that the SDSF traversal feature automatically confirms that there is enough free space around the SDSF before attempting to traverse. Another aspect of the SDSF traversal feature of this teaching is that it is possible to traverse a SDSF of varying geometry. The geometry can include, for example, but not limited to, square and wavy SDSFs. The orientation of the AV relative to the SDSF can determine at what speed and direction the AV proceeds. The SDSF traversal feature can adjust the speed of the AV near the SDSF. When the AV rises the SDSF, the speed can be increased to help the AV traverse the SDSF.

[0038] 1. An autonomous delivery vehicle, the autonomous delivery vehicle comprising: a powered base, the powered base comprising two powered front wheels, two powered rear wheels and an energy storage, the powered base being configured to move at a commanded rate and in a commanded direction to perform a transport of at least one object; a cargo platform, the cargo platform comprising a plurality of short-range sensors, the cargo platform being mechanically attached to the powered base; a cargo container, the cargo container having a volume for receiving the at least one object, the cargo container being mounted on the cargo platform; a long-range sensor suite, the long-range sensor suite comprising a LIDAR and one or more cameras, the long-range sensor suite being mounted on the cargo container; and a controller, the controller receiving data from the long-range sensor suite and the plurality of short-range sensors, the controller determining a commanded rate and a commanded direction based at least on the data, the controller providing the commanded rate and the commanded direction to the powered base to complete the transport. 2. The autonomous delivery vehicle of claim 1, wherein the data from the plurality of short-range sensors comprises at least one characteristic of a surface on which the powered base is traveling. 3. The autonomous delivery vehicle of claim 1, wherein the plurality of short-range sensors comprises at least one stereo camera. 4. The autonomous delivery vehicle of claim 1, wherein the plurality of short-range sensors comprises at least one IR projector, at least one image sensor, and at least one RGB sensor. 5. The autonomous delivery vehicle of claim 1, wherein the plurality of short-range sensors comprises at least one radar sensor. 6. The autonomous delivery vehicle of claim 1, wherein data from the plurality of short-range sensors comprises RGB-D data. 7. The autonomous delivery vehicle of claim 1, wherein the controller determines a geometry of a road surface based on the RGB-D data received from the plurality of short-range sensors. 8. The autonomous delivery vehicle of claim 1, wherein the plurality of short-range sensors detect objects within 4 meters of the AV, and the long-range sensor suite detects objects more than 4 meters from the autonomous delivery vehicle. 9. The autonomous delivery vehicle of claim 1, wherein the plurality of short-range sensors comprises cooling circuitry. 10. The autonomous delivery vehicle of claim 1, wherein the plurality of short-range sensors comprises ultrasonic sensors.11. The autonomous delivery vehicle of claim 2, wherein the controller comprises: executable code, the executable code comprising: accessing a map, the map formed by a map processor, the map processor comprising: a first processor, the first processor accessing point cloud data from the long-range sensor suite, the point cloud data representing a surface; a filter, the filter filtering the point cloud data; a second processor, the second processor forming a processable portion from the filtered point cloud data; a third processor, the third processor merging the processable portion into at least one polygon; a fourth processor, the fourth processor locating and marking at least one substantially discontinuous surface feature (SDSF) in the at least one polygon, if present, to form the marked point cloud data; a fifth processor, the fifth processor creating a graphic polygon from the marked point cloud data; and a sixth processor, the sixth processor selecting a path from the start point to the end point based at least on the graphic polygon, the AV traversing the at least one SDSF along the path. 12. The autonomous delivery vehicle of claim 11, wherein the filter comprises: a seventh processor, the seventh processor executing code, the code comprising: conditionally removing points representing transient objects and points representing outliers from the point cloud data; and replacing the removed points with preselected heights. 13. The autonomous delivery vehicle of claim 11, wherein the second processor comprises executable code, the executable code comprising: segmenting the point cloud data into processable portions; and removing points of preselected heights from the processable portions. 14. The autonomous delivery vehicle of claim 11, wherein the third processor comprises executable code, the executable code comprising: reducing the size of the processable portions by analyzing outliers, voxels, and normals; growing regions from the reduced-size processable portions; determining an initial drivable surface based on the grown regions; segmenting and meshing the initial drivable surface; locating polygons within the segmented and meshed initial drivable surface; and setting at least one drivable surface based at least on the polygons. 15. The autonomous delivery vehicle of claim 14, wherein the fourth processor comprises executable code, the executable code comprising: classifying the point cloud data of the initial drivable surface according to an SDSF filter, the SDSF filter comprising at least three categories of points; and locating at least one SDSF point based at least on whether the points of the at least three categories in combination satisfy at least one first preselected criterion. 16. The autonomous delivery vehicle of claim 15, wherein the fourth processor comprises executable code comprising: creating at least one SDSF trajectory based at least on whether the plurality of at least one SDSF points in combination satisfy at least one second preselected criterion.17. The autonomous delivery vehicle of claim 14, wherein creating the graphics polygon comprises an eighth processor comprising executable code comprising: creating at least one polygon from at least one drivable surface, the at least one polygon comprising an outer edge; smoothing the outer edge; forming a driving margin based on the smoothed outer edge; adding at least one SDSF track to the at least one drivable surface; and removing inner edges from the at least one drivable surface based on at least one third preselected criterion. 18. The autonomous delivery vehicle of claim 17, wherein smoothing the outer edge comprises a ninth processor comprising executable code comprising: trimming the outer edge outwardly to form an outward edge. 19. The autonomous delivery vehicle of claim 18, wherein forming the driving margin of the smoothed outer edge comprises a tenth processor comprising executable code comprising: trimming the outward edge inwardly. 20. The autonomous delivery vehicle of claim 1, wherein the controller comprises: a subsystem for navigating at least one substantially discontinuous surface feature (SDSF) encountered by an autonomous delivery vehicle (AV), the AV traversing a path on a surface, the surface comprising at least one SDSF, the path comprising a start point and a destination, the subsystem comprising: a first processor, the first processor accessing a route topology, the route topology comprising at least one graphic polygon comprising filtered point cloud data, the filtered point cloud data comprising labeled features, the point cloud data comprising a drivable margin; a second processor, the second processor transforming the point cloud data into a global coordinate system; a third processor, the third processor determining a boundary of the at least one SDSF, the third processor creating an SDSF buffer of a preselected size around the boundary; a fourth processor, the fourth processor determining which of the at least one SDSF can be traversed based at least on at least one SDSF traversal criterion; a fifth processor, the fifth processor creating an edge / weight graph based at least on at least one SDSF traversal criterion, the transformed point cloud data, and the route topology; and a base controller, the base controller selecting a path from the start point to the destination based at least on the edge / weight graph. 21. The autonomous delivery vehicle of claim 20, wherein at least one SDSF traversal criterion comprises: a preselected width of at least one and a preselected smoothness of at least one SDSF; a minimum entry distance and a minimum exit distance between at least one SDSF and an AV comprising a drivable surface; and a minimum entry distance between at least one SDSF and the AV that allows the AV to approach approximately 90° to the at least one SDSF.

[0039] 22. A method for managing a global occupancy grid for an autonomous device, the global occupancy grid including global occupancy grid cells, the global occupancy grid cells being associated with occupancy probabilities, the method comprising: receiving sensor data from a sensor associated with the autonomous device; creating a local occupancy grid having local occupancy grid cells based at least on the sensor data; if the autonomous device has moved from a first area to a second area, accessing historical data associated with the second area; creating a static grid based at least on the historical data; moving the global occupancy grid to maintain the autonomous device in a central position of the global occupancy grid; updating the moved global occupancy grid based on the static grid; if at least one of the global occupancy grid cells coincides with a position of the autonomous device, marking at least one of the global occupancy grid cells as unoccupied; for each of the local occupancy grid cells, calculating a position of the local occupancy grid cell on the global occupancy grid; accessing a first occupancy probability from the global occupancy grid cell at the position; accessing a second occupancy probability from the local occupancy grid cell at the position; and calculating a new occupancy probability at the position on the global occupancy grid based at least on the first occupancy probability and the second occupancy probability. 23. The method of claim 22, further comprising: range checking the new occupancy probability. 24. The method of claim 23, wherein the range checking comprises: if the new occupancy probability < 0, setting the new occupancy probability to 0; and if the new occupancy probability > 1, setting the new occupancy probability to 1. 25. The method of claim 22, further comprising: setting the global occupancy grid cell to the new occupancy probability. 26. The method of claim 23, further comprising: setting the global occupancy grid cell to the range-checked new occupancy probability.

[0040] 27. A method for creating and managing an occupancy grid, the method comprising: transforming sensor measurements into a reference frame associated with a device by a local occupancy grid creation node; creating a time-stamped measurement occupancy grid; publishing the time-stamped measurement occupancy grid as a local occupancy grid; creating multiple local occupancy grids; creating a static occupancy grid based on surface features in a repository, the surface features being associated with a location of the device; moving a global occupancy grid associated with the location of the device to maintain the device and the local occupancy grid approximately centered relative to the global occupancy grid; adding information from the static occupancy grid to the global occupancy grid; marking an area in the global occupancy grid currently occupied by the device as unoccupied; for each of at least one cell in each local occupancy grid, determining a location of the at least one cell in the global occupancy grid; accessing a first value at the location; determining a second value at the location based on a relationship between the first value and a cell value at at least one cell in the local occupancy grid; comparing the second value against a preselected probability range; and setting the global occupancy grid with the new value if the probability value is within the preselected probability range. 28. The method of claim 27, further comprising: publishing a global occupancy grid. 29. The method of claim 27, wherein the surface characteristics include surface type and surface discontinuities. 30. The method of claim 27, wherein the relationship includes a summation. 31. A system for creating and managing occupancy grids, the system comprising: a plurality of local grid creation nodes, the plurality of local grid creation nodes creating at least one local occupancy grid, the at least one local occupancy grid being associated with a location of a device, the at least one local occupancy grid comprising at least one cell; a global occupancy grid manager, the global occupancy grid manager accessing at least one local occupancy grid, the global occupancy grid manager creating a static occupancy grid based on surface characteristics in a repository, the surface characteristics being associated with a location of the device, moving a global occupancy grid associated with the location of the device so that the device and at least one local occupancy grid remain approximately centered relative to the global occupancy grid; adding information from the static occupancy grid to at least one global occupancy grid; marking an area in the global occupancy grid currently occupied by the device as unoccupied; for each of at least one cell in each local occupancy grid, determining a location of the at least one cell in the global occupancy grid; accessing a first value at the location; determining a second value at the location based on a relationship between the first value and a cell value at at least one cell in the local occupancy grid; comparing the second value against a preselected probability range; and setting the global occupancy grid with a new value if the probability value is within the preselected probability range.

[0041] 32. A method for updating a global occupancy grid, the method comprising: if the autonomous device has moved to a new location, updating the global occupancy grid with information from a static grid associated with the new location; analyzing a surface at the new location; if the surface is drivable, updating the surface and updating the global occupancy grid with the updated surface; and updating the global occupancy grid with values ​​from a repository of static values, the static values ​​being associated with the new location. 33. The method of claim 32, wherein updating the surface comprises: accessing a local occupancy grid associated with the new position; for each cell in the local occupancy grid, accessing a local occupancy grid surface classification confidence value and a local occupancy grid surface classification; if the local occupancy grid surface classification is the same as the global surface classification in the global occupancy grid in the cell, adding the global surface classification confidence value in the global occupancy grid to the local occupancy grid surface classification confidence value to form a sum, and updating the global occupancy grid at the cell with the sum; if the local occupancy grid surface classification is not the same as the global surface classification in the global occupancy grid in the cell, subtracting the local occupancy grid surface classification confidence value from the global surface classification confidence value in the global occupancy grid to form a difference, and updating the global occupancy grid with the difference; if the difference is less than zero, updating the global occupancy grid with the local occupancy grid surface classification. 34. The method of claim 32, wherein updating the global occupancy grid with values ​​from the repository of static values ​​comprises: for each cell in the local occupancy grid, accessing from the local occupancy grid a local occupancy grid probability that the cell is occupied, a log probability value; updating the log probability value in the global occupancy grid with the local occupancy grid log probability value at the cell; if a preselected certainty that the cell is not occupied is met, and if the autonomous device is driving within a lane barrier, and if the local occupancy grid surface classification indicates a drivable surface, decreasing the log probability that the cell is occupied in the local occupancy grid; if the autonomous device expects to encounter a relatively uniform surface, and if the local occupancy grid surface classification indicates a relatively non-uniform surface, increasing the log probability in the local occupancy grid; and if the autonomous device expects to encounter a relatively uniform surface, and if the local occupancy grid surface classification indicates a relatively uniform surface, decreasing the log probability in the local occupancy grid.

[0042] 35. A method for real-time control of configuration of a device, the device comprising a chassis, at least four wheels, a first side of the chassis operably connected to at least one of the at least four wheels, and an opposing second side of the chassis operably connected to at least one of the at least four wheels, the method comprising: creating a map based at least on prior surface features and an occupancy grid, the map being created non-real time, the map comprising at least one location, at least one location being associated with at least one surface feature, at least one surface feature being associated with at least one surface classification and at least one mode; determining a current surface feature as the device travels; updating the occupancy grid in real time with the current surface feature; and determining from the occupancy grid and the map a path along which the device can travel to traverse the at least one surface feature.

[0043] 36. A method for real-time control of a configuration of a device, the device comprising a chassis, at least four wheels, a first side of the chassis operably coupled to at least one of the at least four wheels, and an opposing second side of the chassis operably coupled to at least one of the at least four wheels, the method comprising: receiving environmental data; determining a surface type based at least on the environmental data; determining a mode based at least on the surface type and a first configuration; determining a second configuration based at least on the mode and the surface type; determining a movement command based at least on the second configuration; and controlling the configuration of the device by using the movement command to change the device from the first configuration to the second configuration. 37. The method of claim 36, wherein the environmental data comprises RGB-D image data. 38. The method of claim 36, the method further comprising: populating an occupancy grid based at least on the surface type and the mode; and determining the movement command based at least on the occupancy grid. 39. The method of claim 38, wherein the occupancy grid comprises information based at least on data from at least one image sensor. 40. The method of claim 36, wherein the environmental data comprises a topology of a road surface. 41. The method of claim 36, wherein the configuration comprises two pairs of clusters of at least four wheels, a first pair of the two pairs being positioned on a first side, a second pair of the two pairs being positioned on a second side, the first pair comprising a first front wheel and a first rear wheel, and the second pair comprising a second front wheel and a second rear wheel. 42. The method of claim 41, wherein control of the configuration comprises: coordinated powering of the first pair and the second pair based at least on environmental data. 43. The method of claim 41, wherein control of the configuration comprises: transitioning from driving at least four wheels and a pair of retracting casters to driving two wheels, the pair of casters being operably coupled to a chassis, wherein the first pair of clusters and the second pair of clusters rotate to elevate the first front wheel and the second front wheel, the device resting on the first rear wheel, the second rear wheel, and the pair of casters. 44. The method of claim 41, wherein control of the configuration comprises: rotating a pair of clusters operably coupled to the first two powered wheels on the first side and the second two powered wheels on the second side based at least on environmental data. 45. The method of claim 36, wherein the device further comprises a cargo container, the cargo container being mounted on the chassis, the chassis controlling the height of the cargo container. 46. ​​The method of claim 45, wherein the height of the cargo container is based at least on environmental data.

[0044] 47. A system for real-time control of a configuration of a device, the device comprising a chassis, at least four wheels, a first side of the chassis, and an opposing second side of the chassis, the system comprising: a device processor, the device processor receiving real-time environmental data surrounding the device, the device processor determining a surface type based at least on the environmental data, the device processor determining a mode based at least on the surface type and a first configuration, the device processor determining a second configuration based at least on the mode and the surface type; and a powered base processor, the powered base processor determining movement commands based at least on the second configuration, the powered base processor controlling the configuration of the device by using the movement commands to change the device from the first configuration to the second configuration. 48. The system of claim 47, wherein the environmental data comprises RGB-D image data. 49. The system of claim 47, wherein the device processor comprises populating an occupancy grid based at least on the surface type and the mode. 50. The system of claim 49, wherein the powered base processor comprises determining movement commands based at least on the occupancy grid. 51. The system of claim 49, wherein the occupancy grid comprises information based at least on data from at least one image sensor. 52. The system of claim 47, wherein the environmental data includes a topology of a road surface. 53. The system of claim 47, wherein the configuration includes two pairs of clusters of at least four wheels, a first pair of the two pairs being positioned on a first side, a second pair of the two pairs being positioned on a second side, the first pair having a first front wheel and a first rear wheel, and the second pair having a second front wheel and a second rear wheel. 54. The system of claim 53, wherein control of the configuration includes: coordinated powering of the first pair and the second pair based at least on the environmental data. 55. The system of claim 53, wherein control of the configuration includes: transitioning from driving the at least four wheels and a pair of retracting casters to driving the two wheels, the pair of casters being operably coupled to the chassis, wherein the first pair of the cluster and the second pair of the cluster rotate to elevate the first front wheel and the second front wheel, the device resting on the first rear wheel, the second rear wheel, and the pair of casters.

[0045] 56. A method for maintaining a global occupancy grid, the method comprising: locating a first position of an autonomous device; when the autonomous device moves to a second position, the second position being associated with a global occupancy grid and a local occupancy grid, updating the global occupancy grid with at least one occupancy probability value associated with the first position; updating the global occupancy grid with at least one drivable surface associated with the local occupancy grid; updating the global occupancy grid with a surface confidence associated with the at least one drivable surface; using a first Bayesian function to update the global occupancy grid with a log probability of at least one occupancy probability value; and adjusting the log probability based at least on a characteristic associated with the second position; and when the autonomous device remains in the first position and the global occupancy grid and the local occupancy grid are co-located, updating the global occupancy grid with at least one drivable surface associated with the local occupancy grid; updating the global occupancy grid with a surface confidence associated with the at least one drivable surface; using a second Bayesian function to update the global occupancy grid with a log probability of at least one occupancy probability value; and adjusting the log probability based at least on a characteristic associated with the second position. 57. The method of claim 35, wherein creating a map comprises: accessing point cloud data representing a surface; filtering the point cloud data; forming the filtered point cloud data into processable portions; merging the processable portions into at least one concave polygon; locating and marking at least one SDSF in the at least one concave polygon, locating and marking the point cloud data forming the marked; creating a graphics polygon based at least on the at least one concave polygon; and selecting a path from a start point to an end point based at least on the graphics polygon, the AV traversing the at least one SDSF along the path. 58. The method of claim 57, wherein filtering the point cloud data comprises: conditionally removing points representing transient objects and points representing outliers from the point cloud data; and replacing the removed points with preselected heights. 59. The method of claim 57, wherein forming the processable portions comprises: segmenting the point cloud data into processable portions; and removing points of preselected heights from the processable portions. 60. The method of claim 57, wherein merging processable portions comprises: reducing the size of processable portions by analyzing outliers, voxels, and normals; growing regions from the reduced-size processable portions; determining an initial drivable surface based on the grown regions; segmenting and meshing the initial drivable surface; locating polygons within the segmented and meshed initial drivable surface; and setting at least one drivable surface based at least on the polygons. 61. The method of claim 60, wherein locating and marking at least one SDSF comprises: classifying point cloud data of the initial drivable surface based on an SDSF filter, the SDSF filter comprising at least three categories of points; and locating at least one SDSF point based at least on whether the at least three categories of points in combination satisfy at least one first preselected criterion.62. The method of claim 61, further comprising: creating at least one SDSF track based at least on whether the plurality of at least one SDSF points in combination satisfy at least one second preselected criterion. 63. The method of claim 62, wherein creating the graphic polygon further comprises: creating at least one polygon from at least one drivable surface, the at least one polygon comprising an exterior edge; smoothing the exterior edge; forming a driving margin based on the smoothed exterior edge; adding at least one SDSF track to the at least one drivable surface; and removing interior edges from the at least one drivable surface based on at least one third preselected criterion. 64. The method of claim 63, wherein the smoothing of the exterior edge comprises: trimming the exterior edge outwardly to form an outward edge. 65. The method of claim 63, wherein forming the driving margin of the smoothed exterior edge comprises: trimming the outward edge inwardly.

[0046] 66. An autonomous delivery vehicle, the autonomous delivery vehicle comprising: a powered base, the powered base comprising two powered front wheels, two powered rear wheels, and an energy storage device, the powered base being configured to move at a commanded rate; a cargo platform, the cargo platform comprising a plurality of short-range sensors, the cargo platform being mechanically attached to the powered base; a cargo container, the cargo container having a volume for receiving one or more objects to be delivered, the cargo container being mounted on the cargo platform; a long-range sensor suite, the long-range sensor suite comprising a LIDAR and one or more cameras, the long-range sensor suite being mounted on the cargo container; and a controller, the controller receiving data from the long-range sensor suite and the plurality of short-range sensors. 67. The autonomous delivery vehicle of claim 66, wherein the plurality of short-range sensors detect at least one characteristic of a drivable surface. 68. The autonomous delivery vehicle of claim 66, wherein the plurality of short-range sensors are stereo cameras. 69. The autonomous delivery vehicle of claim 66, wherein the plurality of short-range sensors include an IR projector, two image sensors, and an RGB sensor. 70. The autonomous delivery vehicle of claim 66, wherein the plurality of short-range sensors are radar sensors. 71. The autonomous delivery vehicle of claim 66, wherein the short range sensors supply RGB-D data to the controller. 72. The autonomous delivery vehicle of claim 66, wherein the controller determines the geometry of the road surface based on the RGB-D data received from the plurality of short range sensors. 73. The autonomous delivery vehicle of claim 66, wherein the plurality of short range sensors detect objects within 4 meters of the autonomous delivery vehicle, and the long range sensor suite detects objects more than 4 meters from the autonomous delivery vehicle.

[0047] 74. An autonomous delivery vehicle, the autonomous delivery vehicle comprising: a powered base, the powered base comprising at least two powered rear wheels, a caster front wheel, and an energy storage device, the powered base being configured to move at a commanded rate; a cargo platform, the cargo platform comprising a plurality of short-range sensors, the cargo platform being mechanically attached to the powered base; a cargo container, the cargo container having a volume for receiving one or more objects to be delivered, the cargo container being mounted on the cargo platform; a long-range sensor suite, the long-range sensor suite comprising a LIDAR and one or more cameras, the long-range sensor suite being mounted on the cargo container; and a controller, the controller receiving data from the long-range sensor suite and the plurality of short-range sensors. 75. The autonomous delivery vehicle of claim 74, wherein the plurality of short-range sensors detect at least one characteristic of a drivable surface. 76. The autonomous delivery vehicle of claim 74, wherein the plurality of short-range sensors are stereo cameras. 77. The autonomous delivery vehicle of claim 74, wherein the plurality of short-range sensors include an IR projector, two image sensors, and an RGB sensor. 78. The autonomous delivery vehicle of claim 74, wherein the plurality of short-range sensors are radar sensors. 79. The autonomous delivery vehicle of claim 74, wherein the short range sensors supply RGB-D data to the controller. 80. The autonomous delivery vehicle of claim 74, wherein the controller determines the geometry of the road surface based on the RGB-D data received from the plurality of short range sensors. 81. The autonomous delivery vehicle of claim 74, wherein the plurality of short range sensors detect objects within 4 meters of the autonomous delivery vehicle, and the long range sensor suite detects objects more than 4 meters from the autonomous delivery vehicle. 82. The autonomous delivery vehicle of claim 74, further comprising a second set of powered wheels that can engage the ground while the casters are lifted off the ground.

[0048] 83. An autonomous delivery vehicle, the autonomous delivery vehicle comprising: a powered base, the powered base comprising at least two powered rear wheels, a caster front wheel, and an energy storage, the powered base being configured to move at a commanded rate; a cargo platform, the cargo platform mechanically attached to the powered base; and a short-range camera assembly mounted to the cargo platform, the short-range camera assembly detecting at least one characteristic of a drivable surface, the short-range camera assembly comprising: a camera; a first light; and a first liquid-cooled heat sink, wherein the first liquid-cooled heat sink cools the first light and the camera. 84. The autonomous delivery vehicle of claim 83, wherein the short-range camera assembly further comprises a thermoelectric cooler between the camera and the liquid-cooled heat sink. 85. The autonomous delivery vehicle of claim 83, wherein the first light and the camera are recessed in a cover having an opening, the opening deflecting illumination from the first light away from the camera. 86. The autonomous delivery vehicle of claim 83, wherein the light is tilted downward at least 15° and recessed at least 4 mm in the cover to minimize illumination that distracts pedestrians. 87. The autonomous delivery vehicle of claim 83, wherein the camera has a field of view and the first light comprises two LEDs with lenses to produce two beams of light that spread to illuminate the field of view of the camera. 88. The autonomous delivery vehicle of claim 87, wherein the lights are tilted approximately 50° apart and the lenses produce a 60° beam of light. 89. The autonomous delivery vehicle of claim 83, wherein the short-range camera assembly comprises an ultrasonic sensor mounted above the camera. 90. The autonomous delivery vehicle of claim 83, wherein the short-range camera assembly is mounted in a central position on the front face of the cargo platform. 91. The autonomous delivery vehicle of claim 83, further comprising at least one corner camera assembly mounted on at least one corner of the front face of the cargo platform, the at least one corner camera assembly comprising: an ultrasonic sensor; a corner camera; a second light; and a second liquid-cooled heat sink, wherein the second liquid-cooled heat sink cools the second light and the corner camera. 92. The method of claim 22, wherein the historical data comprises surface data. 93. The method of claim 22, wherein the historical data comprises discontinuous data. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The present teaching will be more readily understood by reference to the following description accompanying the accompanying drawings, in which:

[0050] Figure 1-1 is a schematic block diagram of the main components of the system of the present teaching;

[0051] Figure 1-2 is a schematic block diagram of the main components of the map processor of the present teachings;

[0052] Figure 1-3 is a schematic block diagram of the main components of the perception processor of the present teachings;

[0053] Figure 1-4 is a schematic block diagram of the main components of an autonomous processor of the present teachings;

[0054] Figure 1A is a schematic block diagram of a system for preparing a driving path for an AV according to the present teachings;

[0055] Figure 1B is a pictorial diagram of an exemplary configuration of a device incorporating a system of the present teachings;

[0056] Figure 1C is a side view of an autonomous delivery vehicle showing the field of view of some long-range sensors and short-range sensors.

[0057] Figure 1D is a schematic block diagram of a map processor of the present teachings;

[0058] Figure 1E is a visual diagram of the first part of the flow of the map processor of the present teaching;

[0059] Figure 1F is an image of a segmented point cloud of the present teaching;

[0060] Figure 1G is a visual diagram of the second part of the map processor of the present teaching;

[0061] Figure 1H is an image of a drivable surface detection result of the present teaching;

[0062] Fig. 1I is a visual diagram of the process of the SDSF finder of this teaching;

[0063] Figure 1J is a visual diagram of the SDSF categories of this teaching;

[0064] Figure 1K is an image of SDSF recognized by the system of the present teachings;

[0065] Figure 1L and Figure 1M It is a visual diagram of the polygon processing of this teaching;

[0066] Figure 1N is an image of a polygon and SDSF recognized by the system of the present teachings;

[0067] Figure 2A is an isometric view of an autonomous vehicle of the present teachings;

[0068] Figure 2B is a top view of a cargo container showing the field of view of a selected long range sensor;

[0069] Figure 2C-2Fis a view of the long range sensor assembly;

[0070] Figure 2G is a top view of a cargo container showing the field of view of a selected short-range sensor;

[0071] Figure 2H is an isometric view of a cargo platform of the present teachings;

[0072] Figure 2I-2L is an isometric view of a short-range sensor;

[0073] Figure 2M-2N is an isometric view of an autonomous vehicle of the present teachings;

[0074] Figure 2O-2P is an isometric view of an autonomous vehicle of the present teachings with outer skin panels removed;

[0075] Figure 2Q is an isometric view of an autonomous vehicle of the present teachings with a portion of a top panel removed;

[0076] Figure 2R-2V is a diagram of a long-range sensor on an autonomous vehicle of the present teachings;

[0077] Figure 2W-2Z is a view of the ultrasonic sensor;

[0078] Figure 2AA-2BB is the view from the central short-range camera assembly;

[0079] Figure 3A is a schematic block diagram of a system of one configuration of the present teachings;

[0080] Figure 3B is a schematic block diagram of a system of another configuration of the present teachings;

[0081] Figure 3C is a schematic block diagram of a system capable of initially creating a global occupancy grid according to the present teachings;

[0082] Figure 3D is a visual representation of the static grid of this teaching;

[0083] Figure 3E and Figure 3F is a visual representation of the creation of an occupancy grid of the present teachings;

[0084] Figure 3G is a visual representation of the prior occupancy grid of the present teachings;

[0085] Figure 3H is a visual representation of the global occupancy grid for updating this teaching;

[0086] Fig. 3Iis a flow chart of a method for publishing a global occupancy grid according to the present teachings;

[0087] Figure 3J is a flow chart of a method for updating a global occupancy grid according to the present teachings;

[0088] Figure 3K-3M is a flow chart of another method of the present teachings for updating a global occupancy grid.

[0089] Figure 4A is a perspective intuitive diagram of a device of the present teachings in various modes;

[0090] Figure 4B is a schematic block diagram of a system of the present teachings;

[0091] Figure 4C is a schematic block diagram of a driving surface processor component of the present teachings;

[0092] Figure 4D is a schematic block diagram / intuitive flow chart of the process of the present teaching;

[0093] Figure 4E and Figure 4F are perspective and side views, respectively, of a configuration of the apparatus of the present teachings in a standard mode;

[0094] Figure 4G and Figure 4H are perspective and side views, respectively, of a configuration of the apparatus of the present teachings in 4-wheel mode;

[0095] Fig. 4I and Figure 4J are perspective and side views, respectively, of the apparatus of the present teachings configured in a raised 4-wheel mode;

[0096] Figure 4K is a flow chart of the method of the present teachings;

[0097] Figure 5A is a schematic block diagram of a device controller of the present teachings;

[0098] Figure 5B is a schematic block diagram of an SDSF processor of the present teachings;

[0099] Figure 5C is an image of a SDSF closeness identified by the system of the present teachings;

[0100] Figure 5D is an image of a route topology created by a system of the present teachings;

[0101] Figure 5E is a schematic block diagram of a mode of the present teaching.

[0102] Figure 5F-5Jis a flow chart of a method for traversing a SDSF according to the present teachings;

[0103] Figure 5K is a schematic block diagram of a system for traversing a SDSF according to the present teachings;

[0104] Figure 5L-5N yes Figure 5F-5H A visual representation of the method; and

[0105] Fig.5O is a visual representation of converting an image into polygons. DETAILED DESCRIPTION

[0106] The systems and methods of the present teachings can use onboard sensors and previously developed maps to develop an occupancy grid and use these aids to navigate the AV across surface features, including reconfiguring the AV based at least on the surface type.

[0107] Reference now Figure 1-1 , the AV system 100 can include a structure on which sensors 10701 can be mounted and within which a device controller 10111 can execute. The structure can include a powered base 10112 that can direct the movement of wheels that are part of the structure and can enable movement of the AV. The device controller 10111 can execute on at least one processor located on the AV and can receive data from sensors 10701 that can be, but are not limited to, located on the AV. The device controller 10111 can provide speed, direction, and configuration information to a base controller 10114 that can provide movement commands to the powered base 10112. The device controller 10111 can receive map information from a map processor 10104 that can prepare a map of an area around the AV. The device controller 10111 can include, but is not limited to, including a sensor processor 10703 that can receive and process input from sensors 10701 (including sensors on the AV). In some configurations, the device controller 10111 can include a perception processor 2143, an autonomous processor 2145, and a driver processor 2127. The perception processor 2143 can, for example, but not limited to, locate static and dynamic obstacles, determine traffic light status, create an occupancy grid, and classify surfaces. The autonomous processor 2145 can, for example, but not limited to, determine the maximum speed of the AV and determine the type of situation in which the AV is navigating, such as on a road, on a sidewalk, at an intersection, and / or under remote control. The driver processor 2127 can, for example, but not limited to, create commands based on the direction of the autonomous processor 2145 and send them to the base controller 10114.

[0108] Reference now Figure 1-2, the map processor 10104 can create a map of surface features and can provide the map through the device controller 10111 to the perception processor 2143 which can update the occupancy grid. The map processor 10104 can include, among many other aspects, a feature extractor 10801, a point cloud organizer 10803, a transient processor 10805, a segmenter 10807, a polygon generator 10809, an SDSF line generator 10811, and a combiner 10813. The feature extractor 10801 can include a first processor accessing point cloud data representing a surface. The point cloud organizer 10803 can include a second processor forming processable portions from the filtered point cloud data. The transient processor 10805 can include a first filter that filters the point cloud data. The segmenter 10807 can include executable code that can include, but is not limited to, segmenting the point cloud data into processable portions and removing points of preselected heights from the processable portions. The first filter can optionally include executable code, which can include, but is not limited to, conditionally removing points representing transient objects and points representing outliers from the point cloud data, and replacing the removed points with a preselected height. The polygon generator 10809 can include a third processor that merges the processable portion into at least one concave polygon. The third processor can optionally include executable code, which can include, but is not limited to, reducing the size of the processable portion by analyzing outliers, voxels, and normals, growing regions from the reduced-size processable portion, determining an initial drivable surface based on the grown regions, segmenting and meshing the initial drivable surface, locating polygons within the segmented and meshed initial drivable surface, and setting the drivable surface based on at least the polygon. The SDSF line generator 10811 can include a fourth processor that locates and marks at least one SDSF in at least one concave polygon, locating and marking the point cloud data forming the mark. The fourth processor can optionally include executable code, which can include, but is not limited to, classifying the point cloud data of the drivable surface according to an SDSF filter, the SDSF filter including at least three categories of points, and locating at least one SDSF point based on at least whether the points of the categories meet at least one first preselected criterion in combination. The combiner 10813 can include a fifth processor that creates a graphic polygon. Creating a graphic polygon can optionally include executable code, which can include, but is not limited to, including: creating at least one polygon from at least one drivable surface, the at least one polygon including edges; smoothing the edges; forming a driving margin based on the smoothed edges; adding at least one SDSF track to at least one drivable surface; and removing edges from at least one drivable surface according to at least one third preselected criterion. Smoothing the edges can optionally include executable code, which can include, but is not limited to, trimming the edges outward.The driving margin forming a smooth edge can optionally include executable code that can include, but is not limited to, trimming the outward edge inward.

[0109] Reference now Figure 1-3 , can provide a map to an AV, which can include onboard sensors, powered wheels, a processor for receiving sensor and map data and using that data to power configure the AV to traverse various surfaces, particularly as the AV, for example, delivers merchandise. The onboard sensors can provide data that can populate an occupancy grid and can be used to detect dynamic obstacles. The occupancy grid can also be populated by a map. The device controller 10111 can include a perception processor 2143 that can receive and process sensor data and map data and can update the occupancy grid with that data.

[0110] Reference now Figure 1-4 , the device controller 10111 can include a configuration processor 41023 that can automatically determine the configuration of the AV based at least on the mode of the AV and the surface features encountered. The autonomous processor 2145 can include a control processor 40325 that can determine what surface needs to be traversed and what configuration the AV needs to adopt to traverse the surface based at least on a map (a planned route to follow), information from the configuration processor 41023, and the mode of the AV. The autonomous processor 2145 can supply commands to the motor drive processor 40326 to execute the commands.

[0111] Reference now Figure 1A , the map processor 10104 can enable a device, such as but not limited to an AV or a semi-autonomous device, to navigate in an environment that can include features such as an SDSF. The features in the map, together with the onboard sensors, can enable the AV to travel on a variety of surfaces. In particular, the SDSF can be accurately identified and marked, enabling the AV to automatically maintain the performance of the AV during entry and exit of the SDSF, and the AV speed, configuration, and direction can be controlled for safe SDSF traversal.

[0112] Continue to refer Figure 1AIn some configurations, the system 100 for managing traversal of the SDSF can include an AV 10101, a core cloud infrastructure 10103, an AV service 10105, a device controller 10111, sensors 10701, and a power base 10112. The AV 10101 can provide, for example, but not limited to, transportation and escort services from an origin to a destination following a dynamically determined path as modified by incoming sensor information. The AV 10101 can include, but is not limited to, devices including autonomous modes, devices capable of fully autonomous operation, devices capable of being at least partially remotely operated, and devices capable of including a combination of those features. The transportation device service 10105 can provide drivable surface information including features to the device controller 10111. The device controller 10111 can modify the drivable surface information based on, for example, but not limited to, the incoming sensor information and the feature traversal requirements, and can select a path for the AV 10101 based on the modified drivable surface information. The device controller 10111 can provide commands to the powered base 10112, which can instruct the powered base 10112 to provide speed, direction, and configuration commands to the wheel motors and cluster motors, which cause the AV 10101 to follow the chosen path and raise and lower its cargo accordingly. The transport device service 10105 can access route related information from the core cloud infrastructure 10103, which can include, but is not limited to, storage and content distribution facilities. In some configurations, the core cloud infrastructure 10103 can include, for example, but not limited to, AMAZON WEB SERVICES®, GOOGLECLOUD TM and ORACLE CLOUD® commercial products.

[0113] Reference now Figure 1B , can include a map processor 10104 ( Figure 1A ) Device controller 10111 receiving information ( Figure 1A) can include a power base assembly, such as, for example, but not limited to, a power base fully described in, for example, but not limited to, U.S. Patent Application No. 16 / 035,205, entitled Mobility Device, filed on July 13, 2018, or U.S. Patent No. 6,571,892, entitled Control System and Method, filed on August 15, 2001, both of which are incorporated herein by reference in their entirety. The exemplary power base assembly is not intended to limit the present teachings but is described herein for the purpose of clarifying the features of any power base assembly that may be useful in implementing the techniques of the present teachings. The exemplary power base assembly can optionally include a power base 10112, a wheel cluster assembly 11100, and a payload carrier height assembly 10068. The exemplary power base assembly can optionally provide electrical and mechanical power to the drive wheels 11203 and the cluster 11100 that can raise and lower the wheels 11203. Power base 10112 can control the rotation of cluster assembly 11100 and the elevation of payload carrier elevation assembly 10068 to support the substantially discontinuous surface traversal of the present teachings. Other such devices can be used to enable SDSF detection and traversal of the present teachings.

[0114] Reference again Figure 1A In some configurations, sensors inside the exemplary power base can detect the orientation and rate of change of orientation of the AV 10101, the motors can enable servo operation, and the controller can incorporate information from the internal sensors and motors. Appropriate motor commands can be calculated to achieve transporter performance and implement path following commands. The left and right wheel motors can drive the wheels on either side of the AV 10101. In some configurations, the front and rear wheels can be coupled to drive together, so that the two left wheels can be driven together and the two right wheels can be driven together. In some configurations, rotation can be achieved by driving the left and right motors at different rates, and the cluster motors can rotate the wheelbase in the front / rear direction. This can allow the AV 10101 to remain level when the front wheels become higher or lower than the rear wheels. This feature can be useful, for example, but not limited to, when climbing up and down the SDSF. The payload carrier 10173 can be automatically raised and lowered based at least on the underlying terrain.

[0115] Continue to refer Figure 1AIn some configurations, the point cloud data can include route information for the area in which the AV 10101 will travel. Point cloud data that may be collected by a mapping device similar to or the same as the AV 10101 can be time-stamped. The path along which the mapping device travels can be referred to as a mapping trajectory. The point cloud data processing described herein can occur as the mapping device traverses the mapping trajectory, or later after the point cloud data collection is completed. After the point cloud data are collected, they can undergo point cloud data processing, which can include initial filtering and point reduction, point cloud segmentation, and feature detection as described herein. In some configurations, the core cloud infrastructure 10103 can provide long-term or short-term storage for the collected point cloud data, and can provide the data to the AV service 10105. The AV service 10105 can select among possible point cloud data sets to find a data set that covers the terrain of the desired starting point around the AV 10101 and the desired destination of the AV 10101. AV service 10105 can include, but is not limited to, including a map processor 10104 that can reduce the size of the point cloud data and determine the features represented in the point cloud data. In some configurations, the map processor 10104 can determine the location of the SDSF based on the point cloud data. In some configurations, polygons can be created from the point cloud data as a technique for segmenting the point cloud data and ultimately setting a drivable surface. In some configurations, the SDSF lookup and drivable surface determination can be performed in parallel. In some configurations, the SDSF lookup and drivable surface determination can be performed sequentially.

[0116] Reference now Figure 1C In some configurations, the AV can be configured to deliver cargo and / or perform other functions involving autonomous navigation to a desired location. In some applications, the AV can be remotely guided. In some configurations, the AV 20100 includes a cargo container that can be remotely opened in response to user input, automatically or manually to allow a user to place or remove packages and other items. The cargo container 20110 is mounted on a cargo platform 20160, which is mechanically connected to a power base 20170. The power base 20170 includes four powered wheels 20174 and two casters 20176. The power base provides speed and direction control to move the cargo container 20110 along the ground and over obstacles including curbs and other discontinuous surface features.

[0117] Continue to refer Figure 1C, the cargo platform 20160 is connected to the power base 20170 by two U-shaped frames 20162. Each U-shaped frame 20162 is rigidly attached to the structure of the cargo platform 20160 and includes two holes that allow a rotatable joint 20164 to be formed on the power base 20170 with the end of each arm 20172. The power base controls the rotational position of the arms, thereby controlling the height and attitude of the cargo container 20110.

[0118] Continue to refer Figure 1C In some configurations, the AV 20100 includes one or more processors to receive data, navigate a path, and select a direction and speed for the powered base 20170 .

[0119] Reference now Figure 1D In some configurations, the map processor 10104 of the present teachings can locate the SDSF on the map. The map processor 10104 can include, but is not limited to, a feature extractor 10801, a point cloud organizer 10803, a transient processor 10805, a segmenter 10807, a polygon generator 10809, a SDSF line generator 10811, and a data combiner 10813.

[0120] Continue to refer Figure 1D , feature extractor 10801 ( Figure 1-2 ) can include, but is not limited to, line-of-sight filtering 10121 including point cloud data 10131 and drawn trajectory 10133. Line-of-sight filtering can remove points that are hidden from the direct line of sight of the sensor that collected the point cloud data and formed the drawn trajectory. Point cloud organizer 10803 ( Figure 1-2 ) can organize 10151 the reduced point cloud data 10132 according to preselected criteria that may be associated with particular features. In some configurations, the transient processor 10805 ( Figure 1-2 ) can remove 10153 transient points from the organized point cloud data and drawn trajectory 10133 by any number of methods, including those described herein. Transient points can complicate processing, especially when certain features are static. Segmenter 10807 ( Figure 1-2) can split the processed point cloud data 10135 into processable chunks. In some configurations, the processed point cloud data 10135 can be segmented 10155 into portions having a preselected minimum number of points, such as but not limited to approximately 100,000 points. In some configurations, further point reduction can be based on preselected criteria that may be related to the features to be extracted. For example, if points above a certain height are not important for locating features, those points may be removed from the point cloud data. In some configurations, the height of at least one of the sensors collecting the point cloud data may be considered to be the origin, and points above the origin may be removed from the point cloud data when, for example, the only points of interest are associated with a surface. After the filtered point cloud data 10135 has been segmented to form segments 10137, the remaining points can be divided into drivable surface portions and surface features can be located. In some configurations, the polygon generator 10809 ( Figure 1-2 ) can locate the drivable surface by, for example, but not limited to, generating 10161 polygons 10139 as described herein. In some configurations, the SDSF line generator 10811 ( Figure 1-2 ) can locate surface features by, for example, but not limited to, generating 10163 SDSF lines 10141 as described herein. In some configurations, the combiner 10813 ( Figure 1-2 ) can create a data set by combining 10165 polygons 10139 and SDSF 10141, which can be further processed to generate AV 10101 ( Figure 1A ) is the actual path that can be traveled.

[0121] Now the main reference Figure 1E , from point cloud data 10131 ( Figure 1D ) to eliminate 10153 ( Figure 1D ) relative to the drawing trajectory 10133 transient objects, such as the exemplary time-stamped points 10751, can include projecting the time-stamped points on the drawing trajectory 10133 onto the point cloud data 10131 ( Figure 1D ) with substantially the same timestamp within the reduced point cloud data 10132. If the ray 10753 intersects a point (e.g., point D 10755) between the timestamp point on the drawn trajectory 10133 and the end point of the ray 10753, it can be assumed that the intersection point D 10755 entered the point cloud data during a different scan of the camera. Intersection points, such as intersection point D 10755, can be assumed to be part of a transient object and can be removed from the reduced point cloud data 10132 ( Figure 1D ) is removed because it does not represent a fixed feature such as SDSF. The result is processed point cloud data 10135 without, for example, but not limited to, transient objects ( Figure 1DPoints that have been removed as part of a transient object but are still substantially at ground level can be returned 10754 to the processed point cloud data 10135 ( Figure 1D Transient objects cannot include, for example but not limited to, SDSF 10141 ( Figure 1D ) and therefore in SDSF 10141 ( Figure 1D ) is the feature being detected, without interfering with the point cloud data 10131 ( Figure 1D ) are removed for integrity reasons.

[0122] Continue to refer Figure 1E , for the processed point cloud data 10135 ( Figure 1D ) segmented 10155 ( Figure 1D ) is capable of generating a portion 10757 of a preselected size and shape, such as, but not limited to, a rectangle 10154 having a minimum preselected side length and comprising approximately 100,000 points ( Figure 1F ). From each portion 10757, 10157 can be removed ( Figure 1D ) points not required for a particular task, such as but not limited to points located above a preselected level, to reduce the dataset size. In some configurations, the preselected level may be AV 10101 ( Figure 1A ). Removing these points can lead to more efficient processing of the dataset.

[0123] Again the main reference Figure 1D , the map processor 10104 can supply at least one data set to the device controller 10111, which can be used to generate direction, speed, and configuration commands to control the AV 10101 ( Figure 1A ). The at least one data set can include points that can be connected to other points in the data set, where each line connecting the points in the data set crosses the drivable surface. To determine such waypoints, the segmented point cloud data 10137 can be divided into polygons 10139, and the vertices of the polygons 10139 may become waypoints. The polygons 10139 can include features such as, for example, SDSF 10141.

[0124] Continue to refer Figure 1D In some configurations, creating the processed point cloud data 10135 can include filtering voxels. To reduce the number of points that will undergo further processing, in some configurations, the centroid of each voxel in the data set can be used to approximate the points in the voxel, and all points except the centroid can be eliminated from the point cloud data. In some configurations, the center of the voxel can be used to approximate the points in the voxel. The segments 10251 ( Figure 1G) size, such as, for example but not limited to, taking a random point subsample so that it is possible to obtain from the filtered segment 10251 ( Figure 1G ) by eliminating a fixed number of points chosen uniformly at random.

[0125] Continue to refer to further Figure 1D In some configurations, creating processed point cloud data 10135 can include calculating normals based on a data set from which outliers have been removed and which has been reduced by voxel filtering. The normals of each point in the filtered data set can be used for various processing possibilities, including curve reconstruction algorithms. In some configurations, estimating and filtering normals in the data set can include using surface meshing techniques to obtain an underlying surface from the data set, and calculating normals based on the surface mesh. In some configurations, estimating normals can include using approximations to directly infer surface normals from the data set, such as, for example, but not limited to, determining the normal of a fitted plane obtained by applying a full least squares method to the k nearest neighbors of a point. In some configurations, the value of k can be selected based on at least empirical data. Filtering normals can include removing any normals that are more than about 45° perpendicular to the xy plane. In some configurations, filters can be used to align normals in the same direction. If a portion of the data set represents a planar surface, redundant information contained in adjacent normals can be filtered out by performing random subsampling or by filtering out a point from a set of related points. In some configurations, picking points can include recursively decomposing the data set into boxes until each box contains at most k points. A single normal can be calculated from the k points in each box.

[0126] Continue to refer Figure 1D In some configurations, creating processed point cloud data 10135 can include growing regions within a data set by clustering points that are geometrically compatible with a surface representing the data set, and refining the surface as the region grows to obtain an approximation of the maximum number of points. Region growing can merge points according to a smoothness constraint. In some configurations, the smoothness constraint can, for example, be determined empirically, or can be based on a desired surface smoothness. In some configurations, the smoothness constraint can include a range of about 10π / 180 to about 20π / 180. The output of region growing is a set of point clusters, each point cluster being a set of points, each point being considered to be part of the same smooth surface. In some configurations, region growing can be based on a comparison of angles between normals. Region growing can be accomplished by, for example, but not limited to, region growing segmentation.

[0127] pointclouds.org / documentation / tutorials / region_growing_segmentation.php and

[0128] pointclouds.org / documentation / tutorials / cluster_extraction.php#cluster-extraction of The algorithm is completed.

[0129] Reference now Figure 1G , segmented point cloud data 10137 ( Figure 1D ) can be used to generate 10161 ( Figure 1D ) polygons 10759, e.g., 5m x 5m polygons. For example, meshing can be used to convert the point sub-clusters into polygons 10759. Meshing can be accomplished by, for example, but not limited to, standard methods such as marching cubes, marching tetrahedrons, surface meshes, greedy meshing, and dual contouring. In some configurations, polygons 10759 can be generated by projecting a local neighborhood of a point along its normal and connecting unconnected points. The resulting polygons 10759 can be based on at least the size of the neighborhood, the maximum acceptable distance of a point to be considered, the maximum edge length of the polygon, the minimum and maximum angles of the polygon, and the maximum deviation that the normals can take from each other. In some configurations, the polygons 10759 can be selected based on whether they are too small to allow AV 10101 ( Figure 1A ) transport to filter polygons 10759. In some configurations, AV 10101 can be dragged around each polygon 10759 by known means ( Figure 1A ) If the circle falls substantially within polygon 10759, then polygon 10759, and therefore the resulting drivable surface, is capable of accommodating AV 10101 ( Figure 1A In some configurations, the area of ​​polygon 10759 can be compared to AV10101 ( Figure 1A ) can be compared to the footprint of AV 10101. It can be assumed that the polygon is irregular, so the first step for determining the area of ​​polygon 10759 is to divide polygon 10759 into regular polygons 10759A by known methods. For each regular polygon 10759A, its size can be determined using the standard area equation. The area of ​​each regular polygon 10759A can be added together to find the area of ​​polygon 10759, and this area can be compared to the footprint of AV 10101 ( Figure 1A ) are compared. The filtered polygons can include a subset of polygons that meet the size criteria. The filtered polygons can be used to set the final drivable surface.

[0130] Continue to refer Figure 1GIn some configurations, polygons 10759 can be processed by removing outliers by conventional means such as, for example, but not limited to, statistical analysis techniques such as those available in the point cloud library pointclouds.org / documentation / tutorials / statistical_outlier.php. Filtering can include reducing the segments 10137 by conventional means including, but not limited to, voxelization grid methods such as those available in the point cloud library pointclouds.org / documentation / tutorials / voxel_grid.php. Figure 1D ). The concave polygon 10263 can be created, for example, but not limited to, by the process set forth in A New ConcaveHull Algorithm and Concaveness Measure for n-dimensional Datasets, Park et al., Journal of Information Science and Engineering 28, pp. 587-600, 2012.

[0131] Now the main reference Figure 1H , in some configurations, the processed point cloud data 10135 ( Figure 1D ) can be used to determine an initial drivable surface 10265. Region growing can generate point clusters that can include points that are part of the drivable surface. In some configurations, to determine the initial drivable surface, a reference plane can be fit to each point cluster. In some configurations, the point clusters can be filtered based on a relationship between the orientation of the point cluster and the reference plane. For example, if the angle between the point cluster plane and the reference plane is less than, for example, but not limited to, about 30°, the point cluster can be preliminarily considered to be part of the initial drivable surface. In some configurations, the point clusters can be filtered based on, for example, but not limited to, size constraints. In some configurations, a point size larger than the point cloud data 10131 ( Figure 1D ) can be considered as too large, and the size is smaller than point cloud data 10131 ( Figure 1D) can be considered too small. The initial drivable surface can include filtered point clusters. In some configurations, the point clusters can be split apart for further processing by any of several known methods. In some configurations, density based spatial clustering of applications with noise (DBSCAN) can be used to split the point clusters, while in some configurations, k-means clustering can be used to split the point clusters. DBSCAN can group points that are closely packed together and mark points that are generally isolated or in low-density areas as outliers. In order to be considered closely packed, a point must be within a preselected distance from a candidate point. In some configurations, a scaling factor for the preselected distance can be determined empirically or dynamically. In some configurations, the scaling factor can be in the range of approximately 0.1 to 1.0.

[0132] Main references Fig. 1I , generating 10163 ( Figure 1D )SDSF lines can include drivable surfaces by further filtering 10265 ( Figure 1H ) to locate the SDSF. In some configurations, points from the point cloud data that make up the polygon can be classified as upper ring points 10351 ( Figure 1J )、lower ring point 10353( Figure 1J ) or cylindrical point 10355 ( Figure 1J ). Upper ring point 10351 ( Figure 1J ) is the shape farthest from the ground that can fall into the SDSF model 10352. The lower ring point 10353 ( Figure 1J ) falls into the shape closest to the ground of the SDSF model 10352. The cylindrical point 10355 ( Figure 1J ) can fall between the upper ring point 10351 ( Figure 1J ) and the lower ring point 10353 ( Figure 1J ) between the shapes. The combination of categories can form a ring 10371. To determine whether a ring 10371 forms an SDSF, certain criteria are tested. For example, in each ring 10371 there must be a point 10351 ( Figure 1J ) the minimum number of points and as the lower ring point 10353 ( Figure 1J ) minimum number of points. In some configurations, the minimum value can be chosen empirically and can fall within a range of about 5-20. Each ring 10371 can be divided into multiple parts, such as two hemispheres. Another criterion for determining whether the points in the ring 10371 represent an SDSF is whether most of the points are located in opposite hemispheres of the parts of the ring 10371. Cylinder points 10355 ( Figure 1J ) can appear in the first cylindrical area 10357 ( Figure 1J ) or the second cylindrical area 10359 ( Figure 1J ). Another criterion for SDSF selection is that in the two cylindrical regions 10357 / 10359 ( Figure 1J ) must have a minimum number of points. In some configurations, the minimum number of points can be chosen empirically and can fall within the range of 3-20. Another criterion for SDSF selection is that the ring 10371 must include three categories of points, namely the upper ring points 10351 ( Figure 1J )、lower ring point 10353( Figure 1J ) and cylindrical point 10355 ( Figure 1J )

[0133] Continue to main reference Fig. 1I , in some configurations, polygons can be processed in parallel. Each class worker 10362 can search its assigned polygon for SDSF points 10789 ( Figure 1N ) and can convert SDSF point 10789 ( Figure 1N ) assigned to category 10763 ( Figure 1G As polygons are processed, the resulting point categories 10763 can be combined 10363 ( Figure 1G ) to form combined category 10366, and category 10365 can be shortened to form shortened combined category 10368. Shortened SDSF point 10789 ( Figure 1N ) can include filtering SDSF points relative to their distance from the ground 10789 ( Figure 1N The shortened combination categories 10368 can be averaged, possibly by an average worker 10373 by searching each SDSF point 10766 ( Figure 1G ) and generates the average point 10765 ( Figure 1G ) are processed in parallel, and the points of this category form a set of average circles 10375. In some configurations, it is possible to empirically determine each SDSF point 10766 ( Figure 1G ) around the radius. In some configurations, each SDSF point 10766 ( Figure 1G ) can include a range between 0.1m and 1.0m. The average point 10765 ( Figure 1G ) of the SDSF trajectory 10377 ( Figure 1G ) between one point and another on the surface. Connecting the average circles 10375 together can generate the SDSF trajectory 10377 ( Figure 1G and 1K ). When creating SDSF track 10377 ( Figure 1G and1K ), if there are two next candidate points within the search radius of the starting point, the next point can be selected based at least on forming a line that is as straight as possible among the previous line segment, the starting point, and the candidate destination point and based on the candidate next point representing the minimum change in SDSF height between the previous point and the candidate next point. In some configurations, the SDSF height can be defined as the upper ring 10351 ( Figure 1J ) and lower ring 10353 ( Figure 1J )’s height.

[0134] Now the main reference Figure 1L , combination 10165 ( Figure 1D ) Convex polygons and SDSF lines can be generated including polygon 10139 ( Figure 1D ) and SDSF 10141 ( Figure 1D ) data set, and can manipulate the data set to generate graphic polygons using SDSF data. Manipulating convex polygons 10263 can include, but is not limited to, merging convex polygons 10263 to form merged convex polygons 10771. Merging convex polygons 10263 can use, for example, but not limited to, ( www.angusj.com / delphi / clipper.php ) can be accomplished by known methods such as those found in . The merged convex polygon 10771 can be enlarged to smooth the edges and form an enlarged polygon 10772. The enlarged polygon 10772 can be retracted to provide a driving margin, thereby forming a retracted polygon 10774, the SDSF trajectory 10377 ( Figure 1M ) can be added to the retracted polygon 10774. Inward trimming (retracting) can ensure that there is room near the edge for AV 10101 ( Figure 1A ) by at least AV10101 ( Figure 1A ) to reduce the size of the drivable surface by a preselected amount for driving. Polygon expansion and contraction can be accomplished by commercially available techniques such as, for example, but not limited to, the ARCGIS® Clipping command (desktop.arcgis.com / en / arcmap / 10.3 / manage-data / editing-existing-features / clipping-a-polygon-feature.htm).

[0135] Now the main reference Figure 1M, the retracted polygon 10774 can be partitioned into polygons 10778, each of which can be traversed without encountering a non-drivable surface. The retracted polygon 10774 can be partitioned by conventional means such as, but not limited to, ear slicing optimized by z-order curve hashing and extended to handle holes, distorted polygons, degeneracy, and self-intersections. Commercially available ear slicing implementations can include, but are not limited to, those found in (github.com / mapbox / earcut.hpp). The SDSF trajectory 10377 can include SDSF points 10789 ( Figure 1N ). Vertices 10781 can be considered to be connected to each other as AV 10101 ( Figure 1A ) form possible path points of possible driving paths. In the data set, SDSF point 10789 can be marked as follows ( Figure 1N ). As partitioning proceeds, it is possible that redundant edges are introduced, such as but not limited to edges 10777 and 10779. Removing one of edges 10777 or 10779 can reduce the complexity of further analysis and can preserve the polygon mesh. In some configurations, the Hertel-Mehlhorn polygon partitioning algorithm can be used to remove edges, thereby skipping edges that have been marked as features. The set of polygons 10778 that include the marked features is further simplified to reduce the number of possible waypoints, and the possible waypoints can be annotated with point data 10379 ( Figure 5B ) is provided to the device controller 10111 ( Figure 1A ).

[0136] Reference now Figure 2A-2B , sensor data collected by the AV can also be used to populate the occupancy grid. The processor in the AV can receive data from sensors in the long-range sensor assembly 20400 mounted above the cargo container 20110 and from short-range sensors 20510, 20520, 20530, 20540 and other sensors located in the cargo platform 20160. In addition, the processor can receive data from an optional short-range sensor 20505 mounted near the top of the front of the cargo container 20110. The processor can also receive data from one or more antennas 20122A, 20122B (including cellular, WiFi, and / or GPS) Figure 1C ) to receive data. In one example, the AV 20100 has a GPS antenna 20122A located on the long-range sensor assembly 20400 ( Figure 1C ) and / or antenna 20122B located on top of cargo container 20110 ( Figure 1C). The processor may be located anywhere in the AV 20100. In some examples, one or more processors are located in the long-range sensor assembly 20400. Additional processors may be located in the cargo platform 20160. In other examples, the processor may be located in the cargo container 20110 and / or as part of the powered base 20170.

[0137] Continue to refer Figure 2A-2B , the long-range sensor assembly 20400 is mounted above the cargo container to provide an improved view of the environment around the AV. In one example, the long-range sensor assembly 20400 is more than 1.2m above the driving surface or ground. In other examples, where the cargo container is higher or the powered base configuration raises the cargo platform 20160, the long-range sensor assembly 20400 can be 1.8m above the ground above which the AV is moving. The long-range sensor assembly 20400 provides information about the environment around the AV from a minimum distance to a maximum range. The minimum distance can be defined by the relative position of the long-range sensor 20400 and the cargo container 20110. The minimum distance can be further defined by the field of view (FOV) of the sensor. The maximum distance can be defined by the range of the long-range sensor in the long-range sensor assembly 20400 and / or by the processor. In one example, the range of the long-range sensor is limited to 20 meters. In one example, the range of the Velodyne Puck LIDAR can be up to 100m. The long-range sensor assembly 20400 can provide data about objects in all directions. The sensor assembly may provide information about structures, surfaces, and obstacles around the AV 20100 over a 360° angle.

[0138] Continue to refer Figure 2A, the three long range cameras viewing through windows 20434, 20436, and 20438 are able to provide horizontal FOVs 20410, 20412, 20414 that together provide a 360° FOV. The horizontal FOV can be defined by the selected camera and the position of the camera within the long range camera assembly 20400. When describing the field of view, zero angle is a ray that is located in a vertical plane through the center of the AV 20100 and is perpendicular to the front of the AV. The zero angle ray passes through the front of the AV. The front long range camera viewing through window 20434 has a 96° FOV 20410 from 311° to 47°. The left long range camera viewing through window 20436 has a FOV 20412 from 47° to 180°. The right long range camera viewing through window 20438 has a FOV 20414 from 180° to 311°. The long range sensor assembly 20400 may include an industrial camera positioned to view through the window 20432 that provides more detailed information about objects and surfaces in front of the AV 20100 than the long range camera. The industrial camera behind the window 20432 may have a FOV 20416 defined by the selected camera and the position of the camera within the long range camera assembly 20400. In one example, the industrial camera behind the window 20432 has a FOV from 23° to 337°.

[0139] Reference now Figure 2B , the LIDAR 20420 provides a 360° horizontal FOV around the AV 20100. The vertical FOV may be limited by the LIDAR instrument. In one example, a vertical FOV 20418 of 40° and mounted 1.2m to 1.8m above the ground sets the minimum distance of the sensor at 3.3m to 5m from the AV 20100.

[0140] Reference now Figure 2C and Figure 2D , the long range sensor assembly 20400 is shown with a cover 20430. The cover 20430 includes windows 20434, 20432, 20436 through which the long range camera and industrial camera observe the environment around the AV 20100. The cover 20430 for the long range sensor assembly 20400 is sealed by an O-ring between the cover 20430 and the top of the cargo container 20110 to prevent weathering.

[0141] Reference now Figure 2E and Figure 2F, the cover 20430 has been removed to reveal an example of a camera and processor. The LIDAR sensor 20420 provides data about the range or distance to the surface around the AV. This data can be provided to the processor 20470 located in the long-range sensor assembly 20400. The LIDAR is mounted on the structure 20405 above the long-range cameras 20440A-C and the cover 20430. The LIDAR sensor 20420 is an example of a ranging sensor based on reflected laser pulse light. Other ranging sensors, such as radars using reflected radio waves, can also be used. In one example, the LIDAR sensor 20420 is a Puck sensor from VELODYNE LIDAR® of San Jose, California. The three long-range cameras 20440A, 20440B, 20440C provide digital images of objects, surfaces, and structures around the AV 20100. Three long range cameras 20440A, 20440B, 20440C are arranged around the structure 20405 relative to the cover 20430 to provide three horizontal FOVs covering a full 360° around the AV. The long range cameras 20440A, 20440B, 20440C are located on an elevated ring structure 20405 mounted to the cargo container 20110. The long range cameras 20440A, 20440B, 20440C receive images through windows 20434, 20436, 20438 mounted in the cover 20430. The long range cameras may include cameras and lenses on a printed circuit board (PCB).

[0142] Reference now Figure 2F , one example of a long range camera 20440A can include a digital camera 20444 having a fisheye lens 20442 mounted in front of the digital camera 20444. The fisheye lens 20442 can expand the FOV of the camera to a much wider angle. In one example, the fisheye lens expands the field of view to 180°. In one example, the digital camera 20444 is similar to the e-cam52A_56540_MOD of E-con Systems of San Jose, California. In one example, the fisheye lens 20442 is similar to the model DSL227 of Sunex of Carlsbad, California.

[0143] Continue to refer Figure 2F , the long range sensor assembly 20400 may also include an industrial camera 20450 that receives visual data through a window 20432 in the cover 20430. The industrial camera 20450 provides additional data about objects, surfaces, and structures in front of the AV to the processor 20470. The camera may be similar to Kowa industrial camera part number LM6HC. The industrial camera 20450 and long range cameras 20440A-C are located 1.2m to 1.8m above the surface over which the AV 20100 is moving.

[0144] Continue to refer Figure 2F , mounting the long-range sensor assembly 20400 above the cargo container provides at least two advantages. When the sensors are mounted farther above the ground, the field of view of the long-range sensors including the long-range cameras 20440A-C, the industrial cameras 20450, and the LIDAR 20420 is less blocked by nearby objects such as people, cars, low walls, etc. Additionally, the sidewalk is structured to provide visual cues including signs, fence heights, etc. for people to perceive, and typical eye levels are in the range of 1.2m to 1.8m. Mounting the long-range sensor assembly 20400 to the top of the cargo container places the long-range cameras 20440A-C, 20450 on the same level as the signs and over the visual cues for pedestrians. The long-range sensors are mounted on a structure 20405 that provides a strong and rigid mount that resists deflection caused by the movement of the AV 20100.

[0145] Reference again Figure 2E and Figure 2F , the long-range sensor assembly may include an inertial measurement unit (IMU) and one or more processors that receive data from the long-range sensor and output the processed data to other processors for navigation. An IMU 20460 with a vertical reference (VRU) is mounted to the structure 20405. The IMU / VRU 20460 may be located directly below the LIDAR 20420 to provide position data about the LIDAR 20420. The position and orientation from the IMU / VRU 20460 may be combined with data from other long-range sensors. In one example, the IMU / VRU 20460 is a model MTi20 supplied by Xsens Technologies of the Netherlands. The one or more processors may include a processor 20465 that receives data from at least an industrial camera 20450. In addition, the processor 20470 may receive data from at least one of the following LIDAR 20420, long-range cameras 20440A-C, industrial cameras 20450, and IMU / VRU 20460. The processor 20470 can be cooled by a liquid-cooled heat exchanger 20475 connected to a circulating coolant system.

[0146] Reference now Figure 2G , the AV 20100 may include a number of short-range sensors that detect driving surfaces and obstacles within a predetermined distance from the AV. Short-range sensors 20510, 20520, 20530, 20540, 20550, and 20560 are located on the perimeter of the container platform 20160. These sensors are located on the cargo container 20110 ( Figure 2B) and below the long range sensor assembly 20400 ( Figure 2C ) is closer to the ground. Short range sensors 20510, 20520, 20530, 20540, 20550 and 20560 are tilted downward to provide FOVs that capture the long range sensor assembly 20400 ( Figure 2C ) cannot see surfaces and objects that sensors in the AV 20100 cannot see. The field of view of sensors that are closer to the ground and tilted downward is less likely to be blocked by nearby objects and pedestrians than sensors that are mounted farther from the ground. In one example, the short-range sensor provides information about ground surfaces and objects up to 4m away from the AV 20100.

[0147] Reference again Figure 2B , the vertical FOVs of two short range sensors are shown in a side view of the AV 20100. The vertical FOV 20542 of the rearward facing sensor 20540 is centered on the centerline 20544. The centerline 20544 is tilted below the top surface of the cargo platform 20160. In one example, the sensor 20540 has a vertical FOV of 42° and a centerline 20546 tilted 22° to 28° below the plane 20547 defined by the top plate of the cargo platform 20160. In the example, the short range sensors 20510 and 20540 are approximately 0.55m to 0.71m above the ground. The resulting vertical FOV 20512, 20542 covers the ground from 0.4m to 4.2m from the AV. The short range sensors 20510, 20520, 20530, 20550 ( Figure 2G )、20560( Figure 2G ) have a similar vertical field of view and centerline angle relative to the top of the cargo platform. The short-range sensor mounted on the cargo platform 20160 is able to observe the ground from 0.4 meters to 4.7 meters outward from the outside edge of the AV 20100.

[0148] Continue to refer Figure 2B , a short range sensor 20505 can be mounted on the front surface near the top of the cargo container 20110. In one example, the sensor 20505 can provide an additional view of the ground in front of the AV to the view provided by the short range sensor 20510. In another example, the sensor 20505 can provide a view of the ground in front of the AV in place of the view provided by the short range sensor 20510. In one example, the short range sensor 20505 can have a vertical FOV 20507 of 42° and the angle of the centerline to the top of the cargo platform 20160 is 39°. The resulting view of the ground extends from 0.7m to 3.75m from the AV.

[0149] Reference again Figure 2G, the horizontal FOV of the short-range sensors 20510, 20520, 20530, 20540, 20550, 20560 covers all directions around the AV 20100. The horizontal FOVs 20522 and 20532 of adjacent sensors such as 20520 and 20530 overlap at a certain distance outward from the AV 20100. In one example, the horizontal FOVs 20522, 20532 and 20562, 20552 of adjacent sensors 20520, 20530 and 20560, 20550 overlap at 0.5 meters to 2 meters from the AV. The short-range sensors are distributed around the perimeter of the cargo base 20160, have a horizontal field of view, and are placed at a specific angle to provide nearly complete visual coverage of the ground around the AV. In one example, the short-range sensors have a horizontal FOV of 69°. The front sensor 20510 faces forward at zero angle relative to the AV and has a FOV 20512. In one example, the two front corner sensors 20520, 20560 are tilted so that the centerline is at an angle 20564 of 65°. In an example, the rear sensors 20530, 20550 are tilted so that the centerline of 20530 and 20560 is at an angle 20534 of 110°. In some configurations, other numbers of sensors with other horizontal FOVs mounted around the perimeter of the cargo bed 20160 to provide a nearly complete view of the ground around the AV 20100 are possible.

[0150] Reference now Figure 2H , short range sensors 20510, 20520, 20530, 20540, 20550, 20560 are located on the perimeter of the cargo base 20160. The short range cameras are mounted in protrusions that set the angle and position of the short range sensors. In another configuration, the sensors are positioned mounted on the interior of the cargo base and receive visual data through windows aligned with the outer skin of the cargo base 20160.

[0151] Reference now Fig.2I and Figure 2J, the short distance sensor 20600 is mounted in the skin member 20516 of the cargo base 20160 and may include a liquid cooling system. The skin member 20516 includes a formed protrusion 20514 that maintains the short distance sensor assembly 20600 at a predetermined position and vertical angle relative to the top of the cargo base 20160 and at an angle relative to the front of the cargo base 20160. In some configurations, the short distance sensor 20510 is tilted downwardly 28° relative to the cargo platform 20160, the short distance sensors 20520 and 20560 are tilted downwardly 18° and tilted forwardly 25°, the short distance sensors 20530 and 20550 are tilted downwardly 34° and tilted rearwardly 20°, and the short distance sensor 20540 is tilted downwardly 28° relative to the cargo platform 20160. The skin member 20516 includes a cavity 20517 for receiving the camera assembly 20600. The skin element 20516 can also include a plurality of elements 20518 to receive mechanical fasteners including, but not limited to, rivets, screws, and buttons. Alternatively, the camera assembly can be mounted with an adhesive or secured in place with a clip secured to the skin element 20516. A gasket 20519 can provide a seal to the front of the camera 20610.

[0152] Reference now Figure 2K and Figure 2L , the short distance sensor assembly 20600 includes a short distance sensor 20610 mounted on a bracket 20622 attached to a water cooling plate 20626. The outer shell 20612, the transparent cover 20614 and the heat sink 20618 are arranged in a Figure 2K and Figure 2L The sensor block 20616 and the electronics block 20620 have been partially removed to better visualize the heat dissipation elements of the short distance sensor 20610. The short distance sensor assembly 20600 may include one or more thermoelectric coolers (TECs) 20630 between the bracket 20622 and the liquid cold plate 20626. The liquid cold plate 20626 is cooled by a coolant pumped through 20628 that is thermally connected to the plate 20626. A TEC is an electrically powered element having a first side and a second side. The electrically powered TEC cools the first side while rejecting thermal energy removed from the first side plus electrical power at the second side. In the short distance sensor assembly 20600, the TEC 20630 cools the bracket 20622 and transmits thermal cooling energy plus electrical energy to the water cold plate 20626. Alternatively, the TEC 20630 can be used to actively control the temperature of the camera 20600 by changing the amplitude and polarity of the voltage supplied to the TEC 20630.

[0153] Operating the TEC 20630 in cooling mode allows the short distance sensor 20610 to operate at a temperature below the coolant temperature. The bracket 20622 is thermally connected to the short distance sensor 20610 in two locations to maximize cooling of the sensor block 20616 and the electronics block 20620. The bracket 20622 includes a lug 20624 that is thermally attached to the heat sink 20618 via 20625. The heat sink 20618 is thermally connected to the sensor block 20616. The bracket is thus thermally connected to the sensor block 20616 via the heat sink 20618, the screws 20625 and the lug 20624. The bracket 20622 is also mechanically attached to the electronics block 20620 to provide direct cooling of the electronics block 20620. The bracket 20622 may include a number of mechanical attachments, including but not limited to Figure 2J The screws and rivets that engage the elements 20518 in the short-range sensor 20610 can incorporate one or more sensors, including but not limited to cameras, stereo cameras, ultrasonic sensors, short-range radars, and infrared projectors and CMOS sensors. An example short-range sensor is similar to Intel's RealSense Depth Camera D435 of Santa Clara, California, which includes an IR projector, two imager chips, and an RGB camera.

[0154] Reference now Figure 2M-2O , another embodiment of an AV 20100 is shown. The AV 20100A includes a cargo container 20110 mounted on a cargo platform 20160 and a powered base 20170. The AV 20100A includes a plurality of long-range and short-range sensors. The primary long-range sensor is mounted in a sensor pylon 20400A above the cargo container 20110. The sensor pylon may include a LIDAR 20420 and a plurality of long-range cameras (not shown) aimed in different directions to provide a wide field of view. In some configurations, the LIDAR 20420 can be used as described elsewhere herein, for example but not limited to providing point cloud data that enables filling of an occupancy grid, and providing information to identify landmarks, locate the AV 20100 within its environment, and / or determine a navigable space. In some configurations, a long-range camera from Leopard Imaging Inc. can be used to identify landmarks, locate the AV 20100 within its environment, and / or determine a navigable space.

[0155] Continue to refer Figure 2M-2O, short-range sensors are primarily mounted in the cargo platform 20160 and provide information about obstacles near the AV 20100A. In some embodiments, the short-range sensors supply data about obstacles and surfaces within 4 m of the AV 20100A. In some configurations, the short-range sensors provide information up to 10 m from the AV 20100A. A plurality of cameras, at least partially facing forward, are mounted in the cargo platform 20160. In some configurations, the plurality of cameras can include three cameras.

[0156] Reference now Fig.2O , the top cover 20830 has been partially cut away to reveal the secondary roof 20810. The secondary roof 20810 provides a single piece on which multiple antennas 20820 can be mounted. In the example, ten antennas 20820 are mounted to the secondary roof 20810. In addition, for example, four cellular communication channels each have two antennas, and there are two WiFi antennas. The antennas are wired as main antennas and auxiliary antennas for cellular transmission and reception. The auxiliary antenna can improve cellular functionality by several methods including but not limited to reducing interference and achieving 4G LTE connection. The secondary roof 20810 and the top cover 20830 are non-metallic. The secondary roof 20810 is a plastic surface within 10mm-20mm of the top cover 20830, which is not structural and allows the antenna to be connected to the processor before the top cover 20830 is attached. Antenna connections are often high impedance and sensitive to dirt, grease and misoperation. Mounting and connecting the antenna to the secondary roof 20810 allows the top cover to be installed and removed without touching the antenna connection. Maintenance and repair operations may include removing the roof without removing the secondary roof or disconnecting the antenna. Assembly of the antenna in addition to installing the roof 20830 facilitates testing / repair. The roof 20830 is weatherproof and prevents water and sand from entering the cargo container 20110. Mounting the antenna on the secondary roof minimizes the number of openings on the roof 20830.

[0157] Reference now Figure 2P , Figure 2Q and Figure 2R, another example of a long range sensor assembly (LRSA) 20400A mounted above a cargo container (not shown) is shown. The LRSA can include a LIDAR and a plurality of long range cameras mounted at different locations on the LRSA structure 20950 to provide a panoramic view of the environment of the AV 20100A. The LIDAR 20420 is mounted on the topmost portion of the LRSA structure 20950 to provide an uninterrupted view. The LIDAR can include a VELODYNELIDAR. A plurality of long range cameras 20910A-20910D are mounted on the LRSA structure 20950 on the next level below the LIDAR 20420. In the example, four cameras are mounted, with one camera every 90° around the structure to provide four views of the environment around the AV 20100. In some examples, the four views will overlap. In some examples, each camera is either aligned with the direction of motion or orthogonal to the direction of motion. In the example, one camera is aligned with each major face of the AV 20100A - front, back, left and right. In the example, the long range camera is model LI-AR01 44-MIPI-M12 manufactured by Leopard Imaging Inc. The long range camera may have a MIPI CSI-2 interface to provide high speed data transfer to the processor. The long range camera may have a horizontal field of view between 50° and 70° and a vertical field of view between 30° and 40°.

[0158] Reference now Figure 2S and Figure 2T, the long range processor 20940 is located on the LRSA structure 20950 below the long range cameras 20910A-20910D and the LIDAR 20420. The long range processor 20940 receives data from the long range cameras and LIDAR. The long range processor communicates with one or more processors elsewhere in the AV 20100A. As described elsewhere herein, the long range processor 20940 provides data derived from the long range cameras and LIDAR to one or more processors located elsewhere on the AV 20100A. The long range processor 20940 can be liquid cooled by a cooler 20930. The cooler 20930 can be mounted to a structure below the long range cameras and LIDAR. The cooler 20930 can provide a mounting location for the long range processor 20940. The cooler 20930 is described in U.S. Patent Application No. 16 / 883,668 (Agent Docket No. AA280), filed on May 26, 2020, entitled Apparatus for Electronic Coolingonan Autonomous Device, which is incorporated herein by reference in its entirety. The cooler is provided with a liquid supply conduit and a return conduit for providing coolant to the cooler 20930.

[0159] Reference again Figure 2M and 2N , short-range camera assemblies 20740A-C are mounted on the front of the container platform 20160 and angled to collect information about the driving surface as well as obstacles, steps, curbs, and other generally discontinuous surface features (SDSF). The camera assemblies 20740A-C include one or more LED lights to illuminate the driving surface, objects on the ground, and the SDSF.

[0160] Reference now Figure 2U-2X, camera assembly 20740A-B includes lamp 20732 to illuminate the ground and objects to provide improved image data from camera 20732. Note that camera assembly 20740A is a mirror image of 20740C and the description of 20740A is implicitly applicable to 20740C. Camera 20732 may include a single-view camera, a stereo camera and / or an infrared projector and a CMOS sensor. An example of a camera is Intel's RealSense Depth D435 camera in Santa Clara, California, which includes an IR projector, two imager chips with lenses, and an RGB camera. LED lamp 20734 can be used at night or in low light conditions or can be used to improve image data all the time. A theory of operation is that the lamp creates contrast by illuminating the projection surface and creating shadows in the recess. In the example, the LED lamp can be a white LED. In the example, LED lamp 20734 is Xlamp XHP50 from Cree Inc. In another example, LED lights may emit in the infrared to provide illumination for the camera 20372 without distracting or annoying nearby pedestrians or drivers.

[0161] Continue to refer Figure 2U-2X , the placement and angle of the light 20374 and the shape of the covers 20736A, 20736B prevent the camera 20732 from seeing the light 20374. The angle and placement of the light 20374 and covers 20736A, 20736B prevent the light from interfering with the driver or annoying pedestrians. It is advantageous for the camera 20732 not to be exposed to the light 20734 to prevent the sensor in the camera 20732 from being blinded by the light 20734 and thus being prevented from detecting lower light signals from the ground and objects in front of and to the sides of the AV 20100A. The camera 20732 and / or the light 20734 can be cooled with liquid that flows into and out of the camera assembly through port 20736.

[0162] Reference now Figure 2W and Figure 2X , the short-range camera assembly 20740A includes an ultrasonic or sonar short-range sensor 20730A. The second short-range camera assembly 20740C also includes an ultrasonic short-range sensor 20730B ( Figure 2N ).

[0163] Reference now Figure 2Y , the ultrasonic sensor 20730A is mounted above the camera 20732. In the example, the centerline of the ultrasonic sensor 20730A is parallel to the base of the cargo container 20110, which often means that the sensor 20730A is horizontal. The sensor 20730A is at a 45° angle to facing forward. The cover 20376A provides a horn 20746 to direct the ultrasound emitted from the ultrasonic sensor 20730A and received by the ultrasonic sensor 20730A.

[0164] Continue to refer Figure 2Y , a cross section of camera 20732, lights 20734 within camera assembly 20740A illustrate the angles and openings in cover 20736. The cameras in the short range camera assemblies are tilted downward to better image the ground in front of and to the sides of the AV. The center camera assembly 20740B is oriented directly forward in the horizontal plane. The corner camera assemblies 20740A, 20740C are angled 25° in the horizontal plane relative to directly forward to their respective sides. Camera 20732 is tilted downward 20° in the vertical plane relative to the top of the cargo platform. Since the AV typically keeps the cargo platform level, the camera is therefore angled 20° below the horizontal. Similarly, the center camera assembly 20740B ( Figure 2M ) is angled 28° below the horizontal. In an example, the cameras in the camera assembly can be tilted downward by 25° to 35°. In another example, the cameras in the camera assembly can be tilted downward by 15° to 45°. The LED lights 20734 are similarly tilted downward to illuminate the ground imaged by the camera 20732 and minimize distraction to pedestrians. In one example, the LED light centerline 20742 is parallel within 5° of the camera centerline 20738. The cover 20736A protects both the camera 20732 and pedestrians from the bright light of the LED 20734 in the camera assembly 20740A-C. The cover that isolates the light emitted by the LED also provides a flared opening 20737 to maximize the field of view of the camera 20732. The light is recessed at least 4 mm from the opening of the cover. The light opening is defined by an upper wall 20739 and a lower wall 20744. The upper wall 20739 is approximately parallel (±5°) to the centerline 2074. The lower wall 20744 flares approximately 18° from the centerline 20742 to maximize illumination of the ground and objects near the ground.

[0165] Reference now Figure 2Z-2AA, in one configuration, the lamp 20734 includes two LEDs 20734A, each LED below a square lens 20734B to produce a light beam. The LED / lenses are tilted and positioned relative to the camera 20372 to illuminate the camera's field of view, with minimal spillage of the lamp outside the FOV of the camera 20732. The two LED / lenses are mounted together on a single PCB 20752 with a defined angle 20762 between the two lamps. In another configuration, the two LED / lenses are individually mounted on a heat sink 20626A on a separate PCB at a certain angle relative to each other. For example, the lamp is an Xlamp XHP50 from Cree, Inc., and the lens is a 60° lens HB-SQ-W from LEDil. The lamps are at an angle of approximately 50° relative to each other, so the angle 20762 between the fronts of the lenses is 130°. The light is located approximately 18 mm (±5 mm) 20764 behind the front of the camera 20732 and approximately 30 mm 20766 below the centerline of the camera 20732 .

[0166] Reference now Figure 2AA-2BB , the camera 20732 is cooled by a thermoelectric cooler (TEC) 20630, which along with the lamp 20734 is cooled by a liquid coolant flowing through a cold block 20626A. The camera is attached to the bracket 20622 via screws 20625, which screw into the sensor block portion of the camera, while the back of the bracket 20622 is bolted to the electronics block of the camera. The bracket 20622 is cooled by two TEzCs in order to maintain the performance of the IR imaging chip (CMOS chip) in the camera 20732. The TECs reject heat from the bracket 20622 and the electrical energy they draw to the cold block 20626A.

[0167] Reference now Figure 2BB , the coolant is directed through a U-shaped path created by the center wing 20626D. The coolant flows directly behind the LED / lens / PCB of the lamp 20734. The wings 20626B, 20626C improve the heat transfer from the lamp 20734 to the coolant. The coolant flows upward to pass over the hot side of the TEC 20630. The fluid path is connected by a plate 20737 ( Figure 2X )create.

[0168] Reference now Figure 3A, sensor data and map data can be used to update an occupancy grid. The systems and methods of the present teachings can manage a global occupancy grid for a device that is autonomously navigating relative to a grid map. The grid map can include routes or paths that a device can follow from a starting point to a destination. The global occupancy grid can include free space indications that can indicate places for the device to safely navigate. Possible paths and free space indications can be combined on the global occupancy grid to establish an optimal path that the device can travel on to safely reach a destination.

[0169] Continue to refer Figure 3A , as the device moves, a global occupancy grid that will be used to determine an obstacle-free navigation route can be accessed based on the location of the device, and the global occupancy grid can be updated as the device moves. The update can be based on at least a current value associated with the global occupancy grid at the location of the device, a static occupancy grid that can include historical information about the neighborhood in which the device is navigating, and data being collected by sensors as the device travels. As described herein, sensors can be located on the device, and they can be located elsewhere.

[0170] Continue to refer to further Figure 3A , a global occupancy grid can include cells, and cells can be associated with occupancy probability values. Each cell of the global occupancy grid can be associated with information such as whether an obstacle has been identified at the location of the cell, characteristics and discontinuities of the driving surface at and around the location as determined based on previously collected data and as determined by data collected as the device navigates, and prior occupancy data associated with the location. Data captured as the device navigates can be stored in a local occupancy grid centered on the device. When updating the global occupancy grid, previously collected static data can be combined with local occupancy grid data and global occupancy data determined in a previous update to create a new global occupancy grid, while marking the space occupied by the device as unoccupied. In some configurations, a Bayesian approach can be used to update the global occupancy grid. The method can include, for each cell in the local occupancy grid, calculating the cell's position on the global occupancy grid, accessing a value at the position from the current global occupancy grid, accessing a value at the position from the static occupancy grid, accessing a value at the position from the local occupancy grid, and calculating a new value for the position on the global occupancy grid based on the current value from the global occupancy grid, the value from the static occupancy grid, and the value from the local occupancy grid. In some configurations, the relationship used to calculate the new value can include the sum of the static value and the local occupancy grid value minus the current value. For example, in some configurations, the new value may be constrained by a preselected value based on a computational constraint.

[0171] Continue to refer Figure 3A, the system 30100 of the present teaching can manage a global occupancy grid. The global occupancy grid can start from initial data and can be updated as the device moves. Creating an initial global occupancy grid can include a first process, and updating the global occupancy grid can include a second process. The system 30100 can include, but is not limited to, a global occupancy server 30121 that can receive information from various sources and can update the global occupancy grid 30505 based on at least the information. The information can be supplied by, for example, but not limited to, sensors, static information, and navigation information located on the device and / or elsewhere. For example, in some configurations, the sensor can include a camera and a radar that can detect surface features and obstacles. The sensor can be advantageously located on the device, for example, to provide sufficient coverage of the surrounding environment to enable the device to travel safely. In some configurations, the LIDAR 30103 can provide a LIDAR point cloud (PC) data 30201 that can enable filling of a local occupancy grid with LIDAR free space information 30213. In some configurations, a conventional ground detection inverse sensor model (ISM) 30113 can process the LIDAR PC data 30201 to generate LIDAR free space information 30213.

[0172] Continue to refer Figure 3AIn some configurations, the RGB-D camera 30101 can provide RGB-D PC data 30202 and RGB camera data 30203. The RGB-D PC data 30202 can populate the local occupancy grid with depth free space information 30209, while the RGB-D camera data 30203 can populate the local occupancy grid with surface data 30211. In some configurations, the RGB-D PC data 30202 can be processed by, for example, but not limited to, a conventional stereo free space ISM 30109, while the RGB-D camera data 30203 can be fed to, for example, but not limited to, a conventional surface detection neural network 30111. In some configurations, the RGB MIPI camera 30105 can provide RGB data 30205 to generate a local occupancy grid with LIDAR / MIPI free space information 30215 in conjunction with the LIDAR PC data 30201. In some configurations, the RGB data 30205 can be fed to a conventional free space neural network 30115, the output of which can be subjected to a pre-selected mask 30221 that can identify which parts of the RGB data 30205 are most important for accuracy before being fed to a conventional 2D-3D registration 30117 along with the LIDAR PC data 30201. The 2D-3D registration 30117 can project an image from the RGB data 30205 onto the LIDAR PC data 30201. In some configurations, 2D-3D registration 30117 is not required. Any combination of sensors and methods for processing sensor data can be used to collect data to update the global occupancy grid. Any number of free space estimation processes can be used and combined to enable the determination and verification of occupancy probabilities in the global occupancy grid.

[0173] Continue to refer Figure 3A In some configurations, historical data can be provided by, for example, a repository 30107 of previously collected and processed data having information associated with a navigation area. In some configurations, the repository 30107 can include, for example, but not limited to, route information such as, for example, polygons 30207. In some configurations, this data can be fed to a conventional polygon parser 30119 that can provide edges 30303, discontinuities 30503, and surfaces 30241 to a global occupancy grid server 30121. The global occupancy grid server 30121 can fuse the local occupancy grid data collected by the sensors with the processed repository data to determine a global occupancy grid 30505. A grid map 30601 can be created from the global occupancy data ( Figure 3D ).

[0174] Reference now Figure 3BIn some configurations, the sensor can include a sonar 30141 that can provide a local occupancy grid with sonar free space 30225 to a global occupancy grid server 30121. The depth data 30209 can be processed by a conventional free space ISM 30143. The local occupancy grid 30225 with sonar free space can be combined with a local occupancy grid with surfaces and discontinuities 30223, a local occupancy grid with LIDAR free space 30213, a local occupancy grid with LIDAR / MIPI free space 30215, a local occupancy grid with stereo free space 30209, and edges 30303 ( Figure 3F )、Discontinuous 30503( Figure 3F )、Navigation point 30501( Figure 3F ), surface confidence 30513 ( Figure 3F ) and surface 30241 ( Figure 3F ) are fused to form a global occupancy grid 30505.

[0175] Reference now Figures 3C-3F To initialize the global occupancy grid, the global occupancy grid initialization 30200 can include creating a global occupancy grid 30505 and a static grid 30249 by the global occupancy grid server 30121. The global occupancy grid 30505 can be created by fusing data from the local occupancy grid 30118 with edges 30303, discontinuities 30503, and surfaces 30241 located in the region of interest. The static grid 30249 can be created ( Figure 3D ) to include data such as, for example, but not limited to, surface data 30241, discontinuity data 30503, edges 30303, and polygons 30207. It is possible to convert the static grid 30249 ( Figure 3E ) is added to the occupancy data derived from the data collected from the sensor 30107A and from the prior 30505A ( Figure 3F ) to calculate the initial global occupancy grid 30505. The local occupancy grid 30118 can include, but is not limited to, the stereoscopic free space estimate 30209 ( Figure 3B ) produces local occupancy grid data, including surface / discontinuity detection results 30223 ( Figure 3B ) local occupancy grid data, free space estimation by LIDAR through ISM 30213 ( Figure 3B ) and the local occupancy grid data generated by following the 2D-3D registration 30117 ( Figure 3B ) of LIDAR / MIPI free space estimation 30215 ( Figure 3B) generated by the sonar free space estimate 30225. In some configurations, the local occupancy grid 30118 can include local occupancy grid data generated by the sonar free space estimate 30225 through the ISM. In some configurations, the various local occupancy grids with free space estimates can be fused into the local occupancy grid 30118 according to a preselected known process. A grid map 30601 (which can include occupancy and surface data near the device) can be created from the global occupancy grid 30505. Figure 3E In some configurations, the grid map 30601 can be published using, for example but not limited to, the Robot Operating System (ROS) subscribe / publish feature ( Figure 3E ) and static grid 30249 ( Figure 3D ).

[0176] Reference now Figure 3G and Figure 3H , to update the occupancy grid as the device moves, occupancy grid update 30300 can include updating the local occupancy grid relative to data measured while the device is moving, and combining that data with static grid 30249. Static grid 30249 is accessed when the device moves out of the working occupancy grid range. The device can be positioned in occupancy grid 30245A at a first position 30513A at a first time. As the device moves to a second position 30513B, the device is positioned in occupancy grid 30245B, which includes a set of values ​​derived from its new position and possibly from values ​​in occupancy grid 30245A. Data from static grid 30249 and from an initial global occupancy grid 30505 (which locally coincides with cells in occupancy grid 30245B at the second time) are combined. Figure 3C ) can be used together with the measured surface data and occupancy probability to update each grid cell according to a preselected relationship. In some configurations, the relationship can include adding static data to the measured data. The resulting occupancy grid 30245C at the third time and third location 30513C can be provided to the mobile manager 30123 to inform the navigation of the device.

[0177] Reference now Fig. 3I , a method 30450 for creating and managing an occupancy grid can include, but is not limited to, including, by a local occupancy grid creation node 30122, transforming 30451 sensor measurements to a reference frame associated with a device, creating 30453 a timestamped measurement occupancy grid, and using the timestamped measurement occupancy grid as a local occupancy grid 30234 ( Figure 3G ) publish 30455. A system associated with method 30450 can include multiple local grid creation nodes 30122, for example, one for each sensor, so that multiple local occupancy grids 30234 ( Figure 3G). The sensors can include but are not limited to RGB-D camera 30325 ( Figure 3G )、LIDAR / MIPI 30231( Figure 3G ) and LIDAR 30233 ( Figure 3G ). A system associated with method 30450 can include a global occupancy grid server 30121 that can receive local occupancy grids and process them according to method 30450. In particular, method 30450 can include loading 30242 surfaces, accessing 30504 surface discontinuities such as, for example, but not limited to, curbs, and creating 30248 a static occupancy grid 30249 from any features available in repository 30107, which can include, for example, but not limited to, surfaces and surface discontinuities. Method 30450 can include receiving 30456 published local occupancy grids and moving 30457 the global occupancy grid to maintain the device in the center of the map. Method 30450 can include setting 30459 a new area on the map with prior information from the static prior occupancy grid 30249, and marking 30461 the area currently occupied by the device as unoccupied. Method 30450 can perform loop 30463 for each cell in each local occupancy grid. Loop 30463 can include, but is not limited to, including calculating the position of the cell on the global occupancy grid, accessing the value at the position on the global occupancy grid, and calculating a new value at the cell position based on the relationship between the previous value and the value at the cell in the local occupancy grid. The relationship can include, but is not limited to, including the sum of the calculated values. Loop 30463 can include comparing the new value against a preselected acceptable probability range and setting the global occupancy grid with the new value. The comparison can include setting the probability to the minimum or maximum acceptable probability if the probability is below or above the minimum or maximum acceptable probability. Method 30450 can include publishing 30467 the global occupancy grid.

[0178] Reference now Figure 3J, an alternative method 30150 for creating a global occupancy grid can include, but is not limited to, including, if 30151 the device has moved, accessing 30153 occupancy probabilities associated with an old map area (where the device was before it moved) and updating the global occupancy grid over a new map area (where the device is after it moved) with values ​​from the old map area, accessing 30155 drivable surfaces associated with cells of the global occupancy grid in the new map area, and updating cells in the updated global occupancy grid with the drivable surfaces, and continuing at step 30159. If 30151 the device has not moved, and if the global occupancy grid is co-located with the local occupancy grid, the method 30150 can include updating 30159 the possibly updated global occupancy grid with surface confidences associated with the drivable surface from at least one local occupancy grid, updating 30161 the updated global occupancy grid with log odds of occupancy probability values ​​from at least one local occupancy grid using, for example, but not limited to, a Bayesian function, and adjusting 30163 the log odds based on at least a characteristic associated with the location. If 30157 the global occupancy grid is not co-located with the local occupancy grid, the method 30150 can include returning to step 30151. The characteristic can include, but is not limited to, including setting the location of the device to unoccupied.

[0179] Reference now Figure 3K In another configuration, method 30250 for creating a global occupancy grid can include, but is not limited to, including, if 30251 the device has moved, updating 30253 the global occupancy grid with information from a static grid associated with the new location of the device. Method 30250 can include analyzing 30257 the surface at the new location. If 30259 the surface is drivable, method 30250 can include updating 30261 the surface on the global occupancy grid and updating 30263 the global occupancy grid with values ​​from a repository of static values ​​associated with the new location on the map.

[0180] Reference now Figure 3L, updating 30261 the surface can include, but is not limited to, including accessing 30351 a local occupancy grid (LOG) for a particular sensor. If 30353 there are more cells in the local occupancy grid to process, then the method 30261 can include accessing 30355 a surface classification confidence value and a surface classification from the local occupancy grid. If 30357 the surface classification at the cell in the local occupancy grid is the same as the surface classification at the location of the cell in the global occupancy grid, then the method 30261 can include setting 30461 a new global occupancy grid (GOG) surface confidence to the sum of the old global occupancy grid surface confidence and the local occupancy grid surface confidence. If 30357 the surface classification at the cell in the local occupancy grid is different than the surface classification at the location of the cell in the global occupancy grid, then the method 30261 can include setting 30359 a new global occupancy grid surface confidence to the difference between the old global occupancy grid surface confidence and the local occupancy grid surface confidence. If 30463 the new global occupancy grid surface confidence is less than zero, then method 30261 can include setting 30469 the new global occupancy grid surface classification to the value of the local occupancy grid surface classification.

[0181] Reference now Figure 3M , updating 30263 the global occupancy grid with values ​​from a repository of static values ​​can include, but is not limited to, including, if 30361 there are more cells to process in the local occupancy grid, then method 30263 can include accessing 30363 the log odds from the local occupancy grid and updating 30365 the log odds in the global occupancy grid with the value from the local occupancy grid at that location. If 30367 the maximum certainty that the cell is empty is met, and if 30369 the device is traveling within a predetermined lane barrier, and if 30371 the surface is drivable, then method 30263 can include updating 30373 the probability that the cell is occupied and returning to continue processing more cells. If 30367 the maximum certainty that the cell is empty is not met, or if 30369 the device is not traveling in a predetermined lane, or if 30371 the surface is not drivable in the mode in which the device is currently traveling, then method 30263 can include returning to consider more cells without updating the log odds. If the device is in standard mode (i.e., a mode in which the device is capable of navigating relatively uniform surfaces), and the surface classification indicates that the surface is not relatively uniform, method 30263 can adjust the device's path by increasing the probability that the cell is occupied by updating 30373 the log odds. If the device is in standard mode, and the surface classification indicates that the surface is relatively uniform, method 30263 can adjust the device's path by decreasing the probability that the cell is occupied by updating 30373 the log odds. If the device is traveling in 4-wheel mode (i.e., a mode in which the device is capable of navigating non-uniform terrain), adjustments to the probability that the cell is occupied may not be necessary.

[0182] Reference now Figure 4A , the AV is capable of traveling in a particular mode, which can be associated with a device configuration, such as the configuration depicted in device 42114A and the configuration depicted in device 42114B. The system for controlling the configuration of a device in real time based on at least one environmental factor and the condition of the device of the present teachings can include, but is not limited to: a sensor; a mobile device; a chassis operably coupled to the sensor and the mobile device, the mobile device being driven by a motor and a power source; a device processor that receives data from the sensor; and a power base processor that controls the mobile device. In some configurations, the device processor can receive environmental data, determine environmental factors, determine configuration changes based on the environmental factors and the condition of the device, and provide the configuration changes to the power base processor. The power base processor can issue commands to the mobile device to move the device from one place to another, thereby physically reconfiguring the device when required by the road surface type.

[0183] Continue to refer Figure 4A , sensors that collect environmental data can include, but are not limited to, including, for example, cameras, LIDAR, radar, thermometers, pressure sensors, and weather condition sensors, several of which are described herein. Based on this data classification, the device processor can determine environmental factors based on which device configuration changes can be based. In some configurations, environmental factors can include surface factors such as, for example, but not limited to, surface type, surface characteristics, and surface conditions. The device processor can determine in real time how to change the configuration to adapt to traversing the detected surface type based on the environmental factors and the current situation of the device.

[0184] Continue to refer Figure 4A In some configurations, configuration changes of devices 42114A / B / C (collectively, devices 42114) can include, for example, changes in the configuration of the mobile device. Other configuration changes are contemplated, such as user information displays and sensor controls that may depend on the current mode and surface type. In some configurations, as described herein, the mobile device can include at least four drive wheels 442101, two on each side of the chassis 42112, and at least two casters 42103 operably coupled to the chassis 42112. In some configurations, the drive wheels 442101 can be operably coupled in pairs 42105, wherein each pair 42105 can include a first drive wheel 42101A and a second drive wheel 42101B of the four drive wheels 442101, and the pairs 42105 are each located on opposite sides of the chassis 42112. The operable coupling can include a wheel cluster assembly 42110. In some configurations, the power base processor 41016 ( Figure 4B) can control the rotation of cluster assembly 42110. Left and right wheel motors 41017 ( Figure 4B ) can drive the wheels 442101 on either side of the chassis 42112. It can drive the left and right wheel motors 41017 at different rates ( Figure 4B ) to complete the rotation. Cluster motor 41019 ( Figure 4B ) can rotate the wheelbase in the fore-aft direction. For example, when encountering discontinuous surface features, the rotation of the wheelbase can allow the cargo to rotate independently of the drive wheel 442101, if any, while the front drive wheel 442101A becomes higher or lower than the rear drive wheel 442101B. The cluster assembly 42110 can independently operate each pair 42105 of two wheels, thereby providing forward, reverse and rotational movement of the device 42114 on command. The cluster assembly 42110 can provide structural support for each pair 42105. The cluster assembly 42110 can provide mechanical power to rotate the wheel drive assembly together, thereby allowing functions that rely on the rotation of the cluster assembly to be implemented, such as but not limited to discontinuous surface feature climbing, various surface types and uneven terrain. More details on the operation of the cluster wheels can be found in U.S. Patent Application No. 16,035,205 (Agent File No. X80) entitled Mobility Device filed on July 13, 2018, which is incorporated herein by reference in its entirety.

[0185] Continue to refer Figure 4A , the configuration of device 42114 can be combined with the mode 41033 of device 42114 ( Figure 4B ) is associated with, but is not limited to being associated with. Device 42114 is capable of several modes 41033 ( Figure 4B ) in standard mode 10100-1 ( Figure 5E ), the device 42114B is capable of operating on two drive wheels 442101B and two casters 42103. Standard Mode 10100-1 ( Figure 5E ) can provide rotational performance and mobility on relatively solid horizontal surfaces such as, but not limited to, indoor environments, sidewalks, and walkways. In Enhanced Mode 10100-2 ( Figure 5E ) or 4-wheel mode, the device 42114A / C can command the four drive wheels 442101A / B, can actively stabilize through on-board sensors, and can raise and / or redirect the chassis 42112, casters 42103, and cargo. 4-wheel mode 10100-2 ( Figure 5E ) can provide mobility in a variety of environments, allowing the device 42114A / C to travel up steep slopes and over soft uneven terrain. In four-wheel mode 10100-2 ( Figure 5E), all four drive wheels 442101A / B can be deployed and casters 42103 can be retracted. Rotation of cluster 42110 can allow operation on uneven terrain, and drive wheels 442101A / B can be driven up and over discontinuous surface features. This functionality can provide device 42114A / C with mobility in a wide variety of outdoor environments. Device 42114B can operate on firm and stable but wet outdoor surfaces. Frost heave and other natural phenomena can degrade outdoor surfaces, creating cracks and loose material. In 4-wheel mode 10100-2 ( Figure 5E ), devices 42114A / C are capable of operating on these degraded surfaces. Mode 41033 ( Figure 4B ) is described in detail in US Pat. No. 6,571,892 ('892), issued Jun. 3, 2003, entitled Control System and Method, which is incorporated herein by reference in its entirety.

[0186] Reference now Figure 4B , the system 41000 is capable of processing input from the sensor 41031 and generating commands to the wheel motor 41017 to drive the wheel 442101 ( Figure 4A ), and generates a command to the cluster motor 41019 to drive the cluster 42110 ( Figure 4A ) to drive device 42114 ( Figure 4A ). System 41000 can include, but is not limited to, a device processor 41014 and a power base processor 41016. Device processor 41014 can receive and process environmental data 41022 from sensor 41031 and provide configuration information 40125 to power base processor 41016. In some configurations, device processor 41014 can include a sensor processor 41021 that can receive and process environmental data 41022 from sensor 41031. As described herein, sensor 41031 can include, but is not limited to, a camera. Based on this data, information about, for example, what is being used by device 42114 ( Figure 4A ) traversing a driving surface. In some configurations, the driving surface information can be processed in real time. The device processor 41014 can include a configuration processor 41023 that can determine a surface type 40121 based on the environment data 41022, for example, a surface being traversed by the device 42114 ( Figure 4A ) The type of surface to be traversed. Configuration processor 41023 can include, for example, driving surface processor 41029 ( Figure 4C), the driving surface processor 41029 can create, for example, a driving surface classification layer, a driving surface confidence layer, and an occupancy layer based on the environmental data 41022. As described herein, these data can be used by the power base processor 41016 to create movement commands 40127 and motor commands 40128, and can be used by the global occupancy grid processor 41025 to update an occupancy grid that can be used for path planning. Configuration 40125 can be based at least in part on surface type 40121. Surface type 40121 and mode 41033 can be used to at least partially determine occupancy grid information 41022, which can include the probability that cells in the occupancy grid are occupied. The occupancy grid can at least partially enable determination of device 42114 ( Figure 4A ) can take.

[0187] Continue to refer Figure 4B , the power base processor 41016 can receive configuration information 40125 from the device processor 41014 and process the configuration information 40125 and other information, such as path information. The power base processor 41016 can include a control processor 40325, which can create a movement command 40127 based on at least the configuration information 40125 and provide the movement command 40127 to the motor drive processor 40326. The motor drive processor 40326 can generate a motor command 40128 ( Figure 4A ). Specifically, the motor drive processor 40326 can generate a motor command 40128 that can drive the wheel motor 41017, and can generate a motor command 40128 that can drive the cluster motor 41019.

[0188] Reference now Figure 4C , real-time surface detection of the present teachings can include, but is not limited to, including a configuration processor 41023 that can include a driving surface processor 41029. The driving surface processor 41029 can determine that the device 42114 ( Figure 4A ) is navigating on. These characteristics can be used to determine the characteristics of the driving surface on which the device 42114 ( Figure 4A ) future configuration. Driving surface processor 41029 can include, but is not limited to, neural network processor 40207, data transforms 40215, 40219, and 40239, layer processor 40241, and occupancy grid processor 40242. Together, these components can generate a signal capable of guiding device 42114 ( Figure 4A ) configuration changes and can achieve the occupation grid 40244 ( Figure 4C ) of the modification information, which can notify the device 42114 ( Figure 4A )’s driving path planning.

[0189] Reference now Figure 4C and Figure 4D , the neural network processor 40207 is capable of making the environmental data 41022 ( Figure 4B ) is subjected to a trained neural network, which is capable of detecting the Figure 4B For each point of the collected data, indicate the type of surface that point is likely to represent. Environmental Data 41022 ( Figure 4B ) can be received as, but is not limited to, a camera image 40202, where the camera can be associated with camera attributes 40204. The camera image 40202 can include a camera having an X resolution 40205 ( Figure 4D ) and Y resolution 40204 ( Figure 4D ) of points 40201. In some configurations, the camera image 40202 can include an RGB-D image with an X resolution 40205 ( Figure 4D ) can contain 40640 pixels, and the Y resolution is 40204 ( Figure 4D ) can include 40480 pixels. In some configurations, the camera image 40202 can be converted to an image formatted according to the requirements of the selected neural network. In some configurations, the data can be normalized, scaled, and converted from 2D to 1D, which can improve the processing efficiency of the neural network. The neural network can be trained in various ways including, but not limited to, training with RGB-D camera images. In some configurations, in the neural network file 40209 ( Figure 4D ) can be provided to the neural network processor 40207 through a direct connection to a processor executing the trained neural network or, for example, through a communication channel. In some configurations, the neural network processor 40207 can use the trained neural network file 40209 ( Figure 4D ) to identify environmental data 41022 ( Figure 4B ) within the surface type 40121 ( Figure 4C ). In some configurations, surface type 40121 ( Figure 4C ) can include, but are not limited to, non-drivable, hard drivable, soft drivable, and curb. In some configurations, surface type 40121 ( Figure 4C) can include, but are not limited to, non-drivable / base, asphalt, concrete, brick, packed dirt, wood, gravel / small stones, grass, mulch, sand, curb, solid metal, metal grille, tactile paving, snow / ice, and train tracks. The result of the neural network processing can include a surface classification grid 40303 of points 40213 and centers 40211 with an X resolution 40205 and a Y resolution 40203. Each point 40213 in the surface classification grid 40303 can be associated with a surface type 40121 ( Figure 4C ) is associated with the possibility of a specific one of them.

[0190] Continue to refer Figure 4C and 4D , driving surface processor 41029 ( Figure 4C ) can include a 2D to 3D transformation 40215 that can classify a grid 40303 ( Figure 4D ) is back-projected into the 3D image cube 40307 in 3D real-world coordinates as seen by the camera ( Figure 4D ). Back-projection is able to recover the 3D properties of 2D data and can transform a 2D image from an RGB-D camera into a 3D camera frame 40305 ( Figure 4C Cube 40307 ( Figure 4D ) in point 40233 ( Figure 4D ) can each be used with surface type 40121 ( Figure 4C ) is associated with a depth coordinate and an X / Y coordinate. It is possible to associate a specific one of the camera attributes 40204 ( Figure 4C ) to define the point cube 40307 ( Figure 4D ) size. For example, camera attributes 40204 can include a maximum range on which a camera can reliably project. In addition, there may be a device 42114 ( Figure 4A ) may interfere with the features of image 40202. For example, caster 42103 ( Figure 4A ) may interfere with camera 40227 ( Figure 4D ) of the view. These factors may limit the point cube 40307 ( Figure 4D ). In some configurations, camera 40227 ( Figure 4D ) cannot be reliably projected beyond about six meters, which could indicate that camera 40227 ( Figure 4D ) and may limit the point cube 40307 ( Figure 4D ). In some configurations, device 42114 ( Figure 4A ) features can be used as camera 40227 ( Figure 4D ) is the minimum limit of the range. For example, caster 42103 ( Figure 4A ) can suggest that in some configurations a minimum limit of about one meter can be set. In some configurations, point cube 40307 ( Figure 4D ) can be limited to points 40227 ( Figure 4D ) more than one meter away from the camera 40227 ( Figure 4D ) Points below six meters.

[0191] Continue to refer Figure 4C and Figure 4D , driving surface processor 41029 ( Figure 4C ) can include a base chain transform 40219 that can transform a 3D cube of points into a cube that is compatible with device 42114 ( Figure 4A ) associated with the coordinates, namely the base chain frame 40309 ( Figure 4C The base chain transformation 40219 can transform the cube 40307 ( Figure 4D ) in 3D data point 40223 ( Figure 4D ) is transformed into a cube 40308 ( Figure 4D ) in point 40233 ( Figure 4D ), where the Z dimension is set to device 42114 ( Figure 4A ) base. Driving surface processor 41029 ( Figure 4C ) can include OG preparation 40239, which can transform cube 40308 ( Figure 4D ) in point 40233 ( Figure 4D ) is projected onto the occupancy grid 40244 ( Figure 4D ) as cube 40311 ( Figure 4D ) in point 40237 ( Figure 4D ). Depends on point 40237 ( Figure 4D ), the layer processor 40241 is able to convert point 40237 ( Figure 4D ) flattened into various layers 40312 ( Figure 4C ). In some configurations, layer processor 40241 can apply a scalar value to layer 40312 ( Figure 4C ). In some configurations, layer 40312 ( Figure 4C ) can include an occupancy layer 40243, a surface classification layer 40245 as determined by the neural network processor 40207 ( Figure 4D ) and surface type confidence layer 40247 ( Figure 4D). In some configurations, the surface type confidence layer 40247 ( Figure 4D ). In some configurations, one or more layers can be replaced or augmented by a layer that provides a probability of a non-drivable surface.

[0192] Continue to refer Figure 4C and Figure 4D , in some configurations, the probability values ​​in the occupied layer 40243 can be represented as log odds (log odds -> ln(p / (1-p)) values. In some configurations, the probability values ​​in the occupied layer 40243 can be represented as log odds (log odds -> ln(p / (1-p)) values. Figure 4B ) and surface type 40121 ( Figure 4B In some configurations, preselected probability values ​​in the occupation layer 40243 can be selected to cover situations such as, for example, but not limited to, the following: (1) when the surface type 40121 ( Figure 4A ) is hard and drivable, and when device 42114 ( Figure 4A ) is in a set of pre-selected modes 41033 ( Figure 4B ), or (2) when surface type 40121 ( Figure 4A ) is soft and drivable, and when device 42114 ( Figure 4A ) when in a specific pre-selected mode such as, for example, standard mode, or (3) when the surface type 40121 ( Figure 4A ) is soft and drivable, and when device 42114 ( Figure 4A ) when in a specific pre-selected mode such as, for example, 4-wheel mode, or (4) when surface type 40121 ( Figure 4A ) is discontinuous, and when device 42114 ( Figure 4A ) when in a specific preselected mode such as, for example, standard mode, or (5) when surface type 40121 ( Figure 4A ) is discontinuous, and when device 42114 ( Figure 4A ) in a specific pre-selected mode such as 4-wheel mode, or (6) when surface type 40121 ( Figure 4A ) is not drivable, and when device 42114 ( Figure 4A ) is in a set of pre-selected modes 41033 ( Figure 4B ). In some configurations, the probability values ​​can include, but are not limited to, those set forth in Table I. In some configurations, the probabilities predicted by the neural network can be tuned if necessary and can replace the probabilities listed in Table I.

[0193]

[0194] Table I.

[0195] Reference again Figure 4C , the occupancy grid processor 40242 can provide parameters that may affect the probability value of the occupancy grid 40244, such as, for example, but not limited to, the surface type 40121 and the occupancy grid information 41022, to the global occupancy grid processor 41025 in real time. Configuration information 40125 (such as, for example, but not limited to, the mode 41033 and the surface type 40121) can be Figure 4B ) is provided to the power base processor 41016 ( Figure 4B Power base processor 41016 ( Figure 4B ) can determine the motor command 40128 ( Figure 4B ), these motor commands can be based on at least configuration information 40125 ( Figure 4B )Set device 42114 ( Figure 4A ) configuration.

[0196] Reference now Figure 4E and Figure 4F , the device 42100A can be configured to operate in a standard mode according to the present teachings. In the standard mode, the casters 42103 and the second drive wheel 42101B can rest on the ground while the device 42100A navigates its path. The first drive wheel 42101A can be raised a preselected amount 42102 ( Figure 4F ) and is able to leave the driving surface. Device 42100A is able to successfully navigate on relatively firm, level surfaces. When driving in standard mode, occupying grid 40244 ( Figure 4C ) can reflect surface type restrictions (see Table I), thus enabling compatible mode selection, or enabling configuration changes based on surface type and current mode.

[0197] Reference now Figure 4G-4J , the device 42100B / C can be configured to operate in a 4-wheel mode according to the present teachings. In one configuration in the 4-wheel mode, the first drive wheel 42101A and the second drive wheel 42101B can rest on the ground while the device 42100A navigates its path. The casters 42103 can be retracted and can be lifted off the driving surface a preselected amount 42104 ( Figure 4H ). In another configuration in 4-wheel mode, the first drive wheel 42101A and the second drive wheel 42101B can be substantially resting on the ground as the device 42100A navigates its path. The casters 42103 can be retracted and the chassis 42111 can be rotated (thereby moving the casters 42103 away from the ground) to accommodate, for example, discontinuous surfaces. In this configuration, the casters 42103 can be lifted off the driving surface by a preselected amount 42108 ( Figure 1J ). The device 42100A / C is able to successfully navigate a variety of surfaces, including soft surfaces and discontinuous surfaces. In another configuration in 4-wheel mode, the second drive wheel 42101B can rest on the ground while the device 42100A navigates its path, while the first drive wheel 42101A can move along the device 42100C ( Figure 1J ) is raised when navigating its path. Casters 42103 can be retracted and chassis 42111 can be rotated (thereby moving casters 42103 away from the ground) to allow, for example, discontinuous surfaces. When driving in 4-wheel mode, grid 40244 ( Figure 4C ) can reflect the surface type (see Table I), thus enabling compatible mode selection, or enabling configuration changes based on the surface type and current mode.

[0198] Reference now Figure 4K , a method 40150 for real-time control of a device configuration of a device such as an AV traversing a path based on, for example, but not limited to, at least one environmental factor and the device configuration can include, but is not limited to, receiving 40151 sensor data, determining 40153 a surface type based at least on the sensor data, and determining 40155 a current mode based at least on the surface type and the current device configuration. The method 40150 can include determining 40157 a next device configuration based at least on the current mode and the surface type, determining 40159 a movement command based at least on the next device configuration, and changing 40161 the current device configuration to the next device configuration based at least on the movement command.

[0199] Now the main reference Figure 5A , in order to respond to objects appearing in the path of the AV, the annotated point data 10379 ( Figure 5B ). Annotated point data 10379 ( Figure 5B ), which may be capable of being used to indicate AV 10101 ( Figure 1A ) based on route information of the traveled path, which can include but is not limited to navigable edges, such as, for example but not limited to, plotted trajectory 10413 / 10415 ( Figure 5D ) drawing trajectory, and such as, but not limited to, SDSF10377 ( Figure 5C ) of the marked features. Plot trajectory 10413 / 10415 ( Figure 5C ) can include an edge graph of the route space and initial weights assigned to portions of the route space. The edge graph can include properties such as, for example but not limited to, directionality and capacity, and edges can be classified according to these properties. Draw Trajectory 10413 / 10415 ( Figure 5C) can include cost modifiers associated with surfaces of the route space, and driving modes associated with edges. Driving modes can include, but are not limited to, path following and SDSF climbing. Other modes can include operating modes such as, for example, but not limited to, autonomous, mapping, and waiting for intervention. Ultimately, a path can be selected based at least on the lower cost modifier. Figure 5C ) Topology that is relatively far away may have higher cost modifiers and may be less interesting when forming a path. Figure 1A ) is executed while adjusting the initial weights, which may cause the path to be modified. The adjusted weights can be used to adjust the edge / weight graph 10381 ( Figure 5B ), and can be based on at least the current driving mode, the current surface and the edge category.

[0200] Continue to refer Figure 5A , the device controller 10111 can include a feature processor that can perform specific tasks related to incorporating the eccentricity of any feature into the path. In some configurations, the feature processor can include, but is not limited to, an SDSF processor 10118. In some configurations, the device controller 10111 can include, but is not limited to, an SDSF processor 10118, a sensor processor 10703, a mode controller 10122, and a base controller 10114, each described herein. The SDSF processor 10118, the sensor processor 10703, and the mode controller 10122 can provide input to the base controller 10114.

[0201] Continue to refer Figure 5A , the base controller 10114 is capable of determining based at least on input provided by the mode controller 10122, the SDSF processor 10118, and the sensor processor 10703 that the power base 10112 can use to perform a task based at least on the edge / weight graph 10381 ( Figure 5B ) Determine the path on the drive AV 10101 ( Figure 1A ) information. In some configurations, the base controller 10114 can ensure that the AV 10101 ( Figure 1A ) can follow a predetermined path from an origin to a destination and modify the predetermined path based on at least external and / or internal conditions. In some configurations, external conditions can include, but are not limited to, traffic lights, SDSF, and when being driven by AV 10101 ( Figure 1A ) in or near the path of the AV 10101. In some configurations, internal conditions can include, but are not limited to, including conditions that reflect the AV 10101 ( Figure 1A) mode transition in response to external conditions. The device controller 10111 can determine commands to send to the power base 10112 based at least on the external and internal conditions. The commands can include, but are not limited to, speed and direction commands that can direct the AV 10101 ( Figure 1A ) in a commanded direction at a commanded speed. Other commands can include, for example, a command group that implements a feature response such as, for example, SDSF climbing. For example, the base controller 10114 can determine the desired speed between waypoints of the path by conventional methods including, but not limited to, Interior Point Optimizer (IPOPT) Large Scale Nonlinear Optimization (https: / / projects.coin-or.org / Ipopt). The base controller 10114 can determine the desired path based on at least conventional techniques such as, for example, but not limited to, techniques based on the Dykstra algorithm, an A* search algorithm, or a breadth-first search algorithm. ...413 / 10415 ( Figure 5C ) to define an area in which obstacle detection can be performed. The height of the payload carrier, when adjustable, can be adjusted based at least in part on the directional velocity.

[0202] Continue to refer Figure 5A , the base controller 10114 can convert the speed and direction determination into motor commands. For example, when encountering an SDSF such as, for example but not limited to, a curb or a slope, the base controller 10114 can direct the powered base 10112 to raise the payload carrier 10173 ( Figure 1A ), AV 10101 ( Figure 1A ) is aligned with the SDSF at an angle of approximately 90° and reduces the speed to a relatively low level. Figure 1A ) when climbing a substantially discontinuous surface, the base controller 10114 can direct the powered base 10112 to transition to a climbing phase with increased speed because increased torque is required to move the AV 10101 ( Figure 1A ) moves up the slope. When AV 10101 ( Figure 1A ) When encountering a relatively level surface, the base controller 10114 can reduce speed in order to remain atop any flat portion of the SDSF. When in the case of a downhill slope associated with a flat portion, the AV 10101 ( Figure 1A) begins to descend a substantially discontinuous surface, and when both wheels are on a downhill ramp, the base controller 10114 can allow the speed to increase. When a SDSF such as, for example, but not limited to, a ramp is encountered, the ramp can be identified and processed as a structure. For example, the characteristics of the structure can include a preselected size of the ramp. The ramp can include an incline of approximately 30° and can optionally be on both sides of the platform without limitation. The device controller 10111 ( Figure 5A ) is capable of distinguishing between obstacles and slopes by comparing the angle of the sensed feature with the expected slope ramp angle, which can be obtained from the sensor processor 10703 ( Figure 5A ) receiving angle.

[0203] Now the main reference Figure 5B , the SDSF processor 10118 can be used to create a drivable surface for use by the AV 10101 ( Figure 1A ) traverses a navigable edge of a path. Figure 5C ) forms an SDSF buffer 10407 of a preselected size around the Figure 5C ), it is possible to erase navigable edges (see Figure 5D ) to prepare for special processing for a given SDSF traverse. It is possible to draw segments such as segment 10409 ( Figure 5C ) is closed at the previously determined SDSF point 10789 ( Figure 1N ) divide the SDSF buffer 10407 equally between the pairs ( Figure 5C In some configurations, for a closed line segment to be considered a candidate for SDSF traversal, the segment end 10411 ( Figure 5C ) can land on an unobstructed portion of the drivable surface with sufficient space for AV 10101 ( Figure 1A ) along the line segment at the adjacent SDSF point 10789 ( Figure 1N ) and SDSF point 10789 ( Figure 1N ) may be a drivable surface. Segment end 10411 ( Figure 5C ) can be connected to the underlying topology, forming vertices and drivable edges. For example, in Figure 5D The line segments 10461, 10463, 10465 and 10467 satisfying the crossing criteria ( Figure 5C ) is shown as part of the topology. In contrast, at least because segment end 10411 ( Figure 5C ) does not fall on the drivable surface, so line segment 10409 ( Figure 5C ) does not meet the criteria. Overlapping SDSF buffer 10506 ( Figure 5C) can indicate SDSF discontinuity, which may be useful for SDSF in overlapping SDSF buffers 10506 ( Figure 5C ) is not conducive to the SDSF crossing. It can make the SDSF line 10377 ( Figure 5C ) smooth, and able to adjust SDSF point 10789 ( Figure 1N ) so that they are approximately a preselected distance apart, the preselected distance being based at least on AV 10101 ( Figure 1A ) area.

[0204] Continue to refer Figure 5B , the SDSF processor 10118 can transform the annotated point data 10379 into an edge / weight graph 10381, including topological modifications for SDSF traversal. The SDSF processor 10118 can include a seventh processor 10601, an eighth processor 10702, a ninth processor 10603, and a tenth processor 10605. The seventh processor 10601 can transform the coordinates of the points in the annotated point data 10379 to a global coordinate system to achieve compatibility with GPS coordinates, thereby generating a GPS-compatible data set 10602. The seventh processor 10601 can use conventional processes such as, for example, but not limited to, affine matrix transformations and PostGIS transformations to generate a GPS-compatible data set 10602. The World Geodetic System (WGS) can be used as a standard coordinate system because it takes into account the curvature of the earth. The map can be stored in a Universal Transverse Mercator (UTM) coordinate system and can be switched to WGS when it is necessary to find where a particular address is located.

[0205] Now the main reference Figure 5C , the eighth processor 10702 ( Figure 5B ) can smooth the SDSF and determine the boundary of the SDSF 10377, create a buffer zone 10407 around the SDSF boundary, and the farther the surface is from the SDSF boundary, the greater the cost correction value of the surface. The drawing track 10413 / 10415 can be a special case lane with the lowest cost correction value. The lower cost correction value 10406 ​​can be generally located near the SDSF boundary, while the higher cost correction value 10408 can be generally located relatively far from the SDSF boundary. The eighth processor 10702 can send a request to the ninth processor 10603 ( Figure 5B ) provided with a cost of 10704 ( Figure 5B ) point cloud data.

[0206] Continue to main reference Figure 5C , the ninth processor 10603 ( Figure 5B ) can calculate approximately 90° close to 10604 ( Figure 5B ) for AV 10101 ( Figure 1A ) traverses SDSF 10377 that have met criteria for marking them as traversable. The criteria can include SDSF width and SDSF smoothness. Line segments, such as line segment 10409, can be created such that their length indicates AV 10101 ( Figure 1A ) The minimum entry distance that may be required to approach SDSF 10377, and the minimum exit distance that may be required to leave SDSF 10377. Segment endpoints, such as endpoint 10411, can be integrated with the underlying route topology. The criteria used to determine whether an SDSF approach is feasible can eliminate some approach possibilities. SDSF buffers such as SDSF buffer 10407 can be used to calculate valid approaches and route topology edge creation.

[0207] Again the main reference Figure 5B , the tenth processor 10605 can create an edge / weight graph 10381 from the topology, where the edge and weight graph developed can be used to calculate a path through the map. The topology can include cost modifiers and driving modes, and the edges can include directionality and capacity. The weights can be adjusted at runtime based on information from any number of sources. The tenth processor 10605 can provide at least one ordered sequence of points to the base controller 10114, plus a recommended driving mode at a particular point, to enable path generation. Each point in each sequence of points represents a location and a label of a possible path point on a processed drivable surface. In some configurations, the label can indicate that the point represents a portion of a feature that may be encountered along the path, such as, for example but not limited to, an SDSF. In some configurations, features may be further labeled with recommended processing based on the type of feature. For example, in some configurations, if a path point is labeled as an SDSF, the further labeling can include a mode. The mode can be determined by the AV 10101 ( Figure 1A ) is interpreted as targeting AV 10101 ( Figure 1A ) suggested driving instructions, such as, for example, placing the AV10101 ( Figure 1A ) Switch to SDSF climbing mode 100-31 ( Figure 5E ) so that AV 10101 ( Figure 1A ) can cross SDSF10377 ( Figure 5C ).

[0208] Reference now Figure 5E In some configurations, the mode controller 10122 can send a signal to the base controller 10114 ( Figure 5A ) provides an indication that a mode transition is to be performed. The mode controller 10122 is capable of establishing the AV 10101 ( Figure 1A) is traveling in the mode in which the vehicle is traveling. For example, the mode controller 10122 can provide a mode change indication to the base controller 10114 to change between, for example, the path following mode 10100-32 and the SDSF climb mode 10100-31 when an SDSF is identified along the travel path. In some configurations, the annotated point data 10379 ( Figure 5B ) can include mode identifiers at various points along the route, for example, when the mode changes to adapt to the route. For example, if you have annotated point data 10379 ( Figure 5B ) is tagged with SDSF10377 ( Figure 5C ), the device controller 10111 can determine the mode identifier associated with the waypoint and possibly adjust the power base 10112 ( Figure 5A ) instructions. In addition to SDSF climbing mode 10100-31 and path following mode 10100-32, in some configurations, AV 10101 ( Figure 1A ) can also support modes of operation that can include, but are not limited to, the Standard Mode 10100-1 and Enhanced (4-round) Mode 10100-2 described herein. Capable of adjusting the payload carrier 10173 ( Figure 1A ) to provide the necessary clearance over obstacles and along slopes.

[0209] Reference now Fig. 5F, a method 11150 for navigating an AV toward a target point across at least one SDSF can include, but is not limited to, receiving 11151 SDSF information related to the SDSF, the location of the target point, and the location of the AV. The SDSF information can include, but is not limited to, a set of points each classified as an SDSF point, and the associated probability of each point being an SDSF point. The method 11150 can include drawing 11153 a closed polygon containing the location of the AV, the location of the target point, and drawing a path line between the location of the target point and the AV. The closed polygon can include a preselected width. Table I includes possible ranges of preselected variables discussed herein. The method 11150 can include selecting 11155 two SDSF points located within the polygon and drawing 11157 an SDSF line between the two points. In some configurations, the selection of the SDSF points can be random or in any other manner. If 11159 there are less than a first preselected number of points within a first preselected distance of the SDSF line, and if 11161 there have been less than a second preselected number of attempts at picking SDSF points, drawing lines between them, and having less than a first preselected number of points around the SDSF line, then method 11150 can include returning to step 11155. If 11161 there have been a second preselected number of attempts at picking SDSF points, drawing lines between them, and having less than a first preselected number of points around the SDSF line, then method 11150 can include noting 11163 that no SDSF line was detected.

[0210] Now the main reference Figure 5G , if 11159 ( Fig. 5F ) there are a first preselected number of points or more, method 11150 can include fitting 11165 a curve to points that fall within a first preselected distance of the SDSF line. If 11167 the number of points within the first preselected distance of the curve exceeds the number of points within the first preselected distance of the SDSF line, and if 11171 the curve intersects the path line, and if 11173 there are no gaps between points on the curve that exceed a second preselected distance, method 11150 can include identifying 11175 the curve as an SDSF line. If 11167 the number of points within the first preselected distance of the curve does not exceed the number of points within the first preselected distance of the SDSF line, or if 11171 the curve does not intersect the path line, or if 11173 there are gaps between points on the curve that exceed a second preselected distance, and if 11177 the SDSF line does not remain stable, and if 11169 curve fits have not been attempted for more than a second preselected number of attempts, method 11150 can include returning to step 11165. A stable SDSF line is the result of subsequent iterations that produce the same or fewer points.

[0211] Now the main reference Figure 5H, if 11169 ( Figure 5G ) has attempted curve fitting for the second preselected number of attempts, or if 11177 ( Figure 5G ) the SDSF line remains stable or degenerates, the method 11150 can include receiving 11179 occupancy grid information. The occupancy grid can provide the probability of the presence of obstacles at certain points. When the occupancy grid includes data captured and / or calculated over a common geographic area having polygons, the occupancy grid information can augment the SDSF and path information found in the polygons surrounding the AV path and SDSF. The method 11150 can include selecting 11181 points and their associated probabilities from the common geographic area. If 11183 the probability of the presence of an obstacle at the selected point is higher than a preselected percentage, and if 11185 the obstacle is located between the AV and the target point, and if 11186 the obstacle is less than a third preselected distance from the SDSF line between the SDSF line and the target point, the method 11150 can include projecting 11187 the obstacle onto the SDSF line. If 11183 there is less than or equal to a preselected percentage probability that the location includes an obstacle, or if 11185 the obstacle is not located between the AV and the target point, or if 11186 the obstacle is located at a distance equal to or greater than a third preselected distance from the SDSF line between the SDSF and the target point, and if 11189 there are more obstacles to process, then method 11150 can include resuming processing at step 11179.

[0212] Now the main reference Fig.5I , if 11189 ( Figure 5H ) there are no more obstacles to deal with, method 11150 can include connecting 11191 the projections and finding the endpoints of the connected projections along the SDSF line. Method 11150 can include marking 11193 the portion of the SDSF line between the projection endpoints as non-traversable. Method 11150 can include marking 11195 the portion of the SDSF line outside the non-traversable portion as traversable. Method 11150 can include turning 11197 the AV to within a fifth preselected amount perpendicular to the traversable portion of the SDSF line. If 11199 the heading error relative to the line perpendicular to the traversable portion of the SDSF line is greater than the first preselected amount, method 11150 can include slowing 11251 the AV by a ninth preselected amount. Method 11150 can include driving 11253 the AV forward toward the SDSF line, slowing by a second preselected amount per meter of distance between the AV and the traversable SDSF line. If 11255 the AV is less than a fourth preselected distance from the traversable SDSF line, and if 11257 the heading error relative to a line perpendicular to the SDSF line is greater than or equal to a third preselected amount, method 11150 can include slowing 11252 the AV by a ninth preselected amount.

[0213] Now the main reference Figure 5J , if 11257 ( Fig.5I ) the heading error relative to a line perpendicular to the SDSF line is less than a third preselected amount, the method 11150 can include ignoring 11260 the updated SDSF information and driving the AV at a preselected speed. If 11259 the elevation of the front of the AV relative to the rear of the AV is between a sixth preselected amount and a fifth preselected amount, the method 11150 can include driving 11261 the AV forward and increasing the speed of the AV to an eighth preselected amount per degree of elevation. If 11263 the front-to-rear elevation of the AV is less than the sixth preselected amount, the method 11150 can include driving 11265 the AV forward at a seventh preselected speed. If 11267 the rear of the AV is more than a fifth preselected distance from the SDSF line, the method 11150 can include noting 11269 that the AV has completed crossing the SDSF. If 11267 the rear of the AV is less than or equal to the fifth preselected distance from the SDSF line, the method 11150 can include returning to step 11260.

[0214] Reference now Figure 5K , a system 51100 for navigating an AV toward a target point across at least one SDSF can include, but is not limited to, a path line processor 11103, an SDSF detector 11109, and an SDSF controller 11127. The system 51100 can be operably coupled to a surface processor 11601 that can process sensor information that can include, for example, but is not limited to, the AV 10101 ( Figure 5L ) of the surrounding environment. Surface processor 11601 can provide real-time surface feature updates, including indications of SDSF. In some configurations, the camera can provide RGB-D data whose points can be classified according to surface type. In some configurations, system 51100 can process points that have been classified as SDSF and their associated probabilities. System 51100 can be operably coupled to system controller 11602, which can manage AV10101 ( Figure 5L ) aspects of the operation of the AV 10101. The system controller 11602 can maintain an occupancy grid 11138 that can include information about the AV 10101 ( Figure 5L ) nearby navigable areas. The occupancy grid 11138 can include the probability of the presence of obstacles. This information can be used in conjunction with the SDSF information to determine the SDSF 10377 ( Figure 5N ) can be AV 10101 ( Figure 5L ) traverse without encountering obstacles 11681 ( Figure 5M ). The system controller 11602 can determine the AV 10101 based on the environment and other information ( Figure 5N ) should not exceed a speed limit 11148. The speed limit 11148 can be used as a guide, or can override the speed set by the system 51100. The system 51100 can be operably coupled to a base controller 10114 that can send drive commands 11144 generated by the SDSF controller 11127 to the AV 10101 ( Figure 5L ) driving components. The base controller 10114 can provide the SDSF controller 11127 with information about the AV 10101 ( Figure 5L ) Orientation information during SDSF traversal.

[0215] Continue to refer Figure 5K , the path line processor 11103 can continuously receive surface classification points 10789 in real time, which surface classification points 10789 can include but are not limited to points classified as SDSF. The path line processor 11103 can receive the location of the target point 11139, as well as the position of the target point 11139 as determined by, for example but not limited to, the AV 10101 ( Figure 5L ) center 11202 ( Figure 5L ) indicated by the AV position 11141. The system 51100 can include a polygon processor 11105 that draws a polygon 11147 that includes the AV position 11141, the position of the target point 11139, and the path 11214 between the target point 11139 and the AV position 11141. The polygon 11147 can include a preselected width. In some configurations, the preselected width can include approximately AV 10101 ( Figure 5L ) width. It is possible to identify SDSF points 10789 that fall within polygon 11147.

[0216] Continue to refer Figure 5K, the SDSF detector 11109 can receive the surface classification points 10789, the paths 11214, the polygons 11147, and the target points 11139, and can determine the most appropriate SDSF line 10377 based on the criteria set forth herein available within the incoming data. The SDSF detector 11109 can include, but is not limited to, including a point processor 11111 and an SDSF line processor 11113. The point processor 11111 can include selecting two SDSF points 10789 that are within the polygon 11147, and drawing the SDSF 10377 line between the two points. If there are less than a first preselected number of points within a first preselected distance of the SDSF line 10377, and if there have been less than a second preselected number of attempts in selecting the SDSF point 10789, drawing the line between the two points, and having less than a first preselected number of points around the SDSF line, the point processor 11111 can include looping through the select-draw-test loop as set forth herein again. If there have been a second preselected number of attempts at picking SDSF points, drawing lines between them, and having fewer than a first preselected number of points around the SDSF line, the point processor 11111 can include noting that no SDSF line was detected.

[0217] Continue to refer Figure 5K , the SDSF line processor 11113 can include, if there are a first preselected number or more of points 10789, dividing the curves 11609-11611 ( Figure 5L ) is fitted to point 10789 that falls within a first preselected distance of SDSF line 10377. If on curve 11609-11611 ( Figure 5L ) exceeds the number of points 10789 within the first preselected distance of SDSF line 10377, and if curve 11609-11611 ( Figure 5L ) intersects with path line 11214, and if on curve 11609-11611 ( Figure 5L ) does not have a gap between points 10789 on the curve 11609-11611 that exceeds a second preselected distance, the SDSF line processor 11113 can include, for example, converting the curve 11609-11611 ( Figure 5L ) is identified as SDSF line 10377. If the curve 11609-11611 ( Figure 5L ) does not exceed the number of points 10789 within the first preselected distance of SDSF line 10377, or if curves 11609-11611 ( Figure 5L ) does not intersect path line 11214, or if on curves 11609-11611 ( Figure 5L) there is a gap between points 10789 on the SDSF line that exceeds a second preselected distance, and if the SDSF line 10377 does not remain stable, and if the curve fit has not been attempted for more than a second preselected number of attempts, the SDSF line processor 11113 is able to perform a curve fitting cycle again.

[0218] Continue to refer Figure 5K , the SDSF controller 11127 can receive the SDSF line 10377, the occupancy grid 11138, the AV orientation change 11142, and the speed limit 11148, and can generate SDSF commands 11144 to drive the AV 10101 ( Figure 5L ) to correctly cross SDSF 10377 ( Figure 5N ). The SDSF controller 11127 can include, but is not limited to, an obstacle processor 11115, an SDSF approach 11131, and an SDSF traverse 11133. The obstacle processor 11115 can receive the SDSF line 10377, the target point 11139, and the occupancy grid 11138, and can determine whether any of the obstacles identified in the occupancy grid 11138 are likely to be within the AV 10101 ( Figure 5N ) Crossing SDSF 10377 ( Figure 5N ) when it is blocked. Obstacle processor 11115 can include, but is not limited to, including an obstacle selector 11117, an obstacle tester 11119, and a traversal locator 11121. Obstacle selector 11117 can include, but is not limited to, including receiving occupancy grid 11138 as described herein. Obstacle selector 11117 can include selecting occupancy grid points and their associated probabilities from a geographic area common to both occupancy grid 11138 and polygon 11147. Obstacle tester 11119 can include, if the probability of an obstacle being present at the selected grid point is above a preselected percentage, and if the obstacle is located within AV 10101 ( Figure 5L ) and the target point 11139, and if the obstacle is less than a third preselected distance from the SDSF line 10377 between the SDSF line 10377 and the target point 11139, the obstacle tester 11119 can include projecting the obstacle onto the SDSF line 10377, thereby forming a projection 11621 that intersects the SDSF line 10377. If the probability that the location includes an obstacle is less than or equal to a preselected percentage, or if the obstacle is not located at the AV 10101 ( Figure 5L ) and the target point 11139, or if the obstacle is located at a distance less than or equal to a third preselected distance from the SDSF line 10377 between the SDSF line 10377 and the target point 11139, the obstacle tester 11119 can include resuming execution upon receiving the occupancy grid 11138 if there are more obstacles to process.

[0219] Continue to refer Figure 5K , the traverse locator 11121 can include connecting projection points and locating the connected projection 11621 along the SDSF line 10377 ( Figure 5M ) endpoints 11622 / 11623 ( Figure 5M ). The traversal locator 11121 can include projecting endpoints 11622 / 11623 ( Figure 5M ) between the portion of SDSF line 10377 to 11624 ( Figure 5M ) as uncrossable. The crossing locator 11121 can include marking the SDSF line 10377 at the uncrossable portion 11624 ( Figure 5M ) other than 11626 ( Figure 5M ) is marked as traversable.

[0220] Continue to refer Figure 5K , SDSF approach 11131 can include sending SDSF command 11144 to cause AV 10101 ( Figure 5N ) turns to the traversable portion 11626 ( Figure 5N ) is within a fifth preselected amount perpendicular to SDSF line 10377. If relative to traversable portion 11626 ( Figure 5N ) vertical vertical line 11627 ( Figure 5N ) is greater than a first preselected amount, the SDSF approach 11131 can include sending a SDSF command 11144 to move the AV 10101 ( Figure 5N ) to slow down the AV 10101 by a ninth preselected amount. In some configurations, the ninth preselected amount can vary from very slow to completely stopped. The SDSF approach 11131 can include sending an SDSF command 11144 to move the AV 10101 ( Figure 5N ) drives forward toward SDSF line 10377, sending SDSF command 11144 to move AV 10101 ( Figure 5N ) slows down by a second preselected amount per meter travelled. If AV 10101 ( Figure 5N ) and can cross SDSF line 11626 ( Figure 5N ) is less than a fourth preselected distance, and if the heading error relative to a line perpendicular to the SDSF line 10377 is greater than or equal to a third preselected amount, the SDSF approach 11131 can include sending an SDSF command 11144 to move the AV 10101 ( Figure 5N ) slows down the ninth preselection amount.

[0221] Continue to refer Figure 5KIf the heading error is less than a third preselected amount relative to a line perpendicular to the SDSF line 10377, then the SDSF traverse 11133 can include ignoring the updated SDSF information and sending an SDSF command 11144 to drive the AV 10101 at a preselected rate ( Figure 5N ). If AV orientation change 11142 indicates AV 10101 ( Figure 5N ) front edge 11701 ( Figure 5N ) relative to AV 10101 ( Figure 5N )'s trailing edge 11703 ( Figure 5N ) is between the sixth preselected amount and the fifth preselected amount, the SDSF traverse 11133 can include sending an SDSF command 11144 to move the AV 10101 ( Figure 5N ) drives forward and sends SDSF command 11144 to move AV 10101 ( Figure 5N ) to a preselected rate. If AV orientation change 11142 indicates that AV 10101 ( Figure 5N ) front edge 11701 ( Figure 5N ) to the trailing edge 11703 ( Figure 5N ) is raised less than a sixth preselected amount, the SDSF traverse 11133 can include sending an SDSF command 11144 to move the AV 10101 ( Figure 5N ) drives forward at the seventh preselected speed. If AV position 11141 indicates trailing edge 11703 ( Figure 5N ) exceeds a fifth preselected distance from SDSF line 10377, then SDSF crossing 11133 can include noting AV 10101 ( Figure 5N ) has completed the traverse of SDSF 10377. If AV position 11141 indicates trailing edge 11703 ( Figure 5N ) is less than or equal to a fifth preselected distance from the SDSF line 10377, the SDSF traversal 11133 can include again executing a loop starting from ignoring the updated SDSF information.

[0222] Some exemplary ranges of preselected values ​​described herein can include, but are not limited to, those listed in Table II.

[0223]

[0224] Table II.

[0225] Reference now Fig.5O, to support real-time data collection, in some configurations, the system of the present teachings can generate locations in three-dimensional space of various surface types upon receiving, such as, for example, but not limited to, RGD-D camera image data. The system can rotate the images 12155 and transform them from a camera coordinate system 12157 to a UTM coordinate system 12159. The system can generate a polygon file from the transformed image, and the polygon file can represent a three-dimensional location associated with a surface type 12161. A method 12150 for locating a feature 12151 from a camera image 12155 received by an AV 10101 (the AV 10101 having a pose 12163) can include, but is not limited to, including receiving, by the AV 10101, a camera image 12155. Each camera image 12155 can include an image timestamp 12171, and each image 12155 can include image color pixels 12167 and image depth pixels 12169. The method 12150 can include receiving a pose 12163 of the AV 10101, the pose 12163 having a pose timestamp 12171, and determining a selected image 12173 by identifying, from the camera images 12155, an image having an image timestamp 12165 that is closest to the pose timestamp 12171. The method 12150 can include separating image color pixels 12167 from image depth pixels 12169 in the selected image 12173, and determining an image surface classification 12161 for the selected image 12173 by providing the image color pixels 12167 to a first machine learning model 12177 and providing the image depth pixels 12169 to a second machine learning model 12179. The method 12150 can include determining perimeter points 12181 of a feature in the camera image 12173, wherein the feature can include feature pixels 12151 within the perimeter, each feature pixel 12151 having the same surface classification 12161, and each perimeter point 12181 having a coordinate set 12157. The method 12150 can include converting each coordinate set 12157 to UTM coordinates 12159.

[0226] The configuration of this teaching is directed to a computer system for implementing the methods discussed in the description herein, and to a computer-readable medium containing a program for implementing these methods. Raw data and results can be stored for future retrieval and processing, printed, displayed, transferred to another computer and / or transferred elsewhere. The communication link can be wired or wireless, for example, using a cellular communication system, a military communication system, and a satellite communication system. The various parts of the system can operate on a computer with a variable number of CPUs. Other alternative computer platforms can be used.

[0227] The present configuration is also directed to software for implementing the methods discussed herein, and computer-readable media storing software for implementing these methods. The various modules described herein can be implemented on the same CPU, or can be implemented on different computers. According to regulations, the present configuration has been described in a language more or less specific to structural and method features. However, it should be understood that the present configuration is not limited to the specific features shown and described, because the means disclosed herein include preferred forms that make the present configuration effective.

[0228] The method can be implemented in whole or in part electronically. Signals representing actions taken by the elements of the system and other disclosed configurations can be propagated through at least one real-time communication network. Control and data information can be executed electronically and stored on at least one computer-readable medium. The system can be implemented to be executed on at least one computer node in at least one live communication network. Common forms of at least one computer-readable medium can include, for example, but not limited to, floppy disks, flexible disks, hard disks, tapes or any other magnetic media, compact disk read-only memory or any other optical media, punched cards, paper tapes or any other physical media with hole patterns, random access memory, programmable read-only memory and erasable programmable read-only memory (EPROM), flash EPROM, or any other memory chip or box, or any other medium from which a computer can read. In addition, at least one computer-readable medium can contain graphics in any form subject to appropriate licensing when necessary, including but not limited to Graphics Interchange Format (GIF), Joint Photographic Experts Group (JPEG), Portable Network Graphics (PNG), Scalable Vector Graphics (SVG) and Tagged Image File Format (TIFF).

[0229] Although the present teachings have been described above in specific configurations, it should be understood that they are not limited to these disclosed configurations. Many modifications and other configurations will occur to those skilled in the art, and these modifications and other configurations are intended to be covered by both the present disclosure and the appended claims. As understood by those skilled in the art relying on the disclosure in this specification and the accompanying drawings, the scope of the present teachings is intended to be determined by proper interpretation and construction of the appended claims and their legal equivalents.

[0230] The rights claimed are as follows.

Claims

1. A method for managing a global occupancy grid of an autonomous device, the global occupancy grid comprising global occupancy grid cells, the global occupancy grid cells being associated with occupancy probabilities, the method comprising: receiving sensor data from a sensor associated with the autonomous device; creating a local occupancy grid based at least on the sensor data, the local occupancy grid having local occupancy grid cells; If the autonomous device has moved from a first area to a second area, then accessing historical data associated with the second region; creating a static grid based at least on the historical data; moving the global occupancy grid so that the autonomous device remains in a central position of the global occupancy grid; updating a shifted global occupancy grid based on the static grid; marking at least one of the global occupancy grid cells as unoccupied if at least one of the global occupancy grid cells coincides with the location of the autonomous device; For each of the locally occupied grid cells, Calculating the position of the local occupied grid cell on the global occupied grid; accessing a first occupancy probability from the global occupancy grid cell at the location; accessing a second occupancy probability from the local occupancy grid cell at the location; and A new occupancy probability at the location on the global occupancy grid is calculated based at least on the first occupancy probability and the second occupancy probability.

2. The method according to claim 1, further comprising: A range check is performed on the new occupancy probability.

3. The method according to claim 2, wherein: The range check includes: If the new occupancy probability is less than 0, setting the new occupancy probability to 0; and If the new occupancy probability>1, the new occupancy probability is set to 1.

4. The method according to claim 1, further comprising: The global occupancy grid cell is set to the new occupancy probability.

5. The method according to claim 2, further comprising: The global occupancy grid cell is set to the new range-checked occupancy probability.

6. The method according to claim 1, wherein: The historical data includes surface data.

7. The method according to claim 1, wherein: The historical data includes discontinuous data.

8. A method for creating and managing an occupancy grid, the method comprising: Creating nodes from the local occupancy grid transforms sensor measurements into a reference frame associated with the device; Create a timestamped measurement occupancy grid; publishing the timestamped measurement result occupancy grid as a local occupancy grid; Create multiple local occupancy grids; creating a static occupancy grid based on surface features in a repository, the surface features being associated with locations of the devices; moving a global occupancy grid associated with the position of the device to maintain the device and the local occupancy grid approximately centered relative to the global occupancy grid; adding information from the static occupancy grid to the global occupancy grid; marking an area in the global occupancy grid currently occupied by the device as unoccupied; For each of at least one cell in each local occupancy grid, determining a location of the at least one cell in the global occupancy grid; accessing a first value at the location; determining a second value at the location based on a relationship between the first value and a cell value at the at least one cell in the local occupancy grid; comparing the second value against a preselected probability range; as well as If the probability value is within the preselected probability range, the global occupancy grid is set with the new value.

9. The method according to claim 8, further comprising: The global occupancy grid is published.

10. The method according to claim 8, wherein: The surface characteristics include surface type and surface discontinuities.

11. The method according to claim 8, wherein: The relationship comprises a summation.

12. A system for creating and managing an occupancy grid, comprising: a plurality of local grid creation nodes, the plurality of local grid creation nodes creating at least one local occupancy grid, the at least one local occupancy grid being associated with a location of the device, the at least one local occupancy grid comprising at least one cell; A global occupancy grid manager, the global occupancy grid manager accessing the at least one local occupancy grid, the global occupancy grid manager: creating a static occupancy grid based on surface characteristics in a repository, the surface characteristics being associated with the location of the device, moving a global occupancy grid associated with the position of the device to maintain the device and at least one of the local occupancy grids approximately centered relative to the global occupancy grid; adding information from the static occupancy grid to at least one global occupancy grid; marking an area in the global occupancy grid currently occupied by the device as unoccupied; For each of the at least one cell in each local occupancy grid, determining a location of the at least one cell in the global occupancy grid; accessing a first value at the location; determining a second value at the location based on a relationship between the first value and a cell value at the at least one cell in the local occupancy grid; comparing the second value against a preselected probability range; as well as If the probability value is within the preselected probability range, the global occupancy grid is set with the new value.

13. A method for updating a global occupancy grid, the method comprising: if the autonomous device has moved to a new location, updating the global occupancy grid with information from a static grid associated with the new location; analyzing the surface at the new position; If the surface is drivable, updating the surface and updating the global occupancy grid with the updated surface; as well as The global occupancy grid is updated with a value from a repository of static values, the static value associated with the new position.

14. The method according to claim 13, wherein: Updating the surface comprises: accessing a local occupancy grid associated with the new position; For each cell in the local occupancy grid, Access local occupancy grid surface classification confidence values ​​and local occupancy grid surface classifications; If the local occupancy grid surface classification is the same as the global surface classification in the global occupancy grid in the cell, adding the global surface classification confidence value in the global occupancy grid to the local occupancy grid surface classification confidence value to form a sum, and updating the global occupancy grid at the cell with the sum; If the local occupancy grid surface classification is not the same as the global surface classification in the global occupancy grid in the cell, subtracting the local occupancy grid surface classification confidence value from the global surface classification confidence value in the global occupancy grid to form a difference, and updating the global occupancy grid with the difference; If the difference is less than zero, the global occupancy grid is updated using the local occupancy grid surface classification.

15. The method according to claim 13, wherein: Updating the global occupancy grid with the values ​​from the repository of static values ​​comprises: For each cell in the local occupancy grid, a local occupancy grid probability, a log-odds value, of accessing the cell from the local occupancy grid as an occupied value; updating the log-probability value in the global occupancy grid with the local occupancy grid log-probability value at the cell; If a preselected certainty that the cell is not occupied is met, and if the autonomous device is traveling within a lane barrier, and if a local occupancy grid surface classification indicates a drivable surface, reducing the log probability that the cell is occupied in the local occupancy grid; If the autonomous device expects to encounter a relatively uniform surface, and if the local occupancy grid surface classification indicates a relatively non-uniform surface, increasing the log odds in the local occupancy grid; and If the autonomous device expects to encounter a relatively uniform surface, and if the local occupancy grid surface classification indicates a relatively uniform surface, then reducing the log odds in the local occupancy grid.

16. A method for real-time control of configuration of an apparatus comprising a chassis, at least four wheels, a first side of the chassis operably coupled to at least one of the at least four wheels, and an opposing second side of the chassis operably coupled to at least one of the at least four wheels, the method comprising: Receive environmental data; determining a surface type based at least on the environmental data; determining a mode based at least on the surface type and the first configuration; determining a second configuration based at least on the mode and the surface type; determining a movement command based at least on the second configuration; as well as The configuration of the device is controlled by using the movement command to change the device from the first configuration to the second configuration.

17. The method according to claim 16, wherein: The environmental data includes RGB-D image data.

18. The method according to claim 16, further comprising: populating an occupancy grid based on at least the surface type and the pattern; as well as The movement command is determined based at least on the occupancy grid.

19. The method according to claim 18, wherein: The occupancy grid includes information based at least on data from at least one image sensor.

20. The method according to claim 16, wherein: The environmental data includes the topology of the road surface.

21. The method according to claim 16, wherein: The configuration includes two pairs of clusters of the at least four wheels, a first pair of the two pairs being located on the first side, a second pair of the two pairs being located on the second side, the first pair including a first front wheel and a first rear wheel, and the second pair including a second front wheel and a second rear wheel.

22. The method according to claim 21, wherein: The controlling of the configuration includes: Powering the first pair and the second pair is coordinated based at least on the environmental data.

23. The method according to claim 21, wherein: The controlling of the configuration includes: Transitioning from driving the at least four wheels and a pair of retractable casters to driving two wheels, the pair of casters being operably connected to the chassis, wherein the first pair of the cluster and the second pair of the cluster are rotated to raise the first front wheel and the second front wheel, the device being resting on the first rear wheel, the second rear wheel and the pair of casters.

24. The method according to claim 21, wherein: The controlling of the configuration includes: A pair of clusters operably coupled with first two powered wheels on the first side and second two powered wheels on the second side are rotated based at least on the environmental data.

25. The method of claim 16, wherein: The apparatus also includes a cargo container mounted on the chassis, the chassis controlling the height of the cargo container.

26. The method according to claim 25, wherein: The height of the cargo container is based at least on the environmental data.

27. A system for real-time control of configuration of an apparatus comprising a chassis, at least four wheels, a first side of the chassis, and an opposing second side of the chassis, the system comprising: a device processor that receives real-time environmental data about the device, the device processor determines a surface type based at least on the environmental data, the device processor determines a mode based at least on the surface type and a first configuration, and the device processor determines a second configuration based at least on the mode and the surface type; as well as A power base processor determines a movement command based at least on the second configuration, the power base processor controlling the configuration of the device by using the movement command to change the device from the first configuration to the second configuration.

28. The system of claim 27, wherein: The environmental data includes RGB-D image data.

29. The system of claim 27, wherein: The device processor includes populating an occupancy grid based at least on the surface type and the mode.

30. The system of claim 29, wherein: The power base processor includes determining the movement command based at least on the occupancy grid.

31. The system of claim 29, wherein: The occupancy grid includes information based at least on data from at least one image sensor.

32. The system of claim 27, wherein: The environmental data includes the topology of the road surface.

33. The system of claim 27, wherein: The configuration includes two pairs of clusters of the at least four wheels, a first pair of the two pairs being positioned on the first side, a second pair of the two pairs being positioned on the second side, the first pair having a first front wheel and a first rear wheel, and the second pair having a second front wheel and a second rear wheel.

34. The system of claim 33, wherein: The controlling of the configuration includes: Powering the first pair and the second pair is coordinated based at least on the environmental data.

35. The system of claim 33, wherein: The controlling of the configuration includes: Transitioning from driving the at least four wheels and a pair of retractable casters to driving two wheels, the pair of casters being operably connected to the chassis, wherein the first pair of the cluster and the second pair of the cluster are rotated to raise the first front wheel and the second front wheel, the device being resting on the first rear wheel, the second rear wheel and the pair of casters.

36. A method for maintaining a global occupancy grid, the method comprising: Locate the first location of the autonomous device; When the autonomous device moves to a second location, the second location is associated with the global occupancy grid and the local occupancy grid, updating the global occupancy grid with at least one occupancy probability value associated with the first location; updating the global occupancy grid with at least one drivable surface associated with the local occupancy grid; updating the global occupancy grid with a surface confidence associated with the at least one drivable surface; updating the global occupancy grid with the log-odds of the at least one occupancy probability value using a first Bayesian function; and adjusting the log odds based on at least a characteristic associated with the second position; as well as while the autonomous device remains in the first position and the global occupancy grid and the local occupancy grid are co-located, updating the global occupancy grid with the at least one drivable surface associated with the local occupancy grid; updating the global occupancy grid with the surface confidence associated with the at least one drivable surface; updating the global occupancy grid with the log-odds of the at least one occupancy probability value using a second Bayesian function; and The log odds are adjusted based on at least a characteristic associated with the second position.

37. A method for real-time control of configuration of an apparatus comprising a chassis, at least four wheels, a first side of the chassis operably coupled to at least one of the at least four wheels, and an opposing second side of the chassis operably coupled to at least one of the at least four wheels, the method comprising: creating a map based at least on the prior surface features and the occupancy grid, the map being created in non-real time, the map comprising at least one location, the at least one location being associated with at least one surface feature, the at least one surface feature being associated with at least one surface classification and at least one mode; determining current surface characteristics as the device travels; updating the occupancy grid in real time using the current surface features; A path that the device can travel to traverse the at least one surface feature is determined from the occupancy grid and the map.

38. The method of claim 37, wherein: Creating the map includes: accessing point cloud data representing the surface; filtering the point cloud data; forming the filtered point cloud data into a processable portion; merging the processable portions into at least one concave polygon; Positioning and marking the at least one SDSF in the at least one concave polygon, the positioning and marking forming marked point cloud data; creating a graphics polygon based at least on the at least one concave polygon; and A path is chosen from a start point to an end point based at least on the graphics polygon, and the AV traverses the at least one SDSF along the path.

39. The method of claim 38, wherein: Filtering the point cloud data includes: Conditionally removing points representing transient objects and points representing outliers from the point cloud data; and Replace the removed points with the preselected height.

40. The method of claim 38, wherein: The formation processing part includes: Segmenting the point cloud data into the processable portions; and Points of preselected heights are removed from the processable portion.

41. The method of claim 38, wherein: The processable parts are merged to include: reducing the size of the processable portion by analyzing outliers, voxels, and normals; growing a region from the processable portion that is reduced in size; determining an initial drivable surface from the growth region; segmenting and meshing the initial drivable surface; Positioning polygons within the segmented and meshed initial drivable surface; and At least one drivable surface is provided based at least on the polygon.

42. The method according to claim 41, wherein: The locating and marking the at least one SDSF comprises: classifying the point cloud data of the initial drivable surface according to an SDSF filter, the SDSF filter comprising at least three classes of points; and At least one SDSF point is located based at least on whether the points of the at least three categories in combination satisfy at least one first pre-selected criterion.

43. The method of claim 42, further comprising: At least one SDSF trajectory is created based at least on whether a plurality of the at least one SDSF points in combination satisfy at least one second preselected criterion.

44. The method of claim 43, wherein: Said creation of the graphics polygon further comprises: creating at least one polygon from the at least one drivable surface, the at least one polygon including an exterior edge; smoothing the outer edge; forming a driving margin based on the smooth outer edge; adding the at least one SDSF track to the at least one drivable surface; and Inner edges are removed from the at least one drivable surface based on at least one third preselected criterion.

45. The method of claim 44, wherein: The smoothing of the outer edge comprises: The outer edges are trimmed outwardly to form an outward edge.

46. ​​The method of claim 44, wherein: The driving margin forming a smooth outer edge includes: The outward edges are trimmed inwards.

47. An autonomous delivery vehicle comprising: a powered base, the powered base comprising at least two powered rear wheels, a caster front wheel, and an energy accumulator, the powered base being configured to move at a commanded rate; a cargo platform mechanically attached to the powered base; as well as a short-range camera assembly mounted to the cargo platform, the short-range camera assembly detecting at least one characteristic of a drivable surface, the short-range camera assembly comprising: camera; First Light; and The first liquid-cooled heat sink, Wherein, the first liquid-cooled heat sink cools the first lamp and the camera.

48. The autonomous delivery vehicle of claim 47, wherein: The short-range camera assembly also includes a thermoelectric cooler between the camera and the liquid-cooled heat sink.

49. The autonomous delivery vehicle of claim 47, wherein: The first light and the camera are recessed in a cover having an opening that deflects illumination from the first light away from the camera.

50. The autonomous delivery vehicle of claim 47, wherein: The lamp is tilted downward at least 15° and recessed at least 4 mm in the cover to minimize illumination that would distract pedestrians.

51. The autonomous delivery vehicle of claim 47, wherein: The camera has a field of view, and The first light includes two LEDs with lenses to produce two beams of light that spread to illuminate the field of view of the camera.

52. The autonomous delivery vehicle of claim 51, wherein: The lamps are tilted approximately 50° apart and the lens produces a 60° beam.

53. The autonomous delivery vehicle of claim 47, wherein: The short-range camera assembly includes an ultrasonic sensor mounted above the camera.

54. The autonomous delivery vehicle of claim 47, wherein: The short-range camera assembly is mounted in a central location on the front face of the cargo platform.

55. The autonomous delivery vehicle of claim 47, further comprising at least one corner camera assembly mounted on at least one corner of a front face of the cargo platform, the at least one corner camera assembly comprising: Ultrasonic Sensors Corner Camera; Second light; as well as A second liquid-cooled heat sink, wherein the second liquid-cooled heat sink cools the second light and the corner camera.

Images