Systems and methods for real-time control of autonomous devices

Abstract

To provide a system and a method for real time control of an autonomous device.SOLUTION: An autonomous vehicle has sensors advantageously varied in capabilities, advantageously positioned, and advantageously impervious to environmental conditions. A system is executed on the autonomous vehicle and can receive a map including, for example, substantially discontinuous surface features along with data from the sensors, create an occupancy grid based upon the map and the data, and change the configuration of the autonomous vehicle based upon the type of surface on which the autonomous vehicle navigates. The device can safely navigate surfaces and surface features, including traversing discontinuous surfaces and other obstacles.SELECTED DRAWING: None

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This utility patent application is a continuation-in-part of U.S. patent application Ser. No. 16 / 800,497 (Attorney Docket No. AA164), filed February 25, 2020, and entitled "System and Method for Surface Feature Detection and Traversal," which is incorporated herein by reference in its entirety. This patent application claims the benefit of U.S. Provisional Patent Application No. 62 / 872,396 (Attorney Docket No. AA028), filed July 10, 2019, entitled "Apparatus for Long and Short Range Sensors on an Autonomous Delivery Device," U.S. Provisional Patent Application No. 62 / 990,485 (Attorney Docket No. AA037), filed March 17, 2020, entitled "System and Method for Managing an Occupancy Grid," and U.S. Provisional Patent Application No. 62 / 872,320 (Attorney Docket No. Z96), filed July 10, 2019, entitled "System and Method for Real-Time Control of the Configuration of an Autonomous Device."

[0002] This application is incorporated herein by reference in its entirety. This application is related to U.S. patent application Ser. No. 16 / 035,205 (Attorney Docket No. X80), filed July 13, 2018, entitled "MOBILITY DEVICE," U.S. patent application Ser. No. 15 / 787,613 (Attorney Docket No. W10), filed October 18, 2017, entitled "MOBILITY DEVICE," U.S. patent application Ser. No. 15 / 600,703 (Attorney Docket No. U22), filed May 17, 2018, entitled "SYSTEM AND METHOD FOR SECURE REMOTE CONTROL OF A MEDICAL This application is related to U.S. Patent Application No. 15 / 982,737 (Attorney Docket No. X55), filed July 15, 2017, entitled "MOBILITY DEVICE IMPROVEMENTS," U.S. Provisional Application No. 62 / 532,993 (Attorney Docket No. U30), filed September 15, 2017, entitled "MOBILITY DEVICE SEAT," U.S. Provisional Application No. 62 / 559,263 (Attorney Docket No. V85), filed September 15, 2017, entitled "MOBILITY DEVICE SEAT," and U.S. Provisional Application No. 62 / 581,670 (Attorney Docket No. W07), filed November 4, 2017, entitled "MOBILITY DEVICE SEAT."

[0003] The present teachings relate generally to AVs and, more particularly, to autonomous route planning, global occupancy grid management, on-vehicle sensors, surface feature detection and traversal, and real-time vehicle configuration changes. [Background technology]

[0004] (background) Navigation for AVs and semi-autonomous vehicles (AVs) typically relies on long-range sensors, including, but not limited to, LIDAR, cameras, stereo cameras, and radar. Long-range sensors can sense 4 to 100 meters from the AV. In contrast, object avoidance and / or surface detection typically relies on short-range sensors, including, but not limited to, stereo cameras, short-range radar, and ultrasonic sensors. These short-range sensors typically observe an area or volume of approximately 5 meters around the AV. Sensors can, for example, enable the AV to orient within its environment and navigate roads, sidewalks, obstacles, and open spaces to reach a desired destination. Sensors can also enable the AV to visualize people, signs, traffic signals, obstacles, and surface features.

[0005] Surface feature traversal can be challenging because surface features, such as, but not limited to, substantially discontinuous surface features (SDSFs), can be found in heterogeneous forms, and the forms can be unique to specific geographies. However, SDSFs, such as, but not limited to, slopes, edges, curbs, steps, and curb-like geometries (referred to herein, in a non-limiting manner, as SDSFs or simply surface features), can contain some typical characteristics that can aid in their identification. Surface / road conditions and surface types can be recognized and classified, for example, by fusing multisensory data, which can be complex and costly. Surface features and conditions can be used to control the physical reconfiguration of the AV in real time.

[0006] Sensors can be used to enable the creation of an occupancy grid that can represent the world for route planning purposes for AVs. Route planning requires a grid that identifies spaces as free, occupied, or unknown. However, the probability that a space is occupied can improve spatial decision-making. A log-odds representation of probability can be used to increase precision in the numerical bounds of probability between 0 and 1. The probability that a cell is occupied can depend on at least new sensor information, previous sensor information, and prior occupancy information.

[0007] What is needed is a system that combines aggregated and real-time sensor data with changes in the vehicle's physical configuration to accomplish variable terrain traversal. What is needed is advantageous sensor placement to accomplish physical configuration changes, variable terrain traversal, and object avoidance. What is needed is the ability to locate SDSFs based on a multipart model associated with several criteria for SDSF identification. What is needed is the determination of candidate surface feature traversals based on criteria such as candidate traversal approach angles, candidate traversal driving surfaces on either side of the candidate surface feature, and real-time determination of candidate traversal path obstacles. What is needed is a system and method for incorporating drivable surface and device mode information into occupancy grid determination. Summary of the Invention [Means for solving the problem]

[0008] (summary) An AV of the present teachings can autonomously navigate to a desired location. In some configurations, the AV can include a device controller including sensors, a perception subsystem, an autonomy subsystem, and a driver subsystem, a power base, four powered wheels, two caster wheels, and a cargo container. In some configurations, the perception and autonomy subsystems can receive and process sensor information (perception) and map information (perception and autonomy) and provide instructions to the driver subsystem. The map information can include surface classifications and associated device modes. Movement of the AV, controlled by the driver subsystem and enabled by the power base, can be sensed by the sensor subsystem and provide a feedback loop. In some configurations, the SDSF can be accurately identified from point cloud data and stored in a map, for example, according to the processes described herein. Portions of the map associated with the AV's location can be provided to the AV during navigation. The perception subsystem can maintain an occupancy grid, which can inform the AV about the probability that the path to be traversed is currently occupied. In some configurations, the AV can operate in multiple distinct modes. Modes can enable complex terrain traversal, among other benefits. A combination of maps (e.g., surface classification), sensor data (sensing features surrounding the AV), occupancy grids (probability that an upcoming route point is occupied), and modes (whether difficult terrain can be traversed) can be used to identify the direction, configuration, and speed of the AV.

[0009] Regarding preparing a map, in some configurations, a method of the present teachings for creating a map for navigating at least one SDSF encountered by an AV, wherein the AV travels a path across a surface, the surface including at least one SDSF, the path including a start point and an end point, the method can include, without limitation, accessing point cloud data representing the surface, filtering the point cloud data, forming the filtered point cloud data into processable portions, and merging the processable portions into at least one concave polygon. The method can include locating and labeling the at least one SDSF within the at least one concave polygon. The locating and labeling can form the labeled point cloud data. The method can include at least creating a graphing polygon based on the at least one concave polygon, and selecting a path from the start point to the end point based at least on the graphing polygon. When navigating, the AV can traverse at least one SDSF along the path.

[0010] Filtering the point cloud data can optionally include conditionally removing points representing transient objects and points representing outliers from the point cloud data and replacing the removed points with a preselected height. Forming the processing portions can optionally include segmenting the point cloud data into processable portions and removing points at the preselected height from the processable portions. Merging the processable portions can optionally include reducing a size of the processable portions by analyzing outliers, voxels, and normals, growing a region from the reduced-sized processable portion, determining an initial drivable surface from the grown-up region, segmenting and meshing the initial drivable surface, locating polygons in the segmented and meshed initial drivable surface, and setting the drivable surface based at least on the polygons. Locating and labeling the at least one SDSF feature may optionally include sorting the point cloud data of the drivable surface according to an SDSF filter, the SDSF filter including at least three categories of points, and locating the at least one SDSF point based at least on whether the categories of points, in combination, satisfy at least one first preselected criterion. The method may optionally include creating at least one SDSF trajectory based at least on whether a plurality of the at least one SDSF point, in combination, satisfies at least one second preselected criterion. Creating a graphing polygon may further optionally include creating at least one polygon from the at least one drivable surface. The at least one polygon may include an edge. Creating the graphing polygon may include smoothing the edge, forming a driving margin based on the smoothed edge, adding the at least one SDSF trajectory to the at least one drivable surface, and removing the edge from the at least one drivable surface according to at least one third preselected criterion. Smoothing the edges can optionally include trimming the edges outward.Providing a smoothed edge running margin can optionally include trimming the outward facing edge inward.

[0011] In some configurations, a system of the present teachings for creating a map for navigating at least one SDSF encountered by an AV, wherein the AV travels a path over a surface, the surface including the at least one SDSF, the path including a start point and an end point, the system can include, without limitation, a first processor that accesses point cloud data representing the surface, a first filter that filters the point cloud data, a second processor that forms a processable portion from the filtered point cloud data, a third processor that merges the processable portion into at least one concave polygon, a fourth processor that locates and labels the at least one SDSF within the at least one concave polygon, where the locating and labeling forms the labeled point cloud data, a fifth processor that creates a graphing polygon, and a path selector that selects a path from the start point to the end point based on at least the graphing polygon. The AV can traverse the at least one SDSF along the path.

[0012] The first filter may optionally include executable code that may include, without limitation, conditionally removing points representing transient objects and points representing outliers from the point cloud data and replacing the removed points with a preselected height. The segmenter may optionally include executable code that may include, without limitation, segmenting the point cloud data into processable portions and removing points at a preselected height from the processable portions. The third processor may optionally include executable code that may include, without limitation, reducing a size of the processable portion by analyzing outliers, voxels, and normals, growing a region from the reduced-sized processable portion, determining an initial drivable surface from the grown-up region, segmenting and meshing the initial drivable surface, locating polygons in the segmented and meshed initial drivable surface, and setting the drivable surface based at least on the polygons. The fourth processor may optionally include executable code that may include, but is not limited to, sorting the drivable surface point cloud data according to an SDSF filter, the SDSF filter including at least three categories of points, and locating at least one SDSF point based at least on whether the categories of points, in combination, satisfy at least one first preselected criterion. The system may optionally include executable code that may include, but is not limited to, creating at least one SDSF trajectory based at least on whether a plurality of the at least one SDSF point, in combination, satisfy at least one second preselected criterion.

[0013] Creating a graphed polygon may optionally include executable code that may include, but is not limited to, creating at least one polygon from at least one drivable surface, where the at least one polygon includes an edge, smoothing the edge, forming a driving margin based on the smoothed edge, adding at least one SDSF trajectory to the at least one drivable surface, and removing the edge from the at least one drivable surface according to at least one third preselected criterion. Smoothing the edge may optionally include executable code that may include, but is not limited to, trimming the edge outward. Forming a running margin of the smoothed edge may optionally include executable code that may include, but is not limited to, trimming the outward edge inward.

[0014] In some configurations, a method of the present teachings for creating a map for navigating at least one SDSF encountered by an AV, wherein the AV travels a path over a surface, the surface including at least one SDSF, the path including a start point and an end point, the method can include, without limitation, accessing a route topology. The route topology can include at least one graphed polygon that can include filtered point cloud data. The point cloud data can include labeled features and a drivable margin. The method can include transforming the point cloud data to a global coordinate system, determining a boundary of the at least one SDSF, creating an SDSF buffer of a preselected size around the boundary, determining at least one SDSF that can be traversed based at least on at least one SDSF traversal criterion, creating an edge / weighted graph based at least on the at least one SDSF traversal criterion, the transformed point cloud data, and the route topology, and selecting a route from the start point to a destination point based at least on the edge / weighted graph.

[0015] The at least one SDSF crossing criterion may optionally include a preselected width of the at least one SDSF and a preselected smoothness of the at least one SDSF, a minimum entry distance and a minimum exit distance between the at least one SDSF and the AV, including a drivable surface, and a minimum entry distance between the at least one SDSF and the AV that may accommodate an approximately 90° approach to the at least one SDSF by the AV.

[0016] In some configurations, a system of the present teachings for creating a map for navigating at least one SDSF encountered by an AV, wherein the AV travels a path over a surface, the surface including the at least one SDSF, the path including a start point and an end point, can include, without limitation, a sixth processor that accesses a route form. The route form can include at least one graphed polygon that can include filtered point cloud data. The point cloud data can include labeled features and a drivable margin. The system can include a seventh processor that transforms the point cloud data to a global coordinate system and an eighth processor that determines a boundary of the at least one SDSF. The eighth processor can create an SDSF buffer of a preselected size around the boundary. The system may include at least a ninth processor that determines at least one SDSF that may be traversed based on at least one SDSF traversal criterion; a tenth processor that creates an edge / weight graph based on at least the at least one SDSF traversal criterion, the transformed point cloud data, and the route topology; and a base controller that plans a route from the start point to the destination point based on at least the edge / weight graph.

[0017] In some configurations, a method of the present teachings for creating a map for navigating at least one SDSF encountered by an AV, wherein the AV travels a path over a surface, the surface including at least one SDSF, the path including a start point and an end point, the method can include, without limitation, accessing point cloud data representing the surface. The method can include filtering the point cloud data, forming the filtered point cloud data into processable portions, and merging the processable portions into at least one concave polygon. The method can include locating and labeling the at least one SDSF within the at least one concave polygon. The locating and labeling can form labeled point cloud data. The method can include creating a graphing polygon based at least on the at least one concave polygon. The graphing polygon can form a route configuration, and the point cloud data can include labeled features and a drivable margin. The method may include transforming the point cloud data to a global coordinate system, determining a boundary of at least one SDSF, creating an SDSF buffer of a preselected size around the boundary, determining at least one SDSF that may be traversed based at least on at least one SDSF traversal criterion, creating an edge / weight graph based at least on the at least one SDSF traversal criterion, the transformed point cloud data, and a route topology, and selecting a route from a start point to a destination point based at least on the edge / weight graph.

[0018] Filtering the point cloud data can optionally include conditionally removing points representing transient objects and points representing outliers from the point cloud data and replacing the removed points with a preselected height. Forming the processing portions can optionally include segmenting the point cloud data into processable portions and removing points at the preselected height from the processable portions. Merging the processable portions can optionally include reducing a size of the processable portions by analyzing outliers, voxels, and normals, growing a region from the reduced-sized processable portion, determining an initial drivable surface from the grown-up region, segmenting and meshing the initial drivable surface, locating polygons in the segmented and meshed initial drivable surface, and setting the drivable surface based at least on the polygons. Locating and labeling the at least one SDSF feature may optionally include sorting the point cloud data of the drivable surface according to an SDSF filter, the SDSF filter including at least three categories of points, and locating the at least one SDSF point based at least on whether the categories of points, in combination, satisfy at least one first preselected criterion. The method may optionally include creating at least one SDSF trajectory based at least on whether a plurality of the at least one SDSF point, in combination, satisfies at least one second preselected criterion. Creating a graphing polygon may further optionally include creating at least one polygon from the at least one drivable surface. The at least one polygon may include an edge. Creating the graphing polygon may include smoothing the edge, forming a driving margin based on the smoothed edge, adding the at least one SDSF trajectory to the at least one drivable surface, and removing the edge from the at least one drivable surface according to at least one third preselected criterion. Smoothing the edges can optionally include trimming the edges outward.Forming the smoothed edge running margin can optionally include inward trimming of the outward edge. The at least one SDSF crossing criterion can optionally include a preselected width of the at least one SDSF and a preselected smoothness of the at least one SDSF, a minimum entry distance and a minimum exit distance between the at least one SDSF and the AV that includes a drivable surface, and a minimum entry distance between the at least one SDSF and the AV that can accommodate an approximately 90° approach to the at least one SDSF by the AV.

[0019] In some configurations, a system of the present teachings for creating a map for navigating at least one SDSF encountered by an AV, wherein the AV travels a path over a surface, the surface including at least one SDSF, and the path includes a start point and an end point, can include, without limitation, a point cloud accessor that accesses point cloud data representing the surface, a first filter that filters the point cloud data, a segmenter that forms processable portions from the filtered point cloud data, a third processor that merges the processable portions into at least one concave polygon, a fourth processor that locates and labels the at least one SDSF within the at least one concave polygon, where the locating and labeling forms labeled point cloud data, and a fifth processor that creates a graphing polygon. The route configuration can include at least one graphing polygon that can include the filtered point cloud data. The point cloud data can include labeled features and a drivable margin. The system may include a seventh processor that transforms the point cloud data to a global coordinate system and an eighth processor that determines a boundary of at least one SDSF. The eighth processor may create an SDSF buffer of a preselected size around the boundary. The system may include a ninth processor that determines at least one SDSF that may be traversed based on at least one SDSF traversal criterion, a tenth processor that creates an edge / weight graph based on at least the at least one SDSF traversal criterion, the transformed point cloud data, and a route topology, and a base controller that plans a route from a start point to a destination point based on at least the edge / weight graph.

[0020] The first filter may optionally include executable code that may include, without limitation, conditionally removing points representing transient objects and points representing outliers from the point cloud data and replacing the removed points with a preselected height. The segmenter may optionally include executable code that may include, without limitation, segmenting the point cloud data into processable portions and removing points at a preselected height from the processable portions. The third processor may optionally include executable code that may include, without limitation, reducing a size of the processable portion by analyzing outliers, voxels, and normals, growing a region from the reduced-sized processable portion, determining an initial drivable surface from the grown-up region, segmenting and meshing the initial drivable surface, locating polygons in the segmented and meshed initial drivable surface, and setting the drivable surface based at least on the polygons. The fourth processor may optionally include executable code that may include, but is not limited to, sorting the drivable surface point cloud data according to an SDSF filter, the SDSF filter including at least three categories of points, and locating at least one SDSF point based at least on whether the categories of points, in combination, satisfy at least one first preselected criterion. The system may optionally include executable code that may include, but is not limited to, creating at least one SDSF trajectory based at least on whether a plurality of the at least one SDSF point, in combination, satisfy at least one second preselected criterion.

[0021] Creating a graphed polygon may optionally include executable code that may include, but is not limited to, creating at least one polygon from at least one drivable surface, where the at least one polygon includes an edge, smoothing the edge, forming a driving margin based on the smoothed edge, adding at least one SDSF trajectory to the at least one drivable surface, and removing the edge from the at least one drivable surface according to at least one third preselected criterion. Smoothing the edge may optionally include executable code that may include, but is not limited to, trimming the edge outward. Forming a running margin of the smoothed edge may optionally include executable code that may include, but is not limited to, trimming the outward edge inward.

[0022] In some configurations, the SDSF can be identified by its dimensions. For example, a curb may include, but is not limited to, a width of approximately 0.6-0.7 m. In some configurations, point cloud data can be processed to locate the SDSF, which can be used to prepare a route for the AV from a start point to a destination. In some configurations, the route can be contained within a map and provided to a perception subsystem. As the AV travels along the route, in some configurations, SDSF traversal can be adapted, in part, through sensor-based positioning of the AV, enabled by the perception subsystem. The perception subsystem can execute on at least one processor within the AV.

[0023] The AV may include a power base, including, but not limited to, two powered front wheels, two powered rear wheels, and an energy storage device, and at least one processor. The power base may be configured to move at a commanded speed. The AV may include a cargo platform mechanically attached to the power base and including a plurality of short-range sensors. The AV, in some configurations, may include a cargo container mounted on the cargo platform and having a volume for receiving one or more objects to be delivered. The AV, in some configurations, may include a long-range sensor suite mounted on the cargo container and may include, but is not limited to, a LIDAR and one or more cameras. The AV may include a controller that may receive data from the long-range sensor suite and the short-range sensor suite.

[0024] The short-range sensor suite can optionally detect at least one characteristic of the drivable surface and can optionally include a stereo camera, an IR projector, two image sensors, an RGB sensor, and a radar sensor. The short-range sensor suite can optionally provide RGB-D data to the controller. The controller can optionally determine the geometry of the road surface based on the RGB-D data received from the short-range sensor suite. The short-range sensor suite can optionally detect objects within 4 meters of the AV, and the long-range sensor suite can optionally detect objects more than 4 meters from the AV.

[0025] The perception subsystem can use data collected by the sensors to capture an occupancy grid. The occupancy grid of the present teachings is configured as a 3D grid of points surrounding the AV, with the AV occupying a center point. In some configurations, the occupancy grid can extend 10 meters to the left, right, rear, and front of the AV. The grid can encompass approximately the height of the AV and virtually travel with the AV as it moves, representing obstacles surrounding the AV. The grid can be converted to two dimensions by shrinking its vertical axis, for example, but not by way of limitation, divided into polygons approximately 5 cm by 5 cm in size. Obstacles that appear in the 3D space around the AV can be reduced to 2D shapes. If the 2D shape overlaps any segment of one of the polygons, the polygon can be given a value of 100, indicating that the space is occupied. Any polygons that remain unfilled can be given a value of 0 and can be referred to as free space through which the AV may move.

[0026] As an AV navigates, it may encounter situations that may require a change in the AV's configuration. A method of the present teachings for real-time control of a device's configuration, in some configurations, the device includes a chassis, at least four wheels, a first side of the chassis operably coupled to at least one of the at least four wheels, and an opposing second side of the chassis operably coupled to at least one of the at least four wheels, the method can include, without limitation, receiving environmental data, determining a surface type based at least on the environmental data, determining a mode based at least on the surface type and the first configuration, determining a second configuration based at least on the mode and the surface type, determining a movement command based on the at least the second configuration, and controlling the configuration of the device by using the movement command to change the device from the first configuration to the second configuration.

[0027] The method optionally includes capturing an occupancy grid based on at least a surface type and a mode. The environmental data optionally includes RGB-D image data and a road surface morphology. The configuration optionally includes two clustered pairs of at least four wheels. A first pair of the two pairs can be positioned on a first side, and a second pair of the two pairs can be positioned on a second side. The first pair can include a first front wheel and a first rear wheel, and the second pair can include a second front wheel and a second rear wheel. Control of the configuration optionally includes coordinated powering of the first pair and the second pair based at least on the environmental data. Control of the configuration optionally includes transitioning from driving the at least four wheels and a pair of casters that are retracted to driving two wheels with the clustered first pair and the clustered second pair rotated to lift the first front wheel and the second front wheel. The pair of casters can be operably coupled to the chassis. The device can be resting on the first rear wheel, the second rear wheel, and the pair of casters. Controlling the configuration can optionally include rotating a pair of clusters operably coupled with the two powered wheels on the first side and the two powered wheels on the second side based on at least the environmental data.

[0028] A system of the present teachings for real-time control of the configuration of an AV can include, but is not limited to, a device processor and a power base processor. The AV can include a chassis, at least four wheels, a first side of the chassis, and an opposing second side of the chassis. The device processor can receive real-time environmental data surrounding the AV, determine a surface type based on at least the environmental data, determine a mode based on at least the surface type and the first configuration, and determine a second configuration based on at least the mode and the surface type. The power base processor can enable the AV to move based on at least the second configuration and can enable the AV to change from the first configuration to the second configuration. The device processor can optionally include populating an occupancy grid based on at least the surface type and the mode.

[0029] During navigation, an AV may encounter an SDSF that may require the AV to steer for successful traversal. In some configurations, a method of the present teachings for navigating an AV along a path line within a going area toward a target point that traverses at least one SDSF, wherein the AV includes a leading edge and a trailing edge, can include, without limitation, receiving SDSF information and obstacle information related to the going area, detecting at least one candidate SDSF from the SDSF information, and selecting an SDSF line from the at least one candidate SDSF line based on at least one selection criterion. The method can include determining at least one traversable portion of the selected SDSF line based on at least one location of at least one obstacle found in the obstacle information near the selected SDSF line, heading the AV toward the at least one traversable portion by turning the AV to travel along a line perpendicular to the traversable portion and operating at a first speed, and constantly correcting the heading of the AV based on a relationship between the heading and the perpendicular line. The method may include traveling the AV at a second speed by adjusting a first speed of the AV based on at least the heading and the distance between the AV and the traversable portion. If an SDSF associated with the at least one traversable portion is elevated relative to a surface of the travel route, the method may include crossing the SDSF by elevating the leading edge relative to the trailing edge and traveling the AV at a third increased speed per degree of elevation, and traveling the AV at a fourth speed until the AV clears the SDSF.

[0030] Detecting at least one candidate SDSF from the SDSF information can optionally include (a) drawing a closed polygon encompassing the location of the AV and the location of the target point, (b) drawing a path line between the target point and the AV location, (c) selecting two SDSF points from the SDSF information, the SDSF points being located within the polygon, and (d) drawing an SDSF line between the two points. Detecting at least one candidate SDSF can include (e) repeating steps (c)-(e) if there are fewer than a first preselected number of points within a first preselected distance of the SDSF line, and if there are fewer than a second preselected number of attempts at selecting SDSF points and drawing a line between them around the SDSF line. Detecting at least one candidate SDSF may include (f) fitting a curve to the SDSF points that fall within a first preselected distance of the SDSF line if a first preselected number of points or more exist; and (g) identifying the curve as an SDSF line if a first number of SDSF points within the first preselected distance of the curve exceeds a second number of SDSF points within the first preselected distance of the SDSF line, and if the curve intersects with a path line, and if there are no gaps between the SDSF points on the curve that exceed the second preselected distance. Detecting at least one candidate SDSF may include (h) repeating steps (f)-(h) if the number of points within a first preselected distance of the curve does not exceed the number of points within a first preselected distance of the SDSF line, or if the curve does not intersect with the path line, or if there are gaps between SDSF points on the curve that exceed a second preselected distance, and if the SDSF line does not remain stable, and if steps (f)-(h) have not been attempted more than a second preselected number of times.

[0031] The closed polygon can optionally include a preselected width, which can optionally include a width dimension of the AV. Selecting the SDSF points can optionally include random selection. The at least one selection criterion can optionally include a first number of SDSF points within a first preselected distance of the curve exceeding a second number of SDSF points within a first preselected distance of the SDSF line, the curve intersecting the path line, and no gaps between the SDSF points on the curve exceeding a second preselected distance.

[0032] Determining at least one traversable portion of the selected SDSF can optionally include selecting a plurality of obstacle points from the obstacle information. Each of the plurality of obstacle points can include a probability that the obstacle point is associated with at least one obstacle. Determining the at least one traversable portion can include projecting the plurality of obstacle points onto the SDSF line to form at least one projection if the probability is higher than a preselected percentage and if any of the plurality of obstacle points is located between the SDSF line and the target point and if any of the plurality of obstacle points is less than a third preselected distance from the SDSF line. Determining the at least one traversable portion can optionally include interconnecting at least two of the at least one projection, locating endpoints of the at least two connected projections along the SDSF line, marking the at least two connected projections as non-traversable SDSF segments, and marking the SDSF line outside the non-traversable segments as at least one traversable segment.

[0033] Traversing at least one traversable portion of the SDSF may optionally include turning the AV to travel along a line perpendicular to the traversable portion, orienting the AV toward the traversable portion, and operating at a first speed, constantly correcting the orientation of the AV based on a relationship between the orientation and the perpendicular line, and traveling the AV at a second speed by adjusting the first speed of the AV based on at least the orientation and the distance between the AV and the traversable portion. Traversing at least one traversable portion of the SDSF may optionally include, if the SDSF rises relative to a surface of the travel route, crossing the SDSF by raising the leading edge relative to the trailing edge and traveling the AV at a third increased speed per degree of rise, and traveling the AV at a fourth speed until the AV has cleared the SDSF.

[0034] Traversing at least one traversable portion of the SDSF may alternatively and optionally include: (a) ignoring updates to the SDSF information and driving the AV at a preselected speed if the heading error is less than a third preselected amount relative to a line perpendicular to the SDSF line; (b) driving the AV forward and increasing the speed of the AV to an eighth preselected speed per degree of elevation if the elevation of the front portion of the AV relative to the rear portion of the AV is between a sixth preselected amount and a fifth preselected amount; (c) driving the AV forward at a seventh preselected speed if the front portion is elevated less than the sixth preselected amount relative to the rear portion; and (d) repeating steps (a)-(d) if the rear portion is less than or equal to a fifth preselected distance from the SDSF line.

[0035] In some configurations, the SDSF and AV wheels can be automatically aligned to avoid system instability. Automatic alignment can be implemented, for example, but not limited to, by continuously testing and correcting the orientation of the AV as it approaches the SDSF. Another aspect of the SDSF traversal feature of the present teachings is that it automatically verifies that sufficient free space exists around the SDSF before attempting crossing. Yet another aspect of the SDSF traversal feature of the present teachings is its ability to traverse SDSFs of various geometries. Geometric shapes can include, for example, but not limited to, square and contoured SDSFs. The orientation of the AV relative to the SDSF can determine the speed and direction the AV travels. The SDSF traversal feature can adjust the speed of the AV in the vicinity of the SDSF. When the AV ascends to the SDSF, speed can be increased to assist the AV in crossing the SDSF.

[0036] 1. An autonomous delivery vehicle comprising: a power base including two powered front wheels, two powered rear wheels, and an energy storage device, the power base configured to move at a commanded speed and in a commanded direction to perform transportation of at least one object; a cargo platform including a plurality of short-range sensors, the cargo platform mechanically attached to the power base; a cargo container with a volume for receiving at least one object, the cargo container mounted on the cargo platform; a long-range sensor suite including a LIDAR and one or more cameras, the long-range sensor suite mounted on the cargo container; and a controller for receiving data from the long-range sensor suite and the plurality of short-range sensors, the controller determining a commanded speed and a commanded direction based on at least the data, and the controller providing the commanded speed and commanded direction to the power base to complete the transportation. 2. The autonomous delivery vehicle of item 1, wherein the data from the plurality of short-range sensors comprises at least one characteristic of a surface over which the power base travels. 3. The autonomous delivery vehicle of item 1, wherein the plurality of short-range sensors comprises at least one stereo camera. 4. The autonomous delivery vehicle of item 1, wherein the plurality of short-range sensors comprises at least one IR projector, at least one image sensor, and at least one RGB sensor. 5. The autonomous delivery vehicle of item 1, wherein the plurality of short-range sensors comprises at least one radar sensor. 6. The autonomous delivery vehicle of item 1, wherein the data from the plurality of short-range sensors comprises RGB-D data. 7. The autonomous delivery vehicle of item 1, wherein the controller determines the geometry of the road surface based on the RGB-D data received from the plurality of short-range sensors. 8. The autonomous delivery vehicle of item 1, wherein the plurality of short-range sensors detect objects within 4 meters of the AV and the long-range sensor suite detects objects more than 4 meters from the autonomous delivery vehicle. 9. The autonomous delivery vehicle described in item 1, wherein the plurality of short-range sensors includes a cooling circuit.10. The autonomous delivery vehicle of item 1, wherein the plurality of short-range sensors comprises ultrasonic sensors. 11. The autonomous delivery vehicle of item 2, wherein the controller comprises: a first processor that accesses a map, the map formed by a map processor that accesses point cloud data from a long-range sensor suite, the point cloud data representing a surface; a second processor that filters the point cloud data, the filter forming a processable portion from the filtered point cloud data; a third processor that merges the processable portion into at least one polygon; a fourth processor that locates and labels at least one substantially discontinuous surface feature (SDSF) within the at least one polygon, if present, wherein the locating and labeling forms labeled point cloud data; a fifth processor that creates a graphing polygon from the labeled point cloud data; and a sixth processor that plans a route from a start point to an end point based at least on the graphing polygon, wherein the AV traverses at least one SDSF along the route. 12. The autonomous delivery vehicle of item 11, wherein the filter comprises a seventh processor that executes code including: conditionally removing points representing transient objects and points representing outliers from the point cloud data; and replacing the removed points with a preselected height. 13. The autonomous delivery vehicle of item 11, wherein the second processor includes executable code including: segmenting the point cloud data into processable portions; and removing points with a preselected height from the processable portions.14. The autonomous delivery vehicle of item 11, wherein the third processor includes executable code including: reducing a size of the processable portion by analyzing outliers, voxels, and normals, growing a region from the reduced-sized processable portion, determining an initial drivable surface from the grown-up region, segmenting and meshing the initial drivable surface, locating polygons in the segmented and meshed initial drivable surface, and setting at least one drivable surface based on at least the polygons. 15. The autonomous delivery vehicle of item 14, wherein the fourth processor includes executable code including: sorting point cloud data of the initial drivable surface according to an SDSF filter, the SDSF filter including at least three categories of points, and locating at least one SDSF point based on whether the at least three categories of points, in combination, satisfy at least one first preselected criterion. 16. The autonomous delivery vehicle of item 15, including an eighth processor including executable code that includes: creating at least one SDSF trajectory based on whether a plurality of at least one SDSF point, in combination, satisfies at least one second preselected criterion. 17. The autonomous delivery vehicle of item 14, including an eighth processor including executable code that includes: creating at least one polygon from at least one drivable surface, where the at least one polygon includes an outer edge; smoothing the outer edge; forming a driving margin based on the smoothed outer edge; adding at least one SDSF trajectory to the at least one drivable surface; and removing an inner edge from the at least one drivable surface according to at least one third preselected criterion. 18. The autonomous delivery vehicle of item 17, including a ninth processor including executable code that includes: smoothing the outer edge19. The autonomous delivery vehicle of item 18, including a tenth processor including executable code for forming a smoothed outer edge driving margin includes inwardly trimming the outward edge. 20. The controller includes a subsystem for navigating at least one substantially discontinuous surface feature (SDSF) encountered by an autonomous delivery vehicle (AV), wherein the AV travels a path over the surface, the surface including at least one SDSF, the path including a start point and an end point, the first processor accessing a route form, the route form including at least one graphed polygon including filtered point cloud data, the filtered point cloud data including labeled features, and the point cloud data including a drivable margin; a second processor for transforming the point cloud data to a global coordinate system; and 20. The autonomous delivery vehicle of claim 1, comprising a subsystem comprising: a third processor that determines a boundary of at least one SDSF, wherein the third processor creates an SDSF buffer of a preselected size around the boundary; a fourth processor that determines at least one SDSF that may be traversed based on at least one SDSF crossing criterion; a fifth processor that creates an edge / weighted graph based on at least the at least one SDSF crossing criterion, the transformed point cloud data, and the route morphology; and a base controller that plans a route from a start point to an end point based on at least the edge / weighted graph. 21. The autonomous delivery vehicle of claim 20, wherein the at least one SDSF crossing criterion comprises a preselected width of the at least one SDSF and a preselected smoothness of the at least one SDSF, a minimum entry distance and a minimum exit distance between the at least one SDSF and the AV including a drivable surface, and a minimum entry distance between the at least one SDSF and the AV that accommodates an approximately 90° approach to the at least one SDSF by the AV.

[0037] 22. A method for managing a global occupancy grid for an autonomous device, the global occupancy grid including global occupancy grid cells, the global occupancy grid cells being associated with an occupancy probability, receiving sensor data from a sensor associated with the autonomous device, creating a local occupancy grid based on at least the sensor data, the local occupancy grid having local occupancy grid cells, when the autonomous device moves from a first area to a second area, accessing historical data associated with the second area, creating a static grid based on at least the historical data, and moving the global occupancy grid to locate the autonomous device at a center location of the global occupancy grid. , updating the moved global occupancy grid based on the static grid, marking at least one of the global occupancy grid cells as unoccupied if at least one of the global occupancy grid cells matches a location of the autonomous device, and for each local occupancy grid cell, calculating a position of the local occupancy grid cell on the global occupancy grid, accessing a first occupancy probability from the global occupancy grid cell at the position, accessing a second occupancy probability from the local occupancy grid cell at the position, and calculating a new occupancy probability at the position on the global occupancy grid based on at least the first occupancy probability and the second occupancy probability. 23. The method of claim 22, further including range checking the new occupancy probability. 24. The method of claim 23, wherein range checking includes setting the new occupancy probability to 0 if the new occupancy probability is <0, and setting the new occupancy probability to 1 if the new occupancy probability is >1. 25. The method of claim 22, further including setting the global occupancy grid cell to the new occupancy probability. 26. The method of item 23, further comprising setting the global occupancy grid cells to the range-checked new occupancy probability.

[0038] 27. A method for creating and managing an occupancy grid, comprising: transforming, by a local occupancy grid creation node, sensor measurements into a reference frame associated with a device; creating a time-stamped measurement occupancy grid; publishing the time-stamped measurement occupancy grid as a local occupancy grid; creating a plurality of local occupancy grids; creating a static occupancy grid based on surface characteristics in a repository, the surface characteristics being associated with a location of the device; moving a global occupancy grid associated with the location of the device to maintain the device and local occupancy grid approximately aligned with the global occupancy grid; 28. The method of claim 27, further comprising: adding information from a local occupancy grid to a global occupancy grid; marking an area in the global occupancy grid currently occupied by the device as unoccupied; and for at least one cell in each local occupancy grid, determining a location of the at least one cell in the global occupancy grid, accessing a first value at the location, determining a second value at the location based on a relationship between the first value and the cell value of the at least one cell in the local occupancy grid, comparing the second value against a preselected probability range, and setting the global occupancy grid with the new value if the probability value is within the preselected probability range. 29. The method of claim 27, further comprising publishing the global occupancy grid. 29. The method of claim 27, wherein the surface characteristics comprise a surface type and a surface discontinuity. 30. The method of claim 27, wherein the relationship comprises summing.31. A system for creating and managing an occupancy grid, comprising: a plurality of local grid creation nodes that create at least one local occupancy grid, the at least one local occupancy grid being associated with a location of a device, the at least one local occupancy grid including at least one cell; and a global occupancy grid manager that accesses the at least one local occupancy grid, the global occupancy grid creating a static occupancy grid based on surface characteristics in a repository, the surface characteristics being associated with the location of the device; and moving a global occupancy grid associated with the location of the device to substantially align the global occupancy grid with the device and the at least one cell. and a global occupancy grid manager that maintains local occupancy grids, adds information from the static occupancy grid to at least one global occupancy grid, marks areas in the global occupancy grid that are currently occupied by the device as unoccupied, and for at least one cell in each local occupancy grid, determines a location of the at least one cell in the global occupancy grid, accesses a first value at the location, determines a second value at the location based on a relationship between the first value and the cell value in the at least one cell in the local occupancy grid, compares the second value against a preselected probability range, and sets the global occupancy grid with the new value if the probability value is within the preselected probability range.

[0039] 32. A method for updating a global occupancy grid, the method including: when an autonomous device moves to a new location, updating the global occupancy grid with information from a static grid associated with the new location; analyzing a surface at the new location; and, if the surface is drivable, updating the surface and updating the global occupancy grid with the updated surface; and updating the global occupancy grid with values ​​from a repository of static values, the static values ​​being associated with the new location. 33. The method of item 32, wherein updating the surface includes accessing a local occupancy grid associated with the new location; for each cell in the local occupancy grid, accessing a local occupancy grid surface classification confidence value and a local occupancy grid surface classification; if the local occupancy grid surface classification is identical to the global surface classification in the global occupancy grid in the cell, adding the global surface classification confidence value in the global occupancy grid to the local occupancy grid surface classification confidence value to form a sum and updating the global occupancy grid at the cell using the sum; if the local occupancy grid surface classification is not identical to the global surface classification in the global occupancy grid in the cell, subtracting the local occupancy grid surface classification confidence value from the global surface classification confidence value in the global occupancy grid to form a difference and updating the global occupancy grid using the difference; and if the difference is less than zero, updating the global occupancy grid with the local occupancy grid surface classification.34. The method of item 32, wherein updating the global occupancy grid with values ​​from the repository of static values ​​includes: for each cell in the local occupancy grid, accessing a log-odds value, which is a local occupancy grid probability that the cell from the local occupancy grid is an occupied value; updating the log-odds value in the global occupancy grid with the local occupancy grid log-odds value at the cell; decreasing the log-odds that the cell is occupied in the local occupancy grid if a preselected certainty that the cell is unoccupied is met and if the autonomous device is proceeding within a lane barrier and the local occupancy grid surface classification indicates a drivable surface; increasing the log-odds in the local occupancy grid if the autonomous device expects to encounter a relatively uniform surface and the local occupancy grid surface classification indicates a relatively non-uniform surface; and decreasing the log-odds in the local occupancy grid if the autonomous device expects to encounter a relatively uniform surface and the local occupancy grid surface classification indicates a relatively uniform surface.

[0040] 35. A method for real-time control of a configuration of a device, the device including a chassis, at least four wheels, a first side of the chassis operatively coupled to at least one of the at least four wheels, and an opposing second side of the chassis operatively coupled to at least one of the at least four wheels; creating a map based on at least prior surface features and an occupancy grid, the map being created in non-real time, the map including at least one location, the at least one location associated with at least one surface feature, the at least one surface feature associated with at least one surface classification and at least one mode; determining current surface features as the device travels; updating the occupancy grid in real time with the current surface features; and determining from the occupancy grid and the map a path the device may travel to traverse the at least one surface feature.

[0041] 36. A method for real-time control of a configuration of a device, the device including a chassis, at least four wheels, a first side of the chassis operably coupled to at least one of the at least four wheels, and an opposing second side of the chassis operably coupled to at least one of the at least four wheels, the method comprising: receiving environmental data; determining a surface type based at least on the environmental data; determining a mode based at least on the surface type and the first configuration; determining a second configuration based at least on the mode and the surface type; determining a movement command based at least on the second configuration; and controlling the configuration of the device by using the movement command to change the device from the first configuration to the second configuration. 37. The method of claim 36, wherein the environmental data comprises RGB-D image data. 38. The method of claim 36, further comprising capturing an occupancy grid based at least on the surface type and the mode, and determining the movement command based at least on the occupancy grid. 39. The method of claim 38, wherein the occupancy grid comprises at least information based on data from at least one image sensor. 40. The method of claim 36, wherein the environmental data comprises a morphology of a road surface. 41. The method of claim 36, wherein the configuration comprises two clustered pairs of at least four wheels, a first pair of the two pairs being positioned on a first side and a second pair of the two pairs being positioned on a second side, the first pair including a first front wheel and a first rear wheel, and the second pair including a second front wheel and a second rear wheel. 42. The method of claim 41, wherein controlling the configuration includes coordinated powering of the first pair and the second pair based on at least the environmental data. 43. The method of item 41, wherein controlling the configuration includes transitioning from driving at least four wheels and pairs of casters that are retracted, the pairs of casters being operably coupled to the chassis, to driving two wheels with a clustered first pair and a clustered second pair that are rotated to lift a first front wheel and a second front wheel, the device resting on a first rear wheel, a second rear wheel, and the pairs of casters.44. The method of item 41, wherein controlling the configuration includes rotating a pair of clusters operably coupled with a first two powered wheels on a first side and a second two powered wheels on a second side based at least on environmental data. 45. The method of item 36, wherein the device further comprises a cargo container, the cargo container mounted on a chassis, the chassis controlling a height of the cargo container. 46. The method of item 45, wherein the height of the cargo container is based at least on environmental data.

[0042] 47. A system for real-time control of a configuration of a device, the device including a chassis, at least four wheels, a first side of the chassis, and an opposing second side of the chassis; a device processor receiving real-time environmental data surrounding the device, the device processor determining a surface type based at least on the environmental data, the device processor determining a mode based at least on the surface type and the first configuration, and the device processor determining a second configuration based at least on the mode and surface type; and a power base processor determining movement commands based at least on the second configuration, the power base processor controlling the configuration of the device by using the movement commands to change the device from the first configuration to the second configuration. 48. The system of item 47, wherein the environmental data comprises RGB-D image data. 49. The system of item 47, wherein the device processor includes capturing an occupancy grid based at least on the surface type and the mode. 50. The system of item 49, wherein the power base processor includes determining movement commands based at least on the occupancy grid. 51. The system of item 49, wherein the occupancy grid comprises at least information based on data from at least one image sensor. 52. The system of item 47, wherein the environmental data comprises a morphology of the road surface. 53. The system of item 47, wherein the configuration comprises two clustered pairs of at least four wheels, a first pair of the two pairs being positioned on a first side and a second pair of the two pairs being positioned on a second side, the first pair having a first front wheel and a first rear wheel, and the second pair having a second front wheel and a second rear wheel. 54. The system of item 53, wherein controlling the configuration includes coordinated powering of the first pair and the second pair based at least on the environmental data.55. The system of item 53, wherein controlling the configuration includes transitioning from driving at least four wheels and pairs of casters that are retracted, the pairs of casters being operably coupled to the chassis, to driving two wheels with a clustered first pair and a clustered second pair that are rotated to lift a first front wheel and a second front wheel, the device resting on a first rear wheel, a second rear wheel, and the pairs of casters.

[0043] 56. A method for maintaining a global occupancy grid, comprising: locating a first position of an autonomous device; and when the autonomous device moves to a second position, the second position is associated with a global occupancy grid and a local occupancy grid; updating the global occupancy grid with at least one occupancy probability value associated with the first position; updating the global occupancy grid with at least one drivable surface associated with the local occupancy grid; updating the global occupancy grid with a surface confidence associated with the at least one drivable surface; and updating the global occupancy grid with log-odds of the at least one occupancy probability value using a first Bayes function. and adjusting the log-odds based on a characteristic associated with at least a second location; when the autonomous device remains at the first location and the global occupancy grid and the local occupancy grid are co-located, updating the global occupancy grid with at least one drivable surface associated with the local occupancy grid; updating the global occupancy grid with a surface confidence associated with the at least one drivable surface; updating the global occupancy grid with the log-odds of the at least one occupancy probability value using a second Bayes function; and adjusting the log-odds based on a characteristic associated with at least the second location. 57. The method of item 35, wherein creating the map includes accessing point cloud data representing the surface, filtering the point cloud data, forming the filtered point cloud data into a processable portion, merging the processable portion into at least one concave polygon, locating and labeling at least one SDSF within the at least one concave polygon, wherein the locating and labeling forms the labeled point cloud data; creating a graphing polygon based at least on the at least one concave polygon; and selecting a route from a start point to an end point based at least on the graphing polygon, wherein the AV traverses the at least one SDSF along the route.58. The method of item 57, wherein filtering the point cloud data includes conditionally removing points representing transient objects and points representing outliers from the point cloud data, and replacing the removed points with a preselected height. 59. The method of item 57, wherein forming the processable portions includes segmenting the point cloud data into processable portions and removing points at a preselected height from the processable portions. 60. The method of item 57, wherein merging the processable portions includes reducing a size of the processable portions by analyzing outliers, voxels, and normals, growing a region from the reduced-sized processable portion, determining an initial drivable surface from the grown-up region, segmenting and meshing the initial drivable surface, locating polygons within the segmented and meshed initial drivable surface, and setting at least one drivable surface based on at least the polygons. 61. The method of item 60, wherein locating and labeling at least one SDSF includes sorting point cloud data of the initial drivable surface according to an SDSF filter, wherein the SDSF filter includes at least three categories of points, and locating the at least one SDSF point based on whether the at least three categories of points, in combination, satisfy at least one first preselected criterion. 62. The method of item 61, further including creating at least one SDSF trajectory based on whether a plurality of at least one SDSF point, in combination, satisfy at least one second preselected criterion. 63. The method of item 62, wherein creating a graphed polygon further includes creating at least one polygon from at least one drivable surface, the at least one polygon including an outer edge; smoothing the outer edge; forming a driving margin based on the smoothed outer edge; adding at least one SDSF trajectory to the at least one drivable surface; and removing an inner edge from the at least one drivable surface according to at least one third preselected criterion.64. The method of item 63, wherein smoothing the outer edge comprises trimming the outer edge outward to form an outward edge. 65. The method of item 63, wherein forming a running margin of the smoothed outer edge comprises trimming the outward edge inward.

[0044] 66. An autonomous delivery vehicle comprising: a power base including two powered front wheels, two powered rear wheels, and an energy storage device, the power base configured to move at a commanded speed; a cargo platform including a plurality of short-range sensors, the cargo platform mechanically attached to the power base; a cargo container with a volume for receiving one or more objects to be delivered, the cargo container mounted on the cargo platform; a long-range sensor suite including a LIDAR and one or more cameras, the long-range sensor suite mounted on the cargo container; and a controller for receiving data from the long-range sensor suite and the plurality of short-range sensors. 67. The autonomous delivery vehicle of item 66, wherein the plurality of short-range sensors detect at least one characteristic of a drivable surface. 68. The autonomous delivery vehicle of item 66, wherein the plurality of short-range sensors are stereo cameras. 69. The autonomous delivery vehicle of item 66, wherein the multiple short-range sensors comprise an IR projector, two image sensors, and an RGB sensor. 70. The autonomous delivery vehicle of item 66, wherein the multiple short-range sensors are radar sensors. 71. The autonomous delivery vehicle of item 66, wherein the short-range sensors provide RGB-D data to a controller. 72. The autonomous delivery vehicle of item 66, wherein the controller determines a geometry of the road surface based on the RGB-D data received from the multiple short-range sensors. 73. The autonomous delivery vehicle of item 66, wherein the multiple short-range sensors detect objects within 4 meters of the autonomous delivery vehicle and the long-range sensor suite detects objects more than 4 meters from the autonomous delivery vehicle.

[0045] 74. An autonomous delivery vehicle comprising: a power base including at least two powered rear wheels, a caster front wheel, and an energy storage device, the power base configured to move at a commanded speed; a cargo platform including a plurality of short-range sensors, the cargo platform mechanically attached to the power base; a cargo container with a volume for receiving one or more objects to be delivered, the cargo container mounted on the cargo platform; a long-range sensor suite including a LIDAR and one or more cameras, the long-range sensor suite mounted on the cargo container; and a controller for receiving data from the long-range sensor suite and the plurality of short-range sensors. 75. The autonomous delivery vehicle of item 74, wherein the plurality of short-range sensors detect at least one characteristic of a drivable surface. 76. The autonomous delivery vehicle of item 74, wherein the plurality of short-range sensors are stereo cameras. 77. The autonomous delivery vehicle of item 74, wherein the multiple short-range sensors comprise an IR projector, two image sensors, and an RGB sensor. 78. The autonomous delivery vehicle of item 74, wherein the multiple short-range sensors are radar sensors. 79. The autonomous delivery vehicle of item 74, wherein the short-range sensors provide RGB-D data to the controller. 80. The autonomous delivery vehicle of item 74, wherein the controller determines the geometry of the road surface based on the RGB-D data received from the multiple short-range sensors. 81. The autonomous delivery vehicle of item 74, wherein the multiple short-range sensors detect objects within 4 meters of the autonomous delivery vehicle, and the long-range sensor suite detects objects more than 4 meters from the autonomous delivery vehicle. 82. The autonomous delivery vehicle of item 74, further comprising a second set of powered wheels that can engage the ground while the caster wheels are lifted off the ground.

[0046] 83. An autonomous delivery vehicle comprising: a power base including at least two powered rear wheels, a caster front wheel, and an energy storage device, the power base configured to move at a commanded speed; a cargo platform, the cargo platform mechanically attached to the power base; and a short-range camera assembly mounted on the cargo platform that detects at least one characteristic of the drivable surface, the short-range camera assembly comprising a camera, a first light, and a first liquid-cooled heat sink, the first liquid-cooled heat sink cooling the first light and the camera. 84. The autonomous delivery vehicle of item 83, wherein the short-range camera assembly further comprises a thermoelectric cooler between the camera and the liquid-cooled heat sink. 85. The autonomous delivery vehicle of item 83, wherein the first light and the camera are recessed in a cover with an opening that deflects illumination from the first light away from the camera. 86. The autonomous delivery vehicle of item 83, wherein the light is angled downward at least 15° and recessed at least 4 mm into the cover to minimize lighting that is distracting to pedestrians. 87. The autonomous delivery vehicle of item 83, wherein the camera has a field of view and the first light comprises two LEDs with lenses for generating two beams of light that are diffused to illuminate the field of view of the camera. 88. The autonomous delivery vehicle of item 87, wherein the lights are angled approximately 50° apart and the lenses generate 60° beams. 89. The autonomous delivery vehicle of item 83, wherein the short-range camera assembly includes an ultrasonic sensor mounted above the camera. 90. The autonomous delivery vehicle of item 83, wherein the short-range camera assembly is mounted in a central position on the front of the cargo platform. 91. The autonomous delivery vehicle described in item 83, further comprising at least one corner camera assembly mounted on at least one corner of the front of the cargo platform, the at least one corner camera assembly comprising an ultrasonic sensor, a corner camera, a second light, and a second liquid-cooled heat sink, the second liquid-cooled heat sink cooling the second light and the corner camera.92. The method of claim 22, wherein the historical data comprises surface data. 93. The method of claim 22, wherein the historical data comprises discontinuity data. The present invention provides, for example, the following. (Item 1) 1. An autonomous delivery vehicle, comprising: a power base including two powered front wheels, two powered rear wheels, and an energy storage device, the power base configured to move at a commanded speed and in a commanded direction to effect transportation of at least one object; a cargo platform including a plurality of short-range sensors, said cargo platform mechanically attached to said power base; a cargo container with a volume for receiving the at least one object, the cargo container mounted on the cargo platform; and a long-range sensor suite comprising a LIDAR and one or more cameras, the long-range sensor suite being mounted on the cargo container; a controller for receiving data from the long-range sensor suite and the plurality of short-range sensors, the controller determining the commanded speed and the commanded direction based at least on the data, the controller providing the commanded speed and the commanded direction to the power base to complete the transport; and An autonomous delivery vehicle comprising: (Item 2) 2. The autonomous delivery vehicle of claim 1, wherein the data from the plurality of short-range sensors comprises at least one characteristic of a surface over which the power base travels. (Item 3) Item 1. The autonomous delivery vehicle of item 1, wherein the plurality of short-range sensors comprises at least one stereo camera. (Item 4) Item 1, wherein the plurality of short-range sensors comprises at least one IR projector, at least one image sensor, and at least one RGB sensor. (Item 5) Item 1 , the autonomous delivery vehicle comprising: a plurality of short-range sensors; and a radar sensor. (Item 6) Item 1. The autonomous delivery vehicle of item 1, wherein the data from the plurality of short-range sensors comprises RGB-D data. (Item 7) Item 1, an autonomous delivery vehicle according to item 1, wherein the controller determines the geometry of the road surface based on the RGB-D data received from the plurality of short-range sensors. (Item 8) 2. The autonomous delivery vehicle of item 1, wherein the plurality of short-range sensors detect objects within 4 meters of the AV and the long-range sensor suite detects objects more than 4 meters from the autonomous delivery vehicle. (Item 9) Item 1, wherein the plurality of short-range sensors are equipped with a cooling circuit. (Item 10) Item 1 , the autonomous delivery vehicle comprising: ultrasonic sensors. (Item 11) The controller executable code, the executable code comprising: accessing a map, the map being formed by a map processor, the map processor comprising: a first processor that accesses point cloud data from the long-range sensor suite, the point cloud data representing the surface; and a filter for filtering the point cloud data; a second processor for forming a processable portion from the filtered point cloud data; a third processor for merging the processable portions into at least one polygon; a fourth processor that locates and labels the at least one substantially discontinuous surface feature (SDSF) within the at least one polygon, if present, wherein the locating and labeling forms labeled point cloud data; and a fifth processor that creates a graphed polygon from the labeled point cloud data; a sixth processor that determines a path from a start point to an end point based at least on the graphed polygon, the AV traversing the at least one SDSF along the path; Item 3. The autonomous delivery vehicle of item 2, comprising: (Item 12) The filter is conditionally removing points representing transient objects and points representing outliers from the point cloud data; replacing the removed points with a preselected height; and Item 12. The autonomous delivery vehicle of item 11, comprising a seventh processor that executes code including: (Item 13) The second processor Segmenting the point cloud data into the processable portions; removing points of a preselected height from said addressable portion; Item 12. The autonomous delivery vehicle of item 11, comprising the executable code comprising: (Item 14) The third processor reducing the size of the processable portion by analyzing outliers, voxels, and normals; Enlarging an area from the reduced size addressable portion; determining an initial drivable surface from the expanded region; and segmenting and meshing the initial drivable surface; Locating polygons within the segmented and meshed initial drivable surface; establishing at least one drivable surface based at least on said polygon; Item 12. The autonomous delivery vehicle of item 11, comprising the executable code comprising: (Item 15) The fourth processor sorting the initial drivable surface point cloud data according to an SDSF filter, the SDSF filter including at least three categories of points; locating at least one SDSF point based on whether the at least three categories of points, in combination, satisfy at least one first preselected criterion; and Item 15. The autonomous delivery vehicle of item 14, comprising the executable code comprising: (Item 16) Item 16. The autonomous delivery vehicle of item 15, wherein the fourth processor includes the executable code to create at least one SDSF trajectory based on whether a plurality of the at least one SDSF point, in combination, satisfies at least one second preselected criterion. (Item 17) Creating a graphing polygon is creating at least one polygon from the at least one drivable surface, the at least one polygon including an outer edge; smoothing the outer edge; forming a running margin based on the smoothed outer edge; adding the at least one SDSF track to the at least one drivable surface; removing an inner edge from the at least one drivable surface according to at least one third preselected criterion; and Item 15. The autonomous delivery vehicle of item 14, further comprising an eighth processor including the executable code, (Item 18) Item 18. The autonomous delivery vehicle of item 17, comprising a ninth processor including the executable code, wherein smoothing the outer edge includes trimming the outer edge outward to form an outward edge. (Item 19) Item 19. The autonomous delivery vehicle of item 18, further comprising a tenth processor including the executable code, wherein forming the smoothed outer edge running margin includes trimming the outward edge inward. (Item 20) The controller a subsystem for navigating at least one substantially discontinuous surface feature (SDSF) encountered by the autonomous delivery vehicle (AV), wherein the AV travels a path across a surface, the surface including the at least one SDSF, the path including a start point and an end point, the subsystem comprising: a first processor that accesses a route configuration, the route configuration including at least one graphed polygon including filtered point cloud data, the filtered point cloud data including labeled features, and the point cloud data including a drivable margin; a second processor for transforming the point cloud data into a global coordinate system; a third processor that determines a boundary of the at least one SDSF, the third processor creating an SDSF buffer of a preselected size around the boundary; a fourth processor that determines the at least one SDSF that may be traversed based on at least one SDSF traversal criterion; a fifth processor that creates an edge / weight graph based on at least the at least one SDSF crossing criterion, the transformed point cloud data, and the route morphology; a base controller that selects the path from the start point to the end point based at least on the edge / weight graph; Item 1. The autonomous delivery vehicle of item 1, comprising: (Item 21) The at least one SDSF cross-sectional criterion is: a preselected width of the at least one SDSF and a preselected smoothness of the at least one SDSF; a minimum entry distance and a minimum exit distance between the at least one SDSF and the AV, including a drivable surface; the minimum approach distance between the at least one SDSF and the AV accommodating an approximately 90° approach to the at least one SDSF by the AV; Item 21. The autonomous delivery vehicle of item 20, comprising: (Item 22) 1. A method for managing a global occupancy grid for an autonomous device, the global occupancy grid including global occupancy grid cells, the global occupancy grid cells associated with occupancy probabilities, the method comprising: receiving sensor data from a sensor associated with the autonomous device; generating a local occupancy grid based at least on the sensor data, the local occupancy grid having local occupancy grid cells; When the autonomous device moves from a first area to a second area, accessing historical data associated with the second area; and creating a static grid based at least on the historical data; moving the global occupancy grid to maintain the autonomous device at a center position of the global occupancy grid; updating the displaced global occupancy grid based on the static grid; marking at least one of the global occupied grid cells as unoccupied if at least one of the global occupied grid cells matches a location of the autonomous device; For each said locally occupied grid cell, calculating a position of the local occupancy grid cell on the global occupancy grid; accessing a first occupancy probability from the global occupancy grid cell at the location; accessing a second occupancy probability from the local occupied grid cell at the location; and calculating a new occupancy probability at the location on the global occupancy grid based on at least the first occupancy probability and the second occupancy probability; A method comprising: (Item 23) 23. The method of claim 22, further comprising range checking the new occupancy probabilities. (Item 24) The range checking step includes: If the new occupancy probability is <0, setting the new occupancy probability to 0; if the new occupancy probability is >1, setting the new occupancy probability to 1; Item 24. The method according to Item 23, comprising: (Item 25) 23. The method of claim 22, further comprising setting the global occupied grid cells to the new occupancy probabilities. (Item 26) 24. The method of claim 23, further comprising setting the global occupied grid cells to the range-checked new occupancy probability. (Item 27) 1. A method for creating and managing an occupancy grid, comprising: Transforming the sensor measurements into a reference frame associated with the device by a local occupancy grid creation node; creating a time-stamped measurement occupancy grid; publishing the time-stamped measured occupancy grid as a local occupancy grid; creating a plurality of local occupancy grids; creating a static occupancy grid based on surface characteristics in a repository, the surface characteristics being associated with the location of the device; moving a global occupancy grid associated with the device's location to maintain the device and the local occupancy grid substantially aligned with the global occupancy grid; adding information from the static occupancy grid to the global occupancy grid; marking an area in the global occupancy grid currently occupied by the device as unoccupied; For at least one cell in each local occupancy grid, determining a location of the at least one cell within the global occupancy grid; accessing a first value at the location; determining a second value at the location based on a relationship between the first value and a cell value for the at least one cell in the local occupancy grid; comparing the second value against a preselected probability range; if the probability value is within the preselected probability range, setting the global occupancy grid with the new value; A method comprising: (Item 28) 28. The method of claim 27, further comprising publishing the global occupancy grid. (Item 29) Item 28. The method of item 27, wherein the surface characteristics comprise a surface type and a surface discontinuity. (Item 30) 28. The method of claim 27, wherein the relationship comprises summing. (Item 31) 1. A system for creating and managing an occupancy grid, comprising: a plurality of local grid creation nodes that create at least one local occupancy grid, the at least one local occupancy grid being associated with a location of a device, the at least one local occupancy grid including at least one cell; a global occupancy grid manager that accesses the at least one local occupancy grid; Equipped with the global occupancy grid manager: creating a static occupancy grid based on surface characteristics in a repository, the surface characteristics being associated with the location of the device; moving a global occupancy grid associated with the location of the device to maintain the device and the at least one local occupancy grid substantially aligned with the global occupancy grid; adding information from the static occupancy grid to at least one global occupancy grid; marking an area in the global occupancy grid currently occupied by the device as unoccupied; For each of the at least one cell in each local occupancy grid, determining a location of the at least one cell within the global occupancy grid; accessing a first value at the location; determining a second value at the location based on a relationship between the first value and a cell value for the at least one cell in the local occupancy grid; comparing the second value against a preselected probability range; if the probability value is within the preselected probability range, setting the global occupancy grid with the new value; The system. (Item 32) 1. A method for updating a global occupancy grid, comprising: When an autonomous device moves to a new location, updating the global occupancy grid with information from a static grid associated with the new location; analyzing the surface at the new location; updating the surface if it is drivable and updating the global occupancy grid with the updated surface; updating the global occupancy grid with values ​​from a repository of static values, the static values ​​being associated with the new location; A method comprising: (Item 33) renewing the surface accessing a local occupancy grid associated with the new location; and For each cell in the local occupancy grid, accessing a local occupancy grid surface classification confidence value and a local occupancy grid surface classification; if the local occupancy grid surface classification is the same as the global surface classification in the global occupancy grid in the cell, adding the global surface classification confidence value in the global occupancy grid to the local occupancy grid surface classification confidence value to form a sum and using the sum to update the global occupancy grid at the cell; if the local occupancy grid surface classification is not identical to the global surface classification in the global occupancy grid within the cell, subtracting the local occupancy grid surface classification confidence value from the global surface classification confidence value in the global occupancy grid to form a difference and updating the global occupancy grid using the difference; if the difference is less than zero, updating the global occupancy grid with the local occupancy grid surface classification; Item 33. The method according to Item 32, comprising: (Item 34) updating the global occupancy grid with the values ​​from the repository of static values; For each cell in the local occupancy grid, accessing a log-odds value, which is a local occupancy grid probability that the cell from the local occupancy grid is an occupancy value; updating the log-odds values ​​in the global occupancy grid with the local occupancy grid log-odds values ​​in the cell; decreasing the log-odds that the cell is occupied in the local occupancy grid if a preselected certainty that the cell is unoccupied is met and if the autonomous device is traveling within a lane barrier and if a local occupancy grid surface classification indicates a drivable surface; increasing the logarithmic odds within the local occupancy grid if the autonomous device expects to encounter a relatively uniform surface and if the local occupancy grid surface classification indicates a relatively non-uniform surface; decreasing the logarithmic odds within the local occupancy grid if the autonomous device expects to encounter a relatively uniform surface and if the local occupancy grid surface classification indicates a relatively uniform surface; Item 33. The method according to Item 32, comprising: (Item 35) 1. A method for real-time control of a configuration of a device, the device including a chassis, at least four wheels, a first side of the chassis operatively coupled to at least one of the at least four wheels, and an opposing second side of the chassis operatively coupled to at least one of the at least four wheels, the method comprising: generating a map based on at least a prior surface feature and an occupancy grid, the map being generated in non-real-time, the map including at least one location, the at least one location associated with at least one surface feature, the at least one surface feature associated with at least one surface classification and at least one mode; determining a current surface characteristic as the device progresses; updating the occupancy grid in real time with the current surface features; determining, from the occupancy grid and the map, a path that the device may take to traverse the at least one surface feature; A method comprising: (Item 36) 1. A method for real-time control of a configuration of a device, the device including a chassis, at least four wheels, a first side of the chassis operatively coupled to at least one of the at least four wheels, and an opposing second side of the chassis operatively coupled to at least one of the at least four wheels, the method comprising: receiving environmental data; determining a surface type based at least on the environmental data; and determining a mode based on at least the surface type and a first configuration; determining a second configuration based on at least the mode and the surface type; determining a movement command based on at least the second configuration; controlling a configuration of the device by using the move command to change the device from the first configuration to the second configuration; A method comprising: (Item 37) Item 37. The method of item 36, wherein the environmental data comprises RGB-D image data. (Item 38) populating an occupancy grid based on at least the surface type and the mode; determining the movement command based at least on the occupancy grid; Item 37. The method of item 36, further comprising: (Item 39) Item 39. The method of item 38, wherein the occupancy grid comprises at least information based on data from at least one image sensor. (Item 40) Item 37. The method of item 36, wherein the environmental data comprises road surface morphology. (Item 41) Item 37. The method of item 36, wherein the configuration comprises two clustered pairs of the at least four wheels, a first pair of the two pairs being positioned on the first side and a second pair of the two pairs being positioned on the second side, the first pair including a first front wheel and a first rear wheel, and the second pair including a second front wheel and a second rear wheel. (Item 42) Item 42. The method of item 41, wherein controlling the configuration includes coordinated powering of the first pair and the second pair based at least on the environmental data. (Item 43) Item 42. The method of claim 41, wherein controlling the configuration includes transitioning from driving the at least four wheels and a pair of casters that are retracted, the pair of casters being operably coupled to the chassis, to driving two wheels with the clustered first pair and the clustered second pair rotated to lift the first front wheel and the second front wheel, and the device is resting on the first rear wheel, the second rear wheel, and the pair of casters. (Item 44) Item 42. The method of item 41, wherein controlling the configuration includes rotating a pair of clusters operatively coupled to a first two powered wheels on the first side and a second two powered wheels on the second side based on at least the environmental data. (Item 45) Item 37. The method of item 36, wherein the device further comprises a cargo container, the cargo container being mounted on the chassis, and the chassis controlling the height of the cargo container. (Item 46) Item 46. The method of item 45, wherein the height of the cargo container is based on at least the environmental data. (Item 47) 1. A system for real-time control of a configuration of a device, the device including a chassis, at least four wheels, a first side of the chassis, and an opposing second side of the chassis, the system comprising: a device processor that receives real-time environmental data surrounding the device, the device processor determining a surface type based at least on the environmental data, the device processor determining a mode based at least on the surface type and a first configuration, and the device processor determining a second configuration based at least on the mode and the surface type; a power base processor that determines a movement command based on at least the second configuration, the power base processor using the movement command to control a configuration of the device to change the device from the first configuration to the second configuration; A system comprising: (Item 48) Item 48. The system of item 47, wherein the environmental data comprises RGB-D image data. (Item 49) Item 48. The system of item 47, wherein the device processor populates an occupancy grid based on at least the surface type and the mode. (Item 50) 50. The system of claim 49, wherein the power base processor determines the movement commands based on at least the occupancy grid. (Item 51) Item 50. The system of item 49, wherein the occupancy grid comprises at least information based on data from at least one image sensor. (Item 52) Item 48. The system of item 47, wherein the environmental data comprises road surface morphology. (Item 53) Item 48. The system of item 47, wherein the configuration comprises two clustered pairs of the at least four wheels, a first pair of the two pairs positioned on the first side and a second pair of the two pairs positioned on the second side, the first pair having a first front wheel and a first rear wheel, and the second pair having a second front wheel and a second rear wheel. (Item 54) Item 54. The system of item 53, wherein controlling the configuration includes coordinated powering of the first pair and the second pair based at least on the environmental data. (Item 55) Item 54. The system of item 53, wherein controlling the configuration includes transitioning from driving the at least four wheels and a pair of casters that are retracted, the pair of casters being operably coupled to the chassis, to driving two wheels with the clustered first pair and the clustered second pair rotated to lift the first front wheel and the second front wheel, and the device is resting on the first rear wheel, the second rear wheel, and the pair of casters. (Item 56) 1. A method for maintaining a global occupancy grid, comprising: Locating a first position of the autonomous device; when the autonomous device moves to a second location, the second location is associated with the global occupancy grid and the local occupancy grid; updating the global occupancy grid with at least one occupancy probability value associated with the first location; updating the global occupancy grid with at least one drivable surface associated with the local occupancy grid; updating the global occupancy grid with a surface confidence associated with the at least one drivable surface; updating the global occupancy grid with log-odds of the at least one occupancy probability value using a first Bayes function; adjusting the log-odds based on a characteristic associated with at least the second location; when the autonomous device remains at the first location and the global occupancy grid and the local occupancy grid are co-located; updating the global occupancy grid using the at least one drivable surface associated with the local occupancy grid; updating the global occupancy grid with the surface confidence associated with the at least one drivable surface; updating the global occupancy grid with log-odds of the at least one occupancy probability value using a second Bayes function; adjusting the logarithmic odds based on a characteristic associated with at least the second location; and A method comprising: (Item 57) Creating the map comprises: accessing point cloud data representing the surface; filtering the point cloud data; forming the filtered point cloud data into a processable portion; merging the processable portions into at least one concave polygon; locating and labeling the at least one SDSF within the at least one concave polygon, wherein the locating and labeling forms labeled point cloud data; and generating a graphing polygon based at least on said at least one concave polygon; determining the path from a start point to an end point based at least on the graphed polygon, the AV traversing the at least one SDSF along the path; Item 36. The method according to Item 35, comprising: (Item 58) filtering the point cloud data conditionally removing points representing transient objects and points representing outliers from the point cloud data; replacing the removed points with a preselected height; and Item 57. The method according to Item 57, comprising: (Item 59) Forming the processing portion comprises: Segmenting the point cloud data into the processable portions; removing points of a preselected height from said addressable portion; Item 57. The method according to Item 57, comprising: (Item 60) Merging the processable portions comprises: reducing the size of the processable portion by analyzing outliers, voxels, and normals; Enlarging an area from the reduced size addressable portion; determining an initial drivable surface from the expanded region; and segmenting and meshing the initial drivable surface; Locating polygons within the segmented and meshed initial drivable surface; establishing at least one drivable surface based at least on said polygon; Item 57. The method according to Item 57, comprising: (Item 61) Locating and labeling the at least one SDSF comprises: sorting the initial drivable surface point cloud data according to an SDSF filter, the SDSF filter including at least three categories of points; locating at least one SDSF point based on whether the at least three categories of points, in combination, satisfy at least one first preselected criterion; and Item 61. The method according to Item 60, comprising: (Item 62) Item 62. The method of item 61, further comprising generating at least one SDSF trajectory based at least on whether a plurality of the at least one SDSF points, in combination, satisfy at least one second preselected criterion. (Item 63) Creating the graphing polygon further comprises: creating at least one polygon from the at least one drivable surface, the at least one polygon including an outer edge; smoothing the outer edge; forming a running margin based on the smoothed outer edge; adding the at least one SDSF track to the at least one drivable surface; removing an inner edge from the at least one drivable surface according to at least one third preselected criterion; and Item 63. The method according to Item 62, comprising: (Item 64) Item 64. The method of item 63, wherein smoothing the outer edge comprises trimming the outer edge outward to form an outward edge. (Item 65) Item 64. The method of item 63, wherein forming the smoothed outer edge running margin includes trimming the outwardly facing edge inward. (Item 66) 1. An autonomous delivery vehicle, comprising: a power base including two powered front wheels, two powered rear wheels, and an energy storage device, the power base configured to move at a commanded speed; a cargo platform including a plurality of short-range sensors, said cargo platform mechanically attached to said power base; a cargo container with a volume for receiving one or more objects to be delivered, said cargo container mounted on said cargo platform; a long-range sensor suite comprising a LIDAR and one or more cameras, the long-range sensor suite being mounted on the cargo container; a controller for receiving data from the long-range sensor suite and the plurality of short-range sensors; An autonomous delivery vehicle comprising: (Item 67) Item 67. The autonomous delivery vehicle of item 66, wherein the plurality of short-range sensors detect at least one characteristic of a drivable surface. (Item 68) Item 67. The autonomous delivery vehicle of item 66, wherein the plurality of short-range sensors are stereo cameras. (Item 69) Item 67. The autonomous delivery vehicle of item 66, wherein the multiple short-range sensors comprise an IR projector, two image sensors, and an RGB sensor. (Item 70) Item 67. The autonomous delivery vehicle of item 66, wherein the plurality of short-range sensors are radar sensors. (Item 71) Item 67. The autonomous delivery vehicle of item 66, wherein the short-range sensor provides RGB-D data to the controller. (Item 72) Item 67. The autonomous delivery vehicle of item 66, wherein the controller determines the geometry of the road surface based on the RGB-D data received from the plurality of short-range sensors. (Item 73) Item 67. The autonomous delivery vehicle of item 66, wherein the plurality of short-range sensors detect objects within 4 meters of the autonomous delivery vehicle and the long-range sensor suite detects objects more than 4 meters from the autonomous delivery vehicle. (Item 74) 1. An autonomous delivery vehicle, comprising: a power base including at least two powered rear wheels, a caster front wheel, and an energy storage device, the power base configured to move at a commanded speed; a cargo platform including a plurality of short-range sensors, said cargo platform mechanically attached to said power base; a cargo container with a volume for receiving one or more objects to be delivered, said cargo container mounted on said cargo platform; a long-range sensor suite comprising a LIDAR and one or more cameras, the long-range sensor suite being mounted on the cargo container; a controller for receiving data from the long-range sensor suite and the plurality of short-range sensors; An autonomous delivery vehicle comprising: (Item 75) Item 75. The autonomous delivery vehicle of item 74, wherein the plurality of short-range sensors detect at least one characteristic of a drivable surface. (Item 76) Item 75. The autonomous delivery vehicle of item 74, wherein the plurality of short-range sensors are stereo cameras. (Item 77) Item 75. The autonomous delivery vehicle of item 74, wherein the plurality of short-range sensors comprises an IR projector, two image sensors, and an RGB sensor. (Item 78) Item 75. The autonomous delivery vehicle of item 74, wherein the plurality of short-range sensors are radar sensors. (Item 79) Item 75. The autonomous delivery vehicle of item 74, wherein the short-range sensor provides RGB-D data to the controller. (Item 80) Item 75. The autonomous delivery vehicle of item 74, wherein the controller determines the geometry of the road surface based on the RGB-D data received from the plurality of short-range sensors. (Item 81) Item 75. The autonomous delivery vehicle of item 74, wherein the plurality of short-range sensors detect objects within 4 meters of the autonomous delivery vehicle and the long-range sensor suite detects objects more than 4 meters from the autonomous delivery vehicle. (Item 82) Item 75. The autonomous delivery vehicle of item 74, further comprising a second set of powered wheels that can engage the ground while the caster wheels are lifted off the ground. (Item 83) 1. An autonomous delivery vehicle, comprising: a power base including at least two powered rear wheels, a caster front wheel, and an energy storage device, the power base configured to move at a commanded speed; a cargo platform, the cargo platform mechanically attached to the power base; a short-range camera assembly mounted on said cargo platform for detecting at least one characteristic of a drivable surface; Equipped with The short-range camera assembly A camera and The first light and a first liquid-cooled heat sink; Equipped with The first liquid-cooled heat sink cools the first light and the camera, the autonomous delivery vehicle. (Item 84) Item 84. The autonomous delivery vehicle of item 83, wherein the short-range camera assembly further comprises a thermoelectric cooler between the camera and the liquid-cooled heat sink. (Item 85) Item 84. The autonomous delivery vehicle of item 83, wherein the first light and the camera are embedded in a cover with an opening that deflects illumination from the first light away from the camera. (Item 86) Item 84. The autonomous delivery vehicle of item 83, wherein the lights are angled downward at least 15° and recessed at least 4 mm into the cover to minimize distracting illumination to pedestrians. (Item 87) Item 84. The autonomous delivery vehicle of item 83, wherein the camera has a field of view and the first light comprises two LEDs with lenses for generating two beams of light that are diffused to illuminate the field of view of the camera. (Item 88) Item 88. The autonomous delivery vehicle of item 87, wherein the lights are angled approximately 50 degrees apart and the lenses produce a 60 degree beam. (Item 89) Item 84. The autonomous delivery vehicle of item 83, wherein the short-range camera assembly includes an ultrasonic sensor mounted above the camera. (Item 90) Item 84. The autonomous delivery vehicle of item 83, wherein the short-range camera assembly is mounted in a central position on the front of the cargo platform. (Item 91) and at least one corner camera assembly mounted on at least one corner of a front of the cargo platform, the at least one corner camera assembly comprising: an ultrasonic sensor; Corner cameras and The second light and a second liquid-cooled heat sink that cools the second light and the corner camera; and Item 84. The autonomous delivery vehicle of item 83, comprising: (Item 92) Item 23. The method of item 22, wherein the historical data comprises surface data. (Item 93) 23. The method of claim 22, wherein the historical data comprises discontinuity data. [Brief explanation of the drawings]

[0047] The present teachings may be more readily understood by reference to the following description taken in conjunction with the accompanying drawings.

[0048] [Figure 1-1] FIG. 1-1 is a schematic block diagram of the major components of the system of the present teachings.

[0049] [Figure 1-2] 1-2 is a schematic block diagram of the major components of the map processor of the present teachings.

[0050] [Figure 1-3] 1-3 are schematic block diagrams of the major components of the perceptual processor of the present teachings.

[0051] [Figure 1-4] 1-4 are schematic block diagrams of the major components of an autonomic processor of the present teachings.

[0052] [Figure 1A] FIG. 1A is a schematic block diagram of a system of the present teachings for preparing a travel path for an AV.

[0053] [Figure 1B] FIG. 1B is a pictorial diagram of an exemplary configuration of a device incorporating a system of the present teachings.

[0054] [Figure 1C] FIG. 1C is a side view of an automated delivery vehicle showing the fields of view of several long-range and short-range sensors.

[0055] [Figure 1D] FIG. 1D is a schematic block diagram of a map processor of the present teachings.

[0056] [Figure 1E] FIG. 1E is a pictorial diagram of a first portion of the map processor flow of the present teachings.

[0057] [Figure 1F] FIG. 1F is an image of a segmented point cloud of the present teachings.

[0058] [Figure 1G] FIG. 1G is a pictorial diagram of a second portion of the map processor of the present teachings.

[0059] [Figure 1H] FIG. 1H is an image of the drivable surface detection results of the present teachings.

[0060] [Figure 1I] FIG. 1I is a pictorial diagram of the SDSF detector flow of the present teachings.

[0061] [Figure 1J] FIG. 1J is a pictorial representation of the SDSF categories of the present teachings.

[0062] [Figure 1K] FIG. 1K is an image of an SDSF identified by the system of the present teachings.

[0063] [Figure 1L] 1L and 1M are pictorial illustrations of the polygon processing of the present teachings. [Figure 1M] 1L and 1M are pictorial illustrations of the polygon processing of the present teachings.

[0064] [Figure 1N] FIG. 1N is an image of a polygon and SDSF identified by the system of the present teachings.

[0065] [Figure 2A] FIG. 2A is an isometric view of an autonomous vehicle of the present teachings.

[0066] [Figure 2B] FIG. 2B is a top view of a cargo container showing the field of view of selected long-range sensors.

[0067] [Figure 2C] 2C-2F are diagrams of a long-range sensor assembly. [Figure 2D] 2C-2F are diagrams of a long-range sensor assembly. [Figure 2E] 2C-2F are diagrams of a long-range sensor assembly. [Figure 2F] 2C-2F are diagrams of a long-range sensor assembly.

[0068] [Figure 2G] FIG. 2G is a top view of a cargo container showing the field of view of selected short-range sensors.

[0069] [Figure 2H] FIG. 2H is an isometric view of a cargo platform of the present teachings.

[0070] [Figure 2I] 2I-2L are isometric views of the short-range sensor. [Figure 2J] 2I-2L are isometric views of the short-range sensor. [Figure 2K] 2I-2L are isometric views of the short-range sensor. [Figure 2L] 2I-2L are isometric views of the short-range sensor.

[0071] [Figure 2M] 2M-2N are isometric views of an autonomous vehicle of the present teachings. [Figure 2N] 2M-2N are isometric views of an autonomous vehicle of the present teachings.

[0072] [Figure 2O] 2O-2P are isometric views of an autonomous vehicle of the present teachings with the skin panels removed. [Figure 2P] 2O-2P are isometric views of an autonomous vehicle of the present teachings with the skin panels removed.

[0073] [Figure 2Q] FIG. 2Q is an isometric view of an autonomous vehicle of the present teachings with a portion of the top panel removed.

[0074] [Figure 2R] 2R-2V are diagrams of long-range sensors on an autonomous vehicle of the present teachings. [Figure 2S] 2R-2V are diagrams of long-range sensors on an autonomous vehicle of the present teachings. [Figure 2T] 2R-2V are diagrams of long-range sensors on an autonomous vehicle of the present teachings. [Figure 2U] 2R-2V are diagrams of long-range sensors on an autonomous vehicle of the present teachings. [Figure 2V] 2R-2V are diagrams of long-range sensors on an autonomous vehicle of the present teachings.

[0075] [Figure 2W] 2W-2Z are diagrams of ultrasonic sensors. [Figure 2X] 2W-2Z are diagrams of ultrasonic sensors. [Figure 2Y] 2W-2Z are diagrams of ultrasonic sensors. [Figure 2Z] 2W-2Z are diagrams of ultrasonic sensors.

[0076] [Figure 2AA] 2AA-2BB are diagrams of the central short-range camera assembly. [Figure 2BB] 2AA-2BB are diagrams of the central short-range camera assembly.

[0077] [Figure 3A] FIG. 3A is a schematic block diagram of a system in one configuration of the present teachings.

[0078] [Figure 3B] FIG. 3B is a schematic block diagram of another configuration of a system of the present teachings.

[0079] [Figure 3C] FIG. 3C is a schematic block diagram of a system of the present teachings that can initially create a global occupancy grid.

[0080] [Figure 3D] FIG. 3D is a pictorial representation of a static grid of the present teachings.

[0081] [Figure 3E] 3E and 3F are pictorial representations of the creation of an occupancy grid of the present teachings. [Figure 3F] 3E and 3F are pictorial representations of the creation of an occupancy grid of the present teachings.

[0082] [Figure 3G]FIG. 3G is a pictorial representation of a priori occupancy grid of the present teachings.

[0083] [Figure 3H] FIG. 3H is a pictorial representation of updating the global occupancy grid of the present teachings.

[0084] [Figure 3I] FIG. 3I is a flow diagram of a method of the present teachings for publishing a global occupancy grid.

[0085] [Figure 3J] FIG. 3J is a flow diagram of a method of the present teachings for updating the global occupancy grid.

[0086] [Figure 3K] 3K-3M are flow diagrams of another method of the present teachings for updating the global occupancy grid. [Figure 3L] 3K-3M are flow diagrams of another method of the present teachings for updating the global occupancy grid. [Figure 3M] 3K-3M are flow diagrams of another method of the present teachings for updating the global occupancy grid.

[0087] [Figure 4A] FIG. 4A is a perspective pictorial view of a device of the present teachings mounted in various modes.

[0088] [Figure 4B] FIG. 4B is a schematic block diagram of a system of the present teachings.

[0089] [Figure 4C] FIG. 4C is a schematic block diagram of the driving surface processor component of the present teachings.

[0090] [Figure 4D] FIG. 4D is a schematic block / pictorial flow diagram of the process of the present teachings.

[0091] [Figure 4E] 4E and 4F are perspective and side views, respectively, of a configuration of a device of the present teachings in a standard mode. [Figure 4F] 4E and 4F are perspective and side views, respectively, of a configuration of a device of the present teachings in a standard mode.

[0092] [Figure 4G] 4G and H are perspective and side views, respectively, of a configuration of a device of the present teachings in four-wheel mode. [Figure 4H] 4G and H are perspective and side views, respectively, of a configuration of a device of the present teachings in four-wheel mode.

[0093] [Figure 4I] 4I and 4J are perspective and side views, respectively, of a configuration of a device of the present teachings in an elevated four-wheel mode. [Figure 4J] 4I and 4J are perspective and side views, respectively, of a configuration of a device of the present teachings in an elevated four-wheel mode.

[0094] [Figure 4K] FIG. 4K is a flowchart of a method of the present teachings.

[0095] [Figure 5A] FIG. 5A is a schematic block diagram of a device controller of the present teachings.

[0096] [Figure 5B] FIG. 5B is a schematic block diagram of an SDSF processor of the present teachings.

[0097] [Figure 5C] FIG. 5C is an image of the SDSF approach identified by the system of the present teachings.

[0098] [Figure 5D]FIG. 5D is an image of the route morphology produced by the system of the present teachings.

[0099] [Figure 5E] FIG. 5E is a schematic block diagram of a mode of the present teachings.

[0100] [Figure 5F] 5F-5J are flowcharts of methods of the present teachings for traversing an SDSF. [Figure 5G] 5F-5J are flowcharts of methods of the present teachings for traversing an SDSF. [Figure 5H] 5F-5J are flowcharts of methods of the present teachings for traversing an SDSF. [Figure 5I] 5F-5J are flowcharts of methods of the present teachings for traversing an SDSF. [Figure 5J] 5F-5J are flowcharts of methods of the present teachings for traversing an SDSF.

[0101] [Figure 5K] FIG. 5K is a schematic block diagram of a system of the present teachings for traversing an SDSF.

[0102] [Figure 5L] 5L-5N are pictorial representations of the method of FIGS. 5F-5H. [Figure 5M] 5L-5N are pictorial representations of the method of FIGS. 5F-5H. [Figure 5N] 5L-5N are pictorial representations of the method of FIGS. 5F-5H.

[0103] [Figure 5O] FIG. 5O is a pictorial representation of converting an image into a polygon. DETAILED DESCRIPTION OF THE INVENTION

[0104] (Detailed explanation) The systems and methods of the present teachings can use onboard sensors and previously developed maps to develop an occupancy grid and use these aids to navigate the AV across surface features, including reconfiguring the AV based on surface type and previous.

[0105] Referring now to FIG. 1-1 , an AV system 100 can include a structure on which sensors 10701 can be mounted and in which a device controller 10111 can execute. The structure can include a power base 10112 that can direct the movement of wheels that are part of the structure and enable movement of the AV. The device controller 10111 can execute on at least one processor located on the AV and can receive data from sensors 10701, which can be located on the AV, including, but not limited to, sensors 10701. The device controller 10111 can provide speed, direction, and configuration information to a base controller 10114, which can provide movement commands to the power base 10112. The device controller 10111 can receive map information from a map processor 10104, which can prepare a map of an area surrounding the AV. The device controller 10111 can include a sensor processor 10703 that can receive and process input from sensors 10701, including, but not limited to, sensors on the AV. In some configurations, the device controller 10111 can include a perception processor 2143, an autonomy processor 2145, and a driver processor 2127. The perception processor 2143 can, for example, without limitation, locate static and dynamic obstacles, determine traffic signal conditions, create occupancy grids, and classify surfaces. The autonomy processor 2145 can, for example, without limitation, determine the maximum speed of the AV and determine the type of situation in which the AV is navigating, e.g., on a road, on a sidewalk, at an intersection, and / or under remote control. The driver processor 2127 can, for example, without limitation, create commands according to the direction of the autonomy processor 2145 and send them on to the base controller 10114.

[0106] 1-2 , the map processor 10104 can create a map of surface features and provide the map, through the device controller 10111, to the perception processor 2143, which can update the occupancy grid. The map processor 10104 can include, among many other aspects, a feature extractor 10801, a point cloud organizer 10803, a transient processor 10805, a segmenter 10807, a polygon generator 10809, an SDSF line generator 10811, and a combiner 10813. The feature extractor 10801 can include a first processor that accesses point cloud data representing the surface. The point cloud organizer 10803 can include a second processor that forms processable portions from the filtered point cloud data. The transient processor 10805 can include a first filter that filters the point cloud data. The segmenter 10807 may include executable code that may include, but is not limited to, segmenting the point cloud data into processable portions and removing points of a preselected height from the processable portions. The first filter may optionally include executable code that may include, but is not limited to, conditionally removing points representing transient objects and points representing outliers from the point cloud data and replacing the removed points with a preselected height. The polygon generator 10809 may include a third processor that merges the processable portions into at least one concave polygon. The third processor may optionally include executable code that may include, but is not limited to, reducing the size of the processable portion by analyzing outliers, voxels, and normals, growing an area from the reduced-size processable portion, determining an initial drivable surface from the grown-up area, segmenting and meshing the initial drivable surface, locating polygons within the segmented and meshed initial drivable surface, and setting the drivable surface based at least on the polygons.The SDSF line generator 10811 may include a fourth processor that locates and labels at least one SDSF within at least one concave polygon, where the locating and labeling may form labeled point cloud data. The fourth processor may optionally include executable code that may include, without limitation, sorting the drivable surface point cloud data according to an SDSF filter, where the SDSF filter includes at least three categories of points, and locating at least one SDSF point based at least on whether the categories of points, in combination, satisfy at least one first preselected criterion. The combiner 10813 may include a fifth processor that creates a graphing polygon. Creating a graphed polygon may optionally include executable code that may include, but is not limited to, creating at least one polygon from at least one drivable surface, where the at least one polygon includes an edge, smoothing the edge, forming a driving margin based on the smoothed edge, adding at least one SDSF trajectory to the at least one drivable surface, and removing the edge from the at least one drivable surface according to at least one third preselected criterion. Smoothing the edge may optionally include executable code that may include, but is not limited to, trimming the edge outward. Forming a running margin of the smoothed edge may optionally include executable code that may include, but is not limited to, trimming the outward edge inward.

[0107] 1-3, a map can be provided to the AV, which can include onboard sensors, powered wheels, and a processor for receiving sensor and map data, using that data, and power-configuring the AV to traverse various types of surfaces, among other things, when delivering goods, for example. The onboard sensors can capture an occupancy grid and can provide data that can be used to detect dynamic obstacles. The occupancy grid can also be captured by the map. The device controller 10111 can include a perception processor 2143 that can receive and process the sensor and map data and use that data to update the occupancy grid.

[0108] 1-4, the device controller 10111 can include a configuration processor 41023 that can automatically determine a configuration for the AV based on at least the mode of the AV and the surface characteristics encountered. The autonomy processor 2145 can include a control processor 40325 that can determine the type of surface that needs to be traversed and the configuration the AV needs to take to traverse the surface based on at least the map (planned route to be followed), information from the configuration processor 41023, and the mode of the AV. The autonomy processor 2145 can provide commands to the motor drive processor 40326 to implement the commands.

[0109] 1A , the map processor 10104 can enable a device, such as, but not limited to, an AV or semi-autonomous device, to navigate within an environment that may include features such as SDSFs. The features in the map, along with onboard sensors, can enable the AV to navigate over various surfaces. In particular, the SDSFs can be precisely identified and labeled so that the AV can automatically maintain AV performance during SDSF entry and exit, and AV speed, configuration, and direction can be controlled for safe SDSF traversal.

[0110] Continuing with reference to FIG. 1A , in some configurations, the system 100 for managing traversal of an SDSF can include an AV 10101, a core cloud infrastructure 10103, an AV service 10105, a device controller 10111, sensors 10701, and a power base 10112. The AV 10101 can provide transportation and shuttle services from origins to destinations, following dynamically determined routes, for example, but not limited to, as modified by incoming sensor information. The AV 10101 can include, but is not limited to, devices with autonomous modes, devices that can operate fully autonomously, devices that can be operated at least partially remotely, and devices that can include combinations of these features. The transportation device service 10105 can provide drivable surface information, including features, to the device controller 10111. The device controller 10111 can modify the drivable surface information according to at least, for example, but not limited to, the incoming sensor information and feature traversal requirements, and can plan a route for the AV 10101 based on the modified drivable surface information. The device controller 10111 can submit commands to the power base 10112, which instructs the power base 10112 to provide speed, direction, and configuration commands to the wheel motors and cluster motors, causing the AV 10101 to follow a selected route and raise and lower its cargo accordingly. The transport device services 10105 can access route-related information from a core cloud infrastructure 10103, which may include, but is not limited to, storage and content distribution facilities. In some configurations, the core cloud infrastructure 10103 may be hosted by, for example, but not limited to, AMAZON WEB SERVICES®, GOOGLE CLOUD TM , and commercial products such as ORACLE CLOUD®.

[0111] 1B , an exemplary AV that may include a device controller 10111 (FIG. 1A) that may receive information from the map processor 10104 (FIG. 1A) of the present teachings may include a power base assembly, such as, for example, but not limited to, a power base, fully described in U.S. Patent Application No. 16 / 035,205, filed July 13, 2018, entitled “Mobility Device,” or U.S. Patent No. 6,571,892, filed August 15, 2001, entitled “Control System and Method,” both of which are incorporated herein by reference in their entireties. The exemplary power base assembly is described herein not to limit the present teachings, but instead to clarify features of any power base assembly that may be useful in implementing the techniques of the present teachings. The exemplary power base assembly may optionally include a power base 10112, a wheel cluster assembly 11100, and a payload carrier height assembly 10068. The exemplary power base assembly can optionally provide electrical as well as mechanical power to drive the wheels 11203 and the cluster 11100, which can raise and lower the wheels 11203. The power base 10112 can control the rotation of the cluster assembly 11100 and the elevation and lowering of the payload carrier height assembly 10068 to support the substantially discontinuous surface traversal of the present teachings. Other such devices can also be used to accommodate SDSF detection and traversal of the present teachings.

[0112] Referring again to FIG. 1A , in some configurations, sensors internal to the exemplary power base can detect the orientation and rate of change of the AV10101, motors can enable servo operation, and a controller can interpret information from the internal sensors and motors. Appropriate motor commands can be calculated to achieve transport performance and implement path-following commands. Left and right wheel motors can drive wheels on both sides of the AV10101. In some configurations, the front and rear wheels can be coupled to drive together, such that the two left wheels can drive together and the two right wheels can drive together. In some configurations, turning can be accomplished by driving the left and right motors at different rates, and a cluster motor can rotate the wheelbase in the forward / rearward direction. This can allow the AV10101 to remain level while the front wheels are higher or lower than the rear wheels. This feature can be useful, for example, but not limited to, when ascending or descending an SDSF. The payload carrier 10173 can be automatically raised and lowered based at least on the underlying terrain.

[0113] Continuing with reference to FIG. 1A , in some configurations, the point cloud data may include route information regarding the area through which the AV 10101 should travel. Possibly collected by a mapping device similar to or the same as the AV 10101, the point cloud data may be time-tagged. The path along which the mapping device travels may be referred to as a mapped trajectory. Point cloud data processing as described herein may occur as the mapping device traverses the mapped trajectory or after point cloud data collection is complete. After the point cloud data are collected, they may undergo point cloud data processing, which may include initial filtering and point reduction, point cloud segmentation, and feature detection as described herein. In some configurations, the core cloud infrastructure 10103 may provide long-term or short-term storage for the collected point cloud data and may provide the data to the AV service 10105. The AV service 10105 can select from among potential point cloud datasets to find one that covers an area surrounding a desired starting point for the AV 10101 and a desired destination for the AV 10101. The AV service 10105 can include, without limitation, a map processor 10104 that can reduce the size of the point cloud data and determine features represented within the point cloud data. In some configurations, the map processor 10104 can determine the location of an SDSF from the point cloud data. In some configurations, polygons can be created from the point cloud data as a technique for segmenting the point cloud data and ultimately establishing a drivable surface. In some configurations, SDSF detection and drivable surface determination can proceed in parallel. In some configurations, SDSF detection and drivable surface determination can proceed sequentially.

[0114] Referring now to FIG. 1C , in some configurations, the AV may be configured to deliver cargo and / or perform other functions involving autonomously navigating to a desired location. In some applications, the AV may be remotely guided. In some configurations, the AV 20100 comprises a cargo container that can be opened automatically, remotely, in response to user input, or manually, allowing a user to place or remove luggage and other items. The cargo container 20110 is mounted on a cargo platform 20160, which is mechanically connected to a power base 20170. The power base 20170 includes four powered wheels 20174 and two caster wheels 20176. The power base provides speed and directional control for moving the cargo container 20110 along the ground and over obstacles, including curbs and other discontinuous surface features.

[0115] 1C , the cargo platform 20160 is connected to the power base 20170 through two U-frames 20162. Each U-frame 20162 is rigidly attached to the structure of the cargo platform 20160 and includes two holes that allow a rotatable joint 20164 to be formed with the end of each arm 20172 on the power base 20170. The power base controls the rotational position of the arms, and therefore the height and tilt angle of the cargo container 20110.

[0116] Continuing with reference to FIG. 1C, in some configurations, the AV20100 includes one or more processors for receiving data, navigating a route, and selecting a direction and speed for the power base 20170.

[0117] 1D , in some configurations, a map processor 10104 of the present teachings can locate the SDSF on a map. The map processor 10104 can include, but is not limited to, a feature extractor 10801, a point cloud organizer 10803, a transient processor 10805, a segmenter 10807, a polygon generator 10809, an SDSF line generator 10811, and a data combiner 10813.

[0118] 1D , the feature extractor 10801 (FIG. 1-2) can include, but is not limited to, line-of-sight filtering 10121 of the point cloud data 10131 and the mapped trajectory 10133. Line-of-sight filtering can remove points that are hidden from the direct line-of-sight of the sensors that collect the point cloud data and form the mapped trajectory. The point cloud organizer 10803 (FIG. 1-2) can organize 10151 the reduced point cloud data 10132, possibly according to preselected criteria associated with specific features. In some configurations, the transient processor 10805 (FIG. 1-2) can remove 10153 transient points from the organized point cloud data and the mapped trajectory 10133 by any number of methods, including those described herein. Transient points can complicate processing, especially if the specific features are stationary. The segmenter 10807 (FIGS. 1-2) can divide the processed point cloud data 10135 into manageable chunks. In some configurations, the processed point cloud data 10135 can be segmented (10155) into sections having a preselected minimum number of points, for example, but not limited to, about 100,000 points. In some configurations, further point reduction can be based on preselected criteria that can be related to the features to be extracted. For example, if points above a certain height are not important for locating features, those points can be deleted from the point cloud data. In some configurations, the height of at least one of the sensors collecting the point cloud data can be considered an origin, and points above the origin can be removed from the point cloud data, for example, when only points of interest are associated with surface features. After the filtered point cloud data 10135 is segmented, forming segments 10137, the remaining points can be divided into drivable surface sections and surface features can be located. In some configurations, a polygon generator 10809 (FIGS. 1-2) can locate the drivable surface by generating (10161) a polygon 10139, for example, but not by way of limitation, as described herein.In some configurations, the SDSF line generator 10811 (FIG. 1-2) can locate surface features by generating (10163) an SDSF line 10141, for example, without limitation, as described herein. In some configurations, the combiner 10813 (FIG. 1-2) can combine (10165) the polygon 10139 and the SDSF 10141 to create a data set that can be further processed to generate an actual path along which the AV 10101 (FIG. 1A) may travel.

[0119] 1E , eliminating 10153 ( FIG. 1D ) objects that are transient with respect to mapped trajectory 10133, such as exemplary time-stamped point 10751, from point cloud data 10131 ( FIG. 1D ) may include casting ray 10753 from the time-stamped point on mapped trajectory 10133 to each time-stamped point in point cloud data 10131 ( FIG. 1D ) that has substantially the same timestamp. If ray 10753 intersects a point between the time-stamped point on mapped trajectory 10133 and the end point of ray 10753, e.g., point D 10755, then intersection point D 10755 can be assumed to have entered the point cloud data during a different sweep of the camera. The intersection point, e.g., intersection point D 10755, can be assumed to be part of a transient object and can be removed from reduced point cloud data 10132 ( FIG. 1D ) as not representing a fixed feature, such as an SDSF. The result is, for example, but not limitation, processed point cloud data 10135 (FIG. 1D) that is free of transient objects. Points that were removed as part of transient objects but are also substantially at ground level can be returned (10754) to the processed point cloud data 10135 (FIG. 1D). Transient objects may not include certain features, such as, for example, but not limitation, SDSF10141 (FIG. 1D), and therefore can be removed without interfering with the integrity of the point cloud data 10131 (FIG. 1D) when SDSF10141 (FIG. 1D) is the feature being detected.

[0120] Continuing with reference to FIG. 1E, segmenting 10155 (FIG. 1D) the processed point cloud data 10135 (FIG. 1D) can generate sections 10757 having a preselected size and shape, for example, but not limited to, a rectangle 10154 (FIG. 1F) having a minimum preselected side length and containing approximately 100,000 points. From each section 10757, points not necessarily relevant to the specific task, for example, but not limited to, points located above a preselected level, can be removed (10157) (FIG. 1D) to reduce the dataset size. In some configurations, the preselected level can be the height of AV 10101 (FIG. 1A). Removing these points can lead to more efficient processing of the dataset.

[0121] Referring again primarily to FIG. 1D , the map processor 10104 can provide the device controller 10111 with at least one dataset that can be used to generate direction, speed, and configuration commands for controlling the AV 10101 (FIG. 1A). The at least one dataset can include points that can be connected to other points in the dataset, and each line connecting points in the dataset traverses the drivable surface. To determine such route points, the segmented point cloud data 10137 can be divided into polygons 10139, and the vertices of the polygons 10139 can potentially become route points. The polygons 10139 can include features such as, for example, SDSFs 10141.

[0122] 1D , in some configurations, creating the processed point cloud data 10135 can include filtering voxels. To reduce the number of points that will undergo future processing, in some configurations, the centroid of each voxel in the dataset can be used to approximate the points within the voxel, and all points except the centroid can be excluded from the point cloud data. In some configurations, the center of the voxel can be used to approximate the points within the voxel. Other methods for reducing the size of the filtered segment 10251 ( FIG. 1G ) can also be used, such as, for example, but not limited to, random point subsampling, such that a fixed number of uniformly randomly selected points can be excluded from the filtered segment 10251 ( FIG. 1G ).

[0123] 1D , in some configurations, creating processed point cloud data 10135 can include calculating normals from a dataset from which outliers have been removed and which has been reduced in size through voxel filtering. The normals for each point in the filtered dataset can be used for various processing possibilities, including curve reconstruction algorithms. In some configurations, estimating and filtering normals in the dataset can include obtaining an underlying surface from the dataset using a surface meshing technique and calculating normals from the surface mesh. In some configurations, estimating normals can include using approximations to infer surface normals directly from the dataset, such as, for example, but not limited to, determining normals to a fitting plane obtained by applying a total least squares method to k nearest neighbors of a point. In some configurations, the value of k can be selected based at least on empirical data. Filtering normals can include removing any normals that are more than approximately 45° from perpendicular to the xy plane. In some configurations, a filter can be used to align normals to the same direction. If a portion of the dataset represents a planar surface, redundant information contained in neighboring normals can be filtered out either by performing random subsampling or by filtering out one point from the set of associated points. In some configurations, selecting a point can include recursively decomposing the dataset into boxes until each box contains at most k points. A single normal can be calculated from the k points in each box.

[0124] Continuing with reference to FIG. 1D , in some configurations, creating the processed point cloud data 10135 can include growing regions within the dataset by clustering points that geometrically fit a surface representing the dataset and refining the surface as the region grows to obtain the best approximation for the maximum number of points. Region growing can merge points in terms of a smoothness constraint. In some configurations, the smoothness constraint can be determined empirically or based on a desired surface smoothness, for example. In some configurations, the smoothness constraint can include a range of about 10π / 180 to about 20π / 180. The output of region growing is a set of point clusters, each of which is a set of points, each of which is considered to be part of the same smooth surface. In some configurations, region growing can be based on a comparison of angles between normals. Region growing can be performed by algorithms such as, but not limited to, region growing segmentation http: / / pointclouds.org / documentation / tutorials / region_growing_segmentation.php and http: / / pointclouds.org / documentation / tutorials / cluster_extraction.php#cluster-extraction.

[0125] Referring now to FIG. 1G, the segmented point cloud data 10137 (FIG. 1D) can be used to generate polygon 10759, for example, a 5m x 5m polygon (10161) (FIG. 1D). The point sub-clusters can be converted into polygon 10759, for example, using meshing. Meshing can be performed by standard methods such as, for example, but not limited to, marching cubes, marching tetrahedra, surface nets, greedy meshing, and dual contouring. In some configurations, polygon 10759 can be generated by projecting a local neighborhood of points along the normals of the points and connecting unconnected points. The resulting polygon 10759 can be based on at least the size of the neighborhood, the maximum allowable distance for the points to be considered, the maximum edge length for the polygon, the minimum and maximum angles of the polygon, and the maximum deviation the normals can have from each other. In some configurations, polygons 10759 can be filtered according to whether polygon 10759 would be too small for AV 10101 (FIG. 1A) to traverse. In some configurations, a circle the size of AV 10101 (FIG. 1A) can be dragged around each of polygons 10759 by known means. If the circle falls substantially within polygon 10759, polygon 10759, and therefore the resulting drivable surface, can accommodate AV 10101 (FIG. 1A). In some configurations, the area of ​​polygon 10759 can be compared to the occupied area of ​​AV 10101 (FIG. 1A). The polygon can be assumed to be irregular; therefore, the first step in determining the area of ​​polygon 10759 would be to separate polygon 10759 into regular polygons 10759A by known methods. For each regular polygon 10759A, a standard area equation can be used to determine its size. The areas of each regular polygon 10759A can be added together to find the area of ​​the polygon 10759, which can be compared to the occupied area of ​​the AV 10101 (FIG. 1A). The filtered polygons can include a subset of polygons that meet the size criteria.The filtered polygons can be used to set the final drivable surface.

[0126] 1G , in some configurations, polygon 10759 can be processed by removing outliers by conventional means, such as, for example, but not limited to, statistical analysis techniques, such as those available in the Point Cloud Library, http: / / pointclouds.org / documentation / tutorials / statistical_outlier.php. Filtering can include downsizing segment 10137 ( FIG. 1D ) by conventional means, including, but not limited to, a voxelization grid approach, such as that available in the Point Cloud Library, http: / / pointclouds.org / documentation / tutorials / voxel_grid.php. Concave polygon 10263 can be created by, for example, but not limited to, the process described in “A New Concave Hull Algorithm and Concaveness Measure for n-dimensional Datasets,” Park et al., Journal of Information Science and Engineering 28, pp. 587-600, 2012.

[0127] Referring now primarily to FIG. 1H , in some configurations, the processed point cloud data 10135 ( FIG. 1D ) can be used to determine an initial drivable surface 10265. Region growing can generate point clusters that may include points that are part of the drivable surface. In some configurations, a reference plane can be fitted to each of the point clusters to determine the initial drivable surface. In some configurations, the point clusters can be filtered according to the relationship between the orientation of the point cluster and the reference plane. For example, if the angle between the point cluster plane and the reference plane is below, for example, but not by way of limitation, about 30°, the point cluster can be preliminarily considered to be part of the initial drivable surface. In some configurations, the point clusters can be filtered based on, for example, but not by way of limitation, a size constraint. In some configurations, point clusters larger in point size than about 20% of the total points in the point cloud data 10131 (FIG. 1D) can be considered too large, and point clusters smaller in size than about 0.1% of the total points in the point cloud data 10131 (FIG. 1D) can be considered too small. The initial drivable surface can include filtered versions of the point clusters. In some configurations, the point clusters can be split for further processing by any of several known methods. In some configurations, density-based spatial clustering for noisy applications (DBSCAN) can be used to split the point clusters, while in some configurations, k-means clustering can be used to split the point clusters. DBSCAN can group points that are densely clustered together and mark points that are substantially isolated or in low-density areas as outliers. To be considered dense, points must be located within a preselected distance from a candidate point. In some configurations, a scale factor for the preselected distance can be determined empirically or dynamically. In some configurations, the scale factor may be in the range of approximately 0.1 to 1.0.

[0128] Referring primarily to FIG. 1I, generating an SDSF line (10163) (FIG. 1D) can include locating the SDSF by further filtering a concave polygon 10263 on the drivable surface 10265 (FIG. 1H). In some configurations, points from the point cloud data that make up the polygon can be categorized as either upper donut points 10351 (FIG. 1J), lower donut points 10353 (FIG. 1J), or cylinder points 10355 (FIG. 1J). The upper donut points 10351 (FIG. 1J) can correspond to the shape of the SDSF model 10352 that is furthest from the ground. The lower donut points 10353 (FIG. 1J) correspond to the shape of the SDSF model 10352 that is closest to the ground or at ground level. Cylinder points 10355 (FIG. 1J) can fall into a shape between upper donut points 10351 (FIG. 1J) and lower donut points 10353 (FIG. 1J). A combination of categories can form a donut 10371. To determine whether a donut 10371 forms an SDSF, certain criteria are tested. For example, in each donut 10371, there must be a minimum number of points that are upper donut points 10351 (FIG. 1J) and a minimum number that are lower donut points 10353 (FIG. 1J). In some configurations, the minimum value can be selected empirically and can fall in the range of approximately 5 to 20. Each donut 10371 can be divided into multiple portions, for example, two hemispheres. Another criterion for determining whether points within donut 10371 represent an SDSF is whether a majority of the points are located within opposing hemispheres of the donut 10371 portion. Cylindrical points 10355 (Figure 1J) can occur in either the first cylindrical region 10357 (Figure 1J) or the second cylindrical region 10359 (Figure 1J). Another criterion for SDSF selection is that a minimum number of points must be present within both cylindrical regions 10357 / 10359 (Figure 1J). In some configurations, the minimum number of points can be selected empirically and can fall in the range of 3 to 20.Another criterion for SDSF selection is that the donut 10371 must contain at least two of three categories of points: upper donut points 10351 (FIG. 1J), lower donut points 10353 (FIG. 1J), and cylinder points 10355 (FIG. 1J).

[0129] Continuing with reference primarily to FIG. 11, in some configurations, polygons can be processed in parallel. Each category worker 10362 can search its assigned polygon for SDSF points 10789 (FIG. 1N) and assign the SDSF points 10789 (FIG. 1N) to categories 10763 (FIG. 1G). As the polygons are processed, the resulting point categories 10763 (FIG. 1G) can be combined (10363) to form combined category 10366, and the categories can be shortened (10365) to form shortened combined category 10368. Shortening the SDSF points 10789 (FIG. 1N) can include filtering the SDSF points 10789 (FIG. 1N) with respect to their distance from the ground. The shortened combined categories 10368 are averaged, possibly processed in parallel by an average worker 10373, by searching the area around each SDSF point 10766 (FIG. 1G) and generating an average point 10765 (FIG. 1G). The category points can form a set of averaged donuts 10375. In some configurations, the radius around each SDSF point 10766 (FIG. 1G) can be empirically determined. In some configurations, the radius around each SDSF point 10766 (FIG. 1G) can include a range of 0.1 m to 1.0 m. The height change between one point on the SDSF trajectory 10377 (FIG. 1G) and another with respect to the SDSF at the average point 10765 (FIG. 1G) can be calculated. Connecting the averaged donuts 10375 together can generate the SDSF trajectory 10377 (FIGS. 1G and 1K). When generating the SDSF trajectory 10377 (FIGS. 1G and 1K), if there are two candidate next points within a search radius of the starting point, the next point can be selected based on at least forming the straightest possible line between the previous line segment, the starting point, and the candidate destination point, where the candidate next point represents the smallest change in SDSF height between the previous point and the candidate next point. In some configurations, the SDSF height can be defined as the difference between the heights of the upper donut 10351 (FIG. 1J) and the lower donut 10353 (FIG. 1J).

[0130] Referring now primarily to FIG. 1L, combining concave polygons and SDSF lines 10165 (FIG. 1D) can generate a dataset including polygon 10139 (FIG. 1D) and SDSF 10141 (FIG. 1D), which can be manipulated to generate a graphed polygon using the SDSF data. Manipulating concave polygons 10263 can include, but is not limited to, merging concave polygons 10263 to form merged polygon 10771. Merging concave polygons 10263 can be accomplished using known methods, such as, but not limited to, those found at (http: / / www.angusj.com / delphi / clipper.php). Merged polygon 10771 can be expanded to smooth edges and form expanded polygon 10772. Expanded polygon 10772 can be contracted to provide a driving margin, forming contracted polygon 10774, to which SDSF trajectory 10377 (FIG. 1M) can be added. Inward trimming (contraction) can ensure that there is room near the edge for AV 10101 (FIG. 1A) to travel by reducing the size of the drivable surface by a preselected amount based at least on the size of AV 10101 (FIG. 1A). Polygon expansion and contraction can be accomplished by commercially available techniques, such as, but not limited to, the ARCGIS® Clip command (http: / / desktop.arcgis.com / en / arcmap / 10.3 / manage-data / editing-existing-features / clipping-a-polygon-feature.htm).

[0131] Referring now primarily to FIG. 1M , the contracted polygon 10774 can be partitioned into polygons 10778, each of which can be traversed without encountering a non-drivable surface. The contracted polygon 10774 can be partitioned by conventional means such as, for example, but not limited to, ear slicing, optimized by z-order curve hashing and extended to handle holes, twisted polygons, degeneracy, and self-intersections. Commercially available ear slicing implementations include, but are not limited to, those found at (https: / / github.com / mapbox / earcut.hpp). The SDSF trajectory 10377 can include SDSF points 10789 ( FIG. 1N ), which can be connected to polygon vertices 10781. The vertices 10781 can be considered to be possible path points that can be connected to each other to form possible travel paths for the AV 10101 ( FIG. 1A ). In the dataset, SDSF points 10789 (FIG. 1N) can be labeled as such. As partitioning progresses, it is possible that redundant edges are introduced, such as, but not limited to, edges 10777 and 10779. Removing one of edges 10777 or 10779 can reduce the complexity of further analysis and preserve the polygonal mesh. In some configurations, a Hertel-Mehlhorn polygon partitioning algorithm can be used to remove edges and omit edges labeled as features. The set of polygons 10778, including the labeled features, can undergo further simplification to reduce the number of possible path points, which can be provided to device controller 10111 (FIG. 1A) in the form of annotated point data 10379 (FIG. 5B), which can be used to populate an occupancy grid.

[0132] 2A-2B, sensor data collected by an AV can also be used to populate an occupancy grid. A processor within the AV can receive data from sensors in a long-range sensor assembly 20400 mounted on the cargo container 20110 and from short-range sensors 20510, 20520, 20530, 20540, and other sensors located in the cargo platform 20160. In addition, the processor may receive data from an optional short-range sensor 20505 mounted near the top of the front of the cargo container 20110. The processor may also receive data from one or more antennas 20122A, 20122B (FIG. 1C), including cellular, WiFi, and / or GPS. In one example, the AV 20100 has a GPS antenna 20122A (FIG. 1C) located on the long-range sensor assembly 20400 and / or an antenna 20122B (FIG. 1C) located on the cargo container 20110. The processor may be located anywhere within the AV 20100. In some embodiments, one or more processors are located within the long range sensor assembly 20400. Additional processors may be located within the cargo platform 20160. In other embodiments, the processor may be located within the cargo container 20110 and / or as part of the power base 20170.

[0133] 2A-2B , the long-range sensor assembly 20400 is mounted on the cargo container to provide an improved view of the environment surrounding the AV. In one embodiment, the long-range sensor assembly 20400 is more than 1.2 m feet above the travel surface or ground. In other embodiments, if the cargo container is taller or a power base configuration elevates the cargo platform 20160, the long-range sensor assembly 20400 may be 1.8 m above the ground across which the AV is traveling. The long-range sensor assembly 20400 provides information about the environment around the AV from a minimum distance to a maximum range. The minimum distance may be defined by the relative positions of the long-range sensor 20400 and the cargo container 20110. The minimum distance may further be defined by the field of view (FOV) of the sensor. The maximum distance may be defined by the range of the long-range sensor in the long-range sensor assembly 20400 and / or by the processor. In one embodiment, the range of the long-range sensor is limited to 20 meters. In one embodiment, the Velodyne Puck LIDAR has a range of up to 100 m. The long-range sensor assembly 20400 may provide data about objects in all directions. The sensor assembly may provide information about structures, surfaces, and obstacles over a 360° angle around the AV20100.

[0134] Continuing with reference to FIG. 2A , three long-range cameras viewing through windows 20434, 20436, and 20438 can provide horizontal FOVs 20410, 20412, 20414, which together provide a 360° FOV. The horizontal FOV may be defined by the camera selected and the camera's location within the long-range camera assembly 20400. In describing the field of view, zero angle is a ray that lies in a vertical plane through the center of the AV 20100 and perpendicular to the front of the AV. A zero-angle ray passes through the front of the AV. The front long-range camera viewing through window 20434 has a 96° FOV 20410 that is between 311° and 47°. The left long-range camera viewing through window 20436 has an FOV 20412 that is between 47° and 180°. The right long-range camera looking through window 20438 has an FOV 20414 of 180° to 311°. The long-range sensor assembly 20400 may include an industrial camera positioned to look through window 20432 that provides more detailed information about objects and surfaces in front of the AV 20100 than the long-range camera. The industrial camera located behind window 20432 may have an FOV 20416 that is defined by the selected camera and the location of the camera within the long-range camera assembly 20400. In one example, the industrial camera behind window 20432 has an FOV of 23° to 337°.

[0135] 2B, the LIDAR 20420 provides a 360° horizontal FOV around the AV 20100. The vertical FOV may be limited by the LIDAR instrument. In one embodiment, the vertical FOV 20418 is 40° and mounted 1.2 m to 1.8 m above the ground, setting the minimum distance of the sensor to 3.3 m to 5 m from the AV 20100.

[0136] 2C and 2D, the long-range sensor assembly 20400 is shown with a cover 20430. The cover 20430 includes windows 20434, 20432, 20436 through which the long-range and industrial cameras observe the environment surrounding the AV 20100. The cover 20430 for the long-range sensor assembly 20400 is sealed from the weather by an O-ring between the cover 20430 and the top of the cargo container 20110.

[0137] 2E and 2F, the cover 20430 has been removed to reveal an embodiment of a camera and processor. The LIDAR sensor 20420 provides data regarding range or distance to surfaces surrounding the AV. These data may be provided to a processor 20470, located within the long-range sensor assembly 20400. The LIDAR is mounted on a structure 20405 above the long-range cameras 20440A-C and the cover 20430. The LIDAR sensor 20420 is an example of a ranging sensor based on reflected laser pulsed light. Other ranging sensors, such as radar that uses reflected radio waves, can also be used. In one example, the LIDAR sensor 20420 is a Puck sensor by VELODYNE LIDAR® (San Jose, CA). The three long-range cameras 20440A, 20440B, 20440C provide digital images of objects, surfaces, and structures surrounding the AV 20100. Three long-range cameras 20440A, 20440B, 20440C are arranged around the structure 20405 relative to the cover 20430 to provide three horizontal FOVs that cover a full 360° around the AV. The long-range cameras 20440A, 20440B, 20440C are on an elevated ring structure 20405 that is mounted to the cargo container 20110. The long-range cameras 20440A, 20440B, 20440C receive images through windows 20434, 20436, 20438 mounted in the cover 20430. The long-range cameras may comprise cameras and lenses on a printed circuit board (PCB).

[0138] 2F, one embodiment of long-range camera 20440A may include a digital camera 20444 with a fisheye lens 20442 mounted on the front of the digital camera 20444. The fisheye lens 20442 may expand the FOV of the camera to a much wider angle. In one embodiment, the fisheye lens expands the field of view to 180°. In one embodiment, the digital camera 20444 is similar to the e-cam52A_56540_MOD by E-con Systems (San Jose, CA). In one embodiment, the fisheye lens 20442 is similar to the model DSL227 by Sunex (Carlsbad, CA).

[0139] 2F, the long-range sensor assembly 20400 may also include an industrial camera 20450 that receives visual data through a window 20432 in the cover 20430. The industrial camera 20450 provides additional data to the processor 20470 regarding objects, surfaces, and structures in front of the AV. The camera may be similar to a Kowa industrial camera (part number LM6HC). The industrial camera 20450 and long-range cameras 20440A-C are positioned 1.2 m to 1.8 m above the surface over which the AV 20100 is moving.

[0140] Continuing with reference to FIG. 2F, mounting the long-range sensor assembly 20400 above a cargo container provides at least two advantages. The field of view for the long-range sensors, including the long-range cameras 20440A-C, industrial camera 20450, and LIDAR 20420, is often less obstructed by nearby objects such as people, cars, and low walls when the sensors are mounted further above the ground. In addition, pedestrian walkways are designed to provide visual cues, including signs, fence heights, and the like, for people to perceive, with typical eye heights ranging from 1.2 m to 1.8 m. Mounting the long-range sensor assembly 20400 above a cargo container places the long-range cameras 20440A-C, 20450 at the same level as the signs and above visual cues directed at pedestrians. The long-range sensors are mounted on a structure 20405 that provides a substantially rigid mount that withstands deflections caused by the AV 20100's movement.

[0141] 2E and 2F, the long-range sensor assembly may include an inertial measurement unit (IMU) and one or more processors that receive data from the long-range sensors and output processed data to other processors for navigation. An IMU 20460 with vertical reference (VRU) is mounted on structure 20405. The IMU / VRU 20460 may be located directly below the LIDAR 20420 to provide position data for the LIDAR 20420. The position and orientation from the IMU / VRU 20460 may be combined with data from other long-range sensors. In one example, the IMU / VRU 20460 is a model MTi 20 supplied by Xsens Technologies (The Netherlands). The one or more processors may include a processor 20465 that receives data from at least the industrial camera 20450. Additionally, processor 20470 may receive data from at least one of the following: LIDAR 20420, long-range cameras 20440A-C, industrial camera 20450, and IMU / VRU 20460. Processor 20470 may be cooled by a liquid-cooled heat exchanger 20475, which is connected to a circulating coolant system.

[0142] 2G, the AV 20100 may include several short-range sensors that detect driving surfaces and obstacles within a predetermined distance from the AV. Short-range sensors 20510, 20520, 20530, 20540, 20550, and 20560 are located on the periphery of the container platform 20160. These sensors are located below the cargo container 20110 (FIG. 2B) and are closer to the ground than the long-range sensor assembly 20400 (FIG. 2C). The short-range sensors 20510, 20520, 20530, 20540, 20550, and 20560 are angled downward to provide an FOV that captures surfaces and objects that cannot be seen by the sensors in the long-range sensor assembly 20400 (FIG. 2C). The field of view of sensors located closer to the ground and angled downward is less likely to be obstructed by nearby objects and pedestrians than sensors mounted farther from the ground. In one embodiment, the short-range sensors provide information about the ground surface and objects up to 4 m from the AV20100.

[0143] Referring again to FIG. 2B, the vertical FOVs for two of the short-range sensors are shown in a side view of the AV 20100. The vertical FOV 20542 of the rear-facing sensor 20540 is centered about a centerline 20544. The centerline 20544 is angled below the upper surface of the cargo platform 20160. In one example, the sensor 20540 has a vertical FOV of 42° and a centerline 20546 that is angled 22° to 28° below a plane 20547 defined by the top surface of the cargo platform 20160. In one example, the short-range sensors 20510 and 20540 are approximately 0.55 m to 0.71 m above the ground. The resulting FOVs 20512, 20542 cover the ground 0.4 m to 4.2 m from the AV. Short-range sensors 20510, 20520, 20530, 20550 (FIG. 2G), 20560 (FIG. 2G) mounted on cargo base 20160 have similar vertical fields of view and centerline angles relative to the top of the cargo platform. The short-range sensors mounted on cargo platform 20160 can view the ground 0.4 to 4.7 meters from the outer edge of the AV 20100.

[0144] 2B , a short-range sensor 20505 may be mounted on the front surface near the top of the cargo container 20110. In one embodiment, the sensor 20505 may provide an additional view of the ground in front of the AV to the view provided by the short-range sensor 20510. In another embodiment, the sensor 20505 may provide a view of the ground in front of the AV in place of the view provided by the short-range sensor 20510. In one embodiment, the short-range sensor 20505 may have a vertical FOV 20507 of 42°, and the angle of its centerline relative to the top of the cargo platform 20160 is 39°. The resulting view of the ground extends from 0.7 m to 3.75 m from the AV.

[0145] 2G , the horizontal FOVs of short-range sensors 20510, 20520, 20530, 20540, 20550, 20560 cover all directions around the AV 20100. The horizontal FOVs 20522 and 20532 of adjacent sensors, such as 20520 and 20530, overlap at some distance away from the AV 20100. In one example, the horizontal FOVs 20522, 20532 and 20562, 20552 of adjacent sensors 20520, 20530 and 20560, 20550 overlap at 0.5 to 2 meters from the AV. The short-range sensors are distributed around the perimeter of the cargo base 20160, have horizontal fields of view, and are installed at specific angles to provide near-complete visual coverage of the ground surrounding the AV. In one embodiment, the short-range sensors have a horizontal FOV of 69°. The front sensor 20510 faces forward at a zero angle relative to the AV and has an FOV 20512. In one embodiment, the two front corner sensors 20520, 20560 are angled so that their centerlines are at a 65° angle 20564. In one embodiment, the rear sensors 20530, 20550 are angled so that the centerlines of 20530 and 20560 are at a 110° angle 20534. Other numbers of sensors with other horizontal FOVs are also possible, mounted around the periphery of the cargo base 20160 to provide a nearly complete view of the ground around the AV 20100 in some configurations.

[0146] 2H, short range sensors 20510, 20520, 20530, 20540, 20550, 20560 are located on the periphery of the cargo base 20160. The short range cameras are mounted in protrusions which set the angle and location of the short range cameras. In another configuration, the sensors are mounted and located on the interior of the cargo base and receive visual data through windows aligned with the outer skin of the cargo base 20160.

[0147] 2I and 2J, the short-range sensor 20600 is mounted within a skin element 20516 of the cargo base 20160 and may include a liquid cooling system. The skin element 20516 includes a formed protrusion 20514 that holds the short-range sensor assembly 20600 in place and at a vertical angle relative to the top of the cargo base 20160 and at an angle relative to the front of the cargo base 20160. In some configurations, the short-range sensor 20510 is angled downward by 28° relative to the cargo platform 20160, the short-range sensors 20520 and 20560 are angled downward by 18° and 25° forward, the short-range sensors 20530 and 20550 are angled downward by 34° and 20° aft, and the short-range sensor 20540 is angled downward by 28° relative to the cargo platform 20160. The skin element 20516 includes a cavity 20517 for receiving the camera assembly 20600. The skin element 20516 may also include a plurality of elements 20518 for receiving mechanical fasteners, including but not limited to rivets, screws, and buttons. Alternatively, the camera assembly may be mounted with an adhesive or held in place with clips fastened to the skin element 20516. A gasket 20519 can provide a seal against the front of the camera 20610.

[0148] 2K and 2L, a short-range sensor assembly 20600 includes a short-range sensor 20610 mounted on a bracket 20622 that is attached to a water-cooled plate 20626. The outer case 20612, transparent cover 20614, and heat sink 20618 have been partially removed in FIGS. 2K and 2L to better visualize the heat-dissipating elements of the short-range sensor 20610: the sensor block 20616 and the electronics block 20620. The short-range sensor assembly 20600 may include one or more thermoelectric coolers (TECs) 20630 between the bracket 20622 and the liquid-cooled plate 20626. The liquid-cooled plate 20626 is cooled by coolant pumped through 20628 that is thermally connected to the plate 20626. The TEC is an electrically powered element with first and second sides. The powered TEC cools a first side while rejecting on a second side the thermal energy removed from the first side plus electrical power. In the short-range sensor assembly 20600, the TEC 20630 cools the bracket 20622 and transfers thermal cooling energy plus electrical energy to the water-cooled plate 20626. Alternatively, the TEC 20630 can be used to actively control the temperature of the camera 20600 by varying the magnitude and polarity of the voltage supplied to the TEC 20630.

[0149] Operating the TEC 20630 in cooling mode allows the short-range sensor 20610 to operate at temperatures below the coolant temperature. A bracket 20622 is thermally connected to the short-range sensor 20610 in two locations, maximizing cooling of the sensor block 20616 and the electronics block 20620. The bracket 20622 includes tabs 20624 that are thermally attached to a heat sink 20618 via screws 20625. The heat sink 20618 is thermally connected to the sensor block 20616. The bracket is thus thermally connected to the sensor block 20616 via the heat sink 20618, screws 20625, and tabs 20624. The bracket 20622 is also mechanically attached to the electronics block 20620, providing direct cooling of the electronics block 20620. Bracket 20622 may include multiple mechanical attachments, including, but not limited to, screws and rivets that engage element 20518 of FIG. 2J. Short-range sensor 20610 may incorporate one or more sensors, including, but not limited to, a camera, a stereo camera, an ultrasonic sensor, a short-range radar, and an infrared projector and a CMOS sensor. One exemplary short-range sensor is similar to the RealSense Depth Camera D435 by Intel (Santa Clara, California), which includes an IR projector, two imager chips, and an RGB camera.

[0150] 2M-2O, another embodiment of the AV20100 is shown. The AV20100A includes a cargo container 20110 mounted on a cargo platform 20160 and a power base 20170. The AV20100A includes multiple long-range and short-range sensors. A primary long-range sensor is mounted in a sensor pylon 20400A above the cargo container 20110. The sensor pylon may include a LIDAR 20420 and multiple long-range cameras (not shown) oriented in divergent directions to provide a wide field of view. In some configurations, the LIDAR 20420 can be used as described elsewhere herein to provide point cloud data that may enable the capture of an occupancy grid, identify landmarks, locate the AV20100 within its environment, and / or provide information for determining a navigable space, for example, without limitation. In some configurations, a long-range camera from Leopard Imaging Inc. can be used to identify landmarks, locate the AV20100 within its environment, and / or determine navigable space.

[0151] 2M-2O, short-range sensors are primarily mounted within the cargo platform 20160 and provide information about obstacles in the vicinity of the AV20100A. In some embodiments, the short-range sensors provide data about obstacles and surfaces within 4 m of the AV20100A. In some configurations, the short-range sensors provide information up to 10 m from the AV20100A. Multiple cameras, at least partially forward-facing, are mounted within the cargo platform 20160. In some configurations, the multiple cameras may include three cameras.

[0152] Referring now to FIG. 2O, the top cover 20830 is partially cut away to reveal the sub-roof 20810. The sub-roof 20810 provides a single component on which multiple antennas 20820 can be mounted. In one embodiment, ten antennas 20820 are mounted on the sub-roof 20810. Furthermore, for example, four cellular communication channels each have two antennas, and there are two WiFi antennas. The antennas are wired as primary and secondary antennas for cellular transmission and reception. The secondary antennas can improve cellular functionality in several ways, including, but not limited to, reducing interference and achieving 4G LTE connectivity. The sub-roof 20810 and top cover 20830 are non-metallic. The sub-roof 20810 is a plastic surface within 10 to 20 mm of the top cover 20830, which is non-structural and allows the antennas to be connected to the processor before the top cover 20830 is attached. Antenna connections are often high impedance and sensitive to dirt, grease, and misuse. Mounting and connecting the antenna to the subroof 20810 allows the top cover to be installed and removed without touching the antenna connections. Maintenance and repair operations may include removing the top cover without removing the subroof or disconnecting the antenna. Assembly of the antenna separately from installation of the top cover 20830 facilitates testing / repair. The top cover 20830 is weatherproof and prevents water and debris from entering the cargo container 20110. Mounting the antenna on the subroof minimizes the number of openings on the top cover 20830.

[0153] 2P, 2Q, and 2R, another embodiment of a long-range sensor assembly (LRSA) 20400A mounted on a cargo container (not shown) is shown. The LRSA may include a LIDAR and multiple long-range cameras mounted at different locations on the LRSA structure 20950 to provide a panoramic view of the AV20100A's environment. The LIDAR 20420 is mounted at the top of the LRSA structure 20950 to provide an uninterrupted view. The LIDAR may include a VELODYNE LIDAR. Multiple long-range cameras 20910A-20910D are mounted on the LRSA structure 20950 on the next level below the LIDAR 20420. In one embodiment, four cameras are mounted, one every 90° around the perimeter of the structure, providing four views of the environment surrounding the AV20100. In some embodiments, the four views will overlap. In some embodiments, each camera is either aligned with the direction of motion or orthogonal to the direction of movement. In one embodiment, one camera is aligned with each of the major faces of the AV20100A, i.e., the front, back, left, and right sides. In one embodiment, the long-range camera is a model LI-AR01 44-MIPI-M12 manufactured by Leopard Imaging Inc. The long-range camera may have a MIPI CSI-2 interface to provide high-speed data transfer to the processor. The long-range camera may have a horizontal field of view of 50° to 70° and a vertical field of view of 30° to 40°.

[0154] 2S and 2T, a long-range processor 20940 is located on the LRSA structure 20950 below the long-range cameras 20910A-20910D and the LIDAR 20420. The long-range processor 20940 receives data from the long-range cameras and the LIDAR. The long-range processor communicates with one or more processors elsewhere within the AV20100A. The long-range processor 20940 provides data derived from the long-range cameras and the LIDAR to one or more processors located elsewhere on the AV20100A, as described elsewhere herein. The long-range processor 20940 may be liquid-cooled by a chiller 20930. The chiller 20930 may be mounted on a structure below the long-range cameras and the LIDAR. The chiller 20930 may provide a mounting location for the long-range processor 20940. The cooler 20930 is described in U.S. Patent Application No. 16 / 883,668 (Attorney Docket No. AA280), filed May 26, 2020, and entitled "Apparatus for Electronic Cooling on an Autonomous Device," which is incorporated herein by reference in its entirety. The cooler is provided with a liquid supply conduit and a return conduit that provide cooling liquid to the cooler 20930.

[0155] 2M and 2N, short-range camera assemblies 20740A-C are mounted on the front of the container platform 20160 and angled to collect information about the traveling surface and obstacles, steps, curbs, and other substantially discontinuous surface features (SDSF). The camera assemblies 20740A-C include one or more LED lights to illuminate the traveling surface, objects on the ground, and SDSF.

[0156] Referring now to Figures 2U-2X, camera assemblies 20740A-B include lights 20732 to illuminate the ground and objects and provide improved image data from camera 20732. Note that camera assembly 20740A is a mirror image of 20740C, and the description of 20740A implicitly applies to 20740C. Camera 20732 may include a single-focus camera, a stereo camera, and / or an infrared projector and CMOS sensor. One example of a camera is the Real-Sense Depth D435 camera by Intel (Santa Clara, CA), which includes an IR projector, two imager chips with lenses, and an RGB camera. LED lights 20734 may be used at night or in low-light conditions, or may be used constantly to improve image data. One operating principle is that the lights create contrast by illuminating the projection surface and creating shadows in recesses. The LED lights may be white LEDs in some examples. In one embodiment, the LED light 20734 is an Xlamp XHP50 from Cree Inc. In another embodiment, the LED light may emit infrared light to provide illumination for the camera 20372 without distracting or disturbing nearby pedestrians or drivers.

[0157] 2U-2X, the placement and angle of light 20374 and the shape of covers 20736A, 20736B prevent camera 20732 from seeing light 20374. The angle and placement of light 20374 and covers 20736A, 20736B prevent the light from interfering with drivers or obstructing pedestrians. It is advantageous for camera 20732 not to be exposed to light 20734 to prevent the sensor in camera 20732 from being blinded by light 20734 and therefore prevented from detecting weaker light signals from the ground and objects in front and to the sides of AV20100A. Camera 20732 and / or light 20734 may be cooled using a liquid flowing in and out of the camera assembly through port 20736.

[0158] 2W and 2X, a short-range camera assembly 20740A includes an ultrasonic or sonar short-range sensor 20730A. A second short-range camera assembly 20740C also includes an ultrasonic short-range sensor 20730B (FIG. 2N).

[0159] 2Y, the ultrasonic sensor 20730A is mounted above the camera 20732. In one embodiment, the centerline of the ultrasonic sensor 20730A is parallel to the base of the cargo container 20110, which often means that the sensor 20730A is horizontal. The sensor 20730A is angled 45° from facing forward. The cover 20376A provides a horn 20746 for directing the ultrasonic waves emerging from and received by the ultrasonic sensor 20730A.

[0160] Continuing with reference to FIG. 2Y, a cross section of the camera 20732 and light 20734 in camera assembly 20740A illustrates the angles and openings in cover 20736. The cameras in the short-range camera assembly are angled downward to better image the ground in front of and to the sides of the AV. The center camera assembly 20740B is oriented directly ahead in the horizontal plane. The corner camera assemblies 20740A, 20740C are angled 25° to their respective sides relative to directly ahead in the horizontal plane. The camera 20732 is angled 20° downward relative to the top of the cargo platform in the vertical plane. Because the AV generally holds the cargo platform horizontally, the cameras are therefore angled 20° downward from horizontal. Similarly, the center camera assembly 20740B (FIG. 2M) is angled 28° downward from horizontal. In some embodiments, the cameras in the camera assembly may be angled downward by 25° to 35°. In another embodiment, the cameras in the camera assemblies may be angled downward by 15° to 45°. The LED lights 20734 are similarly angled downward to illuminate the ground imaged by the cameras 20732 and minimize distraction to pedestrians. In one embodiment, the LED light centerline 20742 is parallel to and within 5° of the camera centerline 20738. The cover 20736A protects both the cameras 20732 and pedestrians from the bright light of the LEDs 20734 in the camera assemblies 20740A-C. The cover, which isolates the light emitted by the LEDs, also provides a tapered opening 20737 to maximize the field of view of the cameras 20732. The lights are recessed at least 4 mm from the cover opening. The light opening is defined by an upper wall 20739 and a lower wall 20744. The upper wall 20739 is approximately parallel (±5°) to the centerline 2074. The lower wall 20744 flares out approximately 18° from the centerline 20742 to maximize illumination of the ground and objects near the ground.

[0161] 2Z-2AA, in one configuration, the light 20734 includes two LEDs 20734A each under a square lens 20734B to generate a beam of light. The LEDs / lenses are angled and positioned relative to the camera 20732 to illuminate the camera's field of view with little light spill outside the camera's 20732 FOV. The two LEDs / lenses are mounted together on a single PCB 20752 with a defined angle 20762 between the two lights. In another configuration, the two LEDs / lenses are individually mounted on a heat sink 20626A on a separate PCB at an angle relative to each other. In one example, the lights are Xlamp XHP50s from Cree, Inc., and the lenses are 60° lens HB-SQ-Ws from LEDil. The lights are angled approximately 50° relative to each other, so the angle 20762 between the front of the lenses is 130°. The light is located approximately 18mm ( ± 5 mm) 20764 rearward and approximately 30 mm 20766 downward from the centerline of the camera 20732.

[0162] 2AA-2BB, camera 20732 is cooled by a thermoelectric cooler (TEC) 20630, which, along with light 20734, is cooled by liquid coolant flowing through cold block 20626A. The camera is attached to bracket 20622 via screws 20625 that thread into the sensor block portion of the camera, while the rear of bracket 20622 is bolted to the electronics block of the camera. Bracket 20622 is cooled by two TECs to maintain the performance of the IR imaging chip (CMOS chip) within camera 20732. The TECs reject heat from bracket 20622 and the power they draw to cold block 20626A.

[0163] Referring now to FIG. 2BB, coolant is directed through a U-shaped channel created by center fin 20626D. The coolant flows directly behind the LED / lens / PCB of light 20734. Fins 20626B, 20626C improve heat transfer from light 20734 to the coolant. The coolant flows upward and passes by the hot side of TEC 20630. The fluid channel is created by plate 20737 (FIG. 2X) attached to the back of cold block 20626A.

[0164] 3A , sensor data and map data can be used to update an occupancy grid. The systems and methods of the present teachings can manage a global occupancy grid for a device navigating autonomously relative to a grid map. The grid map can include routes or paths the device can take from a start point to a destination. The global occupancy grid can include free space indications that can indicate where it is safe for the device to navigate. The potential paths and free space indications can be combined on the global occupancy grid to establish an optimal path the device can proceed on to safely reach its destination.

[0165] 3A , as the device moves, a global occupancy grid that will be used to determine a non-obstructive navigation route can be accessed based on the device's location, and the global occupancy grid can be updated as the device moves. The updates can be based on at least current values ​​associated with the global occupancy grid at the device's location, a static occupancy grid that can include historical information about the neighborhood through which the device is navigating, and data collected by sensors as the device progresses. The sensors can be located on the device, as described herein, or they can be located at different locations.

[0166] Continuing with further reference to FIG. 3A , the global occupancy grid can include cells, which can be associated with occupancy probability values. Each cell of the global occupancy grid can be associated with information such as whether an obstacle has been identified at the cell's location, characteristics and discontinuities of the traveling surface at and surrounding the location, as determined from previously collected data and by data collected as the device navigates, and prior occupancy data associated with the location. Data captured as the device navigates can be stored in a local occupancy grid whose center is the device. When updating the global occupancy grid, static previously collected data can be combined with the local occupancy grid data and global occupancy data determined in previous updates to create a new global occupancy grid with the space occupied by the device marked as unoccupied. In some configurations, a Bayesian method can be used to update the global occupancy grid. The method may include, for each cell in the local occupancy grid, calculating a location of the cell on a global occupancy grid, accessing a value at that location from the current global occupancy grid, accessing a value at that location from a static occupancy grid, accessing a value at that location from the local occupancy grid, and calculating a new value at that location on the global occupancy grid as a function of the current value from the global occupancy grid, the value from the static occupancy grid, and the value from the local occupancy grid. In some configurations, the relationship used to calculate the new value may include the sum of the static value and the local occupancy grid value minus the current value. In some configurations, the new value may be limited by a preselected value, for example, based on a computational limit.

[0167] Continuing with reference to FIG. 3A , the system 30100 of the present teachings can manage a global occupancy grid. The global occupancy grid can start with initial data and can be updated as the device moves. Creating the initial global occupancy grid can include a first process, and updating the global occupancy grid can include a second process. The system 30100 can include, without limitation, a global occupancy server 30121 that can receive information from various sources and update the global occupancy grid 30505 based at least on the information. The information can be provided by, for example, without limitation, sensors located on the device and / or elsewhere, static information, and navigation information. In some configurations, sensors can include cameras and radar that can detect, for example, surface characteristics and obstacles. Sensors can advantageously be located on the device to provide sufficient peripheral coverage to enable safe navigation by the device, for example. In some configurations, the LIDAR 30103 can provide LIDAR point cloud (PC) data 30201, which can enable populating a local occupancy grid with LIDAR free space information 30213. In some configurations, a conventional ground detection inverse sensor model (ISM) 30113 can process the LIDAR PC data 30201 and generate the LIDAR free space information 30213.

[0168] 3A , in some configurations, the RGB-D camera 30101 can provide RGB-D PC data 30202 and RGB camera data 30203. The RGB-D PC data 30202 can capture a local occupancy grid with depth free-space information 30209, and the RGB-D camera data 30203 can capture a local occupancy grid with surface data 30211. In some configurations, the RGB-D PC data 30202 can be processed by, for example, but not limited to, a conventional stereo free-space ISM 30109, and the RGB-D camera data 30203 can be fed into, for example, but not limited to, a conventional surface detection neural network 30111. In some configurations, the RGB MIPI camera 30105 can provide RGB data 30205, which can be combined with the LIDAR PC data 30201 to generate a local occupancy grid with LIDAR / MIPI free-space information 30215. In some configurations, the RGB data 30205 can be fed into a conventional free-space neural network 30115, the output of which, along with the LIDAR PC data 30201, can undergo a pre-selected mask 30221 that can identify portions of the RGB data 30205 that are most important for accuracy before being fed into a conventional 2D-3D registration 30117. The 2D-3D registration 30117 can project an image from the RGB data 30205 onto the LIDAR PC data 30201. In some configurations, the 2D-3D registration 30117 is not required. Any combination of sensors and methods for processing sensor data can be used to aggregate data and update the global occupancy grid. Any number of free-space estimation procedures can be used and combined to enable determination and validation of occupancy probabilities within the global occupancy grid.

[0169] Continuing with reference to FIG. 3A , in some configurations, historical data can be provided by a repository 30107 of previously collected and processed data, for example, with information associated with a navigation area. In some configurations, the repository 30107 can include route information, such as, but not limited to, polygon 30207. In some configurations, these data can be fed to a conventional polygon parser 30119, which can provide edges 30303, discontinuities 30503, and surfaces 30241 to a global occupancy grid server 30121. The global occupancy grid server 30121 can fuse local occupancy grid data collected by sensors with the processed repository data to determine a global occupancy grid 30505. A grid map 30601 ( FIG. 3D ) can be created from the global occupancy data.

[0170] 3B , in some configurations, the sensors may include sonar 30141, which may provide a local occupancy grid with sonar free space 30225 to global occupancy grid server 30121. Depth data 30209 may be processed by conventional free-space ISM 30143. The local occupancy grid with sonar free space 30225 may be fused with the local occupancy grid with surfaces and discontinuities 30223, the local occupancy grid with LIDAR free space 30213, the local occupancy grid with LIDAR / MIPI free space 30215, the local occupancy grid with stereo free space 30209, as well as edges 30303 ( FIG. 3F ), ​​discontinuities 30503 ( FIG. 3F ), ​​navigation points 30501 ( FIG. 3F ), ​​surface confidence 30513 ( FIG. 3F ), ​​and surfaces 30241 ( FIG. 3F ) to form global occupancy grid 30505.

[0171] 3C-3F, to initialize a global occupancy grid, global occupancy grid initialization 30200 may include creating, by global occupancy grid server 30121, a global occupancy grid 30505 and a static grid 30249. Global occupancy grid 30505 may be created by blending data from local occupancy grid 30118 with edges 30303, discontinuities 30503, and surfaces 30241 located within the region of interest. Static grid 30249 (FIG. 3D) may be created to include data such as, for example, but not limited to, surface data 30241, discontinuity data 30503, edges 30303, and polygons 30207. An initial global occupancy grid 30505 can be calculated by adding occupancy probability data from static grid 30249 (FIG. 3E) to occupancy data derived from collected data from sensor 30107A and subtracting occupancy data from prior 30505A (FIG. 3F) of global occupancy grid 30505. Local occupancy grid 30118 can include, but is not limited to, local occupancy grid data resulting from stereo free-space estimation 30209 (FIG. 3B) through ISM, local occupancy grid data including surface / discontinuity detection results 30223 (FIG. 3B), local occupancy grid data resulting from LIDAR free-space estimation 30213 (FIG. 3B) through ISM, and, in some configurations, local occupancy grid data resulting from LIDAR / MIPI free-space estimation 30215 (FIG. 3B) following 2D-3D registration 30117 (FIG. 3B). In some configurations, the local occupancy grid 30118 can include local occupancy grid data coming from the sonar free space estimate 30225 through the ISM. In some configurations, various local occupancy grids with free space estimates can be fused into the local occupancy grid 30118 according to a preselected known process. From the global occupancy grid 30505, a grid map 30601 (FIG. 3E) can be created that can include occupancy and surface data in the vicinity of the device.In some configurations, the grid map 30601 (FIG. 3E) and static grid 30249 (FIG. 3D) can be published using, for example, but not limited to, the Robot Operation System (ROS) subscribe / publish feature.

[0172] 3G and 3H, to update the occupancy grid as the device moves, occupancy grid update 30300 can include updating the local occupancy grid with data measured as the device is moving and combining that data with static grid 30249. Static grid 30249 is accessed when the device moves out of a working occupancy grid range. The device can be positioned within occupancy grid 30245A at a first location 30513A at a first time. When the device moves to a second location 30513B, the device is positioned within occupancy grid 30245B, which includes a set of values ​​derived from its new location and possibly from values ​​in occupancy grid 30245A. Data from static grid 30249 and surface data from initial global occupancy grid 30505 (FIG. 3C) that are geographically aligned with cells in occupancy grid 30245B at the second time can be used, along with the measured surface data and occupancy probabilities, to update each grid cell according to a preselected relationship. In some configurations, the relationship may include summing static data with measured data. The resulting occupancy grid 30245C at the third time and third location 30513C may be made available to the movement manager 30123 to inform the navigation of the device.

[0173] 3I , a method 30450 for creating and managing an occupancy grid can include, but is not limited to, transforming 30451 sensor measurements into a reference frame associated with the device by a local occupancy grid creation node 30122, creating 30453 a time-stamped measurement occupancy grid, and publishing 30455 the time-stamped measurement occupancy grid as a local occupancy grid 30234 ( FIG. 3G ). A system associated with method 30450 can include multiple local grid creation nodes 30122, e.g., one per sensor, such that multiple local occupancy grids 30234 ( FIG. 3G ) can be produced. The sensors can include, but are not limited to, an RGB-D camera 30325 ( FIG. 3G ), a LIDAR / MIPI 30231 ( FIG. 3G ), and a LIDAR 30233 ( FIG. 3G ). A system associated with method 30450 can include a global occupancy grid server 30121 that can receive the local occupancy grids and process them according to method 30450. In particular, the method 30450 may include loading 30242 a surface, accessing 30504 a surface discontinuity, such as, but not limited to, a curb, and creating 30248 a static occupancy grid 30249 from any properties available in the repository 30107, which may include, but is not limited to, surfaces and surface discontinuities. The method 30450 may include receiving 30456 a published local occupancy grid and moving 30457 a global occupancy grid to keep the device centered on the map. The method 30450 may include setting 30459 a new region on the map with prior information from the static prior occupancy grid 30249 and marking 30461 an area currently occupied by the device as unoccupied. The method 30450 may perform a loop 30463 for each cell in each local occupancy grid.The loop 30463 may include, but is not limited to, calculating a location of a cell on a global occupancy grid, accessing a previous value at that location on the global occupancy grid, and calculating a new value at the cell location based on a relationship between the previous value and the value at the cell in the local occupancy grid. The relationship may include, but is not limited to, summing the values. The loop 30463 may include comparing the new value against a preselected range of acceptable probabilities and setting the global occupancy grid with the new value. The comparison may include setting the probability to the minimum or maximum acceptable probability if the probability is lower or higher than the minimum or maximum acceptable probability. The method 30450 may include publishing 30467 the global occupancy grid.

[0174] Referring now to FIG. 3J , an alternative method 30150 for creating a global occupancy grid may include, but is not limited to, in 30151, if the device has moved, accessing occupancy probability values ​​associated with the old map area (where the device was before it moved) and updating the global occupancy grid on the new map area (where the device was after it moved) with the values ​​from the old map area 30153, accessing drivable surfaces associated with cells of the global occupancy grid in the new map area and updating the cells in the updated global occupancy grid with the drivable surfaces 30155, and proceeding to step 30159. If the device has not moved at 30151 and the global occupancy grid is co-located with the local occupancy grid, method 30150 may include updating 30159 the potentially updated global occupancy grid with surface confidences associated with drivable surfaces from at least one local occupancy grid, updating 30161 the updated global occupancy grid with log-odds of occupancy probability values ​​from the at least one local occupancy grid, for example, but not limited to, using a Bayes function, and adjusting 30163 the log-odds based on at least a characteristic associated with the location. If the global occupancy grid is not co-located with the local occupancy grid at 30157, method 30150 may include returning to step 30151. The characteristic may include, but is not limited to, setting the location of the device as unoccupied.

[0175] 3K, in another configuration, a method 30250 for creating a global occupancy grid may include, but is not limited to, at 30251, if the device moves, updating 30253 the global occupancy grid with information from a static grid associated with the new location of the device. Method 30250 may include analyzing 30257 the surface at the new location. At 30259, if the surface is drivable, method 30250 may include updating 30261 the surface on the global occupancy grid and updating 30263 the global occupancy grid with values ​​from a repository of static values ​​associated with the new location on the map.

[0176] 3L , updating the surface 30261 may include, but is not limited to, accessing 30351 a local occupancy grid (LOG) for a particular sensor. If there are additional cells to process in 30353, method 30261 may include accessing 30355 a surface classification confidence value and surface classification from the local occupancy grid. If, at 30357, the surface classification at the cell in the local occupancy grid is identical to the surface classification in the global occupancy grid at the cell's location, method 30261 may include setting 30461 a new global occupancy grid (GOG) surface confidence to the sum of the old global occupancy grid surface confidence and the local occupancy grid surface confidence. If, at 30357, the surface classification at the cell in the local occupancy grid is not identical to the surface classification in the global occupancy grid at the cell's location, method 30261 may include setting 30359 a new global occupancy grid surface confidence to the difference between the old global occupancy grid surface confidence and the local occupancy grid surface confidence. In 30463, if the new global occupancy grid surface confidence is less than zero, the method 30261 may include setting 30469 the new global occupancy grid surface classification to the value of the local occupancy grid surface classification.

[0177] 3M , updating the global occupancy grid 30263 with values ​​from a repository of static values ​​can include, but is not limited to, the following: if there are more cells to process in the local occupancy grid at 30361, method 30263 can include accessing the log-odds from the local occupancy grid 30363 and updating the log-odds in the global occupancy grid with the value from the local occupancy grid at that location 30365. If the maximum certainty that the cell is empty is met at 30367, and if the device is proceeding within a predetermined lane barrier at 30369, and if the surface is drivable at 30371, method 30263 can include updating the probability that the cell is occupied 30373 and returning to continue processing more cells. If the maximum certainty that the cell is empty is not reached in 30367, or if the device is not traveling in the predetermined lane in 30369, or if the surface is not drivable in the mode the device is currently traveling in, in 30371, method 30263 may include returning to consider additional cells without updating the log-odds. If the device is in a standard mode, i.e., a mode in which the device can navigate a relatively uniform surface, and the surface classification indicates that the surface is relatively uneven, method 30263 may adjust the device's path by increasing the probability that the cell is occupied by updating 30373 the log-odds. If the device is in a standard mode and the surface classification indicates that the surface is relatively uniform, method 30263 may adjust the device's path by decreasing the probability that the cell is occupied by updating 30373 the log-odds. If the device is traveling in four-wheel mode, i.e., a mode in which the device can navigate uneven terrain, adjustment of the probability that the cell is occupied may not be necessary.

[0178] 4A , an AV can proceed in a specific mode that may be associated with a device configuration, e.g., the configuration depicted in device 42114A and the configuration depicted in device 42114B. A system of the present teachings for real-time control of the device's configuration based on at least one environmental factor and the device's status can include, but is not limited to, a sensor, a locomotion means, a chassis operatively coupled to the sensor and the locomotion means, the chassis being driven by a motor and a power supply, a device processor that receives data from the sensor, and a power base processor that controls the locomotion means. In some configurations, the device processor can receive environmental data, determine environmental factors, determine configuration changes according to the environmental factors and the device's status, and provide the configuration changes to the power base processor. The power base processor can issue commands to the locomotion means to move the device from location to location and physically reconfigure the device when required by road surface type.

[0179] Continuing with reference to FIG. 4A , sensors that collect environmental data can include, for example, but are not limited to, cameras, LIDAR, radar, thermometers, pressure sensors, and weather condition sensors, some of which are described herein. From this set of data, the device processor can determine environmental factors on which device configuration changes can be based. In some configurations, the environmental factors can include, for example, but are not limited to, surface factors such as surface type, surface features, and surface conditions. Based on the environmental factors and the current status of the device, the device processor can determine in real time how to change its configuration to accommodate traversing a detected surface type.

[0180] Continuing with reference to FIG. 4A , in some configurations, configuration changes of devices 42114A / B / C (collectively referred to as devices 42114) can include, for example, changes in the configuration of the mobility vehicle. Other configuration changes, such as user information displays and sensor controls, which may depend on the current mode and surface type, are also contemplated. In some configurations, the mobility vehicle can include at least four drive wheels 442101, two on each side of the chassis 42112, and at least two caster wheels 42103 operably coupled to the chassis 42112, as described herein. In some configurations, the drive wheels 442101 can be operably coupled in pairs 42105, and each pair 42105 can include a first drive wheel 42101A and a second drive wheel 42101B of the four drive wheels 442101, the pairs 42105 being located on opposite sides of the chassis 42112. The operable coupling can include a wheel cluster assembly 42110. In some configurations, a power base processor 41016 (FIG. 4B) can control the rotation of the cluster assembly 42110. Left and right wheel motors 41017 (FIG. 4B) can drive wheels 442101 on either side of the chassis 42112. Turning can be accomplished by driving the left and right wheel motors 41017 (FIG. 4B) at different rates. A cluster motor 41019 (FIG. 4B) can rotate the wheel base in a forward / rearward direction. The rotation of the wheel base can allow cargo to rotate independently of the drive wheels 442101, if at all, while the front drive wheels 442101A are higher or lower than the rear drive wheels 442101B, for example, when encountering a discontinuous surface feature. The cluster assembly 42110 can operate each pair of two wheels 42105 independently, thereby providing, upon command, forward, reverse, and rotational movement of the device 42114. The cluster assembly 42110 can provide structural support for the pairs 42105.The cluster assembly 42110 provides the mechanical power to rotate the wheel drive assemblies together and can enable functions that rely on cluster assembly rotation, such as, but not limited to, climbing discontinuous surface features, various surface types, and uneven terrain. Further details regarding the operation of clustered wheels can be found in U.S. Patent Application No. 16,035,205 (Attorney Docket No. X80), filed July 13, 2018, and entitled "Mobility Device," which is incorporated herein by reference in its entirety.

[0181] Continuing with reference to FIG. 4A , configurations of device 42114 can be associated, including but not limited to, with mode 41033 (FIG. 4B) of device 42114. Device 42114 can operate in several of modes 41033 (FIG. 4B). In standard mode 10100-1 (FIG. 5E), device 42114B can operate on two of drive wheels 442101B and two of caster wheels 42103. Standard mode 10100-1 (FIG. 5E) can provide turning capability and mobility on relatively firm, level surfaces, such as, but not limited to, indoor environments, sidewalks, and pavements. In enhanced mode 10100-2 (FIG. 5E) or four-wheel mode, device 42114A / C can command four of the drive wheels 442101A / B, can be actively stabilized through onboard sensors, and can elevate and / or reorient the chassis 42112, casters 42103, and cargo. Four-wheel mode 10100-2 (FIG. 5E) can provide mobility in a variety of environments and enable device 42114A / C to climb steep inclines and navigate across soft, uneven terrain. In four-wheel mode 10100-2 (FIG. 5E), all four of drive wheels 442101A / B can be deployed and caster wheels 42103 can be retracted. Rotation of cluster 42110 can enable operation on uneven terrain, and drive wheels 442101A / B can navigate over and across discontinuous surface features. This functionality can provide device 42114A / C with mobility in a wide variety of outdoor environments. Device 42114B can operate on solid and stable, yet wet, outdoor surfaces. Frost heave and other natural phenomena can degrade outdoor surfaces, causing cracks and loose material. In four-wheel mode 10100-2 (FIG. 5E), device 42114A / C can operate on these degraded surfaces.Mode 41033 (FIG. 4B) is described in detail in U.S. Pat. No. 6,571,892 ('892), issued June 3, 2003, and entitled "Control System and Method," which is incorporated herein by reference in its entirety.

[0182] 4B , the system 41000 can drive the device 42114 ( FIG. 4A ) by processing inputs from the sensors 41031 and generating commands to the wheel motors 41017 to drive the wheels 442101 ( FIG. 4A ) and to the cluster motor 41019 to drive the cluster 42110 ( FIG. 4A ). The system 41000 can include, but is not limited to, a device processor 41014 and a power base processor 41016. The device processor 41014 can receive and process environmental data 41022 from the sensors 41031 and provide configuration information 40125 to the power base processor 41016. In some configurations, the device processor 41014 can include a sensor processor 41021, which can receive and process environmental data 41022 from the sensors 41031. The sensor 41031 can include, but is not limited to, a camera, as described herein. From these data, information about the traveling surface being traversed, for example, by the device 42114 (FIG. 4A) can be accumulated and processed. In some configurations, the traveling surface information can be processed in real time. The device processor 41014 can include a configuration processor 41023 that can determine, for example, a surface type 40121 from the environmental data 41022 being traversed by the device 42114 (FIG. 4A). The configuration processor 41023 can include, for example, a traveling surface processor 41029 (FIG. 4C) that can create, for example, a traveling surface classification layer, a traveling surface confidence layer, and an occupancy layer from the environmental data 41022. These data can be used by the power base processor 41016 to create movement commands 40127 and motor commands 40128, as described herein, and can be used by the global occupancy grid processor 41025 to update an occupancy grid that can be used for path planning. The configuration 40125 can be based, at least in part, on the surface type 40121.The surface type 40121 and mode 41033 can be used, at least in part, to determine occupancy grid information 41022, which can include the probability that a cell in the occupancy grid is occupied. The occupancy grid can, at least in part, enable the determination of possible paths that the device 42114 (FIG. 4A) can take.

[0183] 4B , the power base processor 41016 can receive configuration information 40125 from the device processor 41014 and process the configuration information 40125 along with other information, such as routing information. The power base processor 41016 can include a control processor 40325 that can create movement commands 40127 based on at least the configuration information 40125 and provide the movement commands 40127 to a motor drive processor 40326. The motor drive processor 40326 can generate motor commands 40128 that can direct and move the device 42114 ( FIG. 4A ). Specifically, the motor drive processor 40326 can generate motor commands 40128 that can drive the wheel motors 41017, and can generate motor commands 40128 that can drive the cluster motors 41019.

[0184] 4C , the real-time surface detection of the present teachings can include a configuration processor 41023, which may include, but is not limited to, a traveling surface processor 41029. The traveling surface processor 41029 can determine characteristics of the traveling surface over which the device 42114 ( FIG. 4A ) is navigating. The characteristics can be used to determine the future configuration of the device 42114 ( FIG. 4A ). The traveling surface processor 41029 can include, but is not limited to, a neural network processor 40207, data transformations 40215, 40219, and 40239, a layer processor 40241, and an occupancy grid processor 40242. Together, these components can generate information that can direct changes in the configuration of the device 42114 ( FIG. 4A ) and can enable modifications to the occupancy grid 40244 ( FIG. 4C ), which can inform path planning for the progression of the device 42114 ( FIG. 4A ).

[0185] 4C and 4D, the neural network processor 40207 can subject the environmental data 41022 (FIG. 4B) to a trained neural network that can indicate, for each point of data collected by the sensor 41031 (FIG. 4B), the type of surface that the point is likely to represent. The environmental data 41022 (FIG. 4B) can be received as, without limitation, a camera image 40202, where the camera can be associated with a camera property 40204. The camera image 40202 can include a 2D grid of points 40201 having an X resolution 40205 (FIG. 4D) and a Y resolution 40204 (FIG. 4D). In some configurations, the camera image 40202 can include an RGB-D image, where the X resolution 40205 (FIG. 4D) can include 40,640 pixels and the Y resolution 40204 (FIG. 4D) can include 40,480 pixels. In some configurations, the camera image 40202 can be converted into an image formatted according to the requirements of the selected neural network. In some configurations, the data can be normalized, scaled, and converted from 2D to 1D, which can improve the processing efficiency of the neural network. The neural network can be trained in many ways, including, but not limited to, training using RGB-D camera images. In some configurations, the trained neural network, represented in a neural network file 40209 (FIG. 4D), can be made available to the neural network processor 40207 through a direct connection to a processor executing the trained neural network or, for example, through a communication channel. In some configurations, the neural network processor 40207 can use the trained neural network file 40209 (FIG. 4D) to identify surface types 40121 (FIG. 4C) within the environmental data 41022 (FIG. 4B). In some configurations, surface types 40121 (FIG. 4C) can include, but are not limited to, non-driveable, hard driveable, soft driveable, and curb.In some configurations, surface types 40121 (FIG. 4C) can include, but are not limited to, unroadable / background, asphalt, concrete, brick, packed dirt, wood planks, gravel / pebbles, grass, mulch, sand, curb, solid metal, metal grid, tactile paving, snow / ice, and railroad tracks. The results of the neural network processing can include a surface classification grid 40303 of points 40213 having an X resolution 40205, a Y resolution 40203, and a center 40211. Each point 40213 in the surface classification grid 40303 can be associated with a likelihood of being a particular one of the surface types 40121 (FIG. 4C).

[0186] Continuing with reference to Figures 4C and 4D, the travel surface processor 41029 (Figure 4C) can include a 2D-to-3D transform 40215 that can backproject from the 2D surface classification grid 40303 (Figure 4D) in the 2D camera frame to a 3D image cube 40307 (Figure 4D) in 3D real-world coordinates as seen by the camera. The backprojection can restore the 3D nature of the 2D data and transform the 2D image from the RGB-D camera into the 3D camera frame 40305 (Figure 4C). Points 40233 (Figure 4D) in the cube 40307 (Figure 4D) can each be associated with a likelihood of being a particular one of the surface types 40121 (Figure 4C), as well as depth and X / Y coordinates. The dimensions of point cube 40307 (FIG. 4D) can be bounded according to camera properties 40204 (FIG. 4C), such as, for example, but not limited to, focal length x, focal length y, and center of projection 40225. For example, camera properties 40204 can include a maximum range over which the camera can reliably project. Additionally, there may be features of device 42114 (FIG. 4A) that can interfere with image 40202. For example, caster 42103 (FIG. 4A) may interfere with the view of camera 40227 (FIG. 4D). These factors can limit the number of points in point cube 40307 (FIG. 4D). In some configurations, camera 40227 (FIG. 4D) cannot reliably project beyond about 6 meters, which may represent an upper limit for the range of camera 40227 (FIG. 4D) and can limit the number of points in point cube 40307 (FIG. 4D). In some configurations, features of device 42114 (FIG. 4A) can act as a minimum limit for the range of camera 40227 (FIG. 4D). For example, the presence of caster 42103 (FIG. 4A) can imply a minimum limit, which in some configurations may be set at about 1 meter. In some configurations, points in point cube 40307 (FIG. 4D) can be limited to points 1 meter or more from camera 40227 (FIG. 4D) and 6 meters or less from camera 40227 (FIG. 4D).

[0187] 4C and 4D, the driving surface processor 41029 (FIG. 4C) can include a base link transform 40219 that can transform the 3D cube of points into coordinates associated with the device 42114 (FIG. 4A), i.e., the base link frame 40309 (FIG. 4C). The base link transform 40219 can transform a 3D data point 40223 (FIG. 4D) in cube 40307 (FIG. 4D) to a point 40233 (FIG. 4D) in cube 40308 (FIG. 4D), whose Z dimension is set to the base of the device 42114 (FIG. 4A). The travel surface processor 41029 (FIG. 4C) can include an OG preparation 40239 that can project points 40233 (FIG. 4D) in cube 40308 (FIG. 4D) onto an occupancy grid 40244 (FIG. 4D) as points 40237 (FIG. 4D) in cube 40311 (FIG. 4D). The layer processor 40241 can flatten points 40237 (FIG. 4D) into various layers 40312 (FIG. 4C) depending on the data represented by points 40237 (FIG. 4D). In some configurations, the layer processor 40241 can apply a scalar value to the layers 40312 (FIG. 4C). In some configurations, layer 40312 (FIG. 4C) can include the probability of occupancy layer 40243, a surface classification layer 40245 (FIG. 4D) as determined by neural network processor 40207, and a surface type confidence layer 40247 (FIG. 4D). In some configurations, surface type confidence layer 40247 (FIG. 4D) can be determined by converting class scores from neural network processor 40207 to scores that can be determined by normalizing the class scores to a probability distribution over the output classes as log(class score) / Σlog(each class). In some configurations, one or more layers can be replaced or augmented by a layer that provides a probability of an undriveable surface.

[0188] Continuing with reference to FIGS. 4C and 4D, in some configurations, the probability values ​​in occupancy layer 40243 may be expressed as log-odds (log-odds->ln(p / (1-p)) values. In some configurations, the probability values ​​in occupancy layer 40243 may be based on a combination of at least mode 41033 (FIG. 4B) and surface type 40121 (FIG. 4B). In some configurations, the preselected probability values ​​in occupancy layer 40243 may be, for example, without limitation, However, (1) when surface type 40121 (FIG. 4A) is hard and runnable and device 42114 (FIG. 4A) is in a preselected set of modes 41033 (FIG. 4B), or (2) when surface type 40121 (FIG. 4A) is soft and runnable and device 42114 (FIG. 4A) is in a specific preselected mode, such as standard mode, or (3) when surface type 40121 (FIG. 4A) is soft and runnable and device 42114 (FIG. 4A) is in a specific preselected mode, such as standard mode. The probability values ​​can be selected to cover situations such as (1) when device 42114 (FIG. 4A) is in a specific preselected mode, such as, for example, four-wheel mode, or (2) when surface type 40121 (FIG. 4A) is discontinuous and device 42114 (FIG. 4A) is in a specific preselected mode, such as, for example, standard mode, or (3) when surface type 40121 (FIG. 4A) is discontinuous and device 42114 (FIG. 4A) is in a specific preselected mode, such as four-wheel mode, or (4) when surface type 40121 (FIG. 4A) is discontinuous and device 42114 (FIG. 4A) is in a specific preselected mode, such as four-wheel mode, or (5) when surface type 40121 (FIG. 4A) is discontinuous and device 42114 (FIG. 4A) is in a specific preselected mode, such as four-wheel mode, or (6) when surface type 40121 (FIG. 4A) is non-drivable and device 42114 (FIG. 4A) is in a preselected set of modes 41033 (FIG. 4B). In some configurations, the probability values ​​can include, but are not limited to, those set forth in Table I. In some configurations, the neural network predicted probabilities can be adjusted, if necessary, to replace the probabilities listed in Table I. [Table 1]

[0189] 4C , the occupancy grid processor 40242 can provide parameters in real time to the global occupancy grid processor 41025 that can affect the probability values ​​of the occupancy grid 40244, such as, for example, but not limited to, surface type 40121 and occupancy grid information 41022. Configuration information 40125 ( FIG. 4B ), for example, but not limited to, mode 41033 and surface type 40121, can be provided to the power base processor 41016 ( FIG. 4B ). The power base processor 41016 ( FIG. 4B ) can determine motor commands 40128 ( FIG. 4B ), which can set the configuration of the device 42114 ( FIG. 4A ), based on at least the configuration information 40125 ( FIG. 4B ).

[0190] 4E and 4F, the device 42100A can be configured in accordance with the present teachings to operate in a standard mode. In standard mode, the caster 42103 and second drive wheel 42101B can rest on the ground as the device 42100A navigates its path. The first drive wheel 42101A can be elevated a preselected amount 42102 (FIG. 4F) to clear the driving surface. The device 42100A can navigate normally on relatively firm, level surfaces. When traveling in standard mode, the occupancy grid 40244 (FIG. 4C) can reflect surface type limitations (see Table I), thus enabling adaptive selection of modes or configuration changes based on the surface type and current mode.

[0191] 4G-4J, the device 42100B / C can be configured in accordance with the present teachings to operate in a four-wheel mode. In one configuration in the four-wheel mode, the first drive wheel 42101A and the second drive wheel 42101B can rest on the ground as the device 42100A navigates its path. The casters 42103 can be retracted and clear the driving surface by a preselected amount 42104 (FIG. 4H). In another configuration in the four-wheel mode, the first drive wheel 42101A and the second drive wheel 42101B can rest substantially on the ground as the device 42100A navigates its path. The casters 42103 can be retracted and the chassis 42111 can be rotated (thus moving the casters 42103 farther from the ground) to accommodate, for example, an uneven surface. In this configuration, the casters 42103 can clear the driving surface by a preselected amount 42108 (FIG. 1J). The device 42100A / C can navigate successfully on a variety of surfaces, including soft and discontinuous surfaces. In another configuration in four-wheel mode, the second drive wheel 42101B can rest on the ground as the device 42100A, while the first drive wheel 42101A can be elevated as the device 42100C (FIG. 1J) navigates its path. The casters 42103 can be retracted and the chassis 42111 can be rotated (thus moving the casters 42103 farther from the ground), for example, to accommodate the discontinuous surface. When driving in four-wheel mode, the occupancy grid 40244 (FIG. 4C) can reflect the surface type (see Table I), thus enabling adaptive selection of a mode or configuration changes based on the surface type and the current mode.

[0192] 4K, a method 40150 for real-time control of a device configuration of a device, such as, but not limited to, an AV, navigating a path based on at least one environmental factor and a device configuration may include, but is not limited to, receiving sensor data 40151, determining a surface type based at least on the sensor data 40153, and determining a current mode based at least on the surface type and the current device configuration 40155. The method 40150 may include determining a next device configuration 40157 based at least on the current mode and surface type, determining a movement command 40159 based at least on the next device configuration, and changing the current device configuration to the next device configuration 40161 based at least on the movement command.

[0193] Referring now primarily to FIG. 5A , annotated point data 10379 (FIG. 5B) can be provided to device controller 10111 to respond to objects appearing on the path of the AV. Annotated point data 10379 (FIG. 5B), which can be the basis for route information that can be used to instruct AV 10101 (FIG. 1A) to navigate a path, can include, but is not limited to, navigable edges, mapped trajectories, such as, but not limited to, mapped trajectories 10413 / 10415 (FIG. 5D), and labeled features, such as, but not limited to, SDSF 10377 (FIG. 5C). Mapped trajectories 10413 / 10415 (FIG. 5C) can include a graph of edges in the route space and initial weights assigned to portions of the route space. The graph of edges can include characteristics, such as, but not limited to, directionality and capacity, and edges can be categorized according to these characteristics. The mapped trajectory 10413 / 10415 (FIG. 5C) may include cost modifiers associated with surfaces in the route space and driving modes associated with edges. Driving modes may include, but are not limited to, path following and SDSF climbing. Other modes may include, for example, but are not limited to, operational modes such as autonomous, mapping, and waiting for intervention. Ultimately, a path may be selected based at least on lower cost modifiers. Features relatively far from the mapped trajectory 10413 / 10415 (FIG. 5C) may have higher cost modifiers and may receive less attention when forming a path. The initial weights may be adjusted while the AV 10101 (FIG. 1A) is operating, potentially causing path modifications. The adjusted weights may be used to adjust the edge / weight graph 10381 (FIG. 5B) and may be based at least on the current driving mode, the current surface, and the edge category.

[0194] 5A , the device controller 10111 can include a feature processor that can perform specific tasks related to incorporating the eccentricity of any feature into the path. In some configurations, the feature processor can include, but is not limited to, the SDSF processor 10118. In some configurations, the device controller 10111 can include, but is not limited to, the SDSF processor 10118, the sensor processor 10703, the mode controller 10122, and the base controller 10114, each of which are described herein. The SDSF processor 10118, the sensor processor 10703, and the mode controller 10122 can provide inputs to the base controller 10114.

[0195] Continuing with reference to FIG. 5A , the base controller 10114, based on inputs provided by at least the mode controller 10122, the SDSF processor 10118, and the sensor processor 10703, can determine information that the power base 10112 can use to cause the AV 10101 ( FIG. 1A ) to travel on a path determined by the base controller 10114 based at least on the edge / weighted graph 10381 ( FIG. 5B ). In some configurations, the base controller 10114 can ensure that the AV 10101 ( FIG. 1A ) follows a predetermined path from a start point to a destination and can modify the predetermined path based on at least external and / or internal conditions. In some configurations, external conditions can include, but are not limited to, stop signals, SDSFs, and obstacles in or near the path being traveled by the AV 10101 ( FIG. 1A ). In some configurations, internal conditions can include, but are not limited to, mode transitions that reflect the response the AV 10101 ( FIG. 1A ) makes to external conditions. The device controller 10111 can determine commands to send to the power base 10112 based on at least external and internal conditions. The commands can include, but are not limited to, speed and direction commands that can instruct the AV 10101 (FIG. 1A) to proceed in a commanded direction at a commanded speed. Other commands can include, for example, groups of commands that enable characteristic responses, such as SDSF climbing. The base controller 10114 can determine the desired speed between waypoints of the path by conventional methods, including, but not limited to, Interior Point Optimizer (IPOPT) large-scale nonlinear optimization (https: / / projects.coin-or.org / Ipopt). The base controller 10114 can determine at least the desired speed by conventional methods, including, but not limited to, Dijkstra's algorithm, A *The desired path can be determined based on conventional techniques, such as a search algorithm or a technique based on a breadth-first search algorithm. The base controller 10114 can form a box around the mapped trajectory 10413 / 10415 (FIG. 5C) to establish an area in which obstacle detection can be performed. The height of the payload carrier, when adjustable, can be adjusted based, at least in part, on the commanded speed.

[0196] 5A , the base controller 10114 can translate speed and direction determinations into motor commands. For example, without limitation, when encountering an SDSF, such as a curb or a slope, the base controller 10114 can instruct the power base 10112 in SDSF climb mode to raise the payload carrier 10173 ( FIG. 1A ), align the AV 10101 ( FIG. 1A ) with the SDSF at approximately a 90° angle, and reduce speed to a relatively low level. When the AV 10101 ( FIG. 1A ) climbs a substantially discontinuous surface, the base controller 10114 can instruct the power base 10112 to transition to a climb phase in which speed is increased as increased torque is required to move the AV 10101 ( FIG. 1A ) up the slope. When the AV 10101 (FIG. 1A) encounters a relatively level surface, the base controller 10114 can reduce speed to stay on any flat portions of the SDSF. In the case of a downward ramp associated with a flat portion, the base controller 10114 can allow speed to increase when the AV 10101 (FIG. 1A) begins to descend a substantially discontinuous surface and when both wheels are on the downward ramp. When an SDSF is encountered, such as, for example, but not limited to, a slope, the slope can be identified and treated as a structure. The structural feature can include, for example, a ramp of a preselected size. The ramp can include a slope of approximately 30° and, optionally, but not limited to, on both sides of a flat area. The device controller 10111 (FIG. 5A) can distinguish between an obstacle and a slope by comparing the angle of the perceived feature to an expected slope angle, which can be received from the sensor processor 10703 (FIG. 5A).

[0197] Referring now primarily to FIG. 5B, the SDSF processor 10118 can locate navigable edges from blocks of drivable surface formed by the polygonal mesh represented in the annotated point data 10379 that can be used to create paths for traversal by the AV 10101 (FIG. 1A). Within the SDSF buffer 10407 (FIG. 5C), which can form an area of ​​a preselected size around the SDSF line 10377 (FIG. 5C), the navigable edges can be erased for special handling, assuming SDSF traversal (see FIG. 5D). A closed line segment, such as segment 10409 (FIG. 5C), can be drawn to bisect the SDSF buffer 10407 (FIG. 5C) between a pair of previously determined SDSF points 10789 (FIG. 1N). In some configurations, for a closed line segment to be considered a candidate for SDSF traversal, the segment end 10411 (FIG. 5C) can fall on an unobstructed portion of the drivable surface, there can be sufficient room for the AV 10101 (FIG. 1A) to travel along the line segment between adjacent SDSF points 10789 (FIG. 1N), and the area between the SDSF points 10789 (FIG. 1N) can be the drivable surface. The segment end 10411 (FIG. 5C) can be connected to the underlying feature to form a vertex and a drivable edge. For example, line segments 10461, 10463, 10465, and 10467 (FIG. 5C), which meet the traversal criteria, are shown as part of the feature in FIG. 5D. In contrast, line segment 10409 (FIG. 5C) did not meet the criteria because, at least, segment end 10411 (FIG. 5C) does not fall on the drivable surface. The overlapping SDSF buffers 10506 (FIG. 5C) can exhibit SDSF discontinuities that can penalize SDSF traversal of the SDSFs within the overlapped SDSF buffers 10506 (FIG. 5C). The SDSF lines 10377 (FIG. 5C) can be smoothed, and the locations of the SDSF points 10789 (FIG. 1N) can be adjusted so that they fall a preselected distance apart, the preselected distance based at least on the footprint of the AV 10101 (FIG. 1A).

[0198] Continuing with reference to FIG. 5B , the SDSF processor 10118 can convert the annotated point data 10379 into an edge / weighted graph 10381, including morphological corrections for SDSF traversal. The SDSF processor 10118 can include a seventh processor 10601, an eighth processor 10702, a ninth processor 10603, and a tenth processor 10605. The seventh processor 10601 can convert coordinates of points in the annotated point data 10379 to a global coordinate system to achieve compatibility with GPS coordinates and generate a GPS-compatible dataset 10602. The seventh processor 10601 can generate the GPS-compatible dataset 10602 using conventional processes, such as, but not limited to, affine matrix transformations and PostGIS transformations. The World Geodetic System (WGS) can be used as a standard coordinate system because it takes into account the curvature of the Earth. Maps can be stored in the Universal Transverse Mercator (UTM) coordinate system and can be switched to WGS when it is necessary to find where a specific address is located.

[0199] Referring now primarily to FIG. 5C , the eighth processor 10702 ( FIG. 5B ) can smooth the SDSF, determine the boundary of the SDSF 10377, and create a buffer 10407 around the SDSF boundary, increasing the surface cost modifier as it moves away from the SDSF boundary. The mapped trajectory 10413 / 10415 can be a special-case lane with the lowest cost modifier. The lower cost modifier 10406 ​​can generally be located near the SDSF boundary, while the higher cost modifier 10408 can generally be located relatively far from the SDSF boundary. The eighth processor 10702 can provide the point cloud data with costs 10704 ( FIG. 5B ) to the ninth processor 10603 ( FIG. 5B ).

[0200] Continuing with reference primarily to FIG. 5C , the ninth processor 10603 ( FIG. 5B ) can calculate an approximately 90° approach 10604 ( FIG. 5B ) for the AV 10101 ( FIG. 1A ) to traverse SDSF 10377 that meets the criteria for labeling them as traversable. The criteria can include SDSF width and SDSF smoothness. Line segments, such as line segment 10409, can be created whose lengths indicate the minimum approach distance that the AV 10101 ( FIG. 1A ) may require to approach and exit SDSF 10377. Segment endpoints, such as endpoint 10411, can be integrated with underlying routing configurations. The criteria used to determine whether an SDSF approach is possible can eliminate the possibility of some approaches. SDSF buffers, such as SDSF buffer 10407, can be used to calculate valid approach and route configuration edge creation.

[0201] Referring again primarily to FIG. 5B , the tenth processor 10605 can create an edge / weight graph 10381 from the topology, which is a graph of edges and weights developed herein that can be used to calculate a path through the map. The topology can include cost modifiers and driving modes, and the edges can include directionality and capacity. The weights can be adjusted at runtime based on information from any number of sources. The tenth processor 10605 can provide at least one sequence of ordered points to the base controller 10114 to enable path generation, along with a recommended driving mode at a particular point. Each point in each sequence of points represents the location and labeling of a possible path point on the processed drivable surface. In some configurations, the labeling can indicate that the point represents a portion of a feature that may be encountered along the path, such as, for example, but not limited to, an SDSF. In some configurations, the feature can be further labeled with suggested processing based on the type of feature. For example, in some configurations, if a path point is labeled as an SDSF, further labeling can include a mode that can be interpreted by AV10101 (FIG. 1A) as a suggested driving instruction for AV10101 (FIG. 1A), such as switching AV10101 (FIG. 1A) to SDSF climb mode 100-31 (FIG. 5E) to enable AV10101 (FIG. 1A) to traverse SDSF10377 (FIG. 5C).

[0202] 5E, in some configurations, the mode controller 10122 can provide instructions to the base controller 10114 (FIG. 5A) for executing mode transitions. The mode controller 10122 can establish the mode in which the AV 10101 (FIG. 1A) is traveling. For example, the mode controller 10122 can provide a change of mode indication to the base controller 10114, e.g., change between a path following mode 10100-32 and an SDSF climb mode 10100-31, when an SDSF is identified along the travel path. In some configurations, the annotated point data 10379 (FIG. 5B) can include a mode identifier at various points along the route, e.g., when the mode is changed to accommodate the route. For example, if SDSF 10377 (FIG. 5C) is labeled in annotated point data 10379 (FIG. 5B), the device controller 10111 can determine the mode identifier associated with the route point and possibly adjust instructions to the power base 10112 (FIG. 5A) based on the desired mode. In addition to SDSF climb mode 10100-31 and path following mode 10100-32, in some configurations, the AV 10101 (FIG. 1A) can support operational modes that may include, but are not limited to, standard mode 10100-1 and enhanced (four-wheel) mode 10100-2, as described herein. The height of the payload carrier 10173 (FIG. 1A) can be adjusted to provide the necessary clearance over obstacles and along slopes.

[0203] Referring now to FIG. 5F , a method 11150 for navigating an AV toward a target point that intersects at least one SDSF may include, but is not limited to, receiving 11151 SDSF information related to an SDSF, a target point location, and an AV location. The SDSF information may include, but is not limited to, a set of points classified as SDSF points and an associated probability for each point that the point is an SDSF point. The method 11150 may include 11153 drawing a closed polygon encompassing the AV location, the target point location, and drawing a path line between the target point and the AV location. The closed polygon may include a preselected width. Table I includes possible ranges for the preselected variables discussed herein. The method 11150 may include 11155 selecting two of the SDSF points located within the polygon and 11157 drawing an SDSF line between the two points. In some configurations, the selection of the SDSF points may be random or in any other manner. If, at 11159, there are fewer than a first preselected number of points within a first preselected distance of the SDSF line, and if, at 11161, there are fewer than a second preselected number of attempts at selecting SDSF points and drawing lines between them and having fewer than the first preselected number of points around the SDSF line, method 11150 may include returning to step 11155. If, at 11161, there are a second preselected number of attempts at selecting SDSF points and drawing lines between them and having fewer than the first preselected number of points around the SDSF line, method 11150 may include conceding 11163 that no SDSF line was detected.

[0204] 5G , at 11159 (FIG. 5F), if there is a first preselected number of points or more, method 11150 may include fitting 11165 a curve to points that fall within a first preselected distance of the SDSF line. If, at 11167, the number of points within the first preselected distance of the curve exceeds the number of points within the first preselected distance of the SDSF line, and if, at 11171, the curve intersects with a path line, and if, at 11173, there are no gaps between points on the curve that exceed a second preselected distance, method 11150 may include identifying 11175 the curve as an SDSF line. If, at 11167, the number of points within a first preselected distance of the curve does not exceed the number of points within a first preselected distance of the SDSF line, or if, at 11171, the curve does not intersect with the path line, or if, at 11173, there are gaps between points on the curve that exceed a second preselected distance, and if, at 11177, the SDSF line does not remain stable, and if, at 11169, the curve fit has not been attempted more than a second preselected number of attempts, method 11150 may include returning to step 11165. An SDSF line that stabilizes is a result of subsequent iterations that result in the same or fewer points.

[0205] Referring now primarily to FIG. 5H , if the curve fit has been attempted for a second preselected number of attempts at 11169 (FIG. 5G), or if the SDSF line remains stable or deteriorates at 11177 (FIG. 5G), method 11150 may include receiving 11179 occupancy grid information. The occupancy grid may provide a probability that an obstacle is present at a point. The occupancy grid information may enhance the SDSF and route information found within a polygon that encompasses the AV route and SDSF when the occupancy grid includes data captured and / or calculated over a common geographic area with the polygon. Method 11150 may include selecting 11181 a point from the common geographic area and its associated probability. If, at 11183, the probability that an obstacle is present at the selected point is higher than a preselected percentage, and if, at 11185, the obstacle is located between the AV and the target point, and if, at 11186, the obstacle is less than a third preselected distance from the SDSF line between the SDSF line and the target point, method 11150 may include projecting 11187 the obstacle onto the SDSF line. If, at 11183, the probability that the location contains an obstacle is less than or equal to the preselected percentage, or if, at 11185, the obstacle is not located between the AV and the target point, or if, at 11186, the obstacle is located at a distance equal to or greater than the third preselected distance from the SDSF line between the SDSF and the target point, and if, at 11189, there are more obstacles to process, method 11150 may include restarting processing at step 11179.

[0206] Referring now primarily to FIG. 5I , at 11189 (FIG. 5H), if there are no more obstacles to process, method 11150 may include connecting the projections and finding 11191 endpoints of the connected projections along the SDSF line. Method 11150 may include marking 11193 a portion of the SDSF line between the projection endpoints as non-traversable. Method 11150 may include marking 11195 a portion of the SDSF line outside the non-traversable segment as traversable. Method 11150 may include turning 11197 the AV to within a fifth preselected amount perpendicular to the traversable segment of the SDSF line. At 11199, if the heading error relative to a line perpendicular to the traversable segment of the SDSF line exceeds the first preselected amount, method 11150 may include decelerating 11251 the AV by a ninth preselected amount. Method 11150 may include driving the AV forward toward the SDSF line and slowing down 11253 a second preselected amount per meter distance between the AV and the traversable SDSF line. If, at 11255, the distance of the AV from the traversable SDSF line is less than a fourth preselected distance, and if, at 11257, the heading error is greater than or equal to a third preselected amount relative to a line perpendicular to the SDSF line, method 11150 may include slowing down 11252 the AV a ninth preselected amount.

[0207] 5J , at 11257 (FIG. 5I), if the heading error is below a third preselected amount relative to a line perpendicular to the SDSF line, method 11150 may include ignoring the updated SDSF information and driving the AV at a preselected speed 11260. If the elevation of the front portion of the AV relative to the rear portion of the AV is between a sixth preselected amount and a fifth preselected amount, at 11259, method 11150 may include driving the AV forward and increasing the speed of the AV to an eighth preselected amount per degree of elevation 11261. If the elevation of the front portion of the AV relative to the rear portion of the AV is below the sixth preselected amount, at 11263, method 11150 may include driving the AV forward at a seventh preselected speed 11265. If the rear of the AV is greater than a fifth preselected distance from the SDSF line at 11267, the method 11150 may include acknowledging that the AV has completed traversing the SDSF 11269. If the rear of the AV is less than or equal to the fifth preselected distance from the SDSF line at 11267, the method 11150 may include returning to step 11260.

[0208] Referring now to FIG. 5K, a system 51100 for navigating an AV toward a target point that traverses at least one SDSF can include, but is not limited to, a path line processor 11103, an SDSF detector 11109, and an SDSF controller 11127. The system 51100 can be operatively coupled to a surface processor 11601, which can process sensor information, which can include, for example, but is not limited to, images of the AV 10101's (FIG. 5L) surroundings. The surface processor 11601 can provide real-time surface feature updates, including indications of SDSFs. In some configurations, a camera can provide RGB-D data, from which points can be classified according to surface type. In some configurations, the system 51100 can process points classified as SDSFs and their associated probabilities. The system 51100 can be operatively coupled to a system controller 11602, which can manage aspects of the operation of the AV 10101 (FIG. 5L). The system controller 11602 can maintain an occupancy grid 11138, which may include information from available sources regarding the navigable area near the AV 10101 (FIG. 5L). The occupancy grid 11138 may include the probability that an obstacle is present. This information, in conjunction with the SDSF information, can be used to determine whether the SDSF 10377 (FIG. 5N) can be traversed by the AV 10101 (FIG. 5L) without encountering an obstacle 11681 (FIG. 5M). The system controller 11602 can determine a speed limit 11148 that the AV 10101 (FIG. 5N) should not exceed, based on environmental and other information. The speed limit 11148 can be used as a guideline for the speed set by the system 51100 or can override it. The system 51100 can be operatively coupled to a base controller 10114, which can transmit driving commands 11144 generated by the SDSF controller 11127 to the driving components of the AV10101 (FIG. 5L).The proximal controller 10114 can provide information to the SDSF controller 11127 about the orientation of the AV 10101 (FIG. 5L) during SDSF traversal.

[0209] Continuing with reference to FIG. 5K, the path line processor 11103 can continuously receive surface classification points 10789 in real time, which may include, but are not limited to, points classified as SDSF. The path line processor 11103 can receive the location of the target point 11139 and the AV location 11141, for example, but not limited to, as indicated by the center 11202 (FIG. 5L) of the AV 10101 (FIG. 5L). The system 51100 can include a polygon processor 11105 that draws a polygon 11147 that encompasses the AV location 11141, the location of the target point 11139, and the path 11214 between the target point 11139 and the AV location 11141. The polygon 11147 can include a preselected width. In some configurations, the preselected width can include an approximate width of the AV 10101 (FIG. 5L). SDSF points 10789 that fall within polygon 11147 can be identified.

[0210] 5K , the SDSF detector 11109 can receive the surface classification points 10789, the path 11214, the polygon 11147, and the target points 11139 and can determine the most suitable SDSF line 10377 according to the criteria described herein that are available in the incoming data. The SDSF detector 11109 can include, but is not limited to, a point processor 11111 and an SDSF line processor 11113. The point processor 11111 can include selecting two of the SDSF points 10789 that are located within the polygon 11147 and drawing an SDSF 10377 line between the two points. If there are fewer than a first preselected number of points within a first preselected distance of the SDSF line 10377, and if there are fewer than a second preselected number of attempts at selecting an SDSF point 10789, drawing a line between the two points, and having fewer than the first preselected number of points around the SDSF line, the point processor 11111 may again loop through a select-draw-test loop as described herein. If there are a second preselected number of attempts at selecting an SDSF point, drawing a line between them, and having fewer than the first preselected number of points around the SDSF line, the point processor 11111 may include concluding that no SDSF line was detected.

[0211] Continuing with reference to FIG. 5K, the SDSF line processor 11113 may include fitting curves 11609-11611 (FIG. 5L) to points 10789 that fall within a first preselected distance of the SDSF line 10377, if a first preselected number or more of points 10789 exist. If the number of points 10789 within a first preselected distance of curve 11609-11611 (FIG. 5L) exceeds the number of points 10789 within a first preselected distance of SDSF line 10377, and if curve 11609-11611 (FIG. 5L) intersects with path line 11214, and if there are no gaps between points 10789 on curve 11609-11611 (FIG. 5L) that exceed a second preselected distance, SDSF line processor 11113 may include identifying curve 11609-11611 (FIG. 5L) as (for example) SDSF line 10377. If the number of points 10789 within the preselected distance of the curves 11609-11611 (Figure 5L) does not exceed the number of points 10789 within a first preselected distance of the SDSF line 10377, or if the curves 11609-11611 (Figure 5L) do not intersect with the path line 11214, or if there are gaps between the points 10789 on the curves 11609-11611 (Figure 5L) that exceed a second preselected distance, and if the SDSF line 10377 does not remain stable, and if the curve fit has not been attempted more than a second preselected number of times, the SDSF line processor 11113 can run the curve fit loop again.

[0212] Continuing with reference to FIG. 5K, the SDSF controller 11127 can receive the SDSF line 10377, the occupancy grid 11138, the AV orientation change 11142, and the speed limit 11148, and can generate SDSF commands 11144 to navigate the AV 10101 (FIG. 5L) to correctly traverse the SDSF 10377 (FIG. 5N). The SDSF controller 11127 can include, but is not limited to, an obstacle processor 11115, an SDSF approach 11131, and an SDSF traverse 11133. The obstacle processor 11115 can receive the SDSF line 10377, the target point 11139, and the occupancy grid 11138, and can determine, from among the obstacles identified in the occupancy grid 11138, whether any of them may obstruct the AV 10101 (FIG. 5N) as it traverses the SDSF 10377 (FIG. 5N). The obstacle processor 11115 may include, but is not limited to, an obstacle selector 11117, an obstacle tester 11119, and a traversal locator 11121. The obstacle selector 11117 may include, but is not limited to, receiving an occupancy grid 11138 as described herein. The obstacle selector 11117 may include selecting occupancy grid points and their associated probabilities from a geographic area that is common to both the occupancy grid 11138 and the polygon 11147. If the probability that an obstacle exists at a selected grid point is higher than a preselected percentage, and if the obstacle is located between AV10101 (Figure 5L) and target point 11139, and if the obstacle is less than a third preselected distance from SDSF line 10377 between SDSF line 10377 and target point 11139, obstacle tester 11119 may project the obstacle onto SDSF line 10377 and form a projection 11621 that intersects with SDSF line 10377.If the probability that the location contains an obstacle is less than or equal to a preselected percentage, or if the obstacle is not located between AV10101 (FIG. 5L) and the target point 11139, or if the obstacle is located at a distance equal to or greater than a third preselected distance from the SDSF line 10377 between the SDSF line 10377 and the target point 11139, the obstacle tester 11119 may include resuming execution in receiving the occupancy grid 11138 if there are further obstacles to process.

[0213] 5K, the traversal locator 11121 can include connecting the projection points and locating the endpoints 11622 / 11623 (FIG. 5M) of the connected projection 11621 (FIG. 5M) along the SDSF line 10377. The traversal locator 11121 can include marking the portion 11624 (FIG. 5M) of the SDSF line 10377 between the projection endpoints 11622 / 11623 (FIG. 5M) as non-traversable. The traversal locator 11121 can include marking the portion 11626 (FIG. 5M) of the SDSF line 10377 outside the non-traversable portion 11624 (FIG. 5M) as traversable.

[0214] Continuing with reference to FIG. 5K, the SDSF approach 11131 may include transmitting an SDSF command 11144 to turn the AV 10101 ( FIG. 5N ) to within a fifth preselected amount perpendicular to the traversable portion 11626 ( FIG. 5N ) of the SDSF line 10377. If the heading error relative to a perpendicular line 11627 ( FIG. 5N ), which is perpendicular to the traversable section 11626 ( FIG. 5N ) of the SDSF line 10377, exceeds a first preselected amount, the SDSF approach 11131 may include transmitting an SDSF command 11144 to slow down the AV 10101 ( FIG. 5N ) by a ninth preselected amount. In some configurations, the ninth preselected amount may range from very slow to completely stopped. The SDSF approach 11131 may include transmitting an SDSF command 11144 to drive the AV 10101 ( FIG. 5N ) forward toward the SDSF line 10377 and transmitting an SDSF command 11144 to slow the AV 10101 ( FIG. 5N ) by a second preselected amount per meter traveled. If the distance between the AV 10101 ( FIG. 5N ) and the traversable SDSF line 11626 ( FIG. 5N ) is less than a fourth preselected distance, and if the heading error is greater than or equal to a third preselected amount relative to a line perpendicular to the SDSF line 10377, the SDSF approach 11131 may include transmitting an SDSF command 11144 to slow the AV 10101 ( FIG. 5N ) by a ninth preselected amount.

[0215] 5K , if the heading error is below a third preselected amount relative to a line perpendicular to the SDSF line 10377, the SDSF traversal 11133 may include transmitting an SDSF command 11144 to ignore the updated SDSF information and run the AV 10101 ( FIG. 5N ) at a preselected rate. If the AV orientation change 11142 indicates that the climb of the leading edge 11701 ( FIG. 5N ) of the AV 10101 ( FIG. 5N ) relative to the trailing edge 11703 ( FIG. 5N ) of the AV 10101 ( FIG. 5N ) is between a sixth preselected amount and a fifth preselected amount, the SDSF traversal 11133 may include transmitting an SDSF command 11144 to run the AV 10101 ( FIG. 5N ) forward and to increase the speed of the AV 10101 ( FIG. 5N ) to a preselected rate per degree of climb. If the AV orientation change 11142 indicates that the elevation of the leading edge 11701 ( FIG. 5N ) relative to the trailing edge 11703 ( FIG. 5N ) of the AV 10101 ( FIG. 5N ) is below a sixth preselected amount, the SDSF traversal 11133 may include sending an SDSF command 11144 to drive the AV 10101 ( FIG. 5N ) forward at a seventh preselected speed. If the AV location 11141 indicates that the trailing edge 11703 ( FIG. 5N ) is above a fifth preselected distance from the SDSF line 10377, the SDSF traversal 11133 may include acknowledging that the AV 10101 ( FIG. 5N ) has completed traversing the SDSF 10377. If AV location 11141 indicates that trailing edge 11703 (FIG. 5N) is less than or equal to a fifth preselected distance from SDSF line 10377, SDSF traversal 11133 may include executing the loop again beginning with ignoring the updated SDSF information.

[0216] Some example ranges for the preselected values ​​described herein may include, but are not limited to, those outlined in Table II. [Table 2-1] Table 2-2

[0217] 5O, to support real-time data collection, in some configurations, a system of the present teachings can generate locations in three-dimensional space of various surface types in response to receiving data such as, for example, but not limited to, RGD-D camera image data. The system can rotate images 12155 and transform them from a camera coordinate system 12157 to a UTM coordinate system 12159. The system can generate polygon files from the transformed images, which can represent three-dimensional locations associated with the surface type 12161. A method 12150 for locating a feature 12151 from a camera image 12155 received by an AV10101, the AV10101 having a pose 12163, can include, but is not limited to, receiving, by the AV10101, the camera image 12155. Each camera image 12155 can include an image timestamp 12171, and each image 12155 can include image color pixels 12167 and image depth pixels 12169. The method 12150 can include receiving a pose 12163 of the AV 10101, the pose 12163 having a pose timestamp 12171, and determining a selected image 12173 by identifying an image from the camera images 12155 having an image timestamp 12165 that is closest to the pose timestamp 12171. The method 12150 may include separating image color pixels 12167 from image depth pixels 12169 in a selected image 12173, and determining an image surface classification 12161 for the selected image 12173 by providing the image color pixels 12167 to a first machine learning model 12177 and providing the image depth pixels 12169 to a second machine learning model 12179. The method 12150 may include determining perimeter points 12181 of a feature in the camera image 12173, the feature including feature pixels 12151 within the perimeter, each of the feature pixels 12151 having the same surface classification 12161, and each of the perimeter points 12181 having a set of coordinates 12157.The method 12150 may include converting each of the set of coordinates 12157 to a UTM coordinate 12159 .

[0218] The present teachings are directed to computer systems for performing the methods discussed in the description herein and to computer-readable media containing programs for performing these methods. Raw data and results can be stored, printed, displayed, transferred to another computer, and / or transferred to another location for future retrieval and processing. Communication links can be wired or wireless, using, for example, cellular, military, and satellite communication systems. Portions of the system can run on computers with varying numbers of CPUs. Other alternative computer platforms can also be used.

[0219] The present arrangements are also directed to software for performing the methods discussed herein and computer-readable media storing the software for performing these methods. The various modules described herein can be executed on the same CPU or on different computers. In accordance with the statute, the present arrangements have been described in language that is more or less specific with respect to structural and methodological features. However, it should be understood that the present arrangements are not limited to the specific features shown and described, since the means disclosed herein comprise preferred forms of embodying the present arrangements.

[0220] The method can be implemented, in whole or in part, electronically. Signals representing actions taken by elements of the system and other disclosed configurations can travel via at least one live communications network. Control and data information can be executed and stored electronically on at least one computer-readable medium. The system can be implemented to run on at least one computer node in at least one live communications network. General forms of the at least one computer-readable medium can include, for example, but are not limited to, a floppy disk, a flexible disk, a hard disk, a magnetic tape, or any other magnetic medium, a compact disk read-only memory or any other optical medium, a punch card, a paper tape, or any other physical medium with a pattern of holes, a random access memory, a programmable read-only memory, and an erasable programmable read-only memory (EPROM), a flash EPROM, or any other memory chip or cartridge, or any other medium from which a computer can read. Additionally, at least one computer-readable medium may contain graphs in any format, including but not limited to Graphics Interchange Format (GIF), Joint Photographic Experts Group (JPEG), Portable Network Graphics (PNG), Scalable Vector Graphics (SVG), and Tagged Image File Format (TIFF), as appropriate, subject to appropriate licensing.

[0221] While the present teachings have been described above in terms of specific configurations, it should be understood that they are not limited to these disclosed configurations. Numerous modifications and other configurations will occur to those skilled in the art to which this pertains and are intended to be and are covered by both this disclosure and the appended claims. It is intended that the scope of the present teachings should be determined by the proper interpretation and construction of the appended claims and their legal equivalents, as understood by those skilled in the art relying on the disclosure in this specification and the accompanying drawings.

Claims

1. 1. A method for real-time control of a configuration of a device, the device including a chassis, four wheels, a first side of the chassis operatively coupled to one of the four wheels, and a second side of the chassis operatively coupled to another one of the four wheels, the method comprising: receiving environmental data; determining a surface type based on the environmental data; and determining a mode based on the surface type and a first configuration; determining a second configuration based on the mode and the surface type; determining a movement command based on the second configuration; controlling a configuration of the device based on the movement command, the control including changing the device from the first configuration to the second configuration; determining the movement command based on an occupancy grid; Including, method.

2. The method of claim 1 , wherein controlling the configuration includes coordinated powering of the first pair of four-wheel clusters and the second pair of four-wheel clusters based on the environmental data.

3. 2. The method of claim 1, wherein controlling the configuration includes transitioning from driving the four wheels and a pair of casters configured to move up and down, the pair of casters being operably coupled to the chassis, to driving two wheels with a clustered first pair and a clustered second pair that are rotated to lift a first front wheel and a second front wheel, the device resting on a first rear wheel, a second rear wheel, and the pair of casters, the first clustered pair including the first front wheel and the first rear wheel, and the second clustered pair including the second front wheel and the second rear wheel.

4. 3. The method of claim 2, wherein controlling the configuration includes rotating a pair of clusters operatively coupled with a first two powered wheels on the first side and a second two powered wheels on the second side based on the environmental data.

5. The method of claim 1 , wherein the device further comprises a cargo container, the cargo container mounted on the chassis, and the chassis controls the height of the cargo container.

6. The method of claim 5 , wherein the height of the cargo container is based on the environmental data.

7. 1. A system for real-time control of a configuration of a device, the device including a chassis, four wheels, a first side of the chassis, and a second side of the chassis, the system comprising: a device processor configured to receive real-time environmental data surrounding the device, determine a surface type based on the real-time environmental data, determine a mode based on the surface type and a first configuration, and determine a second configuration based on the mode and the surface type; a power base processor that determines a movement command based on the second configuration, the power base processor controlling a configuration of the device based on the movement command to change the device from the first configuration to the second configuration; A system comprising: The power base processor determines the movement commands based on an occupancy grid.

8. 8. The system of claim 7, wherein controlling the configuration includes coordinated powering of the first pair of four wheel clusters and the second pair of four wheel clusters based on the real-time environmental data.

9. 8. The system of claim 7, wherein controlling the configuration includes transitioning from driving the four wheels and pairs of casters configured to move up and down, the pairs of casters being operably coupled to the chassis, to driving two wheels with a clustered first pair and a clustered second pair that are rotated to lift a first front wheel and a second front wheel, the device resting on a first rear wheel, a second rear wheel, and the pairs of casters, the first clustered pair including the first front wheel and the first rear wheel, and the second clustered pair including the second front wheel and the second rear wheel.

Images