System and method for real time control of autonomous device

Abstract

To provide a system and a method for real time control of an autonomous device.SOLUTION: An autonomous vehicle has sensors advantageously varied in capabilities, advantageously positioned, and advantageously impervious to environmental conditions. A system is executed on the autonomous vehicle and can receive a map including, for example, substantially discontinuous surface features along with data from the sensors, create an occupancy grid based upon the map and the data, and change the configuration of the autonomous vehicle based upon the type of surface on which the autonomous vehicle navigates. The device can safely navigate surfaces and surface features, including traversing discontinuous surfaces and other obstacles.SELECTED DRAWING: None

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This utility patent application is a continuation-in-part of U.S. patent application Ser. No. 16 / 800,497 (Attorney Docket No. AA164), filed Feb. 25, 2020, and entitled “System and Method for Surface Feature Detection and Traversal,” which is incorporated by reference in its entirety. This patent application claims the benefit of U.S. Provisional Patent Application No. 62 / 872,396 (Attorney Docket No. AA028), filed on July 10, 2019, and entitled “Apparatus for Long and Short Range Sensors on an Autonomous Delivery Device,” U.S. Provisional Patent Application No. 62 / 990,485 (Attorney Docket No. AA037), filed on March 17, 2020, and entitled “System and Method for Managing an Occupancy Grid,” and U.S. Provisional Patent Application No. 62 / 872,320 (Attorney Docket No. Z96), filed on July 10, 2019, and entitled “System and Method for Real-Time Control of the Configuration of an Autonomous Device.”

[0002] This application is incorporated herein by reference in its entirety, in accordance with U.S. Patent Application No. 16 / 035,205, filed July 13, 2018 and entitled "MOBILITY DEVICE" (Attorney Docket No. X80), U.S. Patent Application No. 15 / 787,613, filed October 18, 2017 and entitled "MOBILITY DEVICE" (Attorney Docket No. W10), U.S. Patent Application No. 15 / 600,703, filed May 20, 2017 and entitled "MOBILITY DEVICE" (Attorney Docket No. U22), U.S. Patent Application No. 15 / 600,703, filed May 17, 2018 and entitled "SYSTEM AND METHOD FOR SECURE REMOTE CONTROL OF A MEDICAL No. 15 / 982,737 (Attorney Docket No. X55), entitled "MOBILITY DEVICE", filed on July 15, 2017, U.S. Provisional Application No. 62 / 532,993 (Attorney Docket No. U30), entitled "MOBILITY DEVICE IMPROVEMENTS", filed on September 15, 2017, U.S. Provisional Application No. 62 / 559,263 (Attorney Docket No. V85), entitled "MOBILITY DEVICE SEAT", and U.S. Provisional Application No. 62 / 581,670 (Attorney Docket No. W07), entitled "MOBILITY DEVICE SEAT", filed on November 4, 2017.

[0003] The present teachings relate generally to AVs and, more specifically, to autonomous route planning, global occupancy grid management, on-vehicle sensors, surface feature detection and traversal, and real-time vehicle configuration changes. [Background technology]

[0004] (background) Navigation of AVs and semi-autonomous vehicles (AVs) typically relies on long-range sensors, including, for example, but not limited to, LIDAR, cameras, stereo cameras, and radar. Long-range sensors can sense 4-100 meters from the AV. In contrast, object avoidance and / or surface detection typically relies on short-range sensors, including, for example, but not limited to, stereo cameras, short-range radar, and ultrasonic sensors. These short-range sensors typically observe an area or volume of about 5 meters around the AV. The sensors can, for example, enable the AV to orient within its environment, as well as navigate roads, sidewalks, obstacles, and open spaces to reach a desired destination. The sensors can also enable visualization of people, signs, traffic signals, obstacles, and surface features.

[0005] Surface feature traversal can be a challenge because surface features, such as but not limited to, substantially discontinuous surface features (SDSFs), can be found in heterogeneous forms, and the forms can be unique to a specific geography. However, SDSFs, such as but not limited to slopes, edges, curbs, steps, and curb-like geometries (referred to herein in a non-limiting manner as SDSFs or simply surface features), can contain some typical characteristics that can help identify them. Surface / road conditions and surface types can be recognized and classified, for example, by fusing multi-sensory data, which can be complex and costly. Surface features and conditions can be used to control the physical reconfiguration of the AV in real time.

[0006] The sensors can be used to enable the creation of an occupancy grid that can represent the world for route planning purposes for AVs. Route planning requires a grid that identifies spaces as free, occupied, or unknown. However, the probability that a space is occupied can improve decision making regarding the space. A log-odds representation of the probability can be used to increase the precision in the numerical bounds of the 0 and 1 probabilities. The probability that a cell is occupied can depend on at least the new sensor information, the previous sensor information, and the prior occupancy information.

[0007] What is needed is a system that combines aggregated and real-time sensor data with changes in the vehicle's physical configuration to accomplish variable terrain traversal. What is needed is advantageous sensor placement to accomplish physical configuration changes, variable terrain traversal, and object avoidance. What is needed is the ability to localize SDSFs based on a multipart model associated with several criteria for SDSF identification. What is needed is determining candidate surface feature traversals based on criteria such as candidate traversal approach angles, candidate traversal driving surfaces on either side of the candidate surface feature, and real-time determination of candidate traversal path obstacles. What is needed is a system and method for incorporating drivable surface and device mode information into occupancy grid determination. Summary of the Invention [Means for solving the problem]

[0008] (summary) An AV of the present teachings can autonomously navigate to a desired location. In some configurations, the AV can include a device controller, including a sensor, a perception subsystem, an autonomy subsystem, and a driver subsystem, a power base, four powered wheels, two caster wheels, and a cargo container. In some configurations, the perception and autonomy subsystem can receive and process sensor information (perception) and map information (perception and autonomy) and can provide instructions to the driver subsystem. The map information can include surface classifications and associated device modes. The movement of the AV, controlled by the driver subsystem and enabled by the power base, can be sensed by the sensor subsystem and provide a feedback loop. In some configurations, the SDSF can be accurately identified from point cloud data and stored in a map, for example, according to the process described herein. The portion of the map associated with the location of the AV can be provided to the AV during navigation. The perception subsystem can maintain an occupancy grid that can inform the AV about the probability that the path to be traversed is currently occupied. In some configurations, the AV can operate in multiple distinct modes. Modes can enable complex terrain traversal, among other benefits: A combination of maps (e.g., surface classification), sensor data (sensing features surrounding the AV), occupancy grids (probability that an upcoming route point is occupied), and modes (whether difficult terrain can be traversed) can be used to identify the AV's direction, configuration, and speed.

[0009] With regard to preparing a map, in some configurations, a method of the present teachings for creating a map for navigating at least one SDSF encountered by an AV, wherein the AV travels a path over a surface, the surface including at least one SDSF, the path including a start point and an end point, the method can include, without limitation, accessing point cloud data representing the surface, filtering the point cloud data, forming the filtered point cloud data into a processable portion, and merging the processable portion into at least one concave polygon. The method can include locating and labeling the at least one SDSF within the at least one concave polygon. The locating and labeling can form the labeled point cloud data. The method can include creating a graphing polygon based at least on the at least one concave polygon, and plotting a route from the start point to the end point based at least on the graphing polygon. When navigating, the AV can traverse at least one SDSF along the path.

[0010] Filtering the point cloud data can optionally include conditionally removing points representing transient objects and points representing outliers from the point cloud data and replacing the removed points with a preselected height. Forming the processing portions can optionally include segmenting the point cloud data into processable portions and removing points at preselected heights from the processable portions. Merging the processable portions can optionally include reducing a size of the processable portions by analyzing outliers, voxels, and normals, growing a region from the reduced-sized processable portions, determining an initial drivable surface from the grown region, segmenting and meshing the initial drivable surface, locating polygons in the segmented and meshed initial drivable surface, and setting the drivable surface based at least on the polygons. Locating and labeling the at least one SDSF feature may optionally include sorting the point cloud data of the drivable surface according to an SDSF filter, the SDSF filter including at least three categories of points, and locating the at least one SDSF point based at least on whether the categories of points, in combination, satisfy at least one first preselected criterion. The method may optionally include creating at least one SDSF trajectory based at least on whether a plurality of the at least one SDSF points, in combination, satisfy at least one second preselected criterion. Creating the graphing polygon may further optionally include creating at least one polygon from the at least one drivable surface. The at least one polygon may include an edge. Creating the graphing polygon may include smoothing the edge, forming a driving margin based on the smoothed edge, adding the at least one SDSF trajectory to the at least one drivable surface, and removing the edge from the at least one drivable surface according to at least one third preselected criterion. Smoothing the edges can optionally include trimming the edges outwardly.Providing a smoothed edge running margin can optionally include inwardly trimming the outward facing edge.

[0011] In some configurations, a system of the present teachings for creating a map for navigating at least one SDSF encountered by an AV, where the AV travels a path over a surface, the surface including the at least one SDSF, the path including a start point and an end point, the system can include, without limitation, a first processor that accesses point cloud data representing the surface, a first filter that filters the point cloud data, a second processor that forms a processable portion from the filtered point cloud data, a third processor that merges the processable portion into at least one concave polygon, a fourth processor that locates and labels the at least one SDSF within the at least one concave polygon, where the locating and labeling forms the labeled point cloud data, a fifth processor that creates a graphing polygon, and a path selector that selects a path from the start point to the end point based at least on the graphing polygon. The AV can traverse the at least one SDSF along the path.

[0012] The first filter may optionally include executable code that may include, but is not limited to, conditionally removing points representing transient objects and points representing outliers from the point cloud data and replacing the removed points with a preselected height. The segmenter may optionally include executable code that may include, but is not limited to, segmenting the point cloud data into processable portions and removing points at preselected heights from the processable portions. The third processor may optionally include executable code that may include, but is not limited to, reducing a size of the processable portion by analyzing outliers, voxels, and normals, growing a region from the reduced-sized processable portion, determining an initial drivable surface from the grown region, segmenting and meshing the initial drivable surface, locating polygons in the segmented and meshed initial drivable surface, and setting the drivable surface based at least on the polygons. The fourth processor may optionally include executable code that may include, but is not limited to, sorting the drivable surface point cloud data according to an SDSF filter, the SDSF filter including at least three categories of points, and locating at least one SDSF point based at least on whether the categories of points, in combination, satisfy at least one first preselected criterion. The system may optionally include executable code that may include, but is not limited to, creating at least one SDSF trajectory based at least on whether a plurality of at least one SDSF point, in combination, satisfies at least one second preselected criterion.

[0013] Creating a graphed polygon may optionally include executable code that may include, but is not limited to, creating at least one polygon from the at least one drivable surface, the at least one polygon including an edge, smoothing the edge, forming a driving margin based on the smoothed edge, adding at least one SDSF trajectory to the at least one drivable surface, and removing the edge from the at least one drivable surface according to at least one third preselected criterion. Smoothing the edge may optionally include executable code that may include, but is not limited to, trimming the edge outward. Forming a running margin of the smoothed edge may optionally include executable code that may include, but is not limited to, trimming the outward edge inward.

[0014] In some configurations, a method of the present teachings for creating a map for navigating at least one SDSF encountered by an AV, where the AV travels a path over a surface, the surface including at least one SDSF, the path including a start point and an end point, the method can include, without limitation, accessing a route configuration. The route configuration can include at least one graphed polygon that can include filtered point cloud data. The point cloud data can include labeled features and a driveable margin. The method can include transforming the point cloud data to a global coordinate system, determining a boundary of the at least one SDSF, creating an SDSF buffer of a preselected size around the boundary, determining at least one SDSF that can be traversed based at least on at least one SDSF traversal criterion, creating an edge / weighted graph based at least on the at least one SDSF traversal criterion, the transformed point cloud data, and the route configuration, and plotting a route from the start point to a destination point based at least on the edge / weighted graph.

[0015] The at least one SDSF crossing criterion may optionally include a preselected width of the at least one SDSF and a preselected smoothness of the at least one SDSF, a minimum entry and exit distance between the at least one SDSF and the AV, including a drivable surface, and a minimum entry distance between the at least one SDSF and the AV that may accommodate an approximately 90° approach by the AV to the at least one SDSF.

[0016] In some configurations, a system of the present teachings for creating a map for navigating at least one SDSF encountered by an AV, where the AV travels a path over a surface, the surface including at least one SDSF, the path including a start point and an end point, the system can include, without limitation, a sixth processor that accesses a route configuration. The route configuration can include at least one graphed polygon that can include filtered point cloud data. The point cloud data can include labeled features and a driveable margin. The system can include a seventh processor that transforms the point cloud data to a global coordinate system and an eighth processor that determines a boundary of the at least one SDSF. The eighth processor can create an SDSF buffer of a preselected size around the boundary. The system may include a ninth processor that determines at least one SDSF that may be traversed based on at least one SDSF traversal criterion; a tenth processor that creates an edge / weighted graph based on at least the at least one SDSF traversal criterion, the transformed point cloud data, and the route morphology; and a base controller that plans a route from the start point to the destination point based on at least the edge / weighted graph.

[0017] In some configurations, a method of the present teachings for creating a map for navigating at least one SDSF encountered by an AV, where the AV travels a path over a surface, the surface including at least one SDSF, the path including a start point and an end point, the method can include, without limitation, accessing point cloud data representing the surface. The method can include filtering the point cloud data, forming the filtered point cloud data into a processable portion, and merging the processable portion into at least one concave polygon. The method can include locating and labeling the at least one SDSF within the at least one concave polygon. The locating and labeling can form the labeled point cloud data. The method can include creating a graphing polygon based at least on the at least one concave polygon. The graphing polygon can form a route configuration, and the point cloud data can include the labeled features and the driveable margin. The method may include transforming the point cloud data to a global coordinate system, determining a boundary of at least one SDSF, creating an SDSF buffer of a preselected size around the boundary, determining at least one SDSF that may be traversed based at least on at least one SDSF traversal criterion, creating an edge / weighted graph based at least on the at least one SDSF traversal criterion, the transformed point cloud data, and a route morphology, and planning a route from a start point to a destination point based at least on the edge / weighted graph.

[0018] Filtering the point cloud data can optionally include conditionally removing points representing transient objects and points representing outliers from the point cloud data and replacing the removed points with a preselected height. Forming the processing portions can optionally include segmenting the point cloud data into processable portions and removing points at preselected heights from the processable portions. Merging the processable portions can optionally include reducing a size of the processable portions by analyzing outliers, voxels, and normals, growing a region from the reduced-sized processable portions, determining an initial drivable surface from the grown region, segmenting and meshing the initial drivable surface, locating polygons in the segmented and meshed initial drivable surface, and setting the drivable surface based at least on the polygons. Locating and labeling the at least one SDSF feature may optionally include sorting the point cloud data of the drivable surface according to an SDSF filter, the SDSF filter including at least three categories of points, and locating the at least one SDSF point based at least on whether the categories of points, in combination, satisfy at least one first preselected criterion. The method may optionally include creating at least one SDSF trajectory based at least on whether a plurality of the at least one SDSF points, in combination, satisfy at least one second preselected criterion. Creating the graphing polygon may further optionally include creating at least one polygon from the at least one drivable surface. The at least one polygon may include an edge. Creating the graphing polygon may include smoothing the edge, forming a driving margin based on the smoothed edge, adding the at least one SDSF trajectory to the at least one drivable surface, and removing the edge from the at least one drivable surface according to at least one third preselected criterion. Smoothing the edges can optionally include trimming the edges outwardly.Forming the smoothed edge running margin can optionally include inwardly trimming the outward edge. The at least one SDSF crossing criterion can optionally include a preselected width of the at least one SDSF and a preselected smoothness of the at least one SDSF, a minimum entry and exit distance between the at least one SDSF and the AV that includes a drivable surface, and a minimum entry distance between the at least one SDSF and the AV that can accommodate an approximately 90° approach by the AV to the at least one SDSF.

[0019] In some configurations, a system of the present teachings for creating a map for navigating at least one SDSF encountered by an AV, where the AV travels a path over a surface, the surface including at least one SDSF, the path including a start point and an end point, the system can include, without limitation, a point cloud accessor that accesses point cloud data representing the surface, a first filter that filters the point cloud data, a segmenter that forms a processable portion from the filtered point cloud data, a third processor that merges the processable portion into at least one concave polygon, a fourth processor that locates and labels the at least one SDSF within the at least one concave polygon, the locating and labeling forming a labeled point cloud data, and a fifth processor that creates a graphing polygon. The route configuration can include at least one graphing polygon that can include the filtered point cloud data. The point cloud data can include labeled features and a driveable margin. The system may include a seventh processor that transforms the point cloud data to a global coordinate system and an eighth processor that determines a boundary of at least one SDSF. The eighth processor may create an SDSF buffer of a preselected size around the boundary. The system may include a ninth processor that determines at least one SDSF that may be traversed based at least on the at least one SDSF traversal criterion, a tenth processor that creates an edge / weight graph based at least on the at least one SDSF traversal criterion, the transformed point cloud data, and a route morphology, and a base controller that plans a route from a start point to a destination point based at least on the edge / weight graph.

[0020] The first filter may optionally include executable code that may include, but is not limited to, conditionally removing points representing transient objects and points representing outliers from the point cloud data and replacing the removed points with a preselected height. The segmenter may optionally include executable code that may include, but is not limited to, segmenting the point cloud data into processable portions and removing points at preselected heights from the processable portions. The third processor may optionally include executable code that may include, but is not limited to, reducing a size of the processable portion by analyzing outliers, voxels, and normals, growing a region from the reduced-sized processable portion, determining an initial drivable surface from the grown region, segmenting and meshing the initial drivable surface, locating polygons in the segmented and meshed initial drivable surface, and setting the drivable surface based at least on the polygons. The fourth processor may optionally include executable code that may include, but is not limited to, sorting the drivable surface point cloud data according to an SDSF filter, the SDSF filter including at least three categories of points, and locating at least one SDSF point based at least on whether the categories of points, in combination, satisfy at least one first preselected criterion. The system may optionally include executable code that may include, but is not limited to, creating at least one SDSF trajectory based at least on whether a plurality of at least one SDSF point, in combination, satisfies at least one second preselected criterion.

[0021] Creating a graphed polygon may optionally include executable code that may include, but is not limited to, creating at least one polygon from the at least one drivable surface, the at least one polygon including an edge, smoothing the edge, forming a driving margin based on the smoothed edge, adding at least one SDSF trajectory to the at least one drivable surface, and removing the edge from the at least one drivable surface according to at least one third preselected criterion. Smoothing the edge may optionally include executable code that may include, but is not limited to, trimming the edge outward. Forming a running margin of the smoothed edge may optionally include executable code that may include, but is not limited to, trimming the outward edge inward.

[0022] In some configurations, the SDSF can be identified by its dimensions. For example, a curb may include, but is not limited to, a width of approximately 0.6-0.7 m. In some configurations, point cloud data can be processed to locate the SDSF, which can be used to prepare a route for the AV from a starting point to a destination. In some configurations, the route can be included in a map and provided to a perception subsystem. As the AV progresses along the route, in some configurations, SDSF traversal can be adapted, in part, through sensor-based positioning of the AV, enabled by the perception subsystem. The perception subsystem can be executed on at least one processor within the AV.

[0023] The AV may include a power base, including, but not limited to, two powered front wheels, two powered rear wheels, and an energy storage device, and at least one processor. The power base may be configured to move at a commanded speed. The AV may include a cargo platform mechanically attached to the power base and including a plurality of short-range sensors. The AV may, in some configurations, include a cargo container mounted on the cargo platform and having a volume for receiving one or more objects to be delivered. The AV may, in some configurations, include a long-range sensor suite mounted on the cargo container and may include, but is not limited to, a LIDAR and one or more cameras. The AV may include a controller that may receive data from the long-range and short-range sensor suites.

[0024] The short-range sensor suite can optionally detect at least one characteristic of the drivable surface and can optionally include a stereo camera, an IR projector, two image sensors, an RGB sensor, and a radar sensor. The short-range sensor suite can optionally provide RGB-D data to the controller. The controller can optionally determine a geometry of the road surface based on the RGB-D data received from the short-range sensor suite. The short-range sensor suite can optionally detect objects within 4 meters of the AV, and the long-range sensor suite can optionally detect objects greater than 4 meters from the AV.

[0025] The perception subsystem can use the data collected by the sensors to populate the occupancy grid. The occupancy grid of the present teachings is configured as a 3D grid of points surrounding the AV, where the AV can occupy a center point. In some configurations, the occupancy grid can extend 10 m to the left, right, rear, and front of the AV. The grid can include approximately the height of the AV and virtually travel with the AV as it moves, representing obstacles surrounding the AV. The grid can be converted to two dimensions by shrinking its vertical axis, for example, but not by way of limitation, divided into polygons approximately 5 cm by 5 cm in size. Obstacles that appear in the 3D space around the AV can be reduced to 2D shapes. If the 2D shape overlaps any segment of one of the polygons, the polygon can be given a value of 100, indicating that the space is occupied. Any polygons that remain unfilled can be given a value of 0, and can be referred to as free space in which the AV may move.

[0026] As an AV navigates, it may encounter situations that may require a change in the AV's configuration. A method of the present teachings for real-time control of a configuration of a device, in some configurations, the device includes a chassis, at least four wheels, a first side of the chassis operably coupled to at least one of the at least four wheels, and an opposing second side of the chassis operably coupled to at least one of the at least four wheels, the method may include, without limitation, receiving environmental data, determining a surface type based at least on the environmental data, determining a mode based at least on the surface type and the first configuration, determining a second configuration based at least on the mode and surface type, determining a movement command based on at least the second configuration, and controlling the configuration of the device by using the movement command to change the device from the first configuration to the second configuration.

[0027] The method may optionally include capturing an occupancy grid based on at least a surface type and a mode. The environmental data may optionally include RGB-D image data and a morphology of a road surface. The configuration may optionally include two clustered pairs of at least four wheels. A first pair of the two pairs may be located on a first side and a second pair of the two pairs may be located on a second side. The first pair may include a first front wheel and a first rear wheel, and the second pair may include a second front wheel and a second rear wheel. The control of the configuration may optionally include coordinated powering of the first pair and the second pair based at least on the environmental data. The control of the configuration may optionally include transitioning from driving the at least four wheels and a pair of casters that are retracted to driving two wheels with the clustered first pair and the clustered second pair rotated to lift the first front wheel and the second front wheel. The pair of casters can be operatively coupled to the chassis. The device can be resting on the first rear wheel, the second rear wheel, and the pair of casters. Controlling the configuration can optionally include rotating a pair of clusters operatively coupled with the two powered wheels on the first side and the two powered wheels on the second side based on at least the environmental data.

[0028] A system of the present teachings for real-time control of a configuration of an AV can include, without limitation, a device processor and a power base processor. The AV can include a chassis, at least four wheels, a first side of the chassis, and an opposing second side of the chassis. The device processor can receive real-time environmental data surrounding the AV, determine a surface type based at least on the environmental data, determine a mode based at least on the surface type and the first configuration, and determine a second configuration based at least on the mode and surface type. The power base processor can enable the AV to move based on at least the second configuration and can enable the AV to change from the first configuration to the second configuration. The device processor can optionally include populating an occupancy grid based at least on the surface type and the mode.

[0029] During navigation, the AV may encounter an SDSF that may require the AV to steer for successful traversal. In some configurations, a method of the present teachings for navigating an AV along a path line in a going area toward a target point that crosses at least one SDSF, the AV including a leading edge and a trailing edge, the method may include, without limitation, receiving SDSF information and obstacle information regarding the going area, detecting at least one candidate SDSF from the SDSF information, and selecting an SDSF line from the at least one candidate SDSF line based on at least one selection criterion. The method may include determining at least one traversable portion of the selected SDSF line based on at least one location of at least one obstacle found in the obstacle information in the vicinity of the selected SDSF line, orienting the AV toward the at least one traversable portion by turning the AV to travel along a line perpendicular to the traversable portion and operating at a first speed, and constantly correcting the orientation of the AV based on a relationship between the orientation and the perpendicular line. The method can include traveling the AV at a second speed by adjusting a first speed of the AV based on at least the heading and a distance between the AV and the traversable portion. If an SDSF associated with the at least one traversable portion is elevated relative to a surface of the travel route, the method can include crossing the SDSF by elevating the leading edge relative to the trailing edge and traveling the AV at a third increased speed per degree of elevation, and traveling the AV at a fourth speed until the AV clears the SDSF.

[0030] Detecting at least one candidate SDSF from the SDSF information may optionally include (a) drawing a closed polygon encompassing the location of the AV and the location of the target point, (b) drawing a path line between the target point and the AV location, (c) selecting two SDSF points from the SDSF information, the SDSF points being located within the polygon, and (d) drawing an SDSF line between the two points. Detecting at least one candidate SDSF may include (e) repeating steps (c)-(e) if there are less than a first preselected number of points within a first preselected distance of the SDSF line and if there are less than a second preselected number of attempts in selecting SDSF points, drawing a line between them and having less than the first preselected number of points around the SDSF line. Detecting at least one candidate SDSF may include (f) fitting a curve to the SDSF points that fall within a first preselected distance of the SDSF line if a first preselected number of points or more exist, and (g) identifying the curve as an SDSF line if a first number of SDSF points within the first preselected distance of the curve exceeds a second number of SDSF points within the first preselected distance of the SDSF line and if the curve intersects with a path line and if there are no gaps between the SDSF points on the curve that exceed the second preselected distance. Detecting at least one candidate SDSF may include (h) repeating steps (f)-(h) if the number of points within a first preselected distance of the curve does not exceed the number of points within a first preselected distance of the SDSF line, or if the curve does not intersect the path line, or if there are gaps between SDSF points on the curve that exceed a second preselected distance, and if the SDSF line does not remain stable, and if steps (f)-(h) have not been attempted more than a second preselected number of attempts.

[0031] The closed polygon may optionally include a preselected width, which may optionally include a width dimension of the AV. Selecting the SDSF points may optionally include random selection. The at least one selection criterion may optionally include a first number of SDSF points within a first preselected distance of the curve exceeding a second number of SDSF points within a first preselected distance of the SDSF line, the curve intersecting the path line, and no gaps exist between the SDSF points on the curve that exceed a second preselected distance.

[0032] Determining at least one traversable portion of the selected SDSF can optionally include selecting a plurality of obstacle points from the obstacle information. Each of the plurality of obstacle points can include a probability that the obstacle point is associated with at least one obstacle. Determining at least one traversable portion can include projecting the plurality of obstacle points onto the SDSF line to form at least one projection if the probability is higher than a preselected percentage and if any of the plurality of obstacle points are located between the SDSF line and the target point and if any of the plurality of obstacle points are less than a third preselected distance from the SDSF line. Determining at least one traversable portion can optionally include connecting at least two of the at least one projection to each other, locating end points of the at least two connected projections along the SDSF line, marking the at least two connected projections as non-traversable SDSF segments, and marking the SDSF line outside the non-traversable segments as at least one traversable segment.

[0033] Traversing at least one traversable portion of the SDSF may optionally include turning the AV to travel along a line perpendicular to the traversable portion, orienting the AV toward the traversable portion and operating at a first speed, constantly correcting an orientation of the AV based on a relationship between the orientation and the perpendicular line, and traveling the AV at a second speed by adjusting the first speed of the AV based on at least the orientation and the distance between the AV and the traversable portion. Traversing at least one traversable portion of the SDSF may optionally include crossing the SDSF by elevating the leading edge relative to the trailing edge and traveling the AV at a third increased speed per degree of elevation if the SDSF rises relative to a surface of the travel route, and traveling the AV at a fourth speed until the AV has cleared the SDSF.

[0034] Traversing at least one traversable portion of the SDSF may alternatively and optionally include: (a) ignoring updates to the SDSF information and driving the AV at a preselected speed if the orientation error is below a third preselected amount relative to a line perpendicular to the SDSF line; (b) driving the AV forward and increasing the speed of the AV to an eighth preselected speed per degree of climb if the elevation of the front portion of the AV relative to the rear portion of the AV is between a sixth preselected amount and a fifth preselected amount; (c) driving the AV forward at a seventh preselected speed if the front portion is elevated below the sixth preselected amount relative to the rear portion; and (d) repeating steps (a)-(d) if the rear portion is less than or equal to a fifth preselected distance from the SDSF line.

[0035] In some configurations, the SDSF and AV wheels can be automatically aligned to avoid system instability. Automatic alignment can be implemented, for example, but not limited to, by continuously testing and correcting the orientation of the AV as it approaches the SDSF. Another aspect of the SDSF traversal feature of the present teachings is that it automatically verifies that sufficient free space exists around the SDSF before attempting to traverse. Yet another aspect of the SDSF traversal feature of the present teachings is that it is capable of traversing SDSFs of various geometries. The geometries can include, for example, but not limited to, square and contoured SDSFs. The orientation of the AV relative to the SDSF can determine the speed and direction the AV travels. The SDSF traversal feature can adjust the speed of the AV in the vicinity of the SDSF. When the AV rises to the SDSF, the speed can be increased to assist the AV in traversing the SDSF.

[0036] 1. An autonomous delivery vehicle comprising: a power base including two powered front wheels, two powered rear wheels, and an energy storage device, the power base configured to move at a commanded speed and in a commanded direction to perform a transport of at least one object; a cargo platform including a plurality of short-range sensors, the cargo platform mechanically attached to the power base; a cargo container with a volume for receiving at least one object, the cargo container mounted on the cargo platform; a long-range sensor suite comprising a LIDAR and one or more cameras, the long-range sensor suite mounted on the cargo container; and a controller for receiving data from the long-range sensor suite and the plurality of short-range sensors, the controller determining a commanded speed and a commanded direction based at least on the data, and the controller providing the commanded speed and commanded direction to the power base to complete the transport. 2. The autonomous delivery vehicle of item 1, wherein the data from the multiple short-range sensors comprises at least one characteristic of a surface over which the power base travels. 3. The autonomous delivery vehicle of item 1, wherein the multiple short-range sensors comprise at least one stereo camera. 4. The autonomous delivery vehicle of item 1, wherein the multiple short-range sensors comprise at least one IR projector, at least one image sensor, and at least one RGB sensor. 5. The autonomous delivery vehicle of item 1, wherein the multiple short-range sensors comprise at least one radar sensor. 6. The autonomous delivery vehicle of item 1, wherein the data from the multiple short-range sensors comprises RGB-D data. 7. The autonomous delivery vehicle of item 1, wherein the controller determines a geometry of a road surface based on the RGB-D data received from the multiple short-range sensors. 8. The autonomous delivery vehicle of item 1, wherein the multiple short-range sensors detect objects within 4 meters of the AV and the long-range sensor suite detects objects greater than 4 meters from the autonomous delivery vehicle. 9. The autonomous delivery vehicle of item 1, wherein the multiple short-range sensors are equipped with a cooling circuit.10. The autonomous delivery vehicle of item 1, wherein the plurality of short-range sensors comprises ultrasonic sensors. 11. The autonomous delivery vehicle of item 2, wherein the controller comprises executable code for: accessing a map, the map formed by a map processor, the map processor accessing point cloud data from the long range sensor suite, the point cloud data representing a surface; a first processor for filtering the point cloud data, the filter, and forming a processable portion from the filtered point cloud data; a third processor for merging the processable portion into at least one polygon; a fourth processor for locating and labeling at least one substantially discontinuous surface feature (SDSF) within the at least one polygon, if present, the locating and labeling forming labeled point cloud data; a fifth processor for creating a graphing polygon from the labeled point cloud data; and a sixth processor for planning a route from a start point to an end point based at least on the graphing polygon, wherein the AV traverses the at least one SDSF along the route. 12. The autonomous delivery vehicle of item 11, wherein the filter comprises a seventh processor that executes code including conditionally removing points representing transient objects and points representing outliers from the point cloud data and replacing the removed points with a preselected height. 13. The autonomous delivery vehicle of item 11, wherein the second processor includes executable code including segmenting the point cloud data into processable portions and removing points of the preselected height from the processable portions.14. The autonomous delivery vehicle of item 11, wherein the third processor includes executable code including: reducing a size of the processable portion by analyzing outliers, voxels, and normals, growing an area from the reduced-sized processable portion, determining an initial drivable surface from the grown-up area, segmenting and meshing the initial drivable surface, locating polygons in the segmented and meshed initial drivable surface, and setting at least one drivable surface based on at least the polygons. 15. The autonomous delivery vehicle of item 14, wherein the fourth processor includes executable code including: sorting point cloud data of the initial drivable surface according to an SDSF filter, the SDSF filter including at least three categories of points, and locating at least one SDSF point based on whether the at least three categories of points in combination satisfy at least one first preselected criterion. 16. The autonomous delivery vehicle of item 15, wherein the fourth processor includes executable code for at least creating at least one SDSF trajectory based on whether a plurality of at least one SDSF point, in combination, satisfies at least one second preselected criterion. 17. The autonomous delivery vehicle of item 14, wherein the eighth processor includes executable code for creating at least one polygon from at least one drivable surface, the at least one polygon including an outer edge, smoothing the outer edge, forming a driving margin based on the smoothed outer edge, adding the at least one SDSF trajectory to the at least one drivable surface, and removing an inner edge from the at least one drivable surface according to at least one third preselected criterion. 18. The autonomous delivery vehicle of item 17, wherein the ninth processor includes executable code for smoothing the outer edge includes trimming the outer edge outward to form an outward edge.19. The autonomous delivery vehicle of item 18, including a tenth processor including executable code, wherein forming a driving margin of the smoothed outer edge includes inwardly trimming the outward edge. 20. The controller includes a subsystem for navigating at least one substantially discontinuous surface feature (SDSF) encountered by an autonomous delivery vehicle (AV), wherein the AV navigates a path over the surface, the surface including at least one SDSF, the path including a start point and an end point, a first processor that accesses a route morphology, the route morphology including at least one graphed polygon including filtered point cloud data, the filtered point cloud data including the labeled features, and the point cloud data including the driveable margin, a second processor that converts the point cloud data to a global coordinate system, and a third processor that converts the point cloud data to a global coordinate system. 21. The autonomous delivery vehicle of item 1, comprising a subsystem comprising: a third processor that determines a boundary of at least one SDSF, where the third processor creates an SDSF buffer of a preselected size around the boundary; a fourth processor that determines at least one SDSF that may be traversed based on at least one SDSF traversal criterion; a fifth processor that creates an edge / weighted graph based on at least the at least one SDSF traversal criterion, the transformed point cloud data, and the route morphology; and a base controller that plans a route from a start point to an end point based on at least the edge / weighted graph. 22. The autonomous delivery vehicle of item 1, comprising: a subsystem comprising: a third processor that determines a boundary of at least one SDSF, where the third processor creates an SDSF buffer of a preselected size around the boundary; a fourth processor that determines at least one SDSF that may be traversed based on at least one SDSF traversal criterion; a fifth processor that creates an edge / weighted graph based on at least the at least one SDSF traversal criterion, the transformed point cloud data, and the route morphology; and a base controller that plans a route from a start point to an end point based on at least the edge / weighted graph. 23. The autonomous delivery vehicle of item 20, wherein the at least one SDSF traversal criterion comprises a preselected width of the at least one SDSF and a preselected smoothness of the at least one SDSF, a minimum entry distance and a minimum exit distance between the at least one SDSF and the AV that includes a drivable surface, and a minimum entry distance between the at least one SDSF and the AV that accommodates an approximately 90° approach to the at least one SDSF by the AV.

[0037] 22. A method for managing a global occupancy grid for an autonomous device, the global occupancy grid including global occupancy grid cells, the global occupancy grid cells being associated with an occupancy probability, receiving sensor data from a sensor associated with the autonomous device, creating a local occupancy grid based on at least the sensor data, the local occupancy grid having local occupancy grid cells, when the autonomous device moves from a first area to a second area, accessing historical data associated with the second area, creating a static grid based on at least the historical data, moving the global occupancy grid to position the autonomous device at a center location of the global occupancy grid. 23. The method of claim 22, further comprising: maintaining a local occupancy probability of 0 if the new occupancy probability is <0; and updating the moved global occupancy grid based on the static grid; marking at least one of the global occupancy grid cells as unoccupied if at least one of the global occupancy grid cells matches a location of the autonomous device; and for each local occupancy grid cell, calculating a position of the local occupancy grid cell on the global occupancy grid; accessing a first occupancy probability from the global occupancy grid cell at the position; accessing a second occupancy probability from the local occupancy grid cell at the position; and calculating a new occupancy probability at the position on the global occupancy grid based on at least the first occupancy probability and the second occupancy probability. 24. The method of claim 23, further comprising: setting the new occupancy probability to 0 if the new occupancy probability is <0; and setting the new occupancy probability to 1 if the new occupancy probability is >1. 25. The method of claim 22, further comprising setting the global occupancy grid cell to the new occupancy probability. 26. The method of claim 23, further comprising setting global occupancy grid cells to the range-checked new occupancy probability.

[0038] 27. A method for creating and managing an occupancy grid, comprising: transforming sensor measurements by a local occupancy grid creation node into a reference frame associated with a device; creating a time-stamped measurement occupancy grid; publishing the time-stamped measurement occupancy grid as a local occupancy grid; creating a plurality of local occupancy grids; creating a static occupancy grid based on surface characteristics in a repository, the surface characteristics being associated with a location of the device; moving a global occupancy grid associated with the location of the device to keep the device and the local occupancy grid approximately centered with respect to the global occupancy grid; 27. The method of claim 26, further comprising: adding information from the local occupancy grid to a global occupancy grid; marking an area in the global occupancy grid currently occupied by the device as unoccupied; and for at least one cell in each local occupancy grid, determining a location of the at least one cell in the global occupancy grid, accessing a first value at the location, determining a second value at the location based on a relationship between the first value and a cell value in the at least one cell in the local occupancy grid, comparing the second value against a preselected probability range, and setting the global occupancy grid with the new value if the probability value is within the preselected probability range. 28. The method of claim 27, further comprising publishing the global occupancy grid. 29. The method of claim 27, wherein the surface characteristics comprise a surface type and a surface discontinuity. 30. The method of claim 27, wherein the relationship comprises summing.31. A system for creating and managing an occupancy grid, comprising: a plurality of local grid creation nodes that create at least one local occupancy grid, the at least one local occupancy grid being associated with a location of a device, the at least one local occupancy grid including at least one cell; and a global occupancy grid manager that accesses the at least one local occupancy grid, and creates a static occupancy grid based on surface characteristics in a repository, the surface characteristics being associated with a location of the device, and moves a global occupancy grid associated with the location of the device to substantially center the device and the at least one cell relative to the global occupancy grid. and a global occupancy grid manager that maintains local occupancy grids, adds information from the static occupancy grid to at least one global occupancy grid, marks areas in the global occupancy grid that are currently occupied by the device as unoccupied, and for at least one cell in each local occupancy grid, determines a location of the at least one cell in the global occupancy grid, accesses a first value at the location, determines a second value at the location based on a relationship between the first value and a cell value in the at least one cell in the local occupancy grid, compares the second value against a preselected probability range, and sets the global occupancy grid with the new value if the probability value is within the preselected probability range.

[0039] 32. A method for updating a global occupancy grid, the method including: when an autonomous device moves to a new location, updating the global occupancy grid with information from a static grid associated with the new location; analyzing a surface at the new location; and, if the surface is drivable, updating the surface and updating the global occupancy grid with the updated surface; and updating the global occupancy grid with values ​​from a repository of static values, the static values ​​associated with the new location. 33. The method of claim 32, wherein updating the surface includes accessing a local occupancy grid associated with the new location; for each cell in the local occupancy grid, accessing a local occupancy grid surface classification confidence value and a local occupancy grid surface classification; if the local occupancy grid surface classification is identical to the global surface classification in the global occupancy grid in the cell, adding the global surface classification confidence value in the global occupancy grid to the local occupancy grid surface classification confidence value to form a sum and updating the global occupancy grid at the cell using the sum; if the local occupancy grid surface classification is not identical to the global surface classification in the global occupancy grid in the cell, subtracting the local occupancy grid surface classification confidence value from the global surface classification confidence value in the global occupancy grid to form a difference and updating the global occupancy grid using the difference; and if the difference is less than zero, updating the global occupancy grid with the local occupancy grid surface classification.34. The method of claim 32, wherein updating the global occupancy grid with values ​​from the repository of static values ​​includes: for each cell in the local occupancy grid, accessing a log-odds value, which is a local occupancy grid probability that the cell from the local occupancy grid is an occupied value; updating the log-odds value in the global occupancy grid with the local occupancy grid log-odds value at the cell; decreasing the log-odds that the cell is occupied in the local occupancy grid if a preselected certainty that the cell is not occupied is met and if the autonomous device is proceeding within a lane barrier and the local occupancy grid surface classification indicates a drivable surface; increasing the log-odds in the local occupancy grid if the autonomous device expects to encounter a relatively uniform surface and the local occupancy grid surface classification indicates a relatively non-uniform surface; and decreasing the log-odds in the local occupancy grid if the autonomous device expects to encounter a relatively uniform surface and the local occupancy grid surface classification indicates a relatively uniform surface.

[0040] 35. A method for real-time control of a configuration of a device, the device including a chassis, at least four wheels, a first side of the chassis operably coupled to at least one of the at least four wheels, and an opposing second side of the chassis operably coupled to at least one of the at least four wheels, the method including: creating a map based on at least prior surface features and an occupancy grid, the map being created in non-real-time, the map including at least one location, the at least one location associated with at least one surface feature, the at least one surface feature associated with at least one surface classification and at least one mode; determining current surface features as the device travels; updating the occupancy grid in real-time with the current surface features; and determining from the occupancy grid and the map a path the device may travel to traverse the at least one surface feature.

[0041] 36. A method for real-time control of a configuration of a device, the device including a chassis, at least four wheels, a first side of the chassis operably coupled to at least one of the at least four wheels, and an opposing second side of the chassis operably coupled to at least one of the at least four wheels, the method including receiving environmental data, determining a surface type based at least on the environmental data, determining a mode based at least on the surface type and the first configuration, determining a second configuration based at least on the mode and the surface type, determining a movement command based at least on the second configuration, and controlling a configuration of the device by using the movement command to change the device from the first configuration to the second configuration. 37. The method of claim 36, wherein the environmental data comprises RGB-D image data. 38. The method of claim 36, further comprising capturing an occupancy grid based at least on the surface type and the mode, and determining the movement command based at least on the occupancy grid. 39. The method of claim 38, wherein the occupancy grid comprises at least information based on data from at least one image sensor. 40. The method of claim 36, wherein the environmental data comprises a morphology of a road surface. 41. The method of claim 36, wherein the configuration comprises two clustered pairs of at least four wheels, a first pair of the two pairs being positioned on a first side and a second pair of the two pairs being positioned on a second side, the first pair including a first front wheel and a first rear wheel, and the second pair including a second front wheel and a second rear wheel. 42. The method of claim 41, wherein the control of the configuration includes coordinated powering of the first pair and the second pair based at least on the environmental data. 43. The method of item 41, wherein controlling the configuration includes transitioning from driving at least four wheels and a pair of casters that are retracted, the pair of casters being operably coupled to the chassis, to driving two wheels with a clustered first pair and a clustered second pair that are rotated to lift a first front wheel and a second front wheel, the device resting on a first rear wheel, a second rear wheel, and the pair of casters.44. The method of claim 41, wherein controlling the configuration includes rotating a pair of clusters operatively coupled with a first two powered wheels on a first side and a second two powered wheels on a second side based at least on the environmental data. 45. The method of claim 36, wherein the device further comprises a cargo container, the cargo container mounted on a chassis, and the chassis controls a height of the cargo container. 46. The method of claim 45, wherein the height of the cargo container is based at least on the environmental data.

[0042] 47. A system for real-time control of a configuration of a device, the device including a chassis, at least four wheels, a first side of the chassis, and an opposing second side of the chassis, the system comprising: a device processor receiving real-time environmental data surrounding the device, the device processor determining a surface type based at least on the environmental data, the device processor determining a mode based at least on the surface type and the first configuration, the device processor determining a second configuration based at least on the mode and the surface type; and a power base processor determining movement commands based at least on the second configuration, the power base processor controlling the configuration of the device by using the movement commands to change the device from the first configuration to the second configuration. 48. The system of claim 47, the environmental data comprising RGB-D image data. 49. The system of claim 47, the device processor including populating an occupancy grid based at least on the surface type and the mode. 50. The system of claim 49, the power base processor including determining movement commands based at least on the occupancy grid. 51. The system of claim 49, wherein the occupancy grid comprises at least information based on data from at least one image sensor. 52. The system of claim 47, wherein the environmental data comprises a morphology of a road surface. 53. The system of claim 47, wherein the configuration comprises two clustered pairs of at least four wheels, a first pair of the two pairs being positioned on a first side and a second pair of the two pairs being positioned on a second side, the first pair having a first front wheel and a first rear wheel, and the second pair having a second front wheel and a second rear wheel. 54. The system of claim 53, wherein the control of the configuration includes coordinated powering of the first pair and the second pair based at least on the environmental data.55. The system of item 53, wherein controlling the configuration includes transitioning from driving at least four wheels and a pair of casters that are retracted, the pair of casters being operably coupled to the chassis, to driving two wheels with a clustered first pair and a clustered second pair that are rotated to lift a first front wheel and a second front wheel, the device resting on a first rear wheel, a second rear wheel, and the pair of casters.

[0043] 56. A method for maintaining a global occupancy grid, comprising: locating a first position of an autonomous device; and when the autonomous device moves to a second position, the second position is associated with a global occupancy grid and a local occupancy grid; updating the global occupancy grid with at least one occupancy probability value associated with the first position; updating the global occupancy grid with at least one drivable surface associated with the local occupancy grid; updating the global occupancy grid with a surface confidence associated with the at least one drivable surface; and updating the global occupancy grid with log-odds of the at least one occupancy probability value using a first Bayes function. and adjusting the log-odds based on a characteristic associated with at least the second location; when the autonomous device remains in the first location and the global occupancy grid and the local occupancy grid are co-located, updating the global occupancy grid with at least one drivable surface associated with the local occupancy grid; updating the global occupancy grid with a surface confidence associated with the at least one drivable surface; updating the global occupancy grid with log-odds of the at least one occupancy probability value using a second Bayes function; and adjusting the log-odds based on a characteristic associated with at least the second location. 57. The method of item 35, wherein creating the map includes accessing point cloud data representing the surface, filtering the point cloud data, forming the filtered point cloud data into a processable portion, merging the processable portion into at least one concave polygon, locating and labeling at least one SDSF within the at least one concave polygon, where the locating and labeling forms the labeled point cloud data, creating a graphing polygon based at least on the at least one concave polygon, and selecting a route from a start point to an end point based at least on the graphing polygon, where the AV traverses the at least one SDSF along the route.58. The method of claim 57, wherein filtering the point cloud data includes conditionally removing points representing transient objects and points representing outliers from the point cloud data, and replacing the removed points with a preselected height. 59. The method of claim 57, wherein forming the processing portions includes segmenting the point cloud data into processable portions, and removing points at a preselected height from the processable portions. 60. The method of claim 57, wherein merging the processable portions includes reducing a size of the processable portions by analyzing outliers, voxels, and normals, growing an area from the reduced-sized processable portions, determining an initial drivable surface from the grown area, segmenting and meshing the initial drivable surface, locating polygons in the segmented and meshed initial drivable surface, and setting at least one drivable surface based at least on the polygons. 61. The method of item 60, wherein locating and labeling at least one SDSF includes sorting the point cloud data of the initial drivable surface according to an SDSF filter, the SDSF filter including at least three categories of points, and locating the at least one SDSF point based on whether the at least three categories of points, in combination, satisfy at least one first preselected criterion. 62. The method of item 61, further including creating at least one SDSF trajectory based on whether a plurality of the at least one SDSF points, in combination, satisfy at least one second preselected criterion. 63. The method of item 62, wherein creating the graphed polygon further includes: creating at least one polygon from the at least one drivable surface, the at least one polygon including an outer edge; smoothing the outer edge; forming a driving margin based on the smoothed outer edge; adding at least one SDSF trajectory to the at least one drivable surface; and removing an inner edge from the at least one drivable surface according to at least one third preselected criterion.64. The method of claim 63, wherein smoothing the outer edge comprises trimming the outer edge outward to form an outward edge. 65. The method of claim 63, wherein forming a running margin of the smoothed outer edge comprises trimming the outward edge inward.

[0044] 66. An autonomous delivery vehicle comprising: a power base including two powered front wheels, two powered rear wheels, and an energy storage device, the power base configured to move at a commanded speed; a cargo platform including a plurality of short-range sensors, the cargo platform mechanically attached to the power base; a cargo container with a volume for receiving one or more objects to be delivered, the cargo container mounted on the cargo platform; a long-range sensor suite including a LIDAR and one or more cameras, the long-range sensor suite mounted on the cargo container; and a controller for receiving data from the long-range sensor suite and the plurality of short-range sensors. 67. The autonomous delivery vehicle of item 66, wherein the plurality of short-range sensors detect at least one characteristic of the drivable surface. 68. The autonomous delivery vehicle of item 66, wherein the plurality of short-range sensors are stereo cameras. 69. The autonomous delivery vehicle of item 66, wherein the multiple short-range sensors comprise an IR projector, two image sensors, and an RGB sensor. 70. The autonomous delivery vehicle of item 66, wherein the multiple short-range sensors are radar sensors. 71. The autonomous delivery vehicle of item 66, wherein the short-range sensors provide RGB-D data to the controller. 72. The autonomous delivery vehicle of item 66, wherein the controller determines a geometry of the road surface based on the RGB-D data received from the multiple short-range sensors. 73. The autonomous delivery vehicle of item 66, wherein the multiple short-range sensors detect objects within 4 meters of the autonomous delivery vehicle and the long-range sensor suite detects objects more than 4 meters from the autonomous delivery vehicle.

[0045] 74. An autonomous delivery vehicle comprising: a power base including at least two powered rear wheels, a caster front wheel, and an energy storage device, the power base configured to move at a commanded speed; a cargo platform including a plurality of short-range sensors, the cargo platform mechanically attached to the power base; a cargo container with a volume for receiving one or more objects to be delivered, the cargo container mounted on the cargo platform; a long-range sensor suite including a LIDAR and one or more cameras, the long-range sensor suite mounted on the cargo container; and a controller for receiving data from the long-range sensor suite and the plurality of short-range sensors. 75. The autonomous delivery vehicle of claim 74, wherein the plurality of short-range sensors detect at least one characteristic of the drivable surface. 76. The autonomous delivery vehicle of claim 74, wherein the plurality of short-range sensors are stereo cameras. 77. The autonomous delivery vehicle of item 74, wherein the plurality of short-range sensors comprises an IR projector, two image sensors, and an RGB sensor. 78. The autonomous delivery vehicle of item 74, wherein the plurality of short-range sensors is a radar sensor. 79. The autonomous delivery vehicle of item 74, wherein the short-range sensor provides RGB-D data to the controller. 80. The autonomous delivery vehicle of item 74, wherein the controller determines a geometry of the road surface based on the RGB-D data received from the plurality of short-range sensors. 81. The autonomous delivery vehicle of item 74, wherein the plurality of short-range sensors detect objects within 4 meters of the autonomous delivery vehicle and the long-range sensor suite detects objects greater than 4 meters from the autonomous delivery vehicle. 82. The autonomous delivery vehicle of item 74, further comprising a second set of powered wheels that can engage the ground while the caster wheels are lifted off the ground.

[0046] 83. An autonomous delivery vehicle comprising: a power base including at least two powered rear wheels, a caster front wheel, and an energy storage device, the power base configured to move at a commanded speed; a cargo platform, the cargo platform mechanically attached to the power base; and a short-range camera assembly mounted on the cargo platform that detects at least one characteristic of the drivable surface, the short-range camera assembly comprising a camera, a first light, and a first liquid-cooled heat sink, the first liquid-cooled heat sink cooling the first light and the camera. 84. The autonomous delivery vehicle of item 83, wherein the short-range camera assembly further comprises a thermoelectric cooler between the camera and the liquid-cooled heat sink. 85. The autonomous delivery vehicle of item 83, wherein the first light and the camera are recessed within a cover with an opening that deflects illumination from the first light away from the camera. 86. The autonomous delivery vehicle of item 83, wherein the light is angled downward at least 15° and recessed at least 4mm into the cover to minimize distracting lighting for pedestrians. 87. The autonomous delivery vehicle of item 83, wherein the camera has a field of view and the first light comprises two LEDs with lenses for generating two beams of light that are diffused to illuminate the field of view of the camera. 88. The autonomous delivery vehicle of item 87, wherein the lights are angled approximately 50° apart and the lenses generate a 60° beam. 89. The autonomous delivery vehicle of item 83, wherein the short-range camera assembly includes an ultrasonic sensor mounted above the camera. 90. The autonomous delivery vehicle of item 83, wherein the short-range camera assembly is mounted in a central position on the front of the cargo platform. 91. The autonomous delivery vehicle of item 83, further comprising at least one corner camera assembly mounted on at least one corner of a front side of the cargo platform, the at least one corner camera assembly comprising an ultrasonic sensor, a corner camera, a second light, and a second liquid-cooled heat sink, the second liquid-cooled heat sink cooling the second light and the corner camera.92. The method of claim 22, wherein the historical data comprises surface data. 93. The method of claim 22, wherein the historical data comprises discontinuity data. The present invention provides, for example, the following: (Item 1) 1. An autonomous delivery vehicle, comprising: a power base including two powered front wheels, two powered rear wheels, and an energy storage device, the power base configured to move at a commanded speed and in a commanded direction to effect transportation of at least one object; a cargo platform including a plurality of short range sensors, said cargo platform being mechanically attached to said power base; a cargo container with a volume for receiving the at least one object, the cargo container being mounted on the cargo platform; and a long range sensor suite comprising a LIDAR and one or more cameras, the long range sensor suite being mounted on the cargo container; a controller for receiving data from the long range sensor suite and the plurality of short range sensors, the controller determining the commanded speed and the commanded direction based at least on the data, the controller providing the commanded speed and the commanded direction to the power base to complete the transport; and An autonomous delivery vehicle comprising: (Item 2) 2. The autonomous delivery vehicle of claim 1, wherein the data from the plurality of short-range sensors comprises at least one characteristic of a surface over which the power base travels. (Item 3) 2. The autonomous delivery vehicle of claim 1, wherein the multiple short-range sensors comprise at least one stereo camera. (Item 4) 2. The autonomous delivery vehicle of claim 1, wherein the multiple short-range sensors comprise at least one IR projector, at least one image sensor, and at least one RGB sensor. (Item 5) 2. The autonomous delivery vehicle of claim 1, wherein the plurality of short-range sensors comprises at least one radar sensor. (Item 6) 2. The autonomous delivery vehicle of claim 1, wherein the data from the multiple short-range sensors comprises RGB-D data. (Item 7) 2. The autonomous delivery vehicle of item 1, wherein the controller determines a geometry of a road surface based on RGB-D data received from the multiple short-range sensors. (Item 8) 2. The autonomous delivery vehicle of claim 1, wherein the plurality of short-range sensors detect objects within 4 meters of the AV and the long-range sensor suite detects objects greater than 4 meters from the autonomous delivery vehicle. (Item 9) 2. The autonomous delivery vehicle of claim 1, wherein the plurality of short-range sensors are equipped with a cooling circuit. (Item 10) 2. The autonomous delivery vehicle of claim 1, wherein the plurality of short-range sensors comprises ultrasonic sensors. (Item 11) The controller: and an executable code, the executable code comprising: accessing a map, the map being formed by a map processor, the map processor comprising: a first processor that accesses point cloud data from the long range sensor suite, the point cloud data representing the surface; a filter for filtering the point cloud data; a second processor for forming a processable portion from the filtered point cloud data; a third processor for merging the processable portions into at least one polygon; a fourth processor for locating and labeling said at least one substantially discontinuous surface feature (SDSF) within said at least one polygon, if present, said locating and labeling forming labeled point cloud data; a fifth processor for generating a graphed polygon from the labeled point cloud data; a sixth processor that determines a path from a start point to an end point based at least on the graphed polygon, the AV traversing the at least one SDSF along the path; 3. The autonomous delivery vehicle of item 2, comprising: (Item 12) The filter comprises: conditionally removing points representing transient objects and points representing outliers from the point cloud data; replacing the removed points with a preselected height; and 12. The autonomous delivery vehicle of claim 11, further comprising a seventh processor that executes code including: (Item 13) The second processor, Segmenting the point cloud data into the processable portions; removing points of a preselected height from said addressable portion; Item 12. The autonomous delivery vehicle of item 11, comprising the executable code. (Item 14) The third processor, Reducing the size of the processable portion by analyzing outliers, voxels, and normals; Enlarging an area from the reduced size addressable portion; and determining an initial drivable surface from the expanded region; and segmenting and meshing the initial drivable surface; Locating polygons within the segmented and meshed initial drivable surface; and establishing at least one drivable surface based at least on said polygon; Item 12. The autonomous delivery vehicle of item 11, comprising the executable code. (Item 15) The fourth processor, sorting the initial drivable surface point cloud data according to an SDSF filter, the SDSF filter including at least three categories of points; locating at least one SDSF point based on whether the at least three categories of points, in combination, satisfy at least one first preselected criterion; Item 15. The autonomous delivery vehicle of item 14, comprising the executable code. (Item 16) 16. The autonomous delivery vehicle of claim 15, wherein the fourth processor comprises the executable code to create at least one SDSF trajectory based on whether a plurality of the at least one SDSF point, in combination, satisfies at least one second preselected criterion. (Item 17) Creating a graphing polygon is creating at least one polygon from the at least one drivable surface, the at least one polygon including an outer edge; smoothing the outer edge; and forming a running margin based on the smoothed outer edge; adding said at least one SDSF track to said at least one drivable surface; removing an inner edge from the at least one drivable surface according to at least one third preselected criterion; and Item 15. The autonomous delivery vehicle of item 14, further comprising an eighth processor comprising the executable code, (Item 18) Item 18. The autonomous delivery vehicle of item 17, further comprising a ninth processor including the executable code, wherein smoothing the outer edge includes trimming the outer edge outward to form an outward edge. (Item 19) Item 19. The autonomous delivery vehicle of item 18, further comprising a tenth processor including the executable code, wherein forming the smoothed outer edge running margin includes trimming the outward edge inward. (Item 20) The controller: a subsystem for navigating at least one substantially discontinuous surface feature (SDSF) encountered by the autonomous delivery vehicle (AV), the AV navigating a path across a surface, the surface including the at least one SDSF, the path including a start point and an end point, the subsystem comprising: a first processor that accesses a route configuration, the route configuration including at least one graphed polygon including filtered point cloud data, the filtered point cloud data including labeled features, and the point cloud data including a driveable margin; A second processor for transforming the point cloud data into a global coordinate system; a third processor that determines a boundary of the at least one SDSF, the third processor creating an SDSF buffer of a preselected size around the boundary; a fourth processor that determines the at least one SDSF that may be traversed based on at least one SDSF traversal criterion; and a fifth processor that creates an edge / weight graph based on at least the at least one SDSF traversal criterion, the transformed point cloud data, and the root morphology; a base controller that selects the path from the start point to the end point based at least on the edge / weight graph; 2. The autonomous delivery vehicle of item 1, comprising: (Item 21) The at least one SDSF cross-sectional criterion is a preselected width of the at least one SDSF and a preselected smoothness of the at least one SDSF; a minimum entry and exit distance between the at least one SDSF and the AV, the minimum entry and exit distance including a drivable surface; the minimum approach distance between the at least one SDSF and the AV accommodating an approximately 90° approach to the at least one SDSF by the AV; 21. The autonomous delivery vehicle of item 20, comprising: (Item 22) 1. A method for managing a global occupancy grid for an autonomous device, the global occupancy grid including global occupancy grid cells, the global occupancy grid cells associated with occupancy probabilities, the method comprising: receiving sensor data from a sensor associated with the autonomous device; creating a local occupancy grid based at least on the sensor data, the local occupancy grid having local occupancy grid cells; When the autonomous device moves from a first area to a second area, accessing historical data associated with the second area; and creating a static grid based at least on the historical data; moving the global occupancy grid to maintain the autonomous device in a central position of the global occupancy grid; updating the moved global occupancy grid based on the static grid; marking at least one of the global occupancy grid cells as unoccupied if the at least one of the global occupancy grid cells matches a location of the autonomous device; For each said locally occupied grid cell, calculating a position of the local occupancy grid cell on the global occupancy grid; accessing a first occupancy probability from the global occupancy grid cells at the location; accessing a second occupancy probability from the local occupancy grid cell at the location; and calculating a new occupancy probability at the location on the global occupancy grid based on at least the first occupancy probability and the second occupancy probability; A method comprising: (Item 23) 23. The method of claim 22, further comprising range checking the new occupancy probabilities. (Item 24) The range checking comprises: if the new occupancy probability is <0, setting the new occupancy probability to 0; if the new occupancy probability is >1, setting the new occupancy probability to 1; 24. The method according to item 23, comprising: (Item 25) 23. The method of claim 22, further comprising setting the global occupied grid cells to the new occupancy probability. (Item 26) 24. The method of claim 23, further comprising setting the global occupancy grid cells to the range-checked new occupancy probabilities. (Item 27) 1. A method for creating and managing an occupancy grid, comprising: Transforming the sensor measurements into a reference frame associated with the device by a local occupancy grid creation node; creating a time-stamped measurement occupancy grid; publishing the time-stamped measured occupancy grid as a local occupancy grid; Creating a plurality of local occupancy grids; creating a static occupancy grid based on surface characteristics in a repository, the surface characteristics being associated with a location of the device; moving a global occupancy grid associated with the location of the device to maintain the device and the local occupancy grid approximately aligned with the global occupancy grid; adding information from the static occupancy grid to the global occupancy grid; marking an area in the global occupancy grid that is currently occupied by the device as unoccupied; For at least one cell in each local occupancy grid, determining a location of the at least one cell within the global occupancy grid; accessing a first value at the location; determining a second value at the location based on a relationship between the first value and a cell value for the at least one cell in the local occupancy grid; comparing the second value against a preselected probability range; if the probability value is within the preselected probability range, setting the global occupancy grid with the new value; A method comprising: (Item 28) 28. The method of claim 27, further comprising publishing the global occupancy grid. (Item 29) 28. The method of claim 27, wherein the surface characteristics comprise a surface type and a surface discontinuity. (Item 30) 28. The method of claim 27, wherein the relationship includes summing. (Item 31) 1. A system for creating and managing an occupancy grid, comprising: a plurality of local grid creating nodes that create at least one local occupancy grid, the at least one local occupancy grid being associated with a location of a device, the at least one local occupancy grid including at least one cell; a global occupancy grid manager that accesses the at least one local occupancy grid; Equipped with The global occupancy grid manager: creating a static occupancy grid based on surface characteristics in a repository, the surface characteristics being associated with a location of the device; moving a global occupancy grid associated with the location of the device to maintain the device and the at least one local occupancy grid substantially aligned with the global occupancy grid; adding information from the static occupancy grid to at least one global occupancy grid; marking an area in the global occupancy grid that is currently occupied by the device as unoccupied; For each of the at least one cell in each local occupancy grid, determining a location of the at least one cell within the global occupancy grid; accessing a first value at the location; determining a second value at the location based on a relationship between the first value and a cell value for the at least one cell in the local occupancy grid; comparing the second value against a preselected probability range; if the probability value is within the preselected probability range, setting the global occupancy grid with the new value; A system that performs the above. (Item 32) 1. A method for updating a global occupancy grid, comprising: When an autonomous device moves to a new location, updating the global occupancy grid with information from a static grid associated with the new location; analyzing the surface at the new location; and if the surface is drivable, updating the surface and updating the global occupancy grid with the updated surface; updating the global occupancy grid with values ​​from a repository of static values, the static values ​​being associated with the new location; A method comprising: (Item 33) updating the surface accessing a local occupancy grid associated with the new location; and For each cell in the local occupancy grid, accessing a local occupancy grid surface classification confidence value and a local occupancy grid surface classification; if the local occupancy grid surface classification is the same as the global surface classification in the global occupancy grid in the cell, adding the global surface classification confidence value in the global occupancy grid to the local occupancy grid surface classification confidence value to form a sum and using the sum to update the global occupancy grid in the cell; if the local occupancy grid surface classification is not identical to the global surface classification in the global occupancy grid in the cell, subtracting the local occupancy grid surface classification confidence value from the global surface classification confidence value in the global occupancy grid to form a difference and updating the global occupancy grid with the difference; if the difference is less than zero, updating the global occupancy grid with the local occupancy grid surface classification; 33. The method according to item 32, comprising: (Item 34) updating the global occupancy grid with the values ​​from the repository of static values ​​comprises: For each cell in the local occupancy grid, accessing a log-odds value, which is a local occupancy grid probability that the cell from the local occupancy grid is an occupancy value; updating the log-odds value in the global occupancy grid with the local occupancy grid log-odds value in the cell; decreasing the log-odds that the cell is occupied in the local occupancy grid if a preselected certainty that the cell is unoccupied is met and if the autonomous device is proceeding within a lane barrier and a local occupancy grid surface classification indicates a drivable surface; increasing the log-odds within the local occupancy grid if the autonomous device expects to encounter a relatively uniform surface and if the local occupancy grid surface classification indicates a relatively non-uniform surface; decreasing the log-odds within the local occupancy grid if the autonomous device expects to encounter a relatively uniform surface and if the local occupancy grid surface classification indicates a relatively uniform surface; 33. The method according to item 32, comprising: (Item 35) 1. A method for real-time control of a configuration of a device, the device including a chassis, at least four wheels, a first side of the chassis operably coupled to at least one of the at least four wheels, and an opposing second side of the chassis operably coupled to at least one of the at least four wheels, the method comprising: generating a map based on at least the prior surface features and an occupancy grid, the map being generated in non-real-time, the map including at least one location, the at least one location associated with at least one surface feature, the at least one surface feature associated with at least one surface classification and at least one mode; determining a current surface characteristic as the device progresses; updating the occupancy grid in real time with the current surface features; and determining, from the occupancy grid and the map, a path that the device may take to traverse the at least one surface feature; A method comprising: (Item 36) 1. A method for real-time control of a configuration of a device, the device including a chassis, at least four wheels, a first side of the chassis operably coupled to at least one of the at least four wheels, and an opposing second side of the chassis operably coupled to at least one of the at least four wheels, the method comprising: Receiving environmental data; determining a surface type based at least on the environmental data; determining a mode based on at least the surface type and a first configuration; determining a second configuration based on at least the mode and the surface type; determining a movement command based on at least the second configuration; Controlling a configuration of the device by using the move command to change the device from the first configuration to the second configuration. A method comprising: (Item 37) Item 37. The method of item 36, wherein the environmental data comprises RGB-D image data. (Item 38) populating an occupancy grid based on at least the surface type and the mode; determining the movement command based at least on the occupancy grid; 37. The method of claim 36, further comprising: (Item 39) Item 39. The method of item 38, wherein the occupancy grid comprises at least information based on data from at least one image sensor. (Item 40) Item 37. The method of item 36, wherein the environmental data comprises a road surface morphology. (Item 41) 37. The method of claim 36, wherein the configuration comprises two clustered pairs of the at least four wheels, a first pair of the two pairs being positioned on the first side and a second pair of the two pairs being positioned on the second side, the first pair including a first front wheel and a first rear wheel, and the second pair including a second front wheel and a second rear wheel. (Item 42) Item 42. The method of item 41, wherein controlling the configuration includes coordinated powering of the first pair and the second pair based at least on the environmental data. (Item 43) 42. The method of claim 41, wherein controlling the configuration includes transitioning from driving the at least four wheels and a pair of casters that are retracted, the pair of casters being operably coupled to the chassis, to driving two wheels with the clustered first pair and the clustered second pair rotated to lift the first front wheel and the second front wheel, and the device is resting on the first rear wheel, the second rear wheel, and the pair of casters. (Item 44) 42. The method of claim 41, wherein controlling the configuration includes rotating a pair of clusters operatively coupled to a first two powered wheels on the first side and a second two powered wheels on the second side based on at least the environmental data. (Item 45) Item 37. The method of item 36, wherein the device further comprises a cargo container, the cargo container mounted on the chassis, and the chassis controls a height of the cargo container. (Item 46) Item 46. The method of item 45, wherein the height of the cargo container is based at least on the environmental data. (Item 47) 1. A system for real-time control of a configuration of a device, the device including a chassis, at least four wheels, a first side of the chassis, and an opposing second side of the chassis, the system comprising: a device processor receiving real-time environmental data surrounding the device, the device processor determining a surface type based at least on the environmental data, the device processor determining a mode based at least on the surface type and a first configuration, and the device processor determining a second configuration based at least on the mode and the surface type; a power base processor that determines a move command based on at least the second configuration, the power base processor using the move command to control a configuration of the device to change the device from the first configuration to the second configuration; A system comprising: (Item 48) Item 48. The system of item 47, wherein the environmental data comprises RGB-D image data. (Item 49) Item 48. The system of item 47, wherein the device processor populates an occupancy grid based on at least the surface type and the mode. (Item 50) 50. The system of claim 49, wherein the power base processor determines the movement commands based on at least the occupancy grid. (Item 51) 50. The system of claim 49, wherein the occupancy grid comprises at least information based on data from at least one image sensor. (Item 52) Item 48. The system of item 47, wherein the environmental data comprises a morphology of a road surface. (Item 53) Item 48. The system of item 47, wherein the configuration comprises two clustered pairs of the at least four wheels, a first pair of the two pairs being positioned on the first side and a second pair of the two pairs being positioned on the second side, the first pair having a first front wheel and a first rear wheel, and the second pair having a second front wheel and a second rear wheel. (Item 54) Item 54. The system of item 53, wherein controlling the configuration includes coordinated powering of the first pair and the second pair based at least on the environmental data. (Item 55) 54. The system of claim 53, wherein controlling the configuration includes transitioning from driving the at least four wheels and a pair of casters that are retracted, the pair of casters being operably coupled to the chassis, to driving two wheels with the clustered first pair and the clustered second pair rotated to lift the first front wheel and the second front wheel, and the device is resting on the first rear wheel, the second rear wheel, and the pair of casters. (Item 56) 1. A method for maintaining a global occupancy grid, comprising: Locating a first position of the autonomous device; When the autonomous device moves to a second location, the second location is associated with the global occupancy grid and the local occupancy grid; updating the global occupancy grid with at least one occupancy probability value associated with the first location; updating the global occupancy grid with at least one drivable surface associated with the local occupancy grid; updating the global occupancy grid with a surface confidence associated with the at least one drivable surface; updating the global occupancy grid with log-odds of the at least one occupancy probability value using a first Bayes function; adjusting the log odds based on a characteristic associated with at least the second location; and when the autonomous device remains in the first location and the global occupancy grid and the local occupancy grid are co-located; updating the global occupancy grid using the at least one drivable surface associated with the local occupancy grid; updating the global occupancy grid with the surface confidence associated with the at least one drivable surface; updating the global occupancy grid with log-odds of the at least one occupancy probability value using a second Bayes function; adjusting the log odds based on a characteristic associated with at least the second location; and A method comprising: (Item 57) Creating the map includes: accessing point cloud data representing the surface; filtering the point cloud data; and forming the filtered point cloud data into a processable portion; and merging said processable portions into at least one concave polygon; locating and labeling the at least one SDSF within the at least one concave polygon, the locating and labeling forming labeled point cloud data; and generating a graphing polygon based at least on said at least one concave polygon; determining the path from a start point to an end point based at least on the graphed polygon, the AV traversing the at least one SDSF along the path; 36. The method according to claim 35, comprising: (Item 58) Filtering the point cloud data includes: conditionally removing points representing transient objects and points representing outliers from the point cloud data; replacing the removed points with a preselected height; and 58. The method according to item 57, comprising: (Item 59) Forming the processing portion includes Segmenting the point cloud data into the processable portions; removing points of a preselected height from said addressable portion; 58. The method according to item 57, comprising: (Item 60) Merging the processable portions comprises: Reducing the size of the processable portion by analyzing outliers, voxels, and normals; Enlarging an area from the reduced size addressable portion; and determining an initial drivable surface from the expanded region; and segmenting and meshing the initial drivable surface; Locating polygons within the segmented and meshed initial drivable surface; and establishing at least one drivable surface based at least on said polygon; 58. The method according to item 57, comprising: (Item 61) Locating and labeling the at least one SDSF comprises: sorting the initial drivable surface point cloud data according to an SDSF filter, the SDSF filter including at least three categories of points; locating at least one SDSF point based on whether the at least three categories of points, in combination, satisfy at least one first preselected criterion; Item 61. The method according to item 60, comprising: (Item 62) Item 62. The method of item 61, further comprising generating at least one SDSF trajectory based on whether a plurality of the at least one SDSF points, in combination, satisfy at least one second preselected criterion. (Item 63) Creating the graphing polygon further comprises: creating at least one polygon from the at least one drivable surface, the at least one polygon including an outer edge; smoothing the outer edge; and forming a running margin based on the smoothed outer edge; adding said at least one SDSF track to said at least one drivable surface; removing an inner edge from the at least one drivable surface according to at least one third preselected criterion; and Item 63. The method according to item 62, comprising: (Item 64) Item 64. The method of item 63, wherein smoothing the outer edge comprises trimming the outer edge outwardly to form an outward edge. (Item 65) Item 64. The method of item 63, wherein forming the smoothed outer edge running margin comprises inwardly trimming the outwardly facing edge. (Item 66) 1. An autonomous delivery vehicle, comprising: a power base including two powered front wheels, two powered rear wheels, and an energy storage device, the power base configured to move at a commanded speed; a cargo platform including a plurality of short range sensors, said cargo platform being mechanically attached to said power base; a cargo container with a volume for receiving one or more objects to be delivered, said cargo container mounted on said cargo platform; a long range sensor suite comprising a LIDAR and one or more cameras, the long range sensor suite being mounted on the cargo container; a controller for receiving data from the long range sensor suite and the plurality of short range sensors; An autonomous delivery vehicle comprising: (Item 67) Item 67. The autonomous delivery vehicle of item 66, wherein the plurality of short-range sensors detect at least one characteristic of a drivable surface. (Item 68) Item 67. The autonomous delivery vehicle of item 66, wherein the multiple short-range sensors are stereo cameras. (Item 69) Item 67. The autonomous delivery vehicle of item 66, wherein the multiple short-range sensors comprise an IR projector, two image sensors, and an RGB sensor. (Item 70) Item 67. The autonomous delivery vehicle of item 66, wherein the plurality of short-range sensors are radar sensors. (Item 71) Item 67. The autonomous delivery vehicle of item 66, wherein the short-range sensor provides RGB-D data to the controller. (Item 72) Item 67. The autonomous delivery vehicle of item 66, wherein the controller determines a geometric shape of a road surface based on RGB-D data received from the multiple short-range sensors. (Item 73) Item 67. The autonomous delivery vehicle of item 66, wherein the plurality of short-range sensors detect objects within 4 meters of the autonomous delivery vehicle and the long-range sensor suite detects objects more than 4 meters from the autonomous delivery vehicle. (Item 74) 1. An autonomous delivery vehicle, comprising: a power base including at least two powered rear wheels, a caster front wheel, and an energy storage device, the power base configured to move at a commanded speed; a cargo platform including a plurality of short range sensors, said cargo platform being mechanically attached to said power base; a cargo container with a volume for receiving one or more objects to be delivered, said cargo container mounted on said cargo platform; a long range sensor suite comprising a LIDAR and one or more cameras, the long range sensor suite being mounted on the cargo container; a controller for receiving data from the long range sensor suite and the plurality of short range sensors; An autonomous delivery vehicle comprising: (Item 75) 75. The autonomous delivery vehicle of item 74, wherein the plurality of short-range sensors detect at least one characteristic of a drivable surface. (Item 76) Item 75. The autonomous delivery vehicle of item 74, wherein the multiple short-range sensors are stereo cameras. (Item 77) Item 75. The autonomous delivery vehicle of item 74, wherein the multiple short-range sensors comprise an IR projector, two image sensors, and an RGB sensor. (Item 78) Item 75. The autonomous delivery vehicle of item 74, wherein the plurality of short-range sensors are radar sensors. (Item 79) Item 75. The autonomous delivery vehicle of item 74, wherein the short-range sensor provides RGB-D data to the controller. (Item 80) 75. The autonomous delivery vehicle of item 74, wherein the controller determines a geometric shape of a road surface based on RGB-D data received from the multiple short-range sensors. (Item 81) 75. The autonomous delivery vehicle of claim 74, wherein the plurality of short-range sensors detect objects within 4 meters of the autonomous delivery vehicle and the long-range sensor suite detects objects more than 4 meters from the autonomous delivery vehicle. (Item 82) Item 75. The autonomous delivery vehicle of item 74, further comprising a second set of powered wheels that can engage the ground while the caster wheels are lifted off the ground. (Item 83) 1. An autonomous delivery vehicle, comprising: a power base including at least two powered rear wheels, a caster front wheel, and an energy storage device, the power base configured to move at a commanded speed; a cargo platform, the cargo platform being mechanically attached to the power base; a short-range camera assembly mounted to the cargo platform for detecting at least one characteristic of a drivable surface; Equipped with The short range camera assembly includes: Camera and The first light, a first liquid-cooled heat sink; Equipped with The first liquid cooled heat sink cools the first light and the camera, an autonomous delivery vehicle. (Item 84) Item 84. The autonomous delivery vehicle of item 83, wherein the short-range camera assembly further comprises a thermoelectric cooler between the camera and the liquid-cooled heat sink. (Item 85) Item 84. The autonomous delivery vehicle of item 83, wherein the first light and the camera are embedded in a cover with an opening that deflects illumination from the first light away from the camera. (Item 86) Item 84. The autonomous delivery vehicle of item 83, wherein the lights are angled downward at least 15° and recessed at least 4 mm into the cover to minimize distracting illumination to pedestrians. (Item 87) Item 84. The autonomous delivery vehicle of item 83, wherein the camera has a field of view and the first light comprises two LEDs with lenses for generating two beams of light that diffuse to illuminate the field of view of the camera. (Item 88) Item 88. The autonomous delivery vehicle of item 87, wherein the lights are angled approximately 50° apart and the lenses produce a 60° beam. (Item 89) Item 84. The autonomous delivery vehicle of item 83, wherein the short-range camera assembly includes an ultrasonic sensor mounted above the camera. (Item 90) Item 84. The autonomous delivery vehicle of item 83, wherein the short-range camera assembly is mounted in a central position on the front of the cargo platform. (Item 91) and at least one corner camera assembly mounted on at least one corner of a front surface of the cargo platform, the at least one corner camera assembly comprising: An ultrasonic sensor; Corner camera and The second light, a second liquid-cooled heat sink, the second liquid-cooled heat sink cooling the second light and the corner camera; Item 84. The autonomous delivery vehicle of item 83, comprising: (Item 92) 23. The method of claim 22, wherein the historical data comprises surface data. (Item 93) 23. The method of claim 22, wherein the historical data comprises discontinuity data. [Brief description of the drawings]

[0047] The present teachings may be more readily understood by reference to the following description taken in conjunction with the accompanying drawings, in which:

[0048] [Figure 1-1] FIG. 1-1 is a schematic block diagram of the major components of the system of the present teachings.

[0049] [Figure 1-2] FIG. 1-2 is a schematic block diagram of the major components of the map processor of the present teachings.

[0050] [Figure 1-3] 1-3 are schematic block diagrams of the major components of the perception processor of the present teachings.

[0051] [Figure 1-4] 1-4 are schematic block diagrams of the major components of the autonomy processor of the present teachings.

[0052] [Figure 1A] FIG. 1A is a schematic block diagram of a system of the present teachings for preparing a travel path for an AV.

[0053] [Figure 1B] FIG. 1B is a pictorial diagram of an exemplary configuration of a device incorporating a system of the present teachings.

[0054] [Figure 1C] FIG. 1C is a side view of an autonomous delivery vehicle showing the fields of view of several long-range and short-range sensors.

[0055] [Figure 1D] FIG. 1D is a schematic block diagram of a map processor of the present teachings.

[0056] [Figure 1E] FIG. 1E is a pictorial diagram of a first portion of the map processor flow of the present teachings.

[0057] [Figure 1F] FIG. 1F is an image of a segmented point cloud of the present teachings.

[0058] [Figure 1G] FIG. 1G is a pictorial diagram of a second portion of the map processor of the present teachings.

[0059] [Figure 1H] FIG. 1H is an image of the drivable surface detection results of the present teachings.

[0060] [Figure 1I] FIG. 1I is a pictorial diagram of the SDSF detector flow of the present teachings.

[0061] [Figure 1J] FIG. 1J is a pictorial representation of the SDSF categories of the present teachings.

[0062] [Figure 1K] FIG. 1K is an image of an SDSF identified by the system of the present teachings.

[0063] [Figure 1L] 1L and 1M are pictorial illustrations of the polygon processing of the present teachings. [Figure 1M] 1L and 1M are pictorial illustrations of the polygon processing of the present teachings.

[0064] [Figure 1N] FIG. 1N is an image of the polygon and SDSF identified by the system of the present teachings.

[0065] [Figure 2A] FIG. 2A is an isometric view of an autonomous vehicle of the present teachings.

[0066] [Figure 2B] FIG. 2B is a top view of a cargo container showing the field of view of selected long-range sensors.

[0067] [Figure 2C] 2C-2F are diagrams of a long range sensor assembly. [Figure 2D] 2C-2F are diagrams of a long range sensor assembly. [Figure 2E] 2C-2F are diagrams of a long range sensor assembly. [Figure 2F] 2C-2F are diagrams of a long range sensor assembly.

[0068] [Figure 2G] FIG. 2G is a top view of a cargo container showing the field of view of selected short-range sensors.

[0069] [Figure 2H] FIG. 2H is an isometric view of a cargo platform of the present teachings.

[0070] [Figure 2I] 2I-2L are isometric views of the short range sensor. [Figure 2J] 2I-2L are isometric views of the short range sensor. [Figure 2K] 2I-2L are isometric views of the short range sensor. [Figure 2L] 2I-2L are isometric views of the short range sensor.

[0071] [Figure 2M] 2M-2N are isometric views of an autonomous vehicle of the present teachings. [Figure 2N] 2M-2N are isometric views of an autonomous vehicle of the present teachings.

[0072] [Figure 2O] 2O-2P are isometric views of an autonomous vehicle of the present teachings with the exterior panels removed. [Figure 2P] 2O-2P are isometric views of an autonomous vehicle of the present teachings with the exterior panels removed.

[0073] [Figure 2Q] FIG. 2Q is an isometric view of an autonomous vehicle of the present teachings with a portion of the top panel removed.

[0074] [Figure 2R] 2R-2V are diagrams of long-range sensors on an autonomous vehicle of the present teachings. [Figure 2S] 2R-2V are diagrams of long-range sensors on an autonomous vehicle of the present teachings. [Figure 2T] 2R-2V are diagrams of long-range sensors on an autonomous vehicle of the present teachings. [Figure 2U] 2R-2V are diagrams of long-range sensors on an autonomous vehicle of the present teachings. [Figure 2V] 2R-2V are diagrams of long-range sensors on an autonomous vehicle of the present teachings.

[0075] [Figure 2W] 2W-2Z are diagrams of an ultrasonic sensor. [Figure 2X] 2W-2Z are diagrams of an ultrasonic sensor. [Figure 2Y] 2W-2Z are diagrams of an ultrasonic sensor. [Figure 2Z] 2W-2Z are diagrams of an ultrasonic sensor.

[0076] [Figure 2AA] 2AA-2BB are diagrams of the central short-range camera assembly. [Figure 2BB] 2AA-2BB are diagrams of the central short-range camera assembly.

[0077] [Figure 3A] FIG. 3A is a schematic block diagram of a system in one configuration of the present teachings.

[0078] [Figure 3B] FIG. 3B is a schematic block diagram of a system in another configuration of the present teachings.

[0079] [Figure 3C] FIG. 3C is a schematic block diagram of a system of the present teachings that can initially create a global occupancy grid.

[0080] [Figure 3D] FIG. 3D is a pictorial representation of a static grid of the present teachings.

[0081] [Figure 3E] 3E and 3F are pictorial representations of the creation of an occupancy grid of the present teachings. [Figure 3F] 3E and 3F are pictorial representations of the creation of an occupancy grid of the present teachings.

[0082] [Figure 3G]FIG. 3G is a pictorial representation of a priori occupancy grid of the present teachings.

[0083] [Figure 3H] FIG. 3H is a pictorial representation of updating the global occupancy grid of the present teachings.

[0084] [Figure 3I] FIG. 3I is a flow diagram of a method of the present teachings for publishing a global occupancy grid.

[0085] [Figure 3J] FIG. 3J is a flow diagram of a method of the present teachings for updating the global occupancy grid.

[0086] [Figure 3K] 3K-3M are a flow diagram of another method of the present teachings for updating the global occupancy grid. [Figure 3L] 3K-3M are a flow diagram of another method of the present teachings for updating the global occupancy grid. [Figure 3M] 3K-3M are a flow diagram of another method of the present teachings for updating the global occupancy grid.

[0087] [Figure 4A] FIG. 4A is a perspective pictorial view of a device of the present teachings mounted in various modes.

[0088] [Figure 4B] FIG. 4B is a schematic block diagram of a system of the present teachings.

[0089] [Figure 4C] FIG. 4C is a schematic block diagram of the driving surface processor component of the present teachings.

[0090] [Figure 4D] FIG. 4D is a schematic block / pictorial flow diagram of the process of the present teachings.

[0091] [Figure 4E] 4E and 4F are perspective and side views, respectively, of a configuration of a device of the present teachings in a standard mode. [Figure 4F] 4E and 4F are perspective and side views, respectively, of a configuration of a device of the present teachings in a standard mode.

[0092] [Figure 4G] 4G and H are perspective and side views, respectively, of a configuration of a device of the present teachings in four-wheel mode. [Figure 4H] 4G and H are perspective and side views, respectively, of a configuration of a device of the present teachings in four-wheel mode.

[0093] [Figure 4I] 4I and 4J are perspective and side views, respectively, of a configuration of a device of the present teachings in an elevated four-wheel mode. [Figure 4J] 4I and 4J are perspective and side views, respectively, of a configuration of a device of the present teachings in an elevated four-wheel mode.

[0094] [Figure 4K] FIG. 4K is a flowchart of the method of the present teachings.

[0095] [Figure 5A] FIG. 5A is a schematic block diagram of a device controller of the present teachings.

[0096] [Figure 5B] FIG. 5B is a schematic block diagram of an SDSF processor of the present teachings.

[0097] [Figure 5C] FIG. 5C is an image of the SDSF approach identified by the system of the present teachings.

[0098] [Figure 5D]FIG. 5D is an image of the route morphology produced by the system of the present teachings.

[0099] [Figure 5E] FIG. 5E is a schematic block diagram of a mode of the present teachings.

[0100] [Figure 5F] 5F-5J are flow charts of methods of the present teachings for traversing an SDSF. [Figure 5G] 5F-5J are flow charts of methods of the present teachings for traversing an SDSF. [Figure 5H] 5F-5J are flow charts of methods of the present teachings for traversing an SDSF. [Figure 5I] 5F-5J are flow charts of methods of the present teachings for traversing an SDSF. [Figure 5J] 5F-5J are flow charts of methods of the present teachings for traversing an SDSF.

[0101] [Figure 5K] FIG. 5K is a schematic block diagram of a system of the present teachings for traversing an SDSF.

[0102] [Figure 5L] 5L-5N are pictorial representations of the method of FIGS. 5F-5H. [Figure 5M] 5L-5N are pictorial representations of the method of FIGS. 5F-5H. [Figure 5N] 5L-5N are pictorial representations of the method of FIGS. 5F-5H.

[0103] [Figure 5O] FIG. 5O is a pictorial representation of converting an image into a polygon. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0104] (Detailed Description) The systems and methods of the present teachings can navigate the AV across surface features using on-board sensors and previously developed maps, developing an occupancy grid, and using these aids to reconfigure the AV based on surface type and previous.

[0105] 1-1, the AV system 100 may include a structure on which sensors 10701 may be mounted and within which a device controller 10111 may execute. The structure may include a power base 10112 that may direct the movement of wheels that are part of the structure and enable the movement of the AV. The device controller 10111 may execute on at least one processor located on the AV and may receive data from the sensors 10701, which may be located on the AV, including but not limited to the AV. The device controller 10111 may provide speed, direction, and configuration information to the base controller 10114, which may provide movement commands to the power base 10112. The device controller 10111 may receive map information from the map processor 10104, which may prepare a map of the area surrounding the AV. The device controller 10111 may include a sensor processor 10703 that may receive and process inputs from the sensors 10701, including but not limited to the AV sensors. In some configurations, the device controller 10111 can include a perception processor 2143, an autonomy processor 2145, and a driver processor 2127. The perception processor 2143 can, for example, but not limited to, locate static and dynamic obstacles, determine traffic signal conditions, create occupancy grids, and classify surfaces. The autonomy processor 2145 can, for example, but not limited to, determine the maximum speed of the AV and determine the type of situation in which the AV is navigating, for example, on a road, on a sidewalk, at an intersection, and / or under remote control. The driver processor 2127 can, for example, but not limited to, create commands and send them on the base controller 10114 according to the instructions of the autonomy processor 2145.

[0106] 1-2, the map processor 10104 can create a map of surface features and provide the map through the device controller 10111 to the perception processor 2143, which can update the occupancy grid. The map processor 10104 can include a feature extractor 10801, a point cloud organizer 10803, a transient processor 10805, a segmenter 10807, a polygon generator 10809, an SDSF line generator 10811, and a combiner 10813, among many other aspects. The feature extractor 10801 can include a first processor that accesses point cloud data representing the surface. The point cloud organizer 10803 can include a second processor that forms a processable portion from the filtered point cloud data. The transient processor 10805 can include a first filter that filters the point cloud data. The segmenter 10807 may include executable code that may include, but is not limited to, segmenting the point cloud data into processable portions and removing points of a preselected height from the processable portions. The first filter may optionally include executable code that may include, but is not limited to, conditionally removing points representing transient objects and points representing outliers from the point cloud data and replacing the removed points with a preselected height. The polygon generator 10809 may include a third processor that merges the processable portions into at least one concave polygon. The third processor may optionally include executable code that may include, without limitation, reducing a size of the processable portion by analyzing outliers, voxels, and normals, growing an area from the reduced-sized processable portion, determining an initial drivable surface from the grown-up area, segmenting and meshing the initial drivable surface, locating polygons in the segmented and meshed initial drivable surface, and setting the drivable surface based at least on the polygons.The SDSF line generator 10811 may include a fourth processor that locates and labels at least one SDSF within at least one concave polygon, where the locating and labeling may form labeled point cloud data. The fourth processor may optionally include executable code that may include, without limitation, sorting the drivable surface point cloud data according to an SDSF filter, where the SDSF filter includes at least three categories of points, and locating at least one SDSF point based at least on whether the categories of points in combination satisfy at least one first preselected criterion. The combiner 10813 may include a fifth processor that creates a graphing polygon. Creating a graphed polygon may optionally include executable code that may include, but is not limited to, creating at least one polygon from the at least one drivable surface, the at least one polygon including an edge, smoothing the edge, forming a driving margin based on the smoothed edge, adding at least one SDSF trajectory to the at least one drivable surface, and removing the edge from the at least one drivable surface according to at least one third preselected criterion. Smoothing the edge may optionally include executable code that may include, but is not limited to, trimming the edge outward. Forming a running margin of the smoothed edge may optionally include executable code that may include, but is not limited to, trimming the outward edge inward.

[0107] 1-3, a map can be provided to the AV, which can include on-board sensors, powered wheels, and a processor for receiving sensor and map data, using the data, and power configuring the AV to traverse various types of surfaces, among other things, as the AV delivers goods, for example. The on-board sensors can capture an occupancy grid and can provide data that can be used to detect dynamic obstacles. The occupancy grid can also be captured by the map. The device controller 10111 can include a perception processor 2143 that can receive and process the sensor and map data and use the data to update the occupancy grid.

[0108] 1-4, the device controller 10111 can include a configuration processor 41023 that can automatically determine a configuration of the AV based on at least the mode of the AV and the surface features encountered. The autonomy processor 2145 can include a control processor 40325 that can determine the type of surface that needs to be traversed and the configuration the AV needs to take to traverse the surface based on at least the map (planned route to be followed), information from the configuration processor 41023, and the mode of the AV. The autonomy processor 2145 can provide commands to the motor drive processor 40326 to implement the commands.

[0109] 1A, the map processor 10104 can enable a device, such as, but not limited to, an AV or semi-autonomous device, to navigate within an environment that may include features such as an SDSF. The features in the map, along with on-board sensors, can enable the AV to navigate over various surfaces. In particular, the SDSF can be precisely identified and labeled such that the AV can automatically maintain AV performance during entry and exit of the SDSF, and AV speed, configuration, and direction can be controlled for safe SDSF traversal.

[0110] Continuing with reference to FIG. 1A, in some configurations, a system 100 for managing traversal of an SDSF can include an AV 10101, a core cloud infrastructure 10103, an AV service 10105, a device controller 10111, a sensor 10701, and a power base 10112. The AV 10101 can provide transportation and shuttle services from an origin to a destination following a dynamically determined route, for example, but not limited to, as modified by incoming sensor information. The AV 10101 can include, but is not limited to, devices having an autonomous mode, devices that can operate fully autonomously, devices that can be operated at least partially remotely, and devices that can include combinations of those features. The transportation device service 10105 can provide drivable surface information including features to the device controller 10111. The device controller 10111 can modify the drivable surface information at least according to, for example, but not limited to, the incoming sensor information and feature traversal requirements, and can plan a route for the AV 10101 based on the modified drivable surface information. The device controller 10111 can submit commands to the power base 10112, which instructs the power base 10112 to provide speed, direction, and configuration commands to the wheel motors and cluster motors, which can cause the AV 10101 to follow a selected route and lift and lower its cargo accordingly. The transport device services 10105 can access route-related information from a core cloud infrastructure 10103, which may include, but is not limited to, storage and content distribution facilities. In some configurations, the core cloud infrastructure 10103 can be, for example, but not limited to, Amazon Web Services, Google Cloud, and other cloud services. TM , and commercial products such as ORACLE CLOUD®.

[0111] 1B, an exemplary AV that may include a device controller 10111 (FIG. 1A) that may receive information from a map processor 10104 (FIG. 1A) of the present teachings may include a power base assembly, such as, for example, but not limited to, a power base, fully described in U.S. Patent Application No. 16 / 035,205, filed July 13, 2018, entitled "Mobility Device," or U.S. Patent No. 6,571,892, filed August 15, 2001, entitled "Control System and Method," both of which are incorporated herein by reference in their entirety. The exemplary power base assembly is described herein not to limit the present teachings, but instead to clarify features of any power base assembly that may be useful in implementing the techniques of the present teachings. The exemplary power base assembly may optionally include a power base 10112, a wheel cluster assembly 11100, and a payload carrier height assembly 10068. The exemplary power base assembly can optionally provide electrical as well as mechanical power to drive the wheels 11203 and the cluster 11100 which can raise and lower the wheels 11203. The power base 10112 can control the rotation of the cluster assembly 11100 and the elevation and lowering of the payload carrier height assembly 10068 to assist in the substantially discontinuous surface traversal of the present teachings. Other such devices can also be used to accommodate the SDSF detection and traversal of the present teachings.

[0112] Referring again to FIG. 1A, in some configurations, sensors inside the exemplary power base can detect the orientation and rate of change of the orientation of the AV10101, motors can enable servo operation, and a controller can interpret the information from the internal sensors as well as the motors. Appropriate motor commands can be calculated to achieve vehicle performance and implement path following commands. Left and right wheel motors can drive wheels on both sides of the AV10101. In some configurations, the front and rear wheels can be coupled to drive together, such that the two left wheels can drive together and the two right wheels can drive together. In some configurations, turning can be accomplished by driving the left and right motors at different rates, and the cluster motors can rotate the wheelbase in the forward / rearward direction. This can allow the AV10101 to remain level while the front wheels are higher or lower than the rear wheels. This feature can be useful, for example, but not limited to, when ascending or descending an SDSF. The payload carrier 10173 can be automatically raised and lowered based at least on the underlying terrain.

[0113] Continuing with reference to FIG. 1A, in some configurations, the point cloud data can include route information regarding the area through which the AV10101 should travel. Possibly, the point cloud data collected by a mapping device similar to or the same as the AV10101 can be time-tagged. The path along which the mapping device travels can be referred to as a mapped trajectory. The point cloud data processing described herein can be performed as the mapping device traverses the mapped trajectory or after point cloud data collection is completed. After the point cloud data are collected, they can be subjected to point cloud data processing, which can include initial filtering and point reduction, point cloud segmentation, and feature detection as described herein. In some configurations, the core cloud infrastructure 10103 can provide long-term or short-term storage for the collected point cloud data and can provide the data to the AV service 10105. The AV service 10105 can select among the possible point cloud datasets to find one that covers an area surrounding a desired starting point for the AV 10101 and a desired destination for the AV 10101. The AV service 10105 can include a map processor 10104 that can reduce the size of the point cloud data and determine features represented in the point cloud data, without limitation. In some configurations, the map processor 10104 can determine the location of the SDSF from the point cloud data. In some configurations, polygons can be created from the point cloud data as a technique to segment the point cloud data and ultimately establish a drivable surface. In some configurations, the SDSF detection and drivable surface determination can proceed in parallel. In some configurations, the SDSF detection and drivable surface determination can proceed sequentially.

[0114] Referring now to FIG. 1C, in some configurations, the AV may be configured to deliver cargo and / or perform other functions involving autonomously navigating to a desired location. In some applications, the AV may be remotely guided. In some configurations, the AV 20100 includes a cargo container that may be opened remotely, automatically in response to user input, or manually to allow a user to place or remove luggage and other items. The cargo container 20110 is mounted on a cargo platform 20160 that is mechanically connected to a power base 20170. The power base 20170 includes four powered wheels 20174 and two caster wheels 20176. The power base provides speed and directional control to move the cargo container 20110 along the ground and over obstacles, including curbs and other discontinuous surface features.

[0115] 1C , the cargo platform 20160 is connected to the power base 20170 through two U-frames 20162. Each U-frame 20162 is rigidly attached to the structure of the cargo platform 20160 and includes two holes that allow a rotatable joint 20164 to be formed with the end of each arm 20172 on the power base 20170. The power base controls the rotational position of the arms and therefore the height and tilt angle of the cargo container 20110.

[0116] Continuing to refer to FIG. 1C, in some configurations, the AV20100 includes one or more processors for receiving data, navigating routes, and selecting the direction and speed of the power base 20170.

[0117] 1D, in some configurations, a map processor 10104 of the present teachings can locate the SDSF on a map. The map processor 10104 can include, but is not limited to, a feature extractor 10801, a point cloud organizer 10803, a transient processor 10805, a segmenter 10807, a polygon generator 10809, an SDSF line generator 10811, and a data combiner 10813.

[0118] Continuing with reference to FIG. 1D, the feature extractor 10801 (FIG. 1-2) can include, but is not limited to, line-of-sight filtering 10121 of the point cloud data 10131 and the mapped trajectory 10133. Line-of-sight filtering can remove points that are hidden from the direct line of sight of the sensor that collects the point cloud data and forms the mapped trajectory. The point cloud organizer 10803 (FIG. 1-2) can organize (10151) the reduced point cloud data 10132, possibly according to preselected criteria associated with specific features. In some configurations, the transient processor 10805 (FIG. 1-2) can remove (10153) transient points from the organized point cloud data and the mapped trajectory 10133 by any number of methods, including those described herein. Transient points can complicate processing, especially when the specific features are stationary. A segmenter 10807 (FIG. 1-2) can divide the processed point cloud data 10135 into processable chunks. In some configurations, the processed point cloud data 10135 can be segmented (10155) into sections having a preselected minimum number of points, for example, but not limited to, about 100,000 points. In some configurations, further point reduction can be based on preselected criteria that can be related to the features to be extracted. For example, if points above a certain height are not important to locating the features, those points can be deleted from the point cloud data. In some configurations, the height of at least one of the sensors collecting the point cloud data can be considered as an origin, and points above the origin can be removed from the point cloud data, for example, when only points of interest are associated with surface features. After the filtered point cloud data 10135 is segmented, forming segments 10137, the remaining points can be divided into drivable surface sections and surface features can be located. In some configurations, a polygon generator 10809 (FIGS. 1-2) can locate the drivable surface by generating (10161) a polygon 10139, for example, but not limited to, as described herein.In some configurations, the SDSF line generator 10811 (FIG. 1-2) can locate surface features by generating (10163) an SDSF line 10141, for example, but not by way of limitation, as described herein. In some configurations, the combiner 10813 (FIG. 1-2) can combine (10165) the polygon 10139 and the SDSF 10141 to create a data set that can be further processed to generate an actual path along which the AV 10101 (FIG. 1A) may proceed.

[0119] Now, referring primarily to FIG. 1E, eliminating 10153 (FIG. 1D) objects that are transient to the mapped trajectory 10133, such as the exemplary time-stamped point 10751, from the point cloud data 10131 (FIG. 1D) may include casting a ray 10753 from the time-stamped point on the mapped trajectory 10133 to each time-stamped point in the point cloud data 10131 (FIG. 1D) that has a substantially identical time stamp. If the ray 10753 intersects a point between the time-stamped point on the mapped trajectory 10133 and the end point of the ray 10753, e.g., point D10755, then it may be assumed that the intersection point D10755 entered the point cloud data during a different sweep of the camera. The intersection point, e.g., intersection point D10755, may be assumed to be part of a transient object and may be removed from the reduced point cloud data 10132 (FIG. 1D) as not representing a fixed feature, such as an SDSF. The result is, for example, but not limitation, processed point cloud data 10135 (FIG. 1D) that is free of transient objects. Points that were removed as part of a transient object, but that are also substantially at ground level, can be returned (10754) to the processed point cloud data 10135 (FIG. 1D). Transient objects may not include certain features, such as, for example, but not limitation, SDSF 10141 (FIG. 1D), and therefore can be removed without interfering with the integrity of the point cloud data 10131 (FIG. 1D) when SDSF 10141 (FIG. 1D) is a feature that is being detected.

[0120] With continued reference to FIG. 1E, segmenting 10155 (FIG. 1D) the processed point cloud data 10135 (FIG. 1D) can generate sections 10757 having a preselected size and shape, for example, but not limited to, a rectangle 10154 (FIG. 1F) having a minimum preselected side length and containing approximately 100,000 points. From each section 10757, points that are not necessarily relevant to the specific task, for example, but not limited to, points located above a preselected level, can be removed (10157) (FIG. 1D) to reduce the dataset size. In some configurations, the preselected level can be the height of AV10101 (FIG. 1A). Removing these points can lead to more efficient processing of the dataset.

[0121] Referring again primarily to FIG. 1D, the map processor 10104 can provide the device controller 10111 with at least one dataset that can be used to generate direction, speed, and configuration commands for controlling the AV 10101 (FIG. 1A). The at least one dataset can include points that can be connected to other points in the dataset, and each line connecting points in the dataset traverses the drivable surface. To determine such route points, the segmented point cloud data 10137 can be divided into polygons 10139, and the vertices of the polygons 10139 can potentially become route points. The polygons 10139 can include features such as, for example, SDSFs 10141.

[0122] Continuing with reference to FIG. 1D, in some configurations, creating the processed point cloud data 10135 can include filtering the voxels. To reduce the number of points that will undergo future processing, in some configurations, the centroid of each voxel in the dataset can be used to approximate the points in the voxel, and all points except the centroid can be excluded from the point cloud data. In some configurations, the center of the voxel can be used to approximate the points in the voxel. Other methods for reducing the size of the filtered segment 10251 (FIG. 1G) can also be used, such as taking a random point subsample, such that a fixed number of points, selected uniformly at random, can be excluded from the filtered segment 10251 (FIG. 1G), for example, but not limited to, taking a random point subsample.

[0123] Continuing with still further reference to FIG. 1D, in some configurations, creating the processed point cloud data 10135 can include calculating normals from a dataset from which outliers have been removed and which has been reduced in size through voxel filtering. The normals for each point in the filtered dataset can be used for various processing possibilities, including curve reconstruction algorithms. In some configurations, estimating and filtering normals in the dataset can include obtaining an underlying surface from the dataset using a surface meshing technique and calculating normals from the surface mesh. In some configurations, estimating normals can include using approximations to infer surface normals directly from the dataset, such as, for example, but not limited to, determining normals to a fitting plane obtained by applying a total least squares method to k nearest neighbors of the point. In some configurations, the value of k can be chosen based at least on empirical data. Filtering normals can include removing any normals that are more than about 45° from perpendicular to the xy plane. In some configurations, a filter can be used to align normals to the same direction. If a portion of the data set represents a planar surface, redundant information contained in adjacent normals can be filtered out, either by performing random subsampling, or by filtering out a point from the set of relevant points. In some configurations, selecting a point can include recursively decomposing the data set into boxes until each box contains at most k points. A single normal can be calculated from the k points in each box.

[0124] Continuing with reference to FIG. 1D, in some configurations, creating the processed point cloud data 10135 can include growing regions in the dataset by clustering points that geometrically fit with a surface representing the dataset, and refining the surface as the region grows to obtain the best approximation of the maximum number of points. Region growing can merge points in terms of a smoothness constraint. In some configurations, the smoothness constraint can be determined empirically, for example, or can be based on a desired surface smoothness. In some configurations, the smoothness constraint can include a range of about 10π / 180 to about 20π / 180. The output of region growing is a set of point clusters, each point cluster being a set of points, each of which is considered to be part of the same smooth surface. In some configurations, region growing can be based on a comparison of angles between normals. Region growing can be accomplished by algorithms such as, for example, but not limited to, region growing segmentation http: / / pointclouds.org / documentation / tutorials / region_growing_segmentation.php and http: / / pointclouds.org / documentation / tutorials / cluster_extraction.php#cluster-extraction.

[0125] Referring now to FIG. 1G, the segmented point cloud data 10137 (FIG. 1D) can be used to generate (10161) (FIG. 1D) a polygon 10759, for example a 5m×5m polygon. The point sub-clusters can be converted into polygons 10759, for example using meshing. Meshing can be accomplished by standard methods such as, for example, but not limited to, marching cubes, marching tetrahedrons, surface nets, greedy meshing, and dual contouring. In some configurations, the polygon 10759 can be generated by projecting a local neighborhood of points along the normals of the points and connecting unconnected points. The resulting polygon 10759 can be based on at least the size of the neighborhood, the maximum allowable distance for the points to be considered, the maximum edge length for the polygon, the minimum and maximum angles of the polygon, and the maximum deviation the normals can have from each other. In some configurations, the polygons 10759 can be filtered according to whether the polygons 10759 would be too small for the AV 10101 (FIG. 1A) to traverse. In some configurations, a circle the size of the AV 10101 (FIG. 1A) can be dragged around each of the polygons 10759 by known means. If the circle falls substantially within the polygon 10759, the polygon 10759, and thus the resulting drivable surface, can accommodate the AV 10101 (FIG. 1A). In some configurations, the area of ​​the polygon 10759 can be compared to the occupied area of ​​the AV 10101 (FIG. 1A). The polygon can be assumed to be irregular, and thus the first step in determining the area of ​​the polygon 10759 would be to separate the polygon 10759 into regular polygons 10759A by known methods. For each regular polygon 10759A, a standard area equation can be used to determine its size. The areas of each regular polygon 10759A can be added together to find the area of ​​the polygon 10759, which can be compared to the occupied area of ​​the AV 10101 (FIG. 1A). The filtered polygons can include a subset of polygons that meet the size criteria.The filtered polygons can be used to set the final drivable surface.

[0126] 1G, in some configurations, polygon 10759 can be processed by removing outliers by conventional means such as, for example, but not limited to, statistical analysis techniques such as those available in the Point Cloud Library, http: / / pointclouds.org / documentation / tutorials / statistical_outlier.php. Filtering can include downsizing segments 10137 (FIG. 1D) by conventional means, including, but not limited to, voxelization grid approaches such as those available in the Point Cloud Library, http: / / pointclouds.org / documentation / tutorials / voxel_grid.php. Concave polygon 10263 can be created by, for example, but not limited to, the process described in “A New Concave Hull Algorithm and Concaveness Measure for n-dimensional Datasets”, Park et al., Journal of Information Science and Engineering 28, pp. 587-600, 2012.

[0127] Now referring primarily to FIG. 1H, in some configurations, the processed point cloud data 10135 (FIG. 1D) can be used to determine an initial drivable surface 10265. Region growing can generate point clusters that may include points that are part of the drivable surface. In some configurations, a reference plane can be fitted to each of the point clusters to determine the initial drivable surface. In some configurations, the point clusters can be filtered according to a relationship between the orientation of the point cluster and the reference plane. For example, if the angle between the point cluster plane and the reference plane is below, for example, but not limited to, about 30°, the point cluster can be preliminarily considered to be part of the initial drivable surface. In some configurations, the point clusters can be filtered based on, for example, but not limited to, a size constraint. In some configurations, a point cluster larger in point size than about 20% of the total points in the point cloud data 10131 (FIG. 1D) can be considered too large, and a point cluster smaller in size than about 0.1% of the total points in the point cloud data 10131 (FIG. 1D) can be considered too small. The initial drivable surface can include a filtered version of the point cluster. In some configurations, the point cluster can be split for further processing by any of several known methods. In some configurations, density-based spatial clustering of noisy applications (DBSCAN) can be used to split the point cluster, while in some configurations, k-means clustering can be used to split the point cluster. DBSCAN can group points that are dense together and mark points that are substantially isolated or in low density areas as outliers. To be considered dense, a point must be located within a preselected distance from a candidate point. In some configurations, a scale factor for the preselected distance can be determined empirically or dynamically. In some configurations, the scale factor may be in the range of approximately 0.1 to 1.0.

[0128] Referring primarily to FIG. 1I, generating the SDSF line (10163) (FIG. 1D) can include locating the SDSF by further filtering of the concave polygon 10263 on the drivable surface 10265 (FIG. 1H). In some configurations, the points from the point cloud data that make up the polygon can be categorized as either an upper donut point 10351 (FIG. 1J), a lower donut point 10353 (FIG. 1J), or a cylinder point 10355 (FIG. 1J). The upper donut point 10351 (FIG. 1J) can correspond to the shape of the SDSF model 10352 that is furthest from the ground. The lower donut point 10353 (FIG. 1J) corresponds to the shape of the SDSF model 10352 that is closest to the ground or at ground level. The cylinder points 10355 (FIG. 1J) can fall into a shape between the upper donut points 10351 (FIG. 1J) and the lower donut points 10353 (FIG. 1J). A combination of categories can form a donut 10371. To determine whether a donut 10371 forms an SDSF, certain criteria are tested. For example, in each donut 10371, there must be a minimum number of points that are upper donut points 10351 (FIG. 1J) and a minimum number that are lower donut points 10353 (FIG. 1J). In some configurations, the minimum value can be empirically selected and can fall in the range of about 5 to 20. Each donut 10371 can be divided into multiple portions, for example, two hemispheres. Another criterion for determining whether points within a donut 10371 represent an SDSF is whether a majority of the points lie within opposing hemispheres of the portions of the donut 10371. The cylindrical points 10355 (FIG. 1J) can occur in either the first cylindrical region 10357 (FIG. 1J) or the second cylindrical region 10359 (FIG. 1J). Another criterion for SDSF selection is that a minimum number of points must be present in both cylindrical regions 10357 / 10359 (FIG. 1J). In some configurations, the minimum number of points can be selected empirically and can fall in the range of 3-20.Another criterion for SDSF selection is that the donut 10371 must contain at least two of three categories of points: upper donut points 10351 (FIG. 1J), lower donut points 10353 (FIG. 1J), and cylinder points 10355 (FIG. 1J).

[0129] Continuing with primary reference to FIG. 1I, in some configurations, polygons can be processed in parallel. Each category worker 10362 can search its assigned polygons for SDSF points 10789 (FIG. 1N) and can assign the SDSF points 10789 (FIG. 1N) to categories 10763 (FIG. 1G). As the polygons are processed, the resulting point categories 10763 (FIG. 1G) can be combined (10363) to form combined category 10366, and the categories can be shortened (10365) to form shortened combined category 10368. Shortening the SDSF points 10789 (FIG. 1N) can include filtering the SDSF points 10789 (FIG. 1N) with respect to their distance from the ground. The shortened combined categories 10368 can be averaged by searching the area around each SDSF point 10766 (FIG. 1G) and generating an average point 10765 (FIG. 1G), possibly processed in parallel by an average worker 10373, and the points of the categories can form a set of averaged donuts 10375. In some configurations, the radius around each SDSF point 10766 (FIG. 1G) can be empirically determined. In some configurations, the radius around each SDSF point 10766 (FIG. 1G) can include a range of 0.1 m to 1.0 m. The height change between one point on the SDSF trajectory 10377 (FIG. 1G) and another for the SDSF at the average point 10765 (FIG. 1G) can be calculated. Connecting the averaged donuts 10375 together can generate the SDSF trajectory 10377 (FIGS. 1G and 1K). In creating the SDSF trajectory 10377 (FIGS. 1G and 1K), if there are two candidate next points within a search radius of the starting point, the next point can be selected based on at least forming a straight line between the previous line segment, the starting point, and the candidate destination point as far as possible, where the candidate next point represents the smallest change in SDSF height between the previous point and the candidate next point. In some configurations, the SDSF height can be defined as the difference between the heights of the upper donut 10351 (FIG. 1J) and the lower donut 10353 (FIG. 1J).

[0130] Now referring primarily to FIG. 1L, combining concave polygons and SDSF lines 10165 (FIG. 1D) can generate a dataset including polygon 10139 (FIG. 1D) as well as SDSF 10141 (FIG. 1D), which can be manipulated to generate a graphed polygon using the SDSF data. Manipulating concave polygon 10263 can include, but is not limited to, merging concave polygon 10263 to form merged polygon 10771. Merging concave polygon 10263 can be accomplished using known methods, such as, for example, but not limited to, those found at (http: / / www.angusj.com / delphi / clipper.php). Merged polygon 10771 can be expanded to smooth edges and form expanded polygon 10772. The expanded polygon 10772 can be contracted to provide a driving margin to form a contracted polygon 10774, to which the SDSF trajectory 10377 (FIG. 1M) can be added. Inward trimming (contraction) can ensure that there is room near the edge for the AV10101 (FIG. 1A) to proceed by reducing the size of the drivable surface by at least a preselected amount based on the size of the AV10101 (FIG. 1A). Polygon expansion and contraction can be accomplished by commercially available techniques such as, for example, but not limited to, the ARCGIS® Clip command (http: / / desktop.arcgis.com / en / arcmap / 10.3 / manage-data / editing-existing-features / clipping-a-polygon-feature.htm).

[0131] Referring now primarily to FIG. 1M, the contracted polygon 10774 can be partitioned into polygons 10778, each of which can be traversed without encountering a non-drivable surface. The contracted polygon 10774 can be partitioned by conventional means such as, for example, but not limited to, ear slicing, optimized by z-order curve hashing and extended to handle holes, twisted polygons, degeneracy, and self-intersections. Commercially available ear slicing implementations can include, but are not limited to, those found at (https: / / github.com / mapbox / earcut.hpp). The SDSF trajectory 10377 can include SDSF points 10789 (FIG. 1N) that can be connected to the polygon vertices 10781. The vertices 10781 can be considered to be possible path points that can be connected to each other to form possible travel paths for the AV 10101 (FIG. 1A). In the data set, SDSF points 10789 (FIG. 1N) can be labeled as such. As partitioning progresses, it is possible that redundant edges are introduced, such as, for example, but not limited to, edges 10777 and 10779. Removing one of edges 10777 or 10779 can reduce the complexity of further analysis and can preserve the polygonal mesh. In some configurations, a Hertel-Mehlhorn polygon partitioning algorithm can be used to remove edges and omit edges labeled as features. The set of polygons 10778, including the labeled features, can undergo further simplification to reduce the number of possible path points, which can be provided to the device controller 10111 (FIG. 1A) in the form of annotated point data 10379 (FIG. 5B), which can be used to populate an occupancy grid.

[0132] 2A-2B, sensor data collected by the AV can also be used to populate the occupancy grid. A processor in the AV can receive data from sensors in a long-range sensor assembly 20400 mounted on the cargo container 20110 and from short-range sensors 20510, 20520, 20530, 20540, and other sensors located in the cargo platform 20160. In addition, the processor may receive data from an optional short-range sensor 20505 mounted near the top of the front of the cargo container 20110. The processor may also receive data from one or more antennas 20122A, 20122B (FIG. 1C), including cellular, WiFi, and / or GPS. In one embodiment, the AV 20100 has a GPS antenna 20122A (FIG. 1C) located on the long-range sensor assembly 20400 and / or an antenna 20122B (FIG. 1C) located on the cargo container 20110. The processor may be located anywhere within the AV 20100. In some embodiments, one or more processors are located within the long range sensor assembly 20400. Additional processors may be located within the cargo platform 20160. In other embodiments, the processors may be located within the cargo container 20110 and / or as part of the power base 20170.

[0133] 2A-2B, the long-range sensor assembly 20400 is mounted on top of the cargo container to provide an improved view of the environment surrounding the AV. In one embodiment, the long-range sensor assembly 20400 is more than 1.2 m feet above the travel surface or ground. In other embodiments, if the cargo container is higher or the power base configuration elevates the cargo platform 20160, the long-range sensor assembly 20400 may be 1.8 m above the ground across which the AV is traveling. The long-range sensor assembly 20400 provides information about the environment around the AV from a minimum distance to a maximum range. The minimum distance may be defined by the relative positions of the long-range sensor 20400 and the cargo container 20110. The minimum distance may further be defined by the field of view (FOV) of the sensor. The maximum distance may be defined by the range of the long-range sensor in the long-range sensor assembly 20400 and / or by the processor. In one embodiment, the range of the long-range sensor is limited to 20 meters. In one embodiment, the Velodyne Puck LIDAR has a range of up to 100 m. The long-range sensor assembly 20400 may provide data about objects in all directions. The sensor assembly may provide information about structures, surfaces, and obstacles over a 360° angle around the AV20100.

[0134] Continuing with reference to FIG. 2A, three long-range cameras viewing through windows 20434, 20436, and 20438 can provide horizontal FOVs 20410, 20412, 20414 that together provide a 360° FOV. The horizontal FOV may be defined by the camera selected and the location of the camera within the long-range camera assembly 20400. In describing the field of view, a zero angle is a ray that lies in a vertical plane through the center of the AV 20100 and perpendicular to the front of the AV. A zero angle ray passes through the front of the AV. The front long-range camera viewing through window 20434 has a 96° FOV 20410 that is between 311° and 47°. The left long-range camera viewing through window 20436 has a FOV 20412 that is between 47° and 180°. The right long range camera, viewing through window 20438, has a FOV 20414 of 180° to 311°. The long range sensor assembly 20400 may include an industrial camera positioned to view through window 20432 that provides more detailed information about objects and surfaces in front of the AV 20100 than the long range camera. The industrial camera positioned behind window 20432 may have a FOV 20416 that is defined by the camera selected and the location of the camera within the long range camera assembly 20400. In one embodiment, the industrial camera behind window 20432 has a FOV of 23° to 337°.

[0135] 2B, the LIDAR 20420 provides a 360° horizontal FOV around the AV 20100. The vertical FOV may be limited by the LIDAR instrument. In one embodiment, the vertical FOV 20418 is 40° and mounted 1.2m-1.8m above the ground, setting the minimum distance of the sensor to 3.3m-5m from the AV 20100.

[0136] 2C and 2D, the long range sensor assembly 20400 is shown with a cover 20430. The cover 20430 includes windows 20434, 20432, 20436 through which the long range and industrial cameras observe the environment surrounding the AV 20100. The cover 20430 for the long range sensor assembly 20400 is sealed from the weather by an O-ring between the cover 20430 and the top of the cargo container 20110.

[0137] 2E and 2F, the cover 20430 has been removed to reveal an embodiment of the camera and processor. The LIDAR sensor 20420 provides data regarding range or distance to surfaces around the AV. These data may be provided to a processor 20470, located in the long-range sensor assembly 20400. The LIDAR is mounted on a structure 20405 above the long-range cameras 20440A-C and the cover 20430. The LIDAR sensor 20420 is an example of a ranging sensor based on reflected laser pulsed light. Other ranging sensors, such as radar using reflected radio waves, can also be used. In one example, the LIDAR sensor 20420 is a Puck sensor by VELODYNE LIDAR® (San Jose, CA). The three long-range cameras 20440A, 20440B, 20440C provide digital images of objects, surfaces, and structures around the AV 20100. Three long range cameras 20440A, 20440B, 20440C are arranged around the structure 20405 relative to the cover 20430 to provide three horizontal FOVs covering a full 360° around the AV. The long range cameras 20440A, 20440B, 20440C are on a lift ring structure 20405 that is mounted to the cargo container 20110. The long range cameras 20440A, 20440B, 20440C receive images through windows 20434, 20436, 20438 mounted in the cover 20430. The long range cameras may comprise cameras and lenses on a printed circuit board (PCB).

[0138] 2F, one embodiment of the long range camera 20440A may comprise a digital camera 20444 with a fisheye lens 20442 mounted on the front of the digital camera 20444. The fisheye lens 20442 may expand the FOV of the camera to a much wider angle. In one embodiment, the fisheye lens expands the field of view to 180°. In one embodiment, the digital camera 20444 is similar to an e-cam52A_56540_MOD by E-con Systems (San Jose, CA). In one embodiment, the fisheye lens 20442 is similar to a model DSL227 by Sunex (Carlsbad, CA).

[0139] 2F, the long range sensor assembly 20400 may also include an industrial camera 20450, which receives visual data through a window 20432 in the cover 20430. The industrial camera 20450 provides additional data to the processor 20470 regarding objects, surfaces, and structures in front of the AV. The camera may be similar to a Kowa industrial camera (part number LM6HC). The industrial camera 20450 and the long range cameras 20440A-C are positioned 1.2m to 1.8m above the surface over which the AV 20100 is moving.

[0140] Continuing with reference to FIG. 2F, mounting the long-range sensor assembly 20400 above the cargo container provides at least two advantages. The field of view for the long-range sensors, including the long-range cameras 20440A-C, industrial cameras 20450, and LIDAR 20420, is often less blocked by nearby objects such as people, cars, low walls, etc., when the sensors are mounted further above the ground. In addition, pedestrian-only streets are designed to provide visual cues, including signs, fence heights, etc., for people to perceive, with typical eye heights being in the range of 1.2m to 1.8m. Mounting the long-range sensor assembly 20400 above the cargo container places the long-range cameras 20440A-C, 20450 at the same level as the signs and above the visual cues that are directed at pedestrians. The long-range sensors are mounted on a structure 20405 that provides a substantially rigid mount that withstands deflections caused by the movement of the AV 20100.

[0141] 2E and 2F, the long-range sensor assembly may include an inertial measurement unit (IMU) and one or more processors that receive data from the long-range sensors and output processed data to other processors for navigation. An IMU 20460 with vertical reference (VRU) is mounted on the structure 20405. The IMU / VRU 20460 may be located directly below the LIDAR 20420 to provide position data for the LIDAR 20420. The position and orientation from the IMU / VRU 20460 may be combined with data from other long-range sensors. In one embodiment, the IMU / VRU 20460 is a model MTi 20 supplied by Xsens Technologies (The Netherlands). The one or more processors may include a processor 20465 that receives data from at least the industrial camera 20450. Additionally, the processor 20470 may receive data from at least one of the following: a LIDAR 20420, a long range camera 20440A-C, an industrial camera 20450, and an IMU / VRU 20460. The processor 20470 may be cooled by a liquid-cooled heat exchanger 20475, which is connected to a circulating coolant system.

[0142] Referring now to FIG. 2G, the AV 20100 may include several short-range sensors that detect driving surfaces and obstacles within a predetermined distance from the AV. The short-range sensors 20510, 20520, 20530, 20540, 20550, and 20560 are located on the periphery of the container platform 20160. These sensors are located below the cargo container 20110 (FIG. 2B) and closer to the ground than the long-range sensor assembly 20400 (FIG. 2C). The short-range sensors 20510, 20520, 20530, 20540, 20550, and 20560 are angled downward to provide a FOV that captures surfaces as well as objects that cannot be seen by the sensors in the long-range sensor assembly 20400 (FIG. 2C). The field of view of the sensors that are located closer to the ground and angled downward is less likely to be obstructed by nearby objects and pedestrians than sensors mounted further from the ground. In one embodiment, the short-range sensors provide information about the ground surface and objects up to 4m from the AV20100.

[0143] Referring again to FIG. 2B, the vertical FOV for two of the short-range sensors is shown in a side view of the AV 20100. The vertical FOV 20542 of the rear-facing sensor 20540 is centered on a centerline 20544. The centerline 20544 is angled below the upper surface of the cargo platform 20160. In one embodiment, the sensor 20540 has a vertical FOV of 42° and a centerline 20546 that is angled 22°-28° below a plane 20547 defined by the top plate of the cargo platform 20160. In one embodiment, the short-range sensors 20510 and 20540 are approximately 0.55m-0.71m above the ground. The resulting FOV 20512, 20542 covers the ground 0.4m-4.2m from the AV. The short-range sensors 20510, 20520, 20530, 20550 (FIG. 2G), 20560 (FIG. 2G) mounted on the cargo base 20160 have similar vertical fields of view and centerline angles relative to the top of the cargo platform. The short-range sensors mounted on the cargo platform 20160 can view the ground 0.4 to 4.7 meters from the outer edge of the AV 20100.

[0144] 2B, a short-range sensor 20505 may be mounted on the front surface near the top of the cargo container 20110. In one embodiment, the sensor 20505 may provide an additional view of the ground in front of the AV to the view provided by the short-range sensor 20510. In another embodiment, the sensor 20505 may provide a view of the ground in front of the AV in place of the view provided by the short-range sensor 20510. In one embodiment, the short-range sensor 20505 may have a vertical FOV 20507 of 42°, and the angle of the centerline relative to the top of the cargo platform 20160 is 39°. The resulting view of the ground extends from 0.7m to 3.75m from the AV.

[0145] Referring again to FIG. 2G, the horizontal FOVs of the short-range sensors 20510, 20520, 20530, 20540, 20550, 20560 cover all directions around the AV 20100. The horizontal FOVs 20522 and 20532 of adjacent sensors, such as 20520 and 20530, overlap at some distance away from the AV 20100. In one embodiment, the horizontal FOVs 20522, 20532 and 20562, 20552 of adjacent sensors 20520, 20530 and 20560, 20550 overlap at 0.5 to 2 meters from the AV. The short-range sensors are distributed around the perimeter of the cargo base 20160, have horizontal fields of view, and are installed at specific angles to provide near complete visual coverage of the ground surrounding the AV. In one embodiment, the short range sensors have a horizontal FOV of 69°. The front sensor 20510 faces forward at a zero angle to the AV and has a FOV 20512. In one embodiment, the two front corner sensors 20520, 20560 are angled such that their centerlines are at an angle 20564 of 65°. In one embodiment, the rear sensors 20530, 20550 are angled such that the centerlines of 20530 and 20560 are at an angle 20534 of 110°. Other numbers of sensors with other horizontal FOVs are also possible, mounted around the periphery of the cargo base 20160 to provide a nearly complete view of the ground around the AV 20100 in some configurations.

[0146] 2H, short range sensors 20510, 20520, 20530, 20540, 20550, 20560 are located on the periphery of the cargo base 20160. The short range cameras are mounted in protrusions that set the angle and location of the short range cameras. In another configuration, the sensors are mounted and positioned on the interior of the cargo base and receive visual data through windows that are aligned with the outer skin of the cargo base 20160.

[0147] 2I and 2J, the short range sensor 20600 is mounted within a skin element 20516 of the cargo base 20160 and may include a liquid cooling system. The skin element 20516 includes a formed protrusion 20514 that holds the short range sensor assembly 20600 in a predetermined location and vertical angle relative to the top of the cargo base 20160 and at an angle relative to the front of the cargo base 20160. In some configurations, the short range sensor 20510 is angled downwardly relative to the cargo platform 20160 by 28°, the short range sensors 20520 and 20560 are angled downwardly 18° and forward 25°, the short range sensors 20530 and 20550 are angled downwardly 34° and aft 20°, and the short range sensor 20540 is angled downwardly relative to the cargo platform 20160 by 28°. The skin element 20516 includes a cavity 20517 for receiving the camera assembly 20600. The skin element 20516 may also include a number of elements 20518 for receiving mechanical fasteners, including but not limited to rivets, screws, and buttons. Alternatively, the camera assembly may be mounted with an adhesive or held in place with clips fastened to the skin element 20516. A gasket 20519 can provide a seal against the front of the camera 20610.

[0148] 2K and 2L, the short-range sensor assembly 20600 includes a short-range sensor 20610 mounted on a bracket 20622 that is attached to a water-cooled plate 20626. The outer case 20612, transparent cover 20614, and heat sink 20618 are partially removed in FIG. 2K and 2L to better visualize the heat-dissipating elements of the short-range sensor 20610, the sensor block 20616, and the electronic block 20620. The short-range sensor assembly 20600 may include one or more thermoelectric coolers (TEC) 20630 between the bracket 20622 and the liquid-cooled plate 20626. The liquid-cooled plate 20626 is cooled by coolant pumped through 20628 that is thermally connected to the plate 20626. The TEC is an electrically-driven element with first and second sides. The powered TEC cools the first side while rejecting the thermal energy removed from the first side plus electrical power on the second side. In the short range sensor assembly 20600, the TEC 20630 cools the bracket 20622 and transfers the thermal cooling energy plus electrical energy to the water plate 20626. Alternatively, the TEC 20630 can be used to actively control the temperature of the camera 20600 by varying the magnitude and polarity of the voltage supplied to the TEC 20630.

[0149] Operating the TEC 20630 in cooling mode allows the short-range sensor 20610 to operate at a temperature below the coolant temperature. The bracket 20622 is thermally connected to the short-range sensor 20610 in two locations to maximize cooling of the sensor block 20616 and the electronics block 20620. The bracket 20622 includes tabs 20624 that are thermally attached to the heat sink 20618 via screws 20625. The heat sink 20618 is thermally connected to the sensor block 20616. The bracket is thus thermally connected to the sensor block 20616 via the heat sink 20618, screws 20625, and tabs 20624. The bracket 20622 is also mechanically attached to the electronics block 20620 to provide direct cooling of the electronics block 20620. The bracket 20622 may include multiple mechanical attachments, including but not limited to, screws and rivets that engage with element 20518 of FIG. 2J. The short-range sensor 20610 may incorporate one or more sensors, including but not limited to, a camera, a stereo camera, an ultrasonic sensor, a short-range radar, and an infrared projector and a CMOS sensor. One exemplary short-range sensor is similar to the RealSense Depth Camera D435 by Intel (Santa Clara, California), which includes an IR projector, two imager chips, and an RGB camera.

[0150] 2M-2O, another embodiment of the AV20100 is shown. The AV20100A includes a cargo container 20110 mounted on a cargo platform 20160 and a power base 20170. The AV20100A includes multiple long-range and short-range sensors. A primary long-range sensor is mounted in a sensor pylon 20400A on the cargo container 20110. The sensor pylon may include a LIDAR 20420 and multiple long-range cameras (not shown) oriented in divergent directions to provide a wide field of view. In some configurations, the LIDAR 20420 can be used as described elsewhere herein to provide point cloud data that may enable, for example, but not limited to, capture of an occupancy grid, identify landmarks, locate the AV20100 within its environment, and / or provide information for determining a navigable space. In some configurations, a long-range camera from Leopard Imaging Inc. can be used to identify landmarks, locate the AV20100 within its environment, and / or determine a navigable space.

[0151] 2M-2O, the short-range sensors are primarily mounted within the cargo platform 20160 and provide information about obstacles in the vicinity of the AV20100A. In some embodiments, the short-range sensors provide data about obstacles and surfaces within 4 m of the AV20100A. In some configurations, the short-range sensors provide information up to 10 m from the AV20100A. Multiple cameras, at least partially facing forward, are mounted within the cargo platform 20160. In some configurations, the multiple cameras may include three cameras.

[0152] Referring now to FIG. 2O, the top cover 20830 is partially cut away to reveal the sub-roof 20810. The sub-roof 20810 provides a single piece on which multiple antennas 20820 may be mounted. In one embodiment, ten antennas 20820 are mounted on the sub-roof 20810. Additionally, there are two WiFi antennas, for example, four cellular communication channels each having two antennas. The antennas are wired as a primary antenna for cellular transmission and reception, as well as an auxiliary antenna. The auxiliary antenna may improve cellular functionality in several ways, including, but not limited to, reducing interference and achieving 4G LTE connectivity. The sub-roof 20810 as well as the top cover 20830 are non-metallic. The sub-roof 20810 is a plastic surface within 10 mm to 20 mm of the top cover 20830, which is non-structural and allows the antennas to be connected to the processor before the top cover 20830 is attached. The antenna connections are often high impedance and sensitive to dirt, grease, and misuse. Mounting and connecting the antenna to the subroof 20810 allows the top cover to be installed and removed without touching the antenna connections. Maintenance and repair operations may include removing the top cover without removing the subroof or disconnecting the antenna. Assembly of the antenna separate from installation of the top cover 20830 facilitates testing / repair. The top cover 20830 is weatherproof and prevents water and debris from entering the cargo container 20110. Mounting the antenna on the subroof minimizes the number of openings on the top cover 20830.

[0153] 2P, 2Q, and 2R, another embodiment of a long range sensor assembly (LRSA) 20400A is shown mounted on a cargo container (not shown). The LRSA may include a LIDAR and multiple long range cameras mounted at different locations on the LRSA structure 20950 to provide a panoramic view of the AV20100A's environment. The LIDAR 20420 is mounted at the top on the LRSA structure 20950 to provide an uninterrupted view. The LIDAR may include a VELODYNE LIDAR. Multiple long range cameras 20910A-20910D are mounted on the LRSA structure 20950 on the next level below the LIDAR 20420. In one embodiment, four cameras are mounted, one every 90° around the perimeter of the structure, providing four views of the AV20100's surrounding environment. In some embodiments, the four views will overlap. In some embodiments, each camera is either aligned with the direction of motion or orthogonal to the direction of movement. In one embodiment, one camera is aligned with each of the major faces of the AV20100A, i.e., front, back, left, and right. In one embodiment, the long-range camera is a model LI-AR01 44-MIPI-M12 made by Leopard Imaging Inc. The long-range camera may have a MIPI CSI-2 interface to provide high-speed data transfer to the processor. The long-range camera may have a horizontal field of view of 50° to 70° and a vertical field of view of 30° to 40°.

[0154] 2S and 2T, a long-range processor 20940 is located on the LRSA structure 20950 below the long-range cameras 20910A-20910D and the LIDAR 20420. The long-range processor 20940 receives data from the long-range cameras and the LIDAR. The long-range processor communicates with one or more processors elsewhere in the AV20100A. The long-range processor 20940 provides data derived from the long-range cameras and the LIDAR to one or more processors elsewhere on the AV20100A, described elsewhere herein. The long-range processor 20940 may be liquid-cooled by a cooler 20930. The cooler 20930 may be mounted on a structure below the long-range cameras and the LIDAR. The cooler 20930 may provide a mounting location for the long-range processor 20940. The cooler 20930 is described in U.S. Patent Application Serial No. 16 / 883,668, filed May 26, 2020, and entitled "Apparatus for Electronic Cooling on an Autonomous Device" (Attorney Docket No. AA280), which is incorporated herein by reference in its entirety. The cooler is provided with liquid supply and return conduits that provide cooling liquid to the cooler 20930.

[0155] 2M and 2N, short-range camera assemblies 20740A-C are mounted on the front of the container platform 20160 and angled to collect information about the travel surface and obstacles, steps, curbs, and other substantially discontinuous surface features (SDSFs). The camera assemblies 20740A-C include one or more LED lights to illuminate the travel surface, objects on the ground, and SDSFs.

[0156] 2U-2X, camera assemblies 20740A-B include lights 20732 to illuminate the ground and objects and provide improved image data from camera 20732. Note that camera assembly 20740A is a mirror image of 20740C, and the description of 20740A implicitly applies to 20740C. Camera 20732 may include a single focus camera, a stereo camera, and / or an infrared projector and a CMOS sensor. One example of a camera is the Real-Sense Depth D435 camera by Intel (Santa Clara, CA), which includes an IR projector, two imager chips with lenses, and an RGB camera. LED lights 20734 may be used at night or in low light conditions, or may be used all the time to improve image data. One principle of operation is that the lights create contrast by illuminating the projection surface and creating shadows in recesses. The LED lights may be white LEDs in one example. In one embodiment, the LED light 20734 is an Xlamp XHP50 from Cree Inc. In another embodiment, the LED light may emit infrared light to provide illumination for the camera 20372 without distracting or disturbing nearby pedestrians or drivers.

[0157] 2U-2X, the placement and angle of the light 20374 and the shape of the covers 20736A, 20736B prevent the camera 20732 from seeing the light 20374. The angle and placement of the light 20374 and covers 20736A, 20736B prevent the light from interfering with drivers or obstructing pedestrians. It is advantageous for the camera 20732 not to be exposed to the light 20734 to prevent the sensor in the camera 20732 from being blinded by the light 20734 and thus prevented from detecting weaker light signals from the ground and objects in front and to the sides of the AV20100A. The camera 20732 and / or light 20734 may be cooled using a liquid flowing in and out of the camera assembly through port 20736.

[0158] 2W and 2X, a short-range camera assembly 20740A includes an ultrasonic or sonar short-range sensor 20730A. A second short-range camera assembly 20740C also includes an ultrasonic short-range sensor 20730B (FIG. 2N).

[0159] 2Y, the ultrasonic sensor 20730A is mounted above the camera 20732. In one embodiment, the centerline of the ultrasonic sensor 20730A is parallel to the base of the cargo container 20110, which often means that the sensor 20730A is horizontal. The sensor 20730A is angled 45° from facing forward. The cover 20376A provides a horn 20746 for directing the ultrasonic waves emerging from and received by the ultrasonic sensor 20730A.

[0160] Continuing with reference to FIG. 2Y, a cross section of the camera 20732, light 20734 in the camera assembly 20740A illustrates the angles and openings in the cover 20736. The cameras in the short-range camera assembly are angled downward to better image the ground in front and to the sides of the AV. The center camera assembly 20740B is oriented directly ahead in the horizontal plane. The corner camera assemblies 20740A, 20740C are angled 25° to their respective sides relative to directly ahead in the horizontal plane. The camera 20732 is angled 20° downward relative to the top of the cargo platform in the vertical plane. Since the AV generally holds the cargo platform horizontally, the cameras are therefore angled 20° downward from the horizontal. Similarly, the center camera assembly 20740B (FIG. 2M) is angled 28° downward from the horizontal. In some embodiments, the cameras in the camera assembly may be angled downward by 25°-35°. In another embodiment, the cameras in the camera assemblies may be angled downward by 15°-45°. The LED lights 20734 are similarly angled downward to illuminate the ground imaged by the camera 20732 and minimize distraction to pedestrians. In one embodiment, the LED light centerline 20742 is parallel to the camera centerline 20738 within 5°. The cover 20736A protects both the camera 20732 and pedestrians from the bright light of the LEDs 20734 in the camera assemblies 20740A-C. The cover, which isolates the light emitted by the LEDs, also provides a tapered opening 20737 to maximize the field of view of the camera 20732. The lights are recessed at least 4 mm from the opening in the cover. The light opening is defined by an upper wall 20739 and a lower wall 20744. The upper wall 20739 is approximately parallel (±5°) to the centerline 2074. The lower wall 20744 flares out approximately 18° from the centerline 20742 to maximize illumination of the ground and objects near the ground.

[0161] 2Z-2AA, in one configuration, the light 20734 includes two LEDs 20734A each under a square lens 20734B to generate a beam of light. The LEDs / lenses are angled and positioned relative to the camera 20372 to illuminate the camera's field of view with little light spill outside the camera's 20732 FOV. The two LEDs / lenses are mounted together on a single PCB 20752 with a defined angle 20762 between the two lights. In another configuration, the two LEDs / lenses are mounted individually on a heat sink 20626A on a separate PCB at an angle to each other. In one embodiment, the lights are Xlamp XHP50 from Cree, Inc. and the lenses are 60° lenses HB-SQ-W from LEDil. The lights are angled at about 50° to each other, so the angle 20762 between the front of the lenses is 130°. The light is located approximately 18 mm ( ± 20732 . 5 mm) 20764 rearward and approximately 30 mm 20766 downward from the centerline of the camera 20732 .

[0162] 2AA-2BB, the camera 20732 is cooled by a thermoelectric cooler (TEC) 20630, which is cooled along with the light 20734 by liquid coolant flowing through the cold block 20626A. The camera is attached to a bracket 20622 via a screw 20625 that threads into a sensor block portion of the camera, while the rear of the bracket 20622 is bolted to the electronics block of the camera. The bracket 20622 is cooled by two TECs to maintain the performance of the IR imaging chip (CMOS chip) within the camera 20732. The TECs reject heat from the bracket 20622 and the power they draw to the cold block 20626A.

[0163] 2BB, the coolant is directed through a U-shaped channel created by the center fin 20626D. The coolant flows directly behind the LEDs / lenses / PCB of the light 20734. The fins 20626B, 20626C improve heat transfer from the light 20734 to the coolant. The coolant flows upward and passes by the hot side of the TEC 20630. The fluid channel is created by a plate 20737 (FIG. 2X) that is attached to the rear of the cold block 20626A.

[0164] 3A, sensor data and map data can be used to update an occupancy grid. The systems and methods of the present teachings can manage a global occupancy grid for a device navigating autonomously with respect to a grid map. The grid map can include a route or path that the device may follow from a start point to a destination. The global occupancy grid can include free space indications that can indicate where it is safe for the device to navigate. The possible paths and free space indications can be combined on the global occupancy grid to establish an optimal path on which the device may proceed to safely reach the destination.

[0165] 3A , as the device moves, a global occupancy grid that will be used to determine a non-obstructive navigation route can be accessed based on the location of the device, and the global occupancy grid can be updated as the device moves. The updates can be based on at least current values ​​associated with the global occupancy grid at the device's location, a static occupancy grid that can include historical information about the neighborhood in which the device is navigating, and data collected by sensors as the device progresses. The sensors can be located on the device, as described herein, or they can be located at different locations.

[0166] Continuing to refer still further to FIG. 3A, the global occupancy grid can include cells, which can be associated with occupancy probability values. Each cell of the global occupancy grid can be associated with information such as whether an obstacle has been identified at the cell's location, characteristics and discontinuities of the traveling surface at and surrounding the location, as determined from previously collected data and by data collected as the device navigates, and prior occupancy data associated with the location. Data captured as the device navigates can be stored in a local occupancy grid whose center is the device. When updating the global occupancy grid, the static previously collected data can be combined with the local occupancy grid data and global occupancy data determined in previous updates to create a new global occupancy grid with the space occupied by the device marked as unoccupied. In some configurations, a Bayesian method can be used to update the global occupancy grid. The method may include, for each cell in the local occupancy grid, calculating a location of the cell on a global occupancy grid, accessing a value at that location from a current global occupancy grid, accessing a value at that location from a static occupancy grid, accessing a value at that location from the local occupancy grid, and calculating a new value at that location on the global occupancy grid as a function of the current value from the global occupancy grid, the value from the static occupancy grid, and the value from the local occupancy grid. In some configurations, the relationship used to calculate the new value may include the sum of the static value and the local occupancy grid value minus the current value. In some configurations, the new value may be limited by a preselected value, for example based on a computational limit.

[0167] Continuing with reference to FIG. 3A, the system 30100 of the present teachings can manage a global occupancy grid. The global occupancy grid can start with initial data and can be updated as the device moves. Creating the initial global occupancy grid can include a first process and updating the global occupancy grid can include a second process. The system 30100 can include, but is not limited to, a global occupancy server 30121 that can receive information from various sources and update the global occupancy grid 30505 based at least on the information. The information can be provided by, for example, but is not limited to, sensors located on the device and / or elsewhere, static information, and navigation information. In some configurations, the sensors can include cameras and radars that can detect, for example, surface characteristics and obstacles. The sensors can be advantageously located on the device to provide sufficient surrounding coverage to enable safe navigation by the device, for example. In some configurations, the LIDAR 30103 can provide LIDAR point cloud (PC) data 30201, which can enable populating a local occupancy grid with LIDAR free space information 30213. In some configurations, a conventional ground detection inverse sensor model (ISM) 30113 can process the LIDAR PC data 30201 and generate the LIDAR free space information 30213.

[0168] Continuing with reference to FIG. 3A, in some configurations, the RGB-D camera 30101 can provide RGB-D PC data 30202 and RGB camera data 30203. The RGB-D PC data 30202 can populate a local occupancy grid with depth free-space information 30209, and the RGB-D camera data 30203 can populate a local occupancy grid with surface data 30211. In some configurations, the RGB-D PC data 30202 can be processed by, for example, but not limited to, a conventional stereo free-space ISM 30109, and the RGB-D camera data 30203 can be fed into, for example, but not limited to, a conventional surface detection neural network 30111. In some configurations, the RGB MIPI camera 30105 can provide RGB data 30205, which can be combined with the LIDAR PC data 30201 to generate a local occupancy grid with LIDAR / MIPI free-space information 30215. In some configurations, the RGB data 30205 can be fed into a conventional free-space neural network 30115, the output of which, along with the LIDAR PC data 30201, can undergo a preselected mask 30221 that may identify the portions of the RGB data 30205 that are most important for accuracy before being fed into a conventional 2D-3D registration 30117. The 2D-3D registration 30117 can project an image from the RGB data 30205 onto the LIDAR PC data 30201. In some configurations, the 2D-3D registration 30117 is not required. Any combination of sensors and methods for processing the sensor data can be used to aggregate the data and update the global occupancy grid. Any number of free-space estimation procedures can be used and combined to enable the determination and validation of occupancy probabilities within the global occupancy grid.

[0169] 3A , in some configurations, the historical data can be provided by a repository 30107 of previously collected and processed data, for example, with information associated with the navigation area. In some configurations, the repository 30107 can include route information, such as, for example, but not limited to, polygons 30207. In some configurations, these data can be fed into a conventional polygon parser 30119, which can provide edges 30303, discontinuities 30503, and surfaces 30241 to a global occupancy grid server 30121. The global occupancy grid server 30121 can fuse local occupancy grid data collected by sensors with the processed repository data to determine a global occupancy grid 30505. A grid map 30601 (FIG. 3D) can be created from the global occupancy data.

[0170] 3B, in some configurations, the sensor may include a sonar 30141, which may provide a local occupancy grid with sonar free space 30225 to a global occupancy grid server 30121. The depth data 30209 may be processed by a conventional free space ISM 30143. The local occupancy grid with sonar free space 30225 may be fused with the local occupancy grid with surfaces and discontinuities 30223, the local occupancy grid with LIDAR free space 30213, the local occupancy grid with LIDAR / MIPI free space 30215, the local occupancy grid with stereo free space 30209, as well as edges 30303 (FIG. 3F), discontinuities 30503 (FIG. 3F), navigation points 30501 (FIG. 3F), surface confidence 30513 (FIG. 3F), and surfaces 30241 (FIG. 3F) to form a global occupancy grid 30505.

[0171] 3C-3F, to initialize a global occupancy grid, global occupancy grid initialization 30200 may include creating, by the global occupancy grid server 30121, a global occupancy grid 30505 and a static grid 30249. The global occupancy grid 30505 may be created by blending data from the local occupancy grid 30118 with edges 30303, discontinuities 30503, and surfaces 30241 located within the region of interest. The static grid 30249 (FIG. 3D) may be created to include data such as, for example, but not limited to, surface data 30241, discontinuities data 30503, edges 30303, and polygons 30207. An initial global occupancy grid 30505 can be calculated by adding occupancy probability data from a static grid 30249 (FIG. 3E) to occupancy data derived from collected data from the sensors 30107A and subtracting occupancy data from a prior 30505A (FIG. 3F) of the global occupancy grid 30505. The local occupancy grid 30118 can include, but is not limited to, local occupancy grid data resulting from a stereo free space estimate 30209 (FIG. 3B) through an ISM, local occupancy grid data including surface / discontinuity detection results 30223 (FIG. 3B), local occupancy grid data resulting from a LIDAR free space estimate 30213 (FIG. 3B) through an ISM, and in some configurations, local occupancy grid data resulting from a LIDAR / MIPI free space estimate 30215 (FIG. 3B) following a 2D-3D registration 30117 (FIG. 3B). In some configurations, the local occupancy grid 30118 can include local occupancy grid data coming from the sonar free space estimate 30225 through the ISM. In some configurations, the various local occupancy grids with the free space estimates can be fused into the local occupancy grid 30118 according to a preselected known process. From the global occupancy grid 30505, a grid map 30601 (FIG. 3E) can be created that can include occupancy and surface data in the vicinity of the device.In some configurations, the grid map 30601 (FIG. 3E) and the static grid 30249 (FIG. 3D) can be published using, for example, but not limited to, the Robot Motion System (ROS) subscribe / publish features.

[0172] 3G and 3H, to update the occupancy grid as the device moves, the occupancy grid update 30300 can include updating the local occupancy grid with data measured as the device is moving and combining those data with the static grid 30249. The static grid 30249 is accessed when the device moves out of the occupancy grid range in which it is working. The device can be positioned in the occupancy grid 30245A at a first location 30513A at a first time. When the device moves to a second location 30513B, the device is positioned in the occupancy grid 30245B, which includes a set of values ​​derived from its new location and possibly from the values ​​in the occupancy grid 30245A. Data from the static grid 30249 and surface data from the initial global occupancy grid 30505 (FIG. 3C) that are geographically located with the cells in the occupancy grid 30245B at the second time can be used with the measured surface data as well as occupancy probabilities to update each grid cell according to a preselected relationship. In some configurations, the relationship can include summing static data with measured data. The resulting occupancy grid 30245C at the third time and third location 30513C can be made available to the movement manager 30123 to inform navigation of the device.

[0173] 3I, a method 30450 for creating and managing an occupancy grid can include, but is not limited to, converting sensor measurements to a reference frame associated with the device by a local occupancy grid creation node 30122 30451, creating a time-stamped measurement occupancy grid 30453, and publishing the time-stamped measurement occupancy grid as a local occupancy grid 30234 (FIG. 3G) 30455. A system associated with the method 30450 can include multiple local grid creation nodes 30122, e.g., one per sensor, such that multiple local occupancy grids 30234 (FIG. 3G) can be produced. The sensors can include, but are not limited to, an RGB-D camera 30325 (FIG. 3G), a LIDAR / MIPI 30231 (FIG. 3G), and a LIDAR 30233 (FIG. 3G). A system associated with the method 30450 can include a global occupancy grid server 30121, which can receive the local occupancy grids and process them according to the method 30450. In particular, the method 30450 may include loading a surface 30242, accessing surface discontinuities such as, but not limited to, curbs 30504, and creating a static occupancy grid 30249 30248 from any properties available in the repository 30107, which may include, but are not limited to, surfaces and surface discontinuities. The method 30450 may include receiving a published local occupancy grid 30456, and moving a global occupancy grid to keep the device at the center of the map 30457. The method 30450 may include setting a new area on the map with prior information from the static pre-occupancy grid 30249 30459, and marking an area currently occupied by the device as unoccupied 30461. The method 30450 may perform a loop 30463 for each cell in each local occupancy grid.The loop 30463 may include, but is not limited to, calculating a location of a cell on a global occupancy grid, accessing a previous value at that location on the global occupancy grid, and calculating a new value at the cell location based on a relationship between the previous value and the value at the cell in the local occupancy grid. The relationship may include, but is not limited to, summing the values. The loop 30463 may include comparing the new value against a preselected acceptable probability range and setting the global occupancy grid with the new value. The comparison may include setting the probability to the minimum or maximum acceptable probability if the probability is lower or higher than the minimum or maximum acceptable probability. The method 30450 may include publishing 30467 the global occupancy grid.

[0174] Referring now to FIG. 3J , an alternative method 30150 for creating a global occupancy grid may include, but is not limited to, in 30151, if the device has moved, accessing occupancy probability values ​​associated with the old map area (where the device was before it moved) and updating the global occupancy grid on the new map area (where the device was after it moved) with the values ​​from the old map area 30153, accessing drivable surfaces associated with cells of the global occupancy grid in the new map area and updating the cells in the updated global occupancy grid with the drivable surfaces 30155, and proceeding to step 30159. If the device has not moved at 30151 and the global occupancy grid is co-located with the local occupancy grid, the method 30150 may include updating 30159 the potentially updated global occupancy grid with surface confidences associated with drivable surfaces from at least one local occupancy grid, updating 30161 the updated global occupancy grid with log odds of occupancy probability values ​​from at least one local occupancy grid, for example, but not limited to, using a Bayesian function, and adjusting 30163 the log odds based on at least a characteristic associated with the location. If the global occupancy grid is not co-located with the local occupancy grid at 30157, the method 30150 may include returning to step 30151. The characteristic may include, but is not limited to, setting the location of the device as unoccupied.

[0175] 3K, in another configuration, a method 30250 for creating a global occupancy grid may include, without limitation, at 30251, if the device is moved, updating 30253 the global occupancy grid with information from a static grid associated with the new location of the device. The method 30250 may include analyzing 30257 the surface at the new location. At 30259, if the surface is drivable, the method 30250 may include updating 30261 the surface on the global occupancy grid and updating 30263 the global occupancy grid with values ​​from a repository of static values ​​associated with the new location on the map.

[0176] 3L, updating the surface 30261 may include, but is not limited to, accessing 30351 a local occupancy grid (LOG) for a particular sensor. If there are additional cells in the local occupancy grid to process at 30353, the method 30261 may include accessing 30355 a surface classification confidence value and a surface classification from the local occupancy grid. If the surface classification at the cell in the local occupancy grid is identical to the surface classification in the global occupancy grid at the location of the cell at 30357, the method 30261 may include setting 30461 a new global occupancy grid (GOG) surface confidence to the sum of the old global occupancy grid surface confidence and the local occupancy grid surface confidence. If the surface classification at the cell in the local occupancy grid is not identical to the surface classification in the global occupancy grid at the location of the cell at 30357, the method 30261 may include setting 30359 a new global occupancy grid surface confidence to the difference between the old global occupancy grid surface confidence and the local occupancy grid surface confidence. In 30463, if the new global occupancy grid surface confidence is less than zero, the method 30261 may include setting 30469 the new global occupancy grid surface classification to the value of the local occupancy grid surface classification.

[0177] 3M, updating the global occupancy grid with values ​​from a repository of static values ​​30263 may include, but is not limited to, the following: if there are more cells to process in the local occupancy grid at 30361, the method 30263 may include accessing the log odds from the local occupancy grid 30363 and updating the log odds in the global occupancy grid with the value from the local occupancy grid at that location 30365. If a maximum certainty that the cell is empty is met at 30367, and if the device is proceeding within a given lane barrier at 30369, and if the surface is drivable at 30371, the method 30263 may include updating the probability that the cell is occupied 30373 and returning to continue processing more cells. If the maximum certainty that the cell is empty is not reached at 30367, or if the device is not proceeding in a given lane at 30369, or if the surface is not drivable in the mode in which the device is currently proceeding at 30371, the method 30263 may include returning to consider additional cells without updating the log-odds. If the device is in a standard mode, i.e., a mode in which the device may navigate a relatively uniform surface, and the surface classification indicates that the surface is relatively non-uniform, the method 30263 may adjust the device's path by increasing the probability that the cell is occupied by updating (30373) the log-odds. If the device is in a standard mode and the surface classification indicates that the surface is relatively uniform, the method 30263 may adjust the device's path by decreasing the probability that the cell is occupied by updating (30373) the log-odds. If the device is traveling in four-wheel mode, ie, a mode in which the device can navigate uneven terrain, adjustment of the probability that the cell is occupied may not be necessary.

[0178] 4A, the AV can proceed in a specific mode that may be associated with a device configuration, for example, the configuration depicted in device 42114A and the configuration depicted in device 42114B. A system of the present teachings for real-time control of the configuration of the device based on at least one environmental factor and the status of the device can include, without limitation, a sensor, a moving means, a chassis operably coupled to the sensor and the moving means, the moving means being driven by a motor and a power supply, a device processor that receives data from the sensor, and a power base processor that controls the moving means. In some configurations, the device processor can receive environmental data, determine environmental factors, determine configuration changes according to the environmental factors and the status of the device, and provide the configuration changes to the power base processor. The power base processor can issue commands to the moving means to move the device from location to location and physically reconfigure the device when required by the road surface type.

[0179] Continuing with reference to FIG. 4A , sensors that collect environmental data can include, for example, but not limited to, cameras, LIDAR, radar, thermometers, pressure sensors, and weather condition sensors, some of which are described herein. From this set of data, the device processor can determine environmental factors upon which device configuration changes can be based. In some configurations, the environmental factors can include surface factors, such as, for example, but not limited to, surface type, surface features, and surface conditions. The device processor can determine in real time, based on the environmental factors and the current status of the device, how to change the configuration to accommodate traversing the detected surface type.

[0180] Continuing with reference to FIG. 4A, in some configurations, the configuration change of the devices 42114A / B / C (collectively referred to as the devices 42114) can include, for example, a change in the configuration of the means of locomotion. Other configuration changes, such as user information displays and sensor controls, that may depend on the current mode and surface type, are also envisioned. In some configurations, the means of locomotion can include at least four drive wheels 442101, two on each side of the chassis 42112, and at least two caster wheels 42103 operably coupled to the chassis 42112, as described herein. In some configurations, the drive wheels 442101 can be operably coupled in pairs 42105, each pair 42105 can include a first drive wheel 42101A and a second drive wheel 42101B of the four drive wheels 442101, the pairs 42105 being located on opposite sides of the chassis 42112, respectively. The operable coupling can include a wheel cluster assembly 42110. In some configurations, a power base processor 41016 (FIG. 4B) can control the rotation of the cluster assembly 42110. Left and right wheel motors 41017 (FIG. 4B) can drive the wheels 442101 on either side of the chassis 42112. Turning can be accomplished by driving the left and right wheel motors 41017 (FIG. 4B) at different rates. A cluster motor 41019 (FIG. 4B) can rotate the wheel base in a forward / rearward direction. The rotation of the wheel base can allow the cargo to rotate independently of the drive wheels 442101, if at all, while the front drive wheels 442101A are higher or lower than the rear drive wheels 442101B, for example, when encountering a discontinuous surface feature. The cluster assembly 42110 can independently operate each pair of two wheels 42105, thereby providing forward, reverse, and rotational movement of the device 42114 upon command. The cluster assembly 42110 can provide structural support for the pairs 42105.The cluster assembly 42110 provides the mechanical power to rotate the wheel drive assemblies together and can enable functions that rely on cluster assembly rotation, such as, but not limited to, climbing discontinuous surface features, various surface types, and uneven terrain. Further details about the operation of the clustered wheels can be found in U.S. Patent Application No. 16,035,205 (Attorney Docket No. X80), filed July 13, 2018, and entitled "Mobility Device," which is incorporated herein by reference in its entirety.

[0181] Continuing with reference to FIG. 4A, the configuration of device 42114 can be associated, including but not limited to, with mode 41033 (FIG. 4B) of device 42114. Device 42114 can operate in several of modes 41033 (FIG. 4B). In standard mode 10100-1 (FIG. 5E), device 42114B can operate on two of drive wheels 442101B and two of caster wheels 42103. Standard mode 10100-1 (FIG. 5E) can provide turning capability and mobility on relatively firm, level surfaces, such as, but not limited to, indoor environments, walkways, and pavements. In the enhanced mode 10100-2 (FIG. 5E) or four-wheel mode, the device 42114A / C can command four of the drive wheels 442101A / B and can be actively stabilized through on-board sensors to raise and / or reorient the chassis 42112, casters 42103, and cargo. The four-wheel mode 10100-2 (FIG. 5E) can provide mobility in a variety of environments and enable the device 42114A / C to climb steep inclines and navigate across soft, uneven terrain. In the four-wheel mode 10100-2 (FIG. 5E), all four of the drive wheels 442101A / B can be deployed and the caster wheels 42103 can be retracted. Rotation of the cluster 42110 can enable operation on uneven terrain and the drive wheels 442101A / B can drive over and across discontinuous surface features. This functionality can provide the device 42114A / C with mobility in a wide variety of outdoor environments. The device 42114B can operate on outdoor surfaces that are firm and stable, but wet. Frost heave and other natural phenomena can degrade outdoor surfaces, causing cracks and loose material. In the four-wheel mode 10100-2 (FIG. 5E), the device 42114A / C can operate on these degraded surfaces.Mode 41033 (FIG. 4B) is described in detail in U.S. Pat. No. 6,571,892 ('892), issued Jun. 3, 2003, and entitled "Control System and Method," which is incorporated herein by reference in its entirety.

[0182] 4B, the system 41000 can drive the device 42114 (FIG. 4A) by processing inputs from the sensor 41031 and generating commands to the wheel motor 41017 to drive the wheel 442101 (FIG. 4A) and to the cluster motor 41019 to drive the cluster 42110 (FIG. 4A). The system 41000 can include, but is not limited to, a device processor 41014 and a power base processor 41016. The device processor 41014 can receive and process environmental data 41022 from the sensor 41031 and provide configuration information 40125 to the power base processor 41016. In some configurations, the device processor 41014 can include a sensor processor 41021 that can receive and process environmental data 41022 from the sensor 41031. The sensor 41031 can include, but is not limited to, a camera, as described herein. From these data, information about the driving surface being traversed, for example, by the device 42114 (FIG. 4A) can be accumulated and processed. In some configurations, the driving surface information can be processed in real time. The device processor 41014 can include a configuration processor 41023 that can determine, for example, a surface type 40121 from the environmental data 41022 that is being traversed by the device 42114 (FIG. 4A). The configuration processor 41023 can include, for example, a driving surface processor 41029 (FIG. 4C) that can create, for example, a driving surface classification layer, a driving surface confidence layer, and an occupancy layer from the environmental data 41022. These data can be used by the power base processor 41016 to create movement commands 40127 and motor commands 40128 as described herein, and can be used by the global occupancy grid processor 41025 to update an occupancy grid that can be used for path planning. The configuration 40125 can be based, at least in part, on the surface type 40121.The surface type 40121 and mode 41033 can be used to determine occupancy grid information 41022, which can include, at least in part, the probability that a cell in the occupancy grid is occupied. The occupancy grid can, at least in part, enable the determination of possible paths that the device 42114 (FIG. 4A) can take.

[0183] 4B, the power base processor 41016 can receive configuration information 40125 from the device processor 41014 and process the configuration information 40125 along with other information, e.g., route information. The power base processor 41016 can include a control processor 40325 that can create movement commands 40127 based on at least the configuration information 40125 and provide the movement commands 40127 to the motor drive processor 40326. The motor drive processor 40326 can generate motor commands 40128 that can direct and move the device 42114 (FIG. 4A). In particular, the motor drive processor 40326 can generate motor commands 40128 that can drive the wheel motors 41017 and can generate motor commands 40128 that can drive the cluster motors 41019.

[0184] 4C, the real-time surface detection of the present teachings can include a configuration processor 41023, which can include, but is not limited to, a travel surface processor 41029. The travel surface processor 41029 can determine characteristics of the travel surface over which the device 42114 (FIG. 4A) is navigating. The characteristics can be used to determine the future configuration of the device 42114 (FIG. 4A). The travel surface processor 41029 can include, but is not limited to, a neural network processor 40207, data conversion 40215, 40219, and 40239, a layer processor 40241, and an occupancy grid processor 40242. Together, these components can generate information that can direct changes in the configuration of the device 42114 (FIG. 4A) and can enable modifications to the occupancy grid 40244 (FIG. 4C) that can inform the path planning for the progression of the device 42114 (FIG. 4A).

[0185] 4C and 4D, the neural network processor 40207 can subject the environmental data 41022 (FIG. 4B) to a trained neural network that can indicate, for each point of data collected by the sensor 41031 (FIG. 4B), the type of surface that the point is likely to represent. The environmental data 41022 (FIG. 4B) can be received as, but not limited to, a camera image 40202, where the camera can be associated with a camera property 40204. The camera image 40202 can include a 2D grid of points 40201 having an X-resolution 40205 (FIG. 4D) and a Y-resolution 40204 (FIG. 4D). In some configurations, the camera image 40202 can include an RGB-D image, where the X-resolution 40205 (FIG. 4D) can include 40,640 pixels and the Y-resolution 40204 (FIG. 4D) can include 40,480 pixels. In some configurations, the camera image 40202 can be converted into an image formatted according to the requirements of the selected neural network. In some configurations, the data can be normalized, scaled, and converted from 2D to 1D, which can improve the processing efficiency of the neural network. The neural network can be trained in many ways, including, but not limited to, training with RGB-D camera images. In some configurations, the trained neural network, represented in a neural network file 40209 (FIG. 4D), can be made available to the neural network processor 40207 through a direct connection to a processor executing the trained neural network, or through, for example, a communication channel. In some configurations, the neural network processor 40207 can use the trained neural network file 40209 (FIG. 4D) to identify surface types 40121 (FIG. 4C) in the environmental data 41022 (FIG. 4B). In some configurations, surface types 40121 (FIG. 4C) can include, but are not limited to, non-driveable, hard driveable, soft driveable, and curb.In some configurations, the surface types 40121 (FIG. 4C) can include, but are not limited to, undriveable / background, asphalt, concrete, brick, packed dirt, wood planks, gravel / pebbles, grass, mulch, sand, curb, solid metal, metal grid, tactile paving, snow / ice, and railroad tracks. The results of the neural network processing can include a surface classification grid 40303 of points 40213 having an X-resolution 40205, a Y-resolution 40203, and a center 40211. Each point 40213 in the surface classification grid 40303 can be associated with a likelihood of being a particular one of the surface types 40121 (FIG. 4C).

[0186] Continuing with reference to Figures 4C and 4D, the travel surface processor 41029 (Figure 4C) may include a 2D-to-3D transformation 40215 that may backproject from the 2D surface classification grid 40303 (Figure 4D) in the 2D camera frame to a 3D image cube 40307 (Figure 4D) in 3D real world coordinates as seen by the camera. The backprojection may restore the 3D nature of the 2D data and transform the 2D image from the RGB-D camera into the 3D camera frame 40305 (Figure 4C). Points 40233 (Figure 4D) in the cube 40307 (Figure 4D) may each be associated with a likelihood of being a particular one of the surface types 40121 (Figure 4C), and a depth coordinate and an X / Y coordinate. The dimensions of the point cube 40307 (FIG. 4D) can be bounded according to the camera properties 40204 (FIG. 4C), such as, for example, but not limited to, focal length x, focal length y, and center of projection 40225. For example, the camera properties 40204 can include a maximum range over which the camera can reliably project. Additionally, there may be features of the device 42114 (FIG. 4A) that can interfere with the image 40202. For example, the caster 42103 (FIG. 4A) may interfere with the view of the camera 40227 (FIG. 4D). These factors can limit the number of points in the point cube 40307 (FIG. 4D). In some configurations, the camera 40227 (FIG. 4D) cannot reliably project beyond about 6 meters, which can represent an upper limit for the range of the camera 40227 (FIG. 4D) and can limit the number of points in the point cube 40307 (FIG. 4D). In some configurations, features of the device 42114 (FIG. 4A) can act as a minimum limit for the range of the camera 40227 (FIG. 4D). For example, the presence of the caster 42103 (FIG. 4A) can imply a minimum limit, which in some configurations may be set at about 1 meter. In some configurations, points in the point cube 40307 (FIG. 4D) can be limited to points 1 meter or more from the camera 40227 (FIG. 4D) and 6 meters or less from the camera 40227 (FIG. 4D).

[0187] 4C and 4D, the driving surface processor 41029 (FIG. 4C) can include a base link transform 40219 that can transform the 3D cube of points into coordinates associated with the device 42114 (FIG. 4A), i.e., the base link frame 40309 (FIG. 4C). The base link transform 40219 can transform a 3D data point 40223 (FIG. 4D) in the cube 40307 (FIG. 4D) to a point 40233 (FIG. 4D) in the cube 40308 (FIG. 4D), whose Z dimension is set to the base of the device 42114 (FIG. 4A). The travel surface processor 41029 (FIG. 4C) can include an OG preparation 40239 that can project points 40233 (FIG. 4D) in the cube 40308 (FIG. 4D) onto an occupancy grid 40244 (FIG. 4D) as points 40237 (FIG. 4D) in the cube 40311 (FIG. 4D). The layer processor 40241 can flatten the points 40237 (FIG. 4D) into various layers 40312 (FIG. 4C) depending on the data represented by the points 40237 (FIG. 4D). In some configurations, the layer processor 40241 can apply a scalar value to the layers 40312 (FIG. 4C). In some configurations, the layer 40312 (FIG. 4C) may include the probability of occupancy layer 40243, a surface classification layer 40245 (FIG. 4D) as determined by the neural network processor 40207, and a surface type confidence layer 40247 (FIG. 4D). In some configurations, the surface type confidence layer 40247 (FIG. 4D) may be determined by converting the class scores from the neural network processor 40207 to scores that may be determined by normalizing the class scores to a probability distribution over the output classes as log(class score) / Σ log(each class). In some configurations, one or more layers may be replaced or augmented by a layer providing a probability of an undriveable surface.

[0188] Continuing with reference to FIGS. 4C and 4D , in some configurations, the probability values ​​in the occupancy layer 40243 can be expressed as log odds (log odds->ln(p / (1-p)) values. In some configurations, the probability values ​​in the occupancy layer 40243 can be based on a combination of at least the mode 41033 ( FIG. 4B ) and the surface type 40121 ( FIG. 4B ). In some configurations, the preselected probability values ​​in the occupancy layer 40243 can be, for example and without limitation, However, (1) when surface type 40121 (FIG. 4A) is hard and runnable and device 42114 (FIG. 4A) is in a preselected set of modes 41033 (FIG. 4B), or (2) when surface type 40121 (FIG. 4A) is soft and runnable and device 42114 (FIG. 4A) is in a specific preselected mode, such as, for example, a standard mode, or (3) when surface type 40121 (FIG. 4A) is soft and runnable and device 42114 (FIG. 4A) is in a specific preselected mode, such as, for example, a standard mode. The probability values ​​may be selected to cover situations such as when the chair 42114 (FIG. 4A) is in a specific preselected mode, such as, for example, a four-wheel mode, or (4) when the surface type 40121 (FIG. 4A) is discontinuous and the device 42114 (FIG. 4A) is in a specific preselected mode, such as, for example, a standard mode, or (5) when the surface type 40121 (FIG. 4A) is discontinuous and the device 42114 (FIG. 4A) is in a specific preselected mode, such as, for example, a four-wheel mode, or (6) when the surface type 40121 (FIG. 4A) is non-drivable and the device 42114 (FIG. 4A) is in a preselected set of modes 41033 (FIG. 4B). In some configurations, the probability values ​​may include, but are not limited to, those listed in Table I. In some configurations, the neural network predicted probabilities may be adjusted, if necessary, to replace the probabilities listed in Table I. [Table 1]

[0189] 4C, the occupancy grid processor 40242 can provide parameters in real time to the global occupancy grid processor 41025 that can affect the probability values ​​of the occupancy grid 40244, such as, for example, but not limited to, surface type 40121 and occupancy grid information 41022. Configuration information 40125 (FIG. 4B), such as, for example, but not limited to, mode 41033 and surface type 40121, can be provided to the power base processor 41016 (FIG. 4B). The power base processor 41016 (FIG. 4B) can determine motor commands 40128 (FIG. 4B), which can set the configuration of the device 42114 (FIG. 4A), based at least on the configuration information 40125 (FIG. 4B).

[0190] 4E and 4F, the device 42100A can be configured according to the present teachings to operate in a standard mode. In the standard mode, the casters 42103 and the second drive wheel 42101B can be resting on the ground as the device 42100A navigates its path. The first drive wheel 42101A can be elevated by a preselected amount 42102 (FIG. 4F) to clear the driving surface. The device 42100A can navigate normally on relatively firm level surfaces. When traveling in the standard mode, the occupancy grid 40244 (FIG. 4C) can reflect the surface type limits (see Table I) and thus can enable adaptive selection of modes or configuration changes based on the surface type and current mode.

[0191] 4G-4J, the device 42100B / C can be configured according to the present teachings to operate in a four-wheel mode. In one configuration in the four-wheel mode, the first drive wheel 42101A and the second drive wheel 42101B can be resting on the ground as the device 42100A navigates its path. The casters 42103 can be retracted and clear the driving surface by a preselected amount 42104 (FIG. 4H). In another configuration in the four-wheel mode, the first drive wheel 42101A and the second drive wheel 42101B can be substantially resting on the ground as the device 42100A navigates its path. The casters 42103 can be retracted and the chassis 42111 can be rotated (thus moving the casters 42103 further from the ground) to accommodate, for example, discontinuous surfaces. In this configuration, the casters 42103 can clear the driving surface by a preselected amount 42108 (FIG. 1J). The device 42100A / C can navigate successfully on a variety of surfaces, including soft and discontinuous surfaces. In another configuration in four-wheel mode, the second drive wheel 42101B can rest on the ground as the device 42100A, while the first drive wheel 42101A can be elevated as the device 42100C (FIG. 1J) navigates its path. The casters 42103 can be retracted and the chassis 42111 can be rotated (thus moving the casters 42103 further from the ground), for example, to accommodate the discontinuous surface. When driving in four-wheel mode, the occupancy grid 40244 (FIG. 4C) can reflect the surface type (see Table I), thus enabling adaptive selection of the mode, or configuration changes based on the surface type and the current mode.

[0192] 4K, a method 40150 for real-time control of a device configuration of a device, such as, for example, but not limited to, an AV, navigating a path based on at least one environmental factor and a device configuration may include, but is not limited to, receiving sensor data 40151, determining a surface type based at least on the sensor data 40153, and determining a current mode based at least on the surface type and the current device configuration 40155. The method 40150 may include determining a next device configuration 40157 based at least on the current mode and surface type, determining a movement command 40159 based at least on the next device configuration, and changing the current device configuration to the next device configuration 40161 based at least on the movement command.

[0193] Referring now primarily to FIG. 5A, annotated point data 10379 (FIG. 5B) can be provided to the device controller 10111 to respond to objects appearing on the path of the AV. The annotated point data 10379 (FIG. 5B), which can be the basis for route information that can be used to instruct the AV 10101 (FIG. 1A) to proceed along a path, can include, but is not limited to, navigable edges, mapped trajectories, such as, but not limited to, the mapped trajectories 10413 / 10415 (FIG. 5D), and labeled features, such as, but not limited to, the SDSF 10377 (FIG. 5C). The mapped trajectories 10413 / 10415 (FIG. 5C) can include a graph of edges of the route space and initial weights assigned to portions of the route space. The graph of edges can include characteristics, such as, but not limited to, directionality and capacity, and edges can be categorized according to these characteristics. The mapped trajectory 10413 / 10415 (FIG. 5C) may include cost modifiers associated with surfaces in the route space and driving modes associated with edges. Driving modes may include, but are not limited to, path following and SDSF climbing. Other modes may include, but are not limited to, operational modes such as autonomous, mapping, and waiting for intervention. Finally, a path may be selected based at least on the lower cost modifier. Features that are relatively far from the mapped trajectory 10413 / 10415 (FIG. 5C) may have higher cost modifiers and may receive less attention when forming a path. The initial weights may be adjusted while the AV 10101 (FIG. 1A) is operating, potentially causing modifications to the path. The adjusted weights may be used to adjust the edge / weight graph 10381 (FIG. 5B) and may be based at least on the current driving mode, the current surface, and the edge category.

[0194] 5A, the device controller 10111 can include a feature processor that can perform specific tasks related to incorporating the eccentricity of any feature into the path. In some configurations, the feature processor can include, but is not limited to, the SDSF processor 10118. In some configurations, the device controller 10111 can include, but is not limited to, the SDSF processor 10118, the sensor processor 10703, the mode controller 10122, and the base controller 10114, each of which are described herein. The SDSF processor 10118, the sensor processor 10703, and the mode controller 10122 can provide inputs to the base controller 10114.

[0195] Continuing with reference to FIG. 5A, the base controller 10114, based on inputs provided by at least the mode controller 10122, the SDSF processor 10118, and the sensor processor 10703, can determine information that the power base 10112 can use to cause the AV10101 (FIG. 1A) to travel on a path determined by the base controller 10114 based at least on the edge / weighted graph 10381 (FIG. 5B). In some configurations, the base controller 10114 can ensure that the AV10101 (FIG. 1A) follows a predetermined path from a start point to a destination and can modify the predetermined path based on at least external and / or internal conditions. In some configurations, the external conditions can include, but are not limited to, stop signals, SDSFs, and obstacles in or near the path being traveled by the AV10101 (FIG. 1A). In some configurations, the internal conditions can include, but are not limited to, mode transitions reflecting responses made by the AV10101 (FIG. 1A) to the external conditions. The device controller 10111 can determine commands to send to the power base 10112 based on at least external and internal conditions. The commands can include, but are not limited to, speed and direction commands that can instruct the AV 10101 (FIG. 1A) to proceed in a commanded direction at a commanded speed. Other commands can include, for example, groups of commands that enable characteristic responses such as SDSF climbing. The base controller 10114 can determine the desired speed between waypoints of the path by conventional methods including, but not limited to, Interior Point Optimizer (IPOPT) large-scale nonlinear optimization (https: / / projects.coin-or.org / Ipopt). The base controller 10114 can determine the desired speed between waypoints of the path by conventional methods including, but not limited to, at least, Dijkstra's algorithm, A *The desired path may be determined based on conventional techniques such as a search algorithm or a technique based on a breadth-first search algorithm. The base controller 10114 may form a box around the mapped trajectory 10413 / 10415 (FIG. 5C) to set an area within which obstacle detection may be performed. The height of the payload carrier, when adjustable, may be adjusted based, at least in part, on the commanded speed.

[0196] Continuing with reference to FIG. 5A, the base controller 10114 can translate speed and direction determinations into motor commands. For example, when encountering an SDSF, such as, but not limited to, a curb or a slope, the base controller 10114 can instruct the power base 10112 in SDSF climb mode to raise the payload carrier 10173 (FIG. 1A), align the AV10101 (FIG. 1A) with the SDSF at approximately a 90° angle, and reduce speed to a relatively low level. When the AV10101 (FIG. 1A) climbs a substantially discontinuous surface, the base controller 10114 can instruct the power base 10112 to transition to a climb phase where speed is increased as increased torque is required to move the AV10101 (FIG. 1A) up the slope. When the AV10101 (FIG. 1A) encounters a relatively level surface, the base controller 10114 can reduce speed to stay on any flat portions of the SDSF. In the case of a downhill ramp associated with a flat portion, the base controller 10114 can allow speed to increase when the AV10101 (FIG. 1A) begins to descend a substantially discontinuous surface and when both wheels are on the downhill ramp. When an SDSF is encountered, such as, for example, but not limited to, a slope, the slope can be identified and treated as a structure. The structural feature can include, for example, a ramp of a preselected size. The ramp can include a slope of about 30° and can optionally be, but is not limited to, on both sides of a plateau. The device controller 10111 (FIG. 5A) can distinguish between an obstacle and a slope by comparing the angle of the perceived feature to an expected slope slope angle, which can be received from the sensor processor 10703 (FIG. 5A).

[0197] Referring now primarily to FIG. 5B, the SDSF processor 10118 can locate navigable edges from the blocks of drivable surface formed by the mesh of polygons represented in the annotated point data 10379 that can be used to create a path for traversal by the AV 10101 (FIG. 1A). Within the SDSF buffer 10407 (FIG. 5C), which can form an area of ​​a preselected size around the SDSF line 10377 (FIG. 5C), the navigable edges can be erased (see FIG. 5D) for special handling, assuming SDSF traversal. A closed line segment, such as segment 10409 (FIG. 5C), can be drawn to bisect the SDSF buffer 10407 (FIG. 5C) between a pair of previously determined SDSF points 10789 (FIG. 1N). In some configurations, for a closed line segment to be considered as a candidate for SDSF traversal, the segment end 10411 (FIG. 5C) can fall on an unobstructed portion of the drivable surface, there can be sufficient room for the AV 10101 (FIG. 1A) to travel along the line segment between adjacent SDSF points 10789 (FIG. 1N), and the area between the SDSF points 10789 (FIG. 1N) can be the drivable surface. The segment end 10411 (FIG. 5C) can be connected to the underlying morphology to form vertices and drivable edges. For example, line segments 10461, 10463, 10465, and 10467 (FIG. 5C) that meet the traversal criteria are shown as part of the morphology in FIG. 5D. In contrast, line segment 10409 (FIG. 5C) did not meet the criteria because, at least, segment end 10411 (FIG. 5C) does not fall on the drivable surface. The overlapping SDSF buffers 10506 (FIG. 5C) can exhibit SDSF discontinuities that can penalize SDSF traversal of SDSFs within the overlapped SDSF buffers 10506 (FIG. 5C). The SDSF lines 10377 (FIG. 5C) can be smoothed and the locations of the SDSF points 10789 (FIG. 1N) can be adjusted such that they fall a preselected distance apart, the preselected distance based at least on the footprint of the AV 10101 (FIG. 1A).

[0198] Continuing with reference to FIG. 5B, the SDSF processor 10118 can convert the annotated point data 10379 into an edge / weighted graph 10381, including morphological corrections for SDSF traversal. The SDSF processor 10118 can include a seventh processor 10601, an eighth processor 10702, a ninth processor 10603, and a tenth processor 10605. The seventh processor 10601 can convert coordinates of points in the annotated point data 10379 into a global coordinate system to achieve compatibility with GPS coordinates and generate a GPS-compatible dataset 10602. The seventh processor 10601 can use conventional processes, such as, for example, but not limited to, affine matrix transformation and PostGIS transformation, to generate the GPS-compatible dataset 10602. The World Geodetic System (WGS) can be used as a standard coordinate system because it takes into account the curvature of the earth. Maps can be stored in the Universal Transverse Mercator (UTM) coordinate system and can be switched to WGS when it is necessary to find where a specific address is located.

[0199] Now, referring primarily to FIG. 5C, the eighth processor 10702 (FIG. 5B) may smooth the SDSF, determine the boundary of the SDSF 10377, and create a buffer 10407 around the SDSF boundary, increasing the cost modifier of the surface as it gets further from the SDSF boundary. The mapped trajectory 10413 / 10415 may be a special case lane with the lowest cost modifier. The lower cost modifier 10406 ​​may generally be located near the SDSF boundary, while the higher cost modifier 10408 may generally be located relatively far from the SDSF boundary. The eighth processor 10702 may provide the point cloud data with cost 10704 (FIG. 5B) to the ninth processor 10603 (FIG. 5B).

[0200] Continuing with reference primarily to FIG. 5C, the ninth processor 10603 (FIG. 5B) can calculate an approximately 90° approach 10604 (FIG. 5B) for the AV 10101 (FIG. 1A) to traverse SDSF 10377 that meets the criteria for labeling them as traversable. The criteria can include SDSF width and SDSF smoothness. Line segments, such as line segment 10409, can be created whose lengths indicate the minimum approach distance that the AV 10101 (FIG. 1A) may require to approach the SDSF 10377 and the minimum exit distance that may be required to exit the SDSF 10377. Segment end points, such as end point 10411, can be integrated with the underlying routing configuration. The criteria used to determine whether an SDSF approach is possible can eliminate the possibility of some approaches. SDSF buffers, such as SDSF buffer 10407, can be used to calculate valid approach and route configuration edge creation.

[0201] Referring again primarily to FIG. 5B, the tenth processor 10605 can create an edge / weight graph 10381 from the morphology, which is a graph of edges and weights developed herein that can be used to calculate a path through the map. The morphology can include cost modifiers and driving modes, and the edges can include directionality and capacity. The weights can be adjusted at runtime based on information from any number of sources. The tenth processor 10605 can provide at least one sequence of ordered points to the base controller 10114 to enable path generation, in addition to a recommended driving mode at a particular point. Each point in each sequence of points represents a location and labeling of a possible path point on the processed drivable surface. In some configurations, the labeling can indicate that the point represents a portion of a feature that may be encountered along a path, such as, for example, but not limited to, an SDSF. In some configurations, the features can be further labeled with suggested processing based on the type of feature. For example, in some configurations, if a path point is labeled as an SDSF, further labeling can include a mode that can be interpreted by the AV10101 (FIG. 1A) as a suggested driving instruction for the AV10101 (FIG. 1A), such as, for example, switching the AV10101 (FIG. 1A) to SDSF climb mode 100-31 (FIG. 5E) to enable the AV10101 (FIG. 1A) to traverse SDSF 10377 (FIG. 5C).

[0202] 5E, in some configurations, the mode controller 10122 can provide instructions to the base controller 10114 (FIG. 5A) to execute a mode transition. The mode controller 10122 can establish the mode in which the AV 10101 (FIG. 1A) is traveling. For example, the mode controller 10122 can provide a change of mode indication to the base controller 10114, e.g., change between a path following mode 10100-32 and an SDSF climb mode 10100-31 when an SDSF is identified along the travel path. In some configurations, the annotated point data 10379 (FIG. 5B) can include a mode identifier at various points along the route, e.g., when the mode is changed to accommodate the route. For example, if the SDSF 10377 (FIG. 5C) is labeled in the annotated point data 10379 (FIG. 5B), the device controller 10111 can determine the mode identifier associated with the route point and possibly adjust instructions to the power base 10112 (FIG. 5A) based on the desired mode. In addition to the SDSF climb mode 10100-31 and the path following mode 10100-32, in some configurations, the AV 10101 (FIG. 1A) can support operational modes that may include, but are not limited to, the standard mode 10100-1 and the enhanced (four-wheel) mode 10100-2 described herein. The height of the payload carrier 10173 (FIG. 1A) can be adjusted to provide the necessary clearance over obstacles and along slopes.

[0203] 5F, a method 11150 for navigating an AV toward a target point that crosses at least one SDSF may include, but is not limited to, receiving SDSF information 11151 related to an SDSF, a target point location, and an AV location. The SDSF information may include, but is not limited to, a set of points classified as SDSF points, and an associated probability for each point that the point is an SDSF point, respectively. The method 11150 may include drawing a closed polygon encompassing the AV location, the target point location, and drawing a path line 11153 between the target point and the AV location. The closed polygon may include a preselected width. Table I includes possible ranges for the preselected variables discussed herein. The method 11150 may include selecting 11155 two of the SDSF points located within the polygon, and drawing 11157 an SDSF line between the two points. In some configurations, the selection of the SDSF points may be random or any other manner. If, at 11159, there is less than a first preselected number of points within a first preselected distance of the SDSF line, and if, at 11161, there is less than a second preselected number of attempts at selecting SDSF points, drawing lines between them, and having less than the first preselected number of points around the SDSF line, the method 11150 may include returning to step 11155. If, at 11161, there is a second preselected number of attempts at selecting SDSF points, drawing lines between them, and having less than the first preselected number of points around the SDSF line, the method 11150 may include conceding 11163 that no SDSF line was detected.

[0204] 5G, at 11159 (FIG. 5F), if there is a first preselected number of points or more, the method 11150 may include fitting a curve to points that fall within a first preselected distance of the SDSF line 11165. If, at 11167, the number of points within the first preselected distance of the curve exceeds the number of points within the first preselected distance of the SDSF line, and if, at 11171, the curve intersects with a path line, and if, at 11173, there are no gaps between points on the curve that exceed a second preselected distance, the method 11150 may include identifying 11175 the curve as an SDSF line. If, at 11167, the number of points within a first preselected distance of the curve does not exceed the number of points within a first preselected distance of the SDSF line, or if, at 11171, the curve does not intersect with the path line, or if, at 11173, there are gaps between points on the curve that exceed a second preselected distance, and if, at 11177, the SDSF line does not remain stable, and if, at 11169, the curve fit has not been attempted more than a second preselected number of attempts, the method 11150 may include returning to step 11165. A stable SDSF line is the result of a subsequent iteration resulting in the same or fewer points.

[0205]

[0046] Referring now primarily to Figure 5H, if the curve fit has been attempted for a second preselected number of attempts at 11169 (Figure 5G) or if the SDSF line remains stable or deteriorates at 11177 (Figure 5G), the method 11150 may include receiving 11179 occupancy grid information. The occupancy grid may provide a probability that an obstacle is present at a point. The occupancy grid information may enhance the SDSF and route information found within a polygon surrounding the AV route and the SDSF when the occupancy grid includes data captured and / or calculated over a common geographic area with the polygon. The method 11150 may include selecting 11181 a point from the common geographic area and its associated probability. If, at 11183, the probability that an obstacle is present at the selected point is higher than a preselected percentage, and if, at 11185, the obstacle is located between the AV and the target point, and if, at 11186, the obstacle is less than a third preselected distance from the SDSF line between the SDSF line and the target point, the method 11150 may include projecting the obstacle onto the SDSF line 11187. If, at 11183, the probability that the location contains an obstacle is less than or equal to the preselected percentage, or if, at 11185, the obstacle is not located between the AV and the target point, or if, at 11186, the obstacle is located at a distance equal to or greater than the third preselected distance from the SDSF line between the SDSF and the target point, and if, at 11189, there are additional obstacles to process, the method 11150 may include resuming processing at step 11179.

[0206] 5I , if there are no more obstacles to process at 11189 (FIG. 5H), the method 11150 may include connecting the projections and finding the end points of the connected projections along the SDSF line 11191. The method 11150 may include marking 11193 a portion of the SDSF line between the projection end points as non-traversable. The method 11150 may include marking 11195 a portion of the SDSF line outside the non-traversable segment as traversable. The method 11150 may include turning 11197 the AV to within a fifth preselected amount perpendicular to the traversable segment of the SDSF line. If at 11199 the orientation error relative to a line perpendicular to the traversable segment of the SDSF line exceeds the first preselected amount, the method 11150 may include decelerating 11251 the AV by a ninth preselected amount. The method 11150 may include driving the AV forward toward the SDSF line and slowing down 11253 a second preselected amount per meter distance between the AV and the crossable SDSF line. If, at 11255, the distance of the AV from the crossable SDSF line is less than a fourth preselected distance, and if, at 11257, the heading error is greater than or equal to a third preselected amount relative to a line perpendicular to the SDSF line, the method 11150 may include slowing down 11252 the AV a ninth preselected amount.

[0207] 5J, at 11257 (FIG. 5I), if the heading error is below a third preselected amount relative to a line perpendicular to the SDSF line, the method 11150 may include ignoring the updated SDSF information and driving the AV at a preselected speed 11260. If the elevation of the front portion of the AV relative to the rear portion of the AV is between a sixth preselected amount and a fifth preselected amount, at 11259, the method 11150 may include driving the AV forward and increasing the speed of the AV to an eighth preselected amount per degree of elevation 11261. If the elevation of the front relative to the rear of the AV is below the sixth preselected amount, at 11263, the method 11150 may include driving the AV forward at a seventh preselected speed 11265. If, at 11267, the rear of the AV is greater than a fifth preselected distance from the SDSF line, the method 11150 may include acknowledging 11269 that the AV has completed traversing the SDSF. If, at 11267, the rear of the AV is less than or equal to the fifth preselected distance from the SDSF line, the method 11150 may include returning to step 11260.

[0208] Referring now to FIG. 5K, a system 51100 for navigating an AV toward a target point that traverses at least one SDSF can include, but is not limited to, a path line processor 11103, an SDSF detector 11109, and an SDSF controller 11127. The system 51100 can be operatively coupled to a surface processor 11601, which can process sensor information, which can include, for example, but is not limited to, images of the surroundings of the AV 10101 (FIG. 5L). The surface processor 11601 can provide real-time surface feature updates, including indications of SDSF. In some configurations, a camera can provide RGB-D data, which points can be classified according to surface type. In some configurations, the system 51100 can process points classified as SDSF and their associated probabilities. The system 51100 can be operatively coupled to a system controller 11602, which can manage aspects of the operation of the AV 10101 (FIG. 5L). The system controller 11602 can maintain an occupancy grid 11138, which may include information from available sources regarding the navigable area in the vicinity of the AV 10101 (FIG. 5L). The occupancy grid 11138 may include the probability that an obstacle is present. This information, in conjunction with the SDSF information, can be used to determine whether the SDSF 10377 (FIG. 5N) can be traversed by the AV 10101 (FIG. 5L) without encountering an obstacle 11681 (FIG. 5M). The system controller 11602 can determine a speed limit 11148 that the AV 10101 (FIG. 5N) should not exceed, based on the environment and other information. The speed limit 11148 can be used as a guide to the speed set by the system 51100, or it can override it. The system 51100 can be operably coupled to a base controller 10114, which can transmit driving commands 11144 generated by the SDSF controller 11127 to the driving components of the AV10101 (FIG. 5L).The base controller 10114 can provide information to the SDSF controller 11127 about the orientation of the AV 10101 (FIG. 5L) during SDSF traversal.

[0209] Continuing with reference to FIG. 5K, the path line processor 11103 can continuously receive surface classification points 10789 in real time, which may include, but are not limited to, points classified as SDSF. The path line processor 11103 can receive the location of the target point 11139 and the AV location 11141, for example, but not limited to, as indicated by the center 11202 (FIG. 5L) of the AV 10101 (FIG. 5L). The system 51100 can include a polygon processor 11105 that draws a polygon 11147 encompassing the AV location 11141, the location of the target point 11139, and the path 11214 between the target point 11139 and the AV location 11141. The polygon 11147 can include a preselected width. In some configurations, the preselected width can include an approximate width of the AV 10101 (FIG. 5L). SDSF points 10789 that fall within polygon 11147 can be identified.

[0210] 5K, the SDSF detector 11109 can receive the surface classification points 10789, the path 11214, the polygon 11147, and the target points 11139 and can determine the most suitable SDSF line 10377 according to the criteria described herein that are available in the incoming data. The SDSF detector 11109 can include, but is not limited to, a point processor 11111 and an SDSF line processor 11113. The point processor 11111 can include selecting two of the SDSF points 10789 that are located within the polygon 11147 and drawing an SDSF 10377 line between the two points. If there are less than a first preselected number of points within a first preselected distance of the SDSF line 10377, and if there are less than a second preselected number of attempts at picking an SDSF point 10789 to draw a line between the two points and have less than the first preselected number of points around the SDSF line, the point processor 11111 may again include looping through a select-draw-test loop as described herein. If there are a second preselected number of attempts at picking an SDSF point to draw a line between them and have less than the first preselected number of points around the SDSF line, the point processor 11111 may include conceding that no SDSF line was detected.

[0211] Continuing with reference to FIG. 5K, the SDSF line processor 11113 may include fitting curves 11609-11611 (FIG. 5L) to points 10789 that fall within a first preselected distance of the SDSF line 10377 if a first preselected number or more of points 10789 are present. If the number of points 10789 within a first preselected distance of the curve 11609-11611 (FIG. 5L) exceeds the number of points 10789 within a first preselected distance of the SDSF line 10377, and if the curve 11609-11611 (FIG. 5L) intersects with the path line 11214, and if there are no gaps between the points 10789 on the curve 11609-11611 (FIG. 5L) that exceed a second preselected distance, the SDSF line processor 11113 may include identifying the curve 11609-11611 (FIG. 5L) as (for example) the SDSF line 10377. If the number of points 10789 within the preselected distance of the curve 11609-11611 (FIG. 5L) does not exceed the number of points 10789 within a first preselected distance of the SDSF line 10377, or if the curve 11609-11611 (FIG. 5L) does not intersect with the path line 11214, or if there is a gap between the points 10789 on the curve 11609-11611 (FIG. 5L) that exceeds a second preselected distance, and if the SDSF line 10377 does not remain stable, and if the curve fit has not been attempted more than a second preselected number of attempts, the SDSF line processor 11113 can run the curve fit loop again.

[0212] Continuing with reference to FIG. 5K, the SDSF controller 11127 can receive the SDSF line 10377, occupancy grid 11138, AV orientation change 11142, and speed limit 11148, and can generate SDSF commands 11144 for navigating the AV 10101 (FIG. 5L) to correctly traverse the SDSF 10377 (FIG. 5N). The SDSF controller 11127 can include, but is not limited to, an obstacle processor 11115, an SDSF approach 11131, and an SDSF traversal 11133. The obstacle processor 11115 can receive the SDSF line 10377, the target point 11139, and the occupancy grid 11138, and can determine from among the obstacles identified in the occupancy grid 11138 whether any of them may obstruct the AV 10101 (FIG. 5N) as it traverses the SDSF 10377 (FIG. 5N). The obstacle processor 11115 may include, but is not limited to, an obstacle selector 11117, an obstacle tester 11119, and a traversal locator 11121. The obstacle selector 11117 may include, but is not limited to, receiving an occupancy grid 11138 as described herein. The obstacle selector 11117 may include selecting occupancy grid points and their associated probabilities from a geographic area that is common to both the occupancy grid 11138 and the polygon 11147. If the probability that an obstacle is present at a selected grid point is higher than a preselected percentage, and if the obstacle is located between the AV10101 (Figure 5L) and the target point 11139, and if the obstacle is less than a third preselected distance from the SDSF line 10377 between the SDSF line 10377 and the target point 11139, the obstacle tester 11119 may include projecting the obstacle onto the SDSF line 10377 and forming a projection 11621 that intersects with the SDSF line 10377.If the probability that the location contains an obstacle is below or equal to a preselected percentage, or if the obstacle is not located between the AV10101 (Figure 5L) and the target point 11139, or if the obstacle is located at a distance equal to or greater than a third preselected distance from the SDSF line 10377 between the SDSF line 10377 and the target point 11139, the obstacle tester 11119 may include resuming execution in receiving the occupancy grid 11138 if there are further obstacles to process.

[0213] 5K, the traversal locator 11121 can include connecting the projection points and locating the end points 11622 / 11623 (FIG. 5M) of the connected projection 11621 (FIG. 5M) along the SDSF line 10377. The traversal locator 11121 can include marking the portion 11624 (FIG. 5M) of the SDSF line 10377 between the projection end points 11622 / 11623 (FIG. 5M) as non-traversable. The traversal locator 11121 can include marking the portion 11626 (FIG. 5M) of the SDSF line 10377 outside the non-traversable portion 11624 (FIG. 5M) as traversable.

[0214] Continuing with reference to FIG. 5K, the SDSF approach 11131 can include transmitting an SDSF command 11144 to redirect the AV 10101 (FIG. 5N) to within a fifth preselected amount perpendicular to the traversable portion 11626 (FIG. 5N) of the SDSF line 10377. If the orientation error relative to a perpendicular line 11627 (FIG. 5N), which is perpendicular to the traversable section 11626 (FIG. 5N) of the SDSF line 10377, exceeds the first preselected amount, the SDSF approach 11131 can include transmitting an SDSF command 11144 to slow down the AV 10101 (FIG. 5N) by a ninth preselected amount. In some configurations, the ninth preselected amount can range from very slow to completely stopped. The SDSF approach 11131 may include transmitting an SDSF command 11144 to drive the AV10101 (FIG. 5N) forward toward the SDSF line 10377 and transmitting an SDSF command 11144 to slow the AV10101 (FIG. 5N) by a second preselected amount per meter traveled. If the distance between the AV10101 (FIG. 5N) and the traversable SDSF line 11626 (FIG. 5N) is below a fourth preselected distance and if the heading error is greater than or equal to a third preselected amount relative to a line perpendicular to the SDSF line 10377, the SDSF approach 11131 may include transmitting an SDSF command 11144 to slow the AV10101 (FIG. 5N) by a ninth preselected amount.

[0215] 5K, if the heading error falls below a third preselected amount relative to a line perpendicular to the SDSF line 10377, the SDSF traversal 11133 may include disregarding the updated SDSF information and transmitting an SDSF command 11144 to run the AV10101 (FIG. 5N) at a preselected rate. If the AV orientation change 11142 indicates that the climb of the leading edge 11701 (FIG. 5N) of the AV10101 (FIG. 5N) relative to the trailing edge 11703 (FIG. 5N) of the AV10101 (FIG. 5N) is between a sixth preselected amount and a fifth preselected amount, the SDSF traversal 11133 may include transmitting an SDSF command 11144 to run the AV10101 (FIG. 5N) forward and transmitting an SDSF command 11144 to increase the speed of the AV10101 (FIG. 5N) to a preselected rate per degree of climb. If the AV orientation change 11142 indicates that the elevation of the leading edge 11701 (FIG. 5N) relative to the trailing edge 11703 (FIG. 5N) of the AV 10101 (FIG. 5N) falls below a sixth preselected amount, the SDSF traversal 11133 can include sending an SDSF command 11144 to drive the AV 10101 (FIG. 5N) forward at a seventh preselected speed. If the AV location 11141 indicates that the trailing edge 11703 (FIG. 5N) is above a fifth preselected distance from the SDSF line 10377, the SDSF traversal 11133 can include acknowledging that the AV 10101 (FIG. 5N) has completed traversing the SDSF 10377. If the AV location 11141 indicates that the trailing edge 11703 (FIG. 5N) is less than or equal to a fifth preselected distance from the SDSF line 10377, the SDSF traversal 11133 may include executing the loop again beginning with ignoring the updated SDSF information.

[0216] Some example ranges for the preselected values ​​described herein can include, but are not limited to, those outlined in Table II. [Table 2-1]

Table 2-2

[0217] 5O, to aid in real-time data collection, in some configurations, a system of the present teachings can generate locations in three-dimensional space of various surface types in response to receiving data, such as, for example, but not limited to, RGD-D camera image data. The system can rotate images 12155 and transform them from a camera coordinate system 12157 to a UTM coordinate system 12159. The system can generate polygon files from the transformed images, which can represent three-dimensional locations associated with the surface types 12161. A method 12150 for locating features 12151 from camera images 12155 received by an AV 10101 having a pose 12163 can include, but is not limited to, receiving, by the AV 10101, a camera image 12155. Each camera image 12155 can include an image timestamp 12171, and each image 12155 can include image color pixels 12167 and image depth pixels 12169. The method 12150 can include receiving a pose 12163 of the AV 10101, the pose 12163 having a pose timestamp 12171, and determining a selected image 12173 by identifying an image from the camera images 12155 having an image timestamp 12165 that is closest to the pose timestamp 12171. The method 12150 may include isolating image color pixels 12167 from image depth pixels 12169 in a selected image 12173, and determining an image surface classification 12161 for the selected image 12173 by providing the image color pixels 12167 to a first machine learning model 12177 and providing the image depth pixels 12169 to a second machine learning model 12179. The method 12150 may include determining perimeter points 12181 of a feature in the camera image 12173, the feature including feature pixels 12151 within the perimeter, each of the feature pixels 12151 having the same surface classification 12161, and each of the perimeter points 12181 may have a set of coordinates 12157.The method 12150 may include converting each of the set of coordinates 12157 to a UTM coordinate 12159.

[0218] Configurations of the present teachings are directed to computer systems for performing the methods discussed in the description herein, and computer-readable media containing programs for performing these methods. Raw data and results can be stored for future retrieval and processing, printed, displayed, transferred to another computer, and / or transferred to another location. Communications links can be wired or wireless, using, for example, cellular, military, and satellite communication systems. Portions of the system can run on computers with varying numbers of CPUs. Other alternative computer platforms can also be used.

[0219] The present configuration is also directed to software for performing the methods discussed herein, and computer-readable media storing the software for performing these methods. The various modules described herein can be performed on the same CPU or on different computers. In accordance with the statute, the present configuration has been described in more or less specific language with respect to structural and method features. However, it should be understood that the present configuration is not limited to the specific features shown and described, since the means disclosed herein comprise preferred forms of embodying the present configuration.

[0220] The method can be implemented, in whole or in part, electronically. Signals representing actions taken by elements of the system and other disclosed configurations can travel via at least one live communication network. Control and data information can be executed and stored electronically on at least one computer readable medium. The system can be implemented to execute on at least one computer node in at least one live communication network. General forms of the at least one computer readable medium can include, for example, but are not limited to, a floppy disk, a flexible disk, a hard disk, a magnetic tape, or any other magnetic medium, a compact disk read-only memory or any other optical medium, a punch card, a paper tape, or any other physical medium with a pattern of holes, a random access memory, a programmable read-only memory, and an erasable programmable read-only memory (EPROM), a flash EPROM, or any other memory chip or cartridge, or any other medium from which a computer can read. Additionally, at least one computer-readable medium may contain graphs in any format, including but not limited to Graphics Interchange Format (GIF), Joint Photographic Experts Group (JPEG), Portable Network Graphics (PNG), Scalable Vector Graphics (SVG), and Tagged Image File Format (TIFF), as appropriate, subject to appropriate licensing.

[0221] Although the present teachings have been described above in terms of specific configurations, it should be understood that they are not limited to these disclosed configurations. Numerous modifications and other configurations will occur to those skilled in the art to which this pertains, and are intended and will be covered by both this disclosure and the appended claims. It is intended that the scope of the present teachings should be determined by proper interpretation and construction of the appended claims and their legal equivalents, as understood by those skilled in the art relying on the disclosure in this specification and the accompanying drawings.

Claims

1. 1. A method for real-time control of a configuration of a device, the device including a chassis, at least four wheels, a first side of the chassis operatively coupled to at least one of the at least four wheels, and an opposing second side of the chassis operatively coupled to at least one of the at least four wheels, the method comprising: receiving environmental data; determining a surface type based at least on the environmental data; and determining a mode based on at least the surface type and a first configuration; determining a second configuration based on at least the mode and the surface type; determining a movement command based on at least the second configuration; controlling a configuration of the device by using the move command to change the device from the first configuration to the second configuration; A method comprising:

2. populating an occupancy grid based on at least the surface type and the mode; determining the movement command based at least on the occupancy grid; The method of claim 1 further comprising:

3. 2. The method of claim 1, wherein controlling the configuration includes coordinated powering of the first pair of clusters of at least four wheels and the second pair of clusters of at least four wheels based on the environmental data.

4. 2. The method of claim 1, wherein controlling the configuration includes transitioning from driving the at least four wheels and a pair of casters retracted, the pair of casters operably coupled to the chassis, to driving two wheels with a clustered first pair and a clustered second pair rotated to lift a first front wheel and a second front wheel, the device resting on a first rear wheel, a second rear wheel, and the pair of casters.

5. 4. The method of claim 3, wherein controlling the configuration includes rotating a pair of clusters operatively coupled with a first two powered wheels on the first side and a second two powered wheels on the second side based on at least the environmental data.

6. The method of claim 1 , wherein the device further comprises a cargo container, the cargo container mounted on the chassis, and the chassis controls the height of the cargo container.

7. The method of claim 6 , wherein the height of the cargo container is based on at least the environmental data.

8. 1. A system for real-time control of a configuration of a device, the device including a chassis, at least four wheels, a first side of the chassis, and an opposing second side of the chassis, the system comprising: a device processor configured to receive real-time environmental data surrounding the device, determine a surface type based on the real-time environmental data, determine a mode based on at least the surface type and a first configuration, and determine a second configuration based on the mode and the surface type; a power base processor that determines a movement command based on at least the second configuration, the power base processor using the movement command to control a configuration of the device to change the device from the first configuration to the second configuration; A system comprising:

9. The system of claim 8 , wherein the power base processor determines the movement commands based at least on an occupancy grid.

10. The system of claim 9 , wherein the occupancy grid comprises information about surface type and / or mode.

11. 9. The system of claim 8, wherein controlling the configuration includes coordinated powering of the first pair of clusters of at least four wheels and the second pair of clusters of at least four wheels based on the real-time environmental data.

12. 10. The system of claim 8, wherein controlling the configuration includes transitioning from driving the at least four wheels and pairs of casters retracted, the pairs of casters operably coupled to the chassis, to driving two wheels with a clustered first pair and a clustered second pair rotated to lift a first front wheel and a second front wheel, the device resting on a first rear wheel, a second rear wheel, and the pairs of casters.

Images