System and method for real-time control of autonomous devices

Abstract

To provide a system and a method for real time control of an autonomous device.SOLUTION: An autonomous vehicle has sensors advantageously varied in capabilities, advantageously positioned, and advantageously impervious to environmental conditions. A system is executed on the autonomous vehicle and can receive a map including, for example, substantially discontinuous surface features along with data from the sensors, create an occupancy grid based upon the map and the data, and change the configuration of the autonomous vehicle based upon the type of surface on which the autonomous vehicle navigates. The device can safely navigate surfaces and surface features, including traversing discontinuous surfaces and other obstacles.SELECTED DRAWING: None

Description

Technical Field

[0001] (Cross - reference to Related Applications) This utility patent application is a partial continuation application of U.S. Patent Application No. 16 / 800,497 (Attorney Docket No. AA164), filed on February 25, 2020, entitled "System and Method for Surface Feature Detection and Traversal", which is hereby incorporated by reference in its entirety. This patent application claims the benefit of U.S. Provisional Patent Application No. 62 / 872,396 (Attorney Docket No. AA028), filed on July 10, 2019, entitled "Apparatus for Long and Short Range Sensors on an Autonomous Delivery Device", U.S. Provisional Patent Application No. 62 / 990,485 (Attorney Docket No. AA037), filed on March 17, 2020, entitled "System and Method for Managing an Occupancy Grid", and U.S. Provisional Patent Application No. 62 / 872,320 (Attorney Docket No. Z96), filed on July 10, 2019, entitled "System and Method for Real - Time Control of the Configuration of an Autonomous Device".

[0002] This application is incorporated herein by reference in its entirety by the following: U.S. Patent Application No. 16 / 035,205 (Patent Attorney No. X80), filed on 13 July 2018 and titled “MOBILITY DEVICE”; U.S. Patent Application No. 15 / 787,613 (Patent Attorney No. W10), filed on 18 October 2017 and titled “MOBILITY DEVICE”; U.S. Patent Application No. 15 / 600,703 (Patent Attorney No. U22), filed on 20 May 2017 and titled “MOBILITY DEVICE”; and "SYSTEM AND METHOD FOR SECURE REMOTE CONTROL OF A MEDICAL This relates to U.S. Patent Application No. 15 / 982,737 (Patent Attorney No. X55) titled "DEVICE", U.S. Provisional Application No. 62 / 532,993 (Patent Attorney No. U30) filed on July 15, 2017, titled "MOBILITY DEVICE IMPROVEMENTS", U.S. Provisional Application No. 62 / 559,263 (Patent Attorney No. V85) filed on September 15, 2017, titled "MOBILITY DEVICE SEAT", and U.S. Provisional Application No. 62 / 581,670 (Patent Attorney No. W07) filed on November 4, 2017, also titled "MOBILITY DEVICE SEAT".

[0003] This instruction generally relates to AV (Aviation Vehicle), and more specifically to autonomous route planning, global occupancy grid management, on-vehicle sensors, surface feature detection and cross-sectioning, and real-time vehicle configuration changes. [Background technology]

[0004] (background) Navigation for AVs and semi-autonomous vehicles (AVs) typically relies on long-range sensors, including, but not limited to, LiDAR, cameras, stereo cameras, and radar. Long-range sensors can sense objects from 4 to 100 meters away from the AV. In contrast, object avoidance and / or surface detection typically relies on short-range sensors, including, but not limited to, stereo cameras, short-range radar, and ultrasonic sensors. These short-range sensors typically observe an area or volume within approximately 5 meters of the AV. Sensors can, for example, enable the AV to orient itself within its environment and navigate roads, sidewalks, obstacles, and open spaces to reach a desired destination. Sensors can also enable visualization of people, signs, traffic signals, obstacles, and surface features.

[0005] Surface feature cross-sections can be challenging because surface features, such as substantial discontinuous surface features (SDSFs), may be found in heterogeneous morphologies, and these morphologies may be unique to specific geographical locations. However, SDSFs, such as slopes, edges, curbs, steps, and curb-like geometric shapes (referred to herein, in an unspecified manner, as SDSFs or simply surface features), may include several typical characteristics that can aid in their identification. Surface / road conditions and surface types can be recognized and classified, for example, by fusing multisensory data, which can be complex and costly. Surface features and conditions can be used to control the physical reconstruction of AVs in real time.

[0006] Sensors can be used to enable the creation of an occupied grid that may represent the world for the purpose of path planning related to AV. Path planning requires a grid that identifies space as free, occupied, or unknown. However, the probability that space is occupied can improve decision-making regarding space. A log-odds representation of probabilities can be used to increase accuracy at the numerical boundaries of probabilities 0 and 1. The probability that a cell is occupied may depend at least on new sensor information, previous sensor information, and prior occupied information.

[0007] What is required is a system that combines accumulated sensor data and real-time sensor data with changes in the vehicle's physical configuration in order to perform variable terrain crossing. What is required is sensor placement that is advantageous for achieving physical configuration changes, variable terrain crossing, and object avoidance. What is required is the ability to locate SDSFs based on a multipart model associated with several criteria for SDSF identification. What is required is the determination of candidate surface feature crossings based on criteria such as candidate crossing approach angles, candidate crossing travel surfaces on both sides of candidate surface features, and real-time determination of obstacles along candidate crossing paths. What is required is a system and method for incorporating drivable surface and device mode information into occupied grid determination. [Overview of the Initiative] [Means for solving the problem]

[0008] (summary) The AV in this instruction can autonomously navigate to a desired location. In some configurations, the AV may include a device controller, a power base, four powered wheels, two caster wheels, and a cargo container, which may include sensors, a perception subsystem, an autonomy subsystem, and a driver subsystem. In some configurations, the perception and autonomy subsystems may receive and process sensor information (perception) and map information (perception and autonomy) and provide instructions to the driver subsystem. The map information may include surface classification and associated device modes. The movement of the AV, controlled by the driver subsystem and enabled by the power base, may be sensed by the sensor subsystem and provide a feedback loop. In some configurations, the SDSF may be precisely identified from point cloud data and stored in the map, for example, according to a process described herein. The portion of the map associated with the AV's location may be provided to the AV during navigation. The perception subsystem may maintain an occupancy grid that can inform the AV of the probability that the path to be traversed is currently occupied. In some configurations, the AV may operate in several distinctly different modes. Among other benefits, the mode can enable traversing complex terrain. A combination of maps (e.g., surface classification), sensor data (sensing features surrounding the AV), an occupancy grid (the probability that an upcoming path point is occupied), and the mode (whether it is in a state where it can traverse difficult terrain) can be used to identify the direction, configuration, and speed of the AV.

[0009] Regarding the preparation of the map, in some configurations, the method of this teaching for creating a map for navigating at least one SDSF encountered by an AV, wherein the AV travels a path across a surface, the surface includes at least one SDSF, and the path includes a start point and an end point. The method may include, but is not limited to, accessing point cloud data representing the surface, filtering the point cloud data, forming the filtered point cloud data into a processable portion, and merging the processable portion into at least one concave polygon. The method may also include locating and labeling at least one SDSF within at least one concave polygon. Locating and labeling can form labeled point cloud data. The method may also include, at least, creating a graphing polygon based on at least one concave polygon, and at least selecting a path from a start point to an end point based on the graphing polygon. When navigating, the AV can traverse at least one SDSF along the path.

[0010] Filtering point cloud data may optionally include conditionally removing points representing transient objects and outliers from the point cloud data, and replacing removed points with pre-selected heights. Forming processing portions may optionally include segmenting the point cloud data into processing portions and removing points with pre-selected heights from the processing portions. Merging processing portions may optionally include reducing the size of the processing portions by analyzing outliers, voxels, and normals, expanding the area from the reduced-size processing portions, determining the initial drivable surface from the expanded area, segmenting and meshing the initial drivable surface, locating polygons within the segmented and meshed initial drivable surface, and defining the drivable surface based on at least the polygons. Localizing and labeling at least one SDSF feature optionally includes sorting point cloud data of drivable surfaces according to an SDSF filter, wherein the SDSF filter includes points of at least three categories, and localizing at least one SDSF point based on whether the points of the categories, in combination, satisfy at least one first pre-selected criterion. The method optionally includes creating at least one SDSF trajectory based on whether a plurality of at least one SDSF points, in combination, satisfy at least one second pre-selected criterion. Creating a graphed polygon further optionally includes creating at least one polygon from at least one drivable surface. The at least one polygon may include edges. Creating a graphed polygon may include smoothing the edges, forming a drivable margin based on the smoothed edges, adding at least one SDSF trajectory to at least one drivable surface, and removing edges from at least one drivable surface according to at least one third pre-selected criterion. Edge smoothing may optionally include trimming the edges outwards.Forming the running margin of the smoothed edge may optionally include trimming the outward-facing edge inward.

[0011] In some configurations, the system of this teaching is for creating a map for navigating at least one SDSF encountered by an AV, wherein the AV travels along a path across a surface, the surface includes at least one SDSF, and the path includes a start point and an end point; the system may include, but is not limited to, a first processor for accessing point cloud data representing the surface; a first filter for filtering the point cloud data; a second processor for forming a processable portion from the filtered point cloud data; a third processor for merging the processable portion into at least one concave polygon; a fourth processor for locating and labeling at least one SDSF within at least one concave polygon, wherein locating and labeling forms labeled point cloud data; a fifth processor for creating a graphing polygon; and a path selector for selecting a path from the start point to the end point based on at least the graphing polygon. The AV can traverse at least one SDSF along the path.

[0012] The first filter may optionally include executable code that conditionally removes points representing transient objects and outliers from point cloud data and replaces the removed points with pre-selected heights. The segmenter may optionally include executable code that segmentes the point cloud data into processable portions and removes points with pre-selected heights from the processable portions. The third processor may optionally include executable code that reduces the size of the processable portions by analyzing outliers, voxels, and normals, expands the area from the reduced-size processable portions, determines the initial drivable surface from the expanded area, segments and meshes the initial drivable surface, locates polygons within the segmented and meshed initial drivable surface, and defines the drivable surface based on at least the polygons. The fourth processor may optionally include executable code that sorts point cloud data of drivable surfaces according to an SDSF filter, wherein the SDSF filter includes points of at least three categories, and that locates at least one SDSF point based on whether the points of the categories, in combination, satisfy at least one first pre-selected criterion. The system may optionally include executable code that creates at least one SDSF trajectory based on whether a plurality of at least one SDSF points, in combination, satisfy at least one second pre-selected criterion.

[0013] Creating a graphed polygon may optionally include, but is not limited to, creating at least one polygon from at least one drivable surface, wherein at least one polygon includes an edge, smoothing the edge, forming a traverse margin based on the smoothed edge, adding at least one SDSF traverse to at least one drivable surface, and removing the edge from at least one drivable surface according to at least one third pre-selected criterion. Smoothing the edge may optionally include, but is not limited to, trimming the edge outwards, and may include executable code. Forming a traverse margin of the smoothed edge may optionally include, but is not limited to, trimming the outward-facing edge inwards, and may include executable code.

[0014] In some configurations, the Method of this Teaching is for creating a map for navigating at least one SDSF encountered by an AV, wherein the AV travels a path across a surface, the surface includes at least one SDSF, the path includes a start point and an end point, and the Method may include, but is not limited to, accessing a route morphology. The route morphology may include at least one graphed polygon which may include filtered point cloud data. The point cloud data may include labeled features and drivable margins. The Method may include transforming the point cloud data into a global coordinate system, determining the boundary of at least one SDSF, creating an SDSF buffer of a pre-selected size around the boundary, determining at least one SDSF that can be traversed based on at least one SDSF crossing criterion, creating an edge / weighted graph based on at least one SDSF crossing criterion, the transformed point cloud data, and the route morphology, and selecting a path from the start point to the destination point based on the edge / weighted graph.

[0015] At least one SDSF crossing criterion may optionally include a pre-selected width of at least one SDSF and a pre-selected smoothness of at least one SDSF, a minimum entry distance and a minimum exit distance between at least one SDSF and AV, including a drivable surface, and a minimum entry distance between at least one SDSF and AV that allows for an approximately 90° approach to at least one SDSF by AV.

[0016] In some configurations, the system of this teaching is for creating a map for navigating at least one SDSF encountered by an AV, wherein the AV travels a path across a surface, the surface contains at least one SDSF, the path contains a start point and an end point, and the system may include a sixth processor that accesses a route morphology, which may include at least one graphed polygon containing filtered point cloud data, the point cloud data containing labeled features and drivable margins. The system may include a seventh processor that transforms the point cloud data into a global coordinate system, and an eighth processor that determines the boundary of at least one SDSF. The eighth processor may create an SDSF buffer of a pre-selected size around the boundary. The system may include at least a ninth processor that determines at least one SDSF that can be traversed based on at least one SDSF traversal criterion, a tenth processor that creates an edge / weighted graph based on at least one SDSF traversal criterion, transformed point cloud data, and route morphology, and a base controller that selects a path from a starting point to a destination point based on at least the edge / weighted graph.

[0017] In some configurations, the Method of this Teaching is for creating a map for navigating at least one SDSF encountered by an AV, wherein the AV travels a path across a surface, the surface includes at least one SDSF, and the path includes a start point and an end point. The Method may include, but is not limited to, accessing point cloud data representing the surface. The Method may include filtering the point cloud data, forming the filtered point cloud data into a tadible portion, and merging the tadible portion into at least one concave polygon. The Method may include locating and labeling at least one SDSF within at least one concave polygon. Locating and labeling can form labeled point cloud data. The Method may include creating a graphing polygon based on at least one concave polygon. The graphing polygon may form a route shape, and the point cloud data may include labeled features and traversable margins. This method may include transforming point cloud data into a global coordinate system, determining the boundary of at least one SDSF, creating an SDSF buffer of a pre-selected size around the boundary, determining at least one SDSF that can be traversed based on at least one SDSF crossing criterion, creating an edge / weighted graph based on at least one SDSF crossing criterion, the transformed point cloud data, and the route morphology, and selecting a path from a starting point to a destination point based on the edge / weighted graph.

[0018] Filtering point cloud data may optionally include conditionally removing points representing transient objects and outliers from the point cloud data, and replacing removed points with pre-selected heights. Forming processing portions may optionally include segmenting the point cloud data into processing portions and removing points with pre-selected heights from the processing portions. Merging processing portions may optionally include reducing the size of the processing portions by analyzing outliers, voxels, and normals, expanding the area from the reduced-size processing portions, determining the initial drivable surface from the expanded area, segmenting and meshing the initial drivable surface, locating polygons within the segmented and meshed initial drivable surface, and defining the drivable surface based on at least the polygons. Localizing and labeling at least one SDSF feature optionally includes sorting point cloud data of drivable surfaces according to an SDSF filter, wherein the SDSF filter includes points of at least three categories, and localizing at least one SDSF point based on whether the points of the categories, in combination, satisfy at least one first pre-selected criterion. The method optionally includes creating at least one SDSF trajectory based on whether a plurality of at least one SDSF points, in combination, satisfy at least one second pre-selected criterion. Creating a graphed polygon further optionally includes creating at least one polygon from at least one drivable surface. The at least one polygon may include edges. Creating a graphed polygon may include smoothing the edges, forming a drivable margin based on the smoothed edges, adding at least one SDSF trajectory to at least one drivable surface, and removing edges from at least one drivable surface according to at least one third pre-selected criterion. Edge smoothing may optionally include trimming the edges outwards.Forming a smoothed edge travel margin may optionally include trimming outward edges inward. At least one SDSF crossing criterion may optionally include a pre-selected width of at least one SDSF and a pre-selected smoothness of at least one SDSF, a minimum entry distance and a minimum exit distance between at least one SDSF and AV including a drivable surface, and a minimum entry distance between at least one SDSF and AV that allows for an approximately 90° approach to at least one SDSF by AV.

[0019] In some configurations, the system of this teaching is for creating a map for navigating at least one SDSF encountered by an AV, wherein the AV travels a path across a surface, the surface includes at least one SDSF, and the path includes a start point and an end point; the system includes, but is not limited to, a point cloud accessor for accessing point cloud data representing the surface; a first filter for filtering the point cloud data; a segmenter for forming a processable portion from the filtered point cloud data; a third processor for merging the processable portion into at least one concave polygon; a fourth processor for locating and labeling at least one SDSF within at least one concave polygon, wherein locating and labeling is formed by a fourth processor for forming labeled point cloud data; and a fifth processor for creating a graphing polygon. The route configuration may include at least one graphing polygon which may include the filtered point cloud data. The point cloud data may include labeled features and drivable margins. The system may include a seventh processor that converts point cloud data to a global coordinate system, and an eighth processor that determines the boundary of at least one SDSF. The eighth processor can create an SDSF buffer of a pre-selected size around the boundary. The system may also include a ninth processor that determines at least one SDSF that can be traversed based on at least one SDSF crossing criterion, a tenth processor that creates an edge / weighted graph based on at least one SDSF crossing criterion, the converted point cloud data, and the route morphology, and a base controller that selects a path from a starting point to a destination point based on at least the edge / weighted graph.

[0020] The first filter may optionally include executable code that conditionally removes points representing transient objects and outliers from point cloud data and replaces the removed points with pre-selected heights. The segmenter may optionally include executable code that segmentes the point cloud data into processable portions and removes points with pre-selected heights from the processable portions. The third processor may optionally include executable code that reduces the size of the processable portions by analyzing outliers, voxels, and normals, expands the area from the reduced-size processable portions, determines the initial drivable surface from the expanded area, segments and meshes the initial drivable surface, locates polygons within the segmented and meshed initial drivable surface, and defines the drivable surface based on at least the polygons. The fourth processor may optionally include executable code that sorts point cloud data of drivable surfaces according to an SDSF filter, wherein the SDSF filter includes points of at least three categories, and that locates at least one SDSF point based on whether the points of the categories, in combination, satisfy at least one first pre-selected criterion. The system may optionally include executable code that creates at least one SDSF trajectory based on whether a plurality of at least one SDSF points, in combination, satisfy at least one second pre-selected criterion.

[0021] Creating a graphed polygon may optionally include, but is not limited to, creating at least one polygon from at least one drivable surface, wherein at least one polygon includes an edge, smoothing the edge, forming a traverse margin based on the smoothed edge, adding at least one SDSF traverse to at least one drivable surface, and removing the edge from at least one drivable surface according to at least one third pre-selected criterion. Smoothing the edge may optionally include, but is not limited to, trimming the edge outwards, and may include executable code. Forming a traverse margin of the smoothed edge may optionally include, but is not limited to, trimming the outward-facing edge inwards, and may include executable code.

[0022] In some configurations, SDSFs can be identified by their dimensions. For example, a curb can have a width of approximately 0.6–0.7 m, though not limited to that. In some configurations, point cloud data can be processed to locate SDSFs, and this data can be used to prepare a route for the AV from the starting point to the destination. In some configurations, the route can be contained in a map and provided to a perception subsystem. As the AV travels along the route, in some configurations, SDSF traverses can be adapted through sensor-based positioning of the AV, which is partially enabled by the perception subsystem. The perception subsystem can run on at least one processor within the AV.

[0023] The AV may include, but is not limited to, a power base including two powered front wheels, two powered rear wheels, and an energy storage device, and at least one processor. The power base may be configured to move at a commanded speed. The AV may include a cargo platform mechanically mounted to the power base and including a number of short-range sensors. In some configurations, the AV may include a cargo container mounted on the cargo platform and having volume for receiving one or more objects to be delivered. In some configurations, the AV may include a long-range sensor suite mounted on the cargo container and may include, but is not limited to, a LiDAR and one or more cameras. The AV may include a controller capable of receiving data from the long-range sensor suite and the short-range sensor suite.

[0024] The short-range sensor suite can optionally detect at least one characteristic of the drivable surface and may optionally include a stereo camera, an IR projector, two image sensors, an RGB sensor, and a radar sensor. The short-range sensor suite can optionally supply RGB-D data to a controller. The controller can optionally determine the geometric shape of the road surface based on the RGB-D data received from the short-range sensor suite. The short-range sensor suite can optionally detect objects within 4 meters of the AV, and the long-range sensor suite can optionally detect objects beyond 4 meters of the AV.

[0025] The perception subsystem can use the data collected by sensors to capture an occupancy grid. The occupancy grid of the present disclosure is configured as a 3D grid of points surrounding the AV, and the AV can occupy the central point. In some configurations, the occupancy grid can extend 10 m to the left, right, rear, and front of the AV. The grid can include approximately the height of the AV, which virtually progresses with the AV as it moves and can represent obstacles surrounding the AV. The grid can be transformed into two dimensions by shrinking its vertical axis, for example, but not limited to, being divided into polygons of approximately 5 cm × 5 cm in size. Obstacles that appear within the 3D space around the AV can be shrunk into 2D shapes. If the 2D shape overlaps any segment of one of the polygons, the polygon is given a value of 100, indicating that the space is occupied. Any polygon that remains unfilled can be given a value of 0 and can be referred to as free space where the AV can move.

[0026] When the AV is navigating, it may encounter situations where a change in the configuration of the AV is required. A method of the present disclosure for real-time control of the configuration of a device, in some configurations, the device includes a chassis, at least four wheels, a first side of the chassis operably coupled to at least one of the at least four wheels, and a second, opposing side of the chassis operably coupled to at least one of the at least four wheels, and the method can include, but is not limited to, receiving environmental data, determining a surface type based at least on the environmental data, determining a mode based at least on the surface type and a first configuration, determining a second configuration based at least on the mode and the surface type, determining a movement command based at least on the second configuration, and controlling the configuration of the device by using the movement command to change the device from the first configuration to the second configuration.

[0027] The method optionally includes incorporating an occupied grid based on at least the surface type and mode. Environmental data optionally includes RGB-D image data and road surface morphology. The configuration optionally includes two clustered pairs of at least four wheels. A first pair of the two pairs can be positioned on the first side, and a second pair of the two pairs can be positioned on the second side. The first pair may include a first front wheel and a first rear wheel, and the second pair may include a second front wheel and a second rear wheel. Control of the configuration optionally includes coordinated power supply of the first and second pairs based on at least the environmental data. Control of the configuration optionally includes transitioning from driving at least four wheels and a pair of reversing casters to driving two wheels with a clustered first pair and a clustered second pair that are rotated to lift the first and second front wheels. A pair of casters can be operably coupled to the chassis. The device can be stationary on a first rear wheel, a second rear wheel, and a pair of casters. Control of the configuration may optionally include rotating a pair of clusters operably coupled to two powered wheels on the first side and two powered wheels on the second side, based on at least environmental data.

[0028] The system of the present teachings for real-time control of the configuration of an AV can include, but is not limited to, a device processor and a power base processor. The AV can include a chassis, at least four wheels, a first side of the chassis, and a second, opposing side of the chassis. The device processor can receive real-time environmental data surrounding the AV, determine a surface type based at least on the environmental data, determine a mode based at least on the surface type and a first configuration, and determine a second configuration based at least on the mode and the surface type. The power base processor can enable the AV to move based at least on the second configuration and can enable the AV to change from the first configuration to the second configuration. The device processor can optionally include capturing an occupancy grid based at least on the surface type and the mode.

[0029] During navigation, the AV may encounter a SDSF that may require the AV to be steered for a normal crossing. In some configurations, the Method of this Teaching is for navigating the AV along a path line within a travel area toward a target point that crosses at least one SDSF, wherein the AV includes a leading edge and a trailing edge, and the Method may include, but is not limited to, receiving SDSF information and obstacle information relating to the travel area, detecting at least one candidate SDSF from the SDSF information, and selecting an SDSF line from at least one candidate SDSF line based on at least one selection criterion. The Method may also include determining at least one traversable portion of the selected SDSF line based on at least one location of at least one obstacle found in the vicinity obstacle information of the selected SDSF line, azimuthing the AV toward the at least one traversable portion by turning the AV to travel along a line perpendicular to the traversable portion and operating it at a first speed, and always correcting the orientation of the AV based on the relationship between the orientation and the perpendicular line. The method may include driving the AV at a second speed by adjusting the first speed of the AV based on at least the orientation and the distance between the AV and the traversable portion. If the SDSF associated with at least one traversable portion rises relative to the surface of the route, the method may include driving the AV across the SDSF by raising the leading edge relative to the trailing edge and driving the AV at a third increased speed per degree of rise, and driving the AV at a fourth speed until the AV has passed the SDSF.

[0030] Detecting at least one candidate SDSF from SDSF information may optionally include (a) drawing a closed polygon encompassing the location of the AV and the location of the target point; (b) drawing a path line between the target point and the location of the AV; (c) selecting two SDSF points from the SDSF information such that the SDSF points are located within the polygon; and (d) drawing an SDSF line between the two points. Detecting at least one candidate SDSF may also include repeating steps (c)-(e) if (e) there are fewer than a first pre-selected number of points within a first pre-selected distance of the SDSF line, and there are fewer than a second pre-selected number of attempts in selecting the SDSF points, drawing a line between them, and having fewer than a first pre-selected number of points around the SDSF line. Detecting at least one candidate SDSF may include (f) fitting the curve to the SDSF points that are within a first pre-selected distance of the SDSF line if there are a first pre-selected number of points or more, and (g) identifying the curve as an SDSF line if the first number of SDSF points within the first pre-selected distance of the curve exceeds the second number of SDSF points within the first pre-selected distance of the SDSF line, and the curve intersects the path line, and there are no gaps between SDSF points on the curve beyond the second pre-selected distance. Detecting at least one candidate SDSF may include repeating steps (f)-(h) if (h) the number of points within a first pre-selected distance of the curve does not exceed the number of points within a first pre-selected distance of the SDSF line, or if the curve does not intersect the path line, or if there are gaps between SDSF points on the curve that exceed a second pre-selected distance, and if the SDSF line is not stable, and if steps (f)-(h) have not been attempted more than a second pre-selected number of times.

[0031] A closed polygon may optionally include a pre-selected width, which may optionally include the width dimension of AV. The selection of SDSF points may optionally include random selection. At least one selection criterion may optionally include that a first number of SDSF points within a first pre-selected distance of the curve exceeds a second number of SDSF points within a first pre-selected distance of the SDSF line, that the curve intersects the path line, and that there are no gaps between SDSF points on the curve beyond a second pre-selected distance.

[0032] Determining at least one traversable portion of a selected SDSF optionally includes selecting multiple obstacle points from obstacle information. Each of the multiple obstacle points may include the probability that the obstacle point is associated with at least one obstacle. Determining at least one traversable portion may include projecting the multiple obstacle points onto the SDSF line to form at least one projection if the probability is higher than a pre-selected percentage and any of the multiple obstacle points are located between the SDSF line and the target point, and any of the multiple obstacle points are less than a third pre-selected distance from the SDSF line. Determining at least one traversable portion may optionally include connecting at least two of the at least one projection, locating the endpoints of the at least two connected projections along the SDSF line, marking the at least two connected projections as a non-traversable SDSF section, and marking the SDSF line outside the non-traversable section as at least one traversable section.

[0033] Crossing at least one traversable portion of the SDSF may optionally include turning the AV to proceed along a line perpendicular to the traversable portion, oriented the AV toward the traversable portion, and operating it at a first speed; constantly correcting the orientation of the AV based on the relationship between the orientation and the perpendicular line; and adjusting the first speed of the AV based on at least the orientation and the distance between the AV and the traversable portion to drive the AV at a second speed. Crossing at least one traversable portion of the SDSF may optionally include crossing the SDSF by raising the leading edge relative to the trailing edge if the SDSF is rising relative to the surface of the route, and driving the AV at a third increased speed per degree of rise; and driving the AV at a fourth speed until the AV has passed the SDSF.

[0034] Alternatively, crossing at least one traversable portion of the SDSF may optionally include: (a) if the azimuth error is less than a third pre-selected amount relative to a line perpendicular to the SDSF line, ignoring the update of the SDSF information and driving the AV at a pre-selected speed; (b) if the rise of the front portion of the AV relative to the rear portion is between a sixth pre-selected amount and a fifth pre-selected amount, driving the AV forward and increasing the speed of the AV to an eighth pre-selected speed per degree of rise; (c) if the rise of the front portion is less than a sixth pre-selected amount relative to the rear portion, driving the AV forward at a seventh pre-selected speed; and (d) if the rear portion is less than or equal to a fifth pre-selected distance from the SDSF line, repeating steps (a)-(d).

[0035] In some configurations, the wheels of the SDSF and AV can be automatically aligned to avoid system instability. Automatic alignment can be implemented, for example, by continuously testing and correcting the orientation of the AV as it approaches the SDSF. Another aspect of the SDSF crossing feature in this teaching is that it automatically verifies that sufficient free space exists around the SDSF before attempting to cross it. Yet another aspect of the SDSF crossing feature in this teaching is that it is possible to cross SDSFs of various geometric shapes. Geometric shapes can include, for example, squares and contoured SDSFs. The orientation of the AV relative to the SDSF can determine the speed and direction of the AV as it travels. The SDSF crossing feature can adjust the speed of the AV in the vicinity of the SDSF. When the AV approaches the SDSF, the speed can be increased to assist the AV in crossing the SDSF.

[0036] 1. An autonomous delivery vehicle comprising: a power base including two powered front wheels, two powered rear wheels, and an energy storage device, the power base being configured to move at a commanded speed and in a commanded direction and to carry out the transport of at least one object; and a cargo platform including a plurality of short-range sensors, the cargo platform being mechanically attached to the power base, and a cargo container having a volume for receiving at least one object, the cargo container being mounted on the cargo platform, and a long-range sensor suite comprising a cargo container, a LIDAR, and one or more cameras, the long-range sensor suite being mounted on the cargo container, and a controller for receiving data from the long-range sensor suite and the plurality of short-range sensors, the controller determining a commanded speed and a commanded direction based on at least the data, and the controller providing the commanded speed and a commanded direction to the power base to complete the transport. 2. The autonomous delivery vehicle according to item 1, wherein data from multiple short-range sensors comprises at least one characteristic of the surface over which a power base travels. 3. The autonomous delivery vehicle according to item 1, wherein the multiple short-range sensors comprise at least one stereo camera. 4. The autonomous delivery vehicle according to item 1, wherein the multiple short-range sensors comprise at least one IR projector, at least one image sensor, and at least one RGB sensor. 5. The autonomous delivery vehicle according to item 1, wherein the multiple short-range sensors comprise at least one radar sensor. 6. The autonomous delivery vehicle according to item 1, wherein data from multiple short-range sensors comprises RGB-D data. 7. The autonomous delivery vehicle according to item 1, wherein the controller determines the geometric shape of the road surface based on the RGB-D data received from the multiple short-range sensors. 8. The autonomous delivery vehicle according to item 1, wherein the multiple short-range sensors detect objects within 4 meters of the AV, and the long-range sensor suite detects objects beyond 4 meters from the autonomous delivery vehicle. 9. Multiple short-range sensors are provided in the autonomous delivery vehicle described in item 1, which also includes a cooling circuit.10. Multiple short-range sensors, including ultrasonic sensors, are provided in the autonomous delivery vehicle as described in item 1. 11. The autonomous delivery vehicle as described in item 2, comprising executable code for the controller, which includes accessing a map, the map being formed by a map processor, the map processor being a first processor that accesses point cloud data from a long-range sensor suite, the point cloud data representing a surface, a filter that filters the point cloud data, a second processor that forms a processable portion from the filtered point cloud data, a third processor that merges the processable portion into at least one polygon, and a fourth processor that, if present, locations and labels at least one substantial discontinuous surface feature (SDSF) within the at least one polygon, the location and labeling being the fourth processor that forms labeled point cloud data, a fifth processor that creates a graphed polygon from the labeled point cloud data, and a sixth processor that selects a path from a start point to an end point based on at least the graphed polygon, the AV traversing at least one SDSF along the path, the sixth processor. 12. The autonomous delivery vehicle according to item 11, comprising a seventh processor that executes code including a filter that conditionally removes points representing transient objects and points representing outliers from point cloud data and replaces the removed points having a pre-selected height. 13. The autonomous delivery vehicle according to item 11, comprising a second processor that includes executable code including segmenting the point cloud data into processable portions and removing points having a pre-selected height from the processable portions.14. The autonomous delivery vehicle according to item 11, comprising executable code, a third processor comprising: reducing the size of the processable portion by analyzing outliers, voxels, and normals; expanding the area from the reduced-size processable portion; determining an initial drivable surface from the expanded area; segmenting and meshing the initial drivable surface; locating polygons within the segmented and meshed initial drivable surface; and setting at least one drivable surface based on at least the polygons. 15. The autonomous delivery vehicle according to item 14, comprising executable code, a fourth processor comprising sorting point cloud data of the initial drivable surface according to an SDSF filter, wherein the SDSF filter includes points of at least three categories; and locating at least one SDSF point based on whether points of at least three categories, in combination, satisfy at least one first pre-selected criterion. 16. An autonomous delivery vehicle according to item 15, comprising executable code, the fourth processor comprising creating at least one SDSF trajectory based on whether a plurality of at least one SDSF points, in combination, satisfy at least one second pre-selected criterion. 17. An autonomous delivery vehicle according to item 14, comprising an eighth processor, comprising executable code, the graphing polygon being the creation of at least one polygon from at least one drivable surface, the at least one polygon including an outer edge, smoothing the outer edge, forming a travel margin based on the smoothed outer edge, adding at least one SDSF trajectory to the at least one drivable surface, and removing an inner edge from the at least one drivable surface according to at least one third pre-selected criterion. 18. An autonomous delivery vehicle according to item 17, comprising a ninth processor, comprising executable code, the smoothing outer edge being the trimming of the outer edge outward to form an outward-facing edge.19. An autonomous delivery vehicle as described in item 18, comprising a 10th processor, comprising executable code, which includes forming a smoothed outer edge travel margin, which includes trimming an outward edge inward. 20. A controller subsystem for navigating at least one substantial discontinuous surface feature (SDSF) encountered by an autonomous delivery vehicle (AV), wherein the AV travels a path across the surface, the surface comprises at least one SDSF, the path comprises a start point, an end point, and accesses a route morphology, comprising a first processor, which includes at least one graphed polygon, which comprises filtered point cloud data, the filtered point cloud data comprises labeled features, the point cloud data comprises a travel margin, and a second processor, which transforms the point cloud data into a global coordinate system, Autonomous delivery vehicle according to item 1, comprising a subsystem comprising: a third processor for determining the boundary of at least one SDSF, the third processor for creating an SDSF buffer of a pre-selected size around the boundary; a fourth processor for determining at least one SDSF that can be traversed based on at least one SDSF crossing criterion; a fifth processor for creating an edge / weighted graph based on at least one SDSF crossing criterion, transformed point cloud data, and route morphology; and a base controller for selecting a path from a start point to an end point based on at least the edge / weighted graph. 21. Autonomous delivery vehicle according to item 20, wherein the at least one SDSF crossing criterion comprises a pre-selected width of at least one SDSF and a pre-selected smoothness of at least one SDSF, a minimum entry distance and a minimum exit distance between at least one SDSF and AV including a drivable surface, and a minimum entry distance between at least one SDSF and AV that adapts an approximately 90° approach to at least one SDSF by AV.

[0037] 22. A method for managing a global occupancy grid for an autonomous device, wherein the global occupancy grid includes global occupancy grid cells, each global occupancy grid cell is associated with an occupancy probability, receives sensor data from a sensor associated with an autonomous device, and creates a local occupancy grid based at least on the sensor data, the local occupancy grid having local occupancy grid cells, and when an autonomous device moves from a first area to a second area, accesses historical data associated with a second area, creates a static grid based at least on the historical data, and moves the global occupancy grid, with the autonomous device at the center of the global occupancy grid. A method comprising: maintaining a static grid; updating a moved global occupancy grid based on a static grid; marking at least one of the global occupancy grid cells as unoccupied if at least one of the global occupancy grid cells coincides with the location of an autonomous device; for each local occupancy grid cell, calculating the location of the local occupancy grid cell on the global occupancy grid; accessing a first occupancy probability from the global occupancy grid cell at that location; accessing a second occupancy probability from the local occupancy grid cell at that location; and calculating a new occupancy probability at that location on the global occupancy grid based on at least the first and second occupancy probabilities. 23. The method according to item 22, further comprising range-checking the new occupancy probability. 24. The method according to item 23, wherein the range-checking comprises setting the new occupancy probability to 0 if the new occupancy probability is < 0, and setting the new occupancy probability to 1 if the new occupancy probability is > 1. 25. The method according to item 22, further comprising setting a global occupancy grid cell to the new occupancy probability. 26. The method of item 23, further comprising setting the global occupancy grid cell to a new occupancy probability that has been range-checked.

[0038] 27. A method for creating and managing occupied grids, comprising: converting sensor measurements into reference frames associated with a device by a local occupied grid creation node; creating a timestamped measurement occupied grid; exposing the timestamped measurement occupied grid as a local occupied grid; creating multiple local occupied grids; and creating a static occupied grid based on surface characteristics in a repository, wherein the surface characteristics are associated with the location of the device; moving a global occupied grid associated with the location of the device, and maintaining the device and local occupied grids that are approximately aligned with the global occupied grid; and static A method comprising: adding information from a local occupied grid to a global occupied grid; marking areas in the global occupied grid currently occupied by the device as unoccupied; determining the location of at least one cell in the global occupied grid for each at least one cell in each local occupied grid; accessing a first value at that location; determining a second value at that location based on the relationship between the first value and the cell value in at least one cell in the local occupied grid; comparing the second value to a pre-selected probability range; and setting up the global occupied grid with the new value if the probability value is within the pre-selected probability range. 28. The method of item 27, further comprising making the global occupied grid public. 29. The method of item 27, wherein surface properties comprise a surface type and surface discontinuity. 30. The method of item 27, wherein relationships comprise summing.31. A system for creating and managing occupancy grids, comprising a plurality of local grid creation nodes that create at least one local occupancy grid, the at least one local occupancy grid being associated with the location of a device, the at least one local occupancy grid containing at least one cell, and a global occupancy grid manager that accesses the at least one local occupancy grid, and which creates a static occupancy grid based on surface characteristics in a repository, the surface characteristics being associated with the location of the device, and moves the global occupancy grid associated with the location of the device, and the device and at least one row that are approximately aligned with the global occupancy grid. A system comprising a global occupancy grid manager that maintains local occupancy grids, adds information from static occupancy grids to at least one global occupancy grid, marks areas in the global occupancy grid currently occupied by the device as unoccupied, determines the location of at least one cell in the global occupancy grid for each at least one cell in each local occupancy grid, accesses a first value at that location, determines a second value at that location based on the relationship between the first value and the cell value in at least one cell in the local occupancy grid, compares the second value against a pre-selected probability range, and sets up a global occupancy grid with the new value if the probability value is within the pre-selected probability range.

[0039] 32. A method for updating a global occupancy grid, comprising: updating the global occupancy grid using information from a static grid associated with a new location when an autonomous device moves to a new location; analyzing a surface at the new location; updating the surface if the surface is drivable, updating the surface and updating the global occupancy grid using the updated surface; and updating the global occupancy grid using values ​​from a repository of static values, wherein the static values ​​are associated with the new location. 33. The method of item 32, wherein updating a surface includes accessing a local occupied grid associated with the new location, accessing a local occupied grid surface classification confidence value and a local occupied grid surface classification for each cell in the local occupied grid, adding the global surface classification confidence value in the global occupied grid to the local occupied grid surface classification confidence value if the local occupied grid surface classification is identical to the global surface classification in the global occupied grid within the cell to form a sum, and using the sum to update the global occupied grid in the cell, subtracting the local occupied grid surface classification confidence value from the global surface classification confidence value in the global occupied grid to form a difference, and using the difference to update the global occupied grid, and if the difference is less than zero, updating the global occupied grid using the local occupied grid surface classification.34. The method of item 32, wherein updating the global occupancy grid with values ​​from a repository of static values ​​includes, for each cell in the local occupancy grid, accessing a log odds value which is the local occupancy grid probability that the cell is an occupying value from the local occupancy grid; updating the log odds value in the global occupancy grid with the local occupancy grid log odds value in the cell; decreasing the log odds that the cell is occupied in the local occupancy grid if a pre-selected certainty that the cell is not occupied is met, and the autonomous device is proceeding within the lane barrier, and the local occupancy grid surface classification indicates a drivable surface; increasing the log odds in the local occupancy grid if the autonomous device expects to encounter a relatively uniform surface, and the local occupancy grid surface classification indicates a relatively non-uniform surface; and decreasing the log odds in the local occupancy grid if the autonomous device expects to encounter a relatively uniform surface, and the local occupancy grid surface classification indicates a relatively uniform surface.

[0040] 35. A method for real-time control of the configuration of a device, the device comprising a chassis, at least four wheels, a first side of the chassis operably coupled to at least one of the at least four wheels, and an opposing second side of the chassis operably coupled to at least one of the at least four wheels, and comprising creating a map based on at least prior surface features and an occupancy grid, the map being created non-real-time, the map comprising at least one location, the at least one location being associated with at least one surface feature, the at least one surface feature being associated with at least one surface classification and at least one mode, and as the device moves, determining the current surface features, updating the occupancy grid in real time using the current surface features, and determining from the occupancy grid and the map a path the device may take to traverse at least one surface feature.

[0041] 36. A method for real-time control of the configuration of a device, the device comprising a chassis, at least four wheels, a first side of the chassis operably coupled to at least one of the at least four wheels, and an opposing second side of the chassis operably coupled to at least one of the at least four wheels, the method comprising receiving environmental data, determining a surface type based at least on the environmental data, determining a mode based at least on the surface type and a first configuration, determining a second configuration based at least on the mode and surface type, determining a move command based at least on the second configuration, and controlling the configuration of the device by using the move command to change the device from a first configuration to a second configuration. 37. The method according to item 36, wherein the environmental data comprises RGB-D image data. 38. The method according to item 36, further comprising capturing an occupied grid based at least on the surface type and mode, and determining a move command based at least on the occupied grid. 39. The method according to item 38, wherein the occupied grid comprises information based on data from at least one image sensor. 40. The method of item 36, wherein environmental data comprises the form of a road surface. 41. The method of item 36, wherein the configuration comprises two clustered pairs of at least four wheels, the first pair of the two pairs being positioned on the first side, the second pair of the two pairs being positioned on the second side, the first pair comprising a first front wheel and a first rear wheel, and the second pair comprising a second front wheel and a second rear wheel. 42. The method of item 41, wherein the control of the configuration comprises coordinated power supply of the first pair and the second pair, based at least on environmental data. 43. The method of item 41, wherein the control of the configuration is to drive at least four wheels and pairs of retractable casters, the pairs of casters being operably coupled to the chassis, and to drive two wheels, with a clustered first pair and a clustered second pair, which are rotated to lift the first front wheels and the second front wheels, and the device is to transition to being stationary on the first rear wheels, the second rear wheels, and pairs of casters.44. The method of item 41, wherein the control of the configuration includes rotating a pair of clusters operably coupled with two first powered wheels on a first side and two second powered wheels on a second side, based on at least environmental data. 45. The method of item 36, wherein the device further comprises a cargo container, the cargo container mounted on a chassis, and the chassis controls the height of the cargo container. 46. The method of item 45, wherein the height of the cargo container is based on at least environmental data.

[0042] 47. A system for real-time control of the configuration of a device, the device comprising a chassis, at least four wheels, a first side of the chassis, and an opposing second side of the chassis, and comprising a device processor that receives real-time environmental data surrounding the device, the device processor determining a surface type based on at least the environmental data, the device processor determining a mode based on at least the surface type and a first configuration, and the device processor determining a second configuration based on at least the mode and surface type, and a power base processor that determines a move command based on at least the second configuration, the power base processor controlling the configuration of the device by using the move command, and changing the device from a first configuration to a second configuration. 48. The system according to item 47, wherein the environmental data comprises RGB-D image data. 49. The system according to item 47, wherein the device processor includes capturing an occupied grid based on at least the surface type and mode. 50. The system according to item 49, wherein the power base processor includes determining a move command based on at least the occupied grid. 51. The system according to item 49, wherein the occupied grid comprises information based on data from at least one image sensor. 52. The system according to item 47, wherein the environmental data comprises the form of a road surface. 53. The system according to item 47, wherein the configuration comprises two clustered pairs of at least four wheels, the first pair of the two pairs positioned on the first side, the second pair of the two pairs positioned on the second side, the first pair having a first front wheel and a first rear wheel, and the second pair having a second front wheel and a second rear wheel. 54. The system according to item 53, wherein the control of the configuration comprises coordinated power supply of the first pair and the second pair based on at least environmental data.55. The system according to item 53, wherein the control of the configuration is to drive at least four wheels and pairs of retractable casters, the pairs of casters being operably coupled to the chassis, and to drive two wheels, with a clustered first pair and a clustered second pair that are rotated to lift the first front wheels and the second front wheels, and the device transitions to being stationary on the first rear wheels, the second rear wheels and the pairs of casters.

[0043] 56. A method for maintaining a global occupancy grid, comprising: locating a first position of an autonomous device; updating the global occupancy grid using at least one occupancy probability value associated with the first position, when the autonomous device moves to a second position, the second position being associated with the global occupancy grid and the local occupancy grid; updating the global occupancy grid using at least one drivable surface associated with the local occupancy grid; updating the global occupancy grid using a surface confidence associated with at least one drivable surface; and using a first Bayesian function, the global occupancy grid using the log odds of at least one occupancy probability value. A method comprising updating a lid; adjusting log odds based on characteristics associated with at least a second location; updating the global occupancy grid using at least one drivable surface associated with the local occupancy grid when an autonomous device remains at a first location and the global occupancy grid and local occupancy grid are jointly installed; updating the global occupancy grid using surface confidence associated with at least one drivable surface; updating the global occupancy grid using log odds of at least one occupancy probability value using a second Bayesian function; and adjusting log odds based on characteristics associated with at least a second location. 57. The method of item 35, wherein creating a map includes accessing point cloud data representing a surface, filtering the point cloud data, forming the filtered point cloud data into a processable portion, merging the processable portion into at least one concave polygon, and locating and labeling at least one SDSF within at least one concave polygon, wherein locating and labeling forms labeled point cloud data, creating a graphing polygon based on at least one concave polygon, and selecting a path from a start point to an end point based on at least one graphing polygon, wherein the AV traverses at least one SDSF along the path.58. Filtering point cloud data is the method according to item 57, comprising conditionally removing points representing transient objects and points representing outliers from the point cloud data and replacing the removed points having a pre-selected height. 59. Forming a processing portion is the method according to item 57, comprising segmenting the point cloud data into a processable portion and removing points with a pre-selected height from the processable portion. 60. Merging processable portions is the method according to item 57, comprising reducing the size of the processable portion by analyzing outliers, voxels, and normals, expanding the area from the reduced-size processable portion, determining an initial drivable surface from the expanded area, segmenting and meshing the initial drivable surface, locating polygons within the segmented and meshed initial drivable surface, and defining at least one drivable surface based on at least the polygons. 61. The method according to item 60, wherein locating and labeling at least one SDSF is a sorting of point cloud data of an initial drivable surface according to an SDSF filter, wherein the SDSF filter includes points of at least three categories, and locating at least one SDSF point on the basis that at least three categories of points together satisfy at least one first pre-selected criterion. 62. The method according to item 61, further comprising creating at least one SDSF trajectory on the basis that at least several at least one SDSF points together satisfy at least one second pre-selected criterion. 63. The method of item 62, wherein creating a graphing polygon further comprises creating at least one polygon from at least one drivable surface, the at least one polygon including an outer edge, smoothing the outer edge, forming a drivable margin based on the smoothed outer edge, adding at least one SDSF orbit to at least one drivable surface, and removing an inner edge from at least one drivable surface according to at least one third pre-selected criterion.64. The method of item 63, wherein the outer edge smoothing comprises trimming the outer edge outward to form an outward-facing edge. 65. The method of item 63, wherein the smoothed outer edge running margin comprises trimming the outward-facing edge inward.

[0044] 66. An autonomous delivery vehicle comprising a power base including two powered front wheels, two powered rear wheels, and an energy storage device, wherein the power base is configured to move at a commanded speed, and a cargo platform including a plurality of short-range sensors, wherein the cargo platform is mechanically attached to the power base, and a cargo container having a volume for receiving one or more objects to be delivered, wherein the cargo container is mounted on the cargo platform, and a long-range sensor suite comprising a LIDAR and one or more cameras, wherein the long-range sensor suite is mounted on the cargo container, and a controller for receiving data from the long-range sensor suite and the plurality of short-range sensors. 67. The autonomous delivery vehicle according to item 66, wherein the plurality of short-range sensors detect at least one characteristic of a drivable surface. 68. The autonomous delivery vehicle according to item 66, wherein the plurality of short-range sensors are stereo cameras. 69. The autonomous delivery vehicle according to item 66, comprising multiple short-range sensors, an IR projector, two image sensors, and an RGB sensor. 70. The autonomous delivery vehicle according to item 66, wherein the multiple short-range sensors are radar sensors. 71. The autonomous delivery vehicle according to item 66, wherein the short-range sensors supply RGB-D data to a controller. 72. The autonomous delivery vehicle according to item 66, wherein the controller determines the geometric shape of the road surface based on the RGB-D data received from the multiple short-range sensors. 73. The autonomous delivery vehicle according to item 66, wherein the multiple short-range sensors detect objects within 4 meters of the autonomous delivery vehicle, and the long-range sensor suite detects objects beyond 4 meters from the autonomous delivery vehicle.

[0045] 74. An autonomous delivery vehicle comprising a power base including at least two powered rear wheels, caster front wheels, and an energy storage device, wherein the power base is configured to move at a commanded speed, and a cargo platform including a plurality of short-range sensors, wherein the cargo platform is mechanically attached to the power base, and a cargo container having a volume for receiving one or more objects to be delivered, wherein the cargo container is mounted on the cargo platform, and a long-range sensor suite comprising a LIDAR and one or more cameras, wherein the long-range sensor suite is mounted on the cargo container, and a controller for receiving data from the long-range sensor suite and the plurality of short-range sensors. 75. The autonomous delivery vehicle according to item 74, wherein the plurality of short-range sensors detect at least one characteristic of a drivable surface. 76. The autonomous delivery vehicle according to item 74, wherein the plurality of short-range sensors are stereo cameras. 77. The autonomous delivery vehicle according to item 74, comprising multiple short-range sensors, an IR projector, two image sensors, and an RGB sensor. 78. The autonomous delivery vehicle according to item 74, wherein the multiple short-range sensors are radar sensors. 79. The autonomous delivery vehicle according to item 74, wherein the short-range sensors supply RGB-D data to a controller. 80. The autonomous delivery vehicle according to item 74, wherein the controller determines the geometric shape of the road surface based on the RGB-D data received from the multiple short-range sensors. 81. The autonomous delivery vehicle according to item 74, wherein the multiple short-range sensors detect objects within 4 meters of the autonomous delivery vehicle, and the long-range sensor suite detects objects beyond 4 meters from the autonomous delivery vehicle. 82. The autonomous delivery vehicle according to item 74, further comprising a second set of powered wheels, the caster wheels being able to engage with the ground while being lifted away from the ground.

[0046] 83. An autonomous delivery vehicle comprising: a power base, which includes at least two powered rear wheels, a caster front wheel, and an energy storage device, the power base being configured to move at a commanded speed; a cargo platform, which is mechanically attached to the power base; and a short-range camera assembly mounted on the cargo platform for detecting at least one characteristic of a drivable surface, comprising a camera, a first light, and a first liquid-cooled heatsink, the first liquid-cooled heatsink cooling the first light and the camera. 84. The autonomous delivery vehicle according to item 83, wherein the short-range camera assembly further comprises a thermoelectric cooler between the camera and the liquid-cooled heatsink. 85. The autonomous delivery vehicle according to item 83, wherein the first light and camera are embedded in a cover with an opening that deflects illumination from the first light away from the camera. 86. The lights are angled downward at least 15° and embedded at least 4mm within the cover to minimize distracting lighting for pedestrians, as described in item 83. 87. The camera has a field of view, and the first light has two LEDs with lenses to produce two beams of light that diffuse to illuminate the camera's field of view, as described in item 83. 88. The lights are angled about 50° apart, and the lenses produce a 60° beam, as described in item 87. 89. The short-range camera assembly includes an ultrasonic sensor mounted above the camera, as described in item 83. 90. The short-range camera assembly is mounted in a central position on the front of the cargo platform, as described in item 83. 91. The autonomous delivery vehicle according to item 83, further comprising at least one corner camera assembly mounted on at least one corner of the front of the cargo platform, the at least one corner camera assembly comprising an ultrasonic sensor, a corner camera, a second light, and a second liquid-cooled heatsink, the second liquid-cooled heatsink comprising a second liquid-cooled heatsink for cooling the second light and the corner camera.92. The method of item 22, wherein the historical data comprises surface data. 93. The method of item 22, wherein the historical data comprises discontinuity data. The present invention provides, for example, the following: (Item 1) An autonomous delivery vehicle, A power base comprising two powered front wheels, two powered rear wheels, and an energy storage device, wherein the power base is configured to move at a commanded speed and in a commanded direction and to transport at least one object, A cargo platform including multiple short-range sensors, wherein the cargo platform is mechanically attached to the power base, A cargo container having a volume for receiving at least one object, wherein the cargo container is mounted on the cargo platform, A long-range sensor suite comprising a LIDAR and one or more cameras, wherein the long-range sensor suite is mounted on the cargo container, A controller for receiving data from the long-range sensor suite and the plurality of short-range sensors, wherein the controller determines the commanded speed and the commanded direction based on at least the data, and provides the commanded speed and the commanded direction to the power base to complete the transport, and An autonomous delivery vehicle equipped with [a specific feature / equipment]. (Item 2) The autonomous delivery vehicle according to item 1, wherein the data from the plurality of short-range sensors comprises at least one characteristic of the surface on which the power base travels. (Item 3) The autonomous delivery vehicle according to item 1, wherein the plurality of short-range sensors comprises at least one stereo camera. (Item 4) The autonomous delivery vehicle according to item 1, wherein the plurality of short-range sensors comprises at least one IR projector, at least one image sensor, and at least one RGB sensor. (Item 5) The autonomous delivery vehicle according to item 1, wherein the plurality of short-range sensors comprises at least one radar sensor. (Item 6) The data from the aforementioned multiple short-range sensors comprises RGB-D data, as described in item 1, for the autonomous delivery vehicle. (Item 7) The controller determines the geometric shape of the road surface based on the RGB-D data received from the plurality of short-range sensors, as described in item 1, for the autonomous delivery vehicle. (Item 8) The autonomous delivery vehicle according to item 1, wherein the plurality of short-range sensors detect objects within 4 meters of the AV, and the long-range sensor suite detects objects beyond 4 meters from the autonomous delivery vehicle. (Item 9) The autonomous delivery vehicle according to item 1, comprising the aforementioned multiple short-range sensors and a cooling circuit. (Item 10) The autonomous delivery vehicle according to item 1, wherein the plurality of short-range sensors are equipped with ultrasonic sensors. (Item 11) The aforementioned controller, It comprises executable code, and the executable code is This includes accessing a map, the map being formed by a map processor, and the map processor A first processor accesses point cloud data from the long-range sensor suite, wherein the point cloud data represents the surface, A filter for filtering the aforementioned point cloud data, A second processor that forms a processable portion from the filtered point cloud data, A third processor that merges the aforementioned processable portion into at least one polygon, A fourth processor, if present, locates and labels the at least one substantial discontinuous surface feature (SDSF) within the at least one polygon, wherein the locating and labeling constitutes the formation of labeled point cloud data. A fifth processor that creates a graphed polygon from the labeled point cloud data, A sixth processor that selects a path from a start point to an end point based on at least the graphed polygon, wherein the AV traverses at least one SDSF along the path, and An autonomous delivery vehicle as described in item 2, comprising the features described above. (Item 12) The aforementioned filter is Conditionally remove points representing transient objects and points representing outliers from the aforementioned point cloud data, Replacing the removed point having a pre-selected height An autonomous delivery vehicle as described in item 11, comprising a seventh processor that executes code, including, (Item 13) The aforementioned second processor is The point cloud data is segmented into the processable portion, Removing a point of a pre-selected height from the aforementioned processable portion. An autonomous delivery vehicle as described in item 11, including the executable code. (Item 14) The aforementioned third processor is By analyzing outliers, voxels, and normals, the size of the processable portion is reduced, To expand the area from the reduced size processing portion, Determining the initial drivable surface from the aforementioned enlarged region, The initial drivable surface is segmented and meshed, The process involves locating polygons within the segmented and meshed initial drivable surface, Based on at least the polygon, at least one drivable surface is defined. An autonomous delivery vehicle as described in item 11, including the executable code. (Item 15) The fourth processor is, The process involves sorting the point cloud data of the initial drivable surface according to an SDSF filter, wherein the SDSF filter includes points of at least three categories. At a minimum, locating at least one SDSF point based on whether the points of the at least three categories, in combination, satisfy at least one first pre-selected criterion. An autonomous delivery vehicle as described in item 14, including the executable code. (Item 16) The autonomous delivery vehicle according to item 15, comprising the executable code, the fourth processor, which includes creating at least one SDSF trajectory based on whether a plurality of the at least one SDSF points, in combination, satisfy at least one second pre-selected criterion. (Item 17) Creating a graphed polygon is, Creating at least one polygon from the at least one drivable surface, wherein the at least one polygon includes an outer edge, Smoothing the outer edge, Based on the smoothed outer edge, a travel margin is formed, Adding the at least one SDSF track to the at least one traversable surface, Removing the inner edge from the at least one drivable surface according to at least one third pre-selected criterion An autonomous delivery vehicle according to item 14, comprising an eighth processor, including the executable code, including the aforementioned. (Item 18) An autonomous delivery vehicle according to item 17, comprising a ninth processor, comprising the executable code, wherein smoothing the outer edge includes trimming the outer edge outward to form an outward-facing edge. (Item 19) An autonomous delivery vehicle according to item 18, comprising a tenth processor, comprising the executable code, wherein forming the smoothed outer edge travel margin includes trimming the outward edge inward. (Item 20) The aforementioned controller, The autonomous delivery vehicle (AV) comprises a subsystem for navigating at least one substantially discontinuous surface feature (SDSF) encountered, wherein the AV travels a path across the surface, the surface includes the at least one SDSF, the path includes a start point and an end point, and the subsystem is A first processor accessing a route configuration, wherein the route configuration includes at least one graphed polygon including filtered point cloud data, the filtered point cloud data includes labeled features, and the point cloud data includes drivable margins, and the first processor accesses a route configuration, A second processor that converts the aforementioned point cloud data into a global coordinate system, A third processor that determines the boundary of at least one SDSF, wherein the third processor creates an SDSF buffer of a pre-selected size around the boundary. A fourth processor that determines the at least one SDSF that can be crossed based on at least one SDSF crossing criterion, A fifth processor that creates an edge / weighted graph based on at least one SDSF cross-sectional criterion, the converted point cloud data, and the root morphology, A base controller selects the path from the starting point to the ending point based at least on the edge / weighted graph. An autonomous delivery vehicle as described in item 1, comprising the features described above. (Item 21) The aforementioned at least one SDSF cross-sectional criterion is, The pre-selected width of the at least one SDSF and the pre-selected smoothness of the at least one SDSF, A minimum entry distance and minimum exit distance between the at least one SDSF and the AV, including a drivable surface, Adapting the AV to an approximately 90° approach to the at least one SDSF, the minimum approach distance between the at least one SDSF and the AV and An autonomous delivery vehicle as described in item 20, comprising: (Item 22) A method for managing a global occupancy grid for an autonomous device, wherein the global occupancy grid includes global occupancy grid cells, the global occupancy grid cells are associated with occupancy probabilities, and the method is Receiving sensor data from a sensor associated with the autonomous device, At least based on the sensor data, a local occupancy grid is created, wherein the local occupancy grid has local occupancy grid cells. When the autonomous device moves from the first area to the second area, Accessing historical data associated with the second area, At least based on the aforementioned historical data, create a static grid, To move the global occupied grid and maintain the autonomous device at the central position of the global occupied grid, Based on the static grid, update the moved global occupancy grid, If at least one of the globally occupied grid cells coincides with the location of the autonomous device, then at least one of the globally occupied grid cells is marked as unoccupied. For each of the aforementioned locally occupied grid cells, Calculating the position of the local occupied grid cell on the global occupied grid, Accessing the first occupancy probability from the global occupancy grid cell at the aforementioned location, Accessing the second occupancy probability from the local occupying grid cell at the aforementioned location, Based on at least the first and second occupancy probabilities, a new occupancy probability is calculated for the position on the global occupancy grid. Methods that include... (Item 23) The method according to item 22, further comprising performing a range check on the new occupancy probability. (Item 24) Checking the range as described above means If the new occupancy probability is <0, the new occupancy probability is set to 0. If the new occupancy probability is >1, set the new occupancy probability to 1. The method described in item 23, including the method described in item 23. (Item 25) The method of item 22, further comprising setting the global occupied grid cell to the new occupied probability. (Item 26) The method of item 23, further comprising setting the global occupancy grid cell to the new occupancy probability that has been range-checked. (Item 27) A method for creating and managing an occupied grid, The local dedicated grid creation node converts sensor measurements into a reference frame associated with the device, Creating a timestamped measurement occupancy grid, The aforementioned timestamped measurement occupancy grid is made public as a local occupancy grid, Creating multiple local dedicated grids, Creating a static occupancy grid based on surface characteristics within a repository, wherein the surface characteristics are associated with the location of the device, To move the global occupancy grid associated with the location of the device, and to maintain the device and the local occupancy grid that are substantially aligned with the global occupancy grid, Adding information from the statically occupied grid to the globally occupied grid, To mark the area within the global occupied grid currently occupied by the aforementioned device as unoccupied, For at least one cell within each local occupied grid, Determining the location of at least one cell within the global occupancy grid, Accessing the first value at the aforementioned location, Based on the relationship between the first value and the cell value in at least one cell within the local occupied grid, a second value is determined at the location. The second value is compared to a pre-selected probability range, If the probability value is within the pre-selected probability range, the global occupancy grid with the new value is set. Methods that include... (Item 28) The method described in item 27, further comprising disclosing the aforementioned global occupancy grid. (Item 29) The surface characteristics are the method according to item 27, comprising a surface type and surface discontinuity. (Item 30) The aforementioned relationship is the method described in item 27, which includes summing. (Item 31) A system for creating and managing occupied grids, A plurality of local grid creation nodes that create at least one local occupied grid, wherein the at least one local occupied grid is associated with the location of a device, and the at least one local occupied grid includes at least one cell, A global occupying grid manager that accesses the aforementioned at least one local occupying grid, Equipped with, The aforementioned global occupancy grid manager is Creating a static occupancy grid based on surface characteristics within a repository, wherein the surface characteristics are associated with the location of the device, To move the global occupancy grid associated with the location of the device, and to maintain the device and the at least one local occupancy grid that are substantially aligned with the global occupancy grid, Adding information from the statically occupied grid to at least one globally occupied grid, To mark the area within the global occupied grid currently occupied by the aforementioned device as unoccupied, For each of the at least one cells in each local occupied grid, Determining the location of at least one cell within the global occupancy grid, Accessing the first value at the aforementioned location, Based on the relationship between the first value and the cell value in at least one cell within the local occupied grid, a second value is determined at the location. The second value is compared to a pre-selected probability range, If the probability value is within the pre-selected probability range, the global occupancy grid with the new value is set. A system that performs this task. (Item 32) A method for updating the global occupancy grid, When an autonomous device moves to a new location, it updates the global occupancy grid using information from the static grid associated with the new location. Analyzing the surface at the aforementioned new location, If the surface is drivable, the surface is updated, and the global occupied grid is updated using the updated surface. Updating the global occupancy grid using values ​​from a repository of static values, wherein the static values ​​are associated with the new location. Methods that include... (Item 33) Updating the aforementioned surface means Accessing the local occupied grid associated with the aforementioned new location, For each cell in the aforementioned local occupied grid, Accessing the confidence value of the local occupied grid surface classification and the local occupied grid surface classification, If the local occupied grid surface classification is the same as the global occupied grid surface classification within the cell, the confidence value of the global occupied grid surface classification is added to the confidence value of the local occupied grid surface classification to form a sum, and the global occupied grid in the cell is updated using the sum. If the local occupied grid surface classification is not the same as the global occupied grid surface classification within the cell, the local occupied grid surface classification confidence value is subtracted from the global occupied grid surface classification confidence value to form a difference, and the global occupied grid is updated using the difference. If the difference is less than zero, the global occupied grid is updated using the local occupied grid surface classification. The method described in item 32, including the method described in item 32. (Item 34) Updating the global occupancy grid using the values ​​from the static value repository is: For each cell in the local occupying grid, Accessing the log odds value, which is the local occupancy grid probability that the cell from the local occupancy grid is the occupancy value, The log-odds value in the global occupancy grid is updated using the local occupancy grid log-odds value in the aforementioned cell. If the pre-selected certainty that the cell is not occupied is met, and the autonomous device is moving within the lane barrier, and the local occupied grid surface classification indicates a drivable surface, then the log odds that the cell is occupied within the local occupied grid are reduced. If the autonomous device anticipates encountering a relatively uniform surface, and the local occupied grid surface classification indicates a relatively non-uniform surface, then the log odds within the local occupied grid are increased. When the autonomous device anticipates encountering a relatively uniform surface, and the local occupied grid surface classification indicates a relatively uniform surface, the log odds within the local occupied grid are reduced. The method described in item 32, including the method described in item 32. (Item 35) A method for real-time control of the configuration of a device, the device comprising a chassis, at least four wheels, a first side of the chassis operably coupled to at least one of the at least four wheels, and an opposing second side of the chassis operably coupled to at least one of the at least four wheels, the method being Creating a map based on at least prior surface features and occupancy grids, wherein the map is created non-real-time, the map includes at least one location, the at least one location is associated with at least one surface feature, and the at least one surface feature is associated with at least one surface classification and at least one mode, As the device progresses, the current surface characteristics are determined, The occupied grid is updated in real time using the current surface features, From the occupied grid and the map, determine the path the device may take to traverse the at least one surface feature. Methods that include... (Item 36) A method for real-time control of the configuration of a device, the device comprising a chassis, at least four wheels, a first side of the chassis operably coupled to at least one of the at least four wheels, and an opposing second side of the chassis operably coupled to at least one of the at least four wheels, the method being Receiving environmental data and, The surface type is determined based at least on the aforementioned environmental data, The mode is determined based at least on the surface type and the first configuration, The second configuration is determined based at least on the mode and the surface type, Determine the move command based on at least the second configuration described above, By using the aforementioned move command, the configuration of the device is controlled, and the device is changed from the first configuration to the second configuration. Methods that include... (Item 37) The aforementioned environmental data comprises RGB-D image data, as described in item 36. (Item 38) Based at least the surface type and the mode, the occupied grid is incorporated, The move command is determined based at least on the occupied grid. The method described in item 36, further including the method described in item 36. (Item 39) The method according to item 38, wherein the occupied grid comprises information based on data from at least one image sensor. (Item 40) The aforementioned environmental data comprises the form of a road surface, as described in item 36. (Item 41) The configuration according to item 36, comprising two clustered pairs of at least four wheels, wherein a first pair of the two pairs is positioned on the first side, a second pair of the two pairs is positioned on the second side, the first pair includes a first front wheel and a first rear wheel, and the second pair includes a second front wheel and a second rear wheel. (Item 42) The method according to item 41, wherein the control of the configuration includes coordinated power supply of the first pair and the second pair, based at least on the environmental data. (Item 43) The control of the configuration comprises a transition from driving the at least four wheels and pairs of retractable casters, the pairs of casters being operably coupled to the chassis, to driving two wheels, the clustered first pair and the clustered second pair, which are rotated to lift the first front wheel and the second front wheel, the device being stationary on the first rear wheel, the second rear wheel, and the pairs of casters, according to item 41. (Item 44) The method according to item 41, wherein the control of the configuration includes rotating a pair of clusters operably coupled with two first powered wheels on the first side and two second powered wheels on the second side, based at least on the environmental data. (Item 45) The device further comprises a cargo container, the cargo container mounted on the chassis, and the chassis controls the height of the cargo container, according to the method of item 36. (Item 46) The height of the cargo container is determined by the method described in item 45, based at least on the environmental data. (Item 47) A system for real-time control of the configuration of a device, wherein the device includes a chassis, at least four wheels, a first side of the chassis, and an opposing second side of the chassis, and the system is A device processor that receives real-time environmental data surrounding the device, wherein the device processor determines a surface type based on at least the environmental data, determines a mode based on at least the surface type and a first configuration, and determines a second configuration based on at least the mode and the surface type. A power base processor that determines a move command based on at least the second configuration, wherein the power base processor controls the configuration of the device by using the move command and changes the device from a first configuration to a second configuration. A system that includes these features. (Item 48) The aforementioned environmental data is the system described in item 47, which includes RGB-D image data. (Item 49) The device processor is part of the system according to item 47, which includes taking in an occupied grid based at least on the surface type and the mode. (Item 50) The system according to item 49, wherein the power base processor includes determining the move command based at least on the occupied grid. (Item 51) The system according to item 49, wherein the occupied grid comprises information based on data from at least one image sensor. (Item 52) The aforementioned environmental data is provided for the system described in item 47, which includes the form of a road surface. (Item 53) The configuration comprises two clustered pairs of at least four wheels, the first pair of the two pairs being positioned on the first side, the second pair of the two pairs being positioned on the second side, the first pair having a first front wheel and a first rear wheel, and the second pair having a second front wheel and a second rear wheel, according to item 47. (Item 54) The control of the configuration includes the coordinated power supply of the first pair and the second pair, based at least on the environmental data, as described in item 53. (Item 55) Control of the configuration includes transitioning from driving the at least four wheels and pairs of reversible casters, the pairs of casters being operably coupled to the chassis, to driving two wheels, the clustered first pair and the clustered second pair, which are rotated to lift the first front wheel and the second front wheel, the device resting on the first rear wheel, the second rear wheel, and the pairs of casters, as in the system of item 53. (Item 56) A method for maintaining a global grid, To locate the first position of the autonomous device, When the autonomous device moves to the second position, the second position is associated with the global occupancy grid and the local occupancy grid. Updating the global occupancy grid using at least one occupancy probability value associated with the first position, Updating the global occupancy grid using at least one drivable surface associated with the local occupancy grid, Updating the global occupancy grid using the surface reliability associated with the at least one drivable surface, The global occupancy grid is updated using the log odds of at least one occupancy probability value using a first Bayesian function, Adjusting the log odds based on at least the characteristics associated with the second position, When the autonomous device remains in the first position and the global occupying grid and the local occupying grid are jointly installed, Updating the global occupancy grid using the at least one drivable surface associated with the local occupancy grid, Updating the global occupancy grid using the surface reliability associated with the at least one drivable surface, The global occupancy grid is updated using the log odds of at least one occupancy probability value, using a second Bayesian function. Adjusting the log odds based on at least the characteristics associated with the second position Methods that include... (Item 57) Creating the aforementioned map means Accessing point cloud data representing the aforementioned surface, Filtering the aforementioned point cloud data, The filtered point cloud data is formed into a processable portion, Merging the aforementioned processable portion into at least one concave polygon, The process involves locating and labeling the at least one SDSF within the at least one concave polygon, wherein the locating and labeling process involves forming labeled point cloud data. At a minimum, a graphed polygon is created based on at least one of the concave polygons, The process involves selecting the path from the starting point to the ending point based on at least the graphed polygon, wherein the AV traverses at least one SDSF along the path. The method described in item 35, including the method described in item 35. (Item 58) Filtering the aforementioned point cloud data is Conditionally remove points representing transient objects and points representing outliers from the aforementioned point cloud data, Replacing the removed point having a pre-selected height The method described in item 57, including the method described in item 57. (Item 59) Forming the processing part is The point cloud data is segmented into the processable portion, Removing a point of a pre-selected height from the aforementioned processable portion. The method described in item 57, including the method described in item 57. (Item 60) Merging the aforementioned processable portions means By analyzing outliers, voxels, and normals, the size of the processable portion is reduced, To expand the area from the reduced size processing portion, Determining the initial drivable surface from the aforementioned enlarged region, The initial drivable surface is segmented and meshed, The process involves locating polygons within the segmented and meshed initial drivable surface, Based on at least the polygon, at least one drivable surface is defined. The method described in item 57, including the method described in item 57. (Item 61) Identifying and labeling the aforementioned at least one SDSF is The process involves sorting the point cloud data of the initial drivable surface according to an SDSF filter, wherein the SDSF filter includes points of at least three categories. At a minimum, locating at least one SDSF point based on whether the points of the at least three categories, in combination, satisfy at least one first pre-selected criterion. The method described in item 60, including the method described in item 60. (Item 62) The method according to item 61, further comprising, at least, creating at least one SDSF trajectory based on whether a plurality of the at least one SDSF points, in combination, satisfy at least one second pre-selected criterion. (Item 63) Creating the aforementioned graphed polygon further involves, Creating at least one polygon from the at least one drivable surface, wherein the at least one polygon includes an outer edge, Smoothing the outer edge, Based on the smoothed outer edge, a travel margin is formed, Adding the at least one SDSF track to the at least one traversable surface, Removing the inner edge from the at least one drivable surface according to at least one third pre-selected criterion The method described in item 62, including the method described in item 62. (Item 64) The method according to item 63, wherein the smoothing of the outer edge includes trimming the outer edge outwards to form an outward-facing edge. (Item 65) The method according to item 63, wherein forming the running margin of the smoothed outer edge includes trimming the outward edge inward. (Item 66) An autonomous delivery vehicle, A power base comprising two powered front wheels, two powered rear wheels, and an energy storage device, wherein the power base is configured to move at a commanded speed, A cargo platform including multiple short-range sensors, wherein the cargo platform is mechanically attached to the power base, A cargo container having volume for receiving one or more objects to be delivered, wherein the cargo container is mounted on the cargo platform, A long-range sensor suite comprising a LIDAR and one or more cameras, wherein the long-range sensor suite is mounted on the cargo container, A controller for receiving data from the long-range sensor suite and the plurality of short-range sensors. An autonomous delivery vehicle equipped with [a specific feature / equipment]. (Item 67) The autonomous delivery vehicle according to item 66, wherein the plurality of short-range sensors detect at least one characteristic of a drivable surface. (Item 68) The aforementioned multiple short-range sensors are stereo cameras, as described in item 66 for the autonomous delivery vehicle. (Item 69) The autonomous delivery vehicle according to item 66 comprises an IR projector, two image sensors, and an RGB sensor. (Item 70) The aforementioned multiple short-range sensors are radar sensors, as described in item 66 for the autonomous delivery vehicle. (Item 71) The short-range sensor supplies RGB-D data to the controller, as described in item 66, for the autonomous delivery vehicle. (Item 72) The autonomous delivery vehicle according to item 66, wherein the controller determines the geometric shape of the road surface based on RGB-D data received from the plurality of short-range sensors. (Item 73) The autonomous delivery vehicle according to item 66, wherein the plurality of short-range sensors detect objects within 4 meters of the autonomous delivery vehicle, and the long-range sensor suite detects objects beyond 4 meters from the autonomous delivery vehicle. (Item 74) An autonomous delivery vehicle, A power base comprising at least two powered rear wheels, a caster front wheel, and an energy storage device, wherein the power base is configured to move at a commanded speed, A cargo platform including multiple short-range sensors, wherein the cargo platform is mechanically attached to the power base, A cargo container having volume for receiving one or more objects to be delivered, wherein the cargo container is mounted on the cargo platform, A long-range sensor suite comprising a LIDAR and one or more cameras, wherein the long-range sensor suite is mounted on the cargo container, A controller for receiving data from the long-range sensor suite and the plurality of short-range sensors. An autonomous delivery vehicle equipped with [a specific feature / equipment]. (Item 75) The autonomous delivery vehicle according to item 74, wherein the plurality of short-range sensors detect at least one characteristic of a drivable surface. (Item 76) The aforementioned multiple short-range sensors are stereo cameras, as described in item 74 for the autonomous delivery vehicle. (Item 77) The autonomous delivery vehicle according to item 74 comprises an IR projector, two image sensors, and an RGB sensor. (Item 78) The aforementioned multiple short-range sensors are radar sensors, as described in item 74 for the autonomous delivery vehicle. (Item 79) The short-range sensor supplies RGB-D data to the controller, as described in item 74, for the autonomous delivery vehicle. (Item 80) The controller determines the geometric shape of the road surface based on RGB-D data received from the plurality of short-range sensors, as described in item 74, for the autonomous delivery vehicle. (Item 81) The autonomous delivery vehicle according to item 74, wherein the plurality of short-range sensors detect objects within 4 meters of the autonomous delivery vehicle, and the long-range sensor suite detects objects beyond 4 meters from the autonomous delivery vehicle. (Item 82) The autonomous delivery vehicle according to item 74, further comprising a second set of powered wheels that can engage with the ground while the caster wheels are lifted away from the ground. (Item 83) An autonomous delivery vehicle, A power base comprising at least two powered rear wheels, a caster front wheel, and an energy storage device, wherein the power base is configured to move at a commanded speed, A cargo platform, wherein the cargo platform is mechanically attached to the power base, A short-range camera assembly mounted on the cargo platform, which detects at least one characteristic of the drivable surface, Equipped with, The aforementioned short-range camera assembly is Camera and, The first light, First liquid cooling heatsink and Equipped with, The first liquid-cooled heatsink cools the first light and the camera of an autonomous delivery vehicle. (Item 84) The short-range camera assembly further comprises a thermoelectric cooler between the camera and the liquid-cooled heat sink, as described in item 83, for the autonomous delivery vehicle. (Item 85) The autonomous delivery vehicle according to item 83, wherein the first light and the camera are embedded in a cover with an opening that deflects the illumination from the first light away from the camera. (Item 86) The lights are angled downward at least 15° and embedded at least 4mm within the cover to minimize distracting lighting for pedestrians, as described in item 83 of the Autonomous Delivery Vehicle. (Item 87) The autonomous delivery vehicle according to item 83, wherein the camera has a field of view, and the first light comprises two LEDs with lenses for producing two beams of light that diffuse to illuminate the field of view of the camera. (Item 88) The lights are angled at a distance of approximately 50°, and the lenses produce a 60° beam, as described in item 87 of the autonomous delivery vehicle. (Item 89) The short-range camera assembly includes an ultrasonic sensor mounted above the camera, as described in item 83, for the autonomous delivery vehicle. (Item 90) The short-range camera assembly is mounted in a central position on the front of the cargo platform of the autonomous delivery vehicle as described in item 83. (Item 91) The cargo platform further comprises at least one corner camera assembly mounted on at least one corner of the front of the cargo platform, the at least one corner camera assembly is Ultrasonic sensor and, Corner camera and The second light, A second liquid-cooled heatsink, wherein the second liquid-cooled heatsink cools the second light and the corner camera, and An autonomous delivery vehicle as described in item 83, comprising: (Item 92) The method according to item 22, wherein the historical data comprises surface data. (Item 93) The method described in item 22, wherein the historical data includes discontinuous data. [Brief explanation of the drawing]

[0047] This instruction will be easier to understand with reference to the following explanation, which is assumed to be accompanied by the diagrams.

[0048] [Figure 1-1] Figure 1-1 is a schematic block diagram of the main components of the system described in this instruction.

[0049] [Figure 1-2] Figure 1-2 is a schematic block diagram of the main components of the map processor in this instruction.

[0050] [Figure 1-3] Figure 1-3 is a schematic block diagram of the main components of the perceptual processor in this instruction.

[0051] [Figure 1-4] Figure 1-4 is a schematic block diagram of the main components of the autonomy processor described in this teaching.

[0052] [Figure 1A] Figure 1A is a schematic block diagram of the system of this instruction for preparing the progress path for AV.

[0053] [Figure 1B] Figure 1B is a pictorial illustration of an exemplary configuration of a device incorporating the system described in this instruction.

[0054] [Figure 1C] Figure 1C is a side view of an automated delivery vehicle showing the fields of view of several long-range and short-range sensors.

[0055] [Figure 1D] Figure 1D is a schematic block diagram of the map processor described in this instruction.

[0056] [Figure 1E] Figure 1E is a diagram illustrating the first part of the map processor flow in this instruction.

[0057] [Figure 1F] Figure 1F is an image of the segmented point cloud from this instruction.

[0058] [Figure 1G] Figure 1G is a diagram of the second part of the map processor in this instruction.

[0059] [Figure 1H] Figure 1H is an image showing the results of detecting drivable surfaces using this instruction.

[0060] [Figure 1I] Figure 1I is a diagram illustrating the flow of the SDSF detector described in this instruction.

[0061] [Figure 1J] Figure 1J is a pictorial illustration of the SDSF category in this instruction.

[0062] [Figure 1K] Figure 1K is an image of the SDSF identified by the system of this instruction.

[0063] [Figure 1L] Figures 1L and 1M illustrate the polygon processing described in this instruction. [Figure 1M] Figures 1L and 1M illustrate the polygon processing described in this instruction.

[0064] [Figure 1N] Figure 1N shows images of polygons and SDSFs identified by the system of this instruction.

[0065] [Figure 2A] Figure 2A is an isometric view of the autonomous vehicle in this instruction.

[0066] [Figure 2B] Figure 2B is a top view of a cargo container showing the field of view of the selected long-range sensor.

[0067] [Figure 2C] Figures 2C-2F show a diagram of a long-range sensor assembly. [Figure 2D] Figures 2C-2F show a diagram of a long-range sensor assembly. [Figure 2E] Figures 2C-2F show a diagram of a long-range sensor assembly. [Figure 2F] Figures 2C-2F show a diagram of a long-range sensor assembly.

[0068] [Figure 2G] Figure 2G is a top view of a cargo container showing the field of view of the selected short-range sensor.

[0069] [Figure 2H] Figure 2H is an isometric view of the cargo platform in this instruction.

[0070] [Figure 2I] Figure 2I-2L is an isometric view of a short-range sensor. [Figure 2J] Figure 2I-2L is an isometric view of a short-range sensor. [Figure 2K] Figure 2I-2L is an isometric view of a short-range sensor. [Figure 2L] Figure 2I-2L is an isometric view of a short-range sensor.

[0071] [Figure 2M] Figure 2M-2N is an isometric view of the autonomous vehicle in this instruction. [Figure 2N] Figure 2M-2N is an isometric view of the autonomous vehicle in this instruction.

[0072] [Figure 2O] Figure 2O-2P is an isometric view of the autonomous vehicle of this instruction with the outer panels removed. [Figure 2P] Figure 2O-2P is an isometric view of the autonomous vehicle of this instruction with the outer panels removed.

[0073] [Figure 2Q] Figure 2Q is an isometric view of the autonomous vehicle in this instruction, with a portion of the upper panel removed.

[0074] [Figure 2R] Figure 2R-2V shows the long-range sensor on the autonomous vehicle in this teaching. [Figure 2S] Figure 2R-2V shows the long-range sensor on the autonomous vehicle in this teaching. [Figure 2T] Figure 2R-2V shows the long-range sensor on the autonomous vehicle in this teaching. [Figure 2U] Figure 2R-2V shows the long-range sensor on the autonomous vehicle in this teaching. [Figure 2V] Figure 2R-2V shows the long-range sensor on the autonomous vehicle in this teaching.

[0075] [Figure 2W] Figure 2W-2Z shows an ultrasonic sensor. [Figure 2X] Figure 2W-2Z shows an ultrasonic sensor. [Figure 2Y] Figure 2W-2Z shows an ultrasonic sensor. [Figure 2Z] Figure 2W-2Z shows an ultrasonic sensor.

[0076] [Figure 2AA] Figure 2AA-2BB shows the central short-range camera assembly. [Figure 2BB] Figure 2AA-2BB shows the central short-range camera assembly.

[0077] [Figure 3A] Figure 3A is a schematic block diagram of one configuration of the system described in this instruction.

[0078] [Figure 3B] Figure 3B is a schematic block diagram of an alternative system configuration of this instruction.

[0079] [Figure 3C] Figure 3C is a schematic block diagram of the system in this instruction, which can initially create a global occupancy grid.

[0080] [Figure 3D] Figure 3D is a pictorial representation of the static grid in this instruction.

[0081] [Figure 3E] Figures 3E and 3F are pictorial representations of the creation of the occupying grid in this instruction. [Figure 3F] Figures 3E and 3F are pictorial representations of the creation of the occupying grid in this instruction.

[0082] [Figure 3G]Figure 3G is a pictorial representation of the prior occupation grid for this instruction.

[0083] [Figure 3H] Figure 3H is a pictorial representation of updating the global occupancy grid in this instruction.

[0084] [Figure 3I] Figure 3I is a flowchart of the method described in this teaching for exposing a global occupied grid.

[0085] [Figure 3J] Figure 3J is a flowchart of the method of this teaching for updating the global occupancy grid.

[0086] [Figure 3K] Figure 3K-3M is a flowchart of another method of this teaching for updating the global occupancy grid. [Figure 3L] Figure 3K-3M is a flowchart of another method of this teaching for updating the global occupancy grid. [Figure 3M] Figure 3K-3M is a flowchart of another method of this teaching for updating the global occupancy grid.

[0087] [Figure 4A] Figure 4A is a perspective drawing of the device described in this instruction, showing it installed in various modes.

[0088] [Figure 4B] Figure 4B is a schematic block diagram of the system described in this instruction.

[0089] [Figure 4C] Figure 4C is a schematic block diagram of the running surface processor component of this instruction.

[0090] [Figure 4D] Figure 4D is an outline block / picture flowchart of the process described in this instruction.

[0091] [Figure 4E] Figures 4E and 4F are perspective and side views, respectively, of the device configuration described in this instruction in standard mode. [Figure 4F] Figures 4E and 4F are perspective and side views, respectively, of the device configuration described in this instruction in standard mode.

[0092] [Figure 4G] Figures 4G and 4H are perspective and side views, respectively, of the configuration of the device in this instruction in four-wheel mode. [Figure 4H] Figures 4G and 4H are perspective and side views, respectively, of the configuration of the device in this instruction in four-wheel mode.

[0093] [Figure 4I] Figures 4I and 4J are perspective and side views, respectively, of the configuration of the device of this instruction in the lifting four-wheel mode. [Figure 4J] Figures 4I and 4J are perspective and side views, respectively, of the configuration of the device of this instruction in the lifting four-wheel mode.

[0094] [Figure 4K] Figure 4K is a flowchart of the method described in this instruction.

[0095] [Figure 5A] Figure 5A is a schematic block diagram of the device controller described in this instruction.

[0096] [Figure 5B] Figure 5B is a schematic block diagram of the SDSF processor described in this instruction.

[0097] [Figure 5C] Figure 5C is an image of the SDSF approach identified by the system in this instruction.

[0098] [Figure 5D]Figure 5D is an image of the route configuration created by the system described in this instruction.

[0099] [Figure 5E] Figure 5E is a schematic block diagram of the modes of instruction.

[0100] [Figure 5F] Figure 5F-5J is a flowchart of the teaching method for traversing the SDSF. [Figure 5G] Figure 5F-5J is a flowchart of the teaching method for traversing the SDSF. [Figure 5H] Figure 5F-5J is a flowchart of the teaching method for traversing the SDSF. [Figure 5I] Figure 5F-5J is a flowchart of the teaching method for traversing the SDSF. [Figure 5J] Figure 5F-5J is a flowchart of the teaching method for traversing the SDSF.

[0101] [Figure 5K] Figure 5K is a schematic block diagram of the system for traversing the SDSF in this instruction.

[0102] [Figure 5L] Figures 5L-5N are pictorial representations of the method shown in Figures 5F-5H. [Figure 5M] Figures 5L-5N are pictorial representations of the method shown in Figures 5F-5H. [Figure 5N] Figures 5L-5N are pictorial representations of the method shown in Figures 5F-5H.

[0103] [Figure 5O] Figure 5O is a pictorial representation that transforms an image into a polygon. [Modes for carrying out the invention]

[0104] (Detailed explanation) The system and method of this teaching can navigate AV across surface features, including using onboard sensors and previously developed maps to develop an occupancy grid, and using these aids to reconstruct AV based on surface type and prior information.

[0105] Referring here to Figure 1-1, the AV system 100 may include a structure on which a sensor 10701 may be mounted, and on which a device controller 10111 may be executed. The structure may include a power base 10112 that can direct the movement of wheels which are part of the structure, and enable the movement of the AV. The device controller 10111 may run on at least one processor located on the AV, and may, but is not limited to, located on the AV, and may receive data from the sensor 10701. The device controller 10111 may provide speed, direction, and configuration information to a base controller 10114, which may provide movement commands to the power base 10112. The device controller 10111 may receive map information from a map processor 10104, which may prepare a map of the area surrounding the AV. The device controller 10111 may include a sensor processor 10703 that can receive and process input from the sensor 10701, which may, but is not limited to, a sensor on the AV. In some configurations, the device controller 10111 may include a perception processor 2143, an autonomy processor 2145, and a driver processor 2127. The perception processor 2143 can, for example, locate static and dynamic obstacles, determine traffic signal conditions, create an occupancy grid, and classify surfaces. The autonomy processor 2145 can, for example, determine the maximum speed of the AV and determine the type of situation in which the AV is navigating, for example, on a road, on a sidewalk, at an intersection, and / or under remote control. The driver processor 2127 can, for example, create commands according to instructions from the autonomy processor 2145 and transmit them to the base controller 10114.

[0106] Referring here to Figure 1-2, the map processor 10104 can create a map of surface features and provide the map to the perception processor 2143 through the device controller 10111, which can update the occupied grid. The map processor 10104 may include, among many other aspects, a feature extractor 10801, a point cloud assembler 10803, a transient processor 10805, a segmenter 10807, a polygon generator 10809, an SDSF line generator 10811, and a combiner 10813. The feature extractor 10801 may include a first processor that accesses point cloud data representing the surface. The point cloud assembler 10803 may include a second processor that forms a processable portion from the filtered point cloud data. The transient processor 10805 may include a first filter that filters the point cloud data. The segmenter 10807 may include executable code that includes, but is not limited to, segmenting point cloud data into processable portions and removing points with pre-selected heights from the processable portions. The first filter may optionally include executable code that includes, but is not limited to, conditionally removing points representing transient objects and outliers from the point cloud data and replacing the removed points with pre-selected heights. The polygon generator 10809 may include a third processor that merges the processable portions into at least one concave polygon. The third processor may include executable code that optionally, but not limited to, analyze outliers, voxels, and normals to reduce the size of the processable portion, expand the region from the reduced-size processable portion, determine the initial drivable surface from the expanded region, segment and mesh the initial drivable surface, locate polygons within the segmented and meshed initial drivable surface, and define the drivable surface based on at least the polygons.The SDSF line generator 10811 includes a fourth processor that localizes and labels at least one SDSF within at least one concave polygon, and localizing and labeling can form labeled point cloud data. The fourth processor may optionally include executable code that sorts point cloud data of drivable surfaces according to an SDSF filter, wherein the SDSF filter includes points of at least three categories, and localizes at least one SDSF point based on whether the points of the categories, in combination, satisfy at least one first pre-selected criterion. The combiner 10813 may include a fifth processor that creates a graphing polygon. Creating a graphed polygon may optionally include, but is not limited to, creating at least one polygon from at least one drivable surface, wherein at least one polygon includes an edge, smoothing the edge, forming a traverse margin based on the smoothed edge, adding at least one SDSF traverse to at least one drivable surface, and removing the edge from at least one drivable surface according to at least one third pre-selected criterion. Smoothing the edge may optionally include, but is not limited to, trimming the edge outwards, and may include executable code. Forming a traverse margin of the smoothed edge may optionally include, but is not limited to, trimming the outward-facing edge inwards, and may include executable code.

[0107] Referring to FIGS. 1-3 here, a map can be provided to the AV, which can include an on-board sensor, a powered wheel, a processor for receiving sensor and map data and using those data to power configure the AV such that the AV can, for example, deliver goods, especially across various types of surfaces. The on-board sensor can capture an occupancy grid and provide data that can be used to detect dynamic obstacles. The occupancy grid can also be captured by the map. The device controller 10111 can include a perception processor 2143 that can receive and process sensor data and map data and update the occupancy grid using those data.

[0108] Referring to FIGS. 1-4 here, the device controller 10111 can include a configuration processor 41023 that can automatically determine the configuration of the AV based at least on the mode of the AV and the surface characteristics encountered. The autonomy processor 2145 can include a control processor 4032* that can determine the type of surface that needs to be traversed and the configuration that the AV needs to take to traverse the surface based at least on the map (the planned route to be traversed), information from the configuration processor 41023, and the mode of the AV. The autonomy processor 2145 can supply commands to a motor drive processor 40326 and implement the commands.

[0109] Referring to FIG. 1A here, a map processor 10104 can enable a device, e.g., but not limited to, an AV or a semi-autonomous device, to navigate within an environment that can include features such as SDSF. The features within the map, together with the on-board sensor, can enable the AV to travel on various surfaces. In particular, the SDSF can be accurately identified and labeled such that the AV can automatically maintain its performance during entry and exit of the SDSF and the AV speed, configuration, and direction can be controlled for a safe SDSF crossing.

[0110] Note: There seems to be a typo in the original text where "4032*" is written instead of a complete number in the description of "control processor 4032*" in the sentence of ID=4. This has been maintained as it is in the translation.Continuing to refer to Figure 1A, in some configurations, the system 100 for managing the SDSF crossing may include AV 10101, core cloud infrastructure 10103, AV service 10105, device controller 10111, sensor 10701, and power base 10112. AV 10101 can provide transportation and shuttle services from origin to destination by following a dynamically determined route, for example, but not limited to, incoming sensor information. AV 10101 may include devices having an autonomous mode, devices capable of operating fully autonomously, devices capable of operating at least partially remotely, and combinations of these features. Transportation device service 10105 can provide the device controller 10111 with drivable surface information, including features. The device controller 10111 can modify the drivable surface information, for example, but not limited to, incoming sensor information and feature crossing requirements, and can select a route for AV 10101 based on the modified drivable surface information. The device controller 10111 can present commands to the power base 10112, instructing it to provide speed, direction, and configuration commands to the wheel motors and cluster motors, which in turn cause the AV 10101 to follow a selected route and, accordingly, raise and lower its cargo. The transport device service 10105 can access route-related information from the core cloud infrastructure 10103, which may include, but is not limited to, storage and content distribution equipment. In some configurations, the core cloud infrastructure 10103 may include, for example, AMAZON WEB SERVICES®, GOOGLE CLOUD TM This may include commercial products such as ORACLE CLOUD®.

[0111] Referring here to Figure 1B, an exemplary AV, which may include a device controller 10111 (Figure 1A) capable of receiving information from the map processor 10104 (Figure 1A) of this teaching, may include, for example, a power base assembly such as a power base, which is fully described in, for example, U.S. Patent Application No. 16 / 035,205, filed July 13, 2018, titled "Mobility Device" or U.S. Patent No. 6,571,892, filed August 15, 2001, titled "Control System and Method" (both of which are incorporated herein by reference as a whole). The exemplary power base assembly is described herein not to limit this teaching, but rather to highlight the features of any power base assembly that may be useful in implementing the art of this teaching. The exemplary power base assembly may optionally include a power base 10112, a wheel cluster assembly 11100, and a payload carrier height assembly 10068. The exemplary power base assembly can optionally provide electrical and mechanical power to drive the wheel 11203 and the cluster 11100, which can raise and lower the wheel 11203. The power base 10112 can control the rotation of the cluster assembly 11100 and the raising and lowering of the payload carrier height assembly 10068 to support the substantial discontinuous surface crossing of this teaching. Other such devices can also be used to adapt the SDSF detection and crossing of this teaching.

[0112] Referring again to Figure 1A, in some configurations, internal sensors in the exemplary power base can detect the orientation of the AV10101 and the rate of change of orientation, the motors can enable servo operation, and the controller can understand the information from the internal sensors and motors. Appropriate motor commands can be calculated to achieve transporter performance and implement path-following commands. Left and right wheel motors can drive the wheels on both sides of the AV10101. In some configurations, the front and rear wheels can be coupled to drive together such that the two left wheels can drive together and the two right wheels can drive together. In some configurations, turning can be accomplished by driving the left and right motors at different rates, and the cluster motors can rotate the wheelbase in the forward / rear direction. This can allow the AV10101 to remain horizontal while the front wheels are higher or lower than the rear wheels. This feature may be useful, for example, when ascending or descending an SDSF. The payload carrier 10173 can be automatically ascended and descended based at least on the lower terrain.

[0113] Continuing to refer to Figure 1A, in some configurations, the point cloud data may include route information relating to the area that AV10101 should traverse. Possibly, point cloud data collected by a mapping device similar to or identical to AV10101 may be time-tagged. The path along which the mapping device traverses may be referred to as a mapped trajectory. The point cloud data processing described herein may be performed as the mapping device traverses the mapped trajectory or after point cloud data collection is complete. After the point cloud data has been collected, it may undergo point cloud data processing, which may include initial filtering and point reduction, point cloud segmentation, and feature detection, as described herein. In some configurations, the core cloud infrastructure 10103 may provide long-term or short-term storage for the collected point cloud data and provide the data to AV service 10105. AV service 10105 can select from possible point cloud datasets to find a dataset that covers the area surrounding a desired starting point and a desired destination related to AV10101. AV service 10105 may include, but is not limited to, a map processor 10104 that can reduce the size of the point cloud data and determine the features represented within the point cloud data. In some configurations, the map processor 10104 can determine the location of SDSFs from the point cloud data. In some configurations, polygons can be created from the point cloud data as a technique for segmenting the point cloud data and ultimately defining drivable surfaces. In some configurations, SDSF detection and drivable surface determination can proceed in parallel. In some configurations, SDSF detection and drivable surface determination can proceed sequentially.

[0114] Referring here to Figure 1C, in some configurations, the AV may be configured to perform other functions involving the delivery of cargo and / or autonomous navigation to a desired location. In some applications, the AV may be remotely guided. In some configurations, the AV 20100 includes a cargo container that can be released remotely, automatically, or manually in response to user input, allowing the user to place or remove cargo and other items. The cargo container 20110 is mounted on a cargo platform 20160, which is mechanically connected to a power base 20170. The power base 20170 includes four powered wheels 20174 and two caster wheels 20176. The power base provides speed and direction control to move the cargo container 20110 along the ground and over obstacles including curbs and other discontinuous surface features.

[0115] Continuing to refer to Figure 1C, the cargo platform 20160 is connected to the power base 20170 through two U-frames 20162. Each U-frame 20162 is rigidly attached to the structure of the cargo platform 20160 and includes two holes, allowing a rotatable joint 20164 to be formed together with the ends of each arm 20172 on the power base 20170. The power base controls the rotational position of the arms and, therefore, controls the height and tilt angle of the cargo container 20110.

[0116] Continuing to refer to Figure 1C, in some configurations, the AV20100 includes one or more processors for receiving data, navigating the path, and selecting the direction and speed of the power base 20170.

[0117] Referring here to Figure 1D, in some configurations, the map processor 10104 of this teaching can position SDSF on the map. The map processor 10104 may include, but is not limited to, a feature extractor 10801, a point cloud compiler 10803, a transient processor 10805, a segmenter 10807, a polygon generator 10809, an SDSF line generator 10811, and a data combiner 10813.

[0118] Continuing with Figure 1D, the feature extractor 10801 (Figure 1-2) may include, but is not limited to, line-of-sight filtering 10121 of the point cloud data 10131 and the mapped trajectory 10133. Line-of-sight filtering can remove points that are hidden from the direct line of sight of the sensor that collects the point cloud data and forms the mapped trajectory. The point cloud organizer 10803 (Figure 1-2) may, as possible, organize the reduced point cloud data 10132 according to pre-selected criteria associated with specific features (10151). In some configurations, the transient processor 10805 (Figure 1-2) may remove transients from the organized point cloud data and mapped trajectory 10133 by any number of methods, including those described herein (10153). Transients can complicate processing, in particular when the specific features are stationary. The segmenter 10807 (Figure 1-2) can divide the processed point cloud data 10135 into processable chunks. In some configurations, the processed point cloud data 10135 can be segmented into chunks having a pre-selected minimum number of points, e.g., approximately 100,000 points (10155), though not limited to these. In some configurations, further point reduction can be based on pre-selected criteria that may be relevant to the features to be extracted. For example, if points above a certain height are not important for localizing features, those points can be removed from the point cloud data. In some configurations, the height of at least one of the sensors collecting the point cloud data can be considered the origin, and points above the origin can be removed from the point cloud data, e.g., when only points of interest are associated with surface features. After the filtered point cloud data 10135 has been segmented, it forms segments 10137, and the remaining points can be divided into drivable surface chunks, from which surface features can be localized. In some configurations, the polygon generator 10809 (Figure 1-2) can locate a traversable surface by generating, for example, a polygon 10139 (10161) as described herein, but not limited to.In some configurations, the SDSF line generator 10811 (Figure 1-2) can locate surface features by generating SDSF lines 10141 (10163), for example, as described herein, but not limited to. In some configurations, the combiner 10813 (Figure 1-2) can combine polygon 10139 and SDSF 10141 (10165) to create a dataset that can be further processed to generate actual paths that AV 10101 (Figure 1A) may travel.

[0119] Here, primarily referring to Figure 1E, removing transient objects with respect to the mapped trajectory 10133, such as exemplary timestamped point 10751, from the point cloud data 10131 (Figure 1D) 10153 (Figure 1D) may include casting a ray 10753 from a timestamped point on the mapped trajectory 10133 to each timestamped point in the point cloud data 10131 (Figure 1D) that has substantially the same timestamp. If the ray 10753 intersects a point between the timestamped point on the mapped trajectory 10133 and the endpoint of the ray 10753, for example, point D10755, then the intersection point D10755 can be assumed to have entered the point cloud data between different sweeps of the camera. The intersection point, for example, intersection point D10755, can be assumed to be part of a transient object and can be removed from the reduced point cloud data 10132 (Figure 1D) as not representing fixed features such as SDSF. The result is, for example, processed point cloud data 10135 (Figure 1D) without transient objects, though not limited to them. Points that were removed as part of transient objects but are also substantially at ground level can be returned to processed point cloud data 10135 (Figure 1D) (10754). Transient objects cannot include certain features, such as, for example, SDSF10141 (Figure 1D), and therefore, when SDSF10141 (Figure 1D) is a detected feature, it can be removed without interfering with the integrity of point cloud data 10131 (Figure 1D).

[0120] Continuing to refer to Figure 1E, segmenting the processed point cloud data 10135 (Figure 1D) 10155 (Figure 1D) can generate segments 10757 having rectangles 10154 (Figure 1F) with pre-selected size and shape, e.g., minimum pre-selected side length and containing approximately 100,000 points. From each segment 10757, points that are not necessarily related to the specific task, e.g., points located above a pre-selected level, e.g., points

[0121] Again, primarily referring to Figure 1D, the map processor 10104 can supply the device controller 10111 with at least one dataset that can be used to generate direction, velocity, and configuration commands for controlling the AV 10101 (Figure 1A). At least one dataset may contain points that can be connected to other points in the dataset, and each line connecting points in the dataset traverses a traversable surface. To determine such root points, the segmented point cloud data 10137 can be divided into polygons 10139, and the vertices of the polygons 10139 can potentially be root points. The polygons 10139 may contain features such as, for example, SDSF 10141.

[0122] Continuing to refer to Figure 1D, in some configurations, creating the processed point cloud data 10135 may include filtering voxels. To reduce the number of points that will undergo future processing, in some configurations, the centroid of each voxel in the dataset may be used to approximate the points within the voxel, and all points other than the centroid may be excluded from the point cloud data. In some configurations, the center of a voxel may be used to approximate the points within the voxel. Other methods for reducing the size of the filtered segment 10251 (Figure 1G) may also be used, such as taking a random point subsample so that a fixed number of points, selected uniformly and randomly, may be excluded from the filtered segment 10251 (Figure 1G), for example.

[0123] Continuing to refer to Figure 1D, in some configurations, creating processed point cloud data 10135 may involve calculating normals from a dataset from which outliers have been removed and which has been resized through voxel filtering. The normals for each point in the filtered dataset can be used for various processing possibilities, including curve reconstruction algorithms. In some configurations, estimating and filtering normals in the dataset may involve obtaining the underlying surface from the dataset using a surface meshing technique and calculating normals from the surface mesh. In some configurations, estimating normals may involve using approximations to infer surface normals directly from the dataset, such as determining the normals to a fitting plane obtained by applying the total least squares method to the k nearest neighbors of a point, for example, but not limited to. In some configurations, the value of k may be selected based on at least empirical data. Filtering normals may involve removing any normals greater than approximately 45° from those perpendicular to the xy-plane. In some configurations, the filter may be used to align normals in the same direction. If a portion of the dataset represents a planar surface, redundant information contained within adjacent normals can be filtered out either by performing random subsampling or by filtering out one point from the set of related points. In some configurations, selecting points may involve recursively decomposing the dataset into boxes until each box contains at most k points. A single normal can be calculated from the k points within each box.

[0124] Continuing to refer to Figure 1D, in some configurations, creating the processed point cloud data 10135 may involve expanding the region within the dataset by clustering points that geometrically fit the surface representing the dataset, and refining the surface as the region expands to obtain the best approximation of the maximum number of points. Region expansion can be achieved by merging points in terms of smoothness constraints. In some configurations, the smoothness constraints may be determined empirically, for example, or based on a desired surface smoothness. In some configurations, the smoothness constraints may range from approximately 10π / 180 to approximately 20π / 180. The output of region expansion is a set of point clusters, each point cluster being a set of points, each of which is considered to be part of the same smooth surface. In some configurations, region expansion may be based on comparing angles between normals. Region expansion can be carried out, for example, by algorithms such as region-growing segmentation (http: / / pointclouds.org / documentation / tutorials / region_growing_segmentation.php) and cluster-extraction (http: / / pointclouds.org / documentation / tutorials / cluster_extraction.php#cluster-extraction), although these are not limited to these methods.

[0125] Referring here to Figure 1G, segmented point cloud data 10137 (Figure 1D) can be used to generate polygons 10759, for example, a 5m × 5m polygon (10161) (Figure 1D). Point subclusters can be transformed into polygons 10759 using, for example, meshing. Meshing can be performed by standard methods such as, but are not limited to, marching cubes, marching tetrahedra, surface nets, greedy meshing, and double contour formation. In some configurations, polygons 10759 can be generated by projecting the local neighborhoods of points along the normals of the points and connecting unconnected points. The resulting polygons 10759 can be based on, at a minimum, the size of the neighborhoods, the maximum allowable distance between points to be considered, the maximum edge length between the polygons, the minimum and maximum angles of the polygons, and the maximum deviation that the normals can take from each other. In some configurations, polygon 10759 can be filtered according to whether polygon 10759 would be too small for AV10101 (Figure 1A) to pass through. In some configurations, circles the size of AV10101 (Figure 1A) can be dragged around each of polygons 10759 by known means. If the circles substantially fit within polygon 10759, then polygon 10759, and therefore the resulting traversable surface, can accommodate AV10101 (Figure 1A). In some configurations, the area of ​​polygon 10759 can be compared to the area occupied by AV10101 (Figure 1A). The polygon can be assumed to be irregular, and therefore the first step in determining the area of ​​polygon 10759 would be to separate polygon 10759 into regular polygons 10759A by known methods. For each regular polygon 10759A, a standard area equation can be used to determine its size. The areas of each regular polygon 10759A can be added together to find the area of ​​polygon 10759, which can then be compared to the area occupied by AV10101 (Figure 1A). The filtered polygons may include a subset of polygons that satisfy the size criteria.The filtered polygons can be used to define the final drivable surface.

[0126] Continuing to refer to Figure 1G, in some configurations, polygon 10759 can be processed by removing outliers using conventional methods such as statistical analysis techniques available in, for example, the Point Cloud Library, http: / / pointclouds.org / documentation / tutorials / statistical_outlier.php. Filtering may include reducing the size of segment 10137 (Figure 1D) by conventional methods, including, for example, a voxelized grid approach available in, for example, the Point Cloud Library, http: / / pointclouds.org / documentation / tutorials / voxel_grid.php. Concave polygon 10263 can be created by, for example, the process described in, for example, the process described in "A New Concave Hull Algorithm and Concaveness Measure for n-dimensional Datasets", Park et al., Journal of Information Science and Engineering 28, pp. 587-600, 2012.

[0127] Here, primarily referring to Figure 1H, in some configurations, the processed point cloud data 10135 (Figure 1D) can be used to determine the initial drivable surface 10265. Region expansion can generate point clusters that may contain points that are part of the drivable surface. In some configurations, a reference plane can be fitted to each point cluster to determine the initial drivable surface. In some configurations, point clusters can be filtered according to the relationship between the orientation of the point cluster and the reference plane. For example, if the angle between the point cluster plane and the reference plane is, for example, less than approximately 30°, then the point cluster can be provisionally considered to be part of the initial drivable surface. In some configurations, point clusters can be filtered based on size constraints, for example, but not limited to. In some configurations, point clusters larger in point size than approximately 20% of the total points in the point cloud data 10131 (Figure 1D) can be considered too large, and point clusters smaller in size than approximately 0.1% of the total points in the point cloud data 10131 (Figure 1D) can be considered too small. The initial traversable surface may include filtered point clusters. In some configurations, point clusters can be split by one of several known methods to continue further processing. In some configurations, density-based spatial clustering of noisy applications (DBSCAN) can be used to split point clusters, while in some configurations, k-means clustering can be used to split point clusters. DBSCAN can group points that are densely clustered together and mark points that are substantially isolated or in low-density areas as outliers. To be considered densely clustered, points must be located within a pre-selected distance from candidate points. In some configurations, a scale factor with respect to the pre-selected distance can be determined empirically or dynamically. In some configurations, the scale factor can be in the range of approximately 0.1 to 1.0.

[0128] Referring primarily to Figure 1I, generating SDSF lines (10163) (Figure 1D) may include locating the SDSF by further filtering of the concave polygon 10263 on the traversable surface 10265 (Figure 1H). In some configurations, points from the point cloud data constituting the polygon can be categorized as either upper donut points 10351 (Figure 1J), lower donut points 10353 (Figure 1J), or cylindrical points 10355 (Figure 1J). Upper donut points 10351 (Figure 1J) may correspond to the shape of the SDSF model 10352 furthest from the ground. Lower donut points 10353 (Figure 1J) may correspond to the shape of the SDSF model 10352 closest to the ground or at ground level. The cylindrical point 10355 (Figure 1J) can correspond to a shape between the upper donut point 10351 (Figure 1J) and the lower donut point 10353 (Figure 1J). Combinations of categories can form a donut 10371. Certain criteria are tested to determine whether a donut 10371 forms an SDSF. For example, in each donut 10371, there must be a minimum number of points that are upper donut points 10351 (Figure 1J) and a minimum number that are lower donut points 10353 (Figure 1J). In some configurations, the minimum value can be chosen empirically and can correspond to a range of about 5 to 20. Each donut 10371 can be divided into multiple parts, for example, into two hemispheres. Another criterion for determining whether points in a donut 10371 represent an SDSF is whether the majority of points are located within the opposing hemispheres of the parts of the donut 10371. The cylindrical point 10355 (Figure 1J) can occur in either the first cylindrical region 10357 (Figure 1J) or the second cylindrical region 10359 (Figure 1J). Another criterion for SDSF selection is that a minimum number of points must exist within both cylindrical regions 10357 / 10359 (Figure 1J). In some configurations, the minimum number of points can be selected empirically and may fall in the range of 3 to 20.Another criterion for SDSF selection is that the donut 10371 must contain at least two of the three categories of points: the upper donut point 10351 (Figure 1J), the lower donut point 10353 (Figure 1J), and the cylindrical point 10355 (Figure 1J).

[0129] Continuing, primarily with reference to Figure 1I, in some configurations, polygons can be processed in parallel. Each category worker 10362 can search for its assigned polygon with respect to an SDSF point 10789 (Figure 1N) and assign the SDSF point 10789 (Figure 1N) to category 10763 (Figure 1G). As the polygons are processed, the resulting point categories 10763 (Figure 1G) can be combined (10363) to form a combined category 10366, and the categories can be shortened (10365) to form a shortened combined category 10368. Shortening the SDSF points 10789 (Figure 1N) may include filtering the SDSF points 10789 (Figure 1N) with respect to their distance from the ground. The shortened combined categories 10368 are averaged by exploring the area around each SDSF point 10766 (Figure 1G) and generating an average point 10765 (Figure 1G), which can potentially be processed in parallel by an average worker 10373, so that the points of the category can form a set of averaged donuts 10375. In some configurations, the radius around each SDSF point 10766 (Figure 1G) can be determined empirically. In some configurations, the radius around each SDSF point 10766 (Figure 1G) can include a range of 0.1m to 1.0m. The height change between one point and another on the SDSF orbit 10377 (Figure 1G) with respect to the SDSF at the average point 10765 (Figure 1G) can be calculated. Connecting the averaged donuts 10375 together can generate the SDSF orbit 10377 (Figures 1G and 1K). When creating the SDSF trajectory 10377 (Figures 1G and 1K), if two next candidate points exist within the search radius of the starting point, the next point can be selected based on the fact that it forms a line as straight as possible between the previous line segment, the starting point, and the candidate destination point, and then the candidate next point represents the smallest change in SDSF height between the previous point and the candidate next point. In some configurations, the SDSF height can be defined as the difference between the heights of the upper donut 10351 (Figure 1J) and the lower donut 10353 (Figure 1J).

[0130] Here, mainly referring to FIG. 1L, combining the concave polygon and the SDSF line 10165 (FIG. 1D) can generate a data set including the polygon 10139 (FIG. 1D) and the SDSF 10141 (FIG. 1D), and the data set can be operated to generate a graphed polygon using the SDSF data. Operating the concave polygon 10263 can include, but is not limited to, merging the concave polygon 10263 to form the merged polygon 10771. Merging the concave polygon 10263 can be accomplished using known methods such as, for example, but not limited to, those found at (http: / / www.angusj.com / delphi / clipper.php). The merged polygon 10771 can be extended to smooth the edges and form the extended polygon 10772. The extended polygon 10772 can be shrunk to provide a running margin and form the shrunk polygon 10774, to which the SDSF track 10377 (FIG. 1M) can be added. Inward trimming (shrinking) can ensure that there is room for the AV 10101 (FIG. 1A) to proceed near the edge by reducing the size of the drivable surface by a preselected amount based at least on the size of the AV 10101 (FIG. 1A). Polygon expansion and contraction can be accomplished using commercially available technologies such as, for example, but not limited to, the ARCGIS (registered trademark) clip command (http: / / desktop.arcgis.com / en / arcmap / 10.3 / manage-data / editing-existing-features / clipping-a-polygon-feature.htm).

[0131] Here, primarily referring to Figure 1M, the contracted polygon 10774 can be partitioned into polygons 10778, each of which can be traversed without encountering non-traversable surfaces. The contracted polygon 10774 can be partitioned by conventional means such as ear slicing, which is optimized, for example, by z-order curve hashing and extended to handle holes, torsional polygons, degeneracy, and self-intersections. Commercially available ear slicing implementations can include, for example, those found at (https: / / github.com / mapbox / earcut.hpp). The SDSF trajectory 10377 can include an SDSF point 10789 (Figure 1N) which can be connected to polygon vertex 10781. Vertices 10781 can be considered possible path points which can be connected to each other to form possible travel paths for AV 10101 (Figure 1A). In the dataset, SDSF point 10789 (Figure 1N) can be labeled in this manner. As partitioning proceeds, it is possible, but not limited to, that redundant edges such as edges 10777 and 10779 may be introduced. Removing either edge 10777 or 10779 can reduce the complexity of further analysis and allow for the retention of the polygonal mesh. In some configurations, the Hertel-Mehlhorn polygonal partitioning algorithm can be used to remove edges and omit edges labeled as features. The set of polygons 10778, including the labeled features, can undergo further simplification to reduce the number of possible path points, which can be provided to the device controller 10111 (Figure 1A) in the form of annotated point data 10379 (Figure 5B), which can be used to incorporate the occupied grid.

[0132] Referring here to Figures 2A-2B, the sensor data collected by the AV can also be used to capture the occupied grid. The processor in the AV can receive data from sensors in the long-range sensor assembly 20400 mounted on top of the cargo container 20110, and from short-range sensors 20510, 20520, 20530, 20540 located within the cargo platform 20160, as well as from other sensors. In addition, the processor may receive data from an optional short-range sensor 20505 mounted near the top of the front of the cargo container 20110. The processor may also receive data from one or more antennas 20122A, 20122B (Figure 1C), including cellular, WiFi, and / or GPS. In one embodiment, the AV 20100 has a GPS antenna 20122A (Figure 1C) located on top of the long-range sensor assembly 20400 and / or an antenna 20122B (Figure 1C) located on top of the cargo container 20110. The processor may be located anywhere within the AV20100. In some embodiments, one or more processors are located within the long-range sensor assembly 20400. An additional processor may be located within the cargo platform 20160. In other embodiments, the processor may be located within the cargo container 20110 and / or as part of the power base 20170.

[0133] Continuing to refer to Figures 2A-2B, the long-range sensor assembly 20400 is mounted on top of a cargo container and provides an improved view of the environment surrounding the AV. In one embodiment, the long-range sensor assembly 20400 is positioned above 1.2 m feet from the surface of travel or the ground. In other embodiments, if the cargo container is higher, or if the power base configuration raises the cargo platform 20160, the long-range sensor assembly 20400 may be positioned 1.8 m from the ground over which the AV is traveling. The long-range sensor assembly 20400 provides information about the environment surrounding the AV from a minimum distance to a maximum range. The minimum distance may be defined by the relative positions of the long-range sensor 20400 and the cargo container 20110. The minimum distance may further be defined by the field of view (FOV) of the sensor. The maximum distance may be defined by the range of the long-range sensor within the long-range sensor assembly 20400 and / or by the processor. In one embodiment, the range of the long-range sensor is limited to 20 m. In one embodiment, the Velodyne Puck LIDAR has a range of up to 100m. The long-range sensor assembly 20400 may provide data on objects in all directions. The sensor assembly may provide information on structures, surfaces, and obstacles across a 360° angle around AV20100.

[0134] Continuing to refer to Figure 2A, the three long-range cameras observed through windows 20434, 20436, and 20438 can provide horizontal FOVs 20410, 20412, and 20414, all of which provide a 360° FOV. The horizontal FOV may be defined by the selected camera and the camera's location within the long-range camera assembly 20400. In describing the field of view, the zero angle is the ray located in a vertical plane perpendicular to the front of the AV, passing through the center of the AV20100. The zero-angle ray passes through the front of the AV. The front long-range camera viewed through window 20434 has a 96° FOV 20410 with a range of 311° to 47°. The left long-range camera viewed through window 20436 has a FOV 20412 with a range of 47° to 180°. The right-side long-range camera, viewed through window 20438, has an FOV of 180° to 311°. The long-range sensor assembly 20400 may also include an industrial camera positioned to observe through window 20432, which provides more detailed information about objects and surfaces in front of AV20100 than the long-range camera. The industrial camera located behind window 20432 may have an FOV of 20416, defined by the selected camera and the camera's location within the long-range camera assembly 20400. In one embodiment, the industrial camera behind window 20432 has an FOV of 23° to 337°.

[0135] Referring here to Figure 2B, the LIDAR20420 provides a 360° horizontal FOV around the AV20100. The vertical FOV may be limited by the LIDAR instrument. In one embodiment, the vertical FOV of the LIDAR20418 is 40° and mounted 1.2m to 1.8m above the ground, setting the minimum sensor distance from the AV20100 at 3.3m to 5m.

[0136] Referring here to Figures 2C and 2D, the long-range sensor assembly 20400 is shown together with the cover 20430. The cover 20430 includes windows 20434, 20432, and 20436 through which the long-range camera and industrial camera observe the environment around AV20100. The cover 20430 for the long-range sensor assembly 20400 is sealed from the weather by an O-ring between the cover 20430 and the top of the cargo container 20110.

[0137] Referring here to Figures 2E and 2F, cover 20430 has been removed to expose the camera and processor embodiment. LiDAR sensor 20420 provides data on range or distance to surfaces around the AV. This data may be provided to processor 20470, located within long-range sensor assembly 20400. The LiDAR is mounted on long-range cameras 20440A-C and structure 20405 above cover 20430. LiDAR sensor 20420 is one embodiment of a ranging sensor based on reflected laser pulsed light. Other ranging sensors, such as radar, which use reflected radio waves, can also be used. In one embodiment, LiDAR sensor 20420 is a Puck sensor by VELODYNE LIDAR® (San Jose, CA). Three long-range cameras 20440A, 20440B, and 20440C provide digital images of objects, surfaces, and structures around the AV20100. Three long-range cameras 20440A, 20440B, and 20440C are arranged around structure 20405 relative to cover 20430 to provide three horizontal FOVs covering the entire 360° around the AV. The long-range cameras 20440A, 20440B, and 20440C are mounted on the lifting ring structure 20405, which is mounted on cargo container 20110. The long-range cameras 20440A, 20440B, and 20440C receive images through windows 20434, 20436, and 20438 mounted within cover 20430. The long-range cameras may comprise a camera and lens on a printed circuit board (PCB).

[0138] Referring here to Figure 2F, one embodiment of the long-range camera 20440A may include a digital camera 20444 with a fisheye lens 20442 mounted on the front of the digital camera 20444. The fisheye lens 20442 can greatly widen the camera's field of view. In one embodiment, the fisheye lens expands the field of view to 180°. In one embodiment, the digital camera 20444 is similar to the e-cam52A_56540_MOD by E-con Systems (San Jose, CA). In one embodiment, the fisheye lens 20442 is similar to the Model DSL227 by Sunex (Carlsbad, CA).

[0139] Continuing to refer to Figure 2F, the long-range sensor assembly 20400 may also include an industrial camera 20450 that receives visual data through a window 20432 within the cover 20430. The industrial camera 20450 provides the processor 20470 with additional data regarding objects, surfaces, and structures in front of the AV. The camera may be similar to a Kowa industrial camera (part number LM6HC). The industrial camera 20450 and the long-range cameras 20440A-C are positioned 1.2m to 1.8m above the surface over which the AV 20100 is moving.

[0140] Continuing to refer to Figure 2F, mounting the long-range sensor assembly 20400 on top of a cargo container offers at least two advantages. The field of view for the long-range sensors, including the long-range cameras 20440A-C, the industrial camera 20450, and the LIDAR 20420, is often less obstructed by nearby objects such as people, cars, and low walls when the sensors are mounted higher above the ground. In addition, pedestrian walkways are designed to provide visual cues for people to perceive, including signs, fence heights, etc., and the typical eye level is in the range of 1.2m to 1.8m. Mounting the long-range sensor assembly 20400 on top of a cargo container places the long-range cameras 20440A-C and 20450 at the same level as signs and on visual cues directed towards pedestrians. The long-range sensors are mounted on a structure 20405 that provides a substantially rigid mount to withstand the deflection caused by the movement of AV20100.

[0141] Referring again to Figures 2E and 2F, the long-range sensor assembly may include an inertial measurement unit (IMU) and one or more processors that receive data from the long-range sensor and output the processed data to other processors for navigation. IMU 20460 with a vertical reference unit (VRU) is mounted on structure 20405. IMU / VRU 20460 may be positioned directly below LIDAR 20420 to provide position data related to LIDAR 20420. Position and orientation from IMU / VRU 20460 may be combined with data from other long-range sensors. In one embodiment, IMU / VRU 20460 is model MTi 20 supplied by Xsens Technologies (The Netherlands). One or more processors may include processor 20465 that receives data from at least industrial camera 20450. In addition, the processor 20470 may receive data from at least one of the following: LIDAR 20420, long-range cameras 20440A-C, industrial camera 20450, and IMU / VRU 20460. The processor 20470 may be cooled by a liquid-cooled heat exchanger 20475 connected to a circulating coolant system.

[0142] Referring here to Figure 2G, AV20100 may include several short-range sensors that detect the travel surface and obstacles within a predetermined distance from AV. Short-range sensors 20510, 20520, 20530, 20540, 20550, and 20560 are located above the perimeter of container platform 20160. These sensors are located below cargo container 20110 (Figure 2B) and are closer to the ground than the long-range sensor assembly 20400 (Figure 2C). Short-range sensors 20510, 20520, 20530, 20540, 20550, and 20560 are angled downward to provide an FOV that captures surfaces and objects that cannot be seen by the sensors in the long-range sensor assembly 20400 (Figure 2C). The field of view of sensors located closer to the ground and angled downward is less likely to be obstructed by nearby objects and pedestrians than sensors mounted further from the ground. In one embodiment, the short-range sensor provides information about the ground surface and objects up to 4m from the AV20100.

[0143] Referring again to Figure 2B, the vertical FOVs for two of the short-range sensors are shown in the side view of AV20100. The vertical FOV 20542 of the rear-facing sensor 20540 is centered on the centerline 20544. The centerline 20544 is angled downwards from the top surface of the cargo platform 20160. In one embodiment, sensor 20540 has a vertical FOV of 42° and a centerline 20546 angled 22° to 28° downwards from the plane 20547 defined by the top plate of the cargo platform 20160. In one embodiment, the short-range sensors 20510 and 20540 are located approximately 0.55m to 0.71m above the ground. The resulting FOVs 20512 and 20542 cover the ground from 0.4m to 4.2m above the AV. The short-range sensors 20510, 20520, 20530, 20550 (Figure 2G), and 20560 (Figure 2G), mounted on the cargo base 20160, have similar vertical field of view and centerline angles relative to the top of the cargo platform. The short-range sensors mounted on the cargo platform 20160 can perceive the ground from 0.4 to 4.7 meters from the outer edge of the AV20100.

[0144] Continuing to refer to Figure 2B, the short-range sensor 20505 may be mounted on the front of the cargo container 20110 in the vicinity. In one embodiment, the sensor 20505 may provide an additional view of the ground in front of the AV to the view provided by the short-range sensor 20510. In another embodiment, the sensor 20505 may provide a view of the ground in front of the AV instead of the view provided by the short-range sensor 20510. In one embodiment, the short-range sensor 20505 may have a vertical FOV of 42°, and the angle of its centerline relative to the top of the cargo platform 20160 is 39°. The resulting view of the ground extends from 0.7m to 3.75m from the AV.

[0145] Referring again to Figure 2G, the horizontal FOVs of the short-range sensors 20510, 20520, 20530, 20540, 20550, and 20560 cover all directions around the AV20100. The horizontal FOVs of adjacent sensors such as 20520 and 20530, 20522 and 20532, overlap at a certain distance from the AV20100. In one embodiment, the horizontal FOVs of adjacent sensors 20520, 20530, 20560, and 20550, 20522, 20532, and 20562 and 20552, overlap at 0.5 to 2 meters from the AV. The short-range sensors are distributed around the cargo base 20160, have a horizontal field of view, are installed at specific angles, and provide nearly complete visual coverage of the ground surrounding the AV. In one embodiment, the short-range sensor has a horizontal FOV of 69°. The front sensor 20510 faces forward at zero angle to the AV and has an FOV of 20512. In one embodiment, the two front corner sensors 20520, 20560 are angled such that their centerlines are at an angle of 65° 20564. In one embodiment, the rear sensors 20530, 20550 are angled such that the centerlines of 20530 and 20560 are at an angle of 110° 20534. In some configurations, it is also possible to consider a number of other sensors with other horizontal FOVs mounted around the cargo base 20160 to provide a nearly complete view of the ground around the AV 20100.

[0146] Referring to Figure 2H, the short-range sensors 20510, 20520, 20530, 20540, 20550, and 20560 are located on the periphery of the cargo base 20160. The short-range camera is mounted within a projection that sets the angle and position of the short-range camera. In another configuration, the sensors are mounted and positioned on the interior of the cargo base and receive visual data through a window that is aligned with the outer casing of the cargo base 20160.

[0147] Referring here to Figures 2I and 2J, the short-range sensor 20600 is mounted within an outer panel element 20516 of the cargo base 20160 and may include a liquid cooling system. The outer panel element 20516 includes a formed projection 20514 that holds the short-range sensor assembly 20600 at a predetermined location and vertical angle to the top of the cargo base 20160 and at a certain angle to the front of the cargo base 20160. In some configurations, the short-range sensor 20510 is angled downward by 28° relative to the cargo platform 20160, the short-range sensors 20520 and 20560 are angled 18° downward and 25° forward, the short-range sensors 20530 and 20550 are angled 34° downward and 20° backward, and the short-range sensor 20540 is angled downward by 28° relative to the cargo platform 20160. The outer panel element 20516 includes a cavity 20517 for receiving the camera assembly 20600. The outer panel element 20516 may also include a plurality of elements 20518 for receiving mechanical fasteners, including, but not limited to, rivets, screws, and buttons. Alternatively, the camera assembly may be mounted using adhesive or held in place using clips fastened to the outer panel element 20516. A gasket 20519 can provide a seal to the front of the camera 20610.

[0148] Referring here to Figures 2K and 2L, the short-range sensor assembly 20600 comprises a short-range sensor 20610 mounted on a bracket 20622 which is attached to a water-cooled plate 20626. The outer case 20612, transparent cover 20614, and heat sink 20618 are partially removed in Figures 2K and 2L to better visualize the sensor block 20616 and electronic block 20620, which are heat dissipation elements for the short-range sensor 20610. The short-range sensor assembly 20600 may include one or more thermoelectric coolers (TECs) 20630 between the bracket 20622 and the water-cooled plate 20626. The water-cooled plate 20626 is cooled by a coolant that is pumped through 20628 which is thermally connected to the plate 20626. The TEC is an electrically powered element with first and second sides. The electric TEC cools the first side while rejecting the thermal energy removed from the first side plus electrical energy on the second side. In the short-range sensor assembly 20600, the TEC 20630 cools the bracket 20622 and transfers the thermal cooling energy plus electrical energy to the water-cooled plate 20626. Alternatively, the TEC 20630 can be used to actively control the temperature of the camera 20600 by varying the magnitude and polarity of the voltage supplied to the TEC 20630.

[0149] Operating the TEC20630 in cooling mode allows the short-range sensor 20610 to operate at temperatures below the coolant temperature. Bracket 20622 is thermally connected to the short-range sensor 20610 in two locations to maximize cooling of the sensor block 20616 and the electronics block 20620. Bracket 20622 includes tabs 20624, which are thermally attached to the heatsink 20618 via screws 20625. The heatsink 20618 is thermally connected to the sensor block 20616. The bracket is therefore thermally connected to the sensor block 20616 via the heatsink 20618, screws 20625, and tabs 20624. Bracket 20622 is also mechanically attached to the electronics block 20620, providing direct cooling of the electronics block 20620. Bracket 20622 may include, but is not limited to, multiple mechanical attachments, including screws and rivets that engage with element 20518 in Figure 2J. Short-range sensor 20610 may incorporate, but is not limited to, one or more sensors, including cameras, stereo cameras, ultrasonic sensors, short-range radars, and infrared projectors and CMOS sensors. One exemplary short-range sensor is similar to the RealSense depth camera D435 by Intel (Santa Clara, California), which includes an IR projector, two imager chips, and an RGB camera.

[0150] Referring here to Figure 2M-2O, another embodiment of the AV20100 is shown. The AV20100A includes a cargo container 20110 mounted on a cargo platform 20160 and a power base 20170. The AV20100A includes a plurality of long-range and short-range sensors. The primary long-range sensor is mounted in a sensor pylon 20400A on the cargo container 20110. The sensor pylon may include a LIDAR 20420 and a plurality of long-range cameras (not shown) oriented in a divergent direction to provide a wide field of view. In some configurations, the LIDAR 20420 can be used, for example, as described elsewhere herein, to provide point cloud data that may enable the capture of an occupied grid, identify landmarks, locate the AV20100 in its environment, and / or provide information for determining navigable space. In some configurations, a long-range camera from Leopard Imaging Inc. can be used to identify landmarks, locate the AV20100 within its environment, and / or determine navigable space.

[0151] Continuing to refer to Figure 2M-2O, the short-range sensors are primarily mounted within the cargo platform 20160 and provide information about obstacles in the vicinity of AV20100A. In some embodiments, the short-range sensors provide data about obstacles and surfaces within 4m of AV20100A. In some configurations, the short-range sensors provide information up to 10m from AV20100A. At least partially, multiple forward-facing cameras are mounted within the cargo platform 20160. In some configurations, the multiple cameras may include three cameras.

[0152] Referring here to Figure 2O, the top cover 20830 is partially cut away to expose the sub-roof 20810. The sub-roof 20810 provides a single component on which multiple antennas 20820 can be mounted. In one embodiment, ten antennas 20820 are mounted on the sub-roof 20810. Furthermore, for example, there are four cellular communication channels, each having two antennas, and two WiFi antennas. The antennas are wired as primary and auxiliary antennas for cellular transmission and reception. The auxiliary antennas can improve cellular functionality in several ways, including, but not limited to, reducing interference and achieving 4G LTE connectivity. The sub-roof 20810 and the top cover 20830 are non-metallic. The sub-roof 20810 is a plastic surface within 10-20 mm of the top cover 20830, which is non-structural and allows the antennas to be connected to the processor before the top cover 20830 is installed. Antenna connections are often high impedance and sensitive to dirt, grease, and misuse. Mounting and connecting the antenna to the sub-roof 20810 allows the top cover to be installed and removed without touching the antenna connections. Maintenance and repair operations may include removing the top cover without removing the sub-roof or disconnecting the antenna. Separate antenna assembly from the installation of the top cover 20830 facilitates testing / repair. The top cover 20830 is weatherproof and prevents water and dust from entering the cargo container 20110. Mounting the antenna on the sub-roof minimizes the number of openings on the top cover 20830.

[0153] Referring here to Figures 2P, 2Q, and 2R, another embodiment of the Long Range Sensor Assembly (LRSA) 20400A mounted on a cargo container (not shown) is shown. The LRSA may include a LiDAR and multiple long-range cameras, mounted at different positions on the LRSA structure 20950 to provide a panoramic view of the environment of the AV20100A. The LiDAR 20420 is mounted on top of the LRSA structure 20950 to provide an uninterrupted view. The LiDAR may include a VELODYNE LiDAR. Multiple long-range cameras 20910A-20910D are mounted on the LRSA structure 20950 on the next level below the LiDAR 20420. In one embodiment, four cameras are mounted one at each 90° interval around the structure to provide four views of the environment around the AV20100. In some embodiments, the four views will overlap. In some embodiments, each camera is either aligned with the direction of motion or perpendicular to the direction of motion. In one embodiment, one camera aligns with each of the main faces of the AV20100A, namely the front, rear, left, and right sides. In one embodiment, the long-range camera is a model LI-AR01 44-MIPI-M12 manufactured by Leopard Imaging Inc. The long-range camera may have a MIPI CSI-2 interface to provide high-speed data transfer to the processor. The long-range camera may have a horizontal field of view of 50°–70° and a vertical field of view of 30°–40°.

[0154] Referring here to Figures 2S and 2T, the long-range processor 20940 is located on the LRSA structure 20950 below the long-range cameras 20910A-20910D and the LIDAR 20420. The long-range processor 20940 receives data from the long-range cameras and LIDAR. The long-range processor communicates with one or more processors located elsewhere in the AV20100A. The long-range processor 20940 provides the data derived from the long-range cameras and LIDAR to one or more processors located elsewhere on the AV20100A, as described elsewhere in this specification. The long-range processor 20940 may be liquid-cooled by a cooler 20930. The cooler 20930 may be mounted in the structure below the long-range cameras and LIDAR. The cooler 20930 may provide a mounting location for the long-range processor 20940. The cooler 20930 is described in U.S. Patent Application No. 16 / 883,668 (Patent Attorney No. AA280), filed on 26 May 2020 and titled “Apparatus for Electronic Cooling on an Autonomous Device” (incorporated herein by reference as a whole). The cooler is provided with a liquid supply conduit and a return conduit for supplying a cooling liquid to the cooler 20930.

[0155] Referring again to Figures 2M and 2N, the short-range camera assembly 20740A-C is mounted on the front of the container platform 20160 and is angled to collect information about the surface underway and obstacles, steps, curbs, and other substantial discontinuous surface features (SDSFs). The camera assembly 20740A-C includes one or more LED lights to illuminate the surface underway, objects on the ground, and SDSFs.

[0156] Referring here to Figure 2U-2X, camera assemblies 20740A-B include a light 20732 to illuminate the ground and objects and provide improved image data from camera 20732. Note that camera assembly 20740A is a mirror image of 20740C, and the description of 20740A implicitly applies to 20740C. Camera 20732 may include a single-focus camera, a stereo camera, and / or an infrared projector and a CMOS sensor. One embodiment of the camera is the Real-Sense Depth D435 camera by Intel (Santa Clara, CA), which includes an IR projector, two imager chips with lenses, and an RGB camera. The LED light 20734 may be used at night or in low-light conditions, or may be used continuously to improve image data. One operating principle is that the light creates contrast by illuminating the projection surface and creating shadows in recesses. In one embodiment, the LED light may be a white LED. In one embodiment, the LED light 20734 is an Xlamp XHP50 from Cree Inc. In another embodiment, the LED light may emit infrared light to provide illumination for the camera 20372 without distracting or interfering with nearby pedestrians or drivers.

[0157] Continuing to refer to Figure 2U-2X, the placement and angle of light 20374 and the shape of covers 20736A, 20736B prevent camera 20732 from viewing light 20374. The angle and placement of light 20374 and covers 20736A, 20736B prevent the light from interfering with the driver or obstructing pedestrians. It is advantageous that camera 20732 is not exposed to light 20734 to prevent the sensor in camera 20732 from being blinded by light 20734 and thus prevented from detecting weaker light signals from the ground and objects in front of and to the sides of AV20100A. Camera 20732 and / or light 20734 may be cooled using a liquid flowing in and out of the camera assembly through port 20736.

[0158] Referring here to Figures 2W and 2X, the short-range camera assembly 20740A includes an ultrasonic or sonar short-range sensor 20730A. The second short-range camera assembly 20740C also includes an ultrasonic short-range sensor 20730B (Figure 2N).

[0159] Referring here to Figure 2Y, the ultrasonic sensor 20730A is mounted above the camera 20732. In one embodiment, the centerline of the ultrasonic sensor 20730A is parallel to the base of the cargo container 20110, which often means that the sensor 20730A is horizontal. The sensor 20730A is angled 45° from a forward-facing position. The cover 20376A provides a horn 20746 for directing the ultrasonic waves emerging from and received by the ultrasonic sensor 20730A.

[0160] Continuing to refer to Figure 2Y, cross-sections of camera 20732 and light 20734 within camera assembly 20740A illustrate the angles and openings in cover 20736. The cameras in the short-range camera assembly are angled downward to better image the ground in front of and to the sides of the AV. The center camera assembly 20740B is oriented directly forward in the horizontal plane. The corner camera assemblies 20740A and 20740C are angled 25° to their respective sides relative to directly forward in the horizontal plane. Camera 20732 is angled 20° downward relative to the top of the cargo platform in the vertical plane. Since the AV generally keeps the cargo platform horizontal, the cameras are therefore angled 20° downward from the horizontal. Similarly, the center camera assembly 20740B (Figure 2M) is angled 28° downward from the horizontal. In some embodiments, the cameras in the camera assemblies may be angled downward by 25° to 35°. In another embodiment, the camera within the camera assembly may be angled downward by only 15° to 45°. The LED light 20734 is similarly angled downward to illuminate the ground imaged by the camera 20732 and to minimize distraction to pedestrians. In one embodiment, the LED light centerline 20742 is parallel to the camera centerline 20738 within 5°. Cover 20736A protects both the camera 20732 and pedestrians from the bright light of the LED 20734 within the camera assemblies 20740A-C. The cover, which isolates the light emitted by the LED, also provides a gradually widening opening 20737 to maximize the field of view of the camera 20732. The light is embedded at least 4 mm from the opening in the cover. The light opening is defined by the upper wall 20739 and the lower wall 20744. The upper wall 20739 is approximately parallel (±5°) to the centerline 2074. The lower wall 20744 is gradually widened by approximately 18° from the centerline 20742 to maximize illumination of the ground and objects near the ground.

[0161] Referring here to Figure 2Z-2AA, in one configuration, each light 20734 contains two LEDs 20734A under a square lens 20734B to produce a beam of light. The LEDs / lenses are angled and positioned relative to camera 20372 to illuminate the camera's field of view with minimal light spill outside the camera's FOV. The two LEDs / lenses are mounted together on a single PCB 20752 with a defined angle 20762 between the two lights. In another configuration, the two LEDs / lenses are mounted individually on heatsinks 20626A on separate PCBs at a certain angle to each other. In one embodiment, the lights are Xlamp XHP50 from Cree, Inc., and the lenses are 60° lenses HB-SQ-W from LEDil. The lights are angled approximately 50° to each other, and therefore the angle 20762 between the fronts of the lenses is 130°. The light is approximately 18mm from the front of camera 20732. ± It is located approximately 5mm (20764) rearward and 30mm (20766) below the center line of camera 20732.

[0162] Referring here to Figure 2AA-2BB, camera 20732 is cooled by a thermoelectric cooler (TEC) 20630, which, along with light 20734, is cooled by a liquid coolant flowing through cryogenic block 20626A. The camera is mounted to bracket 20622 via screws 20625 that are screwed into the sensor block portion of the camera, while the rear of bracket 20622 is bolted to the electronics block of the camera. Bracket 20622 is cooled by two TECs to maintain the performance of the IR imaging chip (CMOS chip) in camera 20732. The TECs displace the heat from bracket 20622 and the power they draw in to cryogenic block 20626A.

[0163] Referring here to Figure 2BB, the coolant is directed through a U-shaped path created by the central fin 20626D. The coolant flows directly behind the LED / lens / PCB of light 20734. Fins 20626B and 20626C improve heat transfer from light 20734 to the coolant. The coolant flows upward and passes alongside the hot side of TEC20630. The fluid path is created by plate 20737 (Figure 2X), which is attached to the rear of the cold block 20626A.

[0164] Referring here to Figure 3A, sensor data and map data can be used to update the occupied grid. The system and method of this teaching can manage a global occupied grid for a device autonomously navigating with respect to a grid map. The grid map may include routes or paths that the device can take from a starting point to a destination. The global occupied grid may include free-space indications that indicate where it is safe for the device to navigate. Possible routes and free-space indications can be combined on the global occupied grid to establish the optimal path that the device can take to safely reach its destination.

[0165] Continuing to refer to Figure 3A, as the device moves, a global occupancy grid, which will be used to determine an unobstructed navigation route, can be accessed based on the device's location, and the global occupancy grid can be updated as the device moves. The update can be based on at least the current value associated with the global occupancy grid at the device's location, a static occupancy grid which may contain historical information about the neighborhood the device is navigating, and data collected by sensors as the device moves. Sensors can be located on the device as described herein, and they can be located elsewhere.

[0166] Continuing to refer to Figure 3A, the global occupancy grid can contain cells, and each cell can be associated with an occupancy probability value. Each cell in the global occupancy grid can be associated with information such as the characteristics and discontinuities of the surrounding traversal surface at that location, as well as prior occupancy data associated with that location, such as whether an obstacle has been identified at the cell's location, determined from previously collected data and from data collected as the device navigates. Data captured as the device navigates can be stored in a local occupancy grid, whose center is the device. When updating the global occupancy grid, static, previously collected data can be combined with the local and global occupancy grid data determined in the previous update to create a new global occupancy grid with spaces occupied by the device that are marked as unoccupied. In some configurations, a Bayesian method can be used to update the global occupancy grid. This method may include, for each cell in the local occupancy grid, calculating the cell's position on the global occupancy grid, accessing the value at that position from the current global occupancy grid, accessing the value at that position from the static occupancy grid, accessing the value at that position from the local occupancy grid, and calculating a new value at that position on the global occupancy grid as a function of the current value from the global occupancy grid, the value from the static occupancy grid, and the value from the local occupancy grid. In some configurations, the relationship used to calculate the new value may include subtracting the current value from the sum of the static value and the local occupancy grid value. In some configurations, the new value may be limited by a pre-selected value, for example, based on computational limits.

[0167] Continuing to refer to Figure 3A, the system 30100 of this instruction can manage a global occupancy grid. The global occupancy grid can start with initial data and can be updated as the device moves. Creating the initial global occupancy grid may include a first process, and updating the global occupancy grid may include a second process. The system 30100 may include a global occupancy server 30121 that can receive information from various sources and update the global occupancy grid 30505 based on at least the information. The information may be supplied by sensors located on and / or elsewhere, static information, and navigation information, for example, but not limited to. In some configurations, the sensors may include cameras and radar, for example, that can detect surface characteristics and obstacles. The sensors may be positioned on the device to provide sufficient peripheral coverage to enable safe movement by the device, for example. In some configurations, the LIDAR 30103 can provide LIDAR point cloud (PC) data 30201, which may enable the acquisition of a locally occupied grid using LIDAR free-space information 30213. In some configurations, a conventional ground detection inverse sensor model (ISM) 30113 can process the LIDAR PC data 30201 and generate LIDAR free-space information 30213.

[0168] Continuing to refer to Figure 3A, in some configurations, the RGB-D camera 30101 can provide RGB-D PC data 30202 and RGB camera data 30203. The RGB-D PC data 30202 can capture a locally occupied grid using depth free-space information 30209, and the RGB-D camera data 30203 can capture a locally occupied grid using surface data 30211. In some configurations, the RGB-D PC data 30202 can be processed by, for example, a conventional stereo free-space ISM 30109, and the RGB-D camera data 30203 can be fed into, for example, a conventional surface detection neural network 30111, for example. In some configurations, the RGB MIPI camera 30105 can provide RGB data 30205, which can be combined with LIDAR PC data 30201 to generate a locally occupied grid with LIDAR / MIPI free-space information 30215. In some configurations, RGB data 30205 can be fed into a conventional free-space neural network 30115, whose output, along with LIDAR PC data 30201, can receive a pre-selected mask 30221 that can identify the most accuracy-critical portion of the RGB data 30205 before being fed into a conventional 2D-3D alignment 30117. The 2D-3D alignment 30117 can project the image from the RGB data 30205 onto the LIDAR PC data 30201. In some configurations, the 2D-3D alignment 30117 is not required. Any combination of sensors and methods for processing sensor data can be used to aggregate data and update the global occupancy grid. Any number of free-space estimation procedures can be used and combined to enable determination and verification of occupancy probabilities within the global occupancy grid.

[0169] Continuing to refer to Figure 3A, in some configurations, historical data can be provided by a repository 30107 of previously collected and processed data, which has information associated with, for example, the navigation area. In some configurations, the repository 30107 can include, for example, route information such as polygons 30207, but is not limited to. In some configurations, this data can be fed into a conventional polygon parser 30119, which can provide edges 30303, discontinuities 30503, and surfaces 30241 to a global occupancy grid server 30121. The global occupancy grid server 30121 can merge the local occupancy grid data collected by the sensors with the processed repository data to determine the global occupancy grid 30505. A grid map 30601 (Figure 3D) can be created from the global occupancy data.

[0170] Referring here to Figure 3B, in some configurations, the sensor may include a sonar 30141 that can provide a local occupied grid with sonar free space 30225 to a global occupied grid server 30121. Depth data 30209 can be processed by a conventional free-space ISM 30143. The local occupied grid with sonar free space 30225 can be merged with a local occupied grid with surface and discontinuity 30223, a local occupied grid with LIDAR free space 30213, a local occupied grid with LIDAR / MIPI free space 30215, a local occupied grid with stereo free space 30209, as well as edge 30303 (Figure 3F), discontinuity 30503 (Figure 3F), navigation point 30501 (Figure 3F), surface confidence 30513 (Figure 3F), and surface 30241 (Figure 3F) to form a global occupied grid 30505.

[0171] Referring here to Figures 3C-3F, in order to initialize the global occupied grid, the global occupied grid initializer 30200 may include the creation of a global occupied grid 30505 and a static grid 30249 by the global occupied grid server 30121. The global occupied grid 30505 can be created by fusing data from the local occupied grid 30118 with edges 30303, discontinuities 30503, and surfaces 30241 located within the area of ​​interest. The static grid 30249 (Figure 3D) can be created to include data such as, for example, surface data 30241, discontinuity data 30503, edges 30303, and polygons 30207, for example. The initial global occupancy grid 30505 can be calculated by adding occupancy probability data from the static grid 30249 (Figure 3E) to occupancy data derived from data collected from sensor 30107A, and subtracting occupancy data from the prior global occupancy grid 30505A (Figure 3F). The local occupancy grid 30118 may include, but are not limited to, local occupancy grid data from stereo free-space estimation 30209 (Figure 3B) via ISM, local occupancy grid data including surface / discontinuity detection results 30223 (Figure 3B), local occupancy grid data from LIDAR free-space estimation 30213 (Figure 3B) via ISM, and, in some configurations, local occupancy grid data from LIDAR / MIPI free-space estimation 30215 (Figure 3B) following 2D-3D alignment 30117 (Figure 3B). In some configurations, the local occupancy grid 30118 can include local occupancy grid data obtained from sonar free-space estimation 30225 via ISM. In some configurations, various local occupancy grids with free-space estimation can be merged into the local occupancy grid 30118 according to a pre-selected known process. From the global occupancy grid 30505, a grid map 30601 (Figure 3E) can be created, which may include occupancy and surface data of the device's vicinity.In some configurations, the grid map 30601 (Figure 3E) and the static grid 30249 (Figure 3D) can be exposed using, for example, a Robot Motion System (ROS) subscribe / publish feature, but are not limited to these.

[0172] Referring here to Figures 3G and 3H, in order to update the occupied grid when the device moves, the occupied grid update 30300 may include updating the local occupied grid with respect to data measured as the device is moving and combining that data with the static grid 30249. The static grid 30249 is accessed when the device moves out of the occupied grid range it is working with. The device can be positioned in occupied grid 30245A at a first location 30513A in a first time. When the device moves to a second location 30513B, the device is positioned in occupied grid 30245B from its new location and, possibly, with a set of values ​​derived from the values ​​in occupied grid 30245A. Data from the static grid 30249 and surface data from the initial global occupied grid 30505 (Figure 3C) that is spatially located to the cells in occupied grid 30245B in a second time can be used together with measured surface data and occupation probabilities to update each grid cell according to a pre-selected relationship. In some configurations, the relationship may include summing static data with measured data. The resulting occupied grid 30245C at a third time and third location 30513C can be made available to the movement manager 30123 to signal the navigation of the device.

[0173] Referring here to Figure 3I, method 30450 for creating and managing occupied grids may include, but are not limited to, converting sensor measurements into reference frames associated with the device by a local occupied grid creation node 30122 30451, creating a timestamped measurement occupied grid 30453, and exposing the timestamped measurement occupied grid as a local occupied grid 30234 (Figure 3G) 30455. A system associated with method 30450 may include multiple local grid creation nodes 30122, for example, one per sensor, so that multiple local occupied grids 30234 (Figure 3G) may be brought about. Sensors may include, but are not limited to, an RGB-D camera 30325 (Figure 3G), a LiDAR / MIPI 30231 (Figure 3G), and a LiDAR 30233 (Figure 3G). A system associated with method 30450 may include a global occupied grid server 30121 that can receive local occupied grids and process them according to method 30450. In particular, method 30450 may include loading a surface 30242, accessing surface discontinuities such as curbs 30504, and creating a static occupancy grid 30249 from any properties available in repository 30107, which may include surfaces and surface discontinuities 30248. Method 30450 may include receiving a published local occupancy grid 30456 and moving a global occupancy grid to keep the device at the center of the map 30457. Method 30450 may include setting a new area on the map with prior information from a static pre-occupancy grid 30249 30459 and marking the area currently occupied by the device as unoccupied 30461. Method 30450 may execute a loop 30463 for each cell in each local occupancy grid.Loop 30463 may include, but is not limited to, calculating the location of a cell on a global occupancy grid, accessing the previous value at that location on the global occupancy grid, and calculating a new value at the cell location based on the relationship between the previous value and the value at the cell in the local occupancy grid. The relationship may include, but is not limited to, summing the values. Loop 30463 may also include comparing the new value against a pre-selected range of acceptable probabilities and setting up the global occupancy grid with the new value. The comparison may include setting the probability to the minimum or maximum acceptable probability if the probability is lower or higher than the minimum or maximum acceptable probability. Method 30450 may include exposing the global occupancy grid 30467.

[0174] Referring here to Figure 3J, an alternative method 30150 for creating a global occupancy grid may include, but is not limited to, step 30151, if the device moves, accessing the occupancy probability value associated with the old map area (where the device was before it moved) and updating the global occupancy grid on the new map area (where the device was after it moved) using the value from the old map area 30153, accessing the drivable surface associated with the cells of the global occupancy grid in the new map area and updating the cells in the updated global occupancy grid using the drivable surface 30155, and proceeding to step 30159. In step 30151, if the device has not moved and the global occupancy grid is co-located with the local occupancy grid, method 30150 may include updating the global occupancy grid as possible using surface confidence associated with drivable surfaces from at least one local occupancy grid 30159, updating the global occupancy grid as possible using log odds of occupancy probability values ​​from at least one local occupancy grid, for example, but not limited to, using a Bayesian function 30161, and adjusting the log odds based on at least a characteristic associated with its location 30163. In step 30157, if the global occupancy grid is not co-located with the local occupancy grid, method 30150 may include returning to step 30151. The characteristic may include, but not limited to, setting the device's location as unoccupied.

[0175] Referring here to Figure 3K, in an alternative configuration, method 30250 for creating a global occupancy grid may include, but is not limited to, 30251, updating the global occupancy grid with information from a static grid associated with the new location of the device 30253 if the device moves 30251. Method 30250 may include analyzing the surface at the new location 30257. In 30259, if the surface is drivable, method 30250 may include updating the surface on the global occupancy grid 30261 and updating the global occupancy grid with values ​​from a repository of static values ​​associated with the new location on the map 30263.

[0176] Referring here to Figure 3L, updating the surface 30261 may include, but is not limited to, accessing the local occupied grid (LOG) for a particular sensor 30351. In 30353, if there are further cells to be processed within the local occupied grid, method 30261 may include accessing the surface classification confidence value and surface classification from the local occupied grid 30355. In 30357, if the surface classification of a cell in the local occupied grid is identical to the surface classification in the global occupied grid at the cell's location, method 30261 may include setting the new global occupied grid (GOG) surface confidence to the sum of the old global occupied grid surface confidence and the local occupied grid surface confidence 30461. In 30357, if the surface classification of a cell in the local occupied grid is not identical to the surface classification in the global occupied grid at the cell's location, method 30261 may include setting the new global occupied grid surface confidence to the difference between the old global occupied grid surface confidence and the local occupied grid surface confidence 30359. In 30463, if the new global occupancy grid surface confidence is less than zero, method 30261 may include setting the new global occupancy grid surface classification to the value of the local occupancy grid surface classification 30469.

[0177] Referring here to Figure 3M, updating the global occupancy grid using values ​​from a repository of static values ​​30263 can include, but is not limited to, the following: if, in 30361, there are further cells to be processed within the local occupancy grid, method 30263 may include accessing the log odds from the local occupancy grid 30363 and updating the log odds in the global occupancy grid using the values ​​from the local occupancy grid at that location 30365. If, in 30367, the greatest certainty that a cell is empty is met, and in 30369, the device is moving within a given lane barrier, and in 30371, the surface is traversable, method 30263 may include updating the probability that a cell is occupied 30373 and returning to continue processing further cells. If, in 30367, the maximum certainty that the cell is empty is not reached, or in 30369, the device is not moving within a given lane, or in 30371, the surface is not traversable in the mode the device is currently moving in, method 30263 may include going back to consider further cells without updating the log odds. If the device is in standard mode, i.e., a mode in which the device can navigate a relatively uniform surface, and surface classification indicates that the surface is not relatively uniform, method 30263 may adjust the path of the device by updating the log odds (30373) to increase the probability that the cell is occupied. If the device is in standard mode and surface classification indicates that the surface is relatively uniform, method 30263 may adjust the path of the device by updating the log odds (30373) to decrease the probability that the cell is occupied. When this device is operating in four-wheel mode, that is, in a mode in which the device can navigate uneven terrain, adjustment of the probability of a cell being occupied may not be necessary.

[0178] Referring here to Figure 4A, AV can proceed in specific modes that can be associated with device configurations, for example, the configuration depicted in device 42114A and the configuration depicted in device 42114B. A system of this teaching for real-time control of the configuration of the device, based on at least one environmental factor and the status of the device, may include, but is not limited to, a sensor, a means of transport, a chassis operably coupled to the sensor and the means of transport, the means of transport being driven by a motor and a power source, a device processor that receives data from the sensor, and a power base processor that controls the means of transport. In some configurations, the device processor may receive environmental data, determine environmental factors, determine configuration changes according to the environmental factors and the status of the device, and provide the configuration changes to the power base processor. The power base processor may issue commands to the means of transport to move the device from place to place and physically reconfigure the device when required by the road surface type.

[0179] Continuing with Figure 4A, sensors that collect environmental data may include, but are not limited to, cameras, LiDAR, radar, thermometers, pressure sensors, and weather condition sensors, some of which are described herein. From this set of data, the device processor can determine the environmental factors on which device configuration changes may be based. In some configurations, environmental factors may include, but are not limited to, surface factors such as surface type, surface features, and surface conditions. Based on the environmental factors and the current status of the device, the device processor can determine in real time how to modify the configuration to adapt to crossing detected surface types.

[0180] Continuing to refer to Figure 4A, in some configurations, configuration changes of device 42114A / B / C (collectively referred to as device 42114) may include, for example, configuration changes of the means of transport. Other configuration changes, such as user information displays and sensor control devices, which may depend on the current mode and surface type, are also conceivable. In some configurations, the means of transport may include at least four drive wheels 442101, two on each side of the chassis 42112, and at least two caster wheels 42103 operably coupled to the chassis 42112, as described herein. In some configurations, the drive wheels 442101 may be operably coupled in pairs 42105, each pair 42105 may include a first drive wheel 42101A and a second drive wheel 42101B of the four drive wheels 442101, with each pair 42105 located on opposite sides of the chassis 42112. The operable coupling may include a wheel cluster assembly 42110. In some configurations, a power base processor 41016 (Figure 4B) can control the rotation of the cluster assembly 42110. Left and right wheel motors 41017 (Figure 4B) can drive the wheels 442101 on either side of the chassis 42112. Direction changes can be performed by driving the left and right wheel motors 41017 (Figure 4B) at different rates. A cluster motor 41019 (Figure 4B) can rotate the wheelbase in the forward / rear direction. Wheelbase rotation can allow the cargo to rotate independently of the drive wheels 442101, if so, while the front drive wheel 442101A is higher or lower than the rear drive wheel 442101B, for example, when encountering a discontinuous surface feature. The cluster assembly 42110 allows each pair of the two wheels 42105 to operate independently, thereby providing forward, backward, and rotational motion of the device 42114 in response to commands. The cluster assembly 42110 can provide structural support for the pair 42105.The cluster assembly 42110 provides mechanical power to rotate the wheel drive assemblies together, enabling functions that depend on the rotation of the cluster assembly, such as, but not limited to, climbing discontinuous surface features, various surface types, and uneven terrain. Further details regarding the operation of the clustered wheels can be found in U.S. Patent Application No. 16,035,205 (Patent Attorney's Reference No. X80), filed on 13 July 2018 and titled “Mobility Device” (which is incorporated herein by reference in its entirety).

[0181] Continuing to refer to Figure 4A, the configuration of device 42114 can be associated with, but is not limited to, mode 41033 (Figure 4B) of device 42114. Device 42114 can operate in some of the modes 41033 (Figure 4B). In standard mode 10100-1 (Figure 5E), device 42114B can operate on two of the drive wheels 442101B and two of the caster wheels 42103. Standard mode 10100-1 (Figure 5E) can provide turning and mobility on relatively firm horizontal surfaces, such as, but is not limited to, indoor environments, sidewalks, and pavements. In reinforced mode 10100-2 (Figure 5E) or four-wheel mode, the device 42114A / C can command four of the drive wheels 442101A / B to be actively stabilized via onboard sensors, raising the chassis 42112, casters 42103, and cargo, as well as / or reorienting them. Four-wheel mode 10100-2 (Figure 5E) provides mobility in various environments, allowing the device 42114A / C to climb steep inclines and travel across soft, uneven terrain. In four-wheel mode 10100-2 (Figure 5E), all four of the drive wheels 442101A / B can be deployed, and the caster wheels 42103 can be retracted. Rotation of the cluster 42110 can enable operation on uneven terrain, and the drive wheels 442101A / B can travel over and across discontinuous surface features. This functionality allows device 42114A / C to provide mobility in a wide variety of outdoor environments. Device 42114B can operate on solid and stable but wet outdoor surfaces. Frost heave and other natural phenomena can degrade outdoor surfaces, causing cracks and loose material. In four-wheel mode 10100-2 (Figure 5E), device 42114A / C can operate on these degraded surfaces.Mode 41033 (Figure 4B) is described in detail in U.S. Patent No. 6,571,892 (No. '892), issued on 3 June 2003 and titled "Control System and Method" (which is incorporated herein by reference in its entirety).

[0182] Referring here to Figure 4B, system 41000 can drive device 42114 (Figure 4A) by processing input from sensor 41031, generating commands to wheel motor 41017 to drive wheel 442101 (Figure 4A), and generating commands to cluster motor 41019 to drive cluster 42110 (Figure 4A). System 41000 may include, but is not limited to, a device processor 41014 and a power base processor 41016. Device processor 41014 may receive and process environmental data 41022 from sensor 41031 and provide configuration information 40125 to power base processor 41016. In some configurations, device processor 41014 may include sensor processor 41021 which may receive and process environmental data 41022 from sensor 41031. Sensor 41031 may include, but is not limited to, a camera as described herein. From this data, information about the travel surface traversed by device 42114 (Figure 4A) can be stored and processed. In some configurations, the travel surface information can be processed in real time. The device processor 41014 may include a configuration processor 41023 that can determine the surface type 40121 from environmental data 41022 traversed by device 42114 (Figure 4A). The configuration processor 41023 may include a travel surface processor 41029 (Figure 4C) that can create a travel surface classification layer, a travel surface confidence layer, and an occupation layer from the environmental data 41022. This data can be used by the power base processor 41016 to create travel commands 40127 and motor commands 40128, as described herein, and by the global occupation grid processor 41025 to update the occupation grid that can be used for route planning. Configuration 40125 can be at least partially based on the surface type 40121.Surface type 40121 and mode 41033 can be used to determine occupied grid information 41022, which may at least partially include the probability that a cell in the occupied grid is occupied. The occupied grid can at least partially enable the determination of possible paths that device 42114 (Figure 4A) can take.

[0183] Continuing to refer to Figure 4B, the power base processor 41016 receives configuration information 40125 from the device processor 41014 and can process the configuration information 40125 together with other information, such as routing information. The power base processor 41016 may include a control processor 40325 that can create a move command 40127 based on at least the configuration information 40125 and provide the move command 40127 to the motor drive processor 40326. The motor drive processor 40326 can generate a motor command 40128 that can instruct and move the device 42114 (Figure 4A). Specifically, the motor drive processor 40326 can generate a motor command 40128 that can drive the wheel motor 41017 and a motor command 40128 that can drive the cluster motor 41019.

[0184] Referring here to Figure 4C, the real-time surface detection of this teaching may include a configuration processor 41023, which may include a traveling surface processor 41029, but is not limited to this. The traveling surface processor 41029 can determine the characteristics of the traveling surface on which device 42114 (Figure 4A) is navigating. The characteristics can be used to determine the future configuration of device 42114 (Figure 4A). The traveling surface processor 41029 may include, but is not limited to, a neural network processor 40207, data transformations 40215, 40219, and 40239, a layer processor 40241, and an occupied grid processor 40242. Together, these components can generate information that can instruct changes to the configuration of device 42114 (Figure 4A) and enable modifications to the occupied grid 40244 (Figure 4C) that can inform the path plan for the progress of device 42114 (Figure 4A).

[0185] Referring here to Figures 4C and 4D, the neural network processor 40207 can expose environmental data 41022 (Figure 4B) to a trained neural network that can indicate the type of surface that each point of data collected by sensor 41031 (Figure 4B) is likely to represent. The environmental data 41022 (Figure 4B) can, but are not limited to, be received as a camera image 40202, and the camera can be associated with camera properties 40204. The camera image 40202 can include a 2D grid of points 40201 having an X resolution 40205 (Figure 4D) and a Y resolution 40204 (Figure 4D). In some configurations, the camera image 40202 can include an RGB-D image, the X resolution 40205 (Figure 4D) can include 40,640 pixels, and the Y resolution 40204 (Figure 4D) can include 40,480 pixels. In some configurations, the camera image 40202 can be converted into an image formatted according to the requirements of the selected neural network. In some configurations, the data can be normalized, scaled, and converted from 2D to 1D, which can improve the processing efficiency of the neural network. The neural network can be trained in many ways, including, but not limited to, training with RGB-D camera images. In some configurations, the trained neural network, represented in the neural network file 40209 (Figure 4D), can be made available to the neural network processor 40207 through a direct connection to the processor running the trained neural network, or, for example, through a communication channel. In some configurations, the neural network processor 40207 can use the trained neural network file 40209 (Figure 4D) to identify surface types 40121 (Figure 4C) in the environmental data 41022 (Figure 4B). In some configurations, surface type 40121 (Figure 4C) may include, but is not limited to, non-drivable, hard-drivable, soft-drivable, and curb types.In some configurations, surface type 40121 (Figure 4C) can include, but are not limited to, impassable / background, asphalt, concrete, brick, compacted soil, wooden planks, gravel / pebbles, grass, mulch, sand, curb, solid metal, metal grid, tactile paving, snow / ice, and railway tracks. The result of the neural network processing can include a surface classification grid 40303 with points 40213 having an X resolution 40205, a Y resolution 40203, and a center 40211. Each point 40213 within the surface classification grid 40303 can be associated with a likelihood that it is one of the specific surface types 40121 (Figure 4C).

[0186] Continuing with Figures 4C and 4D, the traveling surface processor 41029 (Figure 4C) may include a 2D-to-3D transformation 40215 that can backproject a 2D surface classification grid 40303 (Figure 4D) in a 2D camera frame to a 3D image cube 40307 (Figure 4D) in 3D real-world coordinates as seen by the camera. The backprojection can restore the 3D properties of the 2D data and can transform a 2D image from an RGB-D camera into a 3D camera frame 40305 (Figure 4C). Each point 40233 (Figure 4D) in the cube 40307 (Figure 4D) can be associated with a likelihood, depth coordinate, and X / Y coordinate, respectively, which are specific to one of the surface types 40121 (Figure 4C). The dimensions of the point cube 40307 (Figure 4D) can be delimited, for example, according to camera properties 40204 (Figure 4C), such as focal length x, focal length y, and projection center 40225, although this is not limited to these properties. For example, camera properties 40204 may include the maximum range over which the camera can reliably project. Furthermore, there may be features of device 42114 (Figure 4A) that may interfere with the image 40202. For example, the caster 42103 (Figure 4A) may interfere with the view of camera 40227 (Figure 4D). These factors can limit the number of points in the point cube 40307 (Figure 4D). In some configurations, camera 40227 (Figure 4D) cannot reliably project beyond approximately 6 meters, which can represent an upper limit to the range of camera 40227 (Figure 4D) and limit the number of points in the point cube 40307 (Figure 4D). In some configurations, the features of device 42114 (Figure 4A) can act as a minimum limit to the range of camera 40227 (Figure 4D). For example, the presence of caster 42103 (Figure 4A) can impose a minimum limit, which in some configurations may be set to approximately 1 meter. In some configurations, points within point cube 40307 (Figure 4D) can be limited to points that are 1 meter or more from camera 40227 (Figure 4D), and 6 meters or less from camera 40227 (Figure 4D).

[0187] Continuing with reference to Figures 4C and 4D, the traveling surface processor 41029 (Figure 4C) may include a base link transform 40219 that can transform a 3D cube of points into coordinates associated with device 42114 (Figure 4A), i.e., a base link frame 40309 (Figure 4C). The base link transform 40219 can transform a 3D data point 40223 (Figure 4D) in cube 40307 (Figure 4D) into a point 40233 (Figure 4D) in cube 40308 (Figure 4D), where the Z dimension is set to the base of device 42114 (Figure 4A). The running surface processor 41029 (Figure 4C) may include an OG preparation 40239 that can project point 40233 (Figure 4D) in cube 40308 (Figure 4D) onto the occupied grid 40244 (Figure 4D) as point 40237 (Figure 4D) in cube 40311 (Figure 4D). The layer processor 40241 can flatten point 40237 (Figure 4D) into various layers 40312 (Figure 4C) depending on the data represented by point 40237 (Figure 4D). In some configurations, the layer processor 40241 can apply scalar values ​​to the layer 40312 (Figure 4C). In some configurations, layer 40312 (Figure 4C) may include the probability of the occupied layer 40243, a surface classification layer 40245 (Figure 4D) which is determined by the neural network processor 40207, and a surface type confidence layer 40247 (Figure 4D). In some configurations, the surface type confidence layer 40247 (Figure 4D) may be determined by converting the class score from the neural network processor 40207 into a score which can be determined by normalizing the class score to a probability distribution across output classes as log(class score) / Σlog(each class). In some configurations, one or more layers may be replaced or extended by layers that provide the probability of an unmovable surface.

[0188] Continuing to refer to Figures 4C and 4D, in some configurations, the probability value in the occupied layer 40243 can be expressed as a log-odds (log-odds -> ln(p / (1-p)) value). In some configurations, the probability value in the occupied layer 40243 can be based on at least a combination of mode 41033 (Figure 4B) and surface type 40121 (Figure 4B). In some configurations, the pre-selected probability value in the occupied layer 40243 is, for example, not limited to However, (1) when surface type 40121 (Figure 4A) is rigid and drivable, and device 42114 (Figure 4A) is in a pre-selected set of mode 41033 (Figure 4B), or (2) when surface type 40121 (Figure 4A) is soft and drivable, and device 42114 (Figure 4A) is in a specific pre-selected mode such as standard mode, or (3 The configuration can be selected to cover situations such as when chair 42114 (Figure 4A) is in a specific pre-selected mode, such as four-wheel mode, or (4) when surface type 40121 (Figure 4A) is discontinuous and device 42114 (Figure 4A) is in a specific pre-selected mode, such as standard mode, or (5) when surface type 40121 (Figure 4A) is discontinuous and device 42114 (Figure 4A) is in a specific pre-selected mode, such as four-wheel mode, or (6) when surface type 40121 (Figure 4A) is immobile and device 42114 (Figure 4A) is in a pre-selected set of mode 41033 (Figure 4B). In some configurations, probability values ​​may include, but are not limited to, those listed in Table I. In some configurations, the neural network predicted probabilities may be adjusted as needed and may replace the probabilities listed in Table I. [Table 1]

[0189] Referring again to Figure 4C, the occupied grid processor 40242 can provide the global occupied grid processor 41025 in real time with parameters that may affect the probability value of the occupied grid 40244, such as, but not limited to, surface type 40121 and occupied grid information 41022. For example, but not limited to, configuration information 40125 (Figure 4B), such as mode 41033 and surface type 40121, can be provided to the power base processor 41016 (Figure 4B). The power base processor 41016 (Figure 4B) can determine a motor command 40128 (Figure 4B) that can configure the device 42114 (Figure 4A), based at least on the configuration information 40125 (Figure 4B).

[0190] Referring here to Figures 4E and 4F, device 42100A can be configured according to this teaching to operate in standard mode. In standard mode, the caster 42103 and the second drive wheel 42101B can be stationary on the ground as device 42100A navigates its path. The first drive wheel 42101A can be raised by a pre-selected amount 42102 (Figure 4F) to pass over the travel surface. Device 42100A can navigate normally on a relatively firm horizontal surface. When traveling in standard mode, the occupied grid 40244 (Figure 4C) can reflect surface type limits (see Table I), thus allowing for a suitable selection of modes or configuration changes based on the surface type and current mode.

[0191] Referring here to Figure 4G-4J, the devices 42100B / C can be configured according to this teaching to operate in four-wheel mode. In one configuration in four-wheel mode, the first drive wheel 42101A and the second drive wheel 42101B can be stationary on the ground as the device 42100A navigates its path. The caster 42103 can be retracted and can pass over the running surface by a pre-selected amount 42104 (Figure 4H). In another configuration in four-wheel mode, the first drive wheel 42101A and the second drive wheel 42101B can be substantially stationary on the ground as the device 42100A navigates its path. The caster 42103 can be retracted, and the chassis 42111 can be rotated, for example, to adapt to discontinuous surfaces (thus moving the caster 42103 further away from the ground). In this configuration, the caster 42103 can pass over the travel surface by a pre-selected amount 42108 (Figure 1J). Devices 42100A / C can navigate successfully on a variety of surfaces, including soft and discontinuous surfaces. In another configuration in four-wheel mode, the second drive wheel 42101B can be stationary on the ground as device 42100A, while the first drive wheel 42101A can be raised as device 42100C (Figure 1J) navigates its path. The caster 42103 can be retracted, and the chassis 42111 can be rotated, for example, to adapt to discontinuous surfaces (thus moving the caster 42103 further from the ground). When traveling in four-wheel mode, the occupied grid 40244 (Figure 4C) can reflect the surface type (see Table I), thus allowing for the selection of a suitable mode, or enabling configuration changes based on the surface type and the current mode.

[0192] Referring here to Figure 4K, a method 40150 for real-time control of the device configuration of a device, such as an AV, which moves along a path based on at least one environmental factor and device configuration, may include, but is not limited to, receiving sensor data 40151, determining the surface type 40153 based on at least the sensor data, and determining the current mode 40155 based on at least the surface type and the current device configuration. The method 40150 may also include determining the next device configuration 40157 based on at least the current mode and surface type, determining a move command 40159 based on at least the next device configuration, and changing the current device configuration to the next device configuration 40161 based on at least the move command.

[0193] Here, primarily referring to Figure 5A, annotated point data 10379 (Figure 5B) can be provided to the device controller 10111 in response to objects appearing in the AV's path. The annotated point data 10379 (Figure 5B), which can be the basis for route information that can be used to instruct the AV 10101 (Figure 1A) to proceed along the path, may include, but are not limited to, navigable edges, mapped trajectories such as, but are not limited to, mapped trajectories 10413 / 10415 (Figure 5D), and labeled features such as, but are not limited to, SDSF 10377 (Figure 5C). The mapped trajectories 10413 / 10415 (Figure 5C) may include a graph of edges in the route space and initial weights assigned to parts of the route space. The graph of edges may include, but are not limited to, properties such as directionality and capacity, and edges may be categorized according to these properties. The mapped trajectories 10413 / 10415 (Figure 5C) may include cost modifiers associated with the surface of the route space and travel modes associated with the edges. Travel modes may include, but are not limited to, path following and SDSF climbing. Other modes may include, but are not limited to, operating modes such as autonomous, mapping, and wait for intervention. Ultimately, the path can be selected based on at least the lower cost modifier. Forms relatively far from the mapped trajectories 10413 / 10415 (Figure 5C) may have higher cost modifiers and may not be of much interest when forming the path. The initial weighting is adjusted while AV10101 (Figure 1A) is operating and may, as a possibility, cause a modification of the path. The adjusted weighting can be used to adjust the edge / weighting graph 10381 (Figure 5B) and may be based on at least the current travel mode, current surface, and edge category.

[0194] Continuing with Figure 5A, the device controller 10111 may include a feature processor capable of performing specific tasks related to incorporating the eccentricity of any feature into the path. In some configurations, the feature processor may include, but is not limited to, an SDSF processor 10118. In some configurations, the device controller 10111 may include, but is not limited to, an SDSF processor 10118, a sensor processor 10703, a mode controller 10122, and a base controller 10114, each as described herein. The SDSF processor 10118, the sensor processor 10703, and the mode controller 10122 may provide inputs to the base controller 10114.

[0195] Continuing with Figure 5A, the base controller 10114 can determine, based on inputs provided by at least the mode controller 10122, the SDSF processor 10118, and the sensor processor 10703, information that the power base 10112 can use to drive the AV10101 (Figure 1A) along a path determined by the base controller 10114 based on at least the edge / weighted graph 10381 (Figure 5B). In some configurations, the base controller 10114 can ensure that the AV10101 (Figure 1A) follows a predetermined path from a starting point to a destination and can modify the predetermined path based on at least external and / or internal conditions. In some configurations, external conditions may include, but are not limited to, stop signals, SDSFs, and obstacles within or near the path being traveled by the AV10101 (Figure 1A). In some configurations, internal conditions may include, but are not limited to, mode transitions that reflect the response of the AV10101 (Figure 1A) to the external conditions. The device controller 10111 can determine commands to send to the power base 10112 based on at least external and internal conditions. Commands may include, but are not limited to, velocity and direction commands that can instruct AV 10101 (Figure 1A) to proceed in a commanded direction at a commanded velocity. Other commands may include, for example, a set of commands that enable feature responses such as SDSF climbing. The base controller 10114 may determine the desired velocity between waypoints in the path by conventional methods, for example, the Interior Point Optimizer (IPOPT) large-scale nonlinear optimization (https: / / projects.coin-or.org / Ipopt). The base controller 10114 may use, for at least, for example, Dijkstra's algorithm, A *The desired path can be determined based on conventional techniques such as search algorithms or techniques based on breadth-first search algorithms. The base controller 10114 can form a box around the mapped trajectory 10413 / 10415 (Figure 5C) to set an area where obstacle detection can be performed. The height of the payload carrier can be adjusted, at least partially, based on a specified velocity, when adjustable.

[0196] Continuing to refer to Figure 5A, the base controller 10114 can translate speed and direction determinations into motor commands. For example, but not limited to, when encountering an SDSF such as a curb or slope, the base controller 10114 can instruct the power base 10112 to raise the payload carrier 10173 (Figure 1A), align the AV 10101 (Figure 1A) with the SDSF at an angle of approximately 90°, and reduce the speed to a relatively low level in the SDSF climbing mode. When the AV 10101 (Figure 1A) is climbing a substantially discontinuous surface, the base controller 10114 can instruct the power base 10112 to transition to an climbing phase in which the speed is increased because an increased torque is required to move the AV 10101 (Figure 1A) up the incline. When AV10101 (Figure 1A) encounters a relatively horizontal surface, the base controller 10114 can reduce speed to stay over any flat portion of the SDSF. In the case of a downhill ramp associated with a flat portion, when AV10101 (Figure 1A) begins descending a substantially discontinuous surface, and both wheels are on the downhill ramp, the base controller 10114 can allow speed to increase. For example, when encountering an SDSF such as a slope, the slope can be identified and treated as a structure. The structural features may include, for example, a ramp of a pre-selected size. The ramp may include a slope of about 30° and may optionally be on both sides of a flat area. The device controller 10111 (Figure 5A) can distinguish between an obstacle and a slope by comparing the angle of the perceived feature with the expected slope ramp angle, the angle of which can be received from the sensor processor 10703 (Figure 5A).

[0197] Here, primarily referring to Figure 5B, the SDSF processor 10118 can locate navigable edges from blocks of traversable surfaces formed by polygonal meshes represented in annotated point data 10379, which can be used to create paths for crossing by AV 10101 (Figure 1A). Within an SDSF buffer 10407 (Figure 5C), which can form an area of ​​a pre-selected size around an SDSF line 10377 (Figure 5C), the navigable edges can be erased in preparation for special handling in anticipation of an SDSF crossing (see Figure 5D). Closed line segments such as segment 10409 (Figure 5C) can be drawn to bisect the SDSF buffer 10407 (Figure 5C) between pairs of previously determined SDSF points 10789 (Figure 1N). In some configurations, since a closed line segment is considered a candidate for SDSF crossing, the segment end 10411 (Figure 5C) can be located in an unobstructed portion of the traversable surface, and there can be sufficient room for AV 10101 (Figure 1A) to travel along the line segment between adjacent SDSF points 10789 (Figure 1N), and the area between SDSF points 10789 (Figure 1N) can be the traversable surface. The segment end 10411 (Figure 5C) can be connected to the underlying morphology to form a vertex and a traversable edge. For example, line segments 10461, 10463, 10465, and 10467 (Figure 5C) that satisfy the crossing criterion are shown as part of the morphology in Figure 5D. In contrast, line segment 10409 (Figure 5C) did not satisfy the criterion because, at least, the segment end 10411 (Figure 5C) does not lie on the traversable surface. Overlapping SDSF buffers 10506 (Figure 5C) can indicate SDSF discontinuities, which can disadvantage SDSF crossings of SDSFs within the overlapping SDSF buffers 10506 (Figure 5C). SDSF lines 10377 (Figure 5C) can be smoothed, and the locations of SDSF points 10789 (Figure 1N) can be adjusted so that they are separated by a pre-selected distance, the pre-selected distance being based on the area occupied by at least AV 10101 (Figure 1A).

[0198] Continuing to refer to Figure 5B, the SDSF processor 10118 can convert the annotated point data 10379 into an edge / weighted graph 10381, including morphological modifications for SDSF cross-sections. The SDSF processor 10118 may include a seventh processor 10601, an eighth processor 10702, a ninth processor 10603, and a tenth processor 10605. The seventh processor 10601 can convert the coordinates of points in the annotated point data 10379 into a global coordinate system, achieve compatibility with GPS coordinates, and generate a GPS-compatible dataset 10602. The seventh processor 10601 can generate the GPS-compatible dataset 10602 using conventional processes such as, for example, affine matrix transformations and PostGIS transformations, but is not limited to these. The World Geodetic System (WGS) can be used as the standard coordinate system because it takes into account the curvature of the Earth. The map can be stored in the Universal Transverse Mercator (UTM) coordinate system and can be switched to WGS when it is necessary to find the location of a specific address.

[0199] Here, primarily referring to Figure 5C, the eighth processor 10702 (Figure 5B) can smooth the SDSF, determine the boundaries of the SDSF 10377, and create buffers 10407 around the SDSF boundaries, increasing the surface cost modifier as it moves further away from the SDSF boundaries. The mapped trajectories 10413 / 10415 may be special-case lanes with the lowest cost modifiers. Lower cost modifiers 10406 ​​can generally be located near the SDSF boundaries, while higher cost modifiers 10408 can generally be located relatively far from the SDSF boundaries. The eighth processor 10702 can provide point cloud data 10704 (Figure 5B) with costs to the ninth processor 10603 (Figure 5B).

[0200] Continuing, primarily with reference to Figure 5C, the ninth processor 10603 (Figure 5B) can calculate an approximately 90° approach 10604 (Figure 5B) for AV10101 (Figure 1A) to traverse SDSF10377 that meet the criteria for labeling them as traversable. The criteria may include the SDSF width and the SDSF smoothness. Line segments, such as line segment 10409, can be created such that their lengths indicate the minimum approach distance that AV10101 (Figure 1A) may be required to approach SDSF10377 and the minimum exit distance that may be required to exit SDSF10377. Segment endpoints, such as endpoint 10411, can be integrated with the underlying routing configuration. The criteria used to determine whether an SDSF approach is possible may exclude several possible approaches. SDSF buffers, such as SDSF buffer 10407, can be used to calculate valid approaches and route configuration edge creation.

[0201] Referring again, primarily to Figure 5B, the tenth processor 10605 can generate from a morphology an edge / weight graph 10381, which is an edge and weight graph developed herein, which can be used to compute a path through the map. The morphology may include cost modifiers and travel modes, and the edges may include directionality and capacity. The weights can be adjusted at runtime based on information from any number of sources. The tenth processor 10605 can provide the base controller 10114 with at least one sequence of ordered points, in addition to a recommended travel mode at a particular point, to enable path generation. Each point in each sequence of points represents the location and labeling of a possible path point on the processed traversable surface. In some configurations, the labeling may indicate that the point represents a part of the features that may be encountered along the path, such as, for example, an SDSF, etc. In some configurations, the features may be further labeled with suggested processing based on the type of feature. For example, in some configurations, if a path point is labeled as an SDSF, further labeling may include a mode. The mode can be interpreted by AV10101 (Figure 1A) as a suggested travel command for AV10101 (Figure 1A), such as switching AV10101 (Figure 1A) to SDSF uphill mode 100-31 (Figure 5E) to allow AV10101 (Figure 1A) to traverse SDSF 10377 (Figure 5C).

[0202] Referring here to Figure 5E, in some configurations, the mode controller 10122 can provide instructions to the base controller 10114 (Figure 5A) to perform mode transitions. The mode controller 10122 can establish the mode in which AV 10101 (Figure 1A) is progressing. For example, the mode controller 10122 can provide the base controller 10114 with a change in mode indication, for example, when SDSF is identified along the progress path, it can change between path-following mode 10100-32 and SDSF ascent mode 10100-31. In some configurations, annotated point data 10379 (Figure 5B) can include mode identifiers at various points along the route, for example, when the mode is changed to adapt the route. For example, if SDSF 10377 (Figure 5C) is labeled within annotated point data 10379 (Figure 5B), the device controller 10111 can determine the mode identifier associated with the route point and, potentially, adjust the commands to the power base 10112 (Figure 5A) based on the desired mode. In addition to the SDSF climbing mode 10100-31 and path-following mode 10100-32, in some configurations, AV 10101 (Figure 1A) can support operating modes, which may include, but are not limited to, standard mode 10100-1 and enhanced (four-wheel) mode 10100-2 as described herein. The height of the payload carrier 10173 (Figure 1A) can be adjusted to provide the necessary clearance across obstacles and along slopes.

[0203] Referring here to Figure 5F, a method 11150 for navigating an AV towards a target point that traverses at least one SDSF may include, but is not limited to, receiving SDSF information related to the SDSF, the location of the target point, and the location of the AV. The SDSF information may, but is not limited to, a set of points classified as SDSF points and the associated probability for each point that a point is an SDSF point. Method 11150 may include drawing a closed polygon encompassing the location of the AV and the location of the target point, and drawing a path line between the target point and the location of the AV. The closed polygon may include a pre-selected width. Table I contains a range of possible pre-selected variables discussed herein. Method 11150 may include selecting two of the SDSF points located within the polygon, and drawing an SDSF line between the two points. In some configurations, the selection of SDSF points may be random or by any other method. If, in step 11159, there are fewer than a first pre-selected number of points within a first pre-selected distance of the SDSF line, and in step 11161, there are fewer than a second pre-selected number of attempts in selecting SDSF points, drawing lines between them and having fewer than a first pre-selected number of points around the SDSF line, then method 11150 may include returning to step 11155. If, in step 11161, there are a second pre-selected number of attempts in selecting SDSF points, drawing lines between them and having fewer than a first pre-selected number of points around the SDSF line, then method 11150 may include acknowledging that no SDSF lines were detected 11163.

[0204] Here, primarily referring to Figure 5G, if in 11159 (Figure 5F) there are a first pre-selected number of points or more, method 11150 may include fitting the curve to the points that fall within a first pre-selected distance of the SDSF line 11165. If in 11167 the number of points within a first pre-selected distance of the curve exceeds the number of points within a first pre-selected distance of the SDSF line, and in 11171 the curve intersects a path line, and in 11173 there are no gaps between points on the curve beyond a second pre-selected distance, method 11150 may include identifying the curve as an SDSF line 11175. If, in 11167, the number of points within a first pre-selected distance of the curve does not exceed the number of points within a first pre-selected distance of the SDSF line, or in 11171, the curve does not intersect the path line, or in 11173, there are gaps between points on the curve that exceed a second pre-selected distance, and in 11177, the SDSF line does not remain stable, and in 11169, the curve fit has not been attempted more than a second pre-selected number of times, then method 11150 may return to step 11165. A stable SDSF line is the result of subsequent iterations that yield the same or fewer points.

[0205] Here, primarily referring to Figure 5H, if the curve fit has been performed only a second pre-selected number of times in 11169 (Figure 5G), or if the SDSF line remains stable or degrades in 11177 (Figure 5G), method 11150 may include receiving occupy grid information 11179. The occupy grid can provide the probability of an obstacle being present at a point. The occupy grid information can enhance the SDSF and path information found within the polygon surrounding the AV path and the SDSF, when the occupy grid includes data captured and / or calculated over a common geographical area with the polygon. Method 11150 may include selecting a point from the common geographical area and its associated probability 11181. If, in 11183, the probability that an obstacle is present at a selected point is higher than a pre-selected percentage, and in 11185, the obstacle is located between AV and the target point, and in 11186, the obstacle is located less than a third pre-selected distance from the SDSF line between the SDSF line and the target point, then method 11150 may include projecting the obstacle onto the SDSF line 11187. If, in 11183, the probability that a location contains an obstacle is less than or equal to a pre-selected percentage, or in 11185, the obstacle is not located between AV and the target point, or in 11186, the obstacle is located at a distance equal to or greater than a third pre-selected distance from the SDSF line between the SDSF and the target point, and in 11189, there are further obstacles to be dealt with, then method 11150 may include resuming the processing in step 11179.

[0206] Here, referring primarily to Figure 5I, in 11189 (Figure 5H), if there are no further obstacles to be dealt with, method 11150 may include connecting the projections and finding the endpoints of the connected projections along the SDSF line 11191. Method 11150 may include marking a portion of the SDSF line between the projection endpoints as non-crossable 11193. Method 11150 may include marking a portion of the SDSF line outside the non-crossable section as crossable 11195. Method 11150 may include redirecting the AV to within a fifth pre-selected amount perpendicular to the crossable section of the SDSF line 11197. In 11199, if the azimuth error with respect to a line perpendicular to the crossable section of the SDSF line exceeds a first pre-selected amount, method 11150 may include slowing the AV by a ninth pre-selected amount 11251. Method 11150 may include driving the AV forward toward the SDSF line and decelerating it by a second pre-selected amount per meter distance between the AV and the traversable SDSF line 11253. If, in 11255, the distance of the AV from the traversable SDSF line is less than a fourth pre-selected distance, and in 11257, the bearing error is greater than or equal to a third pre-selected amount for a line perpendicular to the SDSF line, Method 11150 may include decelerating the AV by a ninth pre-selected amount 11252.

[0207] Here, referring primarily to Figure 5J, in 11257 (Figure 5I), if the azimuth error is less than a third pre-selected amount relative to a line perpendicular to the SDSF line, method 11150 may include ignoring the updated SDSF information and driving the AV at a pre-selected speed 11260. In 11259, if the rise of the front portion of the AV relative to the rear portion is between a sixth pre-selected amount and a fifth pre-selected amount, method 11150 may include driving the AV forward and increasing the speed of the AV to an eighth pre-selected amount per degree of rise 11261. In 11263, if the rise of the front portion of the AV relative to the rear portion is less than a sixth pre-selected amount, method 11150 may include driving the AV forward at a seventh pre-selected speed 11265. If, in step 11267, the rear of the AV is greater than a fifth pre-selected distance from the SDSF line, method 11150 may include acknowledging that the AV has completed crossing the SDSF 11269. If, in step 11267, the rear of the AV is less than or equal to a fifth pre-selected distance from the SDSF line, method 11150 may include returning to step 11260.

[0208] Referring here to Figure 5K, the system 51100 for navigating the AV towards a target point traversing at least one SDSF may include, but is not limited to, a path line processor 11103, an SDSF detector 11109, and an SDSF controller 11127. The system 51100 may be operably coupled with a surface processor 11601 capable of processing sensor information, which may, for example, include, but is not limited to, images of the vicinity of AV 10101 (Figure 5L). The surface processor 11601 may provide real-time surface feature updates, including indications of SDSFs. In some configurations, a camera may provide RGB-D data, which may classify its points according to surface type. In some configurations, the system 51100 may process points classified as SDSFs and their associated probabilities. The system 51100 may be operably coupled with a system controller 11602 capable of managing aspects of the operation of AV 10101 (Figure 5L). The system controller 11602 can maintain an occupied grid 11138 which may include information from available sources regarding the navigable area in the vicinity of AV10101 (Figure 5L). The occupied grid 11138 may include the probability of obstacles being present. This information, in conjunction with the SDSF information, can be used to determine whether SDSF 10377 (Figure 5N) can be traversed by AV10101 (Figure 5L) without encountering obstacle 11681 (Figure 5M). Based on the environment and other information, the system controller 11602 can determine a speed limit 11148 that AV10101 (Figure 5N) should not exceed. The speed limit 11148 can be used as a guideline for the speed set by system 51100, or it can be overridden. System 51100 can be operably coupled with the base controller 10114, which can transmit the drive command 11144 generated by the SDSF controller 11127 to the drive component of AV10101 (Figure 5L).The base controller 10114 can provide the SDSF controller 11127 with information about the orientation of AV10101 (Figure 5L) during the SDSF crossing.

[0209] Continuing to refer to Figure 5K, the path line processor 11103 can continuously receive surface classification points 10789 in real time, which may include, but are not limited to, points classified as SDSF. The path line processor 11103 can receive the location of the target point 11139 and the AV location 11141, which may be, for example, indicated by the center 11202 (Figure 5L) of AV 10101 (Figure 5L). System 51100 may include a polygon processor 11105 that draws a polygon 11147 encompassing AV location 11141, the location of the target point 11139, and the path 11214 between the target point 11139 and AV location 11141. The polygon 11147 may include a pre-selected width. In some configurations, the pre-selected width may include the approximate width of AV 10101 (Figure 5L). The SDSF point 10789 located within polygon 11147 can be identified.

[0210] Continuing to refer to Figure 5K, the SDSF detector 11109 can receive surface classification points 10789, paths 11214, polygons 11147, and target points 11139, and can determine the most suitable SDSF line 10377 available in the incoming data according to the criteria described herein. The SDSF detector 11109 may include, but is not limited to, a point processor 11111 and an SDSF line processor 11113. The point processor 11111 may include selecting two of the SDSF points 10789 located within polygon 11147 and drawing an SDSF line 10377 between the two points. If there are fewer than a first pre-selected number of points within a first pre-selected distance of SDSF line 10377, and if there are fewer than a second pre-selected number of attempts in selecting SDSF point 10789, and a line is drawn between the two points, and there are fewer than a first pre-selected number of points around the SDSF line, the point processor 11111 may again loop through the selection-draw-test loop as described herein. If there are a second pre-selected number of attempts in selecting SDSF points, and a line is drawn between them, and there are fewer than a first pre-selected number of points around the SDSF line, the point processor 11111 may acknowledge that no SDSF lines were detected.

[0211] Continuing to refer to Figure 5K, the SDSF line processor 11113 may include fitting the curve 11609-11611 (Figure 5L) to points 10789 that fall within a first pre-selected distance of the SDSF line 10377, if a first pre-selected number or more of points 10789 exist. The SDSF line processor 11113 may identify curve 11609-11611 (Figure 5L) as (for example) SDSF line 10377 if the number of points 10789 within a first pre-selected distance of curve 11609-11611 (Figure 5L) exceeds the number of points 10789 within a first pre-selected distance of SDSF line 10377, and if curve 11609-11611 (Figure 5L) intersects with path line 11214, and if there are no gaps between points 10789 on curve 11609-11611 (Figure 5L) beyond a second pre-selected distance. If the number of points 10789 within a pre-selected distance of curve 11609-11611 (Figure 5L) does not exceed the number of points 10789 within a first pre-selected distance of SDSF line 10377, or if curve 11609-11611 (Figure 5L) does not intersect with path line 11214, or if there are gaps between points 10789 on curve 11609-11611 (Figure 5L) that exceed a second pre-selected distance, and if SDSF line 10377 is not stable, and if the curve fit has not been attempted more than the second pre-selected number of times, the SDSF line processor 11113 can run the curve fit loop again.

[0212] Continuing to refer to Figure 5K, the SDSF controller 11127 can receive the SDSF line 10377, the occupied grid 11138, the AV orientation change 11142, and the speed limit 11148, and can generate an SDSF command 11144 to cause the AV 10101 (Figure 5L) to travel so as to correctly traverse the SDSF 10377 (Figure 5N). The SDSF controller 11127 may, but is not limited to, include an obstacle processor 11115, an SDSF approach 11131, and an SDSF traverse 11133. The obstacle processor 11115 can receive the SDSF line 10377, the target point 11139, and the occupied grid 11138, and can determine from among the obstacles identified in the occupied grid 11138 whether any of them could interfere with the AV 10101 (Figure 5N) as it traverses the SDSF 10377 (Figure 5N). The obstacle processor 11115 may include, but is not limited to, an obstacle selector 11117, an obstacle tester 11119, and a cross-sectional locator 11121. The obstacle selector 11117 may include, but is not limited to, receiving an occupied grid 11138 as described herein. The obstacle selector 11117 may also include selecting occupied grid points and their associated probabilities from a geographic area common to both the occupied grid 11138 and the polygon 11147. If the probability of an obstacle being present at a selected grid point is higher than a pre-selected percentage, and the obstacle is located between AV10101 (Figure 5L) and target point 11139, and the obstacle is less than a third pre-selected distance from SDSF line 10377 between SDSF line 10377 and target point 11139, the obstacle tester 11119 may include projecting the obstacle onto SDSF line 10377 to form a projection 11621 intersecting SDSF line 10377.If the probability that a location contains an obstacle is less than or equal to a pre-selected percentage, or if the obstacle is not located between AV10101 (Figure 5L) and target point 11139, or if the obstacle is located at a distance equal to or greater than a third pre-selected distance from SDSF line 10377 between SDSF line 10377 and target point 11139, the obstacle tester 11119 may include resuming execution in receiving the occupied grid 11138 if there are further obstacles to process.

[0213] Continuing with Figure 5K, the cross-sectional locator 11121 may include connecting projection points and locating the endpoints 11622 / 11623 (Figure 5M) of the connected projection 11621 (Figure 5M) along the SDSF line 10377. The cross-sectional locator 11121 may also include marking the portion 11624 (Figure 5M) of the SDSF line 10377 between the projection endpoints 11622 / 11623 (Figure 5M) as non-crossable. The cross-sectional locator 11121 may also include marking the portion 11626 (Figure 5M) of the SDSF line 10377 outside the non-crossable portion 11624 (Figure 5M) as crossable.

[0214] Continuing to refer to Figure 5K, the SDSF approach 11131 may include sending an SDSF command 11144 to redirect AV10101 (Figure 5N) to within a fifth pre-selected amount perpendicular to the traversable portion 11626 (Figure 5N) of the SDSF line 10377. If the azimuth error with respect to the vertical line 11627 (Figure 5N), which is perpendicular to the traversable portion 11626 (Figure 5N) of the SDSF line 10377, exceeds a first pre-selected amount, the SDSF approach 11131 may include sending an SDSF command 11144 to decelerate AV10101 (Figure 5N) by a ninth pre-selected amount. In some configurations, the ninth pre-selected amount can range from very slow to a complete stop. The SDSF approach 11131 may include sending an SDSF command 11144 to move AV10101 (Figure 5N) forward toward SDSF line 10377, and sending an SDSF command 11144 to decelerate AV10101 (Figure 5N) by a second pre-selected amount per meter traveled. If the distance between AV10101 (Figure 5N) and the traversable SDSF line 11626 (Figure 5N) is less than a fourth pre-selected distance, and the azimuth error is greater than or equal to a third pre-selected amount for a line perpendicular to SDSF line 10377, the SDSF approach 11131 may include sending an SDSF command 11144 to decelerate AV10101 (Figure 5N) by a ninth pre-selected amount.

[0215] Continuing to refer to Figure 5K, if the orientation error is less than a third pre-selected amount relative to a line perpendicular to the SDSF line 10377, the SDSF crossing 11133 may ignore the updated SDSF information and send an SDSF command 11144 to drive AV10101 (Figure 5N) at the pre-selected rate. If the AV orientation change 11142 indicates that the rise of the leading edge 11701 (Figure 5N) of AV10101 (Figure 5N) relative to the trailing edge 11703 (Figure 5N) of AV10101 (Figure 5N) is between a sixth pre-selected amount and a fifth pre-selected amount, the SDSF crossing 11133 may send an SDSF command 11144 to drive AV10101 (Figure 5N) forward and an SDSF command 11144 to increase the speed of AV10101 (Figure 5N) to the pre-selected rate per degree of rise. If the AV orientation change 11142 indicates that the rise of the leading edge 11701 (Figure 5N) relative to the trailing edge 11703 (Figure 5N) of AV 10101 (Figure 5N) is less than a sixth pre-selected amount, the SDSF crossing 11133 may include sending an SDSF command 11144 to cause AV 10101 (Figure 5N) to travel forward at a seventh pre-selected speed. If the AV location 11141 indicates that the trailing edge 11703 (Figure 5N) is beyond a fifth pre-selected distance from the SDSF line 10377, the SDSF crossing 11133 may include acknowledging that AV 10101 (Figure 5N) has completed crossing the SDSF 10377. If AV location 11141 indicates that the trailing edge 11703 (Figure 5N) is less than or equal to a fifth pre-selected distance from SDSF line 10377, the SDSF crossing 11133 may include repeating a loop that begins with ignoring the updated SDSF information.

[0216] Some illustrative ranges of the pre-selected values ​​described herein may include, but are not limited to, those outlined in Table II. [Table 2-1] Table 2-2

[0217] Referring here to Figure 5O, in some configurations, to support real-time data aggregation, the system of this teaching can generate locations in three-dimensional space of various surface types in response to receiving data such as, for example, RGD-D camera image data. The system can rotate images 12155 and convert them from camera coordinate system 12157 to UTM coordinate system 12159. The system can generate polygonal files from the converted images, and the polygonal files can represent three-dimensional locations associated with surface type 12161. A method 12150 for locating features 12151 from camera images 12155 received by AV10101, having orientation 12163, may include, for example, receiving camera images 12155 by AV10101. Each camera image 12155 may include an image timestamp 12171, and each image 12155 may include an image color pixel 12167 and an image depth pixel 12169. Method 12150 may include receiving an orientation 12163 of AV 10101, wherein the orientation 12163 has an orientation timestamp 12171, and determining a selected image 12173 by identifying an image from a camera image 12155 having an image timestamp 12165 adjacent to the orientation timestamp 12171. Method 12150 may include separating image color pixels 12167 from image depth pixels 12169 in a selected image 12173, and determining an image surface classification 12161 for a selected image 12173 by providing the image color pixels 12167 to a first machine learning model 12177 and the image depth pixels 12169 to a second machine learning model 12179. Method 12150 may also include determining perimeter points 12181 of a feature in a camera image 12173, where the feature includes feature pixels 12151 within the perimeter, each feature pixel 12151 having the same surface classification 12161, and each perimeter point 12181 having a set of coordinates 12157.Method 12150 may include converting each of the coordinate sets 12157 to UTM coordinates 12159.

[0218] The structure of this instruction relates to a computer system for carrying out the methods discussed herein and a computer-readable medium containing programs for carrying out these methods. Raw data and results can be stored, printed, displayed, transferred to another computer, and / or transferred to another location for future reading and processing. Communication links can be wired or wireless, for example, using cellular communication systems, military communication systems, and satellite communication systems. Parts of the system can run on a computer with a variable number of CPUs. Other alternative computer platforms can also be used.

[0219] This configuration also covers software for performing the methods discussed herein and computer-readable media for storing the software for performing these methods. The various modules described herein may be performed on the same CPU or on different computers. In accordance with the law, this configuration has been described in more or less specific language with respect to its structural and methodological features. However, it should be understood that this configuration is not limited to the specific features shown and described, as the means disclosed herein constitute a preferred form of embodying this configuration.

[0220] The method can be implemented electronically, either entirely or in part. Signals representing actions taken by the System and other disclosed components of the System can be transmitted over at least one live communication network. Control and data information can be electronically executed and stored on at least one computer-readable medium. The System can be implemented to run on at least one computer node in at least one live communication network. General forms of at least one computer-readable medium include, but are not limited to, floppy disks, flexible disks, hard disks, magnetic tapes, or any other magnetic media, compact disk read-only memory or any other optical media, punch cards, paper tapes, or any other physical media with perforation patterns, random access memory, programmable read-only memory, and erasable programmable read-only memory (EPROM), flash EPROM, or any other memory chip or cartridge, or any other medium from which a computer can read. Furthermore, at least one computer-readable medium may contain graphs in any form, provided they are appropriately licensed, including, but are not limited to, Graphics Exchange Format (GIF), Joint Photographic Professional Group (JPEG), Portable Network Graphics (PNG), Scalable Vector Graphics (SVG), and Tagged Image File Format (TIFF).

[0221] While these instructions have been described above in terms of specific configurations, it should be understood that they are not limited to these disclosed configurations. Many modifications and other configurations are intended and will be covered by both the disclosure and the accompanying claims, as this will be recalled by those skilled in the art. The scope of these instructions is intended to be determined by the proper interpretation and construction of the accompanying claims and their legal equivalents, as understood by those skilled in the art relying on the disclosures in this specification and the accompanying drawings.

Claims

1. A method for real-time control of the configuration of a device, the device comprising a chassis, four wheels, a first side of the chassis operably coupled to one of the four wheels, and a second side of the chassis operably coupled to another of the four wheels, the method being: Receiving environmental data and, Based on the aforementioned environmental data, the surface type is determined, Based on the surface type and the first configuration, the mode is determined, Based on the mode and the surface type, a second configuration is determined, Based on the second configuration described above, the movement command is determined, Controlling the configuration of the device based on the move command, including changing the device from a first configuration to a second configuration, To determine the move command based on an occupied grid representing the world for the purpose of route planning for the device, Includes, The environmental data includes RGB-D image data, road surface morphology data, landmark data, long-range camera data, short-range camera data, LIDAR data, radar data, thermometer data, pressure sensor data, weather condition sensor data, and combinations thereof. method.

2. The method according to claim 1, further comprising incorporating the occupied grid based at least on the surface type and the mode.

3. The method according to claim 1, wherein the control of the configuration includes coordinated power supply of a first pair of the four-wheel clusters and a second pair of the four-wheel clusters based on the environmental data.

4. The method according to claim 1, wherein the control of the configuration is to drive the four wheels and pairs of casters configured to be movable in the vertical direction, the pairs of casters being operably coupled to the chassis, and the control is to drive two wheels, including a clustered first pair and a clustered second pair which are rotated to lift the first front wheel and the second front wheel, the device being stationary on the first rear wheel, the second rear wheel and the pairs of casters, the clustered first pair including the first front wheel and the first rear wheel, and the clustered second pair including the second front wheel and the second rear wheel.

5. The method according to claim 3, wherein the control of the configuration includes rotating a pair of clusters operably coupled with two first powered wheels on the first side and two second powered wheels on the second side, based on the environmental data.

6. The method according to claim 1, wherein the device further comprises a cargo container, the cargo container is mounted on the chassis, and the chassis controls the height of the cargo container.

7. The height of the cargo container is determined by the method according to claim 6, based on the environmental data.

8. A system for real-time control of the configuration of a device, wherein the device includes a chassis, four wheels, a first side of the chassis, and a second side of the chassis, and the system is A device processor, the device processor is configured to receive environmental data surrounding the device, determine a surface type based on the environmental data, determine a mode based on the surface type and a first configuration, and determine a second configuration based on the mode and the surface type, A power base processor that determines a move command based on the second configuration, wherein the power base processor controls the configuration of the device based on the move command and changes the device from a first configuration to a second configuration. A system that includes, The power base processor determines the move command based on an occupied grid representing the world for the purpose of route planning for the device, The environmental data includes RGB-D image data, road surface morphology data, landmark data, long-range camera data, short-range camera data, LIDAR data, radar data, thermometer data, pressure sensor data, weather condition sensor data, and combinations thereof. system.

9. The system according to claim 8, wherein the occupied grid comprises information relating to surface type and / or mode.

10. The system according to claim 8, wherein the control of the configuration includes coordinated power supply of a first pair of the four-wheel clusters and a second pair of the four-wheel clusters based on the environmental data.

11. Control of the configuration is to drive the four wheels and pairs of casters configured to be movable in the vertical direction, the pairs of casters being operably coupled to the chassis, and the system is to drive two wheels, including a clustered first pair and a clustered second pair which are rotated to lift the first front wheels and the second front wheels, the device being stationary on the first rear wheels, the second rear wheels and the pairs of casters, the clustered first pair including the first front wheels and the first rear wheels, and the clustered second pair including the second front wheels and the second rear wheels.

Images