System and method for surface feature detection and cross-sectioning

Abstract

Providing a system and method for surface feature detection and cross-sectioning. [Solution] The substantially discontinuous surface feature cross-sections of this teaching can utilize a transport device (TD), for example, an autonomous or semi-autonomous device, to navigate in an environment that may contain features such as substantially discontinuous surface features. The substantially discontinuous surface feature cross-sections can enable the TD to travel over a variety of extended surfaces. In particular, the substantially discontinuous surface features can be precisely identified and labeled so that the TD can automatically maintain its performance during entry into and exit from the substantially discontinuous surface features.

Description

Background Art

[0001] (Cross - reference to Related Applications) This utility patent application claims priority to U.S. Provisional Patent Application No. 62 / 809,973, filed on February 25, 2019, entitled "System and Method for Surface Feature Detection and Traversal" (Attorney Docket No. Z26), which is hereby incorporated by reference in its entirety; U.S. Provisional Patent Application No. 62 / 851,266, filed on May 22, 2019, entitled "System and Method for Surface Feature Traversal" (Attorney Docket No. Z81); and U.S. Provisional Patent Application No. 62 / 851,266, filed on May 23, 2019, entitled "System and Method for Surface Feature Traversal" (Attorney Docket No. Z88), which is hereby incorporated by reference in its entirety.

[0002] The present teachings generally relate to surface feature detection and traversal. Surface feature traversal is problematic because surface features, such as, but not limited to, substantially discontinuous surface features (SDSFs), can be found in heterogeneous forms, which can be unique to a specific geography. However, SDSFs, such as, but not limited to, slopes, edges, curbs, steps, and curb - like geometries (referred to herein in a non - limiting way as SDSFs or simply surface features), can include some typical characteristics that can assist in their identification.

[0003] A wide range of devices and methods for transporting people and goods, including autonomous transport, are publicly known. The designs of these devices address non-uniform travel surfaces in several different ways. However, what is missing is the ability to locate SDSFs based on multipart models associated with several criteria for SDSF identification. Also missing is the integration of the located SDSF trajectory with graphed polygons that can form the route morphology. Furthermore, the determination of candidate surface feature crossings does not rely on criteria such as candidate crossing approach angles, candidate crossing travel surfaces on both sides of the candidate surface feature, and candidate crossing path obstacles. [Overview of the project] [Means for solving the problem]

[0004] The SDSF crossing described in this instruction can utilize a transport device (TD), such as an autonomous or semi-autonomous device, to navigate within an environment that may contain features such as SDSFs. SDSF crossing features can enable the TD to travel across a variety of extended surfaces. In particular, SDSFs can be precisely identified so that the TD can automatically maintain its performance during SDSF crossing. In some configurations, SDSFs can be identified by their dimensions. For example, curbs can include widths of approximately 0.6–0.7 m, but are not limited. In some configurations, point cloud data can be processed to locate SDSFs, and this data can be used to prepare a path for the TD from a starting point to a destination. While the TD is traveling along the path, in some configurations, SDSF crossing can be adapted through the TD's sensor-based positioning.

[0005] In some configurations, the teachings describe a method for navigating at least one SDSF encountered by a TD, wherein the TD travels along a path on a surface, the surface includes at least one SDSF, the path includes a start point and an end point, and the method may include, but is not limited to, accessing point cloud data representing the surface, filtering the point cloud data, forming the filtered point cloud data into a processable portion, and merging the processable portion into at least one concave polygon. The method may include locating and labeling the at least one SDSF within at least one concave polygon. Locating and labeling can form labeled point cloud data. The method may include, at least, creating a graphing polygon based on at least one concave polygon, and at least selecting a path from the start point to the end point based on the graphing polygon. The TD may traverse at least one SDSF along the path.

[0006] Filtering point cloud data may optionally include conditionally removing points representing transient objects and outliers from the point cloud data, and replacing removed points with pre-selected heights. Forming processing portions may optionally include dividing the point cloud data into processing portions and removing points with pre-selected heights from the processing portions. Merging processing portions may optionally include reducing the size of the processing portions by analyzing outliers, voxels, and normals, expanding the area from the reduced-size processing portions, determining an initial drivable surface from the expanded area, subdividing and meshing the initial drivable surface, identifying the location of polygons within the subdivided and meshed initial drivable surface, and defining the drivable surface based on at least the polygons. Identifying the location of at least one SDSF feature and labeling them optionally includes sorting point cloud data of drivable surfaces according to an SDSF filter, wherein the SDSF filter includes points of at least three categories, and identifying the location of at least one SDSF point based on whether the points of the category, in combination, satisfy at least one first pre-selected criterion. The method optionally includes creating at least one SDSF trajectory based on whether a plurality of at least one SDSF points, in combination, satisfy at least one second pre-selected criterion. Creating a graphed polygon further optionally includes creating at least one convex polygon from at least one drivable surface. The at least one convex polygon may include edges. Creating a graphed polygon may include smoothing the edges, forming a drivable margin based on the smoothed edges, adding at least one SDSF trajectory to at least one drivable surface, and removing edges from at least one drivable surface according to at least one third pre-selected criterion. Edge smoothing may optionally include trimming the edges outwards.Forming the running margin of the smoothed edge may optionally include trimming the outer edge inward.

[0007] In some configurations, the system of this teaching for navigating at least one SDSF encountered by a TD, wherein the TD travels along a path on a surface, the surface includes at least one SDSF, the path includes a start point and an end point, and the system may include, but is not limited to, a first processor for accessing point cloud data representing the surface, a first filter for filtering the point cloud data, a second processor for forming a processable portion from the filtered point cloud data, a third processor for merging the processable portion into at least one concave polygon, a fourth processor for locating and labeling at least one SDSF within at least one concave polygon, the fourth processor for locating and labeling, forming labeled point cloud data, a fifth processor for creating a graphing polygon, and a path selector for selecting a path from the start point to the end point based on at least the graphing polygon. The TD can traverse at least one SDSF along the path.

[0008] The first filter may optionally include executable code that conditionally removes points representing transient objects and outliers from the point cloud data, and replaces the removed points with pre-selected heights. The segmenter may optionally include executable code that divides the point cloud data into processable parts, and removes points with pre-selected heights from the processable parts. The third processor may optionally include executable code that reduces the size of the processable parts by analyzing outliers, voxels, and normals, expands the area from the reduced-size processable parts, determines an initial drivable surface from the expanded area, subdivides and meshes the initial drivable surface, identifies the location of polygons within the subdivided and meshed initial drivable surface, and defines the drivable surface based on at least the polygons. The fourth processor may optionally include executable code that sorts point cloud data of drivable surfaces according to an SDSF filter, wherein the SDSF filter includes points of at least three categories, and identifies the location of at least one SDSF point based on whether the points of the categories, in combination, satisfy at least one first pre-selected criterion. The system may optionally include executable code that creates at least one SDSF trajectory based on whether a plurality of at least one SDSF points, in combination, satisfy at least one second pre-selected criterion.

[0009] Creating a graphed polygon may optionally include, but is not limited to, creating at least one convex polygon from at least one drivable surface, wherein at least one convex polygon includes an edge, smoothing the edge, forming a traverse margin based on the smoothed edge, adding at least one SDSF traverse to at least one drivable surface, and removing the edge from at least one drivable surface according to at least one third pre-selected criterion. Smoothing the edge may optionally include, but is not limited to, trimming the edge outwards, and may include executable code. Forming a traverse margin of the smoothed edge may optionally include, but is not limited to, trimming the outer edge inwards, and may include executable code.

[0010] In some configurations, the method of this teaching for navigating at least one SDSF encountered by a TD, wherein the TD travels along a path on a surface, the surface comprises at least one SDSF, the path comprises a start point and an end point, and the method may include, but is not limited to, accessing a route morphology. The route morphology may include at least one graphed polygon which may contain filtered point cloud data. The point cloud data may include labeled features and traversable margins. The method may include transforming the point cloud data into a global coordinate system, determining the boundary of at least one SDSF, creating an SDSF buffer of a pre-selected size around the boundary, determining which of the at least one SDSF is traversable based on at least one SDSF crossing criterion, creating an edge / weight graph based on at least one SDSF crossing criterion, the transformed point cloud data, and the route morphology, and selecting a path from the start point to the destination point based on at least the edge / weight graph.

[0011] At least one SDSF crossing criterion may optionally include a pre-selected width of at least one SDSF and a pre-selected smoothness of at least one SDSF, a minimum entry distance and a minimum exit distance between at least one SDSF and a TD, including a drivable surface, and a minimum entry distance between at least one SDSF and a TD that allows for an approximately 90° approach to at least one SDSF by the TD.

[0012] In some configurations, the system of this teaching is for navigating at least one SDSF encountered by a TD, wherein the TD travels along a path on a surface, the surface includes at least one SDSF, the path includes a start point and an end point, and the system may include a sixth processor for accessing the route morphology, which may include at least one graphed polygon containing filtered point cloud data, the point cloud data including labeled features and traversable margins. The system may include a seventh processor for converting the point cloud data to a global coordinate system, and an eighth processor for determining the boundary of at least one SDSF. The eighth processor may create an SDSF buffer of a pre-selected size around the boundary. The system may include at least a ninth processor that determines which of at least one SDSF is traversable based on at least one SDSF traversal criterion; at least a tenth processor that creates an edge / weight graph based on at least one SDSF traversal criterion, transformed point cloud data, and route morphology; and at least a base controller that selects a path from a starting point to a destination point based on the edge / weight graph.

[0013] In some configurations, the teachings describe a method for navigating at least one SDSF encountered by a TD, wherein the TD travels a path on a surface, the surface includes at least one SDSF, the path includes a start point and an end point, and the method may include, but is not limited to, accessing point cloud data representing the surface. The method may include filtering the point cloud data, forming the filtered point cloud data into a tadible portion, and merging the tadible portion into at least one concave polygon. The method may include locating and labeling the at least one SDSF within at least one concave polygon. Locating and labeling can form labeled point cloud data. The method may include creating a graphed polygon based on at least one concave polygon. The graphed polygon may form a route morphology, and the point cloud data may include labeled features and tadible margins. The method may include transforming point cloud data into a global coordinate system, determining the boundary of at least one SDSF, creating an SDSF buffer of a pre-selected size around the boundary, determining which of the at least one SDSF is traversable based on at least one SDSF crossing criterion, creating an edge / weight graph based on at least one SDSF crossing criterion, the transformed point cloud data, and the route morphology, and selecting a path from the starting point to the destination point based on the edge / weight graph.

[0014] Filtering point cloud data may optionally include conditionally removing points representing transient objects and outliers from the point cloud data, and replacing removed points with pre-selected heights. Forming processing portions may optionally include dividing the point cloud data into processing portions and removing points with pre-selected heights from the processing portions. Merging processing portions may optionally include reducing the size of the processing portions by analyzing outliers, voxels, and normals, expanding the area from the reduced-size processing portions, determining an initial drivable surface from the expanded area, subdividing and meshing the initial drivable surface, identifying the location of polygons within the subdivided and meshed initial drivable surface, and defining the drivable surface based on at least the polygons. Identifying and labeling the locations of at least one SDSF optionally includes sorting point cloud data of drivable surfaces according to an SDSF filter, wherein the SDSF filter includes points of at least three categories, and identifying the locations of at least one SDSF point based on whether the points of the category, in combination, satisfy at least one first pre-selected criterion. The method optionally includes creating at least one SDSF trajectory based on whether a plurality of at least one SDSF points, in combination, satisfy at least one second pre-selected criterion. Creating a graphed polygon further optionally includes creating at least one convex polygon from at least one drivable surface. The at least one convex polygon may include edges. Creating a graphed polygon may include smoothing the edges, forming a drivable margin based on the smoothed edges, adding at least one SDSF trajectory to at least one drivable surface, and removing edges from at least one drivable surface according to at least one third pre-selected criterion. Edge smoothing may optionally include trimming the edges outwards.Forming a smoothed edge travel margin may optionally include trimming the outer edge inward. At least one SDSF crossing criterion may optionally include a pre-selected width of at least one SDSF and a pre-selected smoothness of at least one SDSF, a minimum entry distance and minimum exit distance between at least one SDSF and TD including the travelable surface, and a minimum entry distance between at least one SDSF and TD that can accommodate an approximately 90° approach to at least one SDSF by TD.

[0015] In some configurations, the system of this teaching for navigating at least one SDSF encountered by a TD, wherein the TD travels along a path on a surface, the surface includes at least one SDSF, the path includes a start point and an end point, and the system includes, but is not limited to, a point cloud accessor for accessing point cloud data representing the surface, a first filter for filtering the point cloud data, a segmenter for forming a processable portion from the filtered point cloud data, a third processor for merging the processable portion into at least one concave polygon, a fourth processor for locating and labeling the at least one SDSF within the at least one concave polygon, the fourth processor for locating and labeling the point cloud data for forming the labeled point cloud data, and a fifth processor for creating a graphing polygon. The route configuration may include at least one graphing polygon which may include the filtered point cloud data. The point cloud data may include labeled features and drivable margins. The system may include a seventh processor that converts point cloud data to a global coordinate system, and an eighth processor that determines the boundary of at least one SDSF. The eighth processor can create an SDSF buffer of a pre-selected size around the boundary. The system may also include a ninth processor that determines which of at least one SDSF is traversable based on at least one SDSF traversal criterion, a tenth processor that creates an edge / weight graph based on at least one SDSF traversal criterion, the converted point cloud data, and the route morphology, and a base controller that selects a path from a starting point to a destination point based on at least the edge / weight graph.

[0016] The first filter may optionally include executable code that conditionally removes points representing transient objects and outliers from the point cloud data, and replaces the removed points with pre-selected heights. The segmenter may optionally include executable code that divides the point cloud data into processable parts, and removes points with pre-selected heights from the processable parts. The third processor may optionally include executable code that reduces the size of the processable parts by analyzing outliers, voxels, and normals, expands the area from the reduced-size processable parts, determines an initial drivable surface from the expanded area, subdivides and meshes the initial drivable surface, identifies the location of polygons within the subdivided and meshed initial drivable surface, and defines the drivable surface based on at least the polygons. The fourth processor may optionally include executable code that sorts point cloud data of drivable surfaces according to an SDSF filter, wherein the SDSF filter includes points of at least three categories, and identifies the location of at least one SDSF point based on whether the points of the categories, in combination, satisfy at least one first pre-selected criterion. The system may optionally include executable code that creates at least one SDSF trajectory based on whether a plurality of at least one SDSF points, in combination, satisfy at least one second pre-selected criterion.

[0017] Creating a graphed polygon may optionally include, but is not limited to, creating at least one convex polygon from at least one drivable surface, wherein at least one convex polygon includes an edge, smoothing the edge, forming a traverse margin based on the smoothed edge, adding at least one SDSF traverse to at least one drivable surface, and removing the edge from at least one drivable surface according to at least one third pre-selected criterion. Smoothing the edge may optionally include, but is not limited to, trimming the edge outwards, and may include executable code. Forming a traverse margin of the smoothed edge may optionally include, but is not limited to, trimming the outer edge inwards, and may include executable code.

[0018] In some configurations, the method of this teaching is for navigating a transport device (TD) along a path line in a travel area toward a target point that crosses at least one SDSF, wherein the TD includes a leading edge and a trailing edge, and the method may include, but is not limited to, receiving SDSF information and obstacle information relating to a travel area, detecting at least one candidate SDSF from the SDSF information, and selecting an SDSF line from at least one candidate SDSF line based on at least one selection criterion. The method may also include determining at least one traversable portion of the selected SDSF line based on at least one location of at least one obstacle found in the vicinity obstacle information of the selected SDSF line, orienting the TD toward the at least one traversable portion by redirecting the TD to travel along a line perpendicular to the traversable portion, and operating it at a first speed, and constantly correcting the direction of travel of the TD based on the relationship between the direction of travel and the perpendicular line. The method may also include driving the TD at a second speed by adjusting the first speed of the TD based on at least the direction of travel and the distance between the TD and the traversable portion. If the SDSF associated with at least one traversable portion is higher than the surface of the route, the method may include traversing the SDSF by raising the leading edge relative to the trailing edge and driving the TD at a third increased speed according to the degree of the rise, and driving the TD at a fourth speed until the TD has passed the SDSF.

[0019] Detecting at least one candidate SDSF from SDSF information may optionally include (a) drawing a closed polygon encompassing the location of the TD and the location of the target point; (b) drawing a path line between the target point and the location of the TD; (c) selecting two SDSF points from the SDSF information, wherein the SDSF points are located within the polygon; and (d) drawing an SDSF line between the two points. Detecting at least one candidate SDSF may also include (e) repeating steps (c)-(e) if there are fewer than a first pre-selected number of points within a first pre-selected distance of the SDSF line, and fewer than a second pre-selected number of attempts in selecting the SDSF points, drawing a line between them, and having fewer than a first pre-selected number of points around the SDSF line. Detecting at least one candidate SDSF may include (f) fitting the curve to SDSF points that are within a first pre-selected distance of the SDSF line if there are more than 71 pre-selected points, and (g) identifying the curve as an SDSF line if the first number of SDSF points within the first pre-selected distance of the curve exceeds the second number of SDSF points within the first pre-selected distance of the SDSF line, and the curve intersects the path line, and there are no gaps between SDSF points on the curve beyond the second pre-selected distance. Detecting at least one candidate SDSF may also include (h) repeating steps (f)-(h) if the number of points within the first pre-selected distance of the curve does not exceed the number of points within the first pre-selected distance of the SDSF line, or the curve does not intersect the path line, or there are gaps between SDSF points on the curve beyond the second pre-selected distance, and the SDSF line is not stable, and steps (f)-(h) have not been attempted more than the second pre-selected number of times.

[0020] A closed polygon may optionally include a pre-selected width, and the pre-selected width may optionally include the width dimension of the TD. The selection of SDSF points may optionally include random selection. At least one selection criterion may optionally include that a first number of SDSF points within a first pre-selected distance of the curve exceeds a second number of SDSF points within a first pre-selected distance of the SDSF line, that the curve intersects the path line, and that there are no gaps between SDSF points on the curve beyond a second pre-selected distance.

[0021] Determining at least one traversable portion of a selected SDSF optionally includes selecting multiple obstacle points from obstacle information. Each of the multiple obstacle points may include the probability that the obstacle point is associated with at least one obstacle. Determining at least one traversable portion may include projecting the multiple obstacle points onto the SDSF line to form at least one projection if the probability is higher than a pre-selected percentage, any of the multiple obstacle points are located between the SDSF line and the target point, and any of the multiple obstacle points are closer than a third pre-selected distance from the SDSF line. Determining at least one traversable portion may optionally include connecting at least two of the at least one projection to each other, identifying the location of the endpoints of the at least two connected projections along the SDSF line, marking the at least two connected projections as a non-traversable SDSF section, and marking the SDSF line outside the non-traversable section as at least one traversable section.

[0022] Crossing at least one traversable portion of the SDSF may optionally include turning the TD to travel along a line perpendicular to the traversable portion, orienting the TD toward the traversable portion and operating it at a first speed, constantly correcting the direction of travel of the TD based on the relationship between the direction of travel and the perpendicular line, and adjusting the first speed of the TD based on at least the direction of travel and the distance between the TD and the traversable portion to travel the TD at a second speed. Crossing at least one traversable portion of the SDSF may optionally include crossing the SDSF by raising the leading edge relative to the trailing edge if the SDSF is higher than the surface of the travel route, and traveling the TD at a third increased speed according to the degree of the rise, and traveling the TD at a fourth speed until the TD has passed the SDSF.

[0023] Alternatively, crossing at least one traversable portion of the SDSF may optionally include: (a) if the direction error is less than a third pre-selected amount relative to a line perpendicular to the SDSF line, ignoring updates to the SDSF information and driving the TD at a pre-selected speed; (b) if the rise of the front portion of the TD relative to the rear portion of the TD is between a sixth pre-selected amount and a fifth pre-selected amount, driving the TD forward and increasing the speed of the TD to an eighth pre-selected speed according to the degree of the rise; (c) if the rise of the front portion is less than a sixth pre-selected amount relative to the rear portion, driving the TD forward at a seventh pre-selected speed; and (d) if the rear portion is less than or equal to a fifth pre-selected distance from the SDSF line, repeating steps (a)-(d).

[0024] In some configurations, the wheels of the SDSF and TD can be automatically aligned to avoid system instability. Automatic alignment can be implemented, for example, by continuously testing and correcting the direction of travel of the TD as it approaches the SDSF. Another aspect of the SDSF crossing feature in this teaching is that the SDSF crossing feature automatically ensures that sufficient free space exists around the SDSF before attempting the crossing. Yet another aspect of the SDSF crossing feature in this teaching is that it is possible to cross SDSFs of various geometric shapes. Geometric shapes can include, for example, squares and contoured SDSFs. The orientation of the TD relative to the SDSF can determine the speed and direction of travel of the TD. The SDSF crossing feature can adjust the speed of the TD in the vicinity of the SDSF. When the TD approaches the SDSF, the speed can be increased to assist the TD in crossing the SDSF. The present invention provides, for example, the following: (Item 1) A method for navigating at least one substantially discontinuous surface feature (SDSF) encountered by a transport device (TD), wherein the TD travels along a path on the surface, the surface includes the at least one SDSF, and the path includes a start point and an end point. The aforementioned method, Accessing point cloud data representing the aforementioned surface, Filtering the aforementioned point cloud data, The filtered point cloud data is formed into a processable portion, Merging the aforementioned processable portion into at least one concave polygon, Identifying the location of the at least one SDSF within the at least one concave polygon and labeling them, wherein identifying and labeling the locations constitutes forming labeled point cloud data. At a minimum, a graphed polygon is created based on at least one of the concave polygons, Select the path from the starting point to the ending point based at least on the graphed polygon. including wherein the TD traverses the at least one SDSF along the path. (Item 2) Filtering the point cloud data conditionally removing points representing transient objects and points representing outliers from the point cloud data, replacing the removed points having a preselected height, The method according to item 1, including. (Item 3) Forming the processing part dividing the point cloud data into the processable parts, removing points of a preselected height from the processable parts, The method according to item 1, including. (Item 4) Merging the processable parts reducing the size of the processable parts by analyzing outliers, voxels, and normal vectors, enlarging regions from the processable parts of the reduced size, determining an initial drivable surface from the enlarged regions, dividing and meshing the initial drivable surface, identifying the positions of polygons within the divided and meshed initial drivable surface, setting at least one drivable surface based on at least the polygons, The method according to item 1, including. (Item 5) Identifying the positions of the at least one SDSF and labeling them sorting the point cloud data of the drivable surface according to an SDSF filter, the SDSF filter including at least three categories of points, At a minimum, the location of at least one SDSF point is determined based on whether the points of the at least three categories, in combination, satisfy at least one first pre-selected criterion. The method described in item 4, including the method described in item 4. (Item 6) The method according to item 5, further comprising, at least, creating at least one SDSF trajectory based on whether a plurality of the at least one SDSF points, in combination, satisfy at least one second pre-selected criterion. (Item 7) Creating the aforementioned graphed polygon is, Creating at least one convex polygon from the at least one drivable surface, wherein the at least one convex polygon includes an outer edge, Smoothing the outer edge, Based on the smoothed outer edge, a travel margin is formed, Adding the at least one SDSF track to the at least one traversable surface, Removing the inner edge from the at least one drivable surface according to at least one third pre-selected criterion The method described in item 6, further including the method described in item 6. (Item 8) The method of item 8, wherein the smoothing of the outer edge includes trimming the outer edge outwards to form an outer edge. (Item 9) The method of item 7, wherein forming the running margin of the smoothed outer edge includes trimming the outer edge inward. (Item 10) A system for navigating at least one substantially discontinuous surface feature (SDSF) encountered by a TD, wherein the TD travels along a path on the surface, the surface includes the at least one SDSF, and the path includes a start point and an end point. The aforementioned system, Sensors and, Map processor and Power base and, Device controller and Equipped with, The aforementioned device controller A first processor accesses point cloud data representing the surface, A filter for filtering the aforementioned point cloud data, A second processor that forms a processable portion from the filtered point cloud data, A third processor that merges the processable portion into at least one concave polygon, A fourth processor for identifying and labeling the positions of the at least one SDSF within the at least one concave polygon, wherein identifying and labeling the positions constitutes forming labeled point cloud data, A fifth processor that creates graphed polygons, A sixth processor that selects the path from the starting point to the ending point based at least on the graphed polygon, Includes, The TD is a system that traverses at least one SDSF along the path. (Item 11) The filter comprises a seventh processor, and the seventh processor is Conditionally remove points representing transient objects and points representing outliers from the aforementioned point cloud data, Replacing the removed point having a pre-selected height The system described in item 10 executes the code that includes the following: (Item 12) The aforementioned second processor is The point cloud data is divided into the processable portions, Removing a point of a pre-selected height from the aforementioned processable portion. The system described in item 10, including executable code that includes [the specified code]. (Item 13) The aforementioned third processor is By analyzing outliers, voxels, and normals, the size of the processable portion is reduced, Expanding the area from the reduced size of the processable portion, Determining the initial drivable surface from the aforementioned expanded region, The initial drivable surface is divided and meshed, Identifying the location of polygons within the divided and meshed initial drivable surface, Based on at least the polygon, at least one drivable surface is defined. The system described in item 10, including executable code that includes [the specified code]. (Item 14) The fourth processor is, The process involves sorting the point cloud data of the drivable surface according to an SDSF filter, wherein the SDSF filter includes points of at least three categories. At a minimum, the location of at least one SDSF point is determined based on whether the points of the at least three categories, in combination, satisfy at least one first pre-selected criterion. A system as described in item 13, including executable code that includes [the specified code]. (Item 15) The system according to item 14, wherein the fourth processor includes executable code that creates at least one SDSF trajectory based on whether a plurality of the at least one SDSF points, in combination, satisfy at least one second pre-selected criterion. (Item 16) Creating a graphed polygon is, Creating at least one convex polygon from the at least one drivable surface, wherein the at least one convex polygon includes an outer edge, Smoothing the outer edge, Based on the smoothed outer edge, a travel margin is formed, Adding the at least one SDSF track to the at least one traversable surface, Removing the inner edge from the at least one drivable surface according to at least one third pre-selected criterion The system described in item 13, which includes executable code including an eighth processor. (Item 17) The system according to item 16, wherein the ninth processor includes executable code that smooths the outer edge, which includes trimming the outer edge outwards to form an outer edge. (Item 18) The system according to item 17, wherein the tenth processor includes executable code that includes trimming the outer edge inward in order to form the running margin of the smoothed outer edge. (Item 19) A method for navigating at least one substantially discontinuous surface feature (SDSF) encountered by a transport device (TD), wherein the TD travels along a path on the surface, the surface includes the at least one SDSF, and the path includes a start point and an end point. The aforementioned method, Accessing a route configuration, wherein the route configuration includes at least one graphed polygon containing filtered point cloud data, the filtered point cloud data includes labeled features, and the point cloud data includes drivable margins. The point cloud data is converted to a global coordinate system, Determining the boundary of at least one substantially discontinuous surface feature (SDSF), Creating an SDSF buffer of a pre-selected size around the aforementioned boundary, At a minimum, determine which of the at least one SDSF is cross-sectional based on at least one SDSF cross-sectional criterion, At a minimum, create an edge / weight graph based on the at least one SDSF cross-sectional criterion, the converted point cloud data, and the root morphology. Select the path from the starting point to the ending point based at least on the edge / weight graph. Methods that include... (Item 20) The aforementioned at least one SDSF cross-sectional criterion is, The pre-selected width of the at least one SDSF and the pre-selected smoothness of the at least one SDSF, The minimum entry distance and minimum exit distance between the at least one SDSF including a drivable surface and the TD, Equipped with, The method according to item 19, wherein the minimum approach distance between the at least one SDSF and the TD allows the TD to approach the at least one SDSF at an angle of approximately 90°. (Item 21) A system for navigating at least one substantially discontinuous surface feature (SDSF) encountered by a transport device (TD), wherein the TD travels along a path on the surface, the surface includes the at least one SDSF, the path includes a start point and an end point, and the system A first processor for accessing a route configuration, wherein the route configuration includes at least one graphed polygon containing filtered point cloud data, the filtered point cloud data includes labeled features, and the point cloud data includes drivable margins, and the first processor A second processor that converts the point cloud data into a global coordinate system, A third processor for determining the boundary of at least one SDSF, wherein the third processor creates an SDSF buffer of a pre-selected size around the boundary. A fourth processor that determines which of the at least one SDSF is traversable based on at least one SDSF traversal criterion, A fifth processor that creates an edge / weight graph based on at least one SDSF cross-sectional criterion, the converted point cloud data, and the root morphology, A base controller that selects the path from the start point to the end point based at least on the edge / weight graph, A system equipped with these features. (Item 22) The aforementioned at least one SDSF cross-sectional criterion is, The pre-selected width of the at least one SDSF and the pre-selected smoothness of the at least one SDSF, The minimum entry distance and minimum exit distance between the at least one SDSF including a drivable surface and the TD, Equipped with, The system according to item 21, wherein the minimum approach distance between the at least one SDSF and the TD allows the TD to approach the at least one SDSF at an angle of approximately 90°. (Item 23) A method for navigating at least one substantially discontinuous surface feature (SDSF) encountered by a TD, wherein the TD travels along a path on the surface, the surface includes the at least one SDSF, and the path includes a start point and an end point. The aforementioned method, Accessing point cloud data representing the aforementioned surface, Filtering the aforementioned point cloud data, The filtered point cloud data is formed into a processable portion, Merging the aforementioned processable portion into at least one concave polygon, Identifying the location of the at least one SDSF within the at least one concave polygon and labeling them, wherein identifying and labeling the locations constitutes forming labeled point cloud data. At a minimum, the creation of a graphed polygon based on at least one concave polygon, wherein the graphed polygon forms a root shape. The point cloud data is converted to a global coordinate system, Determining the boundary of at least one SDSF, Creating an SDSF buffer of a pre-selected size around the aforementioned boundary, At a minimum, determine which of the at least one SDSF is cross-sectional based on at least one SDSF cross-sectional criterion, At a minimum, create an edge / weight graph based on the at least one SDSF cross-sectional criterion, the converted point cloud data, and the root morphology. Select the path from the starting point to the ending point based at least on the edge / weight graph. Methods that include... (Item 24) Filtering the aforementioned point cloud data is Conditionally remove points representing transient objects and points representing outliers from the aforementioned point cloud data, Replacing the removed point having a pre-selected height The method described in item 23, including the method described in item 23. (Item 25) Forming the processing part is The point cloud data is divided into the processable portions, Removing a point of a pre-selected height from the aforementioned processable portion. The method described in item 23, including the method described in item 23. (Item 26) Merging the aforementioned processable portions means By analyzing outliers, voxels, and normals, the size of the processable portion is reduced, Expanding the area from the reduced size of the processable portion, Determining the initial drivable surface from the aforementioned expanded region, The initial drivable surface is divided and meshed, Identifying the location of polygons within the divided and meshed initial drivable surface, Based on at least the polygon, at least one drivable surface is defined. The method described in item 23, including the method described in item 23. (Item 27) Identifying the location of at least one SDSF and labeling them is The process involves sorting the point cloud data of the drivable surface according to an SDSF filter, wherein the SDSF filter includes points of at least three categories. At a minimum, the location of at least one SDSF point is determined based on whether the points of the at least three categories, in combination, satisfy at least one first pre-selected criterion. The method described in item 26, including the method described in item 26. (Item 28) The method according to item 27, further comprising, at least, creating at least one SDSF trajectory based on whether a plurality of the at least one SDSF points combine to satisfy at least one second pre-selected criterion. (Item 29) Creating a graphed polygon is, Creating at least one convex polygon from the at least one drivable surface, wherein the at least one convex polygon includes an outer edge, Smoothing the outer edge, Based on the smoothed outer edge, a travel margin is formed, Adding the at least one SDSF track to the at least one traversable surface, Removing the inner edge from the at least one drivable surface according to at least one third pre-selected criterion The method described in item 28, further including the method described in item 28. (Item 30) The method according to item 29, wherein the smoothing of the outer edge includes trimming the outer edge outwards to form an outer edge. (Item 31) The method according to item 30, wherein forming the running margin of the smoothed outer edge includes trimming the outer edge inward. (Item 32) The aforementioned at least one SDSF cross-sectional criterion is, The pre-selected width of the at least one SDSF and the pre-selected smoothness of the at least one SDSF, The minimum entry distance and minimum exit distance between the at least one SDSF including a drivable surface and the TD, Equipped with, The method according to item 23, wherein the minimum approach distance between the at least one SDSF and the TD allows the TD to approach the at least one SDSF at an angle of approximately 90°. (Item 33) A system for navigating at least one substantially discontinuous surface feature (SDSF) encountered by a transport device (TD), wherein the TD travels along a path on the surface, the surface includes the at least one SDSF, and the path includes a start point and an end point. The aforementioned system, A first processor accesses point cloud data representing the surface, A first filter for filtering the point cloud data, A second processor that forms a processable portion from the filtered point cloud data, A third processor that merges the processable portion into at least one concave polygon, A fourth processor for identifying and labeling the positions of the at least one SDSF within the at least one concave polygon, wherein identifying and labeling the positions constitutes forming labeled point cloud data, A fifth processor that creates graphed polygons, A sixth processor for accessing a route configuration, wherein the route configuration includes at least one graphed polygon containing filtered point cloud data, the filtered point cloud data includes labeled features, and the point cloud data includes drivable margins, the sixth processor, A seventh processor that converts the point cloud data into a global coordinate system, An eighth processor that determines the boundary of at least one SDSF, wherein the eighth processor creates an SDSF buffer of a pre-selected size around the boundary, A ninth processor that determines which of the at least one SDSF is traversable based on at least one SDSF traversal criterion, A 10th processor that creates an edge / weight graph based on at least one SDSF cross-sectional criterion, the converted point cloud data, and the root morphology, A base controller that selects the path from the start point to the end point based at least on the edge / weight graph, A system equipped with these features. (Item 34) The first filter described above is Conditionally remove points representing transient objects and points representing outliers from the aforementioned point cloud data, Replacing the removed point having a pre-selected height A system as described in item 33, including executable code that includes [the specified code]. (Item 35) The aforementioned second processor is The point cloud data is divided into the processable portions, Removing a point of a pre-selected height from the aforementioned processable portion. A system as described in item 33, including executable code that includes [the specified code]. (Item 36) The aforementioned third processor is By analyzing outliers, voxels, and normals, the size of the processable portion is reduced, Expanding the area from the reduced size of the processable portion, Determining the initial drivable surface from the aforementioned expanded region, The initial drivable surface is divided and meshed, Identifying the location of polygons within the divided and meshed initial drivable surface, Based on at least the polygon, at least one drivable surface is defined. A system as described in item 33, including executable code that includes [the specified code]. (Item 37) The fourth processor is, The process involves sorting the point cloud data of the drivable surface according to an SDSF filter, wherein the SDSF filter includes points of at least three categories. At a minimum, the location of at least one SDSF point is determined based on whether the points of the at least three categories, in combination, satisfy at least one first pre-selected criterion. A system as described in item 36, including executable code that includes [the specified code]. (Item 38) The system according to item 37, comprising executable code that includes creating at least one SDSF trajectory based on whether a plurality of the at least one SDSF points, in combination, satisfy at least one second pre-selected criterion. (Item 39) Creating a graphed polygon is, Creating at least one convex polygon from the at least one drivable surface, wherein the at least one convex polygon includes an outer edge, Smoothing the outer edge, Based on the smoothed outer edge, a travel margin is formed, Adding the at least one SDSF track to the at least one traversable surface, Removing the inner edge from the at least one drivable surface according to at least one third pre-selected criterion A system as described in item 38, including executable code that includes [the specified element]. (Item 40) The system according to item 39, wherein the smoothing of the outer edge includes executable code that trims the outer edge outwards to form an outer edge. (Item 41) The system according to item 40, comprising executable code that includes trimming the outer edge inward to form the running margin of the smoothed outer edge. (Item 42) The aforementioned at least one SDSF cross-sectional criterion is, The pre-selected width of the at least one SDSF and the pre-selected smoothness of the at least one SDSF, The minimum entry distance and minimum exit distance between the at least one SDSF including a drivable surface and the TD, Equipped with, The system according to item 33, wherein the minimum approach distance between the at least one SDSF and the TD allows the TD to approach the at least one SDSF at an angle of approximately 90°. (Item 43) A method for navigating a transport device (TD) along a path line in a travel area toward a target point traversing at least one substantially discontinuous surface feature (SDSF), wherein the TD includes a leading edge and a trailing edge, and the SDSF includes an SDSF point. The aforementioned method, Receiving SDSF information and obstacle information related to the aforementioned area of ​​travel, To detect at least one candidate SDSF from the aforementioned SDSF information, Selecting an SDSF line from the at least one candidate SDSF based on at least one selection criterion, Based on the location of at least one obstacle found in the obstacle information near the selected SDSF line, at least one traversable portion of the selected SDSF line is determined. By redirecting the TD so that it travels along a path perpendicular to the at least one traversable portion, the TD is directed toward the at least one traversable portion and operated at a first speed. Based on the relationship between the direction of travel and the travel path, the direction of travel of the TD is always corrected, The TD is driven at a second speed by adjusting the first speed of the TD based at least on the direction of travel and the distance between the TD and the at least one traversable portion. If the SDSF associated with the at least one traversable portion is higher than the surface of the area of ​​travel, the SDSF is traversed by raising the leading edge relative to the trailing edge and driving the TD at a third increased speed according to the degree of the rise. The TD is driven at a fourth speed until it passes the SDSF. Methods that include... (Item 44) Detecting at least one candidate SDSF from the aforementioned SDSF information is: (a) Drawing a closed polygon that encloses the TD location of the TD and the target point location of the target point, (b) Drawing a straight line between the target point location and the TD location, (c) Selecting two of the SDSF points from the SDSF information, wherein the two SDSF points are located within the closed polygon. (d) Drawing an SDSF line between the two SDSF points, (e) If there are fewer points than the first pre-selected number within the first pre-selected distance of the SDSF line, and fewer than the second pre-selected number of trials have been performed in selecting the SDSF points, repeat steps (c)-(e), (f) If there are more than the number of points specified in the first pre-selected, the curve is fitted to the SDSF points that are included within the first pre-selected distance of the SDSF line, (g) If the first number of SDSF points within the first pre-selected distance of the curve exceeds the second number of SDSF points within the first pre-selected distance of the SDSF line, and the curve intersects the straight line, and there is no gap between the SDSF points on the curve that exceeds the second pre-selected distance, the curve is identified as the SDSF line. (h) If the first number of points within the first pre-selected distance of the curve does not exceed the second number of points within the first pre-selected distance of the SDSF line, or if the curve does not intersect the straight line, or if there is a gap between the SDSF points on the curve that exceeds the second pre-selected distance, and the SDSF line is not stable, and steps (f)-(h) have not been attempted more than the second pre-selected number of times, then steps (f)-(h) are repeated. The method described in item 43, including the method described in item 43. (Item 45) The method according to item 44, wherein the closed polygon may include a pre-selected width. (Item 46) The method according to item 45, wherein the pre-selected width has the width dimension of the TD. (Item 47) The selection of the aforementioned SDSF points is the method described in item 44, including random selection. (Item 48) The aforementioned at least one selection criterion is, The first number of SDSF points within the first pre-selected distance of the curve exceeds the second number of SDSF points within the first pre-selected distance of the SDSF line, The curve intersects the straight line, The gap between the SDSF points on the curve does not exceed a second pre-selected distance. The method described in item 44, which includes the features of item 44. (Item 49) Determining at least one traversable portion of the selected SDSF is: Selecting a plurality of obstacle points from the aforementioned obstacle information, wherein each of the plurality of obstacle points includes the probability that each of the plurality of obstacle points is associated with at least one obstacle, If the aforementioned probability is higher than a pre-selected percentage, and any of the plurality of obstacle points is located between the SDSF line and the target point, and any of the plurality of obstacle points is closer than a third pre-selected distance from the SDSF line, the plurality of obstacle points are projected onto the SDSF line to form at least one projection, Connecting at least two of the aforementioned at least one projections to each other, Identifying the location of the endpoints of the at least two connected projections along the SDSF line, The at least two connected projections are marked as non-crossable sections, The SDSF line outside the non-crossable section is marked as at least one crossable section. The method described in item 43, including the method described in item 43. (Item 50) A method for identifying the location of a feature from a camera image received by a robot having a certain posture, wherein the method is: The process involves receiving camera images from sensors mounted on the robot, wherein each of the camera images includes an image timestamp, and each of the camera images has image color pixels and image depth pixels. Receiving the posture of the robot, wherein the posture has a posture timestamp. The selected camera image is determined by identifying one of the camera images having the image timestamp closest to the aforementioned posture timestamp. In the selected camera image, the image color pixels are separated from the image depth pixels, The image surface classification of the selected camera image is determined by providing the image color pixels to a first machine learning model and the image depth pixels to a second machine learning model. Determining the outer perimeter points of a feature in the selected camera image, wherein the feature includes feature pixels within the perimeter defined by the outer perimeter points, each of the feature pixels having the same surface classification, and each of the outer perimeter points having a set of coordinates. Convert each of the aforementioned sets of coordinates to UTM coordinates. Methods that include... (Item 51) A method for including at least one substantially discontinuous surface feature (SDSF) in a map, wherein the map is used by a transport device, the transport device traverses the at least one SDSF, and the method is The process involves removing transient data from a point cloud dataset, wherein the removal process results in processed point cloud data. The processed point cloud data is divided into sections having a pre-selected number of points, The process involves removing points from the divided and processed point cloud data, wherein the removed points have pre-selected height values, and the removal process forms divided point cloud data. From the divided point cloud data, a concave polygon having a pre-selected size is generated, The points within the aforementioned concave polygon are organized into a pre-selected number of categories, Applying a first pre-selected criterion to the category to form a filtered category, The filtered categories are averaged, Connecting the averaged and filtered categories together to form at least one SDSF, The concave polygon is combined with at least one SDSF to form a merged polygon, Filter the merged polygons according to a second pre-selected criterion, To provide the aforementioned map to the transport device Methods that include... (Item 52) The method according to item 51, further comprising categorizing the points within the concave polygon as upper donut points, lower donut points, and cylindrical points. [Brief explanation of the drawing]

[0025] This instruction will be easier to understand by referring to the following explanation, which is assumed to be accompanied by the diagrams.

[0026] [Figure 1] Figure 1 is a schematic block diagram of the system of this instruction for preparing the progress path for TD.

[0027] [Figure 2] Figure 2 is a diagram illustrating an exemplary configuration of a device incorporating the system described in this instruction.

[0028] [Figure 3] Figure 3 is a schematic block diagram of the map processor described in this instruction.

[0029] [Figure 4] Figure 4 is a diagram illustrating the first part of the map processor flow in this instruction.

[0030] [Figure 5] Figure 5 shows the divided point cloud image from this instruction.

[0031] [Figure 6] Figure 6 is a diagram illustrating the second part of the map processor in this instruction.

[0032] [Figure 7] Figure 7 shows an image of the drivable surface detection results from this instruction.

[0033] [Figure 8] Figure 8 is a diagram illustrating the flow chart of the SDSF detector described in this instruction.

[0034] [Figure 9] Figure 9 is a diagram illustrating the SDSF categories in this instruction.

[0035] [Figure 10] Figure 10 shows an image of the SDSF identified by the system in this instruction.

[0036] [Figure 11A] Figures 11A and 11B are diagrams illustrating the polygon processing described in this instruction. [Figure 11B] Figures 11A and 11B are diagrams illustrating the polygon processing described in this instruction.

[0037] [Figure 12] Figure 12 shows images of polygons and SDSFs identified by the system of this instruction.

[0038] [Figure 13] Figure 13 is a schematic block diagram of the device controller described in this instruction.

[0039] [Figure 14] Figure 14 is a schematic block diagram of the SDSF processor described in this instruction.

[0040] [Figure 15] Figure 15 is an image of the SDSF approach identified by the system in this instruction.

[0041] [Figure 16] Figure 16 shows an image of the route configuration created by the system described in this instruction.

[0042] [Figure 17] Figure 17 is a schematic block diagram of the modes of instruction.

[0043] [Figure 18A] Figures 18A-18E are flowcharts of the teaching method for traversing the SDSF. [Figure 18B] Figures 18A-18E are flowcharts of the teaching method for traversing the SDSF. [Figure 18C] Figures 18A-18E are flowcharts of the teaching method for traversing the SDSF. [Figure 18D] Figures 18A-18E are flowcharts of the teaching method for traversing the SDSF. [Figure 18E] Figures 18A-18E are flowcharts of the teaching method for traversing the SDSF.

[0044] [Figure 19] Figure 19 is a schematic block diagram of the system for traversing the SDSF in this instruction.

[0045] [Figure 20A] Figures 20A-20C are pictorial representations of the method shown in Figures 18A-18C. [Figure 20B] Figures 20A-20C are pictorial representations of the method shown in Figures 18A-18C. [Figure 20C] Figures 20A-20C are pictorial representations of the method shown in Figures 18A-18C.

[0046] [Figure 21] Figure 21 is a pictorial representation of an image being transformed into a polygon. [Modes for carrying out the invention]

[0047] The SDSF cross-section features of this teaching can utilize TDs, for example, autonomous or semi-autonomous devices, to navigate within environments that may include features such as SDSFs. The SDSF cross-section features can enable TDs to travel across a variety of extended surfaces. In particular, SDSFs can be precisely identified and labeled so that TDs can automatically maintain their performance during entry into and exit from SDSFs, and TD speed, mode, and direction can be controlled for safe SDSF crossing.

[0048] Referring here to Figure 1, the system 100 for managing the SDSF crossing may include TD101, core cloud infrastructure 103, TD service 105, device controller 111, sensor 701, and power base 112. TD101 can transport goods and / or people, for example, to a destination, following a route dynamically determined to be modified by incoming sensor information. TD101 may include devices having autonomous modes, devices capable of operating fully autonomously, devices capable of operating at least partially remotely, and combinations of these features. TD service 105 can provide the device controller 111 with drivable surface information including features. The device controller 111 can modify the drivable surface information according to, for example, incoming sensor information and feature crossing requirements, and based on the modified drivable surface information, it can select a route for TD101. The device controller 111 can present commands to the power base 112, instructing it to provide speed, direction, and vertical movement commands to the wheel motors and cluster motors, which in turn cause the TD 101 to follow a selected route and, accordingly, raise and lower its cargo. The TD service 105 can access route-related information from the core cloud infrastructure 103, which may include, but is not limited to, storage and content delivery equipment. In some configurations, the core cloud infrastructure 103 may include, for example, AMAZON WEB SERVICES®, GOOGLE CLOUD TM This may include commercial products such as ORACLE CLOUD®.

[0049] Referring here to Figure 2, the exemplary TD, which may include the device controller 111 (Figure 1) and map processor 104 (Figure 1) of this teaching, may include, for example, a power base assembly, such as a power base, which is fully described in, for example, U.S. Patent Application No. 16 / 035,205, filed July 13, 2018, titled "Mobility Device," or U.S. Patent No. 6,571,892, filed August 15, 2001, titled "Control System and Method" (both of which are incorporated herein by reference as a whole). The exemplary power base assembly is described herein not to limit this teaching, but rather to highlight the features of any power base assembly that may be useful in implementing the art of this teaching. The exemplary power base assembly may optionally include a power base 112, a wheel cluster assembly 21100, and a payload carrier height assembly 30068. The exemplary power base assembly can optionally provide electrical and mechanical power to drive the wheel 21203 and the cluster 21100, which can raise and lower the wheel 21203. The power base 112 can control the rotation of the cluster assembly 21100 and the raising and lowering of the payload carrier height assembly 30068 to support the substantial discontinuous surface crossing of this teaching. Other such devices can also be used to adapt the SDSF detection and crossing of this teaching.

[0050] Continuing to refer to Figure 1, in some configurations, internal sensors in the exemplary power base can detect the orientation of the TD101 and the rate of change of orientation, the motors can enable servo operation, and the controller can understand the information from the internal sensors and motors. Appropriate motor commands can be calculated to achieve transporter performance and implement path-following commands. Left and right wheel motors can drive the wheels on both sides of the TD101 (Figure 1). In some configurations, the front and rear wheels can be coupled to drive together, such that the two left wheels can drive together and the two right wheels can drive together. In some configurations, turning can be accomplished by driving the left and right motors at different rates, and the cluster motors can rotate the wheelbase in the forward / rear direction. This can allow the TD101 to remain level while the front wheels are higher or lower than the rear wheels. This feature may be useful, for example, when ascending or descending an SDSF. The payload carrier 173 can be automatically ascended and descended based at least on the lower terrain.

[0051] Continuing to refer to Figure 1, in some configurations, the point cloud data may include route information relating to the area that TD101 should traverse. Possibly, point cloud data collected by a mapping device similar to or identical to TD101 can be time-tagged. The path along which the mapping device traverses can be referred to as the mapped trajectory. The point cloud data processing described herein can be performed as the mapping device traverses the mapped trajectory or after point cloud data collection is complete. After the point cloud data has been collected, it can undergo point cloud data processing, which may include initial filtering and point reduction, point cloud segmentation, and feature detection, as described herein. In some configurations, the core cloud infrastructure 103 can provide long-term or short-term storage for the collected point cloud data and can provide the data to the TD service 105. The TD service 105 can select from possible point cloud datasets to find a dataset that covers the area surrounding a desired starting point and a desired destination for TD101. The TD service 105 may include, but is not limited to, a map processor 104 capable of reducing the size of point cloud data and determining the features represented within the point cloud data. In some configurations, the map processor 104 can determine the location of SDSFs from the point cloud data. In some configurations, polygons can be created from the point cloud data as a technique for dividing the point cloud data and ultimately defining drivable surfaces. In some configurations, SDSF detection and drivable surface determination can proceed in parallel. In some configurations, SDSF detection and drivable surface determination can proceed sequentially.

[0052] Referring here to Figure 3, in some configurations, the map processor 104 may include feature extraction, which may include, but is not limited to, line-of-sight filtering 121 of point cloud data 131 and mapped trajectories 133. Line-of-sight filtering can remove points that are hidden from the direct line of sight of the sensor that collects the point cloud data and forms the mapped trajectories. The reduced point cloud data 132 may further be processed by organizing (151) the reduced point cloud data 132 according to pre-selected criteria associated with specific features. In some configurations, the organized point cloud data and mapped trajectories 133 may further be processed by removing transients (153) by any number of methods, including the methods described herein. Transients can complicate processing, in particular, if the specific features are stationary. The processed point cloud data 135 can be divided into processable chunks. In some configurations, the processed point cloud data 135 can be divided into segments having a pre-selected minimum number of points, for example, approximately 100,000 points (155). In some configurations, further point reduction is based on pre-selected criteria that may be relevant to the features to be extracted. For example, if points above a certain height are not important for locating a feature, those points can be removed from the point cloud data. In some configurations, the height of at least one of the sensors collecting the point cloud data can be considered the origin, and points above the origin can be removed from the point cloud data, for example, because only points of interest are associated with surface features. After the filtered point cloud data 135 has been divided, it forms segments 137, and the remaining points can be divided into drivable surface segments, and surface features can be localized. In some configurations, localizing a drivable surface can include, for example, generating polygons 139 as described herein (161) (161). In some configurations, locating surface features may include, for example, generating SDSF lines 141 (163) as described herein, but not limited to these.In some configurations, creating a dataset that can be further processed to generate the actual paths that TD101 (Figure 1) may take may involve combining polygons 139 and SDSF141 (165).

[0053] Here, primarily referring to Figure 4, removing transient objects from the point cloud data 131 (Figure 3) to the mapped trajectory 133, such as an exemplary timestamped point 751, 153 (Figure 3) may include casting a ray 753 from a timestamped point on the mapped trajectory 133 to each timestamped point in the point cloud data 131 (Figure 3) that has substantially the same timestamp. If the ray 753 intersects a point between the timestamped point on the mapped trajectory 133 and the endpoint of the ray 753, for example, point D755, then the intersection point D755 can be assumed to have entered the point cloud data between different sweeps of the camera. The intersection point, for example, intersection point D755, can be assumed to be part of a transient object and can be removed from the reduced point cloud data 132 (Figure 3) as not representing fixed features such as SDSF. The result is, for example, processed point cloud data 135 (Figure 3) without transient objects, for example, but not limited to. Points that were removed as part of a transient object but are also substantially at ground level can be returned to the processed point cloud data 135 (Figure 3) (754). Transient objects cannot contain certain features, such as, but are not limited to, SDSF141 (Figure 3), and therefore, when SDSF141 (Figure 3) is a detected feature, it can be removed without interfering with the integrity of the point cloud data 131 (Figure 3).

[0054] Continuing to refer to Figure 4, dividing the processed point cloud data 135 (Figure 3) into 757 sections 155 (Figure 3) can generate sections 757 having a square 154 (Figure 5) with a pre-selected size and shape, e.g., a minimum pre-selected side length, and containing approximately 100,000 points. From each section 757, points that are not necessarily related to the specific task, e.g., points located above the pre-selected points, can be removed to reduce the dataset size (157) (Figure 3). In some configurations, the pre-selected points may be at the height of TD101 (Figure 1). Removing these points can lead to more efficient processing of the dataset.

[0055] Again, primarily referring to Figure 3, the map processor 104 can supply the device controller 111 with at least one dataset that can be used to generate direction, velocity, and height commands to TD101 (Figure 1). At least one dataset may contain points that can be connected to other points in the dataset, and each line connecting points in the dataset traverses a traversable surface. To determine such root points, the divided point cloud data 137 can be divided into polygons 139, and the vertices of the polygons 139 can potentially become root points. The polygons 139 may include features such as, for example, SDSF141.

[0056] Here, primarily referring to Figure 6, in some configurations, the point cloud data 131 (Figure 3) can be processed by removing outliers using conventional means such as statistical analysis techniques, for example, those available in the Point Cloud Library, http: / / pointclouds.org / documentation / tutorials / statistical_outlier.php, for example. Filtering may include reducing the size of segments 137 (Figure 3) by conventional means, including, for example, a voxelized grid approach, as available in the Point Cloud Library, http: / / pointclouds.org / documentation / tutorials / voxel_grid.php. The divided point cloud data 137 (Figure 3) can be used to generate a concave polygon 759, for example, a 5m × 5m polygon (161) (Figure 3). Concave polygons 759 can be created, for example, by the process described at http: / / pointclouds.org / documentation / tutorials / hull_2d.php (but not limited to), or by the process described in "A New Concave Hull Algorithm and Concaveness Measure for n-dimensional Datasets" by Park et al., Journal of Information Science and Engineering 28, pp. 587-600, 2012.

[0057] Continuing, primarily with reference to Figure 6, in some configurations, creating the processed point cloud data 135 (Figure 3) may include filtering voxels. To reduce the number of points that will undergo future processing, in some configurations, the centroid of each voxel in the dataset may be used to approximate the points within the voxel, and all points other than the centroid may be excluded from the point cloud data. In some configurations, the center of a voxel may be used to approximate the points within the voxel. Other methods for reducing the size of the filtered segment 251 may also be used, such as taking a random point subsample so that a fixed number of points, selected uniformly and randomly, may be excluded from the filtered segment 251.

[0058] Referring again to Figure 3, in some configurations, creating processed point cloud data 135 may involve calculating normals from a dataset from which outliers have been removed and which has been resized through voxel filtering. The normals for each point in the filtered dataset can be used for various processing possibilities, including curve reconstruction algorithms. In some configurations, estimating and filtering normals in the dataset may involve obtaining the underlying surface from the dataset using a surface meshing technique and calculating normals from the surface mesh. In some configurations, estimating normals may involve using approximations to infer surface normals directly from the dataset, such as determining the normals to a fitting plane obtained by applying the total least squares method to the k nearest neighbors of a point, for example, but not limited to. In some configurations, the value of k may be selected based on at least empirical data. Filtering normals may involve removing any normals greater than approximately 45° from those perpendicular to the xy plane. In some configurations, the filter may be used to align normals in the same direction. If a portion of the dataset represents a planar surface, redundant information contained within adjacent normals can be filtered out either by performing random subsampling or by filtering out one point from the set of related points. In some configurations, selecting points may involve recursively decomposing the dataset into boxes until each box contains at most k points. A single normal can be calculated from the k points within each box.

[0059] Continuing to refer to Figure 3, in some configurations, creating the processed point cloud data 135 may involve expanding the region within the dataset by clustering points that geometrically fit the surface representing the dataset, and refining the surface as the region expands to obtain the best approximation of the maximum number of points. Region expansion can merge points in terms of smoothness constraints. In some configurations, the smoothness constraints may be determined empirically, for example, or based on a desired surface smoothness. In some configurations, the smoothness constraints may range from approximately 10π / 180 to approximately 20π / 180. The output of region expansion is a set of point clusters, each point cluster being a set of points, each of which is considered to be part of the same smooth surface. In some configurations, region expansion may be based on a comparison of angles between normals. Region expansion can be carried out, for example, by algorithms such as region-growing segmentation (http: / / pointclouds.org / documentation / tutorials / region_growing_segmentation.php) and cluster-extraction (http: / / pointclouds.org / documentation / tutorials / cluster_extraction.php#cluster-extraction), although these are not limited to the above.

[0060] Here, primarily referring to Figure 7, in some configurations, the processed point cloud data 135 (Figure 3) can be used to determine the initial drivable surface 265. Region expansion can generate point clusters that may contain points that are part of the drivable surface. In some configurations, a reference plane can be fitted to each of the point clusters to determine the initial drivable surface. In some configurations, point clusters can be filtered according to the relationship between the orientation of the point cluster and the reference plane. For example, if the angle between the point cluster plane and the reference plane is, for example, less than approximately 30°, a point cluster can be considered to be part of the initial drivable surface. In some configurations, point clusters can be filtered based on size constraints, for example, less than 20% of the total points in the point cloud data 131 can be considered too large, and point clusters smaller than approximately 0.1% of the total points in the point cloud data 131 can be considered too small. The initial drivable surface may include the filtered point clusters. In some configurations, point clusters can be separated for further processing by one of several known methods. In some configurations, density-based spatial clustering of noisy applications (DBSCAN) can be used to separate point clusters, while in others, k-means clustering can be used. DBSCAN can group together points that are densely clustered together and mark points that are substantially isolated or in low-density regions as outliers. To be considered densely clustered, points must be located within a pre-selected distance from candidate points. In some configurations, a scale factor with respect to the pre-selected distance can be determined empirically or dynamically. In some configurations, the scale factor may be in the range of approximately 0.1 to 1.0.

[0061] Again, primarily referring to Figure 6, the resulting point subclusters can be transformed into concave polygons 759, for example, using meshing. Meshing can be performed by standard methods such as, but are not limited to, marching cubes, marching tetrahedra, surface nets, greedy meshing, and double contour formation. In some configurations, concave polygons 759 can be generated by projecting the local neighborhoods of points along the normals of the points and connecting the unconnected points. The resulting concave polygons 759 can be based on at least the size of the neighborhoods, the maximum allowable distance between the points to be considered, the maximum edge length between the polygons, the minimum and maximum angles of the polygons, and the maximum deviation that the normals can take from each other. In some configurations, concave polygons 759 can be filtered according to whether the concave polygons 759 would be too small for TD101 (Figure 1) to pass through. In some configurations, circles the size of TD101 (Figure 1) can be dragged around each of the concave polygons 759 by known means. If the circles substantially fit within the concave polygons 759, then the concave polygons 759, and therefore the resulting drivable surface, can be adapted to TD101 (Figure 1). In some configurations, the area of ​​the concave polygons 759 can be compared to the occupied area of ​​TD101 (Figure 1). The polygons can be assumed to be irregular, and therefore the first thing to do to determine the area of ​​the concave polygons 759 is to separate the concave polygons 759 into regular polygons 759A by known methods. For each regular polygon 759A, a standard area equation can be used to determine its size. The areas of each regular polygon 759A can be added together to find the area of ​​the concave polygon 759, and that area can be compared to the occupied area of ​​TD101 (Figure 1). The filtered concave polygons may include a subset of concave polygons that meet size criteria. The filtered concave polygons can be used to define the final drivable surface.

[0062] Primarily referring to Figure 8, generating SDSF lines 163 (Figure 3) can include identifying the location of the SDSF by further filtering of the concave polygon 759 (Figure 6). In some configurations, points from the point cloud data constituting the polygon can be categorized as either upper donut points 351 (Figure 9), lower donut points 353 (Figure 9), or cylindrical points 355 (Figure 9). Upper donut points 351 (Figure 9) can correspond to the shape of the SDSF model 352 furthest from the ground. Lower donut points 353 (Figure 9) can correspond to the shape of the SDSF model 352 closest to the ground, or at ground level. Cylindrical points 355 (Figure 9) can correspond to the shapes between upper donut points 351 (Figure 9) and lower donut points 353 (Figure 9). Combinations of categories can form a donut 371. To determine whether a donut 371 forms an SDSF, certain criteria are tested. For example, in each donut 371, there must be a minimum number of points that are upper donut points 351 (Figure 9) and a minimum number that are lower donut points 353 (Figure 9). In some configurations, the minimum value can be chosen empirically and may fall in the range of approximately 5 to 20. Each donut 371 can be divided into multiple parts, for example, two hemispheres. Another criterion for determining whether points within a donut 371 represent an SDSF is whether the majority of points are located within the opposing hemispheres of the parts of the donut 371. Cylindrical points 355 (Figure 9) can occur in either the first cylindrical region 357 (Figure 9) or the second cylindrical region 359 (Figure 9). Another criterion for SDSF selection is that there must be a minimum number of points within both cylindrical regions 357 / 359 (Figure 9). In some configurations, the minimum number of points can be empirically selected and may fall in the range of 3 to 20. Another criterion for SDSF selection is that the donut 371 must contain at least two points from three categories: the upper donut point 351 (Figure 9), the lower donut point 353 (Figure 9), and the cylindrical point 355 (Figure 9).

[0063] Continuing, primarily with reference to Figure 8, in some configurations, polygons can be processed in parallel. Each category worker 362 can search for its assigned polygon with respect to an SDSF point 789 (Figure 12) and assign the SDSF point 789 (Figure 12) to category 763 (Figure 6). As the polygons are processed, the resulting point categories 763 (Figure 6) can be combined (363) to form a combined category 366, and the categories can be shortened (365) to form a shortened combined category 368. Shortening the SDSF points 789 (Figure 12) may include filtering the SDSF points 789 (Figure 12) with respect to their distance from the ground. The shortened combined categories 368 are averaged by exploring the area around each SDSF point 766 (Figure 6) and generating an average point 765 (Figure 6), which can potentially be processed in parallel by an average worker 373, so that the points of the category can form a set of averaged donuts 375. In some configurations, the radius around each SDSF point 766 (Figure 6) can be determined empirically. In some configurations, the radius around each SDSF point 766 (Figure 6) can be in the range of 0.1m to 1.0m. The height change between one point and another on the SDSF orbit 377 (Figure 6) with respect to the SDSF at the average point 765 (Figure 6) can be calculated. Connecting the averaged donuts 375 together can generate the SDSF orbit 377 (Figure 6). When creating the SDSF trajectory 377 (Figures 6 and 10), if two next candidate points exist within the search radius of the starting point, the next point can be selected based on the fact that it forms a line as straight as possible between the previous line segment, the starting point, and the candidate destination point, and then the candidate next point represents the smallest change in SDSF height between the previous point and the candidate next point. In some configurations, the SDSF height can be defined as the difference between the heights of the upper donut 351 (Figure 9) and the lower donut 353 (Figure 9).

[0064] Here, primarily referring to Figure 11A, combining a concave polygon and an SDSF line 165 (Figure 3) can generate a dataset containing polygons 139 (Figure 3) and SDSF 141 (Figure 3), which can be manipulated to generate a graphed polygon using the SDSF data. Manipulating the concave polygon 263 may include, but is not limited to, merging the concave polygon 263 to form a merged polygon 771. Merging the concave polygon 263 can be done using known methods, such as, but is not limited to, those found at (http: / / www.angusj.com / delphi / clipper.php). The merged polygon 771 may be expanded to smooth its edges and form an expanded polygon 772. The expanded polygon 772 may be contracted to provide a running margin to form a contracted polygon 774, to which an SDSF track 377 (Figure 11B) may be added. Inward trimming (shrinking) can ensure that there is room for TD101 (Figure 1) to move near the edges by reducing the size of the traversable surface by at least a pre-selected amount based on the size of TD101 (Figure 1). Polygonal expansion and shrinking can be performed by commercially available techniques such as, for example, the ARCGIS® clip command (http: / / desktop.arcgis.com / en / arcmap / 10.3 / manage-data / editing-existing-features / clipping-a-polygon-feature.htm), for example.

[0065] Here, primarily referring to Figure 11B, the contracted polygon 774 can be partitioned into convex polygons 778, each of which can be traversed without encountering non-traversable surfaces. The contracted polygon 774 can be partitioned by conventional means such as ear slicing, which is optimized, for example, by z-order curve hashing and extended to handle holes, torsional polygons, degeneracy, and self-intersections. Commercially available ear slicing implementations can include, for example, those found at (https: / / github.com / mapbox / earcut.hpp). The SDSF trajectory 377 can include SDSF points 789 (Figure 15) that can be connected to polygon vertices 781. The vertices 781 can be considered possible path points that can be connected to each other to form possible travel paths for TD101 (Figure 1). In the dataset, the SDSF points 789 can be labeled in this way. As partitioning proceeds, redundant edges such as edges 777 and 779 may be introduced, for example, but are not limited to these. Removing one of edges 777 or 779 can reduce the complexity of further analysis and allow for the retention of a convex polygon mesh. In some configurations, the Hertel-Mehlhorn polygon partitioning algorithm can be used to remove edges and omit edges that have been labeled as features. The set of convex polygons 778, including the labeled features, can undergo further simplification to reduce the number of possible path points, which can be provided to the device controller 111 (Figure 1) in the form of annotated point data 379 (Figure 14).

[0066] Here, primarily referring to Figure 13, annotated point data 379 (Figure 14) can be provided to the device controller 111. The annotated point data 379 (Figure 14), which can be the basis for route information that can be used to instruct TD101 (Figure 1) to proceed along a path, may include, but are not limited to, navigable edges, mapped trajectories such as, but are not limited to, mapped trajectories 413 / 415 (Figure 16), and labeled features such as, but are not limited to, SDSF377 (Figure 15). The mapped trajectories 413 / 415 (Figure 15) may include a graph of edges in the route space and initial weights assigned to parts of the route space. The graph of edges may include, but are not limited to, characteristics such as directionality and capacity, and edges may be categorized according to these characteristics. The mapped trajectories 413 / 415 (Figure 15) may include cost modifiers associated with the surface of the route space and travel modes associated with the edges. The driving modes may include, but are not limited to, path following and SDSF climbing. Other modes may include, but are not limited to, autonomous, mapping, and wait for intervention. Ultimately, the path can be selected based on at least a lower cost modifier. Forms relatively far from the mapped trajectory 413 / 415 (Figure 15) may have a higher cost modifier and may not be of much interest when forming the path. The initial weights are adjusted while TD101 (Figure 1) is operating and may, as a possibility, cause a modification of the path. The adjusted weights can be used to adjust the edge / weight graph 381 (Figure 14) and may be based on at least the current driving mode, current surface, and edge category.

[0067] Continuing to refer to Figure 13, the device controller 111 may include a feature processor capable of performing specific tasks related to incorporating the eccentricity of any feature into the path. In some configurations, the feature processor may include, but is not limited to, an SDSF processor 118. In some configurations, the device controller 111 may include, but is not limited to, the SDSF processor 118, a sensor processor 703, a mode controller 122, and a base controller 114, each as described herein. The SDSF processor 118, the sensor processor 703, and the mode controller 122 may provide inputs to the base controller 114.

[0068] Continuing with Figure 13, the base controller 114 can determine, based on inputs provided by at least the mode controller 122, the SDSF processor 118, and the sensor processor 703, information that the power base 112 can use to drive the TD101 (Figure 1) along a path determined by the base controller 114 based on at least the edge / weight graph 381 (Figure 14). In some configurations, the base controller 114 can ensure that the TD101 (Figure 1) follows a predetermined path from a starting point to a destination and can modify the predetermined path based on at least external and / or internal conditions. In some configurations, external conditions may include, but are not limited to, stop signals, SDSFs, and obstacles within or near the path being traveled by the TD101 (Figure 1). In some configurations, internal conditions may include, but are not limited to, mode transitions that reflect the response of the TD101 (Figure 1) to the external conditions. The device controller 111 can determine commands to send to the power base 112 based on at least external and internal conditions. Commands may include, but are not limited to, velocity and direction commands that instruct TD101 (Figure 1) to move in a commanded direction at a commanded velocity. Other commands may include, for example, sets of commands that enable feature responses such as SDSF climbing. The base controller 114 may determine the desired velocity between waypoints of the path by conventional methods, but are not limited to, for example, the Interior Point Optimizer (IPOPT) large-scale nonlinear optimization (https: / / projects.coin-or.org / Ipopt). The base controller 114 may determine the desired path by conventional techniques, such as, for example, Dijkstra's algorithm, A* search algorithm, or techniques based on breadth-first search algorithms. The base controller 114 may form a box around the mapped trajectory 413 / 415 (Figure 15) to set an area where obstacle detection can be performed. The height of the payload carrier may, when adjustable, be adjusted at least in part based on the commanded velocity.

[0069] Continuing to refer to Figure 13, the base controller 114 can translate speed and direction determination into motor commands. For example, when encountering an SDSF such as a curb or slope, the base controller 114 can instruct the power base 112 to raise the payload carrier 173 (Figure 2), align the TD101 (Figure 1) with the SDSF at an angle of approximately 90°, and reduce the speed to a relatively low level in the SDSF climbing mode. When the TD101 (Figure 1) is climbing a substantially discontinuous surface, the base controller 114 can instruct the power base 112 to transition to a climbing phase in which the speed is increased because an increased torque is required to move the TD101 (Figure 1) up the slope. When the TD101 (Figure 1) encounters a relatively horizontal surface, the base controller 114 can reduce the speed to stay on any flat portion of the SDSF. In the case of a downhill ramp associated with a flat area, when TD101 (Figure 1) begins to descend a substantially discontinuous surface, and when both wheels are on the downhill ramp, the base controller 114 can allow the speed to increase. For example, when encountering an SDSF such as a slope, the slope can be identified and treated as a structure. The features of the structure may include, for example, a ramp of a pre-selected size. The ramp may include a slope of about 30° and may optionally be on both sides of a flat area. The device controller 111 (Figure 13) can distinguish between an obstacle and a slope by comparing the angle of the perceived feature with the expected slope ramp angle, the angle of which can be received from the sensor processor 703 (Figure 13).

[0070] Here, primarily referring to Figure 14, the SDSF processor 118 can identify the locations of navigable edges from blocks of traversable surfaces formed by polygonal meshes represented in annotated point data 379, which can be used to create paths for crossing by TD101 (Figure 1). Within an SDSF buffer 407 (Figure 15), which can form an area of ​​a pre-selected size around an SDSF line 377 (Figure 15), the navigable edges can be erased in preparation for special handling in the case of an SDSF crossing (see Figure 16). Closed line segments such as segment 409 (Figure 15) can be drawn to bisect the SDSF buffer 407 (Figure 15) between pairs of previously determined SDSF points 789 (Figure 12). In some configurations, since a closed line segment is considered a candidate for SDSF crossing, the segment end 411 (Figure 15) can be located in an unobstructed portion of the traversable surface, and there can be sufficient room for TD101 (Figure 1) to travel along the line segment between adjacent SDSF points 789 (Figure 12), and the area between the SDSF points 789 (Figure 12) can be the traversable surface. The segment end 411 (Figure 15) can be connected to the underlying morphology to form vertices and traversable edges. For example, line segments 461, 463, 465, and 467 (Figure 15) that satisfy the crossing criterion are shown as part of the morphology in Figure 16. In contrast, line segment 409 (Figure 15) did not satisfy the criterion because, at least, the segment end 411 (Figure 15) does not lie on the traversable surface. Overlapping SDSF buffers 506 (Figure 15) can indicate SDSF discontinuities, which can disadvantage SDSF crossings of SDSFs within the overlapping SDSF buffers 506 (Figure 15). SDSF lines 377 (Figure 15) can be smoothed, and the locations of SDSF points 789 (Figure 12) can be adjusted so that they are separated by a pre-selected distance, the pre-selected distance being based on the area occupied by at least TD101 (Figure 1).

[0071] Continuing to refer to Figure 14, the SDSF processor 118 can convert the annotated point data 379 into an edge / weight graph 381, including morphological correction for SDSF cross-sections. The SDSF processor 118 may include a seventh processor 601, an eighth processor 702, a ninth processor 603, and a tenth processor 605. The seventh processor 601 can convert the coordinates of points in the annotated point data 379 to a global coordinate system, achieve compatibility with GPS coordinates, and generate a GPS-compatible dataset 602. The seventh processor 601 can generate the GPS-compatible dataset 602 using conventional processes such as, for example, affine matrix transformations and PostGIS transformations, but is not limited to these. The World Geodetic System (WGS) can be used as the standard coordinate system because it takes into account the curvature of the Earth. Maps can be stored in the Universal Transverse Mercator (UTM) coordinate system and can be switched to WGS when it is necessary to find the location where a specific address is located.

[0072] Here, primarily referring to Figure 15, the eighth processor 702 (Figure 14) smooths the SDSF, determines the boundary of the SDSF 377, and creates a buffer 407 around the SDSF boundary, which can increase the cost modifier of the surface as it moves further away from the SDSF boundary. The mapped trajectories 413 / 415 may be special-case lanes with the lowest cost modifiers. Lower cost modifiers 406 can generally be located near the SDSF boundary, while higher cost modifiers 408 can generally be located relatively far from the SDSF boundary. The eighth processor 702 can provide point cloud data 704 (Figure 14) with costs to the ninth processor 603 (Figure 14).

[0073] Continuing, primarily with reference to Figure 15, the ninth processor 603 (Figure 14) can calculate an approximately 90° approach 604 (Figure 14) for TD101 (Figure 1) to traverse SDSF377 that meet the criteria for labeling them as traversable. The criteria may include the SDSF width and the SDSF smoothness. Line segments, such as line segments 409, can be created such that their lengths indicate the minimum approach distance that TD101 (Figure 1) may be required to approach SDSF377 and the minimum exit distance that may be required to exit SDSF377. Segment endpoints, such as endpoints 411, can be integrated with the underlying routing configuration. The criteria used to determine whether an SDSF approach is possible may exclude several approach possibilities. SDSF buffers, such as SDSF buffers 407, can be used to calculate valid approach and route configuration edge creation.

[0074] Again, primarily referring to Figure 14, the tenth processor 605 can create an edge / weight graph 381 from the morphology, which is an edge and weight graph developed herein, and can be used to compute a path through the map. The morphology may include cost modifiers and travel modes, and the edges may include directionality and capacity. The weights can be adjusted at runtime based on information from any number of sources. The tenth processor 605 can provide the base controller 114 with at least one sequence of ordered points, in addition to a recommended travel mode at a particular point, to enable path generation. Each point in each sequence of points represents the location and labeling of a possible path point on the processed traversable surface. In some configurations, the labeling may indicate that the point represents a part of the features that may be encountered along the path, such as, for example, an SDSF, etc. In some configurations, the features may be further labeled with suggested processing based on the type of feature. For example, in some configurations, if a path point is labeled as an SDSF, the further labeling may include a certain mode. The mode can be interpreted by TD101 (Figure 1) as a suggested travel command for TD101 (Figure 1), such as switching TD101 (Figure 1) to SDSF uphill mode 100-31 (Figure 17) to enable TD101 (Figure 1) to traverse SDSF377 (Figure 15).

[0075] Referring here to Figure 17, in some configurations, the mode controller 122 can provide instructions to the base controller 114 (Figure 13) for performing mode transitions. The mode controller 122 can establish the mode in which the TD101 (Figure 1) is progressing. For example, the mode controller 122 can provide the base controller 114 with a change in mode indication, for example, when the SDSF is identified along the progress path, it can be changed between the path-following mode 100-32 and the SDSF ascend mode 100-31. In some configurations, the annotated point data 379 (Figure 14) can include mode identifiers at various points along the route, for example, when the mode is changed to adapt the route. For example, if the SDSF 377 (Figure 15) is indicated in the annotated point data 379 (Figure 14), the device controller 111 can determine the mode identifier associated with the route point and, possibly, adjust the instructions to the power base 112 (Figure 13) based on the desired mode. In addition to the SDSF climbing mode 100-31 and path-following mode 100-32, in some configurations, the TD101 (Figure 1) can support operating modes including, but not limited to, a standard mode 21001 and an enhanced mode 100-2, which may include driving two drive wheels and two swivel wheels 21001 (Figure 2) in some configurations. Enhanced mode 100-2 can provide assistance by the TD101 (Figure 1) for traversing uneven terrain, various environments, steep slopes, and soft terrain, as described in detail in U.S. Patent No. 6,571,892 (No. '892), issued June 3, 2003, titled "Control System and Method" (which is incorporated herein by reference as a whole). In enhanced mode 100-2, all four drive wheels 21203 (Figure 2) can be deployed. Driving the four wheels 21203 (Figure 2) and ensuring an even weight distribution on the wheels 21203 (Figure 2) allows the TD101 (Figure 1) to travel up and down steep slopes and through many types of outdoor environments, including, but not limited to, gravel, sand, snow, and mud.The height of the payload carrier 173 (Figure 2) can be adjusted to provide the necessary clearance across obstacles and along slopes.

[0076] Referring here to Figure 18A, a method 1150 for navigating a TD toward a target point that traverses at least one SDSF may include, but is not limited to, receiving SDSF information related to the SDSF, the location of the target point, and the location of the TD 1151. The SDSF information may, but is not limited to, a set of points classified as SDSF points and the associated probability for each point that a point is an SDSF point, respectively. Method 1150 may include drawing a closed polygon encompassing the location of the TD and the location of the target point, and drawing a path line between the target point and the location of the TD 1153. The closed polygon may include a pre-selected width. Table I contains a possible range of pre-selected variables discussed herein. Method 1150 may include selecting two of the SDSF points located within the polygon 1155 and drawing an SDSF line between the two points 1157. In some configurations, the selection of SDSF points may be random or by any other method. If, in step 1159, there are fewer than the first pre-selected number of points within the first pre-selected distance of the SDSF line, and in step 1161, there are fewer than the second pre-selected number of attempts in selecting SDSF points, and a line is drawn between them, with fewer than the first pre-selected number of points around the SDSF line, then method 1150 may include returning to step 1155. If, in step 1161, there are more than the second pre-selected number of attempts in selecting SDSF points, and a line is drawn between them, with fewer than the first pre-selected number of points around the SDSF line, then method 1150 may include acknowledging that no SDSF line was detected 1163.

[0077] Here, primarily referring to Figure 18B, if in 1159 (Figure 18A) there are more than a first pre-selected number of points, method 1150 may include fitting the curve to points that are within a first pre-selected distance of the SDSF line 1165. If in 1167 the number of points within a first pre-selected distance of the curve exceeds the number of points within a first pre-selected distance of the SDSF line, and in 1171 the curve intersects a path line, and in 1173 there are no gaps between points on the curve that exceed a second pre-selected distance, method 1150 may include identifying the curve as an SDSF line 1175. If, in 1167, the number of points within a first pre-selected distance of the curve does not exceed the number of points within a first pre-selected distance of the SDSF line, or in 1171, the curve does not intersect the path line, or in 1173, there are gaps between points on the curve that exceed a second pre-selected distance, and in 1177, the SDSF line does not remain stable, and in 1169, the curve fit has not been attempted more than a second pre-selected number of times, then method 1150 may include returning to 1165. A stable SDSF line is the result of subsequent iterations that yield the same or fewer points.

[0078] Here, primarily referring to Figure 18C, if the curve fit has been attempted only a second pre-selected number of times in 1169 (Figure 18B), or if the SDSF line remains stable or degrades in 1177 (Figure 18B), method 1150 may include receiving occupy grid information 1179. The occupy grid can provide the probability of an obstacle being present at a point. The occupy grid information can enhance the SDSF and path information found within a polygon enclosing the TD path and SDSF, when the occupy grid includes data captured and / or calculated over a common geographical area with the polygon. Method 1150 may include selecting a point from the common geographical area and its associated probability 1181. Method 1150 may include projecting the obstacle onto the SDSF line 1187 if, in 1183, the probability that the obstacle is located at a selected point is greater than a pre-selected percentage, and in 1185, the obstacle is located between the TD and the target point, and in 1186, the obstacle is closer than a third pre-selected distance from the SDSF line between the SDSF line and the target point. Method 1150 may include resuming processing in 1179 if, in 1183, the probability that the location contains an obstacle is less than or equal to a pre-selected percentage, or in 1185, the obstacle is not located between the TD and the target point, or in 1186, the obstacle is located at a distance equal to or greater than a third pre-selected distance from the SDSF line between the SDSF and the target point, and in 1189, there are further obstacles to be processed.

[0079] Here, primarily referring to Figure 18D, if there are no further obstacles to be dealt with in 1189 (Figure 18C), method 1150 may include connecting the projections and finding the endpoints of the connected projections along the SDSF line 1191. Method 1150 may include marking a portion of the SDSF line between the projection endpoints as non-crossable 1193. Method 1150 may include marking a portion of the SDSF line outside the non-crossable section as crossable 1195. Method 1150 may include redirecting the TD to within a fifth pre-selected amount perpendicular to the crossable section of the SDSF line 1197. If, in 1199, the direction error relative to the line perpendicular to the crossable section of the SDSF line exceeds a first pre-selected amount, method 1150 may include decelerating the TD by a ninth pre-selected amount 1251. Method 1150 may include moving the TD forward toward the SDSF line and decelerating it by a second pre-selected amount per meter distance between the TD and the traversable SDSF line 1253. If, in 1255, the distance of the TD from the traversable SDSF line is less than a fourth pre-selected distance, and in 1257, the direction error is greater than or equal to a third pre-selected amount relative to a line perpendicular to the SDSF line, Method 1150 may include decelerating the TD by a ninth pre-selected amount 1252.

[0080] Here, primarily referring to Figure 18E, in 1257 (Figure 18D), if the direction error is less than a third pre-selected amount relative to a line perpendicular to the SDSF line, method 1150 may include ignoring the updated SDSF information and driving the TD at a pre-selected speed 1260. In 1259, if the rise of the front portion of the TD relative to the rear portion of the TD is between a sixth pre-selected amount and a fifth pre-selected amount, method 1150 may include driving the TD forward and increasing the speed of the TD to an eighth pre-selected amount according to the degree of the rise 1261. In 1263, if the rise of the front portion of the TD relative to the rear portion is less than a sixth pre-selected amount, method 1150 may include driving the TD forward at a seventh pre-selected speed 1265. In 1267, if the rear of the TD is greater than a fifth pre-selected distance from the SDSF line, method 1150 may include acknowledging that the TD has completed crossing the SDSF 1269. In 1267, if the rear of the TD is less than or equal to a fifth pre-selected distance from the SDSF line, method 1150 may include returning to 1260.

[0081] Referring here to Figure 19, the system 1100 for navigating a TD toward a target point traversing at least one SDSF may include, but is not limited to, a pathline processor 1103, an SDSF detector 1109, and an SDSF controller 1127. The system 1100 can be operably coupled with a surface processor 1601 capable of processing sensor information, which may, for example, include, but is not limited to, images of the vicinity of TD101 (Figure 20A). The surface processor 1601 can provide real-time surface feature updates, including indications of SDSFs. In some configurations, a camera may provide RGB-D data, which allows its points to be classified according to surface type. In some configurations, the system 1100 can process points classified as SDSFs and their associated probabilities. The system 1100 can be operably coupled with a system controller 1602 capable of managing aspects of the operation of TD101 (Figure 20A). The system controller 1602 can maintain an occupied grid 1138 which may include information from available sources regarding the navigable area in the vicinity of TD101 (Figure 20A). The occupied grid 1138 may include the probability of obstacles being present. This information, in conjunction with SDSF information, can be used to determine whether SDSF 377 (Figure 20C) can be traversed by TD101 (Figure 20A) without encountering obstacle 1681 (Figure 20B). Based on the environment and other information, the system controller 1602 can determine a speed limit 1148 that TD101 (Figure 20C) should not exceed. The speed limit 1148 can be used as a guideline for the speed set by system 1100, or it can be overridden. System 1100 can be operably coupled with a base controller 114 which can transmit travel commands 1144 generated by the SDSF controller 1127 to the travel components of TD101 (Figure 20A). The base controller 114 can provide the SDSF controller 1127 with information about the orientation of TD101 (Figure 20A) during the SDSF crossing.

[0082] Continuing to refer to Figure 19, the path line processor 1103 can continuously receive surface classification points 789 in real time, which may include, but are not limited to, points classified as SDSF. The path line processor 1103 can receive the location of the target point 1139 and the TD location 1141, which may be indicated by, for example, the center 1202 (Figure 20A) of the TD 101 (Figure 20A). The system 1100 may include a polygon processor 1105 that draws a polygon 1147 encompassing the TD location 1141, the location of the target point 1139, and the path 1214 between the target point 1139 and the TD location 1141. The polygon 1147 may include a pre-selected width. In some configurations, the pre-selected width may include the approximate width of the TD 101 (Figure 20A). The corresponding SDSF points 789 within the polygon 1147 can be identified.

[0083] Continuing to refer to Figure 19, the SDSF detector 1109 can receive surface classification points 789, paths 1214, polygons 1147, and target points 1139, and can determine the most suitable SDSF line 377 available in the incoming data according to the criteria described herein. The SDSF detector 1109 may include, but is not limited to, a point processor 1111 and an SDSF line processor 1113. The point processor 1111 may include selecting two of the SDSF points 789 located within polygon 1147 and drawing an SDSF line 377 between the two points. If there are fewer than a first pre-selected number of points within a first pre-selected distance of the SDSF line 377, and if there are fewer than a second pre-selected number of attempts in selecting an SDSF point 789, and a line is drawn between the two points, and there are fewer than a first pre-selected number of points around the SDSF line, the point processor 1111 may again loop through the selection-draw-test loop as described herein. If there are a second pre-selected number of attempts in selecting SDSF points, and a line is drawn between them, and there are fewer than a first pre-selected number of points around the SDSF line, the point processor 1111 may acknowledge that no SDSF lines were detected.

[0084] Continuing with Figure 19, the SDSF line processor 1113 may include fitting the curve 1609-1611 (Figure 20A) to points 789 that are contained within a first pre-selected distance of the SDSF line 377 if there are a first pre-selected number or more of points 789. If the number of points 789 within the first pre-selected distance of the curve 1609-1611 (Figure 20A) exceeds the number of points 789 within the first pre-selected distance of the SDSF line 377, and the curve 1609-1611 (Figure 20A) intersects with the path line 1214, and there are no gaps between points 789 on the curve 1609-1611 (Figure 20A) that exceed a second pre-selected distance, the SDSF line processor 1113 may include identifying the curve 1609-1611 (Figure 20A) as (for example) the SDSF line 377. If the number of points 789 within the first pre-selected distance of curve 1609-1611 (Figure 20A) does not exceed the number of points 789 within the first pre-selected distance of SDSF line 377, or if curve 1609-1611 (Figure 20A) does not intersect with path line 1214, or if there are gaps between points 789 on curve 1609-1611 (Figure 20A) that exceed the second pre-selected distance, and if SDSF line 377 is not stable, and if curve fitting has not been attempted more than the second pre-selected number of times, the SDSF line processor 1113 can run the curve fitting loop again.

[0085] Continuing to refer to Figure 19, the SDSF controller 1127 can receive the SDSF line 377, the occupied grid 1138, the TD orientation change 1142, and the speed limit 1148, and can generate an SDSF command 1144 to cause the TD 101 (Figure 20A) to travel so as to correctly traverse the SDSF 377 (Figure 20C). The SDSF controller 1127 may include, but is not limited to, an obstacle processor 1115, an SDSF approach 1131, and an SDSF crossing 1133. The obstacle processor 1115 can receive the SDSF line 377, the target point 1139, and the occupied grid 1138, and can determine from among the obstacles identified in the occupied grid 1138 whether any of them could interfere with the TD 101 (Figure 20C) as it traverses the SDSF 377 (Figure 20C). The obstacle processor 1115 may include, but is not limited to, an obstacle selector 1117, an obstacle tester 1119, and a cross-sectional locator 1121. The obstacle selector 1117 may include, but is not limited to, receiving an occupied grid 1138 as described herein. The obstacle selector 1117 may also include selecting an occupied grid point 1608 (Figure 20B) and its associated probability from a geographic area common to both the occupied grid 1138 and the polygon 1147. If the probability that an obstacle is present at a selected grid point 1608 (Figure 20B) is higher than a pre-selected percentage, and the obstacle is located between TD101 (Figure 20A) and target point 1139, and the obstacle is closer than a third pre-selected distance from SDSF line 377 between SDSF line 377 and target point 1139, the obstacle tester 1119 may include projecting the obstacle onto SDSF line 377 to form a projection 1621 intersecting SDSF line 377.If the probability that a location contains an obstacle is less than or equal to a pre-selected percentage, or if the obstacle is not located between TD101 (Figure 20A) and target point 1139, or if the obstacle is located at a distance equal to or greater than a third pre-selected distance from SDSF line 377 between SDSF line 377 and target point 1139, the obstacle tester 1119 may include resuming execution in receiving the occupied grid 1138 if there are further obstacles to process.

[0086] Continuing with Figure 19, the cross-sectional locator 1121 may include connecting projection points and identifying the locations of the endpoints 1622 / 1623 (Figure 20B) of the connected projection 1621 (Figure 20B) along the SDSF line 377. The cross-sectional locator 1121 may also include marking the portion 1624 (Figure 20B) of the SDSF line 377 between the projection endpoints 1622 / 1623 (Figure 20B) as non-crossable. The cross-sectional locator 1121 may also include marking the portion 1626 (Figure 20B) of the SDSF line 377 outside the non-crossable portion 1624 (Figure 20B) as crossable.

[0087] Continuing to refer to Figure 19, the SDSF approach 1131 may include sending an SDSF command 1144 to redirect TD101 (Figure 20C) to within a fifth pre-selected amount perpendicular to the traversable portion 1626 (Figure 20C) of the SDSF line 377. If the directional error relative to the vertical line 1627 (Figure 20C), which is perpendicular to the traversable portion 1626 (Figure 20C) of the SDSF line 377, exceeds a first pre-selected amount, the SDSF approach 1131 may include sending an SDSF command 1144 to decelerate TD101 (Figure 20C) by a ninth pre-selected amount. In some configurations, the ninth pre-selected amount can range from very slow to a complete stop. The SDSF approach 1131 may include sending an SDSF command 1144 to move TD101 (Figure 20C) forward toward the SDSF line 377, and sending an SDSF command 1144 to decelerate TD101 (Figure 20C) by a second pre-selected amount per meter traveled. If the distance between TD101 (Figure 20C) and the traversable SDSF line 1626 (Figure 20C) is less than a fourth pre-selected distance, and the direction error is greater than or equal to a third pre-selected amount relative to a line perpendicular to the SDSF line 377, the SDSF approach 1131 may include sending an SDSF command 1144 to decelerate TD101 (Figure 20C) by a ninth pre-selected amount.

[0088] Continuing to refer to Figure 19, if the direction error is less than a third pre-selected amount relative to a line perpendicular to the SDSF line 377, the SDSF crossing 1133 may ignore the updated SDSF information and send an SDSF command 1144 to drive TD101 (Figure 20C) at the pre-selected rate. If the TD orientation change 1142 indicates that the rise of the leading edge 1701 (Figure 20C) of TD101 (Figure 20C) relative to the trailing edge 1703 (Figure 20C) of TD101 (Figure 20C) is between a sixth pre-selected amount and a fifth pre-selected amount, the SDSF crossing 1133 may send an SDSF command 1144 to drive TD101 (Figure 20C) forward and an SDSF command 1144 to increase the speed of TD101 (Figure 20C) to the pre-selected rate according to the degree of the rise. If the TD orientation change 1142 indicates that the rise of the leading edge 1701 (Figure 20C) relative to the trailing edge 1703 (Figure 20C) of TD101 (Figure 20C) is less than a sixth pre-selected amount, the SDSF crossing 1133 may include sending an SDSF command 1144 to move TD101 (Figure 20C) forward at a seventh pre-selected speed. If the TD location 1141 indicates that the trailing edge 1703 (Figure 20C) is beyond a fifth pre-selected distance from the SDSF line 377, the SDSF crossing 1133 may include acknowledging that TD101 (Figure 20C) has completed crossing the SDSF 377. If TD location 1141 indicates that the trailing edge 1703 (Figure 20C) is less than or equal to a fifth pre-selected distance from SDSF line 377, the SDSF crossing 1133 may include repeating a loop that begins with ignoring the updated SDSF information.

[0089] Some illustrative ranges of the pre-selected values ​​described herein may include, but are not limited to, those outlined in Table I. [Table 1]

[0090] Referring here to Figure 21, in some configurations, to support real-time data aggregation, the system of this teaching can generate locations in three-dimensional space of various surface types in response to receiving data such as, for example, RGD-D camera image data. The system can rotate images 2155 and convert them from camera coordinate system 2157 to UTM coordinate system 2159. The system can generate polygonal files from the converted images, and the polygonal files can represent three-dimensional locations associated with surface type 2161. A method 2150 for determining the location of a feature 2151 from camera images 2155 received by TD101, having orientation 2163, can include, for example, receiving camera images 2155 by TD101. Each of the camera images 2155 may include an image timestamp 2171, and each of the images 2155 may include image color pixels 2167 and image depth pixels 2169. Method 2150 may include receiving the orientation 2163 of TD101, wherein the orientation 2163 has an orientation timestamp 2171, and determining a selected image 2173 by identifying an image from a camera image 2155 having an image timestamp 2165 closest to the orientation timestamp 2171. Method 2150 may also include separating image color pixels 2167 from image depth pixels 2169 in the selected image 2173, and determining an image surface classification 2161 for the selected image 2173 by providing the image color pixels 2167 to a first machine learning model 2177 and the image depth pixels 2169 to a second machine learning model 2179. Method 2150 may include determining the perimeter points 2181 of a feature in the camera image 2173, where the feature includes feature pixels 2151 within the perimeter, each of the feature pixels 2151 having the same surface classification 2161, and each of the perimeter points 2181 having a set of coordinates 2157. Method 2150 may include converting each of the coordinate sets 2157 to UTM coordinates 2159.

[0091] The structure of this instruction relates to a computer system for carrying out the methods discussed herein and a computer-readable medium containing programs for carrying out these methods. Raw data and results can be stored, printed, displayed, transferred to another computer, and / or transferred to another location for future reading and processing. Communication links can be wired or wireless, for example, using cellular communication systems, military communication systems, and satellite communication systems. Parts of the system can run on a computer with a variable number of CPUs. Other alternative computer platforms can also be used.

[0092] This configuration also covers software / firmware / hardware for performing the methods discussed herein, and computer-readable media for storing the software for performing these methods. The various modules described herein may be performed on the same CPU or on different CPUs. In accordance with the law, this configuration has been described in more or less specific language with respect to its structural and methodological features. However, it should be understood that this configuration is not limited to the specific features shown and described, as the means disclosed herein constitute a preferred form for embodying this configuration.

[0093] The method can be implemented electronically, either entirely or in part. Signals representing actions taken by the System and other disclosed components can be transmitted over at least one live communication network. Control and data information can be electronically executed and stored on at least one computer-readable medium. The System can be implemented to run on at least one computer node in at least one live communication network. General forms of at least one computer-readable medium include, but are not limited to, floppy disks, flexible disks, hard disks, magnetic tapes, or any other magnetic media, compact disk read-only memory or any other optical media, punch cards, paper tapes, or any other physical media with perforation patterns, random access memory, programmable read-only memory, and erasable programmable read-only memory (EPROM), flash EPROM, or any other memory chip or cartridge, or any other medium from which a computer can read. Furthermore, at least one computer-readable medium may include, but is not limited to, graphs in any form, provided it is appropriately licensed, including Graphics Exchange Format (GIF), Joint Photo Professional Group (JPEG), Portable Network Graphics (PNG), Scalable Vector Graphics (SVG), and Tagged Image File Format (TIFF).

[0094] While these teachings have been described above in terms of specific configurations, it should be understood that they are not limited to these disclosed configurations. Many modifications and other configurations are intended and will be covered by both the disclosure and the accompanying claims, as this will be recalled by those skilled in the art. The scope of these teachings is intended to be determined by the proper interpretation and construction of the accompanying claims and their legal equivalents, as understood by those skilled in the art relying on the disclosures in this specification and the accompanying drawings.

Claims

1. A method for navigating at least one substantially discontinuous surface feature (SDSF) encountered by a transport device (TD), wherein the TD travels along a path on the surface, the surface includes the at least one SDSF, and the path includes a start point and an end point. The aforementioned method, Accessing point cloud data representing the aforementioned surface, Filtering the aforementioned point cloud data, The filtered point cloud data is formed into a processable portion, Merging the aforementioned processable portion into at least one concave polygon, Identifying the location of the at least one SDSF within the at least one concave polygon and labeling them, wherein identifying and labeling the locations constitutes forming labeled point cloud data. At a minimum, a graphed polygon is created based on at least one of the concave polygons, Select the path from the starting point to the ending point based at least on the graphed polygon. Includes, The method wherein the TD traverses the at least one SDSF along the path.

2. Filtering the aforementioned point cloud data is Conditionally remove points representing transient objects and points representing outliers from the aforementioned point cloud data, Replacing the removed point having a pre-selected height The method according to claim 1, including the method described in claim 1.

3. Forming the processing part is The point cloud data is divided into the processable portions, Removing a point of a pre-selected height from the aforementioned processable portion. The method according to claim 1, including the method described in claim 1.

4. Merging the aforementioned processable portions means By analyzing outliers, voxels, and normals, the size of the processable portion is reduced, Expanding the area from the reduced size of the processable portion, Determining the initial drivable surface from the aforementioned expanded region, The initial drivable surface is divided and meshed, Identifying the location of polygons within the divided and meshed initial drivable surface, Based on at least the polygon, at least one drivable surface is defined. The method according to claim 1, including the method described in claim 1.

5. Identifying the location of the at least one SDSF and labeling them is The method involves sorting the point cloud data of the drivable surface according to an SDSF filter, wherein the SDSF filter includes points of at least three categories. At a minimum, the location of at least one SDSF point is determined based on whether the points of the at least three categories, in combination, satisfy at least one first pre-selected criterion. The method according to claim 4, including the method described in claim 4.

6. The method according to claim 5, further comprising creating at least one SDSF trajectory based on whether a plurality of the at least one SDSF points, in combination, satisfy at least one second pre-selected criterion.

7. Creating the aforementioned graphed polygon is, Creating at least one convex polygon from the at least one drivable surface, wherein the at least one convex polygon includes an outer edge, Smoothing the outer edge, Based on the smoothed outer edge, a travel margin is formed, Adding the at least one SDSF orbit to the at least one traversable surface, Removing the inner edge from the at least one drivable surface according to at least one third pre-selected criterion The method according to claim 6, further comprising:

8. The smoothing of the outer edge includes trimming the outer edge outwards to form an outer edge. The method according to claim 8.

9. The method according to claim 7, wherein forming the running margin of the smoothed outer edge includes trimming the outer edge inward.

10. A system for navigating at least one substantially discontinuous surface feature (SDSF) encountered by a TD, wherein the TD travels along a path on the surface, the surface includes the at least one SDSF, and the path includes a start point and an end point. The aforementioned system, Sensors and, Map processor and Power base and, Device controller and Equipped with, The aforementioned device controller A first processor accesses point cloud data representing the surface, A filter for filtering the aforementioned point cloud data, A second processor that forms a processable portion from the filtered point cloud data, A third processor that merges the processable portion into at least one concave polygon, A fourth processor for identifying and labeling the positions of the at least one SDSF within the at least one concave polygon, wherein identifying and labeling the positions constitutes forming labeled point cloud data, A fifth processor for creating graphed polygons, A sixth processor that selects the path from the starting point to the ending point based at least on the graphed polygon, Includes, The TD is a system that traverses the at least one SDSF along the path.

11. The filter comprises a seventh processor, and the seventh processor is Conditionally remove points representing transient objects and points representing outliers from the aforementioned point cloud data, Replacing the removed point having a pre-selected height The system according to claim 10, which executes code including the following:

12. The second processor is, The point cloud data is divided into the processable portions, The system according to claim 10, comprising executable code that includes removing a point of a pre-selected height from the processable portion.

13. The third processor described above is By analyzing outliers, voxels, and normals, the size of the processable portion is reduced, Expanding the area from the reduced size of the processable portion, Determining the initial drivable surface from the aforementioned expanded region, The initial drivable surface is divided and meshed, Identifying the location of polygons within the divided and meshed initial drivable surface, Based on at least the polygon, at least one drivable surface is defined. The system according to claim 10, comprising executable code including the following.

14. The fourth processor is, The method involves sorting the point cloud data of the drivable surface according to an SDSF filter, wherein the SDSF filter includes points of at least three categories. At a minimum, the location of at least one SDSF point is determined based on whether the points of the at least three categories, in combination, satisfy at least one first pre-selected criterion. The system according to claim 13, comprising executable code including the following.

15. The system according to claim 14, wherein the fourth processor includes executable code that creates at least one SDSF trajectory based on whether a plurality of the at least one SDSF points, in combination, satisfy at least one second pre-selected criterion.

16. Creating a graphed polygon is, Creating at least one convex polygon from the at least one drivable surface, wherein the at least one convex polygon includes an outer edge, Smoothing the outer edge, Based on the smoothed outer edge, a travel margin is formed, Adding the at least one SDSF orbit to the at least one traversable surface, Removing the inner edge from the at least one drivable surface according to at least one third pre-selected criterion The system according to claim 13, further comprising an eighth processor, which includes executable code including the following.

17. The system according to claim 16, wherein the ninth processor includes executable code that smooths the outer edge, which includes trimming the outer edge outwards to form an outer edge.

18. The system according to claim 17, wherein the tenth processor includes executable code that includes trimming the outer edge inward in order to form the running margin of the smoothed outer edge.

19. A method for navigating at least one substantially discontinuous surface feature (SDSF) encountered by a transport device (TD), wherein the TD travels along a path on the surface, the surface includes the at least one SDSF, and the path includes a start point and an end point. The aforementioned method, Accessing a route configuration, wherein the route configuration includes at least one graphed polygon containing filtered point cloud data, the filtered point cloud data includes labeled features, and the point cloud data includes drivable margins. The point cloud data is converted to a global coordinate system, Determining the boundary of the at least one substantially discontinuous surface feature (SDSF), Creating an SDSF buffer of a pre-selected size around the aforementioned boundary, At a minimum, determine which of the at least one SDSF is traversable based on at least one SDSF traversal criterion, At a minimum, create an edge / weight graph based on the at least one SDSF cross-sectional criterion, the converted point cloud data, and the root morphology. Select the path from the starting point to the ending point based at least on the edge / weight graph. Methods that include...

20. The aforementioned at least one SDSF cross-sectional criterion is, The pre-selected width of the at least one SDSF and the pre-selected smoothness of the at least one SDSF, The minimum entry distance and minimum exit distance between the at least one SDSF including a drivable surface and the TD, Equipped with, The method according to claim 19, wherein the minimum approach distance between the at least one SDSF and the TD allows the TD to approach the at least one SDSF at an angle of approximately 90°.

21. A system for navigating at least one substantially discontinuous surface feature (SDSF) encountered by a transport device (TD), wherein the TD travels along a path on the surface, the surface includes the at least one SDSF, the path includes a start point and an end point, and the system A first processor for accessing a route configuration, wherein the route configuration includes at least one graphed polygon containing filtered point cloud data, the filtered point cloud data includes labeled features, and the point cloud data includes drivable margins, the first processor, A second processor that converts the point cloud data into a global coordinate system, A third processor for determining the boundary of at least one SDSF, wherein the third processor creates an SDSF buffer of a pre-selected size around the boundary. A fourth processor that determines which of the at least one SDSF is traversable based on at least one SDSF traversal criterion, A fifth processor that creates an edge / weight graph based on at least one SDSF cross-sectional criterion, the converted point cloud data, and the root morphology, A base controller that selects the path from the start point to the end point based at least on the edge / weight graph, A system equipped with these features.

22. The aforementioned at least one SDSF cross-sectional criterion is, The pre-selected width of the at least one SDSF and the pre-selected smoothness of the at least one SDSF, The minimum entry distance and minimum exit distance between the at least one SDSF including a drivable surface and the TD, Equipped with, The system according to claim 21, wherein the minimum approach distance between the at least one SDSF and the TD allows the TD to approach the at least one SDSF at an angle of approximately 90°.

23. A method for navigating at least one substantially discontinuous surface feature (SDSF) encountered by a TD, wherein the TD travels along a path on the surface, the surface includes the at least one SDSF, and the path includes a start point and an end point. The aforementioned method, Accessing point cloud data representing the aforementioned surface, Filtering the aforementioned point cloud data, The filtered point cloud data is formed into a processable portion, Merging the aforementioned processable portion into at least one concave polygon, Identifying the location of the at least one SDSF within the at least one concave polygon and labeling them, wherein identifying and labeling the locations constitutes forming labeled point cloud data. At a minimum, a graphed polygon is created based on at least one concave polygon, wherein the graphed polygon forms a root shape. The point cloud data is converted to a global coordinate system, Determining the boundary of at least one SDSF, Creating an SDSF buffer of a pre-selected size around the aforementioned boundary, At a minimum, determine which of the at least one SDSF is traversable based on at least one SDSF traversal criterion, At a minimum, create an edge / weight graph based on the at least one SDSF cross-sectional criterion, the converted point cloud data, and the root morphology. Select the path from the starting point to the ending point based at least on the edge / weight graph. Methods that include...

24. Filtering the aforementioned point cloud data is Conditionally remove points representing transient objects and points representing outliers from the aforementioned point cloud data, Replacing the removed point having a pre-selected height The method according to claim 23, including the method described in claim 23.

25. Forming the processing part is The point cloud data is divided into the processable portions, Removing a point of a pre-selected height from the aforementioned processable portion. The method according to claim 23, including the method described in claim 23.

26. Merging the aforementioned processable portions means By analyzing outliers, voxels, and normals, the size of the processable portion is reduced, Expanding the area from the reduced size of the processable portion, Determining the initial drivable surface from the aforementioned expanded region, The initial drivable surface is divided and meshed, Identifying the location of polygons within the divided and meshed initial drivable surface, Based on at least the polygon, at least one drivable surface is defined. The method according to claim 23, including the method described in claim 23.

27. Identifying the location of the at least one SDSF and labeling them is The method involves sorting the point cloud data of the drivable surface according to an SDSF filter, wherein the SDSF filter includes points of at least three categories. At a minimum, the location of at least one SDSF point is determined based on whether the points of the at least three categories, in combination, satisfy at least one first pre-selected criterion. The method according to claim 26, including the method described in claim 26.

28. The method according to claim 27, further comprising creating at least one SDSF trajectory based on whether a plurality of the at least one SDSF points, in combination, satisfy at least one second pre-selected criterion.

29. Creating a graphed polygon is, Creating at least one convex polygon from the at least one drivable surface, wherein the at least one convex polygon includes an outer edge, Smoothing the outer edge, Based on the smoothed outer edge, a travel margin is formed, Adding the at least one SDSF orbit to the at least one traversable surface, Removing the inner edge from the at least one drivable surface according to at least one third pre-selected criterion The method according to claim 28, further comprising:

30. The method according to claim 29, wherein the smoothing of the outer edge includes trimming the outer edge outwards to form an outer edge.

31. The method according to claim 30, wherein forming the running margin of the smoothed outer edge includes trimming the outer edge inward.

32. The aforementioned at least one SDSF cross-sectional criterion is, The pre-selected width of the at least one SDSF and the pre-selected smoothness of the at least one SDSF, The minimum entry distance and minimum exit distance between the at least one SDSF including a drivable surface and the TD, Equipped with, The method according to claim 23, wherein the minimum approach distance between the at least one SDSF and the TD allows the TD to approach the at least one SDSF at an angle of approximately 90°.

33. A system for navigating at least one substantially discontinuous surface feature (SDSF) encountered by a transport device (TD), wherein the TD travels along a path on the surface, the surface includes the at least one SDSF, and the path includes a start point and an end point. The aforementioned system, A first processor accesses point cloud data representing the surface, A first filter for filtering the point cloud data, A second processor that forms a processable portion from the filtered point cloud data, A third processor that merges the processable portion into at least one concave polygon, A fourth processor for identifying and labeling the positions of the at least one SDSF within the at least one concave polygon, wherein identifying and labeling the positions constitutes forming labeled point cloud data, A fifth processor for creating graphed polygons, A sixth processor for accessing a route configuration, wherein the route configuration includes at least one graphed polygon containing filtered point cloud data, the filtered point cloud data containing labeled features, and the point cloud data containing drivable margins, and the sixth processor A seventh processor that converts the point cloud data into a global coordinate system, An eighth processor for determining the boundary of at least one SDSF, wherein the eighth processor creates an SDSF buffer of a pre-selected size around the boundary, A ninth processor that determines which of the at least one SDSF is traversable based on at least one SDSF traversal criterion, A tenth processor that creates an edge / weight graph based on at least one SDSF cross-sectional criterion, the converted point cloud data, and the root morphology, A base controller that selects the path from the start point to the end point based at least on the edge / weight graph, A system equipped with these features.

34. The first filter is, Conditionally remove points representing transient objects and points representing outliers from the aforementioned point cloud data, Replacing the removed point having a pre-selected height The system according to claim 33, comprising executable code including the following.

35. The second processor is, The point cloud data is divided into the processable portions, Removing a point of a pre-selected height from the aforementioned processable portion. The system according to claim 33, comprising executable code including the following.

36. The third processor described above is By analyzing outliers, voxels, and normals, the size of the processable portion is reduced, Expanding the area from the reduced size of the processable portion, Determining the initial drivable surface from the aforementioned expanded region, The initial drivable surface is divided and meshed, Identifying the location of polygons within the divided and meshed initial drivable surface, Based on at least the polygon, at least one drivable surface is defined. The system according to claim 33, comprising executable code including the following.

37. The fourth processor is, The method involves sorting the point cloud data of the drivable surface according to an SDSF filter, wherein the SDSF filter includes points of at least three categories. At a minimum, the location of at least one SDSF point is determined based on whether the points of the at least three categories, in combination, satisfy at least one first pre-selected criterion. The system according to claim 36, comprising executable code including the following.

38. The system according to claim 37, comprising executable code that includes creating at least one SDSF trajectory based on whether a plurality of the at least one SDSF points, in combination, satisfy at least one second pre-selected criterion.

39. Creating a graphed polygon is, Creating at least one convex polygon from the at least one drivable surface, wherein the at least one convex polygon includes an outer edge, Smoothing the outer edge, Based on the smoothed outer edge, a travel margin is formed, Adding the at least one SDSF orbit to the at least one traversable surface, Removing the inner edge from the at least one drivable surface according to at least one third pre-selected criterion The system according to claim 38, comprising executable code including the following.

40. The system according to claim 39, wherein the smoothing of the outer edge includes executable code that trims the outer edge outwards to form an outer edge.

41. The system according to claim 40, wherein forming the running margin of the smoothed outer edge includes executable code that trims the outer edge inward.

42. The aforementioned at least one SDSF cross-sectional criterion is, The pre-selected width of the at least one SDSF and the pre-selected smoothness of the at least one SDSF, The minimum entry distance and minimum exit distance between the at least one SDSF including a drivable surface and the TD, Equipped with, The system according to claim 33, wherein the minimum approach distance between the at least one SDSF and the TD allows the TD to approach the at least one SDSF at an angle of approximately 90°.

43. A method for navigating a transport device (TD) along a path line in a travel area toward a target point traversing at least one substantially discontinuous surface feature (SDSF), wherein the TD includes a leading edge and a trailing edge, and the SDSF includes an SDSF point. The aforementioned method, Receiving SDSF information and obstacle information related to the aforementioned area of ​​travel, To detect at least one candidate SDSF from the SDSF information, Selecting an SDSF line from the at least one candidate SDSF based on at least one selection criterion, Based on the location of at least one obstacle found in the obstacle information near the selected SDSF line, at least one traversable portion of the selected SDSF line is determined. By changing the direction of the TD so that it travels along a path perpendicular to the at least one traversable portion, the TD is directed toward the at least one traversable portion and operated at a first speed. Based on the relationship between the direction of travel and the aforementioned travel path, the direction of travel of the TD is constantly corrected, The TD is driven at a second speed by adjusting the first speed of the TD based at least on the direction of travel and the distance between the TD and the at least one traversable portion. If the SDSF associated with the at least one traversable portion is higher than the surface of the area of ​​travel, the SDSF is traversed by raising the leading edge relative to the trailing edge and driving the TD at a third increased speed according to the degree of the rise. The TD is driven at a fourth speed until it passes the SDSF. Methods that include...

44. Detecting at least one candidate SDSF from the aforementioned SDSF information is: (a) Drawing a closed polygon that encloses the location of the TD and the location of the target point, (b) Drawing a straight line between the target point location and the TD location, (c) Selecting two of the SDSF points from the SDSF information, wherein the two SDSF points are located within the closed polygon. (d) Drawing an SDSF line between the two SDSF points, (e) If there are fewer points than the first pre-selected number within the first pre-selected distance of the SDSF line, and fewer than the second pre-selected number of trials have been performed in selecting the SDSF points, then repeat steps (c) to (e), (f) If there are more than the number of points specified in the first pre-selected, the curve is fitted to the SDSF points that are included within the first pre-selected distance of the SDSF line, (g) If the first number of SDSF points within the first pre-selected distance of the curve exceeds the second number of SDSF points within the first pre-selected distance of the SDSF line, and the curve intersects the straight line, and there is no gap between the SDSF points on the curve beyond the second pre-selected distance, the curve is identified as the SDSF line. (h) If the first number of points within the first pre-selected distance of the curve does not exceed the second number of points within the first pre-selected distance of the SDSF line, or if the curve does not intersect the straight line, or if there is a gap between the SDSF points on the curve that exceeds the second pre-selected distance, and the SDSF line is not stable, and step (f)-(h) has not been attempted more than the second pre-selected number of times, then step (f)-(h) is repeated. The method according to claim 43, including the method described in claim 43.

45. The method according to claim 44, wherein the closed polygon may include a pre-selected width.

46. The method according to claim 45, wherein the pre-selected width has the width dimension of TD.

47. The method according to claim 44, wherein the selection of the SDSF point includes random selection.

48. The aforementioned at least one selection criterion is, The first number of SDSF points within the first pre-selected distance of the curve exceeds the second number of SDSF points within the first pre-selected distance of the SDSF line, The curve intersects the straight line, The gap between the SDSF points on the curve does not exceed a second pre-selected distance. The method according to claim 44, comprising:

49. Determining at least one traversable portion of the selected SDSF is: Selecting a plurality of obstacle points from the aforementioned obstacle information, wherein each of the plurality of obstacle points includes the probability that each of the plurality of obstacle points is associated with at least one of the aforementioned obstacles. If the aforementioned probability is higher than a pre-selected percentage, and any of the plurality of obstacle points is located between the SDSF line and the target point, and any of the plurality of obstacle points is closer than a third pre-selected distance from the SDSF line, the plurality of obstacle points are projected onto the SDSF line to form at least one projection, Connecting at least two of the aforementioned at least one projections to each other, Identifying the positions of the endpoints of the at least two connected projections along the SDSF line, The at least two connected projections are marked as non-crossable sections, The SDSF line outside the non-crossable section is marked as at least one crossable section. The method according to claim 43, including the method described in claim 43.

50. A method for identifying the location of a feature from a camera image received by a robot having a certain posture, wherein the method is: The process involves receiving camera images from sensors mounted on the robot, wherein each of the camera images includes an image timestamp, and each of the camera images has image color pixels and image depth pixels. Receiving the posture of the robot, wherein the posture has a posture timestamp. The selected camera image is determined by identifying one of the camera images having the image timestamp closest to the posture timestamp. In the selected camera image, the image color pixels are separated from the image depth pixels, The image surface classification of the selected camera image is determined by providing the image color pixels to a first machine learning model and the image depth pixels to a second machine learning model. Determining the outer perimeter points of a feature in the selected camera image, wherein the feature includes feature pixels within the perimeter defined by the outer perimeter points, each of the feature pixels having the same surface classification, and each of the outer perimeter points having a set of coordinates. Convert each of the aforementioned sets of coordinates to UTM coordinates. Methods that include...

51. A method for including at least one substantially discontinuous surface feature (SDSF) in a map, wherein the map is used by a transport device, the transport device traverses the at least one SDSF, and the method is Removing transient data from a point cloud dataset, wherein the removal is performed by: To form processed point cloud data, The processed point cloud data is divided into sections having a pre-selected number of points, The process involves removing points from the divided and processed point cloud data, wherein the removed points have pre-selected height values, and the removal process forms divided point cloud data. From the divided point cloud data, a concave polygon having a pre-selected size is generated, The points within the aforementioned concave polygon are organized into a pre-selected number of categories, Applying a first pre-selected criterion to the category to form a filtered category, The filtered categories are averaged, Connecting the averaged filtered categories together to form at least one SDSF, The concave polygon is combined with at least one SDSF to form a merged polygon, Filter the merged polygons according to a second pre-selected criterion, To provide the aforementioned map to the transport device Methods that include...

52. The method according to claim 51, further comprising categorizing the points within the concave polygon as upper donut points, lower donut points, and cylindrical points.

Images