Obstacle identification method and path planning method based on depth point cloud

Abstract

The invention provides a depth point cloud-based obstacle identification method and a path planning method. The depth point cloud-based obstacle identification method comprises the following steps: acquiring original depth point cloud data through a depth camera and converting three-dimensional information of the original depth point cloud data into a two-dimensional matrix; the pitch angle change of the adjacent point clouds is compared with a set threshold value in the two-dimensional matrix, and the plane point clouds and the initial obstacle point clouds are distinguished; and extracting obstacle information based on the initial obstacle point cloud. According to the obstacle identification method based on the depth point cloud and the path planning method, the accuracy, adaptability and rationality of the obstacle identification method are improved, and the safety of the robot during path planning based on the cost map is improved.

Description

Technical Field

[0001] This application belongs to the field of robot navigation technology, and more specifically, it relates to an obstacle recognition method and path planning method based on depth point clouds. Background Technology

[0002] The ability of a robot to perform tasks safely and efficiently depends heavily on its path planning and motion control capabilities. However, to achieve efficient path planning and motion control, a robot must possess extremely strong perception capabilities in complex environments. This is especially true in changing environments, where the demands on a robot's perception capabilities are even more stringent. This not only ensures the robot's safety during task execution but also improves its efficiency and enhances its intelligence. The foundation of path planning is the establishment of a safe and reliable real-time map.

[0003] In existing technologies, robots employ single-line / multi-line radar or depth camera solutions for obstacle detection. This method relies on selecting points on a reference plane using existing ground segmentation algorithms (such as RANSAC plane fitting). Points with height differences exceeding a preset range on the reference plane are considered obstacles. Therefore, traditional height difference detection methods face significant challenges in complex terrain: the height difference between gentle slopes (e.g., 1cm / 10cm) and steep slopes (e.g., 5cm / 10cm) is significant, while real obstacles (e.g., steps) typically exhibit abrupt height changes (>5cm). Setting a high threshold (e.g., 5cm) may miss low obstacles (e.g., 2cm toys), while a low threshold (e.g., 1cm) may misclassify slopes as obstacles. This inherent contradiction in threshold setting, coupled with the dynamic changes in mixed terrain (e.g., slope transition zones, uneven surfaces) in real-world environments, means that traditional methods cannot guarantee the completeness of obstacle detection and are prone to terrain misjudgments. Thus, existing obstacle recognition methods suffer from insufficient accuracy, adaptability, and rationality.

[0004] Furthermore, once an obstacle is detected, it will be mapped onto the cost map. That is, the initial cost value of a planar point cloud mapped onto the cost map is zero, while the obstacle is assigned a grid cost value. However, a planar surface does not necessarily mean low safety and energy consumption. The lack of quantitative assessment of subtle undulations in the road surface can lead to robot wobbling or increased energy consumption, thus resulting in insufficient safety when the robot plans its path based on the cost map. Summary of the Invention

[0005] The purpose of this application is to provide an obstacle recognition method and a path planning method based on deep point clouds, so as to solve the technical problems of insufficient accuracy, adaptability and rationality of existing obstacle recognition methods, as well as the problem of insufficient safety when robots plan paths based on cost maps.

[0006] To achieve the above objectives, the technical solution adopted in this application is: to provide an obstacle recognition method based on deep point clouds, comprising the following steps: The raw depth point cloud data is acquired through a depth camera and its three-dimensional information is converted into a two-dimensional matrix. By comparing the pitch angle changes of adjacent point clouds with a set threshold using a two-dimensional matrix, planar point clouds can be distinguished from preliminary obstacle point clouds. Obstacle information is extracted based on the preliminary obstacle point cloud.

[0007] The beneficial effects of the above embodiments are as follows: After the 3D point cloud is projected into a two-dimensional matrix based on the depth point cloud of the depth camera, the planar point cloud and the preliminary obstacle point cloud are distinguished by comparing the pitch angle changes of adjacent point clouds with the set threshold. Compared with the prior art, it has the characteristics of pitch angle reflecting local geometric features, robustness to terrain changes, universality of threshold design, and "discrimination" between uphill and downhill. At the same time, it improves the accuracy, adaptability and rationality of obstacle avoidance.

[0008] This application also provides a path planning method based on the obstacle recognition method based on deep point clouds as described above. The path planning method includes the following steps: The planar point cloud in the two-dimensional matrix is ​​clustered according to the pitch angle change to generate several planar clusters; The first generation of values ​​is generated based on the pitch angle variation characteristics of each planar cluster in the same row or column of a two-dimensional matrix. After mapping the two-dimensional matrix to the cost map, the initial generation value of the raster corresponding to the planar cluster is assigned based on the first generation value; Path planning is based on this cost map.

[0009] The beneficial effects of the above embodiments are as follows: by projecting point clouds onto a two-dimensional matrix and performing cluster analysis based on pitch angle variation characteristics, efficient modeling and evaluation of complex terrain are achieved. Specifically, planar clusters are first generated through pitch angle variation clustering to effectively distinguish continuous terrain from real obstacles; then, planar clusters are generated based on pitch angle variation clustering, and a first-generation value is generated based on the pitch angle variation rate and amplitude in the row / column directions to quantify the traversability of the terrain; finally, by mapping to a cost map and assigning an initial generation value, a refined terrain evaluation is provided for path planning. This approach preserves the three-dimensional geometric features of the point cloud while achieving a balance between computational efficiency and navigation accuracy, significantly improving the robot's terrain perception and decision-making capabilities in complex environments, thus comprehensively enhancing the robot's safety.

[0010] This application also provides another path planning method, based on the obstacle recognition method based on deep point clouds as described above, the path planning method comprising the following steps: The planar point cloud in the two-dimensional matrix is ​​clustered according to the pitch angle change to generate several planar clusters; The target type is obtained by classifying the RGB image region corresponding to each planar cluster, and a second-generation value is generated based on the target type. After mapping the two-dimensional matrix to the cost map, the initial generation value of the raster corresponding to the planar cluster is assigned based on the second generation value; Path planning is based on this cost map.

[0011] The beneficial effects of the above embodiments are as follows: The above method achieves refined classification and evaluation of planar regions by combining the geometric features of 3D point clouds with the semantic information of RGB images. First, planar clusters are generated based on pitch angle changes. Then, target types with different physical characteristics are identified through RGB images, and a second-generation value reflecting the actual difficulty of passage is generated. Finally, different plane types are assigned differentiated values ​​in the cost map, enabling the robot path planning to not only avoid obstacles but also intelligently select the optimal road surface type, significantly improving the safety, smoothness, and energy efficiency of navigation. At the same time, it solves the planning defect of traditional methods that are "geometrically feasible but practically infeasible" due to ignoring semantic information.

[0012] This application also provides another path planning method, based on the obstacle recognition method based on deep point clouds as described above, the path planning method including the steps of... The planar point cloud in the two-dimensional matrix is ​​clustered according to the pitch angle change to generate several planar clusters; The first generation of values ​​is generated based on the pitch angle variation characteristics of each planar cluster in the same row or column of a two-dimensional matrix. The target type is obtained by classifying the RGB image region corresponding to each planar cluster, and a second-generation value is generated based on the target type. After mapping the two-dimensional matrix onto the cost map, the initial generation value of the raster corresponding to the planar cluster is assigned based on the first generation value and the second generation value. Path planning is based on this cost map.

[0013] The beneficial effects of the above embodiments are as follows: This method innovatively integrates point cloud geometric features with visual semantic information reflecting physical properties, achieving complementary advantages in multi-dimensional environmental assessment: the first-generation value generated through pitch angle variation characteristics accurately quantifies terrain geometric features, solving the problem of pure visual methods being insensitive to geometric features; at the same time, the second-generation value generated based on RGB classification supplements the physical attributes at the semantic level, making up for the lack of semantic understanding in pure point cloud methods. The intelligent integration of the two ensures that the robot accurately perceives the geometric features of complex terrain and realizes the functional judgment of road surface type, enabling the path planning system to simultaneously consider "geometric physical drivability" and "practical applicability," achieving the optimal balance between safety and passage efficiency in complex scenarios. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 A flowchart illustrating the obstacle recognition method based on deep point clouds provided in the first embodiment of this application; Figure 2 This is a schematic diagram showing the angle between the point cloud and the X-axis in the depth camera coordinate system. Figure 3 A schematic diagram illustrating the conversion of 3D information from depth point cloud data into a 2D matrix; Figure 4 A flowchart illustrating the obstacle recognition method based on deep point clouds provided in the second embodiment of this application; Figure 5 A flowchart illustrating the path planning method provided in the fifth embodiment of this application; Figure 6 Example diagram of the deep point cloud visualization scene effect provided in the fifth embodiment of this application. Detailed Implementation

[0016] To make the technical problems, technical solutions, and beneficial effects to be solved by this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this application.

[0017] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0018] Please refer to the following: Figure 1 The obstacle recognition method based on depth point clouds provided in the first embodiment of this application will now be described. The obstacle recognition method based on depth point clouds includes: Step S1: Obtain the original depth point cloud data through the depth camera and convert its three-dimensional information into a two-dimensional matrix; Step S2: Compare the pitch angle changes of adjacent point clouds with a set threshold in a two-dimensional matrix to distinguish between planar point clouds and preliminary obstacle point clouds; Step S3: Extract obstacle information based on the preliminary obstacle point cloud.

[0019] It is understandable that in step S1, the original depth point cloud data can be obtained directly through the SDK interface of the depth camera, and then the three-dimensional information of the depth point cloud data can be converted into a two-dimensional matrix based on the existing calibration method; thus, the spatial relative relationship between the depth camera and any point cloud in the two-dimensional matrix can be obtained.

[0020] Thus, in step S2, after projecting the 3D point cloud onto a two-dimensional matrix, the planar point cloud and the preliminary obstacle point cloud are distinguished by comparing the pitch angle changes of adjacent point clouds with a set threshold. Compared with the prior art, the advantages are: (1) The pitch angle reflects the local geometric features In a two-dimensional matrix, the pitch angle is the ratio of the height difference to the horizontal distance, essentially a measure of local slope. Therefore, in gentle slope scenarios, a small pitch angle (e.g., 5°) means that even if the cumulative height difference is large, the angle change is gradual, avoiding false detections of obstacles. Conversely, where the pitch angle of an obstacle changes abruptly (e.g., above 30°), it forms a clearly discontinuous geometry with the surrounding ground, preventing missed detections of obstacles.

[0021] (2) Robustness to topographic changes When facing a gentle slope, the pitch angles of adjacent point clouds are close, and even with accumulated height differences, the angle change remains below the threshold. When facing a steep slope or uneven ground, if the slope is consistent (e.g., a fixed tilt angle), the pitch angle change is still small. When facing an obstacle, a sudden change in pitch angle (e.g., from 0° to 45°) directly triggers the threshold determination.

[0022] (3) Universality of threshold design By simply setting an angle threshold (e.g., 15°), you can simultaneously filter out gentle slopes, steep slopes, and ground undulations, retaining only real obstacles. There is no need to adjust parameters for different terrains.

[0023] (4) The view that uphill and downhill slopes create "discrimination" In uphill scenarios within a two-dimensional matrix, the pitch angle change of adjacent point clouds is often greater than the actual slope, while in downhill scenarios, the pitch angle change of adjacent point clouds is often less than the actual slope. Therefore, when faced with uphill and downhill scenarios of the same slope, a "discriminatory" viewpoint emerges: uphill slopes are more easily identified as obstacles, while downhill slopes are relatively easier to identify as roads. However, this "discriminatory" viewpoint corresponds precisely to the relative difficulty of uphill and downhill slopes. Generally, uphill slopes are more difficult than downhill slopes, thus requiring a smaller slope (e.g., less than 14°) to be identified as flat surfaces; conversely, downhill slopes are easier than uphill slopes, thus requiring a larger slope (e.g., greater than 16°) to be identified as obstacles, thus improving the rationality of the obstacle definition.

[0024] Thus, in step S3, obstacle information is extracted based on the preliminary obstacle point cloud, thereby improving the accuracy, adaptability, and rationality of obstacle avoidance.

[0025] It is worth noting that the adjacent point clouds can be two adjacent point clouds in the same column of a two-dimensional matrix, or two adjacent point clouds in the same row of a two-dimensional matrix. Preferably, the adjacent point clouds are two adjacent point clouds in the same column of a two-dimensional matrix.

[0026] It is worth noting that, prior to step S1, an extrinsic parameter calibration step for the depth camera and radar can be included. By accurately calibrating the relative poses (rotation matrix and translation vector) of the two, sensor data from different coordinate systems are unified into the radar coordinate system, solving the spatial alignment problem. This step can eliminate spatial deviations caused by differences in installation position. The rotation matrix and translation vector of the radar between two time points can be used to solve the problem by matching point cloud data between two frames. Furthermore, dynamic calibration can compensate for sensor pose drift during robot movement in real time, ensuring strict spatiotemporal alignment between the depth camera point cloud and radar data. This provides a high-precision geometric consistency guarantee for subsequent obstacle recognition, coordinate transformation, and obstacle avoidance decisions.

[0027] It is worth noting that after step S3, an obstacle point cloud data conversion step may also be included. The extracted obstacle information is converted into obstacle information usable by the obstacle avoidance module and then published. In this way, a depth camera can replace multi-line LiDAR, which not only improves the accuracy of obstacle recognition but also breaks down software ecosystem barriers.

[0028] For further details, please refer to Figure 2In step S1, after acquiring the original depth point cloud data and before converting its three-dimensional information into a two-dimensional matrix, the following steps are also included: Calculate the angle between each depth point cloud and the X-axis in the depth camera coordinate system. x ; If angle x >a*f, where a is the scaling factor and f is the horizontal field of view angle of the depth camera, then the depth point cloud is filtered. Where the included angle x The formula is: , In the formula, x and y are the X and Y axis coordinates of the corresponding depth point cloud.

[0029] Understandably, due to the limited field of view of a camera, edge data may be inaccurate. The angle at each point is calculated using the formula described above. x By removing points with excessively large angles and retaining only the valid point data directly in front, the accuracy of depth point cloud data can be improved.

[0030] For details, please refer to Figure 3 In step S1, converting the 3D information of the original depth point cloud data into a 2D matrix includes the following steps: Calculate the row number of the point cloud projection into the two-dimensional matrix for each depth. Id row number Id The calculation formula is: , In the formula, x, y, and z are the X, Y, and Z axis coordinates of the corresponding depth point cloud, respectively. xy Let ang be the horizontal distance of the depth point cloud projected onto the XY plane in the camera coordinate system. z Let ang be the angle between the depth point cloud and the XY plane. vmin ang is the minimum pitch angle of the depth camera. v This represents the angular resolution of the depth camera in the vertical direction. Calculate the column number col of the point cloud projection into the two-dimensional matrix at each depth. Id Column number col Id The calculation formula is: , In the formula, ang y Let ang be the angle between the depth point cloud and the Y-axis. hmin ang is the minimum horizontal angle of the depth camera. h The horizontal angular resolution of the depth camera; According to the row number Id With column number colId Determine the position of each depth point cloud in a two-dimensional matrix.

[0031] Understandably, the above method, by combining the installation parameters of the depth camera (such as horizontal / vertical field of view and angular resolution) with precise geometric calculation formulas, achieves high-fidelity projection of point cloud data into a two-dimensional matrix. This not only fully preserves the spatial distribution characteristics of the original 3D point cloud but also establishes quantifiable topological relationships through row and column mapping. This projection method overcomes the information loss problem caused by fixed resolution or simple dimensionality reduction in traditional methods. It is particularly suitable for the top-down installation of depth cameras in this application, providing a high-precision data structure foundation for subsequent pitch angle calculation and obstacle recognition based on row and column relationships, significantly improving the detection accuracy and real-time performance of the algorithm in complex terrain.

[0032] Specifically, the steps for calculating the pitch angle changes of adjacent point clouds include: Let P1(x1,y1,z1) and P2(x2,y2,z2) be adjacent point clouds in a two-dimensional matrix; Then the pitch angle change diff between adjacent point clouds P1 and P2 ang The calculation formula is: , In the formula, diff x diff y diff z These are the coordinate differences between points P1 and P2 on the X-axis, Y-axis, and Z-axis, respectively.

[0033] Understandably, the pitch angle calculation formula described above significantly improves obstacle detection accuracy by normalizing height changes and horizontal distance. This formula considers not only vertical height differences but also horizontal displacement, ensuring the calculation results accurately reflect actual slope changes between point clouds and effectively eliminating errors caused by uneven point cloud density or varying measurement distances. Compared to simple plane fitting detection, this angle calculation method based on geometric relationships can more accurately distinguish between real obstacles and terrain undulations, making it particularly suitable for detecting low obstacles in complex environments and significantly reducing the false positive rate caused by fixed height thresholds in traditional methods.

[0034] Please refer to the following: Figure 4 The obstacle recognition method based on depth point clouds provided in the second embodiment of this application will now be described. The second embodiment is based on the first embodiment.

[0035] Specifically, the steps for extracting obstacle information based on the preliminary obstacle point cloud include: The initial obstacle point cloud is clustered, and noise is removed by setting the number of points in the clustered point cloud as a condition. The height of each clustered point cloud is calculated. The height difference of this type of point cloud is calculated and compared with the height of a qualified obstacle. If the calculated height difference is greater than the height of a qualified obstacle, the point cloud is considered to be a real obstacle point cloud. Obstacles are generated in the map based on real obstacle point clouds.

[0036] Understandably, the combination of point cloud clustering and height calculation for each type of point cloud significantly improves the accuracy and robustness of obstacle detection: the clustering operation classifies the discrete initial obstacle point cloud into physically meaningful independent objects, effectively filtering out noise points and isolated false positives; subsequently, the height difference is calculated for each type of point cloud to further verify the authenticity and size characteristics of the obstacle, ensuring that only significant obstacles that meet the height threshold are finally identified, thereby achieving high-precision obstacle detection in complex environments while avoiding false positives for minor ground undulations or sensor noise.

[0037] The following is a description of the path planning methods provided in the third to seventh embodiments of this application.

[0038] In existing cost maps for path planning, the initial cost value of planar point clouds mapped to the cost map is zero. Only obstacles are assigned grid cost value. This means that the cost map does not provide fine-grained classification of planar surfaces, failing to distinguish the geometric and physical characteristics of different planar terrains (such as gentle slopes, uneven surfaces, or slippery surfaces). The robot may choose a geometrically feasible but actually unsafe path (such as a steep slope or a slippery area). Secondly, the lack of quantitative assessment of subtle undulations in the road surface can lead to robot bumps or increased energy consumption. More importantly, this coarse-grained processing can mask potentially risky transition areas (such as the junction of a slope and a plane), preventing the robot from avoiding potential motion control risks or slippage hazards in advance. This results in insufficient safety when the robot plans paths based on the cost map.

[0039] The path planning method provided in the third embodiment of this application will now be described. The third embodiment is based on the first and / or second embodiments.

[0040] The path planning method includes the following steps: The planar point cloud in the two-dimensional matrix is ​​clustered according to the pitch angle change to generate several planar clusters; The first generation of values ​​is generated based on the pitch angle variation characteristics of each planar cluster in the same row or column of a two-dimensional matrix. After mapping the two-dimensional matrix to the cost map, the initial generation value of the raster corresponding to the planar cluster is assigned based on the first generation value; Path planning is based on this cost map.

[0041] However, the aforementioned method achieves efficient modeling and evaluation of complex terrain by projecting point clouds onto a two-dimensional matrix and performing cluster analysis based on pitch angle variation characteristics. Specifically, it first generates planar clusters through pitch angle variation clustering, effectively distinguishing continuous terrain from real obstacles; then, it generates a first-generation value based on the pitch angle variation rate and amplitude in the row / column directions, quantifying the terrain's drivability; finally, by mapping to a cost map and assigning the initial generation value, it provides a refined terrain assessment for path planning. This approach preserves the three-dimensional geometric features of the point cloud while achieving a balance between computational efficiency and navigation accuracy, significantly improving the robot's terrain perception and decision-making capabilities in complex environments, thus comprehensively enhancing the robot's safety.

[0042] The path planning method provided in the fourth embodiment of this application will now be described. The fourth embodiment is based on the first and / or second embodiments.

[0043] The path planning method includes the following steps: The planar point cloud in the two-dimensional matrix is ​​clustered according to the pitch angle change to generate several planar clusters; The target type is obtained by classifying the RGB image region corresponding to each planar cluster, and a second-generation value is generated based on the target type. After mapping the two-dimensional matrix to the cost map, the initial generation value of the raster corresponding to the planar cluster is assigned based on the second generation value; Path planning is based on this cost map.

[0044] Understandably, the above method achieves refined classification and evaluation of planar regions by combining the geometric features of 3D point clouds with the semantic information of RGB images: First, planar clusters are generated based on pitch angle changes. Then, target types with different physical characteristics (such as rubber roads, cement roads, grass, water accumulation, sand, etc.) are identified through RGB images, and a second-generation value reflecting the actual difficulty of passage is generated. Finally, different plane types are assigned differentiated values ​​in the cost map, enabling the robot path planning not only to avoid obstacles, but also to intelligently select the optimal road surface type (such as prioritizing hardened roads over grass), significantly improving the safety, smoothness, and energy efficiency of navigation. At the same time, it solves the planning defect of traditional methods that are "geometrically feasible but practically infeasible" due to the neglect of semantic information.

[0045] Please refer to the following: Figure 5 and Figure 6 The path planning method provided in the fifth embodiment of this application will now be described. The fifth embodiment is based on the first and / or second embodiments.

[0046] The path planning method includes the following steps: The planar point cloud in the two-dimensional matrix is ​​clustered according to the pitch angle change to generate several planar clusters; The first generation of values ​​is generated based on the pitch angle variation characteristics of each planar cluster in the same row or column of a two-dimensional matrix. The target type is obtained by classifying the RGB image region corresponding to each planar cluster, and a second-generation value is generated based on the target type. After mapping the two-dimensional matrix onto the cost map, the initial generation value of the raster corresponding to the planar cluster is assigned based on the first generation value and the second generation value. Path planning is based on this cost map.

[0047] Understandably, this method innovatively integrates point cloud geometric features with visual semantic information reflecting physical properties, achieving complementary advantages in multi-dimensional environmental assessment: the first-generation value generated through pitch angle variation accurately quantifies terrain geometric features (such as slope and flatness), solving the problem of pure visual methods being insensitive to geometric features; simultaneously, the second-generation value generated based on RGB classification supplements the semantic level with physical attributes (such as road surface material, softness, friction coefficient, and safety), compensating for the lack of semantic understanding in pure point cloud methods. This intelligent fusion ensures both accurate perception of complex terrain geometric features by the robot and functional judgment of road surface types, enabling the path planning system to simultaneously consider "geometric physical drivability" and "practical applicability," achieving an optimal balance between safety and efficiency in complex scenarios.

[0048] The path planning method provided in the sixth embodiment of this application will now be described. The sixth embodiment is based on the fifth embodiment.

[0049] The steps for assigning an initial generation value to the raster corresponding to the planar cluster based on the first-generation and second-generation values ​​include: The sum of the first-generation value and the second-generation value is used as the initial generation value of the raster corresponding to the planar cluster.

[0050] The path planning method provided in the seventh embodiment of this application will now be described. The seventh embodiment is based on the fifth embodiment.

[0051] The steps for assigning an initial generation value to the raster corresponding to the planar cluster based on the first-generation and second-generation values ​​include: Based on the degree of influence of geometric terrain features on robot passage according to different classification results, a ratio coefficient between the first-generation value and the second-generation value is preset. The initial generation value of the raster corresponding to the planar cluster is obtained by weighted summation using scaling factors.

[0052] Understandably, this method achieves intelligent and adaptive optimization of environmental assessment by dynamically adjusting the weight ratio of geometric features and semantic information: based on the actual impact of different types of terrain on robot passage (e.g., steep slopes emphasize geometric features, while slippery surfaces emphasize material properties), differentiated value ratio coefficients are preset, and then the final generation value is generated through weighted fusion. This flexible weighting mechanism retains the advantage of point cloud data in accurately quantifying terrain geometric characteristics while fully leveraging the ability of visual information to identify road surface materials. This allows the cost map to automatically adjust the assessment focus according to different scene characteristics, providing more accurate environmental assessment results that better meet actual passage needs for path planning, thereby significantly improving the robot's navigation adaptability and decision-making rationality in complex and changing environments.

[0053] It is worth noting that in the third to seventh embodiments of this application, the pitch angle change characteristic can refer to either the pitch angle change rate or the pitch angle change magnitude. Preferably, the pitch angle change characteristic includes both the pitch angle change rate and the pitch angle change magnitude, in order to more accurately assess the degree of influence of geometric terrain features on the robot's passage.

[0054] It is worth noting that in the fourth to seventh embodiments of this application, the classification refers to classification detection using a deep learning classification model to distinguish target types, wherein the method of classification detection using a deep learning classification model is a mature existing technology.

[0055] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An obstacle recognition method based on deep point clouds, characterized in that, include: The raw depth point cloud data is acquired by a depth camera and its three-dimensional information is converted into a two-dimensional matrix. By comparing the pitch angle changes of adjacent point clouds with a set threshold using a two-dimensional matrix, planar point clouds can be distinguished from preliminary obstacle point clouds. Obstacle information is extracted based on the preliminary obstacle point cloud.

2. The obstacle recognition method based on deep point clouds as described in claim 1, characterized in that, After acquiring the raw depth point cloud data and before converting its 3D information into a 2D matrix, the following steps are also included: Calculate the angle between each depth point cloud and the X-axis in the depth camera coordinate system. x ; If angle x >a*f, where a is the scaling factor and f is the horizontal field of view angle of the depth camera, then the depth point cloud is filtered. Where the included angle x The formula is: ， In the formula, x and y are the X and Y axis coordinates of the corresponding depth point cloud.

3. The obstacle recognition method based on deep point clouds as described in claim 1, characterized in that, The steps to convert the 3D information of the original depth point cloud data into a 2D matrix include: Calculate the row number of the point cloud projection into the two-dimensional matrix for each depth. Id row number Id The calculation formula is: ， In the formula, x, y, and z are the X, Y, and Z axis coordinates of the corresponding depth point cloud, respectively. xy Let ang be the horizontal distance of the depth point cloud projected onto the XY plane in the camera coordinate system. z Let ang be the angle between the depth point cloud and the XY plane. vmin ang is the minimum pitch angle of the depth camera. v This represents the angular resolution of the depth camera in the vertical direction. Calculate the column number col of the point cloud projection into the two-dimensional matrix at each depth. Id Column number col Id The calculation formula is: ， In the formula, ang y Let ang be the angle between the depth point cloud and the Y-axis. hmin ang is the minimum horizontal angle of the depth camera. h The horizontal angular resolution of the depth camera; According to the row number Id With column number col Id Determine the position of each depth point cloud in a two-dimensional matrix.

4. The obstacle recognition method based on deep point clouds as described in claim 3, characterized in that, The steps for calculating the pitch angle variation of adjacent point clouds include: Let P1(x1,y1,z1) and P2(x2,y2,z2) be adjacent point clouds in a two-dimensional matrix; Then the pitch angle change diff between adjacent point clouds P1 and P2 ang The calculation formula is: ， In the formula, diff x diff y diff z These are the coordinate differences between points P1 and P2 on the X-axis, Y-axis, and Z-axis, respectively.

5. The obstacle recognition method based on deep point clouds as described in claim 1, characterized in that, The steps for extracting obstacle information based on the preliminary obstacle point cloud include: The initial obstacle point cloud is clustered, and noise is removed by setting the number of points in the clustered point cloud as a condition. The height of each clustered point cloud is calculated. The height difference of this type of point cloud is calculated and compared with the height of a qualified obstacle. If the calculated height difference is greater than the height of a qualified obstacle, the point cloud is considered to be a real obstacle point cloud. Obstacles are generated in the map based on real obstacle point clouds.

6. A path planning method, characterized in that, The obstacle recognition method based on deep point clouds as described in any one of claims 1-5 includes the following steps: The planar point cloud in the two-dimensional matrix is ​​clustered according to the pitch angle change to generate several planar clusters; The first generation of values ​​is generated based on the pitch angle variation characteristics of each planar cluster in the same row or column of a two-dimensional matrix. After mapping the two-dimensional matrix to the cost map, the initial generation value of the raster corresponding to the planar cluster is assigned based on the first generation value; Path planning is based on this cost map.

7. A path planning method, characterized in that, The obstacle recognition method based on deep point clouds as described in any one of claims 1-5 includes the following steps: The planar point cloud in the two-dimensional matrix is ​​clustered according to the pitch angle change to generate several planar clusters; The target type is obtained by classifying the RGB image region corresponding to each planar cluster, and a second-generation value is generated based on the target type. After mapping the two-dimensional matrix to the cost map, the initial generation value of the raster corresponding to the planar cluster is assigned based on the second generation value; Path planning is based on this cost map.

8. A path planning method, characterized in that, The obstacle recognition method based on deep point clouds as described in any one of claims 1-5 includes the following steps: The planar point cloud in the two-dimensional matrix is ​​clustered according to the pitch angle change to generate several planar clusters; The first generation of values ​​is generated based on the pitch angle variation characteristics of each planar cluster in the same row or column of a two-dimensional matrix. The target type is obtained by classifying the RGB image region corresponding to each planar cluster, and a second-generation value is generated based on the target type. After mapping the two-dimensional matrix onto the cost map, the initial generation value of the raster corresponding to the planar cluster is assigned based on the first generation value and the second generation value. Path planning is based on this cost map.

9. The path planning method as described in claim 8, characterized in that, The steps for assigning an initial generation value to the raster corresponding to the planar cluster based on the first-generation and second-generation values ​​include: The sum of the first-generation value and the second-generation value is used as the initial generation value of the raster corresponding to the planar cluster.

10. The path planning method as described in claim 8, characterized in that, The steps for assigning an initial generation value to the raster corresponding to the planar cluster based on the first-generation and second-generation values ​​include: Based on the degree of influence of geometric terrain features on robot passage according to different classification results, a ratio coefficient between the first-generation value and the second-generation value is preset. The initial generation value of the raster corresponding to the planar cluster is obtained by weighted summation using scaling factors.

Images