Dynamic point cloud segmentation and prediction label correction method of inspection robot

Abstract

The invention discloses a dynamic point cloud segmentation and prediction label correction method of an inspection robot. The dynamic point cloud segmentation and prediction label correction method is mainly divided into two parts: dynamic point cloud identification and prediction point cloud label correction; through filtering is carried out on collected point cloud data, dimension reduction projection is carried out on a region of interest so as to optimize processing efficiency, appearance and motion characteristics of the preprocessed point cloud are extracted by using a sliding time window, error residual errors possibly caused by noise and shielding are solved by using an appearance-motion bridging double-branch network, and the accuracy of the point cloud processing is improved. And finally generating a predicted moving object label. In the prediction process, aiming at a possible false label problem, a density-based spatial clustering method is adopted to carry out clustering analysis on false labels, and calibration is carried out again. According to the method, the dynamic point cloud can be separated from the input data, the segmentation effect is enhanced through label correction, and the accuracy of dynamic point identification is improved.

Description

[0001] Technology Neighborhood

[0002] The present invention relates to the field of robot data processing technology, and in particular to a method for dynamic point cloud segmentation and predicted label correction of an inspection robot. Background Art

[0003] The train undercarriage inspection robot has attracted widespread attention due to its potential value in improving rail transit safety and maintenance efficiency. The main task of the train undercarriage inspection robot is to conduct regular inspections of the undercarriage of subway trains in the maintenance trench to ensure operational safety. The robot moves in the maintenance trench, stops at preset inspection points, and collects images and features of the undercarriage components of the train through the camera at the end of its robotic arm. Based on the collected data, the robot can evaluate the maintenance needs of the train and make maintenance recommendations. During the inspection process, the robot uses simultaneous localization and mapping (SLAM) based on hybrid solid-state LiDAR (laser radar).

[0004] The environmental features in the maintenance trench are highly repetitive and usually have a long and narrow shape, which is a degenerate scenario. In the absence of artificial external assistance such as QR codes or magnetic strips, a 3D LiDAR-based SLAM system is needed to achieve robot positioning. At present, the robustness of the 3D laser SLAM system to dynamic object interference is significantly lower than that of the 2D laser SLAM system, which brings difficulties to the actual robot operation in the shared human-machine maintenance trench environment. During robot deployment teaching or autonomous inspection, the deployment personnel or maintenance personnel of human-machine collaboration must leave the maintenance pit when the robot needs to move to the next position. After the robot stops, the personnel re-enter the maintenance pit for robotic arm teaching or collaborative maintenance to avoid the dynamic point cloud generated by "personnel following the robot" from polluting the robot point cloud map, thereby affecting the positioning accuracy. Repeated entry and exit of the maintenance trench affects the efficiency of deployment teaching and reduces the flexibility of human-machine collaboration. Summary of the invention

[0005] In order to overcome the shortcomings and deficiencies of the prior art, the present invention provides a method for dynamic point cloud segmentation and prediction label correction of an inspection robot. It can be added as a data preprocessing module at the SLAM front end to solve the impact of dynamic point cloud on the SLAM system of the train undercarriage inspection robot.

[0006] A method for dynamic point cloud segmentation and prediction label correction of an inspection robot installs a hybrid solid-state 3D laser radar on the robot body to realize the collection of 3D point cloud data, autonomous mapping and path planning. The industrial computer is responsible for running the core algorithm to realize the dynamic point cloud segmentation and prediction label correction functions.

[0007] The dynamic point cloud segmentation and predicted label correction method is mainly divided into two parts: dynamic point cloud recognition and predicted point cloud label correction; due to the huge amount of point cloud data and heavy computational burden, this method performs straight-through filtering on the collected point cloud data and performs dimensionality reduction projection on the region of interest to optimize processing efficiency. The appearance and motion features of the preprocessed point cloud are extracted using a sliding time window, and the double-branch network of appearance-motion bridging is used to solve the erroneous residuals that may be caused by noise and occlusion, and finally generate the predicted moving object label; in the prediction process, for possible mislabeling problems, a density-based spatial clustering method is used to cluster the mislabels and recalibrate them to improve the accuracy of the labels.

[0008] The steps for implementing the dynamic point cloud segmentation and prediction label correction method are as follows:

[0009] Step S1, Li DAR input preprocessing: define the point cloud data as: In order to prevent interference from objects outside the inspection channel, a straight-through filter is used to crop the point cloud and only extract the point cloud within the inspection channel area. For solid-state LiDAR with a small field of view and non-repetitive scanning, a bird's-eye view (BEV) projection method is used to project the 3D point cloud into a polar coordinate system to balance the uneven spatial distribution of the point cloud. Define F (u,v),i is the Grid defined by the (u,v) coordinates in the polar coordinate BEV image th The set of all points in a cell is expressed as:

[0010]

[0011] Among them, F (u,v),i Represents the polar coordinate top view image in the grid cell defined by the (u,v) coordinates, belonging to the point cloud frame F i The set of all points of p. j Represents the point cloud frame F i A point in the point cloud, located at the polar coordinates (θ j ,p j )Down. This inequality means that the point p j The angle θ j The range that is satisfied. max and θ min are the maximum and minimum values ​​of the angle, respectively. h represents the height of the polar coordinate top view image, which divides the angle into h units. v represents the row index in the polar coordinate image. This condition constrains ρ j Must be within the angle range of row v. This inequality means that the point p j The distance ρ j The range that is satisfied. max and ρmin are the maximum and minimum values ​​of the distance, respectively. w represents the width of the polar coordinate top view image, which divides the distance into w units. u represents the column index in the polar coordinate image. This condition constrains p j Must be within the u-th column distance range.

[0012] Step S2, dynamic object recognition module: when processing the current frame F i When the current frame and its previous and next frames are integrated, a sliding window is constructed for comprehensive analysis. First, the laser odometer is used to determine the transformation matrix describing the spatial evolution between the current frame and the previous n frames, which is expressed as:

[0013]

[0014] in, Represents the overall transformation matrix from time in to time i. This matrix transforms the point cloud data at time in to the coordinate system of time i. Represents the single-step transformation matrix from time in to time i-k+1. This matrix can be regarded as the pose transformation of the i-k+1th frame relative to the inth frame. It means that from k = 1 to k = n Multiply each term to get the overall transformation This represents a step-by-step accumulation over multiple frames.

[0015] Then, the corresponding point cloud set is calculated using the transformation matrix:

[0016]

[0017] in, It means that after transformation, the point cloud frame F at time in i-n The new point cloud frame converted to the coordinate system of time i. This set contains all the points in F i-n The point in the i i-n After transformation, it is located in the coordinate system of time i. represents the overall transformation matrix from time in to time i. This matrix is ​​for each point p j Perform coordinate transformation and convert F i-n The point cloud is transformed from the coordinate system at time in to the coordinate system at time i. j Represents the point cloud frame F i-n A point in the j is the coordinate in the coordinate system at time in.

[0018] The motion features required for identifying dynamic objects are extracted by examining the height differences in the BEV images. To address the problem caused by the low density of the point cloud, point clouds extracted from two consecutive time windows W1 and W2 are used. The motion features are generated by evaluating the height variance of the corresponding grid cells in the two time periods.

[0019] First, ego motion compensation is performed by relative pose transformation to align W1 and W2 to the current local coordinate system. Define H Grid,a Each pixel value represents the height of the grid in Wa, and its expression is:

[0020] H Grid,a =Max{Z Grid,a}-Min{Z Grid,a},a∈{1,2}

[0021] Among them, H Grid,a Indicates the height difference of the cell grid Grid in the ath sliding window. This value is represented by the difference between the maximum height and the minimum height in the window. Grid,a Represents all height values ​​of the unit grid Grid in the ath sliding window, that is, the height of each point in the grid area. Max{Z Grid,a} represents the maximum height of all points in the unit grid Grid in the a-th sliding window. Min{Z Grid,a} represents the minimum height of all points of the unit grid Grid in the ath sliding window.

[0022] in,

[0023] Z Grid,a ={z j ∈p j |p j ∈W Grid,a,}

[0024] Among them, Z Grid,a Represents the set of height values ​​of the cell grid Grid in the a-th sliding window. j Represents the Z coordinate of a specific point, that is, point p j The value on the Z axis. p j Represents a point in the grid a. Grid,a, Represents the Grid grid area in the a-th sliding window.

[0025] definition represents the motion features in the nth channel of the i-th frame, where the value of n is determined by the position of the frame in the time window. When a new frame arrives, both time windows move to the next position. The residual is obtained by subtracting H1 and H2:

[0026]

[0027] in, These are the motion features of each channel of the grid cell Grid at different time frames. Represents the motion features of the 0th channel of the i-th frame. Represents the motion features of the 1st channel of the i-1th frame. Represents the motion features of the 2nd channel of the i-2th frame. Grid,1 and H Grid,2 They represent the height difference of the grid cell Grid in the first sliding window and the height difference of the second sliding window on the grid cell Grid, that is, the difference between the maximum and minimum heights in each area. Grid,1 -H Grid,2 This is the height difference between the first sliding window unit and the second sliding window. By calculating the height difference, we get the residual, which is the height change between the two sliding windows.

[0028]

[0029] in, These are the motion features of each channel of the grid cell Grid at different time frames. Represents the motion features of the 3rd channel of the i-3th frame. Represents the motion features of the 4th channel of the i-4th frame. Represents the motion features of the 5th channel of the i-5th frame. Grid,1 and H Grid,2 They represent the height difference of the first sliding window on the grid unit Grid and the height difference of the second sliding window on the grid unit Grid, that is, the difference between the maximum and minimum heights in each area. Grid,2 -H Grid,1 This is the height difference between the second sliding window unit and the first sliding window. By calculating the height difference, we get the residual, which is the height change between the two sliding windows.

[0030] Step S3, network architecture: Based on the dual-branch structure of PolarNet, combined with the appearance-motion co-attention mechanism. The maximum pooling operation is used to capture the distribution characteristics of the BEV projected point cloud within the vertical range of the grid unit. In order to enhance the cross-modal interaction between appearance and motion features, the appearance-motion co-attention module (AMCM) is introduced. This module consists of two important components: a co-attention gate for aligning the salient features of the two modalities, and a motion-guided attention module based on motion cues for refining the focus points according to motion cues.

[0031] Step S4, segmentation correction based on spatial clustering: This dynamic point cloud recognition method may still encounter the problem of misidentification, and some dynamic point clouds are not correctly marked. In order to solve this problem, a combination of density-based spatial clustering (DBSCAN) and cloth simulation filtering (CSF) is adopted. DBSCAN is a density-based spatial clustering algorithm that can identify and segment high-density areas in the data set into multiple clusters, while identifying isolated points as noise. The core of the algorithm is to identify the core points in the neighborhood and perform clustering based on this. The specific DBSCAN requires two parameters, namely the neighborhood radius ε (eps) and the minimum number of points minPts required to form a dense area. The ε neighborhood of the starting point is retrieved. If the number of points in the neighborhood exceeds minPts, the point is defined as the core point of the cluster and a new cluster is started; otherwise, the point is marked as noise. Once a point is identified as a dense part of a cluster, the points in its ε neighborhood will also be included in the same cluster. If these points also meet the dense condition, the points in their ε neighborhood will continue to be added to the cluster. This process continues to iterate until all density-connected points in the cluster are identified. After the current cluster is fully determined, the algorithm reselects new unvisited points for cluster identification or noise marking until all data points have been processed.

[0032] Considering the existence of ground point clouds, there is a correlation between spatial point clouds. DBSCAN clustering requires that the point clouds in the target cluster meet the density accessibility condition, which easily leads to a large number of ground and non-ground point clouds being clustered in the same cluster. To this end, CSF filtering is applied to effectively separate ground point clouds so that non-ground point clouds can meet the requirements of DBSCAN clustering, thereby more accurately identifying and segmenting dynamic point clouds.

[0033] The beneficial technical effects of the present invention are:

[0034] Aiming at the working scene characteristics of the under-vehicle inspection robot, this paper proposes a low-cost hybrid solid-state Li DAR dynamic point cloud segmentation method, which can separate dynamic point clouds from input data, enhance the segmentation effect through label correction, and improve the accuracy of dynamic point recognition. This method can effectively identify and segment dynamic point clouds, and minimize the interference of dynamic point clouds to SLAM, thereby improving the overall work efficiency and accuracy of the under-vehicle inspection robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a schematic diagram of the robot working scene structure;

[0036] Figure 2 It is the overall technical flow chart of the present invention;

[0037] Figure 3 It is a simulated inspection channel diagram of the present invention;

[0038] Figure 4 It is a dynamic point cloud recognition module diagram of the present invention;

[0039] Figure 5 It is a flow chart of dynamic point cloud tag clustering correction of the present invention;

[0040] Figure 6 This is a dynamic point cloud tag clustering correction result diagram of the present invention; DETAILED DESCRIPTION

[0041] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application may be combined with each other. The present application is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0042] like Figure 1 As shown in the figure, a method for dynamic point cloud segmentation and predicted label correction of an inspection robot is installed with: Livox-Mid 70 solid-state laser radar, Zivid 2 camera, UR5e robotic arm and industrial computer, and dynamic object point cloud recognition and segmentation are performed in the constructed simulated inspection channel. The steps are as follows:

[0043] Combination Figure 2 As shown, the specific steps include the following steps.

[0044] Step S1, LiDAR input preprocessing: define point cloud data as: In order to prevent interference from objects outside the simulated inspection channel, a straight-through filter is used to crop the point cloud and only extract the point cloud within the simulated inspection channel area. Here, the point cloud with a height of 2m is extracted. For solid-state LiDAR with a small field of view and non-repetitive scanning, the bird's eye view (BEV) projection method is used to project the 3D point cloud into the polar coordinate system to balance the uneven spatial distribution of the point cloud. Define F (u,v),i is the Grid defined by the (u,v) coordinates in the polar coordinate BEV image th The set of all points in a cell is expressed as:

[0045]

[0046] Among them, F (u,v),i Represents the polar coordinate top view image in the grid cell defined by the (u,v) coordinates, belonging to the point cloud frame F i The set of all points of p. j Represents the point cloud frame F i A point in the point cloud, located at the polar coordinates (θ j ,p j )Down. This inequality means that the point p j The angle θ j The range that is satisfied.max and θ min are the maximum and minimum values ​​of the angle, respectively. h represents the height of the polar coordinate top view image, which divides the angle into h units. v represents the row index in the polar coordinate image. This condition constrains ρ j Must be within the angle range of row v. This inequality means that the point p j The distance ρ j The range that is satisfied. max and ρ min are the maximum and minimum values ​​of the distance, respectively. w represents the width of the polar coordinate top view image, which divides the distance into w units. u represents the column index in the polar coordinate image. This condition constrains p j Must be within the u-th column distance range.

[0047] Step S2, dynamic object recognition module: when processing the current frame F i When the current frame and its previous and next frames are integrated, a sliding window is constructed for comprehensive analysis. First, the laser odometer is used to determine the transformation matrix describing the spatial evolution between the current frame and the previous n frames, where n = 5, expressed as:

[0048]

[0049] in, Represents the overall transformation matrix from time in to time i. This matrix transforms the point cloud data at time in to the coordinate system of time i. Represents the single-step transformation matrix from time in to time i-k+1. This matrix can be regarded as the pose transformation of the i-k+1th frame relative to the inth frame. It means that from k = 1 to k = n Multiply each term to get the overall transformation This represents a step-by-step accumulation over multiple frames.

[0050] Then, the corresponding point cloud set is calculated using the transformation matrix:

[0051]

[0052] in, It means that after transformation, the point cloud frame F at time in i-n The new point cloud frame converted to the coordinate system of time i. This set contains all the points in F i-n point in After transformation, it is located in the coordinate system of time i. represents the overall transformation matrix from time in to time i. This matrix is ​​for each point p j Perform coordinate transformation and convert F i-nThe point cloud is transformed from the coordinate system of time in to the coordinate system of time i. j Represents the point cloud frame F i-n A point in the j is the coordinate in the coordinate system at time in.

[0053] The motion features required for identifying dynamic objects are extracted by examining the height differences in the BEV images. To address the problem caused by the low density of the point cloud, point clouds extracted from two consecutive time windows W1 and W2 are used. The motion features are generated by evaluating the height variance of the corresponding grid cells in the two time periods.

[0054] First, ego motion compensation is performed by relative pose transformation to align W1 and W2 to the current local coordinate system. Define H Grid,a Each pixel value represents the height of the grid in Wa, and its expression is:

[0055] H Grid,a =Max{Z Grid,a}-Min{Z Grid,a},a∈{1,2}

[0056] Among them, H Grid,a Indicates the height difference of the cell grid Grid in the ath sliding window. This value is represented by the difference between the maximum height and the minimum height in the window. Grid,a Represents all height values ​​of the unit grid Grid in the a-th sliding window, that is, the height of each point in the grid area. Max{Z Grid,a} represents the maximum height of all points in the unit grid Grid in the a-th sliding window. Min{Z Grid,a} represents the minimum height of all points of the unit grid Grid in the ath sliding window.

[0057] Z Grid,a ={z j ∈p j |p j ∈W Grid,a,}

[0058] Among them, Z Grid,a Represents the set of height values ​​of the cell grid Grid in the a-th sliding window. j Represents the Z coordinate of a specific point, that is, point p j The value on the Z axis. p j Represents a point in the grid a. Grid,a, Represents the Grid grid area in the a-th sliding window.

[0059] definition represents the motion features in the nth channel of the i-th frame, where the value of n is determined by the position of the frame in the time window. When a new frame arrives, both time windows move to the next position. The residual is obtained by subtracting H1 and H2:

[0060]

[0061] in, These are the motion features of each channel of the grid cell Grid at different time frames. Represents the motion features of the 0th channel of the i-th frame. Represents the motion features of the 1st channel of the i-1th frame. Represents the motion features of the 2nd channel of the i-2th frame. Grid,1 and H Grid,2 They represent the height difference of the grid cell Grid in the first sliding window and the height difference of the second sliding window on the grid cell Grid, that is, the difference between the maximum and minimum heights in each area. Grid,1 -H Grid,2 This is the height difference between the first sliding window unit and the second sliding window. By calculating the height difference, we get the residual, which is the height change between the two sliding windows.

[0062]

[0063] in, These are the motion features of each channel of the grid cell Grid at different time frames. Represents the motion features of the 3rd channel of the i-3th frame. Represents the motion features of the 4th channel of the i-4th frame. Represents the motion features of the 5th channel of the i-5th frame. Grid,1 and H Grid,2 They represent the height difference of the first sliding window on the grid unit Grid and the height difference of the second sliding window on the grid unit Grid, that is, the difference between the maximum and minimum heights in each area. Grid,2 -H Grid,1 This is the height difference between the second sliding window unit and the first sliding window. By calculating the height difference, we get the residual, which is the height change between the two sliding windows.

[0064] Step S3, network architecture: Based on the dual-branch structure of PolarNet, combined with the appearance-motion co-attention mechanism. The maximum pooling operation is used to capture the distribution characteristics of the BEV projected point cloud within the vertical range of the grid unit. In order to enhance the cross-modal interaction between appearance and motion features, the appearance-motion co-attention module (AMCM) is introduced. This module consists of two important components: a co-attention gate for aligning the salient features of the two modalities, and a motion-guided attention module based on motion cues for refining the focus points according to motion cues.

[0065] Step S4, segmentation correction based on spatial clustering: This dynamic point cloud recognition method may still encounter the problem of misidentification, and some dynamic point clouds are not correctly marked. In order to solve this problem, a combination of density-based spatial clustering (DBSCAN) and cloth simulation filtering (CSF) is adopted. DBSCAN is a density-based spatial clustering algorithm that can identify and segment high-density areas in the data set into multiple clusters, while identifying isolated points as noise. The core of the algorithm is to identify the core points in the neighborhood and perform clustering based on this. The specific DBSCAN requires two parameters, namely the neighborhood radius ε (eps) and the minimum number of points minPts required to form a dense area. The ε neighborhood of the starting point is retrieved. If the number of points in the neighborhood exceeds minPts, the point is defined as the core point of the cluster and a new cluster is started; otherwise, the point is marked as noise. Once a point is identified as a dense part of a cluster, the points in its ε neighborhood will also be included in the same cluster. If these points also meet the dense condition, the points in their ε neighborhood will continue to be added to the cluster. This process continues to iterate until all density-connected points in the cluster are identified. After the current cluster is completely determined, the algorithm reselects new unvisited points for cluster identification or noise marking until all data points are processed. Here, ε(eps) = 0.5 and minPts = 10.

[0066] Considering the existence of ground point clouds, there is a correlation between spatial point clouds. DBSCAN clustering requires that the point clouds in the target cluster meet the density accessibility condition, which easily leads to a large number of ground and non-ground point clouds being clustered in the same cluster. To this end, CSF filtering is applied to effectively separate ground point clouds so that non-ground point clouds can meet the requirements of DBSCAN clustering, thereby more accurately identifying and segmenting dynamic point clouds.

[0067] like Figure 3 The figure shows the comparison between the real inspection channel and the simulated channel built with cartons. The real channel diagram shows the actual inspection scene of the inspection robot, while the simulated inspection channel uses materials such as cartons to build the basic structure of the channel, providing a controlled experimental environment.

[0068] Combination Figure 4The dynamic point cloud recognition module shown in the figure was tested in a simulated train inspection channel. Considering the actual application environment of the algorithm design, only people were classified as dynamic objects in the data set, while other objects such as walls were classified as static background point clouds. The specific steps include:

[0069] Step Q1, time window processing: push the current scan data into the time window, and convert all the scan data in the window into point cloud data under the current perspective

[0070] Step Q2, motion feature extraction: Divide the 6 frames of point cloud data into two sliding time windows for processing, so as to extract motion features for each frame of point cloud data. Specifically, after 6 motion iterations, 6 channels of motion features are extracted, which represent the motion information of the point cloud over time. More specifically, Figure 4 The black, white, and gray circles in the middle simulate the existence states of the point clouds of pedestrians in different frames after the point clouds of each frame are converted to the current frame in the sliding window. This process can effectively capture the motion trajectory of dynamic objects and provide a key basis for subsequent dynamic point cloud recognition and positioning.

[0071] Step Q3, appearance feature learning: Pop out the scan data from the time window, use the PointNet algorithm to learn the appearance features of the pedestrian point cloud, and extract the geometric feature information of the point cloud.

[0072] Step Q4, spatiotemporal information fusion: Input the appearance features and motion features into the dual-branch network bridged by AMCM to perform joint learning and fusion of spatiotemporal information.

[0073] In terms of the dataset format, Semantic KITTI was used as a reference, and the dataset coordinates were normalized. Table 1 shows the results of the comparison with PointNet and PointNet++ algorithms, focusing on the two indicators of ACC (accuracy) and mIoU (mean intersection over union). To better understand the contents in the table, the following is a detailed introduction to the two algorithms and the two evaluation indicators. PointNet is a network model that directly processes three-dimensional point clouds without converting point cloud data into grids or voxels. It handles the disorder of point clouds by aggregating the features of each point. PointNet uses a shared multi-layer perceptron (MLP) to extract features point by point and finally summarizes them into global features. Its advantages are simple structure and high computational efficiency, which are suitable for fast processing of simple geometric structures. However, its limitation is that it is not good at capturing local geometric features, and may not perform well in processing complex or detailed point clouds. PointNet++ enhances the ability to capture local structures on the basis of PointNet, and is suitable for point clouds with multi-scale and complex geometry. Through hierarchical sampling, multi-scale local features are extracted and aggregated, and features in local areas are summarized through neighborhood definitions (such as spherical neighborhoods), so as to better adapt to complex geometric environments. Its advantage is that it is good at capturing multi-scale geometric details and is suitable for scenes with rich details. However, its computational cost is high and its processing speed is relatively slow. ACC represents the overall accuracy of the model in classifying or segmenting point cloud data, which is specifically defined as:

[0074]

[0075] The higher the Acc value, the higher the overall accuracy of the model in recognition and classification tasks. mIoU (Mean Intersection over Union) is a commonly used segmentation task metric that evaluates segmentation performance by comparing the overlap area between the model prediction and the actual label. mIoU is the average of all categories of IoU. The calculation formula for IoU is:

[0076]

[0077] mIoU is the average of the IoUs of all categories to evaluate the overall accuracy in multi-category segmentation tasks:

[0078]

[0079] Among them, the higher the mIoU value, the stronger the model's ability to distinguish all categories in the segmentation task; (detailed introduction and explanation of the algorithms in the table and what ACC means) Figure 6 Results are shown using the label correction method. Figure 6In the figure, for the two randomly selected frames of point cloud data, we can see that the point cloud predicted by the point cloud recognition module is not completely correctly labeled, and there are parts of the pedestrian point cloud data that are not correctly recognized. Figure 5 The dynamic point cloud label clustering correction method is as follows:

[0080] Step R1, point cloud segmentation and clustering: First, the CSF filtering algorithm is used for the original point cloud data to divide the original point cloud into ground point cloud and non-ground point cloud. Then, the DBSCAN clustering algorithm is used to cluster the non-ground point cloud.

[0081] Step R2, label correction: perform label correction based on clustering results. Specifically, if all point clouds in a cluster are marked with the same label in the clustering results, and in the prediction results, some point clouds belonging to the cluster are incorrectly labeled with other labels, then these mislabeled point clouds are uniformly corrected to a unified label. This correction strategy assumes that point clouds of the same type of objects have a high density in space and similar properties, so cluster labels can effectively serve as a basis for correcting predicted labels.

[0082] Finally, we can see Figure 6 The predicted point cloud labels of pedestrians that are misidentified are unified into the same category point cloud labels.

[0083] Table 1 Comparison results between PointNet and PointNet++ algorithms

[0084] method QUR Acc PointNet 84.04％ 91.28％ PointNet++ 87.28％ 93.17％ The method proposed in this paper 89.21％ 95.62％

[0085] The dynamic point cloud segmentation and prediction label correction design method is applicable to robots equipped with low-cost hybrid solid-state laser radar. The method effectively segments the dynamic point cloud by using the appearance and motion features in 2D BEV. The segmentation accuracy and dynamic point recognition ability are improved by integrating the spatial density clustering results. The effectiveness of the method is verified by experiments in a simulated channel environment.

[0086] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", "connect", and "fix" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood by specific circumstances.

[0087] Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various equivalent changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for dynamic point cloud segmentation and prediction label correction of an inspection robot, characterized in that , the method comprises: Step S1, laser radar input preprocessing: define point cloud data as p j =[x j ,y j ,z j ] T , where F i represents the point cloud data of the i-th frame; p j represents the jth point cloud point, which belongs to the point cloud data of the i-th frame; M represents the total number of point clouds in the i-th frame point cloud data; R 3 Represents three-dimensional space, each point of the point cloud data is in three-dimensional space; j =[x j ,y j ,z j ] T represents the three-dimensional coordinates of the jth point in the point cloud; x j ,y j 、z j They represent the coordinate values ​​of the point in the x, y, and z axes respectively. A straight-through filter is used to crop the point cloud and only extract the point cloud in the inspection channel area. For solid-state laser radars with a small field of view and non-repetitive scanning, a bird's-eye view projection method is used to project the 3D point cloud into a polar coordinate system to balance the uneven spatial distribution of the point cloud. Define F (u,v),i is the grid defined by the (u,v) coordinates in the polar coordinate bird's-eye view image th The set of all points within a cell; Step S2, dynamic object recognition module: when processing the current frame F i When , the data of the current frame and its previous and next frames are integrated to construct a sliding window for comprehensive analysis; Step S3, network architecture: Based on the dual-branch structure of PolarNet, combined with the appearance-motion co-attention mechanism, the maximum pooling operation is used to capture the distribution characteristics of the bird's-eye view projection point cloud within the vertical range of the grid unit, and the appearance-motion co-attention module is introduced to enhance the cross-modal interaction between appearance and motion features; Step S4, segmentation correction based on spatial clustering: A combination of density-based spatial clustering algorithm and cloth simulation filtering is used to identify misidentification and some dynamic point clouds are not correctly labeled. The density-based spatial clustering algorithm is used to identify and segment the high-density areas in the data set into multiple clusters, and identify isolated points as noise.

2. The method for dynamic point cloud segmentation and predicted label correction of an inspection robot according to claim 1, characterized in that: The F (u,v),i , the expression is: Among them, F (u,v),i Represents the polar coordinate top view image in the grid cell defined by the (u,v) coordinates, belonging to the point cloud frame F i The set of all points, p j Represents the point cloud frame F i A point in the point cloud, located at the polar coordinates (θ j ,p j )Down, This inequality means that the point p j The angle θ j The range of satisfaction, θ max and θ min are the maximum and minimum values ​​of the angle, respectively. h represents the height of the polar coordinate top view image, which divides the angle into h units. v represents the row index in the polar coordinate image. This condition constrains ρ j must be within the angle range of row v, This inequality means that the point p j The distance ρ j The range satisfied, ρ max and ρ min are the maximum and minimum values ​​of the distance, respectively. w represents the width of the polar coordinate top view image, which divides the distance into w units. u represents the column index in the polar coordinate image. This condition constrains p j Must be within the u-th column distance range.

3. The method for dynamic point cloud segmentation and predicted label correction of an inspection robot according to claim 1, characterized in that: The construction of the sliding window for comprehensive analysis includes: The laser odometry is used to determine the transformation matrix describing the spatial evolution between the current frame and the previous n frames, which is expressed as: in, Represents the overall transformation matrix from time in to time i, which transforms the point cloud data at time in to the coordinate system at time i. Represents the single-step transformation matrix from time in to time i-k+1. This matrix is ​​the pose transformation of the i-k+1th frame relative to the inth frame. It means that from k = 1 to k = n Multiply each term to get the overall transformation This represents a gradual accumulation of multiple frames; Use the transformation matrix to calculate the corresponding point cloud set: in, It means that after transformation, the point cloud frame F at time in i-n The new point cloud frame converted to the coordinate system of time i contains all the points in F i-n point in After transformation, it is located in the coordinate system of time i, Represents the overall transformation matrix from time in to time i. This matrix is ​​for each point p j Perform coordinate transformation and convert F i-n The point cloud is transformed from the coordinate system of time in to the coordinate system of time i, p j Represents the point cloud frame F i-n A point in j is the coordinate in the coordinate system at time in; The motion features required for identifying dynamic objects are extracted by examining the height differences in the bird's-eye view images. The point clouds extracted from two consecutive time windows W1 and W2 are used to generate motion features by evaluating the height variance of the corresponding grid cells in the two time periods. Compensate for self-motion by relative posture transformation, align W1 and W2 to the current local coordinate system, and define H Grid,a Each pixel value represents the height of the grid in Wa, and its expression is: H Grid,a =Max{Z Grid,a }-Min{Z Grid,a },a∈{1,2} Among them, H Grid,a Indicates the height difference of the cell grid Grid in the ath sliding window. This value is represented by the difference between the maximum height and the minimum height in the window. Grid,a Represents all height values ​​of the unit grid Grid in the a-th sliding window, that is, the height of each point in the grid area, Max{Z Grid,a } represents the maximum height of all points in the unit grid Grid in the a-th sliding window, Min{Z Grid,a } represents the minimum height of all points of the cell grid Grid in the a-th sliding window, WITH Grid,a ={z j ∈p j |p j ∈W Grid,a, } Among them, Z Grid,a Represents the set of height values ​​of the cell grid Grid in the a-th sliding window, z j Represents the Z coordinate of a specific point, that is, point p j The value on the Z axis. j represents a point in the grid a, W Grid,a, Represents the Grid area in the a-th sliding window; definition Represents the motion features in the nth channel of the i-th frame. The value of n is determined by the position of the frame in the time window. When a new frame arrives, both time windows move to the next position. The residual is obtained by subtracting H1 and H2: in, These are the motion features of each channel of the grid cell Grid at different time frames. represents the motion features of the 0th channel of the i-th frame, represents the motion features of the 1st channel of the i-1th frame, represents the motion feature of the second channel of the i-2th frame, H Grid,1 and H Grid,2 They represent the height difference of the grid cell Grid in the first sliding window and the height difference of the second sliding window on the grid cell Grid, that is, the difference between the maximum and minimum heights in each area, H Grid,1 -H Grid,2 This is the height difference between the first sliding window unit and the second sliding window. By calculating the height difference, we get the residual, which is the height change between the two sliding windows. in, These are the motion features of each channel of the grid cell Grid at different time frames. represents the motion features of the 3rd channel of the i-3th frame, represents the motion features of the 4th channel of the i-4th frame, represents the motion feature of the 5th channel of the i-5th frame, H Grid,1 and H Grid,2 They represent the height difference of the first sliding window on the grid unit Grid and the height difference of the second sliding window on the grid unit Grid, that is, the difference between the maximum and minimum heights in each area, H Grid,2 -H Grid,1 This is the height difference between the second sliding window unit and the first sliding window. By calculating the height difference, we get the residual, which is the height change between the two sliding windows.

4. The method for dynamic point cloud segmentation and predicted label correction of an inspection robot according to claim 1, characterized in that: The appearance-motion co-attention module consists of two parts: a co-attention gate for aligning the salient features of the two modalities, and a motion-guided attention module based on motion cues for refining the focus points according to the motion cues.

5. The method for dynamic point cloud segmentation and predicted label correction of an inspection robot according to claim 1, characterized in that: The density-based spatial clustering algorithm includes two parameters, the neighborhood radius ε and the minimum number of points minPts required to form a dense area. The ε neighborhood of the starting point is retrieved. If the number of points in the neighborhood exceeds minPts, the point is defined as the core point of the cluster and a new cluster is started; otherwise, the point is marked as noise; once a point is identified as a dense part of a cluster, the points in its ε neighborhood will also be included in the same cluster; If these points also meet the dense condition, continue to add the points in their ε neighborhood to the cluster; This process continues to iterate until all density-connected points in the cluster are identified. After the current cluster is fully determined, the algorithm reselects new unvisited points for cluster identification or noise marking until all data points are processed.

6. The method for dynamic point cloud segmentation and predicted label correction of an inspection robot according to claim 1, characterized in that: This method installs a hybrid solid-state 3D laser radar on the robot body to collect three-dimensional point cloud data, autonomously build maps and plan paths; an industrial computer is used to run the core algorithm to perform dynamic point cloud segmentation and correction of predicted labels.

Images