Laser radar point cloud segmentation clustering system based on multi-module cooperation

Abstract

The invention discloses a laser radar point cloud segmentation clustering system based on multi-module cooperation, and relates to the technical field of image processing, and the system comprises the steps: removing reflection noisy points and non-ground vertical points through a preprocessing module, so as to obtain a pure ground point cloud; performing pre-segmentation on the ground point cloud by using the annular elevation association graph, and optimizing low-confidence point classification through credibility propagation and jump convolution; identifying and removing an outlier in the obstacle point cloud by adopting a conditional value-at-risk; density is reduced through grid sampling and voxelization processing, fine-grained features are mined, and point-voxel features are fused to enhance global feature expression; and finally, improving the resolution by adopting a hierarchical progressive strategy, optimizing the feature information, realizing accurate segmentation and clustering of the obstacle, and outputting a fine clustering result with a bounding box. According to the invention, laser radar point cloud data is processed through multi-module cooperation, accurate segmentation and clustering of the ground and obstacles are realized, and the accuracy and efficiency of point cloud data processing are improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a laser radar point cloud segmentation and clustering system based on multi-module collaboration. Background Art

[0002] In the fields of autonomous driving and surveying and mapping, the processing of LiDAR point cloud data is crucial, but it faces many problems and interference. In terms of ground segmentation, existing algorithms are difficult to balance robustness and real-time issues. Especially in complex scenes, when the ground slope and flatness vary greatly, such as when there are objects such as grass, flower beds, and trees, the difficulty of ground segmentation increases dramatically. Improving processing efficiency in the processing of LiDAR point cloud data is also a key research direction.

[0003] Traditional methods based on distance images, high-order reasoning, and plane fitting have low computational requirements but poor accuracy. Methods based on deep learning have high accuracy but consume large amounts of computing resources and are difficult to meet real-time tasks. The cost of manually annotating data sets is also very high. Point cloud data has prominent noise and outlier problems. There are noise interferences such as reflection noise, vertical non-ground points, and various outliers in the data. How to effectively and accurately filter out these interfering point cloud data is also a key technical issue in point cloud data processing.

[0004] Traditional methods such as statistical outlier removal rely on manual design, which is not effective for some outliers such as backdoor attack trigger points, and do not consider the impact on high-level tasks such as classification. There are challenges in the wide applicability of outlier data filtering models. Point cloud classification also has limitations. Traditional methods based on single point methods or voxel methods are prone to losing key geometric features. Although methods that integrate multiple structural data can learn multi-level and multi-scale features, it is difficult to balance the differences between different data, which reduces classification accuracy. In addition, in 3D object detection, the mismatch between positioning and classification confidence affects detection accuracy. Traditional methods are insufficient in dealing with these problems of point cloud data. Summary of the invention

[0005] In order to solve the above problems, the present invention proposes a lidar point cloud segmentation and clustering system based on multi-module collaboration. The preprocessing module filters out reflection noise and non-ground vertical points to obtain a pure ground point cloud. Then, the ground point cloud is pre-segmented using the annular elevation association map and optimized for classification through credibility propagation and jump convolution. Finally, the fine-grained features of the obstacle point cloud and the coarse-grained feature alignment are optimized through feature alignment. This solves the problem of accurate segmentation and clustering of the ground and obstacles in the lidar point cloud data, and improves the accuracy and efficiency of data processing.

[0006] The specific plan is as follows:

[0007] The laser radar point cloud segmentation and clustering system based on multi-module collaboration includes:

[0008] Input module, receives the lidar point cloud and transmits it to the pre-processing module;

[0009] The preprocessing module includes a regional reflection noise removal submodule and a regional vertical non-ground point cloud removal submodule; the regional reflection noise removal submodule is used to filter out the reflection noise set that interferes with ground fitting in the laser radar point cloud, and obtain the laser radar point cloud with the reflection noise set filtered out; the regional vertical non-ground point cloud removal submodule is used to filter out the vertical non-ground point set that interferes with ground fitting in the laser radar point cloud, and obtain the laser radar point cloud with the vertical non-ground point set filtered out; the laser radar point cloud of the ground and obstacles is obtained based on the laser radar point cloud with the reflection noise set filtered out and the laser radar point cloud with the vertical non-ground point set filtered out;

[0010] The ground point cloud segmentation module sorts the lidar point clouds of the ground and obstacles, pre-segments the sorted lidar point clouds through the annular elevation association map, propagates the credibility of the pre-segmented lidar point clouds, generates low-confidence points, performs jump convolution on the low-confidence points and classifies them as ground point clouds or obstacle point clouds. The classified low-confidence points are reverse mapped to obtain the final ground lidar point cloud and the unaggregated obstacle lidar point cloud.

[0011] The outlier cleaning network module calculates the risk value of each point of the unaggregated obstacle part lidar point cloud, performs weighted summing and averaging of the risk values ​​of each point to obtain the conditional risk value, obtains the tail risk of the unaggregated obstacle part lidar point cloud based on the conditional risk value, regards the unaggregated obstacle part lidar point cloud whose tail risk exceeds the threshold as an outlier and removes it, and obtains the lidar point cloud without outliers;

[0012] The encoding module reduces the density of the LiDAR point cloud after outliers are removed by grid sampling, inputs the LiDAR point cloud after the density reduction into the point branch and mines the fine-grained features of the LiDAR point cloud in the point branch, voxelizes the LiDAR point cloud after the density reduction to obtain voxels, mines the coarse-grained voxel features of the LiDAR point cloud based on the voxels, and aligns the fine-grained features with the coarse-grained voxel features;

[0013] The point-voxel self-attention module performs tensor splicing on the aligned fine-grained features and coarse-grained voxel features to obtain spliced ​​features, fuses the spliced ​​features through the point-voxel self-attention fusion module, and inputs the fused features into the decoding module;

[0014] The decoding module decodes the fused features to obtain the laser radar point cloud decoding features with original resolution;

[0015] The output module performs feature enhancement on the decoded features of the lidar point cloud with original resolution, performs obstacle clustering and bounding box approximation on the feature-enhanced lidar point cloud, and then performs error minimization processing to obtain fine clustering and obstacle lidar point cloud with bounding box; outputs fine clustering and obstacle lidar point cloud with bounding box and the final ground lidar point cloud.

[0016] Furthermore, in the preprocessing module, the calculation formula for the reflection noise point set is as follows:

[0017] ;

[0018] in, Represents the set of reflection noise points to be filtered out; Represents the laser radar point cloud dataset extracted from the laser radar point cloud in the z-axis direction in space with a height less than the height threshold; Indicates from The reflection intensity is further extracted from the lidar point cloud dataset below the reflection threshold; Indicates that when judging the reflection noise set, The upper limit of the number of LiDAR point clouds with heights less than the height threshold in the axial LiDAR point cloud; Indicates the number of LiDAR point clouds whose height is lower than the height threshold in the LiDAR point cloud in the z-axis direction in the current space; Indicates the upper limit of the number of LiDAR point clouds whose reflection intensity is lower than the reflection threshold when judging the reflection noise point set; Indicates the number of lidar point clouds whose current reflection intensity is below the reflection threshold.

[0019] Furthermore, in the preprocessing module, the determination formula for filtering out the vertical non-ground point set that interferes with ground fitting in the lidar point cloud is as follows:

[0020] ;

[0021] ;

[0022] in, Represents the plane normal vector in Coordinate components on the axis; Represents the plane normal vector in Coordinate components on the axis; Represents the plane normal vector in Coordinate components on the axis; express Axis direction unit vector; Indicates the radian angle threshold; Represents the laser radar point cloud dataset extracted from the laser radar point cloud in the z-axis direction in space with a height greater than the height threshold; express The threshold for the number of lidar point clouds.

[0023] Furthermore, in the ground point cloud segmentation module, the lidar point clouds of the ground and obstacles are sorted, and the formula is as follows:

[0024] ;

[0025] in, represents a laser beam; Indicates laser beam Horizontal angular resolution; Indicates the serial number of the LiDAR point cloud arranged from top to bottom, that is, the LiDAR point cloud serial number; Indicates the horizontal arrangement sequence number of the LiDAR point cloud. and Composing LiDAR point cloud sequence ; Indicates laser beam With the laser radar point cloud serial number The corresponding connections between Indicates the total number of laser beams; Indicates the rotation angle of the LiDAR point cloud; Represents the total number of lidar point clouds obtained by a laser beam scanning 360°.

[0026] Furthermore, in the ground point cloud segmentation module, the sorted lidar point cloud is pre-segmented through the annular elevation association map, specifically including:

[0027] Establish a circular elevation association diagram, the formula is as follows:

[0028] ;

[0029] in, For LiDAR point cloud The sequence corresponds to On the plane Coordinates in the axis direction; For LiDAR point cloud The sequence corresponds to On the plane Coordinates in the axis direction; It indicates the number of rings cut into the circular elevation correlation map with the laser radar as the center on the plane; V indicates the number of equal parts of each ring; Indicates the first The radius of the ring; is the center of the laser radar; m represents the mth ring in the annular elevation association diagram; v represents the vth equally divided part of the mth ring in the annular elevation association diagram; represents the floor function;

[0030] To deal with the under-segmentation problem in pre-segmentation, the calculation formula is as follows:

[0031] ;

[0032] in, Express judgment Conditions of high authenticity; Indicates the ring spacing; Represents the height difference between LiDAR point clouds; Indicates The ring The lowest height value of all lidar point clouds in the part is used as the ground height. The calculation formula is as follows:

[0033] ;

[0034] in, is the maximum slope of the ground; take the height value , when The ring Some are higher than The points are pre-segmented as obstacle parts, otherwise they are ground parts.

[0035] Furthermore, in the ground point cloud segmentation module, the pre-segmented lidar point cloud is subjected to credibility propagation to generate low-confidence points, and the low-confidence points are subjected to jump convolution and classified as ground point clouds or obstacle point clouds, as follows:

[0036] The credibility of the pre-segmented lidar point cloud is propagated, and the calculation formula is as follows:

[0037] ;

[0038] in, It is the obstacle point cloud; For the LiDAR point cloud Plane coordinates in the axis direction; For the LiDAR point cloud Plane coordinates in the axis direction; For the LiDAR point cloud The offset in the axis direction; For the LiDAR point cloud The offset in the axis direction; Indicates the restricted range of values, that is, only when the coordinates on the xoy plane When the laser radar point cloud at is not 0, The lidar point cloud at the location will be included in the confidence propagation; after the credibility propagation is completed, the lidar point cloud that has not entered the credibility propagation is marked as a low confidence point;

[0039] Perform a skip convolution operation on low confidence points, and the calculation formula is as follows:

[0040] ;

[0041] in, is the ground lidar point cloud, is the serial number relative to the laser radar point cloud The offset of for relative to The offset of For one The convolution kernel, The coordinates inside the convolution kernel are The weight value of the lidar point cloud at is the convolution score of the obstacle part, Convolution score for the ground part; Indicates the serial number of the lidar point cloud arranged from top to bottom. Indicates the horizontal arrangement sequence of the LiDAR point cloud.

[0042] Furthermore, in the outlier cleaning network module, the risk value of each point of the unaggregated obstacle part of the lidar point cloud is calculated, and the formula is as follows:

[0043] ;

[0044] in, is the unaggregated obstacle part of the LiDAR point cloud. It represents the comprehensive point risk value; is the total number of feature layers for each point; For the LiDAR point cloud arrive Feature layer cumulative calculation; For the The LiDAR point cloud is Characteristics of the layer; is the output score function; is the point feature gradient operator; is the L2 norm; For the Frobenius norm normalization of layer feature gradients.

[0045] Furthermore, in the outlier cleaning network module, the risk values ​​of each point are weighted and averaged to obtain the conditional risk value, and the formula is as follows:

[0046] ;

[0047] ;

[0048] in, is the probability level; is a risk collection; is the indicator function; is the empirical distribution; and Risk Collection The elements in represent specific risk values; Indicated at the probability level Next, risk set Value at risk; Value at Risk The corresponding probability distribution; is the normalization coefficient; Indicated at the probability level Next, risk set The conditional value at risk.

[0049] Furthermore, in the encoding module, the fine-grained features and the coarse-grained voxel features are aligned, and the calculation formula is as follows:

[0050] ;

[0051] ;

[0052] in, It is a fine-grained feature; is the fine-grained feature after alignment; Indicates feature normalization processing; Represents the perceptron encoding process; Indicates feature flattening and expansion processing; is a coarse-grained voxel feature; is the coarse-grained voxel feature after alignment; To reduce the density of the LiDAR point cloud, Represents deconvolution processing.

[0053] Furthermore, in the encoding module, during the process of aligning the fine-grained features and the coarse-grained voxel features, minimization processing is performed to reduce the difference in feature distribution. The calculation formula is as follows:

[0054] ;

[0055] in, To minimize the feature distribution difference value, N is the number of obstacle lidar point clouds after reducing density; is the square of the L2 norm.

[0056] The present invention adopts the above technical solution and has the following beneficial effects:

[0057] (1) The present invention accurately filters out reflection noise points and vertical non-ground points through point cloud data preprocessing, providing a pure and accurate data basis for subsequent processing. At the same time, the outlier cleaning network can effectively filter outliers and improve the accuracy of clustering in the next stage;

[0058] (2) The present invention achieves a clear preliminary segmentation of the ground and obstacles and improves the accuracy through the annular elevation association map and jump convolution;

[0059] (3) The present invention reduces repeated calculations and avoids overfitting problems through efficient collaboration and targeted optimization of multiple modules. At the same time, it uses point-voxel self-attention to enhance global feature expression, improve the stability and generalization ability of the model, and achieve high efficiency and reliability of point cloud data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 A diagram of a laser radar point cloud segmentation and clustering system based on multi-module collaboration according to an embodiment of the present invention;

[0061] Figure 2 This is a diagram of a ground point cloud segmentation module according to an embodiment of the present invention;

[0062] Figure 3 This is a diagram of an outlier cleaning network module according to an embodiment of the present invention;

[0063] Figure 4 This is an output module diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0064] The present invention is further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0065] like Figure 1 As shown, the laser radar point cloud segmentation and clustering system based on multi-module collaboration of the present invention includes:

[0066] The input module 11 receives the laser radar point cloud and transmits it to the laser radar point cloud data preprocessing module.

[0067] Specifically, in this embodiment, the received laser radar point cloud is presented in the form of a three-dimensional point cloud in data format, which is composed of a large number of discrete points. These points follow the actual spatial distribution law of the target object and outline the general outline of the surrounding objects. , which includes the three-dimensional coordinates of space and reflection intensity information, extract it, Including ground point cloud and obstacle point cloud, the ground point cloud set is defined as , the obstacle point cloud set is , lidar point cloud collection Expressed as ,here The point cloud sequence is input into the regional reflection noise removal submodule and the regional vertical non-ground point cloud removal submodule, and then the reflection noise set and the vertical ground point cloud set can be filtered out.

[0068] The preprocessing module 12 includes a regional reflection noise removal submodule and a regional vertical non-ground point cloud removal submodule; the regional reflection noise removal submodule is used to filter out the reflection noise set that interferes with ground fitting in the lidar point cloud, and obtain the lidar point cloud with the reflection noise set filtered out; the regional vertical non-ground point cloud removal submodule is used to filter out the vertical non-ground point set that interferes with ground fitting in the lidar point cloud, and obtain the lidar point cloud with the vertical non-ground point set filtered out; based on the lidar point cloud with the reflection noise set filtered out and the lidar point cloud with the vertical non-ground point set filtered out, the lidar point cloud of the ground and obstacles is obtained.

[0069] Specifically, the calculation formula for the reflection noise point set is as follows:

[0070] ;

[0071] in, Represents the set of reflection noise points to be filtered out; Represents the laser radar point cloud dataset extracted from the laser radar point cloud in the z-axis direction in space with a height less than the height threshold; Indicates from The reflection intensity is further extracted from the lidar point cloud dataset below the reflection threshold; Indicates that when judging the reflection noise point set, The upper limit of the number of LiDAR point clouds with heights less than the height threshold in the axial LiDAR point cloud; Indicates the number of LiDAR point clouds whose height is lower than the height threshold in the LiDAR point cloud in the z-axis direction in the current space; Indicates the upper limit of the number of LiDAR point clouds whose reflection intensity is lower than the reflection threshold when judging the reflection noise point set; Indicates the number of lidar point clouds whose current reflection intensity is below the reflection threshold.

[0072] Specifically, the determination formula for filtering out the vertical non-ground point set that interferes with ground fitting in the lidar point cloud is as follows:

[0073] ;

[0074] ;

[0075] in, Represents the plane normal vector in Coordinate components on the axis; Represents the plane normal vector in Coordinate components on the axis; Represents the plane normal vector in Coordinate components on the axis; express Axis direction unit vector; Indicates the radian angle threshold; Represents the laser radar point cloud dataset extracted from the laser radar point cloud in the z-axis direction in space with a height greater than the height threshold; express The threshold for the number of lidar point clouds.

[0076] The ground point cloud segmentation module 13 sorts the lidar point clouds of the ground and obstacles, pre-segments the sorted lidar point clouds through a circular elevation association map, performs credibility propagation on the pre-segmented lidar point clouds to generate low-confidence points, performs jump convolution on the low-confidence points and classifies them as ground point clouds or obstacle point clouds, and reverse maps the classified low-confidence points to obtain the final ground part lidar point cloud and the unaggregated obstacle part lidar point cloud.

[0077] Specifically, the LiDAR point clouds of the ground and obstacles are sorted according to the following formula:

[0078] ;

[0079] in, represents a laser beam; Indicates laser beam Horizontal angular resolution; Indicates the serial number of the LiDAR point cloud arranged from top to bottom, that is, the LiDAR point cloud serial number; Indicates the horizontal arrangement sequence number of the LiDAR point cloud. and Composing LiDAR point cloud sequence ; Indicates laser beam With the laser radar point cloud serial number The corresponding connections between Indicates the total number of laser beams; Indicates the rotation angle of the LiDAR point cloud; Represents the total number of lidar point clouds obtained by a laser beam scanning 360°.

[0080] Specifically, the pre-segmentation of the sorted LiDAR point cloud by means of a ring elevation association map specifically includes:

[0081] Establish a circular elevation association diagram, the formula is as follows:

[0082] ;

[0083] in, For LiDAR point cloud The sequence corresponds to On the plane Coordinates in the axis direction; For LiDAR point cloud The sequence corresponds to On the plane Coordinates in the axis direction; It indicates the number of rings cut into the circular elevation correlation map with the laser radar as the center on the plane; V indicates the number of equal parts of each ring; Indicates the first The radius of the ring; is the center of the laser radar; m represents the mth ring in the annular elevation association diagram; v represents the vth equally divided part of the mth ring in the annular elevation association diagram; represents the floor function;

[0084] To deal with the under-segmentation problem in pre-segmentation, the calculation formula is as follows:

[0085] ;

[0086] in, Express judgment Conditions of high authenticity; Indicates the ring spacing; Represents the height difference between LiDAR point clouds; Indicates The ring The lowest height value of all lidar point clouds in the part is used as the ground height. The calculation formula is as follows:

[0087] ;

[0088] in, is the maximum slope of the ground; take the height value , when The ring Some are higher than The points are pre-segmented as obstacle parts, otherwise they are ground parts.

[0089] Specifically, the pre-segmented lidar point cloud is propagated with credibility to generate low-confidence points, and the low-confidence points are classified as ground point clouds or obstacle point clouds by skip convolution, as follows:

[0090] The credibility of the pre-segmented lidar point cloud is propagated, and the calculation formula is as follows:

[0091] ;

[0092] in, It is the obstacle point cloud; For the LiDAR point cloud Plane coordinates in the axis direction; For the LiDAR point cloud Plane coordinates in the axis direction; For the LiDAR point cloud The offset in the axis direction; For the LiDAR point cloud The offset in the axis direction; Indicates the restricted range of values, that is, only when the coordinates on the xoy plane When the laser radar point cloud at is not 0, The lidar point cloud at the location will be included in the confidence propagation; after the credibility propagation is completed, the lidar point cloud that has not entered the credibility propagation is marked as a low confidence point;

[0093] Perform a skip convolution operation on low confidence points, and the calculation formula is as follows:

[0094] ;

[0095] in, is the ground lidar point cloud, is the serial number relative to the laser radar point cloud The offset of for relative to The offset of For one The convolution kernel, The coordinates inside the convolution kernel are The weight value of the lidar point cloud at is the convolution score of the obstacle part, Convolution score for the ground part; Indicates the serial number of the lidar point cloud arranged from top to bottom. Indicates the horizontal arrangement sequence of the LiDAR point cloud.

[0096] Specifically, Figure 2As shown, in this embodiment, the segmentation is divided into two parts: coarse segmentation and fine segmentation. In the coarse segmentation, the original point cloud generates a pre-classification result through the annular elevation association map, and then generates a propagation image through RGB image mapping and credibility propagation; in the fine segmentation, the low confidence points are sequentially slid and convolved in multiple steps with a 5*5 convolution kernel through skip convolution to update the image, and the reclassification result is obtained through fusion update and reverse mapping.

[0097] The outlier cleaning network module 14 calculates the risk value of each point of the unaggregated obstacle part lidar point cloud, performs weighted summation and average of the risk values ​​of each point to obtain the conditional risk value, obtains the tail risk of the unaggregated obstacle part lidar point cloud based on the conditional risk value, regards the unaggregated obstacle part lidar point cloud whose tail risk exceeds the threshold as an outlier and removes it to obtain the lidar point cloud without outliers.

[0098] Specifically, the risk value of each point in the unaggregated obstacle part of the LiDAR point cloud is calculated using the following formula:

[0099] ;

[0100] in, is the unaggregated obstacle part of the LiDAR point cloud. It represents the comprehensive point risk value; is the total number of feature layers for each point; For the LiDAR point cloud arrive Feature layer cumulative calculation; For the The LiDAR point cloud is Characteristics of the layer; is the output score function; is the point feature gradient operator; is the L2 norm; For the Frobenius norm normalization of layer feature gradients.

[0101] Specifically, the risk values ​​at each point are weighted and averaged to obtain the conditional risk value. The formula is as follows:

[0102] ;

[0103] ;

[0104] in, is the probability level; is a risk collection; is the indicator function; is the empirical distribution; and Risk Collection The elements in represent specific risk values; Indicated at the probability level Next, risk set Value at risk; Value at Risk The corresponding probability distribution; is the normalization coefficient; Indicated at the probability level Next, risk set The conditional value at risk.

[0105] like Figure 3 As shown, in this embodiment, the outlier cleaning network module performs attribution analysis based on the classification model of the outlier samples to obtain point risks, then distinguishes the outlier samples from the standard samples and determines the long-tail boundary, obtains the risk minimization sample by minimizing the tail risk, and then combines the binary weights to perform outlier cleaning, and finally removes the outlier samples for classification tasks.

[0106] The encoding module 15 reduces the density of the LiDAR point cloud after outliers are removed by grid sampling, inputs the LiDAR point cloud after the density reduction into the point branch and mines the fine-grained features of the LiDAR point cloud in the point branch, voxelizes the LiDAR point cloud after the density reduction to obtain voxels, mines the coarse-grained voxel features of the LiDAR point cloud based on the voxels, and aligns the fine-grained features with the coarse-grained voxel features.

[0107] Specifically, the fine-grained features and the coarse-grained voxel features are aligned, and the calculation formula is as follows:

[0108] ;

[0109] ;

[0110] in, It is a fine-grained feature; is the fine-grained feature after alignment; Indicates feature normalization processing; Represents the perceptron encoding process; Indicates feature flattening and expansion processing; is a coarse-grained voxel feature; is the coarse-grained voxel feature after alignment; To reduce the density of the LiDAR point cloud, Represents deconvolution processing.

[0111] Specifically, the fine-grained features and the coarse-grained voxel features are aligned, and the calculation formula is as follows:

[0112] ;

[0113] ;

[0114] in, It is a fine-grained feature; is the fine-grained feature after alignment; Indicates feature normalization processing; Represents the perceptron encoding process; Indicates feature flattening and expansion processing; is a coarse-grained voxel feature; is the coarse-grained voxel feature after alignment; To reduce the density of the LiDAR point cloud, Represents deconvolution processing.

[0115] The point-voxel self-attention module 16 performs tensor splicing on the aligned fine-grained features and the coarse-grained voxel features to obtain spliced ​​features, fuses the spliced ​​features through the point-voxel self-attention fusion module, and inputs the fused features into the decoding module.

[0116] Specifically, firstly, the fine-grained features of the lidar point cloud after the alignment feature distribution and the coarse-grained voxel features of the lidar point cloud are tensor-joined, and the calculation formula is as follows:

[0117] ;

[0118] in, is the splicing feature, It is the tensor concatenation process.

[0119] The spliced ​​features are then fused by the point-voxel self-attention fusion module to obtain the fused features of the fine-grained features of the lidar point cloud and the coarse-grained voxel features of the lidar point cloud after the feature distribution is aligned, and the global expression ability is enhanced. The calculation formula is as follows:

[0120] ;

[0121] ;

[0122] ;

[0123] in, Indicates query, Indicates the key, represents the value matrix, is the query weight matrix, is the key weight matrix, is the value weight matrix, is the query and key matrix dimension, is the attention weight matrix, For fusion features.

[0124] The decoding module 17 decodes the fused features to obtain the laser radar point cloud decoding features with the original resolution.

[0125] Specifically, in the decoding module, the fusion features are distributed at the point branch by aligning the feature distribution, and the feature distribution difference is minimized during the alignment of the feature distribution to obtain the point branch decoding features of the aligned feature distribution and the voxel branch decoding features of the aligned feature distribution. The point branch decoding features of the aligned feature distribution and the voxel branch decoding features of the aligned feature distribution are further tensor-concatenated, and the calculation formula is as follows:

[0126] ;

[0127] in, Decode features for point branches that align feature distributions, Align the voxel-wise branch decoding features of the feature distribution, Decode features for lidar point cloud data at native resolution.

[0128] The output module 18 performs feature enhancement on the decoded features of the lidar point cloud with the original resolution, performs obstacle clustering and bounding box approximation on the feature-enhanced lidar point cloud, and then performs error minimization processing to obtain a fine clustering and obstacle lidar point cloud with a bounding box; and outputs the fine clustering and obstacle lidar point cloud with a bounding box and the final ground lidar point cloud.

[0129] Specifically, through the output module, the decoded features of the lidar point cloud data with original resolution are transmitted to the output front layer network through two fully connected layers and a random inactivation layer to prevent overfitting and enhance clustering stability, thereby obtaining the enhanced features of the lidar point cloud data; the enhanced features of the lidar point cloud data are further transmitted to the point intersection and union ratio perception refinement network, and the predicted obstacle clustering and bounding box are obtained through point intersection and union ratio calculation. The calculation formula is as follows:

[0130] ;

[0131] in, is the point intersection and comparison score, is the segmentation score for enhancing feature region division of LiDAR point cloud data, is the total number of divisions of the region, is the area currently being calculated, It is the slight deviation between the true point intersection and union and the calculated approximation of the true bounding box, that is, the approximate calculation compensation value.

[0132] The predicted obstacle clustering and bounding box are processed to minimize the error. The formula for minimizing the error is as follows:

[0133] ;

[0134] in, is the total loss of point intersection and union, that is, the deviation between the predicted bounding box and the true bounding box, is the cross entropy loss for segmentation, For smooth function, For smooth Deviation loss, is the loss weight, is the deviation between the predicted point intersection and the true bounding box.

[0135] By continuously reducing The value of , obtains the fine clustering and obstacle lidar point cloud part with bounding box.

[0136] like Figure 4 As shown, in this embodiment, the output module includes a module of an output front-layer network and a point intersection-over-unit-perception refinement network. The output front-layer network consists of a random dropout layer, a fully connected layer, etc., and is used for classification and bounding box regression; the point intersection-over-unit-perception refinement network includes bounding box optimization, bounding box scoring, point intersection-over-unit-perception deviation, etc., and performs grid point segmentation based on the box features of the grid, and is connected to the output front-layer network through approximate 3D IoU, and finally outputs the segmentation and clustering results.

[0137] Although the present invention has been specifically shown and described in conjunction with the preferred embodiments, it should be understood by those skilled in the art that various changes may be made to the present invention in form and details without departing from the spirit and scope of the present invention as defined by the appended claims, all of which are within the scope of protection of the present invention.

Claims

1. A laser radar point cloud segmentation and clustering system based on multi-module collaboration, characterized by: include: Input module, receives the lidar point cloud and transmits it to the pre-processing module; The preprocessing module includes a regional reflection noise removal submodule and a regional vertical non-ground point cloud removal submodule; the regional reflection noise removal submodule is used to filter out the reflection noise set that interferes with ground fitting in the laser radar point cloud, and obtain the laser radar point cloud with the reflection noise set filtered out; the regional vertical non-ground point cloud removal submodule is used to filter out the vertical non-ground point set that interferes with ground fitting in the laser radar point cloud, and obtain the laser radar point cloud with the vertical non-ground point set filtered out; the laser radar point cloud of the ground and obstacles is obtained based on the laser radar point cloud with the reflection noise set filtered out and the laser radar point cloud with the vertical non-ground point set filtered out; The ground point cloud segmentation module sorts the lidar point clouds of the ground and obstacles, pre-segments the sorted lidar point clouds through the annular elevation association map, propagates the credibility of the pre-segmented lidar point clouds, generates low-confidence points, performs jump convolution on the low-confidence points and classifies them as ground point clouds or obstacle point clouds. The classified low-confidence points are reverse mapped to obtain the final ground lidar point cloud and the unaggregated obstacle lidar point cloud. The outlier cleaning network module calculates the risk value of each point of the unaggregated obstacle part lidar point cloud, performs weighted summing and averaging of the risk values ​​of each point to obtain the conditional risk value, obtains the tail risk of the unaggregated obstacle part lidar point cloud based on the conditional risk value, regards the unaggregated obstacle part lidar point cloud whose tail risk exceeds the threshold as an outlier and removes it, and obtains the lidar point cloud without outliers; The encoding module reduces the density of the LiDAR point cloud after outliers are removed by grid sampling, inputs the LiDAR point cloud after the density reduction into the point branch and mines the fine-grained features of the LiDAR point cloud in the point branch, voxelizes the LiDAR point cloud after the density reduction to obtain voxels, mines the coarse-grained voxel features of the LiDAR point cloud based on the voxels, and aligns the fine-grained features with the coarse-grained voxel features; The point-voxel self-attention module performs tensor splicing on the aligned fine-grained features and coarse-grained voxel features to obtain spliced ​​features, fuses the spliced ​​features through the point-voxel self-attention fusion module, and inputs the fused features into the decoding module; The decoding module decodes the fused features to obtain the laser radar point cloud decoding features with original resolution; The output module performs feature enhancement on the decoded features of the lidar point cloud with original resolution, performs obstacle clustering and bounding box approximation on the feature-enhanced lidar point cloud, and then performs error minimization processing to obtain fine clustering and obstacle lidar point cloud with bounding box; outputs fine clustering and obstacle lidar point cloud with bounding box and the final ground lidar point cloud.

2. The laser radar point cloud segmentation and clustering system based on multi-module collaboration according to claim 1 is characterized in that: In the preprocessing module, the calculation formula for the reflection noise set is as follows: ； in, Represents the set of reflection noise points to be filtered out; Represents the laser radar point cloud dataset extracted from the laser radar point cloud in the z-axis direction in space with a height less than the height threshold; Indicates from The reflection intensity is further extracted from the lidar point cloud dataset below the reflection threshold; Indicates that when judging the reflection noise point set, The upper limit of the number of LiDAR point clouds with heights less than the height threshold in the axial LiDAR point cloud; Indicates the number of LiDAR point clouds whose height is lower than the height threshold in the LiDAR point cloud in the z-axis direction in the current space; Indicates the upper limit of the number of LiDAR point clouds whose reflection intensity is lower than the reflection threshold when judging the reflection noise point set; Indicates the number of lidar point clouds whose current reflection intensity is below the reflection threshold.

3. The laser radar point cloud segmentation and clustering system based on multi-module collaboration according to claim 2 is characterized in that: In the preprocessing module, the determination formula for filtering out the vertical non-ground point set that interferes with ground fitting in the lidar point cloud is as follows: ； ； in, Represents the plane normal vector in Coordinate components on the axis; Represents the plane normal vector in Coordinate components on the axis; Represents the plane normal vector in Coordinate components on the axis; express Axis direction unit vector; Indicates the radian angle threshold; Represents the laser radar point cloud dataset extracted from the laser radar point cloud in the z-axis direction in space with a height greater than the height threshold; express The threshold for the number of lidar point clouds.

4. The laser radar point cloud segmentation and clustering system based on multi-module collaboration according to claim 1, characterized in that: In the ground point cloud segmentation module, the lidar point clouds of the ground and obstacles are sorted. The formula is as follows: ； in, represents a laser beam; Indicates laser beam Horizontal angular resolution; Indicates the serial number of the LiDAR point cloud arranged from top to bottom, that is, the LiDAR point cloud serial number; Indicates the horizontal arrangement sequence number of the LiDAR point cloud. and Composing LiDAR point cloud sequence ; Indicates laser beam With the laser radar point cloud serial number The corresponding connections between Indicates the total number of laser beams; Indicates the rotation angle of the LiDAR point cloud; Represents the total number of lidar point clouds obtained by a laser beam scanning 360°.

5. The laser radar point cloud segmentation and clustering system based on multi-module collaboration according to claim 4 is characterized in that: In the ground point cloud segmentation module, the sorted lidar point cloud is pre-segmented through a ring elevation association map, specifically including: Establish a circular elevation association diagram, the formula is as follows: ； in, For LiDAR point cloud The sequence corresponds to On the plane Coordinates in the axis direction; For LiDAR point cloud The sequence corresponds to On the plane Coordinates in the axis direction; It indicates the number of rings cut into the circular elevation correlation map with the laser radar as the center on the plane; V indicates the number of equal parts of each ring; Indicates the first The radius of the ring; is the center of the laser radar; m represents the mth ring in the annular elevation association diagram; v represents the vth equally divided part of the mth ring in the annular elevation association diagram; represents the floor function; To deal with the under-segmentation problem in pre-segmentation, the calculation formula is as follows: ； in, Express judgment Conditions of high authenticity; Indicates the ring spacing; Represents the height difference between LiDAR point clouds; Indicates The ring The lowest height value of all lidar point clouds in the part is used as the ground height. The calculation formula is as follows: ； in, is the maximum slope of the ground; take the height value , when The ring Some are higher than The points are pre-segmented as obstacle parts, otherwise they are ground parts.

6. The laser radar point cloud segmentation and clustering system based on multi-module collaboration according to claim 5, characterized in that: In the ground point cloud segmentation module, the pre-segmented lidar point cloud is propagated for credibility to generate low-confidence points, and the low-confidence points are skipped and convolved to be classified as ground point clouds or obstacle point clouds, as follows: The credibility of the pre-segmented lidar point cloud is propagated, and the calculation formula is as follows: ； in, It is the obstacle point cloud; For the LiDAR point cloud Plane coordinates in the axis direction; For the LiDAR point cloud Plane coordinates in the axis direction; For the LiDAR point cloud The offset in the axis direction; For the LiDAR point cloud The offset in the axis direction; Indicates the restricted range of values, that is, only when the coordinates on the xoy plane When the laser radar point cloud at is not 0, The lidar point cloud at the location will be included in the confidence propagation; after the credibility propagation is completed, the lidar point cloud that has not entered the credibility propagation is marked as a low confidence point; Perform a skip convolution operation on low confidence points, and the calculation formula is as follows: ； in, is the ground lidar point cloud, is the serial number relative to the laser radar point cloud The offset of for relative to The offset of For one The convolution kernel, The coordinates inside the convolution kernel are The weight value of the lidar point cloud at is the convolution score of the obstacle part, Convolution score for the ground part; Indicates the serial number of the lidar point cloud arranged from top to bottom. Indicates the horizontal arrangement sequence of the LiDAR point cloud.

7. The laser radar point cloud segmentation and clustering system based on multi-module collaboration according to claim 1, characterized in that: In the outlier cleaning network module, the risk value of each point in the unaggregated obstacle part of the lidar point cloud is calculated. The formula is as follows: ； in, is the unaggregated obstacle part of the LiDAR point cloud. It represents the comprehensive point risk value; is the total number of feature layers for each point; For the LiDAR point cloud arrive Feature layer cumulative calculation; For the The LiDAR point cloud is Characteristics of the layer; is the output score function; is the point feature gradient operator; is the L2 norm; For the Frobenius norm normalization of layer feature gradients.

8. The laser radar point cloud segmentation and clustering system based on multi-module collaboration according to claim 1, characterized in that: In the outlier cleaning network module, the risk values ​​of each point are weighted and averaged to obtain the conditional risk value. The formula is as follows: ； ； in, is the probability level; is a risk collection; is the indicator function; is the empirical distribution; and Risk Collection The elements in represent specific risk values; Indicated at the probability level Next, risk set Value at risk; Value at Risk The corresponding probability distribution; is the normalization coefficient; Indicated at the probability level Next, risk set The conditional value at risk.

9. The laser radar point cloud segmentation and clustering system based on multi-module collaboration according to claim 1, characterized in that: In the encoding module, the fine-grained features and the coarse-grained voxel features are aligned, and the calculation formula is as follows: ； ； in, It is a fine-grained feature; is the fine-grained feature after alignment; Indicates feature normalization processing; Represents the perceptron encoding process; Indicates feature flattening and expansion processing; is a coarse-grained voxel feature; is the coarse-grained voxel feature after alignment; To reduce the density of the LiDAR point cloud, Represents deconvolution processing.

10. The laser radar point cloud segmentation and clustering system based on multi-module collaboration according to claim 9, characterized in that: In the encoding module, during the alignment of fine-grained features and coarse-grained voxel features, minimization processing is performed to reduce the difference in feature distribution. The calculation formula is as follows: ； in, To minimize the feature distribution difference value, N is the number of obstacle lidar point clouds after reducing density; is the square of the L2 norm.

Images