A Point Cloud Panoramic Segmentation Method Based on Sparse Instance Proposal and Aggregation

Abstract

This invention relates to the field of 3D vision technology and proposes a point cloud panoramic segmentation method based on sparse instance proposal and aggregation. An efficient, accurate, and non-learning-based sparse instance proposal module is designed based on a "sampling-movement-clustering" scheme, eliminating the dependency on offset branches and accelerating the clustering process. It can be easily extended to other backbone networks and datasets and used in a plug-and-play manner. Furthermore, an instance aggregation module based on KNN-Transformer is designed to aggregate potentially fragmented instances generated by the instance proposal module, improving its performance on large targets and outputting more accurate and complete instance segmentation results. The proposed method (PANet) can accurately perform panoramic segmentation of input point clouds and exhibits good robustness to problems such as target occlusion and rapid movement in real-world 3D environments.

Description

Technical Field

[0001] This invention relates to the field of 3D vision technology, specifically to a point cloud panoramic segmentation method based on sparse instance proposal and aggregation. Background Technology

[0002] LiDAR panoptic segmentation (LPS) plays a crucial role in 3D scene understanding using point clouds, a fundamental task in many robotics applications such as autonomous driving. LPS combines semantic segmentation and instance segmentation within a single framework, providing semantic labels for points in a scene and instance IDs for points belonging to instances (objects). With the emergence of large-scale point cloud benchmarks such as SemanticKITTI and nuScenes, LPS has made rapid progress. However, reliable panoptic segmentation remains extremely challenging due to the sparse, unordered, and non-uniform sampling characteristics of point clouds.

[0003] Existing LPS methods can be mainly divided into detection-based methods and clustering-based methods. The former utilizes 3D object detection networks to discover instances, but is usually limited by detection accuracy. The latter achieves instance segmentation through center regression and clustering algorithms. For clustering-based methods, clustering performance is easily affected by the distribution of regression center points. We found that most existing methods heavily rely on learnable offset branches to provide geometric offsets for center regression. However, due to the sparsity, non-uniform density, and various shapes / sizes of LiDAR point clouds, it is difficult to predict ideal geometric offsets. Recently, DSNet designed a learnable dynamic offset (DS) module to iteratively move regression center points further to cluster center points, but the moved center points may not match the true instance centers, and this inconsistency may reduce the learning performance of the DSNet module. Summary of the Invention

[0004] To address the aforementioned issues, this invention proposes a non-learning Sparse Instance Proposal (SIP) module that can efficiently cluster instances directly from the original points of objects. The SIP module is designed using a "sampling-movement-clustering" scheme. Specifically, to avoid the significant computational burden and memory overhead caused by moving and clustering all object points, this invention introduces a Balanced Point Sampling (BPS) strategy. The proposed BPS generates sparse seed points that are more evenly distributed within a distance range, simultaneously achieving point sampling and allocation, thus improving efficiency. Furthermore, this invention designs a simple yet effective movement module, "Bubble Shrinking" (BS), which iteratively and efficiently moves seed points to cluster centers. Finally, this invention clusters the moved points into instances using a Connected Component Labeling (CCL) algorithm, which can be implemented through an efficient depth-first search. Due to the cascaded design of BPS, BS, and CCL-based grouping, the non-learning SIP module of this invention is effective and efficient, and can be easily extended to other backbone networks and datasets in a plug-and-play manner.

[0005] However, due to the sparsity of LiDAR point clouds, clustering algorithms (including the SIP module of this invention) may generate fragmented / over-segmented instances when grouping large objects such as trucks and buses. To improve the integrity of instance segmentation, this invention proposes an instance aggregation (IA) module to further integrate potentially fragmented instances. Specifically, this invention applies a KNN-Transformer to enhance the interaction between instance proposals and merges instance proposals belonging to the same instance ID based on instance similarity. The IA module of this invention can complement the SIP module to further improve the segmentation performance of large objects.

[0006] To achieve the above objectives, this invention provides a point cloud panoramic segmentation method based on sparse instance proposal and aggregation, comprising the following steps:

[0007] S1. Construct a sparse instance proposal module based on the "sampling-movement-clustering" method and output instance proposals;

[0008] S2. Based on KNN-Transformer, an instance aggregation module is constructed to aggregate the potentially fragmented instances generated by the sparse instance proposal module, and output accurate and complete instance segmentation results.

[0009] S3. Utilize the server to optimize network parameters by reducing the network loss function until the network converges, thereby obtaining a point cloud panorama segmentation method based on sparse instance proposal and aggregation.

[0010] S4. Perform panoramic segmentation of the new point cloud using the point cloud panoramic segmentation method based on sparse instance proposal and aggregation.

[0011] Preferably, step S1 specifically includes the following steps:

[0012] S11. Construct an equilibrium point sampling strategy to perform sparse sampling on the foreground points and obtain sparse seed points. To reduce computational burden and memory consumption;

[0013] S12. Construct a "bubble shrinkage" moving module to move the sparse seed points. Iteratively move towards the instance center, causing seed points belonging to different instances to separate and seed points belonging to the same instance to aggregate;

[0014] S13. Cluster the moved sparse seed points using a depth-first search-based connected component labeling algorithm, and assign other unsampled points to the clusters to which the corresponding seed points belong, thus obtaining the final instance proposal point clusters. N is the number of point clusters. For the first The set of points in a point cluster.

[0015] Preferably, the equilibrium point sampling strategy specifically involves: dividing the input point cloud into voxel blocks using voxelization, and then averaging the points within the same voxel as the seed point corresponding to all points in that voxel; this seed point governs all points within its voxel, including semantic category and instance ID; the voxelization operation specifically involves:

[0016] Given a point cloud Hit the mark Its voxel index ,in For voxel resolution, For floor operations; all points with the same voxel index are in the same voxel block;

[0017] The "bubble shrinkage" movement module specifically involves: first, empirically determining the minimum radius for each type of instance. Construct a connected graph using all seed points as vertices. If two vertices belong to the same category... And the distance between them is less than the minimum radius. If they are connected, then each seed point is assigned a bubble containing all points connected to it. The bubble shrinks iteratively based on the points within it, achieving the effect of clustering seed points belonging to the same instance and separating seed points belonging to different instances. The shrinking process is as follows:

[0018]

[0019]

[0020]

[0021]

[0022]

[0023] in, The connection matrix for constructing the connection graph. This is the number of iterations, with a default value of 4.

[0024] The connected component labeling algorithm specifically involves: establishing a connected graph using the translated seed point as the vertex; connecting two vertices if they belong to the same category and the distance between them is less than the minimum radius corresponding to that category; and then using the connected component labeling algorithm to identify each cluster. The minimum radius in the connected component labeling is set to the value in the "bubble shrinking" moving module. Half of it.

[0025] Preferably, step S2 specifically includes the following steps:

[0026] S21. Based on the instance proposal point clusters output by the sparse instance proposal module. Combine point cloud features to extract corresponding instance features. ;

[0027] S22. Use KNN-Transformer to process the instance features. Model the dependencies between them and output enhanced instance features. ;

[0028] S23. Using a lightweight multilayer perceptron, calculate the similarity between pairs of instances based on their features and center point positions. If the similarity is greater than a certain threshold, merge the two instances.

[0029] Preferably, the specific process of step S21 is as follows:

[0030] Where MLP stands for Multilayer Perceptron, and MaxPool is the max pooling function. These are the features and coordinate sets of the point cluster, respectively; the feature dimension of the extracted instance features is 64.

[0031] Preferably, the specific process of step S22 is as follows:

[0032] Based on the spatial location of the center point, k-nearest neighbors (KNN) are used to select clusters of points closest to the instance. The nearest k-point clusters Then, through a linear layer, the point clusters are... Corresponding instance features Mapped to query feature map Cluster k points Corresponding instance features Mapped to key-value feature pairs Then, self-attention is calculated to obtain the enhanced instance features. :

[0033] Where C is 64.

[0034] Preferably, the similarity calculation process in step S23 is as follows:

[0035]

[0036] in, For point clusters The coordinates of the center point are the average values ​​of the points in the point cluster set.

[0037] Preferably, step S3 specifically includes the following steps:

[0038] S31. The server executes the training point cloud generation unit to perform random coordinate flipping, random point cloud rotation, and random point cloud scaling on the input original point cloud to generate an enhanced training point cloud input.

[0039] S32. Utilize the server to execute the semantic backbone network, generating semantic prediction results and point cloud features of the input point cloud. Supervise the semantic backbone network, with the supervision loss being the sum of cross-entropy loss and Lovasz loss:

[0040]

[0041] in, For predicted values, The actual value; A Lovasz extension for IoU, Let C be the error vector for category c, where C is the total number of semantic categories;

[0042] S33. Utilize the sparse instance proposal module described in step S1 on the server, input the foreground points, and generate instance proposal point clusters. ;

[0043] S34. Utilize the instance aggregation module described in step S2 via the server to calculate the similarity between the instances. The supervision is performed using the binary cross-entropy loss:

[0044] Among them, if point cluster If they belong to the same instance It is 1 if it is true, otherwise it is 0;

[0045] S35. Optimize the objective function using a server. and To obtain locally optimal network parameters.

[0046] Preferably, the semantic backbone network in step S32 consists of a 3D sparse convolutional network and a 2D U-Net; the 3D sparse convolutional network consists of four feature extraction blocks to extract multi-scale features; the 2D U-Net consists of a 2D convolutional network, which first performs two progressive downsampling steps at a scale of 2, and then upsampling to the original resolution and projecting it onto point features; the multi-scale features extracted by the 3D sparse convolutional network and the back-projected point features extracted by the 2D U-Net are concatenated and fed into a fully connected layer for feature compression to obtain point cloud features; the semantic backbone network is followed by a two-layer fully connected layer to predict semantic segmentation.

[0047] Preferably, step S4 specifically includes the following steps:

[0048] S41. Obtain radar point cloud in 3D environment;

[0049] S42. Input the radar point cloud into the semantic backbone network to obtain semantic prediction results and point cloud features;

[0050] S43. Obtain the foreground points based on the semantic prediction results, and input the foreground points into the sparse instance proposal module to obtain instance proposal point clusters;

[0051] S44. Input the instance proposed point clusters and point cloud features into the instance aggregation module to obtain aggregated instance point clusters;

[0052] S45. Based on semantic prediction, assign a semantic category to each aggregated instance point cluster through the maximum vote, and then fuse it with the semantic segmentation result to obtain the final panoramic segmentation result.

[0053] Compared with the prior art, the beneficial effects of the present invention are:

[0054] This invention provides a point cloud panoramic segmentation method based on sparse instance proposal and aggregation. It designs an efficient, accurate, and non-learning-based sparse instance proposal module based on a "sampling-movement-clustering" scheme, eliminating dependence on offset branches and accelerating the clustering process. This method can be easily extended to other backbone networks and datasets and used in a plug-and-play manner. Furthermore, an instance aggregation module based on KNN-Transformer is designed to aggregate potentially fragmented instances generated by the instance proposal module, improving its performance on large targets and outputting more accurate and complete instance segmentation results. The proposed method (PANet) can accurately perform panoramic segmentation of input point clouds and exhibits good robustness to target occlusion and rapid movement in real-world 3D environments, achieving state-of-the-art performance on large-scale datasets SemanticKITTI and nuScenes. Attached Figure Description

[0055] Figure 1 The overall algorithm framework diagram of a point cloud panoramic segmentation method based on sparse instance proposal and aggregation provided by the present invention is shown below.

[0056] Figure 2 A framework diagram of the three steps of the sparse proposal module provided by the present invention;

[0057] Figure 3 A visual comparison of the proposed algorithm with other state-of-the-art methods (DSNet, panoptic-polarnet). Detailed Implementation

[0058] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] To address the problems and shortcomings of existing technologies, this invention proposes a point cloud panoramic segmentation method based on sparse instance proposal and aggregation, which mainly includes four stages: designing a sparse instance proposal module, designing an instance aggregation module, model training, and model inference.

[0060] This invention proposes a point cloud panoramic segmentation method based on sparse instance proposal and aggregation, comprising the following steps:

[0061] S1. Construct an efficient, accurate, and non-learning-based sparse instance proposal module based on the "sampling-movement-clustering" approach, and output instance proposals;

[0062] S2. Based on KNN-Transformer, an instance aggregation module is constructed to aggregate the potentially fragmented instances generated by the sparse instance proposal module, thereby improving its performance on large targets and outputting accurate and complete instance segmentation results.

[0063] S3. Utilize the server to optimize network parameters by reducing the network loss function until the network converges, thereby obtaining a point cloud panorama segmentation method based on sparse instance proposal and aggregation.

[0064] S4. Perform panoramic segmentation of the new point cloud using the point cloud panoramic segmentation method based on sparse instance proposal and aggregation.

[0065] The following is a detailed explanation of each step.

[0066] Specifically, step S1 involves constructing an efficient, accurate, and non-learning-based sparse instance proposal module based on a "sampling-moving-clustering" approach, and outputting instance proposals. For example... Figure 2 (b), (c), and (d) show the framework diagram of the three steps of the sparse proposal module provided by the present invention. The specific steps are as follows:

[0067] S11. Construct an equilibrium point sampling strategy to perform sparse sampling on the foreground points and obtain sparse seed points. To reduce computational burden and memory consumption, the input point cloud is divided into voxel blocks using a voxelization operation. Then, the average value of points within the same voxel is used as the seed point for all points within that voxel. This seed point governs all points within its voxel, including semantic category and instance ID. The voxelization operation specifically involves:

[0068] Given a point cloud Hit the mark Its voxel index ,in For voxel resolution, This is a floor operation; all points with the same voxel index are in the same voxel block.

[0069] S12. Construct a "bubble shrinkage" moving module to move the sparse seed points. The process iteratively moves towards the instance center, separating seed points belonging to different instances while aggregating seed points belonging to the same instance; specifically, the minimum radius of each instance category is first empirically determined. Construct a connected graph using all seed points as vertices. If two vertices belong to the same category... And the distance between them is less than the minimum radius. If they are connected, then each seed point is assigned a bubble containing all points connected to it. Similar to other moving modules, the bubble iteratively shrinks based on the points within it, achieving the effect of clustering seed points belonging to the same instance while separating seed points belonging to different instances. The shrinking process is shown below:

[0070]

[0071]

[0072]

[0073]

[0074]

[0075] in, The connection matrix for constructing the connection graph. This represents the number of iterations, which defaults to 4.

[0076] S13. Cluster the moved sparse seed points using a depth-first search-based connected component labeling algorithm, and assign other unsampled points to the clusters to which the corresponding seed points belong, thus obtaining the final instance proposal point clusters. N is the number of point clusters. For the first The set of points in a cluster is specifically constructed by establishing a connected graph with the translated seed point as the vertex. If two vertices belong to the same category and the distance between them is less than the minimum radius corresponding to that category, they are connected. Then, the connected component labeling algorithm is used to find each cluster. The minimum radius in the connected component labeling is set to the minimum radius of the "bubble shrinking" moving module. Half of it.

[0077] Step S2: Based on Transformer, an instance aggregation module is constructed to aggregate the potentially fragmented instances generated by the sparse instance proposal module, improving its performance on large targets and outputting accurate and complete instance segmentation results; the specific steps are as follows:

[0078] S21. Based on the instance proposal point clusters output by the sparse instance proposal module. Combine point cloud features to extract corresponding instance features. Instance feature extraction specifically involves:

[0079] Where MLP stands for Multilayer Perceptron, and MaxPool is the max pooling function. These are the features and coordinate sets of the point cluster, respectively; the feature dimension of the extracted instance features is 64.

[0080] S22. Use KNN-Transformer to process the instance features. Model the dependencies between them and output enhanced instance features. The specific process of KNN-Transformer is as follows:

[0081] Based on the spatial location of the center point, k-nearest neighbors (KNN) are used to select clusters of points closest to the instance. The nearest k-point clusters Then, through a linear layer, the point clusters are... Corresponding instance features Mapped to query feature map Cluster k points Corresponding instance features Mapped to key-value feature pairs Then, self-attention is calculated to obtain the enhanced instance features. :

[0082] Where C is 64.

[0083] S23. Using a lightweight multilayer perceptron, calculate the similarity between pairs of instances based on their features and center point positions. If the similarity exceeds a certain threshold, merge the two instances. The specific similarity calculation process is as follows:

[0084]

[0085] in, For point clusters The coordinates of the center point are the average values ​​of the points in the point cluster set.

[0086] Step S3: Using a server, optimize network parameters by reducing the network loss function until the network converges, obtaining a point cloud panorama segmentation method based on sparse instance proposal and aggregation. For example... Figure 1 The diagram shows the overall algorithm framework of a point cloud panoramic segmentation method based on sparse instance proposal and aggregation provided by this invention. The specific steps are as follows:

[0087] S31. The server executes the training point cloud generation unit to perform data augmentation such as random coordinate flipping, random point cloud rotation, and random point cloud scaling on the input original point cloud, and generates an enhanced training point cloud input.

[0088] S32. Utilize the server to execute the semantic backbone network, generating semantic prediction results and point cloud features of the input point cloud. Supervise the semantic backbone network, with the supervision loss being the sum of cross-entropy loss and Lovasz loss:

[0089]

[0090] in, For predicted values, The actual value; A Lovasz extension for IoU, Let C be the error vector for category c, and C be the total number of semantic categories. The semantic backbone network consists of a 3D sparse convolutional network and a 2D U-Net. The 3D sparse convolutional network consists of four feature extraction blocks that extract multi-scale features. The 2D U-Net consists of a 2D convolutional network that first performs two progressive downsampling steps at a scale of 2, then upsampling to the original resolution and projecting it onto point features. The multi-scale features extracted by the 3D sparse convolutional network and the back-projected point features extracted by the 2D U-Net are concatenated and fed into a fully connected layer for feature compression to obtain point cloud features. The semantic backbone network is followed by a two-layer fully connected layer to predict semantic segmentation.

[0091] S33. Utilize the sparse instance proposal module described in step S1 on the server, input the foreground points, and generate instance proposal point clusters. ;

[0092] S34. Utilize the instance aggregation module described in step S2 via the server to calculate the similarity between the instances. The supervision is performed using the binary cross-entropy loss:

[0093] Among them, if point cluster If they belong to the same instance It is 1 if it is true, otherwise it is 0;

[0094] S35. Optimize the objective function using a server. and To obtain locally optimal network parameters.

[0095] Step S4: Perform panoramic segmentation of the new point cloud using the point cloud panoramic segmentation method based on sparse instance proposal and aggregation. The specific steps are as follows:

[0096] S41. Obtain radar point cloud in 3D environment;

[0097] S42. Input the radar point cloud into the semantic backbone network to obtain semantic prediction results and point cloud features;

[0098] S43. Obtain the foreground points based on the semantic prediction results, and input the foreground points into the sparse instance proposal module to obtain instance proposal point clusters;

[0099] S44. Input the instance proposed point clusters and point cloud features into the instance aggregation module to obtain aggregated instance point clusters;

[0100] S45. Based on semantic prediction, a semantic category is assigned to each aggregated instance point cluster through maximum voting, and this category is then fused with the semantic segmentation result to obtain the final panoptic segmentation result. For example... Figure 3 The image shows a visual comparison of the algorithm proposed in this invention with other advanced methods (DSNet, panoptic-polarnet). The algorithm proposed in this invention has better segmentation results in scenarios with crowded targets such as crowds and large targets such as buses.

[0101] The main innovative points of this invention are as follows:

[0102] (1) A sparse instance proposal (SIP) module was developed that groups instances directly from the original points of objects without additional learning tasks. It can be easily extended to other backbone networks and datasets and is used in a plug-and-play manner.

[0103] (2) SIP eliminates the dependence on offset branches and accelerates the clustering process through a cascaded design based on BPS, BS and CCL.

[0104] (3) An instance aggregation module is proposed to integrate possible fragmented instances and to supplement the SIP module to improve the segmentation performance of large objects.

[0105] (4) Advanced performance was achieved on SemanticKITTI and nuScenes, as shown in the table below:

[0106] Table 1 compares the proposed method (PANet) and other methods on the SemanticKITTI validation set.

[0107]

[0108] Table 2. Comparison of the proposed method (PANet) and other methods on the nuScenes validation set.

[0109]

[0110] While the invention has been described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the invention. Therefore, it should be understood that many modifications can be made to the exemplary embodiments, and other arrangements can be designed without departing from the spirit and scope of the invention as defined by the appended claims. It should be understood that different dependent claims and features described herein can be combined in ways different from those described in the original claims. It is also understood that features described in conjunction with individual embodiments can be used in other described embodiments.

Claims

1. A point cloud panoramic segmentation method based on sparse instance proposal and aggregation, characterized in that, Includes the following steps: S1. Construct a sparse instance proposal module based on the "sampling-movement-clustering" method and output instance proposals; S2. Based on KNN-Transformer, an instance aggregation module is constructed to aggregate the potentially fragmented instances generated by the sparse instance proposal module, and output accurate and complete instance segmentation results. S3. Utilize the server to optimize network parameters by reducing the network loss function until the network converges, thereby obtaining a point cloud panorama segmentation method based on sparse instance proposal and aggregation. S4. Perform panoramic segmentation of the new point cloud using the point cloud panoramic segmentation method based on sparse instance proposal and aggregation. Step S1 specifically includes the following steps: S11. Construct an equilibrium point sampling strategy to perform sparse sampling on the foreground points and obtain sparse seed points. To reduce computational burden and memory consumption; S12. Construct a "bubble shrinkage" moving module to move the sparse seed points. Iteratively move towards the instance center, causing seed points belonging to different instances to separate and seed points belonging to the same instance to aggregate; S13. Cluster the moved sparse seed points using a depth-first search-based connected component labeling algorithm, and assign other unsampled points to the clusters to which the corresponding seed points belong, thus obtaining the final instance proposal point clusters. N is the number of point clusters. For the first The set of points in a point cluster.

2. The point cloud panoramic segmentation method based on sparse instance proposal and aggregation according to claim 1, characterized in that, The equilibrium point sampling strategy specifically involves: dividing the input point cloud into voxel blocks using voxelization, then averaging the points within the same voxel as the seed point corresponding to all points in that voxel; this seed point governs all points within its voxel, including semantic category and instance ID; the voxelization operation specifically involves: Given a point cloud Hit the mark Its voxel index ,in For voxel resolution, For floor operations; all points with the same voxel index are in the same voxel block; The "bubble shrinkage" movement module specifically involves: first, empirically determining the minimum radius for each type of instance. Construct a connected graph using all seed points as vertices. If two vertices belong to the same category... And the distance between them is less than the minimum radius. If they are connected, then each seed point is assigned a bubble containing all points connected to it. The bubble shrinks iteratively based on the points within it, achieving the effect of clustering seed points belonging to the same instance and separating seed points belonging to different instances. The shrinking process is as follows: ； ； ； ； ； in, The connection matrix for constructing the connection graph. This is the number of iterations, with a default value of 4. The connected component labeling algorithm specifically involves: establishing a connected graph using the translated seed point as the vertex; connecting two vertices if they belong to the same category and the distance between them is less than the minimum radius corresponding to that category; and then using the connected component labeling algorithm to identify each cluster. The minimum radius in the connected component labeling is set to the value in the "bubble shrinking" moving module. Half of it.

3. The point cloud panoramic segmentation method based on sparse instance proposal and aggregation according to claim 2, characterized in that, Step S2 specifically includes the following steps: S21. Based on the instance proposal point clusters output by the sparse instance proposal module. Combine point cloud features to extract corresponding instance features. ; S22. Use KNN-Transformer to process the instance features. Model the dependencies between them and output enhanced instance features. ; S23. Using a lightweight multilayer perceptron, calculate the similarity between pairs of instances based on their features and center point positions. If the similarity is greater than a certain threshold, merge the two instances.

4. The point cloud panoramic segmentation method based on sparse instance proposal and aggregation according to claim 3, characterized in that, The specific process of step S21 is as follows: ； Where MLP stands for Multilayer Perceptron, and MaxPool is the max pooling function. These are the features and coordinate sets of the point cluster, respectively; the feature dimension of the extracted instance features is 64.

5. A point cloud panoramic segmentation method based on sparse instance proposal and aggregation according to claim 4, characterized in that, The specific process of step S22 is as follows: Based on the spatial location of the center point, k-nearest neighbors (KNN) are used to select clusters of points closest to the instance. The nearest k-point clusters Then, through a linear layer, the point clusters are... Corresponding instance features Mapped to query feature map Cluster k points Corresponding instance features Mapped to key-value feature pairs Then, self-attention is calculated to obtain the enhanced instance features. : ； Where C is 64.

6. A point cloud panoramic segmentation method based on sparse instance proposal and aggregation according to claim 5, characterized in that, The calculation process of the similarity in step S23 is as follows: ； in, For point clusters The coordinates of the center point are the average values ​​of the points in the point cluster set.

7. A point cloud panoramic segmentation method based on sparse instance proposal and aggregation according to claim 6, characterized in that, Step S3 specifically includes the following steps: S31. The server executes the training point cloud generation unit to perform random coordinate flipping, random point cloud rotation, and random point cloud scaling on the input original point cloud to generate an enhanced training point cloud input. S32. Utilize the server to execute the semantic backbone network, generating semantic prediction results and point cloud features of the input point cloud. Supervise the semantic backbone network, with the supervision loss being the sum of cross-entropy loss and Lovasz loss: ； in, For predicted values, The actual value; A Lovasz extension for IoU, Let C be the error vector for category c, where C is the total number of semantic categories; S33. Utilize the sparse instance proposal module described in step S1 on the server, input the foreground points, and generate instance proposal point clusters. ; S34. Utilize the instance aggregation module described in step S2 via the server to calculate the similarity between the instances. The supervision is performed using the binary cross-entropy loss: ； Among them, if point cluster If they belong to the same instance It is 1 if it is true, otherwise it is 0; S35. Optimize the objective function using a server. and To obtain locally optimal network parameters.

8. A point cloud panoramic segmentation method based on sparse instance proposal and aggregation according to claim 7, characterized in that, The semantic backbone network in step S32 consists of a 3D sparse convolutional network and a 2D U-Net; the 3D sparse convolutional network consists of four feature extraction blocks to extract multi-scale features; The 2D U-Net consists of a 2D convolutional network. It first performs two successive downsampling steps at a scale of 2, and then upsamples to the original resolution and projects it onto point features. The multi-scale features extracted by the 3D sparse convolutional network and the back-projected point features extracted by the 2D U-Net are concatenated and fed into a fully connected layer for feature compression to obtain point cloud features. The semantic backbone network is followed by a two-layer fully connected layer to predict semantic segmentation.

9. A point cloud panoramic segmentation method based on sparse instance proposal and aggregation according to claim 8, characterized in that, Step S4 specifically includes the following steps: S41. Obtain radar point cloud in 3D environment; S42. Input the radar point cloud into the semantic backbone network to obtain semantic prediction results and point cloud features; S43. Obtain the foreground points based on the semantic prediction results, and input the foreground points into the sparse instance proposal module to obtain instance proposal point clusters; S44. Input the instance proposed point clusters and point cloud features into the instance aggregation module to obtain aggregated instance point clusters; S45. Based on semantic prediction, assign a semantic category to each aggregated instance point cluster through the maximum vote, and then fuse it with the semantic segmentation result to obtain the final panoramic segmentation result.

Images