Point cloud segmentation method and system based on scene guidance, electronic equipment and storage medium
By introducing scene-guided feature selection and hollow attention compensation device in three-dimensional point cloud segmentation technology, the problem of missing details of point cloud data and data sparsity is solved, segmentation performance and computing efficiency are improved, and more accurate scene information capture is achieved and detailed feature representation.
Patent Information
- Application Number
- CN202411202648.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-29
- Publication Date
- 2025-01-03
AI Technical Summary
The existing three-dimensional point cloud segmentation technology faces the problems of missing details, data sparsity and high computing costs when processing point cloud data, which leads to difficulties in capturing context information and aggregating local features, and the flexibility of network structure design is limited. , it is difficult to fully restore the details of the scene.
Using a point cloud segmentation method based on scene guidance, a model including feature extraction encoder, feature decoder and hollow attention compensation device is designed. The high-level feature receptive field is expanded through the hollow attention compensation device, and the decoding process is optimized by combining the scene-guided feature selection mechanism. , reduce errors in multi-scale information aggregation.
Significantly improves the processing capability and segmentation performance of the point cloud segmentation model for complex point cloud data, improves the segmentation performance on recognized data sets, reduces the computational burden, enhances the correlation between points and captures scene information ability.
Smart Images
Figure CN119251489A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of three-dimensional vision, and specifically relates to a point cloud segmentation method, system, electronic device and storage medium based on scene guidance. Background Art
[0002] Recently, with the advancement of 3D radar sensing technology and data collection methods, 3D point cloud data has been widely used in many fields such as autonomous driving, augmented reality and virtual reality, which not only promotes the development of point cloud processing technology, but also enhances the in-depth understanding of 3D scenes. 3D point cloud semantic segmentation has become the key to achieving 3D vision tasks. In addition, with the rapid development of deep neural networks, deep learning-based methods such as the groundbreaking Pointnet that learns the global features of point clouds through multi-layer perceptrons, and the use of convolutional neural networks and Transformer networks based on self-attention mechanisms have made significant progress in 3D point cloud segmentation tasks.
[0003] Although neural network-based strategies have achieved remarkable success in 3D point cloud segmentation tasks, they still face challenging problems inherent in 3D radar point cloud data, such as the lack of detailed texture and the irregularity, unstructuredness and disorder of point cloud data.
[0004] First, the lack of detailed textures in point clouds makes the algorithm inadequate in the representation of shallow details, which makes it significantly difficult for the algorithm to capture contextual information and aggregate local features. Second, the sparse nature of point cloud data not only leads to high video memory usage and increased computing time, but also seriously restricts the flexibility of network structure design.
[0005] Existing research mainly focuses on optimizing the backbone network for feature extraction, and often uses a simple U-net-shaped feature fusion neck to integrate multi-level features. Although these enhanced feature extraction networks have made significant progress and greatly enhanced encoding capabilities by relying on mechanisms such as self-attention, relatively simple decoders fail to fully utilize the multi-scale information features provided by these efficient encoders and are unable to fully restore scene details.
[0006] Existing methods generally have difficulty in identifying the details of objects. Scenes play a vital role in recognizing objects. For example, a sofa exists in a living room and a bed appears in a bedroom. Such associations are often ignored in past models. Studies have shown that a powerful decoder is also crucial to network performance. Despite this, high computational costs remain the main obstacle to development in this field. Although some existing algorithms have been optimized for these problems and have optimized the computational occupancy of the algorithms to a certain extent, the high memory usage still brings problems such as poor versatility. Summary of the invention
[0007] In view of the shortcomings of several current point cloud segmentation technologies, the present invention focuses on using scene information to guide feature selection to fully analyze encoder information, and provides a scene-guided point cloud segmentation method, system, electronic device and storage medium.
[0008] In a first aspect, the present invention provides a point cloud segmentation method based on scene guidance, comprising the following steps:
[0009] Step 1: Prepare a point cloud dataset of any size and number of points and perform data preprocessing operations.
[0010] Step 2: Design a point cloud segmentation model, which includes a feature extraction encoder, a feature decoder, and a hole attention complementor.
[0011] The feature extraction encoder is used to extract multi-layer scale features from point cloud data;
[0012] The feature decoder is used to decode and fuse the features extracted by the feature extraction encoder from bottom to top, and output a segmentation result with the same resolution as the input image;
[0013] The hole attention compensator is used to expand the high-level feature receptive field with the hole result to provide the decoder with better feature restoration.
[0014] Step 3: Calculate the cross entropy loss between the predicted classification information generated by the point cloud segmentation model and the true value, and train the point cloud segmentation model.
[0015] Step 4: Repeat step 3 until the preset number of iterations is reached. Save the model parameters at the end of each iteration. Use an independent validation set to calculate the MIoU index of the generated point cloud labels after each round of training.
[0016] Step 5: Select the model parameters corresponding to the best MIoU indicator, import it into the point cloud segmentation model, and use different input point clouds to input the point cloud segmentation model to generate segmentation results.
[0017] In a second aspect, the present invention provides a point cloud segmentation system based on scene guidance, comprising:
[0018] Data processing module: used to prepare point cloud datasets of any size and number of points, and perform data preprocessing operations.
[0019] Design module: used to design a point cloud segmentation model, which includes a feature extraction encoder, a feature decoder and a hole attention complementor.
[0020] The feature extraction encoder is used to extract multi-layer scale features from point cloud data;
[0021] The feature decoder is used to decode and fuse the features extracted by the feature extraction encoder from bottom to top, and output a segmentation result with the same resolution as the input image;
[0022] The hole attention compensator is used to expand the high-level feature receptive field with the hole result to provide the decoder with better feature restoration.
[0023] Training module: used to calculate the cross entropy loss between the predicted classification information generated by the point cloud segmentation model and the true value, and train the point cloud segmentation model.
[0024] Iteration module: Repeat step 3 until the preset number of iterations is reached, save the model parameters at the end of each iteration, and use an independent validation set to calculate the MIoU indicator of the generated point cloud labels after each round of training.
[0025] Select output module: Select the model parameters corresponding to the best MIoU indicator, import it into the point cloud segmentation model, and use different input point clouds to input the point cloud segmentation model to generate segmentation results.
[0026] In a third aspect, the present invention provides an electronic device for scene-guided point cloud segmentation, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned scene-guided point cloud segmentation method when executing the program.
[0027] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute the above-mentioned scene-guided point cloud segmentation method.
[0028] Beneficial effects of the present invention:
[0029] (1) In order to make full use of the selection of scene-driven features, this paper proposes a scene-guided feature selection decoder, which is used to utilize the attention mechanism between scenes and features, optimize the feature decoding and fusion process, enhance the correlation between points, and reduce the error in multi-scale information aggregation.
[0030] (2) In order to address the limitations of traditional feature extraction networks in processing large-scale scene-level features, the present invention proposes a hole attention complementor, which is used to increase the receptive field through a specific hole strategy, thereby effectively capturing a wider range of scene information with a smaller computational burden.
[0031] (3) The present invention not only improves the point cloud segmentation model’s ability to process complex point cloud data, but also significantly improves the segmentation performance on recognized point cloud datasets (such as S3DIS and ScanNet). BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 This is a diagram of the structure of the point cloud segmentation model for scene-guided feature selection in an embodiment of the present application;
[0033] Figure 2 This is a structural diagram of the Oct-Mixed module in the feature extraction encoder of the embodiment of the present application;
[0034] Figure 3 This is a structural diagram of a scene-guided feature selection decoder according to an embodiment of the present application;
[0035] Figure 4 This is a specific structural diagram of the scene guide block in the embodiment of the present application;
[0036] Figure 5 This is a structural diagram of a hole attention compensator according to an embodiment of the present application;
[0037] Figure 6 A flow chart of training a point cloud segmentation model for an embodiment of the present application;
[0038] Figure 7 , 8 This is a comparison chart of the results of the embodiment of the present application and other methods. DETAILED DESCRIPTION
[0039] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0040] The present application provides a scene-guided point cloud segmentation method, including:
[0041] Step 1: Prepare a point cloud dataset of any size and number of points and perform data preprocessing operations.
[0042] Step 2: Design a point cloud segmentation model, which includes a feature extraction encoder, a feature decoder, and a hole attention complementor.
[0043] The feature extraction encoder is used to extract multi-layer scale features from point cloud data;
[0044] The feature decoder is used to decode and fuse the features extracted by the feature extraction encoder from bottom to top, and output a segmentation result with the same resolution as the input image;
[0045] The hole attention compensator is used to expand the high-level feature receptive field with the hole result to provide the decoder with better feature restoration.
[0046] Step 3: Calculate the cross entropy loss between the predicted classification information generated by the point cloud segmentation model and the true value, and train the point cloud segmentation model.
[0047] Step 4: Repeat step 3 until the preset number of iterations is reached. Save the model parameters at the end of each iteration. Use an independent validation set to calculate the MIoU index of the generated point cloud labels after each round of training.
[0048] Step 5: Select the model parameters corresponding to the best MIoU indicator, import it into the point cloud segmentation model, and use different input point clouds to input the point cloud segmentation model to generate segmentation results.
[0049] Furthermore, the data preprocessing operation in step 1 includes: standardizing the point cloud size to a set number of points, normalizing the coordinates and colors, and Z-aligning the coordinate information to improve the accuracy and consistency of the point cloud input.
[0050] Further, the step 2 comprises:
[0051] Step 2-1: Design a feature extraction encoder, which relies on octree and self-attention mechanism, has multi-layer downsampling results, and extracts multi-layer scale features from unordered and sparse point clouds.
[0052] Step 2-2: Create a feature decoder to decode and fuse the features extracted from the encoder, and finally output a segmentation result with the same resolution as the input image.
[0053] Step 2-3: Design a hole attention compensator to expand the high-level feature receptive field with the hole result to provide the decoder with better feature restoration.
[0054] Furthermore, in step 2-2, the feature decoder is composed of a bidirectional weighted pyramid module and a scene-guided feature selection module.
[0055] Further, the training of the point cloud segmentation model in step 3 includes:
[0056] Step 3-1: Perform data enhancement on point cloud data;
[0057] Step 3-2: Input the point cloud file into the encoder, calculate the feature map set under the multi-level and multi-scale receptive field, and input the feature map of the maximum receptive field of the last layer into the compensator;
[0058] Step 3-3: Put the feature map into the decoder to obtain a point-by-point classified feature map, calculate the CE loss between the point-by-point classification result and the classification label, and back-propagate to update the weights of the entire network.
[0059] Furthermore, the data enhancement in step 3-1 is specifically as follows: introducing a nonlinear transformation of coordinates to simulate possible distortion in the data acquisition process, and introducing a rotation angle of ±5 degrees, a scaling ratio of ±0.2, and a position jitter of 0.1 to enhance the original point cloud data set.
[0060] In the following, by way of example, Figure 1-Figure 6 , describing a scene-guided point cloud segmentation method provided by the present application, the specific implementation steps include:
[0061] Step 1: Prepare a point cloud dataset (.npz format) of any size and number of points, and perform a series of data preprocessing operations such as standardization and octree conversion. The initial steps include:
[0062] The point cloud size is standardized to a specific number of points, the coordinates and colors are normalized to ensure that their values are in the range of [0,1], and the coordinate information is Z-aligned to improve the accuracy and consistency of the point cloud input.
[0063] Step 2: In order to fully utilize the relationship between the upper and lower layers to achieve a better segmentation effect, this embodiment designs a point cloud segmentation model. Figure 1 As shown, the point cloud segmentation model includes a feature extraction encoder, a feature decoder and a hole attention compensator.
[0064] The feature extraction encoder relies on octree and self-attention mechanism, and has multiple layers of downsampling results, which are used to extract multi-layer scale features from unordered and sparse point clouds.
[0065] The feature decoder is used to decode and fuse the features extracted by the feature extraction encoder from bottom to top, and output a segmentation result with the same resolution as the input image.
[0066] The hole attention compensator is used to expand the high-level feature receptive field with a simple hole result to provide the decoder with better feature restoration.
[0067] Further, if Figure 1As shown in , the feature extraction encoder adopts a single-scale and multi-scale hybrid fusion encoder to provide richer shallow semantic information for fusion. The point cloud data first passes through an embedding module (Embedding) to vectorize the input point cloud, and then passes through N1 convolutional neural networks to form the first scale feature block. Next, this embodiment performs N2 Oct-Mixed modules on it without changing the size of the feature block to form a feature block of the same scale as the upper layer. Then, downsampling and Oct-Mixed or Oct-Transformer are used alternately to aggregate information to form feature blocks of the next three scales, where Oct-Mixed or Oct-Transformer is performed N3, N4, and N5 times respectively.
[0068] Furthermore, the specific structure of the Oct-Mixed module is as follows: Figure 2 As shown in the figure, the point cloud data is processed by the multi-head self-attention mechanism and the convolutional neural network at the same time, combining the advantages of both. The feature blocks generated by the two are concatenated and a multi-layer perceptron (MLP) is used to aggregate information.
[0069] Further, if Figure 3 As shown, the feature decoder is composed of a bidirectional weighted pyramid module and a scene-guided feature selection module.
[0070] The bidirectional weighted pyramid module consists of two parts, top-down and bottom-up, and is intended to achieve more balanced information processing to retain the original and processed information. In this embodiment, the feature fusion of the bottom-up L-1 layer is taken as an example:
[0071]
[0072] Among them, w1 and w2 are learnable parameters, Upsample() is a function that upsamples and aligns features. It refers to the feature input of layer l-1 in the network input to neck. It refers to the features of the l-2th layer in the bottom-up feature fusion, and ε is 0.0001 to avoid data instability.
[0073] In the top-down fusion process, the L-1 layer fuses three features from different scales:
[0074]
[0075] Among them, w′1, w′2, and w′3 represent the learnable parameters, It refers to the feature input of layer l-1 in the network input to neck. It is the feature generated in the bottom-up feature fusion. It is the feature generated in the top-down fusion in the L+1 layer. And before the final fusion and addition, the features must be normalized:
[0076]
[0077] Among them, n is 1 or 2, representing the features generated by bottom-up or top-down feature fusion, and l represents the number of layers in the octree.
[0078] After this processing, the scene-guided feature selection module introduces a cross-attention mechanism to generate bottom-up scene-guided feature selection. The cross-attention mechanism uses scene-guided instance-level features (shown as shallow features in the figure) to select, enabling it to bring more powerful error filtering capabilities, such as Figure 3 As shown, the scene guidance feature selection module is composed of four scene guidance blocks, each of which is specifically composed as follows: Figure 4 As shown in, the details are as follows:
[0079]
[0080] in, Represents the semantic features of the point cloud at the current scale, refers to the semantic features at a larger scene level, f MSA () refers to the multi-head self-attention mechanism, f rev () is a compensator used to make up for the gap between scene-level semantic features and shallow semantic features, f MCA () is a multi-head cross attention mechanism.
[0081] Furthermore, the hole attention compensator has a specific structure as follows: Figure 5 As shown in the figure, the input point cloud data is introduced into holes of different scales to form information of different receptive fields, and is then processed by a multi-head self-attention mechanism to provide scene information with a wider receptive field for scene guidance.
[0082] Step 3: Use the cross entropy loss function to calculate the distance between the probability distribution of the predicted classification information generated by the model and the true value:
[0083]
[0084] Among them, q(x) and p(x) represent the predicted sample and the correct sample respectively, and the difficulty of calculating the probability distribution p(x) expressed by the probability distribution q(x) is calculated.
[0085] The point cloud segmentation model training process is as follows Figure 6 shown.
[0086] First, the point cloud data is enhanced by introducing nonlinear transformation of coordinates to simulate possible distortion in the data acquisition process, and introducing a rotation angle of ±5 degrees, a scaling ratio of ±0.2, and a position jitter of 0.1 to enhance the original point cloud dataset.
[0087] Secondly, N point cloud files are input into the encoder to calculate the feature map set under multi-level and multi-scale receptive fields, and the feature map with the largest receptive field in the last layer is input into the complement to enhance its receptive field range.
[0088] Finally, the last layer feature map with a larger receptive field is used to replace the original feature map and input into the decoder. The decoder further integrates the multi-layer and multi-scale information obtained from the encoder and gradually restores the high-resolution feature map. The segmentation head obtains point-by-point prediction classification information, calculates the cross entropy loss between the true value, and back-propagates to update the entire model parameters.
[0089] Step 4: Repeat step 3 until the preset number of iterations is reached (the default is 800), traversing all point cloud data in each round, and saving the model parameters at the end of each iteration. At the same time, a part of the original data set is divided as a validation set. After each round of training, the mIoU index of the point cloud label generated by the validation set is predicted.
[0090] Among them, the evaluation index is Mean Intersection over Union (MIoU):
[0091] The standard metric for semantic segmentation is the ratio of the intersection and union of the model's predictions for each class and the true value, summed and averaged. The calculation formula for MIoU is as follows:
[0092]
[0093] where p ij Predicting i as j is a false negative; p ji Predicting j as i is a false positive; p ii Predict i as i, which is true.
[0094] Step 5: Select the model parameters corresponding to the best MIoU indicator, import it into the point cloud segmentation model, and use the point cloud input segmentation model with different inputs to generate segmentation results.
[0095] The specific data and parameters of the embodiments of the present application are described below. Figure 7The comparison results of the embodiment of the present application with other models are shown. The first column represents the input 3D model, the second column represents the true value of the input 3D model segmentation result, the third column represents the deformation result of other models, and the fourth column represents the segmentation result of the present invention. The specific details are as follows:
[0096] (1) The training data comes from two scene 3D point cloud datasets, and the test set is a part of these two datasets. The 3D point cloud data of different scales are cut into 120,000 points and 80,000 points respectively.
[0097] Among them, the ScanNet dataset is a large-scale 3D reconstruction and semantic segmentation dataset for indoor scene understanding, containing 2.5 million views from more than 1,500 scans, a total of 1,513 collected scene data, and 21 categories, of which 1,201 scenes are used for training and 312 scenes are used for testing.
[0098] S3DIS dataset: It is a commonly used indoor scene understanding and 3D semantic segmentation dataset, which contains point cloud data and detailed semantic labels of multiple indoor buildings, including 6 regions and 12 semantic categories in total, covering 11 scenes such as offices, conference rooms and corridors. Because it is obtained from real indoor environments and retains information such as the geometric shape of the real environment, this authenticity makes it one of the important benchmarks for algorithm evaluation and comparison in the field of indoor scene understanding.
[0099] (2) The embodiment of the present application is based on the PyTorch deep learning framework, and the computer configuration is an Intel Core i7-13700K processor and an NVIDIA RTX4090 graphics card.
[0100] The training was performed on two datasets, ScanNet and S3DIS, for 800 rounds. The initial learning rate was set to 0.0015 and was reduced to one-tenth of the original rate at the 360th and 520th rounds. The training input batch size was set to 4 and the depth of the octree was 11. Due to the irregular size of point cloud data, the data preprocessing method was used on the training set to cut the point cloud files to 120,000 points and 800,000 points respectively, and then use the network to train them, while no processing was done on the validation set.
[0101] (3) The embodiment of the present application uses the training parameters of the round with the highest MIoU during the training process, and is tested on the ScanNet dataset and achieves an MIoU index of 76.5; it is tested on the S3DIS dataset and achieves an MIoU index of 71.8.
[0102] Tables 1 and 2 respectively show the comparison of the MIoU index between the embodiments of the present application and other methods. The results show that the segmentation effect of the embodiments of the present application is generally better than that of other methods.
[0103] Table 1 Accuracy of different methods on ScanNet dataset
[0104]
[0105]
[0106] Table 2 Accuracy of different methods in Area 5 of the S3DIS dataset
[0107]
[0108] (4) Figure 8 More segmentation results of the embodiment of the present application are shown. The results show that the embodiment of the present application did not have overfitting during the training process, and achieved good performance in scene semantic segmentation. It can also introduce a scene-guided feature selection mechanism to establish a forced cooperation of high receptive field with detail features, thereby improving its ability to filter out erroneous information and achieving more accurate global and detail feature representation. It is superior to the existing technology in all evaluation indicators, showing its excellent performance in point cloud segmentation tasks.
[0109] On the other hand, this embodiment provides a system, applying the scene-guided point cloud segmentation method, including:
[0110] Data processing module: used to prepare point cloud datasets of any size and number of points, and perform data preprocessing operations;
[0111] Design module: used to design a point cloud segmentation model, wherein the point cloud segmentation model includes a feature extraction encoder, a feature decoder and a hole attention complementor;
[0112] The feature extraction encoder is used to extract multi-layer scale features from point cloud data;
[0113] The feature decoder is used to decode and fuse the features extracted by the feature extraction encoder from bottom to top, and output a segmentation result with the same resolution as the input image;
[0114] The hole attention compensator is used to expand the high-level feature receptive field with the hole result to provide the decoder with better feature restoration;
[0115] Training module: used to calculate the cross entropy loss between the predicted classification information generated by the point cloud segmentation model and the true value, and train the point cloud segmentation model;
[0116] Iteration module: Repeat the training until the preset number of iterations is reached, save the model parameters at the end of each iteration, and use an independent validation set to calculate the MIoU index of the generated point cloud labels after each round of training;
[0117] Select output module: Select the model parameters corresponding to the best MIoU indicator, import it into the point cloud segmentation model, and use different input point clouds to input the point cloud segmentation model to generate segmentation results.
[0118] Furthermore, the design module adopts an encoder-decoder architecture and designs a scene-guided feature selection decoder, which uses a bidirectional weighted pyramid to balance the deep and shallow feature map information, and then uses cross-attention to guide the shallow feature selection from bottom to top, calculates the loss based on the cross-entropy loss function, and uses the back-propagation mechanism to pass the loss information back to the segmentation model for optimization and update.
[0119] On the other hand, the embodiment of the present application also discloses an electronic device for scene-guided point cloud segmentation, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the scene-guided point cloud segmentation method when executing the computer program, wherein the memory may include a memory, such as a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk memory, etc.; the processor, the network interface, and the memory are interconnected through an internal bus, and the internal bus may be an industrial standard architecture bus, a peripheral component interconnection standard bus, an extended industrial standard architecture bus, etc., and the bus may be divided into an address bus, a data bus, a control bus, etc. The memory is used to store programs, and specifically, the program may include a program code, and the program code includes computer operation instructions. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.
[0120] The embodiment of the present application also discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of a point cloud segmentation method based on scene guidance are implemented. Specifically, the computer-readable storage medium includes, but is not limited to, for example, volatile memory and / or non-volatile memory. The volatile memory may include random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include a read-only memory (ROM), a hard disk, a flash memory, an optical disk, a magnetic disk, etc.
[0121] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A point cloud segmentation method based on scene guidance, characterized in that: The following steps are involved: Step 1: Prepare a point cloud dataset of any size and number of points and perform data preprocessing operations; Step 2: Design a point cloud segmentation model, which includes a feature extraction encoder, a feature decoder, and a hole attention filler; The feature extraction encoder is used to extract multi-layer scale features from point cloud data; The feature decoder is used to decode and fuse the features extracted by the feature extraction encoder from bottom to top, and output a segmentation result with the same resolution as the input image; The hole attention compensator is used to expand the high-level feature receptive field with the hole result to provide the decoder with better feature restoration; Step 3: Calculate the cross entropy loss between the predicted classification information generated by the point cloud segmentation model and the true value, and train the point cloud segmentation model; Step 4: Repeat step 3 until the preset number of iterations is reached, save the model parameters at the end of each iteration, and use an independent validation set to calculate the MIoU index of the generated point cloud labels after each round of training; Step 5: Select the model parameters corresponding to the best MIoU indicator, import it into the point cloud segmentation model, and use different input point clouds to input the point cloud segmentation model to generate segmentation results.
2. The scene-guided point cloud segmentation method according to claim 1, characterized in that: The data preprocessing operation in step 1 includes: standardizing the point cloud size to a set number of points, normalizing the coordinates and colors, and Z-aligning the coordinate information to improve the accuracy and consistency of the point cloud input.
3. The scene-guided point cloud segmentation method according to claim 1, characterized in that: The step 2 comprises: Step 2-1: Design a feature extraction encoder, which relies on octree and self-attention mechanism, has multi-layer downsampling results, and extracts multi-layer scale features from unordered and sparse point clouds; Step 2-2: Create a feature decoder to decode and fuse the features extracted from the encoder, and finally output a segmentation result with the same resolution as the input image; Step 2-3: Design a hole attention compensator to expand the high-level feature receptive field with the hole result to provide the decoder with better feature restoration.
4. The scene-guided point cloud segmentation method according to claim 3, characterized in that: In step 2-2, the feature decoder is composed of a bidirectional weighted pyramid module and a scene-guided feature selection module.
5. The scene-guided point cloud segmentation method according to claim 1, characterized in that: The training of the point cloud segmentation model in step 3 includes: Step 3-1: Perform data enhancement on point cloud data; Step 3-2: Input the point cloud file into the encoder, calculate the feature map set under the multi-level and multi-scale receptive field, and input the feature map of the maximum receptive field of the last layer into the compensator; Step 3-3: Put the feature map into the decoder to obtain a point-by-point classified feature map, calculate the CE loss between the point-by-point classification result and the classification label, and back-propagate to update the weights of the entire network.
6. The scene-guided point cloud segmentation method according to claim 5, characterized in that: The data enhancement in step 3-1 is specifically as follows: introducing a nonlinear transformation of coordinates to simulate possible distortion in the data acquisition process, and introducing a rotation angle of ±5 degrees, a scaling ratio of ±0.2, and a position jitter of 0.1 to enhance the original point cloud dataset.
7. A point cloud segmentation system based on scene guidance, characterized in that: include: Data processing module: used to prepare point cloud datasets of any size and number of points, and perform data preprocessing operations; Design module: used to design a point cloud segmentation model, wherein the point cloud segmentation model includes a feature extraction encoder, a feature decoder and a hole attention complementor; The feature extraction encoder is used to extract multi-layer scale features from point cloud data; The feature decoder is used to decode and fuse the features extracted by the feature extraction encoder from bottom to top, and output a segmentation result with the same resolution as the input image; The hole attention compensator is used to expand the high-level feature receptive field with the hole result to provide the decoder with better feature restoration; Training module: used to calculate the cross entropy loss between the predicted classification information generated by the point cloud segmentation model and the true value, and train the point cloud segmentation model; Iteration module: Repeat the training until the preset number of iterations is reached, save the model parameters at the end of each iteration, and use an independent validation set to calculate the MIoU index of the generated point cloud labels after each round of training; Select output module: Select the model parameters corresponding to the best MIoU indicator, import it into the point cloud segmentation model, and use different input point clouds to input the point cloud segmentation model to generate segmentation results.
8. The scene-guided point cloud segmentation system according to claim 7, characterized in that: The design module: An encoder-decoder architecture is adopted, and a scene-guided feature selection decoder is designed to: After using a bidirectional weighted pyramid to balance the information of deep and shallow feature maps, cross attention is used to guide shallow feature selection from bottom to top, and the loss is calculated based on the cross entropy loss function. The back propagation mechanism is used to pass the loss information back to the segmentation model for optimization and update.
9. An electronic device based on scene-guided point cloud segmentation, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a scene-guided point cloud segmentation method according to any one of claims 1 to 6 is implemented.
10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the scene-guided point cloud segmentation method described in any one of claims 1 to 6.