Point cloud semantic segmentation method and system based on expansion attention and multi-prototype fusion

Abstract

The invention discloses a point cloud semantic segmentation method and system based on extended attention and multi-prototype fusion. The method comprises the following steps: training a to-be-processed point cloud sample to obtain a pre-training model; screening the to-be-processed point cloud sample to obtain a target point cloud sample; performing feature extraction on the target point cloud sample by using the pre-training model to obtain sample feature data; building an expanded attention block model, and training the expanded attention block model based on multi-view prototype loss and the sample feature data to obtain a trained neural network; and performing semantic segmentation on the point cloud data based on the trained neural network. Therefore, a small amount of existing information can be fully extracted, and a segmentation network model with stronger generalization ability is constructed.

Description

Technical Field

[0001] The present invention relates to the technical field of point cloud data processing, and in particular to a point cloud semantic segmentation method and system based on expanded attention and multi-prototype fusion. Background Art

[0002] In the actual application of point cloud semantic segmentation tasks, researchers and staff may also encounter the problem of lack of sufficient annotated data sets, and accurately annotating 3D point cloud data sets requires a lot of manpower and time costs. In addition, when faced with new data sets, existing models are often difficult to directly migrate and apply, which requires the model to be retrained to adapt to new tasks, further increasing the consumption of time and computing resources. In view of these challenges, the use of meta-learning methods to conduct research on point cloud semantic segmentation has become an effective solution. Meta-learning aims to design models that can quickly adapt to new tasks and perform well even with very small amounts of annotated data. More specifically for point cloud semantic segmentation tasks, a feasible method is to use pre-trained models as a basis, make full use of existing knowledge, freeze the encoding layer parameters of the model, and only fine-tune the parameters for new categories encountered. In this way, only a few annotated samples are needed to adapt the model to new categories, thereby saving a lot of manpower annotation and training costs.

[0003] Current methods have made significant progress in improving the representation ability of prototype networks, solving the problem of unknown category segmentation, and reducing dependence on large amounts of labeled data, but this field still faces a series of problems that need to be solved. First, the characteristics of few-shot learning determine that the network cannot extract information from massive data, which requires the network to have stronger detail capture capabilities. In addition, existing network models do not pay attention to the quality of samples and the training bias caused by intra-class differences. Therefore, it is necessary to propose a few-shot point cloud semantic segmentation network to fully extract the existing small amount of information and build a model with stronger generalization capabilities. Summary of the invention

[0004] The present invention provides a point cloud semantic segmentation method and system based on expanded attention and multi-prototype fusion, which can fully extract the existing small amount of information and construct a segmentation network model with stronger generalization ability.

[0005] In a first aspect, a point cloud semantic segmentation method based on dilated attention and multi-prototype fusion is provided, comprising the following steps:

[0006] The pre-trained model is obtained by training using the point cloud samples to be processed;

[0007] Screen the point cloud samples to be processed to obtain target point cloud samples;

[0008] Using the pre-trained model to extract features from the target point cloud sample to obtain sample feature data;

[0009] Building an extended attention block model, and training the extended attention block model based on multi-view prototype loss and the sample feature data to obtain a trained neural network;

[0010] Perform semantic segmentation on point cloud data based on the trained neural network.

[0011] According to the first aspect, in a first possible implementation manner of the first aspect, the step of “screening the point cloud samples to be processed to obtain target point cloud samples” specifically includes the following steps:

[0012] Divide the point cloud to be processed into multiple sample blocks, calculate the proportion of target category points in each of the sample blocks, and regard the corresponding sample blocks when the proportion is greater than a preset threshold as target point cloud samples;

[0013] The preset threshold is the ratio of the target category points in the point cloud samples to be processed multiplied by a preset coefficient.

[0014] According to the first aspect, in a second possible implementation manner of the first aspect, the processing flow of the expanded attention block model is:

[0015] Using two layers of self-attention blocks and a multi-layer perceptron to fuse the sample feature data to obtain fused features;

[0016] The fused features are processed using a foreground prototype generator, a background prototype generator and a three-layer multi-head attention block to obtain prototype features.

[0017] According to the second possible implementation manner of the first aspect, in a third possible implementation manner of the first aspect, a calculation formula for the fusion feature f4 is as follows:

[0018]

[0019] Where, f1 = EC(f0);

[0020] f2=MLP(f1);

[0021] f3=SA(f1) 2 ;

[0022] Where f0 is the target point cloud sample; f1 is the sample feature data obtained by extracting features from the target point cloud sample using the feature extractor EC of the pre-trained model; f2 is the feature encoding of the sample feature data using the multi-layer perceptron MLP; and f3 is the weighted processing of the sample feature data using the two-layer self-attention block SA.

[0023] According to the second possible implementation manner of the first aspect, in a fourth possible implementation manner of the first aspect, a calculation formula for the prototype feature P is as follows:

[0024]

[0025] in,

[0026] Where bg and fg are the background prototype and foreground prototype obtained by processing the fusion features using the background prototype generator and the foreground prototype generator respectively; To deeply fuse bg and fg using the three-layer multi-head attention block MA.

[0027] According to the first aspect, in a fifth possible implementation manner of the first aspect, the multi-view prototype loss The calculation formula is as follows:

[0028]

[0029] Among them, L i =CE(r i );

[0030]

[0031] In the formula, is the prototype feature corresponding to each point cloud block in the target point cloud sample generated by the expanded attention module; n is the number of point cloud blocks; r i is the prediction result corresponding to each prototype feature obtained by the predictor Pred; L i is the cross entropy loss corresponding to each prediction result.

[0032] According to the first aspect, in a sixth possible implementation manner of the first aspect, the step of “performing semantic segmentation on point cloud data based on a trained neural network” specifically includes the following steps:

[0033] Obtain query sample features and prototype features after point cloud data is processed by the trained neural network;

[0034] Calculate query sample feature x p With prototype feature y f Gaussian similarity between them for semantic segmentation;

[0035] The calculation formula of the Gaussian similarity is as follows:

[0036] similarity(x p ,y f ) = exp(-||x p -y f||^2 / 2σ^2);

[0037] Where σ is the standard deviation of the Gaussian distribution.

[0038] According to the first aspect, in a seventh possible implementation manner of the first aspect, the step of “using the to-be-processed point cloud samples for training to obtain a pre-trained model” specifically includes the following steps:

[0039] The point cloud samples to be processed are divided into a support set and a query set, and the DGCNN backbone network is trained using the support set and the query set to obtain a pre-trained model.

[0040] Secondly, a point cloud semantic segmentation system based on dilated attention and multi-prototype fusion is provided, including:

[0041] A pre-training module is used to obtain a pre-training model by training with the point cloud samples to be processed;

[0042] A screening module is used to screen the point cloud samples to be processed to obtain target point cloud samples;

[0043] A feature extraction module, which is in communication with the pre-training module and the screening module, and is used to extract features from the target point cloud sample using the pre-training model to obtain sample feature data;

[0044] an extended attention training module, which is in communication with the feature extraction module and is used to build an extended attention block model, train the extended attention block model based on multi-view prototype loss and the sample feature data, and obtain a trained neural network; and

[0045] The semantic segmentation module is communicatively connected to the dilated attention training module and is used to perform semantic segmentation on the point cloud data based on the trained neural network.

[0046] In a third aspect, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that when the computer program is executed by a processor, the point cloud semantic segmentation method based on dilated attention and multi-prototype fusion as described above is implemented.

[0047] Compared with the prior art, the advantages of the present invention are as follows: by screening the point cloud samples to be processed, representative and diverse samples can be selected from limited samples to address the problem of sample inhomogeneity. This is different from the conventional random sample selection strategy. The present invention can significantly improve the quality and representativeness of the samples and provide a more effective learning basis for the model.

[0048] Then, an extended attention block model is built. Therefore, the extended attention block is used to deeply explore the effective information of the foreground and background. Without greatly increasing the computational burden, the receptive field of the model can be significantly expanded, and various detailed changes in the scene can be effectively distinguished and understood.

[0049] The multi-view prototype loss proposed at the same time can solve the impact of style diversity within the category on few-sample learning. By integrating prototypes generated from different viewpoints and samples, a comprehensive category representation is formed, thereby enhancing the model's generalization and recognition capabilities for different styles under the same category. This method not only improves the accuracy of the model when dealing with style diversity, but also provides a new perspective for in-depth understanding and processing of complex semantic relationships in 3D point cloud data. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a flow chart of an embodiment of a point cloud semantic segmentation method based on dilated attention and multi-prototype fusion according to the present invention;

[0051] Figure 2 It is a flow chart of another embodiment of a point cloud semantic segmentation method based on expanded attention and multi-prototype fusion of the present invention;

[0052] Figure 3 It is a schematic diagram of the network structure of the expanded attention block model of the present invention;

[0053] Figure 4 It is a structural schematic diagram of a point cloud semantic segmentation system based on expanded attention and multi-prototype fusion in the present invention. DETAILED DESCRIPTION

[0054] Reference will now be made in detail to specific embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Although the present invention will be described in conjunction with specific embodiments, it will be understood that it is not intended to limit the present invention to the embodiments described. On the contrary, it is intended to cover changes, modifications and equivalents included in the spirit and scope of the present invention as defined by the appended claims. It should be noted that the method steps described herein can all be implemented by any functional block or functional arrangement, and any functional block or functional arrangement can be implemented as a physical entity or a logical entity, or a combination of the two.

[0055] In order to enable those skilled in the art to better understand the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0056] Note: The example to be introduced below is only a specific example, and is not intended to limit the embodiments of the present invention to the following specific steps, values, conditions, data, sequences, etc. Those skilled in the art can apply the concept of the present invention to construct more embodiments not mentioned in this specification by reading this specification.

[0057] See also Figure 1 and Figure 2 As shown, an embodiment of the present invention provides a point cloud semantic segmentation method based on dilated attention and multi-prototype fusion, comprising the following steps:

[0058] S100, using the point cloud samples to be processed to perform training to obtain a pre-trained model;

[0059] S200, screening the point cloud samples to be processed to obtain target point cloud samples;

[0060] S300, extracting features from the target point cloud sample using the pre-trained model to obtain sample feature data;

[0061] S400, building an extended attention block model, and training the extended attention block model based on multi-view prototype loss and the sample feature data to obtain a trained neural network;

[0062] S500, performs semantic segmentation on point cloud data based on a trained neural network.

[0063] Specifically, in this embodiment, the present invention first screens the point cloud samples to be processed, and can select representative and diverse samples from limited samples to address the problem of sample inhomogeneity. This is different from the conventional random sample selection strategy. The present invention can significantly improve the quality and representativeness of the samples, and provide a more effective learning basis for the model.

[0064] Then, an extended attention block model is built. Therefore, the extended attention block is used to deeply explore the effective information of the foreground and background. Without greatly increasing the computational burden, the receptive field of the model can be significantly expanded, and various detailed changes in the scene can be effectively distinguished and understood.

[0065] The multi-view prototype loss proposed at the same time can solve the impact of style diversity within the category on few-sample learning. By integrating prototypes generated from different viewpoints and samples, a comprehensive category representation is formed, thereby enhancing the model's generalization and recognition capabilities for different styles under the same category. This method not only improves the accuracy of the model when dealing with style diversity, but also provides a new perspective for in-depth understanding and processing of complex semantic relationships in 3D point cloud data.

[0066] Preferably, in another embodiment of the present application, the step of “S100, using the point cloud samples to be processed to perform training to obtain a pre-trained model” specifically includes the following steps:

[0067] The point cloud samples to be processed are divided into a support set and a query set, and the DGCNN backbone network is trained using the support set and the query set to obtain a pre-trained model.

[0068] Specifically, in this embodiment, in the pre-training stage, the point cloud samples to be processed are divided into several small blocks, and these blocks are regarded as independent samples to construct support sets and query sets, and then the DGCNN backbone network is used for pre-training to obtain a pre-training model.

[0069] Preferably, in another embodiment of the present application, the step of “S200, screening the point cloud samples to be processed to obtain target point cloud samples” specifically includes the following steps:

[0070] Divide the point cloud to be processed into multiple sample blocks, calculate the proportion of target category points in each of the sample blocks, and regard the corresponding sample blocks when the proportion is greater than a preset threshold as target point cloud samples;

[0071] The preset threshold is the ratio of the target category points in the point cloud samples to be processed multiplied by a preset coefficient.

[0072] Specifically, in this embodiment, using better quality support samples can enable the network to learn more accurate foreground and background features, thereby improving the training effect. Calculate the proportion of target category points in the total number of points in the sample block c k , when the proportion exceeds the set threshold t k When a point cloud is found, it is marked as a high-quality sample block and is selected first. In addition, since the proportion of different categories in the block is different (for example, walls are more likely to obtain a larger proportion of points than chairs), different thresholds should be set for different categories. For ease of calculation, the total proportion of each category in the point cloud to be processed is multiplied by the preset coefficient 0.8 as the threshold of each target category.

[0073] The high-quality sample block set (target point cloud sample) Q is expressed as follows:

[0074]

[0075]

[0076] Preferably, in another embodiment of the present application, the processing flow of the expanded attention block model is:

[0077] Using two layers of self-attention blocks and a multi-layer perceptron to fuse the sample feature data to obtain fused features;

[0078] The fused features are processed using a foreground prototype generator, a background prototype generator and a three-layer multi-head attention block to obtain prototype features.

[0079] Specifically, in this embodiment, in order to solve the problem that the direct use of attention has a relatively shallow understanding of the dependency relationship on the target features and a relatively narrow receptive field, the present invention proposes an expanded attention block. By reusing the attention module on the features extracted by attention, the receptive field of the network can be increased, and the problem of uneven information distribution can be solved at a deeper level. Figure 3 This is the network structure diagram of the proposed dilated attention block model, which is mainly composed of a self-attention module, a multi-head attention module and a multi-head attention module, where N is the number of points and K is the number of prototypes.

[0080] Preferably, in another embodiment of the present application, the calculation formula of the fusion feature f4 is as follows:

[0081]

[0082] Where, f1 = EC(f0);

[0083] f2=MLP(f1);

[0084] f3=SA(f1) 2 ;

[0085] Where f0 is the target point cloud sample; f1 is the sample feature data obtained by extracting features from the target point cloud sample using the feature extractor EC of the pre-trained model; f2 is the feature encoding of the sample feature data using the multi-layer perceptron MLP; and f3 is the weighted processing of the sample feature data using the two-layer self-attention block SA.

[0086] Specifically, in this embodiment, by stacking two layers of self-attention blocks, a deep understanding of the feature data can be gradually built up, and the model can learn more global features to deepen the understanding of the overall structure and contextual relationships of the point cloud object.

[0087] For the target point cloud sample f0 (divided into support point set and query point set), the pre-learned feature f1 is first obtained through the feature extractor EC of the pre-trained model. The multi-layer perceptron MLP is used to encode the features of f1 to obtain the high-dimensional feature f2. Subsequently, the two-layer self-attention module S is used on f1. A Get the weighted feature f3. Finally, fuse the features f1, f2 and f3 to get the support or query encoding feature f4.

[0088] Preferably, in another embodiment of the present application, the calculation formula of the prototype feature P is as follows:

[0089]

[0090] in,

[0091] Where bg and fg are the background prototype and foreground prototype obtained by processing the fusion features using the background prototype generator and the foreground prototype generator respectively; To deeply fuse bg and fg using three-layer multi-head attention blocks.

[0092] Specifically, in the present embodiment, in the task of semantic segmentation of a few-sample point cloud, it is particularly important to accurately understand the differences and dependencies between foreground and background information. Therefore, the present invention stacks three layers of multi-head attention blocks, and through the complex feature fusion between layers, the network can take into account the local details of the deeper foreground and background features, thereby capturing more complex and abstract features and dependencies, and further amplifying the adaptive ability to feature focus points. Specifically, for the encoded fusion feature f4, first obtain its background prototype bg and foreground prototype fg. Then use the three-layer multi-head attention module MA to obtain the deep fusion background prototype Finally, the foreground prototype fg is fused with the background prototype Get the prototype feature P of this iteration.

[0093] Preferably, in another embodiment of the present application, the calculation formula of the multi-view prototype loss L is as follows:

[0094]

[0095] Among them, L i =CE(r i );

[0096]

[0097] In the formula, is the prototype feature corresponding to each point cloud block in the target point cloud sample generated by the expanded attention module; n is the number of point cloud blocks; r i is the prediction result corresponding to each prototype feature obtained by the predictor Pred; L i is the cross entropy loss corresponding to each prediction result.

[0098] Specifically, in this embodiment, although a high-quality support sample set is used, due to the irregularity of the three-dimensional point cloud object, the generated prototype and the query feature will still deviate. Objects of the same category may have different colors, orientations, sizes, and shapes. This difference may cause deviations in the model prediction process. In order to solve this problem, the present invention proposes a multi-view prototype loss, which uses high-quality samples in different blocks to generate prototypes of multiple perspectives and uses each prototype for prediction, and then uses multiple prediction results to calculate the cross entropy loss function and average it, so as to obtain prototype features with stronger generalization ability.

[0099] Specifically, for different selected high-quality samples {s1 …s i …s n}, use the expanded attention block G to generate the prototype corresponding to each point cloud block in the high-quality samples (target point cloud samples), and get the prototype set

[0100]

[0101] Then use the predictor to obtain the prediction result corresponding to each prototype feature {r 1 …r i …r n}, and calculate the corresponding cross entropy loss {L 1 ...L i ...L n}, and finally find the mean of the cross entropy loss as a multi-view prototype loss.

[0102] Preferably, in another embodiment of the present application, the step of “S500, performing semantic segmentation on the point cloud data based on the trained neural network” specifically includes the following steps:

[0103] Obtain query sample features and prototype features after point cloud data is processed by the trained neural network;

[0104] Calculate query sample feature x p With prototype feature y f Gaussian similarity between them for semantic segmentation;

[0105] The calculation formula of the Gaussian similarity is as follows:

[0106] similarity(x p ,y f ) = exp(-||x p -y f ||^2 / 2σ^2);

[0107] Where σ is the standard deviation of the Gaussian distribution.

[0108] Specifically, in this embodiment, by calculating the Gaussian similarity between the query sample vector and the prototype feature vector, the vectors are classified and merged according to the size of the similarity, and the vectors with large similarity are classified into one category to achieve semantic segmentation of point cloud data. The calculation method of Gaussian similarity is based on the probability density function of Gaussian distribution.

[0109] In the calculation formula of Gaussian similarity, ||x p -y f || represents vector x p and fThe Euclidean distance between them; σ is the standard deviation of the Gaussian distribution. When σ is larger, the value of the Gaussian similarity function is closer to 1, indicating that the similarity between the two vectors is higher; when σ is smaller, the value of the Gaussian similarity function is closer to 0, indicating that the similarity between the two vectors is lower.

[0110] See also Figure 4 As shown, the embodiment of the present invention further provides a point cloud semantic segmentation system based on dilated attention and multi-prototype fusion, including:

[0111] A pre-training module is used to obtain a pre-training model by training with the point cloud samples to be processed;

[0112] A screening module is used to screen the point cloud samples to be processed to obtain target point cloud samples;

[0113] A feature extraction module, which is in communication with the pre-training module and the screening module, and is used to extract features from the target point cloud sample using the pre-training model to obtain sample feature data;

[0114] an extended attention training module, which is in communication with the feature extraction module and is used to build an extended attention block model, train the extended attention block model based on multi-view prototype loss and the sample feature data, and obtain a trained neural network; and

[0115] The semantic segmentation module is communicatively connected to the dilated attention training module and is used to perform semantic segmentation on the point cloud data based on the trained neural network.

[0116] Therefore, in order to solve the problem that few-sample point cloud semantic segmentation tasks cannot extract information from massive data, the present invention proposes an expanded attention block strategy to enhance the model's understanding of foreground and background information and its information extraction capabilities, and significantly expand the model's receptive field without significantly increasing the computational burden. In order to solve the problem that the selected samples lack representativeness due to random sample selection, predefined criteria and algorithms are used to comprehensively evaluate the candidate samples, so as to screen out those samples that contain rich target category information and can provide more learning value in the candidate sample pool. In order to solve the problem of diversity that may exist within the same category of point cloud data, the model's understanding and adaptability to intra-class diversity are enhanced through multi-view prototype loss calculation, thereby improving the model's generalization ability when facing new styles or design styles.

[0117] Specifically, this embodiment corresponds one-to-one to the above method embodiment, and the functions of each module have been described in detail in the corresponding method embodiment, so they will not be repeated here.

[0118] Based on the same inventive concept, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, all or part of the method steps of the above method are implemented.

[0119] The present invention implements all or part of the process in the above method, and can also be completed by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of each of the above method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.

[0120] Based on the same inventive concept, an embodiment of the present application also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program running on the processor, and when the processor executes the computer program, all or part of the method steps in the above method are implemented.

[0121] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of a computer device, and uses various interfaces and lines to connect various parts of the entire computer device.

[0122] The memory can be used to store computer programs and / or modules. The processor realizes various functions of the computer device by running or executing the computer programs and / or modules stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required for a function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created according to the use of the mobile phone (such as audio data, video data, etc.). In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0123] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, servers or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program codes.

[0124] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), servers, and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as a combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0125] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0126] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0127] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A point cloud semantic segmentation method based on dilated attention and multi-prototype fusion, It is characterized in that The following steps are involved: The pre-trained model is obtained by training using the point cloud samples to be processed; Screen the point cloud samples to be processed to obtain target point cloud samples; Using the pre-trained model to extract features from the target point cloud sample to obtain sample feature data; Building an extended attention block model, and training the extended attention block model based on multi-view prototype loss and the sample feature data to obtain a trained neural network; Perform semantic segmentation on point cloud data based on the trained neural network.

2. The point cloud semantic segmentation method based on dilated attention and multi-prototype fusion as claimed in claim 1, It is characterized in that The step of "screening the point cloud samples to be processed to obtain target point cloud samples" specifically includes the following steps: Divide the point cloud to be processed into multiple sample blocks, calculate the proportion of target category points in each of the sample blocks, and regard the corresponding sample blocks when the proportion is greater than a preset threshold as target point cloud samples; The preset threshold is the ratio of the target category points in the point cloud samples to be processed multiplied by a preset coefficient.

3. The point cloud semantic segmentation method based on dilated attention and multi-prototype fusion as claimed in claim 1, It is characterized in that The processing flow of the dilated attention block model is as follows: Using two layers of self-attention blocks and a multi-layer perceptron to fuse the sample feature data to obtain fused features; The fused features are processed using a foreground prototype generator, a background prototype generator and a three-layer multi-head attention block to obtain prototype features.

4. The point cloud semantic segmentation method based on dilated attention and multi-prototype fusion as claimed in claim 3, It is characterized in that The fusion feature f 4 The calculation formula is as follows: Among them, f 1 =EC(f 0 ); f 2 =MLP(f 1 ): f 3 =SA(f 1 ) 2 ; In the formula, f 0 is the target point cloud sample; f 1 The sample feature data is obtained by extracting features from the target point cloud samples using the feature extractor EC of the pre-trained model; 2 To use the multi-layer perceptron MLP to encode the sample feature data; 3 In order to use the two-layer self-attention block SA to perform weighted processing on the sample feature data.

5. The point cloud semantic segmentation method based on dilated attention and multi-prototype fusion as claimed in claim 3, It is characterized in that The calculation formula of the prototype feature P is as follows: in, Where bg and fg are the background prototype and foreground prototype obtained by processing the fusion features using the background prototype generator and foreground prototype generator respectively; To deeply fuse bg and fg using the three-layer multi-head attention block MA.

6. The point cloud semantic segmentation method based on dilated attention and multi-prototype fusion as claimed in claim 1, It is characterized in that The multi-view prototype loss The calculation formula is as follows: Among them, L i =CE(r i ); In the formula, is the prototype feature corresponding to each point cloud block in the target point cloud sample generated by the expanded attention module; n is the number of point cloud blocks; r i is the prediction result corresponding to each prototype feature obtained by the predictor Pred; L i is the cross entropy loss corresponding to each prediction result.

7. The point cloud semantic segmentation method based on dilated attention and multi-prototype fusion as claimed in claim 1, It is characterized in that The step of "performing semantic segmentation on point cloud data based on the trained neural network" specifically includes the following steps: Obtain query sample features and prototype features after point cloud data is processed by the trained neural network; Calculate query sample feature x p With prototype feature y f Gaussian similarity between them for semantic segmentation; The calculation formula of the Gaussian similarity is as follows: similarity(x p ,y f )=exp(-||x p -y f ||^2 / 2σ^2); Where σ is the standard deviation of the Gaussian distribution.

8. The point cloud semantic segmentation method based on dilated attention and multi-prototype fusion as claimed in claim 1, It is characterized in that The step of "using the point cloud samples to be processed to train and obtain a pre-trained model" specifically includes the following steps: The point cloud samples to be processed are divided into a support set and a query set, and the DGCNN backbone network is trained using the support set and the query set to obtain a pre-trained model.

9. A point cloud semantic segmentation system based on dilated attention and multi-prototype fusion, It is characterized in that include: A pre-training module is used to obtain a pre-training model by training with the point cloud samples to be processed; A screening module is used to screen the point cloud samples to be processed to obtain target point cloud samples; A feature extraction module, which is in communication with the pre-training module and the screening module, and is used to extract features from the target point cloud sample using the pre-training model to obtain sample feature data; an extended attention training module, which is in communication with the feature extraction module and is used to build an extended attention block model, train the extended attention block model based on multi-view prototype loss and the sample feature data, and obtain a trained neural network; and The semantic segmentation module is communicatively connected to the dilated attention training module and is used to perform semantic segmentation on the point cloud data based on the trained neural network.

10. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the point cloud semantic segmentation method based on dilated attention and multi-prototype fusion is implemented as described in any one of claims 1 to 8.

Images