Point cloud classification segmentation method and system of global mask auto-encoder based on fused voxel

Abstract

The invention discloses a fusion voxel-based global mask auto-encoder point cloud classification segmentation method and system, and belongs to the technical field of three-dimensional point semantic segmentation and deep learning, and the method comprises the steps: obtaining and processing a non-labeled point cloud data set and a labeled point cloud data set, constructing a fusion voxel-based global mask auto-encoder model, and obtaining a fusion voxel-based global mask auto-encoder model; parameters obtained through pre-training are migrated, a downstream task model is constructed, then the model is finely adjusted through a labeled data set, and an optimal deep network model is obtained and used for classification or segmentation of three-dimensional point clouds. According to the method, a self-supervised learning method is adopted, the problem of insufficient mark data sets is solved to a certain extent, the problem of low efficiency of a model caused by dividing local blocks based on a mask auto-encoder in the past is effectively improved, no farthest point sampling and K-nearest neighbor operation exist, global voxel features are used as prediction targets, and the prediction efficiency is improved. And the problem of information redundancy caused by block division is also avoided.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional point semantic segmentation and deep learning, and in particular to a point cloud classification and segmentation method and system based on a global mask autoencoder with fused voxels. Background Art

[0002] 3D point cloud classification is an important research direction in the field of computer vision and graphics, which mainly involves analyzing and processing point cloud data in 3D space to identify and classify objects or scenes. 3D point clouds can be obtained through various sensors such as lidar, and usually contain position information and normal vector information. 3D point cloud classification is a complex and challenging task involving a variety of technologies and methods. It has promoted the progress of autonomous driving, robotics, urban modeling and other fields. In the future, point cloud classification technology will play an important role in a wider range of application scenarios. Designing an effective point cloud classification network is still of great significance.

[0003] Supervised learning has demonstrated its excellent feature learning capabilities in point cloud processing. Supervised learning methods can be roughly divided into voxel-based, multi-view-based and point-based methods according to the different types of features extracted. The voxel-based method divides the point cloud into a regular voxel grid, and then uses deep learning technology to extract voxel features; the multi-view-based method projects the three-dimensional shape onto multiple two-dimensional planes, and then extracts information on these two-dimensional planes. Voxel-based and multi-view-based methods lose a lot of fine geometric structure information during the voxelization and projection process, resulting in limited classification capabilities of the model. The point-based method directly performs feature extraction operations on the point cloud, avoiding these problems. However, supervised learning increases the opportunity for performance improvement by learning labeled data in the task, but a large number of manually labeled data sets are expensive. To address this problem, the present invention adopts a self-supervised learning method to learn point cloud features for point cloud segmentation.

[0004] Self-supervised learning performs a pre-task on an unlabeled dataset to learn a general feature representation, and then performs downstream tasks on a smaller labeled dataset and fine-tunes. Self-supervised learning algorithms solve the problem of insufficient labeled datasets to a certain extent. Among the self-supervised learning methods, masked autoencoders have shown their broad application prospects in fields such as text understanding and image understanding, and many studies have also proved the effectiveness of this scheme in point cloud data learning. Most point cloud self-supervised learning models based on masked autoencoder methods usually divide the point cloud into multiple local area features, mask a part of them, and then use the unmasked area features to learn high-level potential features to predict the masked information. However, the division of local areas mostly uses the farthest point sampling and K nearest neighbor methods, which requires a lot of time. In addition, in order to make the local area completely cover the entire point cloud shape, the local neighborhood divided by the K nearest neighbor will inevitably contain redundant information. In response to this problem, the present invention constructs a global masked autoencoder based on voxels to perform point cloud self-supervised learning. It uses the global point cloud shape of different voxel resolutions as features, and then uses the autoencoder with the standard Transformer as the backbone to predict the masked voxel features of different resolutions from the visible features. Based on voxels, a global mask autoencoder is proposed to directly voxelize the point cloud. There is no time consumption caused by the farthest point sampling and K nearest neighbor operation, and each point in the point cloud will only be divided into one voxel block, and no information redundancy will be generated.

[0005] Through the above analysis, the problems and defects of the existing technology are as follows: the supervised learning method requires a large amount of manually labeled data sets to improve model performance, and a large amount of manually labeled data sets are very expensive; the point cloud self-supervised learning method based on masked autoencoder will consume a lot of time when dividing local blocks, resulting in a decrease in model efficiency; the point cloud self-supervised learning method based on masked autoencoder will inevitably contain redundant information when dividing local blocks.

[0006] Based on this, the present invention designs a point cloud classification and segmentation method and system based on a global mask autoencoder with fused voxels to solve the above problems. Summary of the invention

[0007] The purpose of the present invention is to solve the problems that supervised learning methods require a large number of manually labeled data sets to improve model performance, and a large number of manually labeled data sets are very expensive; the point cloud self-supervised learning method based on masked autoencoder will consume a lot of time when dividing local blocks, resulting in a decrease in model efficiency; the point cloud self-supervised learning method based on masked autoencoder will inevitably contain redundant information when dividing local blocks, and a point cloud classification and segmentation method and system based on a global masked autoencoder with fused voxels are proposed.

[0008] In order to achieve the above object, the present invention adopts the following technical solutions: A point cloud classification and segmentation method based on a global mask autoencoder with fused voxels, comprising: Step 1: Obtain two point cloud datasets, where the unlabeled dataset is used for pre-training self-supervised learning, and the labeled dataset is divided into a training set, a validation set, and a test set according to a certain ratio for fine-tuning downstream tasks, and the point cloud dataset is preprocessed; Step 2: construct a multi-scale voxel feature generation module and a global mask autoencoder model based on fused voxels; Step 3: Use an unlabeled dataset to pre-train the fused voxel-based global mask autoencoder model, and select the network model parameters with the goal of minimizing the reconstruction loss of the previous task; Step 4: Based on the pre-trained model, build a point cloud deep learning network for downstream tasks; Step 5: Transfer the parameters learned by the pre-trained model in step 3 to the point cloud deep learning network for downstream tasks, and use the labeled dataset for fine-tuning, where the training set is used to train the network, and the validation set is used during the training process to obtain the optimal deep network model; Step 6: Input the test set in the labeled data set into the selected optimal network model to obtain the point cloud classification result.

[0009] As a further description of the above technical solution: The preprocessing of the point cloud data set in step 1 includes: downsampling each point cloud in the point cloud data set, and normalizing and standardizing the sampled point coordinates.

[0010] As a further description of the above technical solution: The step 2 of constructing a multi-scale voxel feature generation module and constructing a global mask autoencoder model based on fused voxels includes: (1) Construct a multi-scale voxel feature generation module; The generation of multi-scale voxel features includes the following steps: point cloud shallow feature extraction, voxelization, channel expansion, shape reshaping and feature embedding; (2) Construct a global mask autoencoder model based on fused voxels; The fused voxel-based global mask autoencoder uses an asymmetric encoder-decoder structure with global point cloud shapes of different voxel resolutions as the encoder input.

[0011] As a further description of the above technical solution: In the step 3, the global mask autoencoder model based on fused voxels is pre-trained using an unlabeled dataset, and the network model parameters are selected with the goal of minimizing the reconstruction loss of the pre-task, including: The model is trained using an unlabeled dataset and fed into the network. The predicted mask voxel features are used as the prediction target to supervise the learning of the model.

[0012] As a further description of the above technical solution: The step 4 of constructing a point cloud deep learning network for downstream tasks based on the pre-trained model includes: The decoder part of the global mask autoencoder based on fused voxels is discarded, and the high-level latent features extracted by the encoder are concatenated, followed by a maximum pooling layer, and then followed by 5 linear layers. Each linear layer uses normalization, Swish activation function, and random dropout connections, thereby forming a point cloud deep learning network for downstream classification tasks.

[0013] As a further description of the above technical solution: The step 5 of transferring the parameters learned by the pre-trained model in step 3 to the point cloud deep learning network for downstream tasks, and fine-tuning using a labeled data set, wherein the network is trained using a training set, and the validation set is used during the training process to obtain the optimal deep network model, including: (1) Migrate the parameters learned by the pre-trained model to the point cloud deep learning network for downstream tasks; (2) Use a labeled dataset for fine-tuning, input the training set into the network, calculate the cross entropy loss, and update the network parameters; (3) Use the validation set to test and select the optimal network model; The validation set is input into the model, and data processing includes point cloud downsampling, normalization and standardization, and then the predicted labels are obtained. The predicted labels are compared with the actual labels to calculate the validation set accuracy, and the optimal model on the validation set during the training process is saved.

[0014] As a further description of the above technical solution: In step 6, the test set in the labeled data set is input into the selected optimal network model to obtain the point cloud classification result, including: (1) Input the test set into the model selected in step 5. Data processing includes point cloud downsampling, normalization, and standardization, and then obtain the predicted label. (2) Calculate the accuracy of the test set.

[0015] As a further description of the above technical solution: A point cloud classification system based on a voxel-fused global mask autoencoder includes: The 3D point cloud preprocessing module is used to preprocess the data set, downsample the labeled and unlabeled point cloud data, normalize and standardize them, and divide the unlabeled data set into training set, validation set, and test set according to a certain ratio; Point cloud pre-training model building module, used to build a multi-scale voxel feature generation module and a global mask autoencoder model based on fused voxels; The point cloud pre-training model pre-training module uses an unlabeled dataset to pre-train the global mask autoencoder model based on fused voxels, and selects the network model parameters with the goal of minimizing the reconstruction loss of the previous task; Point cloud classification network building module, based on the pre-trained model, builds a point cloud deep learning network for downstream tasks; The point cloud classification network training module is used to transfer the parameters learned by the pre-trained model to the point cloud deep learning network for downstream tasks, and fine-tune it using a labeled dataset. The training set is used to train the network, and the validation set is used during the training process to obtain the optimal deep network model. The point cloud classification module is used to input the test set in the labeled data set into the selected optimal network model to obtain the point cloud classification result.

[0016] As a further description of the above technical solution: A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of a point cloud classification method based on a global mask autoencoder with fused voxels as described in any one of the above.

[0017] As a further description of the above technical solution: A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the steps of a point cloud classification method based on a global mask autoencoder with fused voxels as described in any one of the above.

[0018] As a further description of the above technical solution: An information data processing terminal is used to implement a point cloud classification system based on a global mask autoencoder with fused voxels as described above.

[0019] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: The present invention provides a point cloud classification method based on a global mask autoencoder with fused voxels. The method adopts a self-supervised learning method to solve the problem of insufficient labeled data sets to a certain extent, and effectively improves the problem of inefficiency of the model caused by dividing local blocks based on the mask autoencoder in the past. It does not have any farthest point sampling and K nearest neighbor operations, but uses global voxel features as prediction targets, and also avoids the problem of information redundancy caused by block division.

[0020] The present invention improves the problem of model inefficiency and information redundancy caused by dividing local blocks based on masked autoencoders. The present invention does not use any farthest point sampling and K-nearest neighbor operations, and voxelizes the point cloud to avoid the problem of local block information redundancy, making the model execution efficiency higher and the classification accuracy of the point cloud higher.

[0021] The present invention aims to improve the problems of insufficient labeled data sets and model inefficiency and information redundancy caused by dividing local blocks based on masked autoencoders. It will bring many expected benefits and commercial value. When the classification accuracy and reliability are equivalent, fewer labeled data sets are required, which can save a lot of manpower and material resources, and is more conducive to the advancement of technology in the field of computer vision, and promote innovation and development in the fields of autonomous driving, robotics, urban modeling, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A point cloud classification and segmentation method based on a global masked autoencoder with fused voxels and a schematic diagram of a point cloud classification method of the system proposed by the present invention; Figure 2 A schematic diagram of the network structure of a point cloud classification and segmentation method and system based on a global masked autoencoder with fused voxels proposed by the present invention; Figure 3 A point cloud classification and segmentation method based on a global masked autoencoder with fused voxels and a channel expansion module principle of the system proposed by the present invention; Figure 4 This is a system flow chart of a point cloud classification and segmentation method and system based on a global mask autoencoder with fused voxels proposed by the present invention. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] Please refer to the attached Figure 1 -Attached Figure 4The present invention provides a technical solution: a point cloud classification and segmentation method based on a global mask autoencoder with fused voxels, comprising: Step 1: Obtain two point cloud datasets, where the unlabeled dataset is used for pre-training self-supervised learning, and the labeled dataset is divided into a training set, a validation set, and a test set according to a certain ratio for fine-tuning downstream tasks, and the point cloud dataset is preprocessed; Step 2: construct a multi-scale voxel feature generation module and a global mask autoencoder model based on fused voxels; Step 3: Use an unlabeled dataset to pre-train the fused voxel-based global mask autoencoder model, and select the network model parameters with the goal of minimizing the reconstruction loss of the previous task; Step 4: Based on the pre-trained model, build a point cloud deep learning network for downstream tasks; Step 5: Transfer the parameters learned by the pre-trained model in step 3 to the point cloud deep learning network for downstream tasks, and use the labeled dataset for fine-tuning, where the training set is used to train the network, and the validation set is used during the training process to obtain the optimal deep network model; Step 6: Input the test set in the labeled data set into the selected optimal network model to obtain the point cloud classification result.

[0025] As a preferred embodiment, Figure 1 As shown, a point cloud classification method based on a global mask autoencoder with fused voxels provided by an embodiment of the present invention specifically includes the following steps: (1) Preprocessing of point cloud dataset includes: Each point cloud in the point cloud dataset is downsampled, and the sampled point coordinates are normalized and standardized.

[0026] (2) Construct a multi-scale voxel feature generation module and a global mask autoencoder model based on fused voxels. Figure 2 ,include: (2.1) Construct a multi-scale voxel feature generation module; The generation of multi-scale voxel features includes the following steps: point cloud shallow feature extraction, voxelization, channel expansion, shape reshaping and feature embedding.

[0027] (2.1.1) Point cloud shallow feature extraction. For an input point cloud containing n points , before voxelization, high-dimensional point cloud features need to be extracted first, so that voxel features can be obtained during the voxelization process. A simple MiniPointNet is used to extract shallow features of point clouds. The MiniPointNet consists of two MLPs and a maximum pooling layer. Each MLP consists of a one-dimensional convolution, batch normalization, a nonlinear activation function, and a one-dimensional convolution in order.

[0028] Specifically, the first MLP is used to extract 256-dimensional point cloud features from the 3D point cloud , and then the global features of the point cloud are obtained through maximum pooling : ; ; mlp1 is the first MLP, and MaxPool represents the maximum pooling. and global features Splicing, and then passing through the second MLP to obtain the shallow features of the c-dimensional point cloud containing local and global information : ; Where mlp2 is the second MLP, is the feature concatenation function.

[0029] (2.1.2) Voxelization. After obtaining the shallow features of the point cloud, the point cloud is voxelized at different resolutions using the average voxelization method, and the features of the points within the voxel are averaged as the features of the voxel block.

[0030] First, we need to divide the point cloud into a regular voxel grid. Let the point cloud be represented as For convenience, we divide the point cloud space into r equal parts along the X, Y, and Z axes, so that the size of each cube grid is , and the dimension is The point cloud is then assigned to the corresponding voxel block according to its spatial coordinates, and the features of the points in each voxel block are aggregated as voxel features: ; in represents the i-th point cloud after transformation, represents the j-th voxel block, , Represents the features of the j-th voxel block; Indicates whether point a is within voxel block b. If so, it returns True, otherwise it returns False. The aggre operation averages the corresponding point features.

[0031] (2.1.3) Channel expansion. Directly voxelizing point clouds at different resolutions will result in different number of channels for features of different scales, which cannot be directly fed into the encoder composed of the Transformer. Therefore, it is necessary to expand the number of feature channels of the first three dimensions of voxel features of different scales to the highest voxel resolution. .

[0032] In order to keep the distribution shape of the original scale voxel block as much as possible, we evenly replicate a certain number of voxel blocks in each dimension to achieve channel expansion. Figure 3 As shown, if you want to change the dimension to The voxel grid is expanded to , which can be roughly divided into 5 situations.

[0033] For convenience, only one dimension is expanded during analysis, that is, r sequentially arranged voxel blocks (indicated by serial number as Expand to pcs. ,①If , then each voxel block is copied times. ,②When or When, starting from the first voxel block, each interval or Individual voxel blocks, copy the corresponding voxel blocks times, the remaining voxel blocks only replicate times; when ③If , then the first a number of pixel blocks are replicated times, the last voxel block is replicated Second, if , then starting from the first voxel block, each interval Individual voxel blocks, copy the corresponding voxel blocks times, the remaining voxel blocks only replicate Second-rate. ⑤When For simplicity, the first a blocks of pixels are copied times, the remaining voxel blocks are replicated Second-rate.

[0034] The above five expansion plans are arranged in order of priority from high to low.

[0035] (2.1.4) Shape reshaping and feature embedding. Suppose that the global voxel shapes of l different scales are sorted from small to large in voxel resolution as the feature sequence. The voxel features expanded to the same number of channels are denoted as To generate global voxel features of different scales, you need to first convert the voxel features Reshape , and then undergoes feature embedding consisting of 2D convolution, batch normalization, and Swish activation function. Multi-scale voxel features : ; Where reshape represents the shape reshaping operation, is a two-dimensional convolution, is batch normalization, is the Swish activation function.

[0036] (2.2) Construct a global mask autoencoder model based on fused voxels; The fused voxel-based global mask autoencoder uses an asymmetric encoder-decoder structure with global point cloud shapes of different voxel resolutions as the encoder input.

[0037] The encoder learns high-level latent features from visible features. It consists of multiple Transformer stacks (with e layers) to generate multi-scale voxel feature sequences. The visible part of as input. Through A random mask is performed to obtain, where m is the number of masked voxel features.

[0038] ; in , Represents the input of the i-th layer Transformer and the first layer Transformer , Latent features extracted for the encoder.

[0039] Compared to the encoder, the decoder consists of fewer Transformer stacks (with d layers), which transforms the encoder output and learnable mask features The concatenation is used as the input of the first layer Transformer.

[0040] ; in , . The output of the last layer of Transformer The predicted mask features can be obtained by separating the features of the last m voxels. ; (3) Use an unlabeled dataset to pre-train the voxel-based global mask autoencoder model. The network model parameters are selected with the goal of minimizing the reconstruction loss of the pre-task, including: Use the unlabeled dataset to train the model and input the unlabeled dataset into the network. Use the predicted mask voxel features as the prediction target to supervise the learning of the model. Let the predicted voxel features be , whose corresponding ground truth value is . Update the pre-trained model using the L1 loss function:

[0041] Where m is the number of masked voxel features. After a certain number of pre-training rounds, the model with the smallest loss is selected as the optimal pre-training model; (4) Based on the pre-trained model, a point cloud deep learning network for downstream tasks is constructed, including: The decoder part of the global mask autoencoder based on fused voxels is discarded, and the high-level latent features extracted by the encoder are concatenated, followed by a maximum pooling layer, and then followed by 5 linear layers. Each linear layer uses normalization, Swish activation function, and random dropout connections, thereby forming a point cloud deep learning network for downstream classification tasks.

[0042] (5) The parameters learned by the pre-trained model in (3) are transferred to the point cloud deep learning network for downstream tasks and fine-tuned using a labeled dataset. The training set is used to train the network, and the validation set is used during the training process to obtain the optimal deep network model. The following steps are performed: (5.1) Transfer the parameters learned by the pre-trained model to the point cloud deep learning network for downstream tasks; (5.2) Use a labeled dataset for fine-tuning, input the training set into the network, calculate the cross entropy loss, and update the network parameters; (5.3) Use the validation set to test and select the optimal network model; The validation set is input into the model, and data processing includes point cloud downsampling, normalization and standardization, and then the predicted labels are obtained. The predicted labels are compared with the actual labels to calculate the validation set accuracy, and the optimal model on the validation set during the training process is saved.

[0043] (6) Input the test set in the labeled data set into the selected optimal network model, and obtain the point cloud classification results including: (6.1) The test set is input into the model selected in step 5. Data processing includes point cloud downsampling, normalization and standardization, and then the predicted label is obtained.

[0044] (6.2) Calculate the accuracy of the test set.

[0045] A point cloud classification system based on a global mask autoencoder with fused voxels provided by an embodiment of the present invention includes: The 3D point cloud preprocessing module is used to preprocess the data set, downsample the labeled and unlabeled point cloud data, normalize and standardize them, and divide the unlabeled data set into training set, validation set, and test set according to a certain ratio; Point cloud pre-training model building module, used to build a multi-scale voxel feature generation module and a global mask autoencoder model based on fused voxels; The point cloud pre-training model pre-training module uses an unlabeled dataset to pre-train the global mask autoencoder model based on fused voxels, and selects the network model parameters with the goal of minimizing the reconstruction loss of the previous task; Point cloud classification network building module, based on the pre-trained model, builds a point cloud deep learning network for downstream tasks; The point cloud classification network training module is used to transfer the parameters learned by the pre-trained model to the point cloud deep learning network for downstream tasks, and fine-tune it using a labeled dataset. The training set is used to train the network, and the validation set is used during the training process to obtain the optimal deep network model. The point cloud classification module is used to input the test set in the labeled data set into the selected optimal network model to obtain the point cloud classification result.

[0046] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of a point cloud classification method based on a global mask autoencoder with fused voxels as described in any one of the above.

[0047] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the steps of a point cloud classification method based on a global mask autoencoder with fused voxels as described in any one of the above.

[0048] An information data processing terminal is used to implement a point cloud classification system based on a global mask autoencoder with fused voxels as described above.

[0049] In order to prove the creativity and technical value of the technical solution of the present invention, this section provides application examples of the claimed technical solution on specific products or related technologies.

[0050] The present invention is used as a main algorithm in a space object intelligent recognition system. The system receives unlabeled and labeled point cloud data sets, and sends the unlabeled data to a global masked autoencoder model based on fused voxels for pre-training to obtain the optimal pre-trained model. The pre-trained model parameters are then transferred to the point cloud deep learning network, and fine-tuned using a labeled data set to obtain a point cloud classification model, which is stored in the space object intelligent recognition system. The system also receives three-dimensional point cloud data as test input, and then sends the test data to the trained deep learning model to obtain a predicted output. Finally, the space object intelligent recognition system outputs the predicted category result of the point cloud.

[0051] As a preferred embodiment, the difference from the above is that: The present invention obtains two public point cloud datasets, which are unlabeled and labeled datasets. The unlabeled dataset is used for model pre-training, and it contains more than 500,000 three-dimensional shapes in 55 categories; the labeled dataset is used for downstream classification network fine-tuning, and it contains 12,311 point cloud files in 40 categories, of which 9,843 are used as training sets and 2,468 are used as test sets. The validation set is set to be consistent with the test set; Then the point cloud is preprocessed, 1024 points are randomly downsampled for each point cloud file in the data set as the model input, and the coordinates of all points in the point cloud in the point cloud data set are normalized and standardized so as to perform voxelization operation on the point cloud; As a preferred embodiment, the difference from the above is that: The present invention voxelizes the point cloud extracted from the shallow feature of the point cloud at voxel resolutions r=2, 3, ..., 11, and then expands the obtained voxel features of 10 different scales to the same number of channels and inputs them into the Transformer; As a preferred embodiment, the difference from the above is that: In the global masked autoencoder of the fused voxels, the present invention uses a 50% probability to randomly mask the multi-scale voxel feature sequence, that is, m=5. In addition, a 12-layer Transformer is used in the encoder and a 2-layer Transformer is used in the decoder; As a preferred embodiment, the difference from the above is that: The initial learning rate used in the pre-training of the present invention is 0.0001, the AdamW optimizer is used to optimize the model parameters, the weight decay rate is 0.05, and the cosine annealing algorithm is used to adjust the learning rate. 300 rounds of pre-training are performed with a batch size of 20; As a preferred embodiment, the difference from the above is that: When fine-tuning the downstream classification task, the present invention uses an initial learning rate of 0.0001, uses the AdamW optimizer to optimize the model parameters, the weight decay rate is 0.05, and uses the cosine annealing algorithm to adjust the learning rate. 300 rounds of pre-training are performed with a batch size of 20; The evaluation criteria for evaluating the point cloud classification method based on a global masked autoencoder with fused voxels provided by an embodiment of the present invention is the accuracy of the test set, and the overall accuracy rate is used as the evaluation index. The present invention is compared with three methods PointNet, CrossMoCo, and PointGrid; in the point cloud segmentation task, the present invention is compared with five methods PointNet, SO-Net, SPLATNet3D, DynamicScale, and Point-PlaneNet. As shown in Table 1, the experimental results show that compared with previous methods, the point cloud classification method based on a global masked autoencoder with fused voxels provided by an embodiment of the present invention has achieved better experimental results on the test set on the public point cloud dataset, indicating that the point cloud classification method of the present invention has better generalization ability and robustness.

[0052] Table 1 Comparative experimental results Method Name Overall accuracy (%) PointNet 89.2 CrossMoCo 91.5 PointGrid 92.0 DynamicScale 92.1 Point-PlaneNet 92.1 The present invention 92.3 It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0053] The above are only preferred specific implementation modes of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical solutions and inventive concepts of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A point cloud classification and segmentation method based on a global mask autoencoder with fused voxels, characterized in that: include: Step 1: Obtain two point cloud datasets, where the unlabeled dataset is used for pre-training self-supervised learning, and the labeled dataset is divided into a training set, a validation set, and a test set according to a certain ratio for fine-tuning downstream tasks, and the point cloud dataset is preprocessed; Step 2: construct a multi-scale voxel feature generation module and a global mask autoencoder model based on fused voxels; Step 3: Use an unlabeled dataset to pre-train the fused voxel-based global mask autoencoder model, and select the network model parameters with the goal of minimizing the reconstruction loss of the previous task; Step 4: Based on the pre-trained model, build a point cloud deep learning network for downstream tasks; Step 5: Transfer the parameters learned by the pre-trained model in step 3 to the point cloud deep learning network for downstream tasks, and use the labeled dataset for fine-tuning, where the training set is used to train the network, and the validation set is used during the training process to obtain the optimal deep network model; Step 6: Input the test set in the labeled data set into the selected optimal network model to obtain the point cloud classification result.

2. The point cloud classification and segmentation method based on a global masked autoencoder with fused voxels according to claim 1, characterized in that: The preprocessing of the point cloud data set in step 1 includes: downsampling each point cloud in the point cloud data set, and normalizing and standardizing the sampled point coordinates.

3. The point cloud classification and segmentation method based on a global mask autoencoder with fused voxels according to claim 1, characterized in that: The step 2 of constructing a multi-scale voxel feature generation module and constructing a global mask autoencoder model based on fused voxels includes: (1) Construct a multi-scale voxel feature generation module; The generation of multi-scale voxel features includes the following steps: point cloud shallow feature extraction, voxelization, channel expansion, shape reshaping and feature embedding; (2) Construct a global mask autoencoder model based on fused voxels; The fused voxel-based global mask autoencoder uses an asymmetric encoder-decoder structure with global point cloud shapes of different voxel resolutions as the encoder input.

4. The point cloud classification and segmentation method based on a global masked autoencoder with fused voxels according to claim 1, characterized in that: In the step 3, the global mask autoencoder model based on fused voxels is pre-trained using an unlabeled dataset, and the network model parameters are selected with the goal of minimizing the reconstruction loss of the pre-task, including: The model is trained using an unlabeled dataset and fed into the network. The predicted mask voxel features are used as the prediction target to supervise the learning of the model.

5. The point cloud classification and segmentation method based on a global mask autoencoder with fused voxels according to claim 1, characterized in that: The step 4 of constructing a point cloud deep learning network for downstream tasks based on the pre-trained model includes: The decoder part of the global mask autoencoder based on fused voxels is discarded, and the high-level latent features extracted by the encoder are concatenated, followed by a maximum pooling layer, and then followed by 5 linear layers. Each linear layer uses normalization, Swish activation function, and random dropout connections, thereby forming a point cloud deep learning network for downstream classification tasks.

6. The point cloud classification and segmentation method based on a global mask autoencoder with fused voxels according to claim 1, characterized in that: The step 5 of transferring the parameters learned by the pre-trained model in step 3 to the point cloud deep learning network for downstream tasks, and fine-tuning using a labeled data set, wherein the network is trained using a training set, and the validation set is used during the training process to obtain the optimal deep network model, including: (1) Migrate the parameters learned by the pre-trained model to the point cloud deep learning network for downstream tasks; (2) Use a labeled dataset for fine-tuning, input the training set into the network, calculate the cross entropy loss, and update the network parameters; (3) Use the validation set to test and select the optimal network model; The validation set is input into the model, and data processing includes point cloud downsampling, normalization and standardization, and then the predicted labels are obtained. The predicted labels are compared with the actual labels to calculate the validation set accuracy, and the optimal model on the validation set during the training process is saved.

7. The point cloud classification and segmentation method based on a global mask autoencoder with fused voxels according to claim 1, characterized in that: In step 6, the test set in the labeled data set is input into the selected optimal network model to obtain the point cloud classification result, including: (1) Input the test set into the model selected in step 5. Data processing includes point cloud downsampling, normalization, and standardization, and then obtain the predicted label. (2) Calculate the accuracy of the test set.

8. A point cloud classification system based on a global masked autoencoder with fused voxels according to any one of claims 1 to 7, characterized in that: A point cloud classification system based on a voxel-fused global mask autoencoder includes: The 3D point cloud preprocessing module is used to preprocess the data set, downsample the labeled and unlabeled point cloud data, normalize and standardize them, and divide the unlabeled data set into training set, validation set, and test set according to a certain ratio; Point cloud pre-training model building module, used to build a multi-scale voxel feature generation module and a global mask autoencoder model based on fused voxels; The point cloud pre-training model pre-training module uses an unlabeled dataset to pre-train the global mask autoencoder model based on fused voxels, and selects the network model parameters with the goal of minimizing the reconstruction loss of the previous task; Point cloud classification network building module, based on the pre-trained model, builds a point cloud deep learning network for downstream tasks; The point cloud classification network training module is used to transfer the parameters learned by the pre-trained model to the point cloud deep learning network for downstream tasks, and fine-tune it using a labeled dataset. The training set is used to train the network, and the validation set is used during the training process to obtain the optimal deep network model. The point cloud classification module is used to input the test set in the labeled data set into the selected optimal network model to obtain the point cloud classification result.

9. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of a point cloud classification method based on a global mask autoencoder with fused voxels as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of a point cloud classification method based on a global mask autoencoder with fused voxels as claimed in any one of claims 1 to 7; An information data processing terminal, used to implement the point cloud classification system based on the global mask autoencoder of fused voxels as described in claim 8.

Images