A point cloud classification segmentation method and system based on a fusion voxel global mask autoencoder

Abstract

The application discloses a point cloud classification and segmentation method and system based on a global mask autoencoder of fused voxels, belongs to the technical field of three-dimensional point semantic segmentation and deep learning, and comprises the following steps: acquiring and processing unlabeled and labeled point cloud datasets, constructing a global mask autoencoder model based on fused voxels, migrating parameters obtained through pre-training to construct a downstream task model, fine-tuning the model by using a labeled dataset to obtain an optimal deep network model, and then using the optimal deep network model for classification or segmentation of three-dimensional point clouds. The self-supervised learning method is adopted, the problem of insufficient labeled datasets is solved to a certain extent, and the problem of low efficiency of a model caused by previous division of local blocks based on a mask autoencoder is effectively improved. The method does not have any farthest point sampling and K nearest neighbor operation, but uses global voxel features as a prediction target, and the problem of information redundancy caused by block division is also avoided.

Description

Technical Field

[0001] This invention belongs to the field of 3D point semantic segmentation and deep learning technology, and particularly relates to a point cloud classification and segmentation method and system based on a global mask autoencoder with fused voxels. Background Technology

[0002] 3D point cloud classification is an important research direction in computer vision and graphics, mainly involving the analysis and processing of point cloud data in 3D space to identify and classify objects or scenes within it. 3D point clouds can be acquired through various sensors such as LiDAR, and typically contain positional and normal vector information. 3D point cloud classification is a complex and challenging task, involving multiple technologies and methods, and it has driven progress in fields such as autonomous driving, robotics, and urban modeling. In the future, point cloud classification technology will play a vital role in a wider range of applications. Designing an effective point cloud classification network remains of great significance.

[0003] Supervised learning has demonstrated excellent feature learning capabilities in point cloud processing. Supervised learning methods can be broadly categorized into voxel-based, multi-view-based, and point-based methods based on the type of features extracted. Voxel-based methods divide the point cloud into a regular voxel grid before using deep learning techniques to extract voxel features; multi-view-based methods project the 3D shape onto multiple 2D planes and then extract information from these planes. Voxel-based and multi-view-based methods lose a significant amount of fine geometric structure information during voxelization and projection, resulting in limited model classification capabilities. Point-based methods, on the other hand, directly extract features from the point cloud, avoiding these problems. However, supervised learning increases performance by learning from labeled data in the task, but large amounts of manually labeled datasets are expensive. To address this issue, this invention employs a self-supervised learning method to learn point cloud features for point cloud segmentation.

[0004] Self-supervised learning performs a pre-task on an unlabeled dataset to learn a general feature representation, and then performs downstream tasks on a smaller labeled dataset for fine-tuning. Self-supervised learning algorithms address the problem of insufficient labeled datasets to some extent. Among self-supervised learning methods, masked autoencoders have shown broad application prospects in text understanding and image understanding, and many studies have also demonstrated their effectiveness in point cloud data learning. Most point cloud self-supervised learning models based on masked autoencoders typically divide the point cloud into multiple local feature regions, mask a portion of them, and then use the features of the unmasked regions to learn high-level latent features to predict the masked information. However, the division of local regions mostly uses farthest point sampling and K-nearest neighbor methods, which requires a significant amount of time. Furthermore, in order for the local regions to completely cover the entire point cloud shape, the local neighborhoods divided by K-nearest neighbors inevitably contain redundant information. To address this problem, this invention constructs a voxel-based global masked autoencoder for point cloud self-supervised learning. It uses the global point cloud shape at different voxel resolutions as features, and then uses an autoencoder with a standard Transformer as its backbone to predict the masked voxel features at different resolutions from the visible features. Based on voxels, a global mask autoencoder is proposed to directly perform voxelization operations on point clouds. This eliminates the time consumption caused by farthest point sampling and K-nearest neighbor operations, and each point in the point cloud is divided into only one voxel block, thus avoiding information redundancy.

[0005] Based on the above analysis, the problems and shortcomings of the existing technology are as follows: supervised learning methods require a large amount of manually labeled datasets to improve model performance, but a large amount of manually labeled datasets are very expensive; the point cloud self-supervised learning method based on mask autoencoders consumes a lot of time when dividing local blocks, resulting in a decrease in model efficiency; the point cloud self-supervised learning method based on mask autoencoders inevitably contains redundant information when dividing local blocks.

[0006] Based on this, the present invention designs a point cloud classification and segmentation method and system based on a global mask autoencoder with fused voxels to solve the above problems. Summary of the Invention

[0007] The purpose of this invention is to address the problems of supervised learning methods requiring large amounts of manually labeled datasets to improve model performance, which are very expensive; point cloud self-supervised learning methods based on mask autoencoders consume a lot of time when dividing local blocks, leading to a decrease in model efficiency; and point cloud self-supervised learning methods based on mask autoencoders inevitably contain redundant information when dividing local blocks. Therefore, this invention proposes a point cloud classification and segmentation method and system based on a global mask autoencoder with fused voxels.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] A point cloud classification and segmentation method based on a global mask autoencoder with fused voxels includes:

[0010] Step 1: Obtain two types of point cloud datasets. The unlabeled dataset is used for pre-training self-supervised learning, while the labeled dataset is divided into training, validation, and test sets according to a certain ratio for fine-tuning downstream tasks. The point cloud dataset is also preprocessed.

[0011] Step 2: Construct a multi-scale voxel feature generation module and a global mask autoencoder model based on fused voxels;

[0012] Step 3: Pre-train the global mask autoencoder model based on fused voxels using an unlabeled dataset, and select network model parameters with the goal of minimizing the reconstruction loss of the previous task.

[0013] Step 4: Based on the pre-trained model, construct a point cloud deep learning network for downstream tasks;

[0014] Step 5: Transfer the parameters learned by the pre-trained model in Step 3 to the point cloud deep learning network for downstream tasks, and fine-tune it using a labeled dataset. The network is trained using the training set, and the optimal deep network model is obtained using the validation set during the training process.

[0015] Step 6: Input the test set from the labeled dataset into the selected optimal network model to obtain the point cloud classification results.

[0016] As a further description of the above technical solution:

[0017] The preprocessing of the point cloud dataset in step one includes: downsampling each point cloud in the point cloud dataset, and normalizing and standardizing the coordinates of the sampled points.

[0018] As a further description of the above technical solution:

[0019] Step two, which involves constructing a multi-scale voxel feature generation module and building a global mask autoencoder model based on fused voxels, includes:

[0020] (1) Construct a multi-scale voxel feature generation module;

[0021] The generation of multi-scale voxel features includes the following steps: shallow feature extraction of point cloud, voxelization, channel expansion, shape reshaping and feature embedding;

[0022] (2) Construct a global mask autoencoder model based on fused voxels;

[0023] The global mask autoencoder based on fused voxels uses an asymmetric encoder-decoder structure, with global point cloud shapes at different voxel resolutions as the encoder input.

[0024] As a further description of the above technical solution:

[0025] In step three, the unlabeled dataset is used to pre-train the global mask autoencoder model based on fused voxels. The network model parameters are selected with the goal of minimizing the reconstruction loss of the previous task.

[0026] The model is trained using an unlabeled dataset, which is then fed into the network. Predicting mask voxel features is used as the prediction target to encourage the model's learning.

[0027] As a further description of the above technical solution:

[0028] Step four, which involves constructing a point cloud deep learning network for downstream tasks based on a pre-trained model, includes:

[0029] The decoder part in the global mask autoencoder based on voxels is discarded. The high-level latent features extracted by the encoder are concatenated, followed by a max pooling layer, and then 5 linear layers. Each linear layer is normalized, has a Swish activation function, and random lossy connections, thus forming a point cloud deep learning network for downstream classification tasks.

[0030] As a further description of the above technical solution:

[0031] Step five involves transferring the parameters learned by the pre-trained model in step three to a point cloud deep learning network for downstream tasks, and fine-tuning it using a labeled dataset. This includes training the network using a training set and obtaining the optimal deep network model using a validation set during training.

[0032] (1) Transfer the parameters learned by the pre-trained model to the point cloud deep learning network used for downstream tasks;

[0033] (2) Fine-tune using a labeled dataset, input the training set into the network, calculate the cross-entropy loss, and update the network parameters;

[0034] (3) Use the validation set to test and select the optimal network model;

[0035] The validation set is input into the model, and data processing includes point cloud downsampling, normalization, and standardization to obtain predicted labels. The predicted labels are compared with the actual labels to calculate the validation set accuracy, and the best model on the validation set during training is saved.

[0036] As a further description of the above technical solution:

[0037] In step six, the selected optimal network model is input into the test set from the labeled dataset to obtain the point cloud classification results, including:

[0038] (1) Input the test set into the model selected in step five. Data processing includes point cloud downsampling, normalization and standardization, and then obtain the predicted labels;

[0039] (2) Calculate the accuracy of the test set.

[0040] As a further description of the above technical solution:

[0041] A point cloud classification system based on a global mask autoencoder with fused voxels includes:

[0042] The 3D point cloud preprocessing module is used to preprocess the dataset, downsample labeled and unlabeled point cloud data, normalize and standardize them, and divide the unlabeled dataset into training set, validation set and test set according to a certain ratio.

[0043] The point cloud pre-trained model building module is used to build a multi-scale voxel feature generation module and a global mask autoencoder model based on fused voxels.

[0044] The point cloud pre-training model pre-training module uses an unlabeled dataset to pre-train a global mask autoencoder model based on fused voxels, and selects network model parameters with the goal of minimizing the reconstruction loss of the previous task.

[0045] The point cloud classification network construction module, based on a pre-trained model, constructs a point cloud deep learning network for downstream tasks;

[0046] The point cloud classification network training module is used to transfer the parameters learned by the pre-trained model to the point cloud deep learning network for downstream tasks. It uses a labeled dataset for fine-tuning, where the network is trained using the training set and the optimal deep network model is obtained using the validation set during the training process.

[0047] The point cloud classification module is used to input the test set from the labeled dataset into the selected optimal network model to obtain the point cloud classification result.

[0048] As a further description of the above technical solution:

[0049] A computer device includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of a point cloud classification method based on a fused voxel-based global mask autoencoder as described in any of the preceding claims.

[0050] As a further description of the above technical solution:

[0051] A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of a point cloud classification method based on a fused voxel-based global mask autoencoder as described in any of the preceding claims.

[0052] As a further description of the above technical solution:

[0053] An information data processing terminal is provided for implementing a point cloud classification system based on a global mask autoencoder using fused voxels, as described above.

[0054] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0055] This invention provides a point cloud classification method based on a global mask autoencoder with fused voxels. It adopts a self-supervised learning method, which solves the problem of insufficient labeled dataset to a certain extent and effectively improves the inefficiency of the model caused by the division of local blocks in previous mask autoencoders. It does not have any farthest point sampling and K-nearest neighbor operations, but uses global voxel features as the prediction target, and also avoids the problem of information redundancy caused by block division.

[0056] This invention improves upon the problems of inefficiency and information redundancy caused by dividing local blocks based on mask autoencoders. This invention avoids the problem of local block information redundancy by voxelizing the point cloud without using any farthest point sampling and K-nearest neighbor operations, thus making the model more efficient and the point cloud classification accuracy higher.

[0057] This invention aims to improve the problems of insufficient labeled datasets and the inefficiency and information redundancy caused by the division of local blocks based on mask autoencoders. It will bring many expected benefits and commercial value. With comparable classification accuracy and reliability, it requires less labeled dataset, which can save a lot of human and material resources and is more conducive to the advancement of computer vision technology, promoting innovation and development in fields such as autonomous driving, robotics, and urban modeling. Attached Figure Description

[0058] Figure 1 This is a schematic diagram of the point cloud classification method and system based on a global mask autoencoder for voxel fusion proposed in this invention.

[0059] Figure 2 This is a schematic diagram of the network structure of a point cloud classification and segmentation method and system based on a global mask autoencoder with fused voxels proposed in this invention.

[0060] Figure 3This invention presents a point cloud classification and segmentation method and system based on a global mask autoencoder with fused voxels, and the principle of its channel extension module.

[0061] Figure 4 This invention presents a system flowchart of a point cloud classification and segmentation method and system based on a global mask autoencoder with fused voxels. Detailed Implementation

[0062] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0063] Please see the appendix Figure 1 -Appendix Figure 4 This invention provides a technical solution: a point cloud classification and segmentation method based on a global mask autoencoder with fused voxels, comprising:

[0064] Step 1: Obtain two types of point cloud datasets. The unlabeled dataset is used for pre-training self-supervised learning, while the labeled dataset is divided into training, validation, and test sets according to a certain ratio for fine-tuning downstream tasks. The point cloud dataset is also preprocessed.

[0065] Step 2: Construct a multi-scale voxel feature generation module and a global mask autoencoder model based on fused voxels;

[0066] Step 3: Pre-train the global mask autoencoder model based on fused voxels using an unlabeled dataset, and select network model parameters with the goal of minimizing the reconstruction loss of the previous task.

[0067] Step 4: Based on the pre-trained model, construct a point cloud deep learning network for downstream tasks;

[0068] Step 5: Transfer the parameters learned by the pre-trained model in Step 3 to the point cloud deep learning network for downstream tasks, and fine-tune it using a labeled dataset. The network is trained using the training set, and the optimal deep network model is obtained using the validation set during the training process.

[0069] Step 6: Input the test set from the labeled dataset into the selected optimal network model to obtain the point cloud classification results.

[0070] As a preferred embodiment, such as Figure 1 As shown, the point cloud classification method based on a global mask autoencoder using fused voxels provided in this embodiment of the invention specifically includes the following steps:

[0071] (1) Preprocessing of the point cloud dataset includes:

[0072] Each point in the point cloud dataset is downsampled, and the coordinates of the sampled points are normalized and standardized.

[0073] (2) Construct a multi-scale voxel feature generation module, and construct a global mask autoencoder model based on fused voxels, such as... Figure 2 ,include:

[0074] (2.1) Construct a multi-scale voxel feature generation module;

[0075] The generation of multi-scale voxel features includes the following steps: shallow feature extraction of point cloud, voxelization, channel expansion, shape reshaping and feature embedding.

[0076] (2.1.1) Shallow feature extraction of point cloud. For an input point cloud containing n points... Before voxelization, high-dimensional point cloud features need to be extracted to obtain voxel features during the voxelization process. A simple MiniPointNet is used to extract shallow features of the point cloud. This MiniPointNet consists of two MLPs and a max pooling layer. Each MLP is composed of one-dimensional convolution, batch normalization, non-linear activation function, and one-dimensional convolution in that order.

[0077] Specifically, the first MLP is used to extract 256-dimensional point cloud features from the 3D point cloud. Then, the global features of the point cloud are obtained through max pooling. :

[0078] ; ;

[0079] Where mlp1 is the first MLP, and MaxPool represents max pooling. The features of each point... and global features The data is then stitched together and processed by a second MLP to obtain shallow c-dimensional point cloud features containing both local and global information. :

[0080] ;

[0081] mlp2 is the second MLP. This is the feature splicing function.

[0082] (2.1.2) Voxelization. After obtaining the shallow features of the point cloud, the point cloud is voxelized at different resolutions using the average voxelization method, and the features of the points within the voxel are averaged as the features of the voxel block.

[0083] First, the point cloud needs to be divided into a regular voxel mesh. Let the point cloud be represented as... For convenience, we divide the point cloud space into r equal parts along the X, Y, and Z axes, making the size of each cube mesh... The dimension is obtained as The point cloud is then assigned to corresponding voxel blocks according to its spatial coordinates, and the features of points within each voxel block are aggregated as voxel features:

[0084] ;

[0085] in This represents the i-th point cloud after transformation. Represents the j-th voxel block. , Represents the characteristics of the j-th voxel block; This indicates whether point a is located within voxel block b; it returns True if it is, and False otherwise. The aggre operation averages the features of the corresponding point.

[0086] (2.1.3) Channel expansion. Directly converting point clouds into voxels at different resolutions results in different numbers of channels for features at different scales, making them unsuitable for direct input into an encoder composed of Transformers. Therefore, it is necessary to expand the number of feature channels in the first three dimensions of voxel features at different scales to the highest voxel resolution. .

[0087] To preserve the original scale voxel block distribution shape as much as possible, we uniformly replicate a certain number of voxel blocks in each dimension to achieve channel expansion. For example... Figure 3 As shown, if we want to make the dimension of voxel mesh expanded to There are roughly five situations.

[0088] For simplicity, the analysis will only expand by one dimension, namely r sequentially arranged voxel blocks (represented by their numbers). Expand to One. Order , ①If Then copy each voxel block. Next. If , ② when or At that time, starting from the first voxel block, every interval or Individual voxel blocks, copy the corresponding voxel block. Next, the remaining voxel blocks are only copied. times; when When, ③ if Then the first a voxel block is copied. Next, the last voxel block is copied. Next, ④ if Then, starting from the first voxel block, every interval Individual voxel blocks, copy the corresponding voxel block. Next, the remaining voxel blocks are only copied. Second-rate. ⑤When For simplicity, the first a voxel blocks are copied. Next, the remaining voxel blocks are copied. Second-rate.

[0089] The five expansion schemes mentioned above are listed in order of priority from highest to lowest.

[0090] (2.1.4) Shape Reshaping and Feature Embedding. Let l global voxel shapes at different scales be sorted from smallest to largest voxel resolution as the feature sequence. Let the voxel features expanded to the same number of channels be denoted as... To generate global voxel features at different scales, it is necessary to first process the voxel features... Reshape Then, it undergoes feature embedding consisting of 2D convolution, batch normalization, and the Swish activation function. Multi-scale voxel features :

[0091] ;

[0092] Where reshape represents the shape reshaping operation. It is a two-dimensional convolution. For batch normalization, This is the Swish activation function.

[0093] (2.2) Construct a global mask autoencoder model based on fused voxels;

[0094] The global mask autoencoder based on fused voxels uses an asymmetric encoder-decoder structure, with global point cloud shapes at different voxel resolutions as the encoder input.

[0095] The encoder learns high-level latent features from visible features. It consists of multiple stacked Transformers (with e layers) to produce multi-scale voxel feature sequences. Visible part As input. Through the The result is obtained by performing a random mask, where m is the number of voxel features that are masked.

[0096] ;

[0097] in , This represents the input of the i-th layer Transformer and the 1-th layer Transformer. , The latent features extracted by the encoder.

[0098] Compared to the encoder, the decoder consists of fewer Transformer stacks (with d layers), and it takes the output of the encoder. and learnable mask features The concatenation is used as input to the first layer of the Transformer.

[0099] ;

[0100] in , The output of the last Transformer layer. The predicted mask features can be obtained by separating the features of the last m voxels. ;

[0101] (3) The global mask autoencoder model based on fused voxels was pre-trained using an unlabeled dataset. The network model parameters were selected with the goal of minimizing the reconstruction loss of the previous task.

[0102] The model is trained using an unlabeled dataset, which is then input into the network. Predicting mask voxel features is used as the prediction target to encourage the model's learning. Let the predicted voxel features be... Its corresponding ground truth value Update the pre-trained model using the L1 loss function:

[0103]

[0104] Where m is the number of masked voxel features, and after a certain number of pre-training rounds, the model with the minimum loss is selected as the optimal pre-trained model.

[0105] (4) Based on the pre-trained model, the point cloud deep learning network for downstream tasks is constructed, including:

[0106] The decoder part in the global mask autoencoder based on voxels is discarded. The high-level latent features extracted by the encoder are concatenated, followed by a max pooling layer, and then 5 linear layers. Each linear layer is normalized, has a Swish activation function, and random lossy connections, thus forming a point cloud deep learning network for downstream classification tasks.

[0107] (5) Transfer the parameters learned by the pre-trained model in (3) to the point cloud deep learning network for downstream tasks, and fine-tune it using a labeled dataset. The network is trained using the training set, and the optimal deep network model is obtained using the validation set during the training process.

[0108] (5.1) Transfer the parameters learned by the pre-trained model to the point cloud deep learning network used for downstream tasks;

[0109] (5.2) Fine-tune using a labeled dataset. Input the training set into the network, calculate the cross-entropy loss, and update the network parameters.

[0110] (5.3) Test the network using the validation set and select the optimal network model;

[0111] The validation set is input into the model, and data processing includes point cloud downsampling, normalization, and standardization to obtain predicted labels. The predicted labels are compared with the actual labels to calculate the validation set accuracy, and the best model on the validation set during training is saved.

[0112] (6) Input the test set from the labeled dataset into the selected optimal network model to obtain the point cloud classification results, including:

[0113] (6.1) Input the test set into the model selected in step five. Data processing includes point cloud downsampling, normalization and standardization, and then obtain the predicted labels.

[0114] (6.2) Calculate the accuracy of the test set.

[0115] An embodiment of the present invention provides a point cloud classification system based on a global mask autoencoder using fused voxels, comprising:

[0116] The 3D point cloud preprocessing module is used to preprocess the dataset, downsample labeled and unlabeled point cloud data, normalize and standardize them, and divide the unlabeled dataset into training set, validation set and test set according to a certain ratio.

[0117] The point cloud pre-trained model building module is used to build a multi-scale voxel feature generation module and a global mask autoencoder model based on fused voxels.

[0118] The point cloud pre-training model pre-training module uses an unlabeled dataset to pre-train a global mask autoencoder model based on fused voxels, and selects network model parameters with the goal of minimizing the reconstruction loss of the previous task.

[0119] The point cloud classification network construction module, based on a pre-trained model, constructs a point cloud deep learning network for downstream tasks;

[0120] The point cloud classification network training module is used to transfer the parameters learned by the pre-trained model to the point cloud deep learning network for downstream tasks. It uses a labeled dataset for fine-tuning, where the network is trained using the training set and the optimal deep network model is obtained using the validation set during the training process.

[0121] The point cloud classification module is used to input the test set from the labeled dataset into the selected optimal network model to obtain the point cloud classification result.

[0122] A computer device includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of a point cloud classification method based on a fused voxel-based global mask autoencoder as described in any of the preceding claims.

[0123] A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of a point cloud classification method based on a fused voxel-based global mask autoencoder as described in any of the preceding claims.

[0124] An information data processing terminal is provided for implementing a point cloud classification system based on a global mask autoencoder using fused voxels, as described above.

[0125] To demonstrate the inventiveness and technical value of the technical solution of this invention, this section provides specific product or related technology application examples of the technical solution claimed.

[0126] This invention, as the main algorithm, is applied to a spatial object intelligent recognition system. This system receives unlabeled and labeled point cloud datasets. The unlabeled data is fed into a global mask autoencoder model based on voxel fusion for pre-training to obtain the optimal pre-trained model. Then, the parameters of the pre-trained model are transferred to a point cloud deep learning network and fine-tuned using a labeled dataset to obtain a point cloud classification model, which is stored in the spatial object intelligent recognition system. This system also receives 3D point cloud data as test input, and then feeds the test data into the trained deep learning model to obtain the prediction output. Finally, the spatial object intelligent recognition system outputs the predicted category result of the point cloud.

[0127] As a preferred embodiment, the difference from the above is:

[0128] This invention obtains two publicly available point cloud datasets: an unlabeled dataset and a labeled dataset. The unlabeled dataset, containing over 500,000 3D shapes across 55 categories, is used for model pre-training. The labeled dataset, containing 12,311 point cloud files across 40 categories, is used for fine-tuning the downstream classification network. Of these, 9,843 are used as the training set and 2,468 as the test set, with the validation set set consistent with the test set.

[0129] Then, the point cloud is preprocessed by randomly downsampling 1024 points from each point cloud file in the dataset as model input, and normalizing and standardizing the coordinates of all points in the point cloud dataset in order to perform voxelization on the point cloud.

[0130] As a preferred embodiment, the difference from the above is:

[0131] This invention voxelizes the point cloud after shallow feature extraction at voxel resolutions r=2,3,...,11 respectively, and then expands the obtained 10 voxel features at different scales to the same number of channels and inputs them into the Transformer.

[0132] As a preferred embodiment, the difference from the above is:

[0133] In this invention, a global mask autoencoder for fused voxels uses a 50% probability to randomly mask the multi-scale voxel feature sequence, i.e., m=5. In addition, a 12-layer Transformer is used in the encoder and a 2-layer Transformer is used in the decoder.

[0134] As a preferred embodiment, the difference from the above is:

[0135] The invention uses an initial learning rate of 0.0001 during pre-training, uses the AdamW optimizer to optimize model parameters, has a weight decay rate of 0.05, and uses a cosine annealing algorithm to adjust the learning rate. It performs 300 rounds of pre-training with a batch size of 20.

[0136] As a preferred embodiment, the difference from the above is:

[0137] In this invention, when fine-tuning the downstream classification task, the initial learning rate is 0.0001, the AdamW optimizer is used to optimize the model parameters, the weight decay rate is 0.05, and the cosine annealing algorithm is used to adjust the learning rate. 300 rounds of pre-training are performed with a batch size of 20.

[0138] The evaluation criterion for the point cloud classification method based on a fused voxel-based global mask autoencoder provided in this invention is the accuracy on the test set, using overall accuracy as the evaluation metric. This invention is compared with three methods: PointNet, CrossMoCo, and PointGrid; in point cloud segmentation tasks, it is compared with five methods: PointNet, SO-Net, SPLATNet3D, DynamicScale, and Point-PlaneNet. As shown in Table 1, experimental results demonstrate that, compared with previous methods, the point cloud classification method based on a fused voxel-based global mask autoencoder provided in this invention achieves better experimental results on the test set of a publicly available point cloud dataset, indicating that the point cloud classification method of this invention has good generalization ability and robustness.

[0139] Table 1 Comparison of experimental results

[0140] Method Name Overall accuracy (%) PointNet 89.2 CrossMoCo 91.5 PointGrid 92.0 DynamicScale 92.1 Point-PlaneNet 92.1 This invention 92.3

[0141] It should be noted that embodiments of the present invention can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuitry such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field-programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of the above-described hardware circuitry and software, such as firmware.

[0142] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A point cloud classification and segmentation method based on a global mask autoencoder with fused voxels, characterized in that, include: Step 1: Obtain two types of point cloud datasets. The unlabeled dataset is used for pre-training self-supervised learning, while the labeled dataset is divided into training, validation, and test sets according to a certain ratio for fine-tuning downstream tasks. The point cloud dataset is also preprocessed. Step 2: Construct a multi-scale voxel feature generation module and a global mask autoencoder model based on fused voxels; The second step involves constructing a multi-scale voxel feature generation module and constructing a global mask autoencoder model based on fused voxels, including: (1) constructing a multi-scale voxel feature generation module; The generation of multi-scale voxel features includes the following steps: shallow feature extraction of point cloud, voxelization, channel expansion, shape reshaping and feature embedding; (2) Construct a global mask autoencoder model based on fused voxels; The global mask autoencoder based on fused voxels uses an asymmetric encoder-decoder structure, with global point cloud shapes at different voxel resolutions as the input to the encoder. Step 3: Pre-train the global mask autoencoder model based on fused voxels using an unlabeled dataset, and select network model parameters with the goal of minimizing the reconstruction loss of the previous task. Step 4: Based on the pre-trained model, construct a point cloud deep learning network for downstream tasks; The step four, which involves constructing a point cloud deep learning network for downstream tasks based on a pre-trained model, includes: discarding the decoder part in a global mask autoencoder based on fused voxels, concatenating the high-level latent features extracted by the encoder, followed by a max pooling layer, and then five linear layers. Each linear layer is normalized, uses a Swish activation function, and has random lost connections, thereby forming a point cloud deep learning network for downstream classification tasks. Step 5: Transfer the parameters learned by the pre-trained model in Step 3 to the point cloud deep learning network for downstream tasks, and fine-tune it using a labeled dataset. The network is trained using the training set, and the optimal deep network model is obtained using the validation set during the training process. The fifth step involves transferring the parameters learned by the pre-trained model in the third step to the point cloud deep learning network used for downstream tasks, and fine-tuning it using a labeled dataset. The network is trained using a training set, and the optimal deep network model is obtained using a validation set during the training process. This includes: (1) transferring the parameters learned by the pre-trained model to the point cloud deep learning network used for downstream tasks. (2) Fine-tune using a labeled dataset, input the training set into the network, calculate the cross-entropy loss, and update the network parameters; (3) Test the network model using the validation set and select the optimal network model; The validation set is input into the model, and the data processing includes point cloud downsampling, normalization, and standardization, and then the predicted labels are obtained. Compare the predicted labels with the actual labels to calculate the accuracy on the validation set, and save the best model on the validation set during the training process. Step 6: Input the test set from the labeled dataset into the selected optimal network model to obtain the point cloud classification results.

2. The point cloud classification and segmentation method based on a global mask autoencoder using fused voxels as described in claim 1, characterized in that, The preprocessing of the point cloud dataset in step one includes: downsampling each point cloud in the point cloud dataset, and normalizing and standardizing the coordinates of the sampled points.

3. The point cloud classification and segmentation method based on a global mask autoencoder using fused voxels as described in claim 1, characterized in that, Step three involves pre-training the global mask autoencoder model based on fused voxels using an unlabeled dataset. The network model parameters are selected with the goal of minimizing the reconstruction loss of the preceding task. This includes training the model using an unlabeled dataset and inputting the unlabeled dataset into the network. The prediction of mask voxel features is used as the prediction target to encourage the model's learning.

4. The point cloud classification and segmentation method based on a global mask autoencoder using fused voxels as described in claim 1, characterized in that, The step six of inputting the test set from the labeled dataset into the selected optimal network model to obtain the point cloud classification result includes: (1) inputting the test set into the model selected in step five, data processing including point cloud downsampling, normalization and standardization, and then obtaining the predicted label; (2) Calculate the accuracy of the test set.

5. A point cloud classification system based on a global mask autoencoder based on voxels, as described in any one of claims 1 to 4, characterized in that, A point cloud classification system based on a global mask autoencoder with fused voxels includes: a 3D point cloud preprocessing module, which is used to preprocess the dataset, downsample labeled and unlabeled point cloud data, normalize and standardize them, and divide the unlabeled dataset into training set, validation set and test set according to a certain ratio; The point cloud pre-trained model building module is used to build a multi-scale voxel feature generation module and a global mask autoencoder model based on fused voxels. The point cloud pre-training model pre-training module uses an unlabeled dataset to pre-train a global mask autoencoder model based on fused voxels, and selects network model parameters with the goal of minimizing the reconstruction loss of the previous task. The point cloud classification network construction module, based on a pre-trained model, constructs a point cloud deep learning network for downstream tasks; The point cloud classification network training module is used to transfer the parameters learned by the pre-trained model to the point cloud deep learning network for downstream tasks. It uses a labeled dataset for fine-tuning, where the network is trained using the training set and the optimal deep network model is obtained using the validation set during the training process. The point cloud classification module is used to input the test set from the labeled dataset into the selected optimal network model to obtain the point cloud classification result.

6. A computer device, characterized in that, The computer device includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of a point cloud classification method based on a fused voxel global mask autoencoder as described in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of a point cloud classification method based on a fused voxel-based global mask autoencoder as described in any one of claims 1 to 4.

Images