A point cloud self-supervised classification method and system based on a multi-view mask autoencoder

Abstract

The application discloses a point cloud self-supervised classification method and system based on a multi-view mask autoencoder, and belongs to the technical field of computer vision, and comprises the following steps: S101, three-dimensional point cloud data sets are acquired, the three-dimensional point cloud data sets are divided according to training, verification and testing, and a sampling technology is used to pre-process the three-dimensional point cloud; S102, a self-supervised deep learning network combined with a multi-view technology is trained with training data, and is optimized through a verification data set to obtain an optimal deep learning model; and S103, the optimal-effect deep learning network model is used to test and verify the test set and the verification set. The application adopts a point cloud multi-view self-supervised classification based on deep learning, the network converts the point cloud into multiple views, integrates the multiple views together by using scene and viewpoint information, maximally reduces projection loss, and uses a mask autoencoder to perform mask reconstruction on the combined view, thereby enhancing the generalization ability of the model.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer vision, and particularly relates to a point cloud self-supervised classification method and system based on a multi-view mask autoencoder. BACKGROUND

[0002] Point cloud is a three-dimensional data format composed of a set of discrete points. This format can accurately capture the geometric details of objects and also contain additional information such as color and normal, providing highly accurate data. The progress of technology and hardware has greatly facilitated the acquisition of point clouds, making them increasingly the preferred data format in various industries, including medical and other professional fields. As the application grows, the demand for accurate point cloud processing models is also growing. However, the rapid development of model complexity and parameter size has already exceeded the growth rate of available labeled data sets. Annotating large point cloud data sets, which often contain millions of points, is both time-consuming and labor-intensive.

[0003] Self-supervised learning is an important research direction to solve this problem. By using unlabeled data for pre-training and using a small labeled data set for fine-tuning, self-supervised models can achieve comparable performance to supervised models while reducing the need for large amounts of labeled data. Unlike images, point cloud data is irregular and unordered, and multi-view methods can effectively address these challenges. These methods extract features from multiple perspectives, capturing global features and providing more comprehensive data analysis, which has shown its superiority in various tasks.

[0004] Compared with self-supervised models directly applied to point clouds, multi-view methods are less affected by data size and scale. They always learn features from a global perspective. In addition, multi-view technology connects different dimensional visual techniques by utilizing mature two-dimensional technology to process three-dimensional data, effectively preserving three-dimensional information when converting to two-dimensional images. In order to reduce the dependence of point cloud classification models on labeled data, it is of great theoretical research and practical application significance to find a self-supervised classification method for point clouds.

[0005] Through the above analysis, the problems and defects of the prior art are that the rapid development of model complexity and parameter size has already exceeded the growth rate of available labeled data sets. Annotating large point cloud data sets, which often contain millions of points, is both time-consuming and labor-intensive, but existing supervised methods rely on labeled data; comprehensive feature extraction for classification is still challenging because local feature learning methods are not always ideal, and the additional computational cost of segmenting point clouds is also a limiting factor; existing multi-view models are mainly supervised and heavily dependent on labeled data.

[0006] Based on this, the present application designs a point cloud self-supervised classification method and system based on a multi-view mask autoencoder to solve the above problems. SUMMARY

[0007] The purpose of the present application is to solve the problem that the rapid development of model complexity and parameter scale has exceeded the growth rate of available labeled data sets. It is both time-consuming and laborious to annotate large point cloud data sets, which contain millions of points, and existing supervised methods rely on labeled data; comprehensive feature extraction for classification is still challenging because local feature learning methods are not always ideal, and the additional computational cost of segmenting point clouds is a limiting factor; existing multi-view models are mainly supervised and heavily dependent on labeled data, and a point cloud self-supervised classification method and system based on a multi-view mask autoencoder are proposed.

[0008] To achieve the above purpose, the present application adopts the following technical solutions:

[0009] A point cloud self-supervised classification method and system based on a multi-view mask autoencoder, comprising the following steps:

[0010] S101, obtain a three-dimensional point cloud data set, divide the three-dimensional point cloud data set according to training, verification and testing, and use sampling technology to pre-process the three-dimensional point cloud;

[0011] S102, combine a self-supervised deep learning network with multi-view technology, and train with training data, optimize through a verification data set to obtain an optimal deep learning model;

[0012] S103, use the deep learning network model with optimal effect to test and verify the test set and the verification set, and output a three-dimensional point cloud classification result.

[0013] Further description of the above technical solutions:

[0014] The method of training the model for classifying point clouds in a self-supervised manner comprises the following steps:

[0015] Step one: use laser scanning or obtain a three-dimensional point cloud data set from a public point cloud database, divide the three-dimensional point cloud data set into three parts of training, verification and testing. The training set is used to train the deep learning model, the verification set is used to optimize the model parameters and select the best model, and the test set is used to evaluate the performance of the model to ensure its reliability in actual application. After dividing the data, we also need to use sampling technology to pre-process the three-dimensional point cloud to ensure the representativeness of the data set and improve the training effect and generalization ability of the model;

[0016] Step two: build a multi-view conversion network, a self-supervised agent task network and a downstream classification task fine-tuning network, and build a point cloud self-supervised classification method based on a multi-view mask autoencoder;

[0017] Step three: training on the self-supervised agent network using the training set; fine-tuning the trained model using the downstream classification task to adapt to the needs of the classification task, and using the validation set to verify the model performance during the training process and selecting the network model;

[0018] Step four: using the network to evaluate the performance on the test set and the validation set, and obtaining the classification results of the three-dimensional point cloud.

[0019] As a further description of the above technical solutions:

[0020] The preprocessing of the three-dimensional point cloud data set in step one includes:

[0021] Each three-dimensional point cloud object in the three-dimensional point cloud data set is sampled using the farthest point sampling method, and each point cloud contains 1024 data points for matching the input size of the deep learning network.

[0022] As a further description of the above technical solutions:

[0023] The construction of the multi-view conversion network, the self-supervised agent task network and the downstream classification task fine-tuning network in step two, and the construction of a point cloud self-supervised classification method based on multi-view mask autoencoder, includes:

[0024] (1) The point cloud is converted to multiple views and spliced into a whole through rendering;

[0025] For point cloud , virtual cameras are arranged around the point cloud at fixed intervals through pre-designed spherical model camera parameters, and the weighted average value of the points is calculated to determine the centroid of the point cloud, and the cameras are aligned to generate views of multiple perspectives;

[0026] (2) The masked reconstruction task is performed on the spliced picture through the mask autoencoder;

[0027] First, for a given input image, the image is evenly divided into image blocks according to the number of perspectives, and a part of the image blocks is randomly selected according to uniform distribution. 75% of the image blocks are masked. The remaining visible part is passed to the encoder of the model to obtain the features, and the decoder uses the features of the visible part to fill in the missing information to reconstruct the entire image, and optimizes itself by comparing the difference between the reconstructed image and the original image;

[0028] (3) Fine-tune the model on the classification task;

[0029] After the pre-training process with mask reconstruction, the encoder saves the rich feature representations extracted from the data. These feature representations are highly abstract and can capture the underlying structure and complex patterns of the data. The encoder is connected to a neural network designed for the point cloud classification task. Through supervised learning, the model continuously adjusts its parameters to minimize the gap between the predicted output and the actual label. During fine-tuning, the model learns the precise features required for the point cloud classification task while retaining the general features learned during pre-training, thereby improving the model's performance on the classification task.

[0030] As a further description of the above technical solutions:

[0031] In step three, the training set is used to train the self-supervised agent network; the fine-tuned network is fine-tuned to adapt to the needs of the classification task, and the validation set is used to verify the performance of the model during training and select the network model, including:

[0032] (1) First, import the training data set into the deep learning network for training, calculate the loss function and adjust the model parameters according to its value. Then, use the validation set to test and select the trained network model. In the validation phase, the data will bypass the sample weighting module and directly pass through the feature extractor and classifier to generate the predicted label;

[0033] (2) After obtaining the predicted label, compare it with the true label of the validation set to calculate the accuracy of the validation set.

[0034] As a further description of the above technical solutions:

[0035] In step four, the network is used to evaluate the performance on the test set and the validation set, and the classification results of the three-dimensional point cloud are obtained, including:

[0036] In step three, the selected model is used to verify the test set and the validation set, and the normalized and standardized data is output through the feature extractor and the classifier to output the predicted label, and the classification accuracy on the test set and the validation set is calculated.

[0037] As a further description of the above technical solutions:

[0038] A point cloud self-supervised classification system of a multi-view mask autoencoder includes:

[0039] A data preprocessing module is used to divide the data in the data set according to a specific proportion into a training set, a validation set and a test set;

[0040] A multi-view conversion module is used to convert the three-dimensional point cloud input into the system into images from multiple perspectives and splice the images together;

[0041] The self-supervised agent task module is used for the agent task of mask reconstruction, and the model learns the generalization features from the task, and improves the understanding ability of the model.

[0042] The downstream classification task fine-tuning module is used for further fine-tuning of the model on the downstream classification task, so that the model is more suitable for the classification task on the basis of the generalization ability learned by the agent task.

[0043] As a further description of the above technical solutions:

[0044] A point cloud self-supervised classification system of a multi-view mask autoencoder, comprising: a computer device, the computer device has a point cloud self-supervised classification system program of a multi-view mask autoencoder which can be run, and has a processor which can execute any one of the point cloud self-supervised classification methods based on the multi-view mask autoencoder according to claims 1-6.

[0045] As a further description of the above technical solutions:

[0046] A point cloud self-supervised classification system of a multi-view mask autoencoder, comprising: a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by a processor to make the processor execute any one of the point cloud self-supervised classification methods based on the multi-view mask autoencoder according to claims 1-6.

[0047] As a further description of the above technical solutions:

[0048] A point cloud self-supervised classification system of a multi-view mask autoencoder, comprising, an information data processing terminal, the information data processing terminal is used to realize the system.

[0049] As described above, due to the adoption of the above technical solutions, the beneficial effects of the present application are:

[0050] 1. The present application adopts a point cloud multi-view self-supervised classification based on deep learning, which converts point cloud into multiple views, integrates multiple views together using scene and viewpoint information, maximally reduces projection loss, and uses a mask autoencoder to reconstruct the combined view, thereby enhancing the generalization ability of the model.

[0051] 2. The point cloud multi-view self-supervised classification method based on deep learning provided by the present application can effectively improve the problems of high calculation cost and lack of global features of traditional methods, and through converting point cloud into multiple views and using a mask autoencoder for self-supervised learning, not only reduces the calculation cost, but also improves the global feature extraction ability and generalization ability of the model, and can maintain a relatively fast processing speed even in the case of high model complexity when processing complex data.

[0052] 3. The present application improves the problems of high calculation cost, strong dependence on labeled data and lack of global feature extraction capability of the existing point cloud processing method, can bring many expected benefits and business interlacing, improves the accuracy and calculation speed of point cloud classification, enables unmanned vehicles to judge the situation in front of them more quickly and make judgments, improves driving safety, and the model training is more convenient without labeled data;

[0053] 4. The present application improves the problems of dependence on labeled data of traditional supervised learning method, high calculation cost and lack of global features, and can train with unlabeled data, improve calculation cost and lack of global features, and maintain fast processing speed under high calculation complexity. BRIEF DESCRIPTION OF DRAWINGS

[0054] Figure 1 is a flowchart of a point cloud self-supervised classification method based on a multi-view mask autoencoder provided by the present application;

[0055] Figure 2 is a principle diagram of a point cloud self-supervised classification method based on a multi-view mask autoencoder provided by the present application;

[0056] Figure 3 is a general structure diagram of a neural network of a point cloud self-supervised classification method based on a multi-view mask autoencoder provided by the present application;

[0057] Figure 4 is a flowchart of rendering a three-dimensional point cloud into a complete multi-view image of a point cloud self-supervised classification method based on a multi-view mask autoencoder provided by the present application;

[0058] Figure 5 is a neural network pre-training structure diagram of a point cloud self-supervised classification method based on a multi-view mask autoencoder provided by the present application;

[0059] Figure 6 is a structure diagram of a neural network of a point cloud self-supervised classification method based on a multi-view mask autoencoder provided by the present application. DETAILED DESCRIPTION

[0060] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the present application.

[0061] Please refer to the drawingsFigure 1 -attach Figure 6 The application provides a technical solution: a point cloud self-supervised classification method and system based on a multi-view mask autoencoder, comprising the following steps:

[0062] S101, acquire a three-dimensional point cloud dataset, divide the three-dimensional point cloud dataset according to training, verification and testing, and use a sampling technique to pre-process the three-dimensional point cloud;

[0063] S102, combine a self-supervised deep learning network based on multi-view technology, and train using training data, and optimize through a verification dataset to obtain an optimal deep learning model;

[0064] S103, test and verify the optimal deep learning network model on a test set and a verification set, and output a three-dimensional point cloud classification result.

[0065] As a preferred embodiment, the following steps are included:

[0066] (1) The point cloud is converted into multiple views by rendering and spliced into a whole (see Figure 4 )

[0067] For the point cloud , a virtual camera is arranged at a fixed interval around the point cloud by using a pre-designed spherical model camera parameter, the weighted average value of the points is calculated to determine the centroid of the point cloud, and the camera is aligned to generate views of multiple perspectives.

[0068] (2) The spliced picture is subjected to a mask reconstruction task by a mask autoencoder (see Figure 5 )

[0069] First, for a given input image, the image is uniformly divided into image blocks according to the number of perspectives, a part is randomly extracted according to uniform distribution, and 75% of the image blocks are subjected to a mask operation. The remaining visible part is passed to the encoder of the model to obtain features, and the decoder uses the features of the visible part to fill in the missing information to reconstruct the entire image, and self-optimizes by comparing the difference between the reconstructed image and the original image.

[0070] (3) Fine-tune the model on the classification task (see Figure 6 )

[0071] After the pre-training process with mask reconstruction, the encoder will save the rich feature representation extracted from the data. These feature representations are highly abstract and can capture the underlying structure and complex patterns of the data. The encoder is connected to a neural network designed for the point cloud classification task. Through the method of supervised learning, the model continuously adjusts its parameters to minimize the gap between the predicted output and the actual label. During the fine-tuning process, the model learns the precise features required for the point cloud classification task while retaining the general features learned during pre-training, thereby improving the model's performance on the classification task.

[0072] The point cloud self-supervised classification system based on a multi-view mask autoencoder provided by the present application comprises:

[0073] The data preprocessing module is used to divide the data in the data set according to a specific proportion into a training set, a validation set and a test set.

[0074] The multi-view conversion module is used to convert the three-dimensional point cloud input into images from multiple perspectives and splice the images together.

[0075] The self-supervised agent task module is used to train the model to learn generalization features from the agent task with mask reconstruction, and improve the understanding ability of the model.

[0076] The downstream classification task fine-tuning module is used to further fine-tune the model on the downstream classification task, so that it is more suitable for the classification task based on the generalization ability learned from the agent task.

[0077] The present application includes a computer device, a processor, a computer readable storage medium and an information data processing terminal for executing the method.

[0078] In order to prove the creativity and technical value of the technical scheme of the present application, this part is an application embodiment of the technical scheme of the claim on a specific product or related technology.

[0079] As a preferred embodiment, as shown in Figure 2 The point cloud self-supervised classification method based on a multi-view mask autoencoder provided by the present application specifically comprises the following steps:

[0080] (1) Use laser scanning or obtain a three-dimensional point cloud data set from a public point cloud database, and divide the three-dimensional point cloud data set into three parts: training, validation and testing. The training set is used to train the deep learning model, the validation set is used to optimize the model parameters and select the best model, and the test set is used to evaluate the performance of the model to ensure its reliability in actual application. After dividing the data, we also need to use sampling technology to preprocess the three-dimensional point cloud to ensure the representativeness of the data set and improve the training effect and generalization ability of the model.

[0081] As a preferred embodiment, the present application obtains three published three-dimensional point cloud data sets. ShapeNet, ModelNet and ScanObjectNN.

[0082] ShapeNet is a training dataset for training the self-supervised proxy task module. ShapeNet is an annotated three-dimensional shape dataset containing 51,300 three-dimensional models of 55 common object types, widely used in many research fields such as shape retrieval and shape classification. For each instance, we use farthest point sampling to sample 1024 points to create input point clouds. Our initial learning rate is set to 0.001, and the weight decay is 0.05. The model is pre-trained for 300 batches with a batch size of 20. The results show that the model can reconstruct images with different mask rates. However, we set the default mask rate to 0.75.

[0083] ModelNet and ScanObjectNN are training, testing and validation datasets for downstream fine-tuning classification tasks. ModelNet40 is a widely used benchmark for evaluating point cloud processing, containing 12,311 models of 40 classes. We use the standard setting, where 9843 instances are used for training and 2468 instances are used for testing. The model is trained for 300 epochs using the AdamW optimizer with a batch size of 20. We set the initial learning rate to 0.0005 and use a cosine annealing scheme to adjust the learning rate over time. ScanObjectNN is a real-world point cloud dataset containing 2902 three-dimensional objects of 15 classes. The dataset contains background noise, occlusion and distortion, making it particularly challenging for point cloud processing.

[0084] The method of preprocessing three-dimensional point clouds is to sample each three-dimensional point cloud object in the three-dimensional point cloud dataset using the farthest point sampling method, each point cloud containing 1024 data points, matching the input size of the deep learning network.

[0085] (2) Constructing a multi-view conversion network, a self-supervised proxy task network and a downstream classification task fine-tuning network, and constructing a point cloud self-supervised classification method based on a multi-view mask autoencoder, specifically including:

[0086] The multi-view method is combined with the self-supervised method to design the model. Figure 1A specific procedure is shown. First, the point cloud is rendered into images from multiple viewpoints using preset camera parameters. Then, these images are arranged in a specific order and stitched into a comprehensive composite image containing all the viewpoints. This method maximizes the preservation of the three-dimensional structure and contour information of the point cloud in the images. Subsequently, each viewpoint is treated as a block, and a masked autoencoder strategy is used to train the model. This training reconstructs the images from the individual viewpoints to obtain the complete features.

[0087] (2.1) Building a multi-view conversion network

[0088] As shown in Figure 4 , the generation of point cloud multi-view images relies on preset camera parameters, and the process is as shown in Figure 2 . Common camera arrangements include a ring mode, in which rendered views are generated by placing virtual cameras at fixed positions. The camera is configured in a spherical mode, which can minimize the loss caused by the projection process and preserve the original features. First, the position of the centroid is determined using the weighted average of the point cloud. Then, according to the spherical ring setting, place planes at fixed angles. On each plane, place cameras at equal angles, all pointing to the centroid. Using a 3D point cloud or a mesh, input it into the rendering function to obtain m rendered images and stitch them together, as follows:

[0089]

[0090] where is the two-dimensional coordinate of the i-th viewpoint, is the camera intrinsic reference matrix of the i-th viewpoint, is the camera anomaly (rotation and translation) of the i-th viewpoint, represents the three-dimensional point coordinates in the global coordinate system. When the input is a mesh, a rasterizer converts the mesh into coordinates according to the viewpoint and assigns faces to pixels. The number of multi-view perspectives m and the generation process of the final fused image is as follows:

[0091]

[0092]

[0093] where denotes the stitched image, denotes the stitching operation, denotes the rendering operation.

[0094] As a preferred embodiment, the present application uses a value of 90° for α and a value of 16 for m.

[0095] (2.2) Building self-supervised agent task network

[0096] As shown in Figure 5 , the mask autoencoder uses the mask reconstruction principle to reconstruct the original image from the latent representation and mask label after partially masking the original image. This method enhances the model's ability to extract generalized features from the latent representation, as shown in Figure 3 .

[0097] Mask division. The image is divided according to different viewpoints. According to the number of viewpoints M, the input image is divided into m image blocks . Each viewpoint corresponds to a region, and a part is randomly extracted according to uniform distribution. Then mask 75% of the image segments, significantly reduce information redundancy by designing a higher mask rate, increase the difficulty of the reconstruction task, so that the model obtains stronger feature extraction ability in the training process.

[0098] Encoder. Only the visible part of the image is passed to the encoder for processing, and the position patches p are embedded through a linear projection layer, then they are passed to the Transformer, as shown in the formula below:

[0099]

[0100] Because only the visible part is processed, the memory and computational cost required to train the model is minimal. The input to the decoder is the complete information , which includes the visible part and the masked part , as shown in the formula below:

[0101]

[0102] Decoder. The working principle of the mask autoencoder is to reconstruct the input image and predict the pixel values of the hidden image blocks. Each output produced by the decoder is a vector representing the pixel values of an image block, as shown in the formula below:

[0103]

[0104] The key step for the decoder to reconstruct the complete image is to match the network output with the image pixels, and determine the loss by comparing with the original image pixel by pixel, as shown in the formula below:

[0105]

[0106] Where i represents the i-th pixel and N represents the total number of pixels. The mask autoencoder reconstructs the entire image by filling in the missing information and optimizes itself by comparing the difference between the reconstructed image and the original image.

[0107] As a preferred embodiment, the present invention uses a value of m of 16 and a value of N of 102400.

[0108] (2.3) Build a fine-tuning network for downstream classification tasks

[0109] like Figure 6 As shown in , after the pre-training process of mask reconstruction, the encoder will save the rich feature representation extracted from the data, the process is as follows Figure 4 As shown in the figure, these feature representations are highly abstract and can capture the underlying structure and complex patterns of the data. The encoder is connected to a neural network designed for point cloud classification. Through supervised learning, the model continuously adjusts its parameters to minimize the gap between the predicted output and the actual label. During fine-tuning, the model learns the precise features required for point cloud classification while retaining the general features learned during pre-training, thereby improving the model's performance on classification tasks.

[0110] (3) Use the training set to train the self-supervised agent network; use the downstream classification task to fine-tune the trained model to adapt to the needs of the classification task. During the training process, use the validation set to verify the model performance and select the network model, including:

[0111] (3.1) First, the training dataset is fed into the deep learning network for training. The loss function is calculated and the model parameters are adjusted accordingly. Next, the validation set is used to test and select the trained network model. During the validation phase, the data bypasses the sample weighting module and passes directly through the feature extractor and classifier to generate predicted labels.

[0112] (3.2) After obtaining the predicted labels, compare them with the true labels of the validation set to calculate the accuracy of the validation set.

[0113] (4) Use the network to evaluate performance on the test set and validation set, and obtain the classification results of the 3D point cloud, including:

[0114] (4.1) Use the test set and validation set for verification, pass the normalized and standardized data through the feature extractor and classifier to output the predicted labels, and calculate the classification accuracy on the test set and validation set.

[0115] The evaluation standard of the point cloud multi-view self-supervised classification method based on deep learning provided by the embodiment of the application is the average accuracy of the validation set, so as to obtain the prediction accuracy of the model for three-dimensional point cloud classification. The classification results on ModelNet40 and ScanObjectNN are listed in Tables 1 and 2. Compared with PointNet, the method improves by 0.9%, and achieves results comparable to MVCNN. In addition, we perform time tests on ModelNet40, as shown in Table 3. Although the computational complexity of our method is similar to that of MVCNN, and the number of model parameters is higher than that of other models, it maintains a faster processing speed.

[0116]

[0117]

[0118] MVMAE in the table is an abbreviation of a point cloud self-supervised classification method based on multi-view mask autoencoder;

[0119] The embodiments of the application can be completed by hardware, software or a combination of both. The hardware components can be constructed by special logic circuits; the software part can be stored in a storage device and operated by appropriate hardware such as microprocessor or specially designed hardware. Those skilled in the art can understand that the above device and its method can be realized by computer executable instructions or processor control code, which can be saved on storage media such as disk, CD or DVD-ROM, or provided in data media such as read-only memory, programmable memory, optical or electronic signal carrier. The device and its modules of the application can be realized by super large scale integrated circuit, gate array, logic chip, transistor and other semiconductor circuits, or by software executed by various types of processors, or by the combination of the above hardware circuit and software.

[0120] The above is only the preferred specific embodiment of the application, but the protection scope of the application is not limited to this. Any skilled person in the art can make equivalent replacement or change according to the technical solution and the inventive concept of the application within the technical range disclosed by the application, which should be covered in the protection scope of the application.

Claims

1. A point cloud self-supervised classification method based on multi-view mask autoencoder, characterized in that: The following steps are involved: Step 1: Use laser scanning or obtain a 3D point cloud dataset from a public point cloud database. Divide the 3D point cloud dataset into three parts: training, validation, and testing. The training set is used to train the deep learning model, the validation set is used to tune model parameters and select the best model, and the test set is used to evaluate the model's performance and ensure its reliability in practical applications. After the data is divided, the 3D point cloud is preprocessed using sampling technology to ensure the representativeness of the dataset and improve the training effect and generalization ability of the model. Step 2: Build a multi-view conversion network, a self-supervised proxy task network, and a downstream classification task fine-tuning network, and construct a point cloud self-supervised classification method based on a multi-view mask autoencoder, including: (1) Convert the point cloud into multiple views through rendering and stitch them into a whole; For the point cloud P, virtual cameras are arranged at fixed intervals around the point cloud using the pre-designed spherical model camera parameters. The weighted average of their points is then calculated to determine the center of mass of the point cloud, and the cameras are aligned to generate views from multiple perspectives. (2) Perform mask reconstruction on the spliced ​​image through the mask autoencoder; First, for a given input image, it is evenly divided into image blocks according to the number of viewpoints. A portion of the image blocks is randomly sampled according to a uniform distribution, and a mask operation is performed on 75% of the image blocks. The remaining visible portion is passed to the model's encoder to obtain features. The decoder uses the features of the visible portion to fill in the missing information and reconstruct the entire image. The decoder then performs self-optimization by comparing the difference between the reconstructed image and the original image. (3) Fine-tune the model on the classification task; After pre-training for mask reconstruction, the encoder saves rich feature representations extracted from the data. These feature representations are highly abstract and can capture the underlying structure and complex patterns of the data. The encoder is connected to a neural network designed for point cloud classification tasks. Through supervised learning, the model continuously adjusts its parameters to minimize the gap between the predicted output and the actual label. During fine-tuning, the model learns the precise features required for point cloud classification tasks while retaining the general features learned during pre-training, thereby improving the model's performance on classification tasks. Step 3: Use the training set to train the self-supervised agent network; use the downstream classification task to fine-tune the trained model to meet the needs of the classification task. During the training process, use the validation set to verify the model performance and select the network model; Step 4: Use the network to evaluate performance on the test set and validation set, and obtain the classification results of the 3D point cloud.

2. The point cloud self-supervised classification method based on multi-view mask autoencoder according to claim 1, characterized in that: The pre-processing of the three-dimensional point cloud dataset in step 1 includes: Each 3D point cloud object in the 3D point cloud dataset is sampled using the farthest point sampling method. Each point cloud contains 1024 data points to match the input size of the deep learning network.

3. The point cloud self-supervised classification method based on multi-view mask autoencoder according to claim 2, characterized in that: In step 3, the training set is used to train the self-supervised agent network; the trained model is fine-tuned using the downstream classification task fine-tuning network to adapt to the needs of the classification task. During the training process, the validation set is used to verify the model performance and select the network model, including: (1) First, the training dataset is imported into the deep learning network for training, the loss function is calculated and the model parameters are adjusted according to its value. Then, the validation set is used to test and select the trained network model. In the validation stage, the data will bypass the sample weighting module and directly pass through the feature extractor and classifier to generate the predicted label; (2) After obtaining the predicted labels, they are compared with the true labels of the validation set to calculate the accuracy of the validation set.

4. The point cloud self-supervised classification method based on multi-view mask autoencoder according to claim 2, characterized in that: In step 4, the network is used to perform performance evaluation on the test set and the validation set, and the classification results of the three-dimensional point cloud are obtained, including: The model selected in step 3 is verified using the test set and validation set. The normalized and standardized data is passed through the feature extractor and classifier to output the predicted labels, and the classification accuracy on the test set and validation set is calculated.

5. A point cloud self-supervised classification system based on a multi-view masked autoencoder according to any one of claims 1 to 4, characterized in that: include: The data preprocessing module is used to divide the data in the dataset into training set, validation set and test set according to a specific ratio; A multi-view conversion module, which converts the 3D point cloud input to the system into images from multiple viewpoints and stitches the images together; The self-supervised proxy task module is used to reconstruct proxy tasks through set masks, train the model to learn generalized features from the tasks, and improve the model's understanding ability; The downstream classification task fine-tuning module is used to make further fine-tuning of the model on the downstream classification task, so that it can be more suitable for classification tasks based on the generalization ability learned by the proxy task.

6. A point cloud self-supervised classification system based on a multi-view mask autoencoder, characterized in that: The invention comprises a computer device having a stored executable program of a point cloud self-supervised classification system of a multi-view mask autoencoder and a processor capable of executing any one of the point cloud self-supervised classification methods based on the multi-view mask autoencoder according to claims 1 to 4.

7. A point cloud self-supervised classification system based on a multi-view masked autoencoder, characterized in that: The invention comprises a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes a point cloud self-supervised classification method based on a multi-view mask autoencoder according to any one of claims 1 to 4.

8. A point cloud self-supervised classification system based on a multi-view mask autoencoder, characterized in that: It comprises an information data processing terminal, which is used to implement any one of the point cloud self-supervised classification methods based on a multi-view mask autoencoder according to claims 1 to 4.

Images