Point cloud self-supervision classification method and system based on multi-view mask auto-encoder

Abstract

The invention discloses a point cloud self-supervision classification method and system based on a multi-view mask auto-encoder, and belongs to the technical field of computer vision, and the method comprises the following steps: S101, obtaining a three-dimensional point cloud data set, dividing the three-dimensional point cloud data set according to training, verification and testing, and carrying out the preprocessing of the three-dimensional point cloud through employing a sampling technology; s102, in combination with a self-supervised deep learning network of a multi-view technology, training is carried out by using training data, and an optimal deep learning model is obtained by verifying data set optimization; and S103, testing and verifying in the test set and the verification set by using the deep learning network model with the optimal effect. According to the method, point cloud multi-view self-supervised classification based on deep learning is adopted, the network converts the point cloud into multiple views, the multiple views are integrated together by using scene and viewpoint information, projection loss is reduced to the maximum extent, mask reconstruction is performed on the combined view by using a mask auto-encoder, and the generalization ability of the model is enhanced.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology, and in particular relates to a point cloud self-supervised classification method and system based on a multi-view mask autoencoder. Background Art

[0002] Point cloud is a 3D data format consisting of a set of discrete points. This format can accurately capture the geometric details of objects and can also contain additional information such as color and normals, providing highly accurate data. Advances in technology and hardware have greatly facilitated the acquisition of point clouds, making it increasingly the preferred data format for various industries, including medical and other professional fields. As the number of applications grows, the demand for accurate point cloud processing models is also growing. However, the rapid development of model complexity and parameter size has outstripped the growth rate of available labeled datasets. Annotating large point cloud datasets, which often contain millions of points, is both time-consuming and laborious.

[0003] Self-supervised learning is an important research direction to solve this problem. By using unlabeled data for pre-training and fine-tuning with a smaller labeled dataset, self-supervised models can achieve comparable performance to supervised models while reducing the need for large amounts of labeled data. Unlike images, point cloud data is irregular and disordered, and the use of multi-view methods can effectively address these challenges. These methods extract features from multiple viewpoints, capture global features, and provide more comprehensive data analysis, showing their superiority in a variety of tasks.

[0004] Compared with self-supervised models directly applied to point clouds, multi-view methods are less affected by data size and scale. They always learn features from a global perspective. In addition, multi-view technology connects different dimensions of vision technology by utilizing mature 2D technology to process 3D data, thereby effectively retaining 3D information when converted to 2D images. In order to reduce the dependence of point cloud classification models on labeled data, finding a self-supervised classification method for point clouds has important theoretical research and practical application significance.

[0005] Through the above analysis, the problems and defects of the existing technology are as follows: the rapid development of model complexity and parameter scale has exceeded the growth rate of available labeled data sets. Annotating large point cloud data sets usually contains millions of points, which is time-consuming and laborious, but existing supervised methods rely on labeled data; comprehensive feature extraction for classification remains challenging because local feature learning methods are not always ideal, and the additional computational cost of segmenting point clouds is also a limiting factor; existing multi-view models are mainly supervised and heavily dependent on labeled data.

[0006] Based on this, the present invention designs a point cloud self-supervised classification method and system based on a multi-view mask autoencoder to solve the above problems. Summary of the invention

[0007] The purpose of the present invention is to solve the problem that the rapid development of model complexity and parameter size has exceeded the growth rate of available labeled datasets. Annotating large point cloud datasets usually contains millions of points, which is time-consuming and laborious, and existing supervised methods rely on labeled data; comprehensive feature extraction for classification remains challenging because local feature learning methods are not always ideal, and the additional computational cost of segmenting point clouds is also a limiting factor; existing multi-view models are mainly supervised and heavily dependent on labeled data. A point cloud self-supervised classification method and system based on a multi-view mask autoencoder is proposed.

[0008] In order to achieve the above object, the present invention adopts the following technical solutions: A point cloud self-supervised classification method and system based on a multi-view mask autoencoder, comprising the following steps: S101, obtaining a three-dimensional point cloud data set, dividing the three-dimensional point cloud data set into training, validation and testing, and preprocessing the three-dimensional point cloud using a sampling technique; S102, a self-supervised deep learning network combined with multi-view technology is trained with training data and optimized with validation data sets to obtain the best deep learning model; S103, using the best deep learning network model to test and verify on the test set and validation set, and output the three-dimensional point cloud classification result.

[0009] As a further description of the above technical solution: The self-supervised method of training a model to classify point clouds comprises the following steps: Step 1: Use laser scanning or obtain a 3D point cloud dataset from a public point cloud database, and divide the 3D point cloud dataset into three parts: training, validation, and testing. The training set is used to train the deep learning model, the validation set is used to tune the model parameters and select the best model, and the test set is used to evaluate the performance of the model to ensure its reliability in practical applications. After the data is divided, we also need to use sampling technology to pre-process the 3D point cloud to ensure the representativeness of the dataset and improve the training effect and generalization ability of the model; Step 2: Build a multi-view conversion network, a self-supervised proxy task network, and a downstream classification task fine-tuning network, and construct a point cloud self-supervised classification method based on a multi-view mask autoencoder; Step 3: Use the training set to train on the self-supervised proxy network; use the downstream classification task to fine-tune the trained model to meet the needs of the classification task. Use the validation set to verify the model performance and select the network model during the training process; Step 4: Use the network to evaluate performance on the test set and validation set, and obtain the classification results of the 3D point cloud.

[0010] As a further description of the above technical solution: The preprocessing of the three-dimensional point cloud data set in step 1 includes: Each 3D point cloud object in the 3D point cloud dataset is sampled using the farthest point sampling method. Each point cloud contains 1024 data points to match the input size of the deep learning network.

[0011] As a further description of the above technical solution: The step 2 of constructing a multi-view conversion network, a self-supervised proxy task network and a downstream classification task fine-tuning network, and constructing a point cloud self-supervised classification method based on a multi-view mask autoencoder, includes: (1) Convert the point cloud into multiple views through rendering and stitch them together into a whole; For point clouds , virtual cameras are arranged at fixed intervals around the point cloud through pre-designed spherical model camera parameters, and the weighted average of their points is calculated to determine the center of mass of the point cloud, and the cameras are aligned to generate views from multiple perspectives; (2) Perform mask reconstruction on the spliced ​​image through a mask autoencoder; First, for a given input image, it is evenly divided into image blocks according to the number of viewing angles, a portion is randomly selected from it according to uniform distribution, and 75% of the image blocks are masked. The remaining visible part is passed to the encoder of the model to obtain features. The decoder uses the features of the visible part to fill in the missing information and reconstruct the entire image, and self-optimizes by comparing the difference between the reconstructed image and the original image; (3) Fine-tune the model on the classification task; After the pre-training process of mask reconstruction, the encoder saves rich feature representations extracted from the data. These feature representations are highly abstract and can capture the underlying structure and complex patterns of the data. The encoder is connected to a neural network designed for point cloud classification tasks. Through supervised learning methods, the model continuously adjusts its own parameters to minimize the gap between the predicted output and the actual label. During the fine-tuning process, the model learns the precise features required for the point cloud classification task while retaining the general features learned during pre-training, thereby improving the model's performance on the classification task.

[0012] As a further description of the above technical solution: In step 3, the training set is used to train the self-supervised proxy network; the trained model is fine-tuned using the downstream classification task fine-tuning network to adapt to the needs of the classification task. During the training process, the validation set is used to verify the model performance and select the network model, including: (1) First, the training data set is imported into the deep learning network for training, the loss function is calculated and the model parameters are adjusted according to its value. Then, the validation set is used to test and select the trained network model. In the validation stage, the data will bypass the sample weighting module and directly pass through the feature extractor and classifier to generate the predicted label; (2) After obtaining the predicted label, compare it with the true label of the validation set to calculate the accuracy of the validation set.

[0013] As a further description of the above technical solution: In step 4, the network is used to perform performance evaluation on the test set and the validation set, and the classification results of the three-dimensional point cloud are obtained, including: The model selected in step three is verified using a test set and a validation set, the normalized and standardized data is passed through a feature extractor and a classifier to output predicted labels, and the classification accuracy on the test set and the validation set is calculated.

[0014] As a further description of the above technical solution: A point cloud self-supervised classification system based on a multi-view mask autoencoder includes: The data preprocessing module is used to divide the data in the data set into training set, validation set and test set according to a specific ratio; A multi-view conversion module, which converts the 3D point cloud input into images from multiple viewpoints and stitches the images together; The self-supervised proxy task module is used to reconstruct proxy tasks through set masks, train the model to learn generalized features from tasks, and improve the model's understanding ability; The downstream classification task fine-tuning module is used to make further fine-tuning of the model on the downstream classification task, so that it can be more suitable for classification tasks based on the generalization ability learned by the proxy task.

[0015] As a further description of the above technical solution: A point cloud self-supervised classification system of a multi-view mask autoencoder comprises: a computer device having a stored executable program of a point cloud self-supervised classification system of a multi-view mask autoencoder and a processor capable of executing any one of the point cloud self-supervised classification methods based on the multi-view mask autoencoder as claimed in claims 1 to 6.

[0016] As a further description of the above technical solution: A point cloud self-supervised classification system of a multi-view mask autoencoder comprises: a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes any one of the point cloud self-supervised classification methods based on the multi-view mask autoencoder according to claims 1 to 6.

[0017] As a further description of the above technical solution: A point cloud self-supervised classification system of a multi-view mask autoencoder includes an information data processing terminal, wherein the information data processing terminal is used to implement the system.

[0018] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: 1. The present invention adopts multi-view self-supervised classification of point clouds based on deep learning. The network converts point clouds into multiple views, integrates multiple views together using scene and viewpoint information, minimizes projection loss, and uses mask autoencoders to perform mask reconstruction on the combined views, thereby enhancing the generalization ability of the model.

[0019] 2. The point cloud multi-view self-supervised classification method based on deep learning provided by the present invention can effectively improve the problems of high computational cost and lack of global features in traditional methods. By converting the point cloud into multiple views and using a masked autoencoder for self-supervised learning, it not only reduces the computational cost, but also improves the global feature extraction and generalization capabilities of the model. When processing complex data, it can maintain a faster processing speed even in the case of higher model complexity.

[0020] 3. The present invention improves the existing point cloud processing methods, which have high computational costs, strong dependence on labeled data, and lack of global feature extraction capabilities. It can bring many expected benefits and commercial interweaving, improve the accuracy and computational speed of point cloud classification, enable unmanned driving to more quickly judge the situation ahead and make judgments, improve driving safety, and make model training easier without the need for labeled data. 4. The present invention improves the dependence of traditional supervised learning methods on labeled data, as well as the high computational cost and lack of global features of traditional methods. The present invention can use unlabeled data for training, improve the computational cost and lack of global features, and maintain a faster processing speed even with high computational complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flowchart of a point cloud self-supervised classification method based on a multi-view mask autoencoder provided by an example of the present invention; Figure 2 It is a schematic diagram of a point cloud self-supervised classification method based on a multi-view mask autoencoder provided by an example of the present invention; Figure 3 It is a schematic diagram of the overall structure of a neural network of a point cloud self-supervised classification method based on a multi-view mask autoencoder provided by an example of the present invention; Figure 4 It is a flowchart of rendering a three-dimensional point cloud into a multi-view complete image based on a point cloud self-supervised classification method of a multi-view mask autoencoder provided by an example of the present invention; Figure 5 It is a neural network pre-training structure diagram of a point cloud self-supervised classification method based on a multi-view mask autoencoder provided by an example of the present invention; Figure 6 It is a structural schematic diagram of fine-tuning a neural network on a classification task for a point cloud self-supervised classification method based on a multi-view mask autoencoder provided by an example of the present invention. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] Please see attached Figure 1 -Attached Figure 6 The present invention provides a technical solution: a point cloud self-supervised classification method and system based on a multi-view mask autoencoder, comprising the following steps: S101, obtaining a three-dimensional point cloud data set, dividing the three-dimensional point cloud data set into training, validation and testing, and preprocessing the three-dimensional point cloud using a sampling technique; S102, a self-supervised deep learning network combined with multi-view technology is trained with training data and optimized with validation data sets to obtain the best deep learning model; S103, using the best deep learning network model to test and verify on the test set and validation set, and output the three-dimensional point cloud classification result.

[0024] As a preferred embodiment, the following steps are included: (1) The point cloud is converted into multiple views through rendering and then stitched into a whole (see Figure 4 ) For point clouds , virtual cameras are arranged at fixed intervals around the point cloud through pre-designed spherical model camera parameters, and the weighted average of their points is calculated to determine the center of mass of the point cloud, and the cameras are aligned to generate views from multiple perspectives.

[0025] (2) The mask autoencoder is used to perform mask reconstruction on the spliced ​​image (see Figure 5 ) First, for a given input image, it is evenly divided into image blocks according to the number of viewing angles, a portion is randomly selected from it according to uniform distribution, and 75% of the image blocks are masked. The remaining visible part is passed to the encoder of the model to obtain features. The decoder uses the features of the visible part to fill in the missing information to reconstruct the entire image, and self-optimizes by comparing the difference between the reconstructed image and the original image.

[0026] (3) Fine-tune the model on the classification task (see Figure 6 ) After the pre-training process of mask reconstruction, the encoder saves rich feature representations extracted from the data. These feature representations are highly abstract and can capture the underlying structure and complex patterns of the data. The encoder is connected to a neural network designed for point cloud classification tasks. Through supervised learning methods, the model continuously adjusts its own parameters to minimize the gap between the predicted output and the actual label. During the fine-tuning process, the model learns the precise features required for the point cloud classification task while retaining the general features learned during pre-training, thereby improving the model's performance on the classification task.

[0027] The present invention provides a point cloud self-supervised classification system based on a multi-view mask autoencoder, comprising: The data preprocessing module is used to divide the data in the data set into training set, validation set and test set according to a specific ratio.

[0028] A multi-view conversion module, which converts the 3D point cloud input into images from multiple viewpoints and stitches the images together; The self-supervised proxy task module is used to reconstruct proxy tasks through set masks, train the model to learn generalized features from tasks, and improve the model's understanding ability.

[0029] The downstream classification task fine-tuning module is used to make further fine-tuning of the model on the downstream classification task, so that it can be more suitable for classification tasks based on the generalization ability learned by the proxy task.

[0030] The invention comprises a computer device and a processor for executing the method, a computer-readable storage medium and an information data processing terminal.

[0031] In order to prove the creativity and technical value of the technical solution of the present invention, this section provides application examples of the claimed technical solution on specific products or related technologies.

[0032] As a preferred embodiment, Figure 2As shown, a point cloud self-supervised classification method based on a multi-view mask autoencoder provided by an example of the present invention specifically includes the following steps: (1) Use laser scanning or obtain a 3D point cloud dataset from a public point cloud database, and divide the 3D point cloud dataset into three parts: training, validation, and testing. The training set is used to train the deep learning model, the validation set is used to tune the model parameters and select the best model, and the test set is used to evaluate the performance of the model to ensure its reliability in practical applications. After the data is divided, we also need to use sampling technology to pre-process the 3D point cloud to ensure the representativeness of the dataset and improve the training effect and generalization ability of the model; As a preferred embodiment, the present invention obtains three public 3D point cloud datasets: ShapeNet, ModelNet and ScanObjectNN.

[0033] ShapeNet is used as a training dataset for the self-supervised agent task module. ShapeNet is a richly annotated 3D shape dataset containing 51,300 3D models of 55 common object types, which are widely used in multiple research fields such as shape retrieval and shape classification. For each instance, we sample 1024 points using farthest point sampling to create the input point cloud. Our initial learning rate is set to 0.001 and the weight decay is 0.05. The model is pre-trained for 300 batches with a batch size of 20. The results show that the model can reconstruct images with different masking rates. However, we set the default masking rate to 0.75.

[0034] ModelNet and ScanObjectNN serve as training, testing, and validation datasets for downstream fine-tuning classification tasks. ModelNet40 is a widely used benchmark for evaluating point cloud processing and contains 12,311 models from 40 categories. We use the standard setting, where 9843 instances are used for training and 2468 instances are used for testing. The models are trained for 300 epochs with a batch size of 20 using the AdamW optimizer. We set the initial learning rate to 0.0005 and use a cosine annealing scheme to adjust the learning rate over time. ScanObjectNN is a real-world point cloud dataset containing 2902 3D objects from 15 categories. This dataset contains background noise, occlusions, and distortions, making it particularly challenging for point cloud processing.

[0035] The method for preprocessing the 3D point cloud is to sample each 3D point cloud object in the 3D point cloud dataset using the farthest point sampling method. Each point cloud contains 1024 data points to match the input size of the deep learning network.

[0036] (2) Construct a multi-view conversion network, a self-supervised proxy task network, and a downstream classification task fine-tuning network, and construct a point cloud self-supervised classification method based on a multi-view mask autoencoder, specifically including: The model is designed by combining the multi-view method with the self-supervision method. Figure 1 The detailed process is shown. First, the point cloud is rendered into images from multiple viewpoints using preset camera parameters. Then, these images are arranged in a specific order and stitched into a comprehensive composite image containing all viewpoints. This method maximizes the preservation of the 3D structure and contour information of the point cloud in the image. Subsequently, each viewpoint is treated as a block and the model is trained using a masked autoencoder strategy. This training reconstructs the image from each viewpoint to derive complete features.

[0037] (2.1) Building a multi-view conversion network like Figure 4 As shown in Figure 2, the generation of point cloud multi-view images depends on the preset camera parameters. The process is as follows: Figure 2 Common camera arrangements include the annular pattern, where a virtual camera is placed at a fixed position to generate The camera is configured in spherical mode, which minimizes the loss caused by the projection process and preserves the original features. First, the center of mass is determined using a weighted average of the point cloud. Then, it is placed at a fixed angle according to the spherical surround settings. planes. On each plane, The cameras are placed at equal angles, all pointing towards the centroid. Using 3D point cloud or a grid, which is fed into the rendering function to obtain m rendered images and stitch them together, as follows:

[0038] in is the 2D coordinate of the i-th viewpoint, is the camera intrinsic reference matrix of the i-th viewpoint, is the camera outlier (rotation and translation) of the ith viewpoint, Represents the coordinates of a 3D point in the global coordinate system. When the input is a mesh, the rasterizer converts the mesh into coordinates according to the viewpoint and assigns faces to pixels. The number of multi-view views m and the final fused image The generation process formula is as follows:

[0039]

[0040] in represents the stitched image, Represents a splicing operation, Represents a rendering operation.

[0041] As a preferred embodiment, the present invention uses a value of 90° for α and a value of 16 for m.

[0042] (2.2) Building a self-supervised agent task network like Figure 5 As shown in , the masked autoencoder uses the mask reconstruction principle to reconstruct the original image from the latent representation and mask marks based on partial observations after partially masking the original image. This method enhances the model's ability to extract generalized features from the latent representation. The process is as follows Figure 3 shown.

[0043] Mask segmentation. Segment the image according to different viewpoints. Divide into m image blocks Each viewpoint corresponds to a region, and a portion is randomly extracted from it according to uniform distribution. Then 75% of the image fragments are masked. By designing a higher masking rate, information redundancy is significantly reduced, the difficulty of the reconstruction task is increased, and the model obtains stronger feature extraction capabilities during the training process.

[0044] Encoder. Only the visible part of the image is Passed to the encoder for processing, the position patches p are embedded through a linear projection layer, and then passed to the Transformer, the formula is as follows:

[0045] Since only the visible part is processed, the memory and computational cost required to train the model is minimal. The input to the decoder is the complete information , including the visible part and the mask part , the formula is as follows:

[0046] Decoder. The masked autoencoder works by reconstructing the input image and predicting the pixel values ​​of the hidden image blocks. Each output produced by the decoder is a vector representing the pixel values ​​of the image block, as follows:

[0047] The decoder reconstructs the complete image The key step is to match the network output to the image pixels and to The loss is determined by pixel-by-pixel comparison, and the formula is as follows:

[0048] Where i represents the i-th pixel and N represents the total number of pixels. The masked autoencoder reconstructs the entire image by filling in the missing information and optimizes itself by comparing the difference between the reconstructed image and the original image.

[0049] As a preferred embodiment, the present invention uses a value of 16 for m and a value of 102400 for N.

[0050] (2.3) Build a fine-tuning network for downstream classification tasks like Figure 6 As shown in Figure 2, after the pre-training process of mask reconstruction, the encoder saves the rich feature representation extracted from the data. Figure 4 As shown in the figure. These feature representations are highly abstract and can capture the underlying structure and complex patterns of the data. The encoder is connected to a neural network designed for point cloud classification tasks. Through supervised learning methods, the model continuously adjusts its own parameters to minimize the gap between the predicted output and the actual label. During the fine-tuning process, the model learns the precise features required for the point cloud classification task while retaining the general features learned during pre-training, thereby improving the model's performance on the classification task.

[0051] (3) Use the training set to train on the self-supervised agent network; use the downstream classification task to fine-tune the network to fine-tune the trained model to meet the needs of the classification task. During the training process, use the validation set to verify the model performance and select the network model, including: (3.1) First, the training data set is imported into the deep learning network for training, the loss function is calculated and the model parameters are adjusted according to its value. Then, the validation set is used to test and select the trained network model. During the validation phase, the data will bypass the sample weighting module and directly pass through the feature extractor and classifier to generate the predicted label.

[0052] (3.2) After obtaining the predicted labels, they are compared with the true labels of the validation set to calculate the accuracy of the validation set.

[0053] (4) Use the network to evaluate performance on the test set and validation set, and obtain the classification results of the 3D point cloud, including: (4.1) Use the test set and validation set for verification, pass the normalized and standardized data through the feature extractor and classifier to output the predicted label, and calculate the classification accuracy on the test set and validation set.

[0054] The evaluation criteria for evaluating the point cloud multi-view self-supervised classification method based on deep learning provided by the embodiment of the present invention is the mean accuracy of the validation set to obtain the prediction accuracy of the model for three-dimensional point cloud classification. The classification results on ModelNet40 and ScanObjectNN are listed in Tables 1 and 2. Compared with PointNet, our method is improved by 0.9% and achieves comparable results to MVCNN. In addition, we conducted a time test on ModelNet40, as shown in Table 3. Although the computational complexity of our method is similar to that of MVCNN and the number of model parameters is higher than other models, it maintains a faster processing speed.

[0055]

[0056]

[0057] In the table, MVMAE is the abbreviation of a point cloud self-supervised classification method based on multi-view mask autoencoder; The embodiments of the present invention may be implemented by hardware, software or a combination of the two. The hardware components may be constructed by specialized logic circuits; the software portion may be stored in a storage device and operated by appropriate hardware such as a microprocessor or specially designed hardware. A person of ordinary skill in the art will appreciate that the above-mentioned device and method may be implemented by computer executable instructions or processor control codes, which may be stored on a storage medium such as a disk, CD or DVD-ROM, or provided in a data medium such as a read-only memory, a programmable memory, an optical or electronic signal carrier, etc. The device and its modules of the present invention may be implemented by semiconductor circuits such as ultra-large-scale integrated circuits, gate arrays, logic chips, transistors, etc., or by software executed by various types of processors, or by a combination of the aforementioned hardware circuits and software.

[0058] The above are only preferred specific implementation modes of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical solutions and inventive concepts of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A point cloud self-supervised classification method and system based on multi-view mask autoencoder, characterized in that: The following steps are involved: S101, obtaining a three-dimensional point cloud data set, dividing the three-dimensional point cloud data set into training, validation and testing, and preprocessing the three-dimensional point cloud using a sampling technique; S102, a self-supervised deep learning network combined with multi-view technology is trained with training data and optimized with validation data sets to obtain the best deep learning model; S103, using the best deep learning network model to test and verify on the test set and validation set, and output the three-dimensional point cloud classification result.

2. According to claim 1, a point cloud self-supervised classification method based on a multi-view mask autoencoder is characterized in that: The self-supervised method of training a model to classify point clouds comprises the following steps: Step 1: Use laser scanning or obtain a 3D point cloud dataset from a public point cloud database, and divide the 3D point cloud dataset into three parts: training, validation, and testing. The training set is used to train the deep learning model, the validation set is used to tune the model parameters and select the best model, and the test set is used to evaluate the performance of the model to ensure its reliability in practical applications. After the data is divided, we also need to use sampling technology to pre-process the 3D point cloud to ensure the representativeness of the dataset and improve the training effect and generalization ability of the model; Step 2: Build a multi-view conversion network, a self-supervised proxy task network, and a downstream classification task fine-tuning network, and construct a point cloud self-supervised classification method based on a multi-view mask autoencoder; Step 3: Use the training set to train on the self-supervised proxy network; use the downstream classification task to fine-tune the trained model to meet the needs of the classification task. Use the validation set to verify the model performance and select the network model during the training process; Step 4: Use the network to evaluate performance on the test set and validation set, and obtain the classification results of the 3D point cloud.

3. The point cloud self-supervised classification method based on multi-view mask autoencoder according to claim 2, characterized in that: The preprocessing of the three-dimensional point cloud data set in step 1 includes: Each 3D point cloud object in the 3D point cloud dataset is sampled using the farthest point sampling method. Each point cloud contains 1024 data points to match the input size of the deep learning network.

4. The point cloud self-supervised classification method based on multi-view mask autoencoder according to claim 2, characterized in that: The step 2 of constructing a multi-view conversion network, a self-supervised proxy task network and a downstream classification task fine-tuning network, and constructing a point cloud self-supervised classification method based on a multi-view mask autoencoder, includes: (1) Convert the point cloud into multiple views through rendering and stitch them together into a whole; For point clouds , virtual cameras are arranged at fixed intervals around the point cloud through pre-designed spherical model camera parameters, and the weighted average of their points is calculated to determine the center of mass of the point cloud, and the cameras are aligned to generate views from multiple perspectives; (2) Perform mask reconstruction on the spliced ​​image through a mask autoencoder; First, for a given input image, it is evenly divided into image blocks according to the number of viewing angles, a portion is randomly selected from it according to uniform distribution, and 75% of the image blocks are masked. The remaining visible part is passed to the encoder of the model to obtain features. The decoder uses the features of the visible part to fill in the missing information and reconstruct the entire image, and self-optimizes by comparing the difference between the reconstructed image and the original image; (3) Fine-tune the model on the classification task; After the pre-training process of mask reconstruction, the encoder saves rich feature representations extracted from the data. These feature representations are highly abstract and can capture the underlying structure and complex patterns of the data. The encoder is connected to a neural network designed for point cloud classification tasks. Through supervised learning methods, the model continuously adjusts its own parameters to minimize the gap between the predicted output and the actual label. During the fine-tuning process, the model learns the precise features required for the point cloud classification task while retaining the general features learned during pre-training, thereby improving the model's performance on the classification task.

5. The point cloud self-supervised classification method based on multi-view mask autoencoder according to claim 2, characterized in that: In step 3, the training set is used to train the self-supervised proxy network; the trained model is fine-tuned using the downstream classification task fine-tuning network to adapt to the needs of the classification task. During the training process, the validation set is used to verify the model performance and select the network model, including: (1) First, the training data set is imported into the deep learning network for training, the loss function is calculated and the model parameters are adjusted according to its value. Then, the validation set is used to test and select the trained network model. In the validation stage, the data will bypass the sample weighting module and directly pass through the feature extractor and classifier to generate the predicted label; (2) After obtaining the predicted label, compare it with the true label of the validation set to calculate the accuracy of the validation set.

6. The point cloud self-supervised classification method based on multi-view mask autoencoder according to claim 2, characterized in that: In step 4, the network is used to perform performance evaluation on the test set and the validation set, and the classification results of the three-dimensional point cloud are obtained, including: The model selected in step three is verified using a test set and a validation set, the normalized and standardized data is passed through a feature extractor and a classifier to output predicted labels, and the classification accuracy on the test set and the validation set is calculated.

7. A point cloud self-supervised classification system based on a multi-view masked autoencoder according to any one of claims 1 to 6, characterized in that: include: The data preprocessing module is used to divide the data in the data set into training set, validation set and test set according to a specific ratio; A multi-view conversion module, which converts the 3D point cloud input into images from multiple viewpoints and stitches the images together; The self-supervised proxy task module is used to reconstruct proxy tasks through set masks, train the model to learn generalized features from tasks, and improve the model's understanding ability; The downstream classification task fine-tuning module is used to make further fine-tuning of the model on the downstream classification task, so that it can be more suitable for classification tasks based on the generalization ability learned by the proxy task.

8. A point cloud self-supervised classification system based on a multi-view masked autoencoder, characterized in that: The invention comprises a computer device having a stored executable point cloud self-supervised classification system program of a multi-view mask autoencoder and a processor capable of executing any one of the point cloud self-supervised classification methods based on the multi-view mask autoencoder as claimed in claims 1 to 6.

9. A point cloud self-supervised classification system based on a multi-view masked autoencoder, characterized in that: The invention comprises a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes a point cloud self-supervised classification method based on a multi-view mask autoencoder according to any one of claims 1 to 6.

10. A point cloud self-supervised classification system based on a multi-view masked autoencoder, characterized in that: It comprises an information data processing terminal, and the information data processing terminal is used to implement the system.

Images