An end-to-end neural network-based neighborhood point cloud down-sampling classification method and device

Abstract

The present disclosure relates to a kind of based on end-to-end neural network neighborhood point cloud downsampling classification method and device.The method includes: input the three-dimensional point cloud data to be classified, and the pre-processing operation is carried out to three-dimensional point cloud data;Through the disclosed data set, the point cloud downsampling model based on neural network is trained, the trained point cloud downsampling model based on neural network is obtained, and the ordered regular three-dimensional point cloud data is input into the trained point cloud downsampling model based on neural network, and the simplified sampled three-dimensional point cloud data is obtained;Through Pointnet network, the point cloud classification of simplified sampled three-dimensional point cloud data is carried out, and the final point cloud classification result is obtained.The present disclosure can effectively retain the semantic information of point cloud, remove unnecessary elements existing in three-dimensional point cloud data, provide effective three-dimensional point cloud data for downstream classification, and improve point cloud classification precision.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to a method and apparatus for classifying neighborhood point clouds based on end-to-end neural networks. Background Technology

[0002] In recent years, the prevalence of 3D sensing devices has increased. These devices typically capture data in the form of point clouds. Various applications, such as classification, registration, and shape reconstruction, consume raw point cloud data. These applications are inefficient when processing large amounts of point cloud data; therefore, it is often necessary to reduce the size of the point cloud data to improve computational efficiency and reduce communication costs.

[0003] Before running downstream tasks, this often involves downsampling the data to reduce the amount of point cloud data. Because downsampling preserves the data structure (i.e., both input and output are in point cloud form), and sampling maintains data fidelity and keeps the data in an interpretable representation, downsampling of 3D point clouds has become a popular research direction.

[0004] 3D point cloud data classification has always been a challenging problem. In related technologies, the PointNet network, which can directly process point cloud data, has been proposed. It learns the features of each point through a multilayer perceptron, while employing a symmetric function to maintain the permutation invariance of the point cloud. Finally, these aggregated features are passed through a fully connected layer to output a one-dimensional matrix. Each component of the matrix represents the probability of the point cloud being classified into that category, and the category with the highest probability is taken as the class assigned by the network. Considering the substitutability of neighboring points in a point cloud, removing redundant information from the data still ensures that the PointNet network has very high classification accuracy. However, existing point cloud downsampling classification methods can only sample according to specific criteria and cannot consider the needs of downstream classification tasks. The downsampled point cloud loses a large amount of semantic information, which can easily affect the performance of downstream tasks.

[0005] Therefore, it is necessary to provide a new technical solution to improve one or more of the problems existing in the above solutions.

[0006] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0007] The purpose of this disclosure is to provide a method and apparatus for classifying neighborhood point clouds based on end-to-end neural networks, thereby overcoming, to at least some extent, one or more problems caused by the limitations and defects of related technologies.

[0008] According to a first aspect of the present disclosure, a method for classification based on downsampling of neighborhood point clouds using an end-to-end neural network is provided, the method comprising:

[0009] Input the 3D point cloud data to be classified, and perform preprocessing operations on the 3D point cloud data; wherein, the preprocessing operations include: denoising the 3D point cloud data, converting the disordered 3D point cloud data into ordered 3D point cloud data, and performing voxelization on the ordered 3D point cloud data to obtain ordered and regular 3D point cloud data, the formula of which is as follows:

[0010] X = Voxel(PCD) (1)

[0011] In the formula, PCD represents the denoised 3D point cloud data, Voxel(·) represents the voxelization operation, and X represents the voxelized 3D point cloud data.

[0012] The neural network-based point cloud downsampling model is trained using a public dataset to obtain a trained neural network-based point cloud downsampling model. The ordered and regularized 3D point cloud data is then input into the trained neural network-based point cloud downsampling model to obtain simplified sampled 3D point cloud data.

[0013] The simplified sampled 3D point cloud data is classified using the Pointnet network to obtain the final point cloud classification result.

[0014] In the embodiments of this disclosure, the point cloud downsampling model includes a key point extraction network and an adaptive neighborhood soft projection attention generation network.

[0015] In the embodiments of this disclosure, the key point extraction network includes a multilayer perceptron layer, a pooling layer, and a fully connected layer;

[0016] The adaptive neighborhood soft projection attention generation network includes an adaptive neighborhood soft projection layer and an attention layer.

[0017] In embodiments of this disclosure, the key point extraction network, when processing the preprocessed 3D point data, includes:

[0018] The feature matrix of the 3D point cloud data is extracted through the multilayer perceptron layer;

[0019] Aggregate the aggregated features of the 3D point cloud data using the max pooling layer feature matrix;

[0020] The required set of key points is obtained through the fully connected layer.

[0021] In embodiments of this disclosure, when inputting the keypoint set obtained by the keypoint extraction network into the software projection attention generation network, the following steps are included:

[0022] The size of the neighborhood is adaptively obtained based on the importance of the keypoint set. The keypoint set is then adaptively offset using the following formula:

[0023] X key =MLP(X key )+X key (2)

[0024] In the formula, X key This represents the preprocessed keypoint set. Soft projection technology is used to project this keypoint set onto the preprocessed 3D point cloud data, and an adaptive nearest neighbor clustering method is used. The specific formula is as follows:

[0025] K = K0 × D (3)

[0026]

[0027]

[0028] In the formula, K0 represents the average density of the preprocessed 3D point cloud data, U represents the size of the preprocessed 3D point cloud data region, N0 represents the number of points in the preprocessed 3D point cloud data, D represents the density stretching coefficient, and d represents the anti-local density. This will obtain the set of neighboring points of the key point set in the preprocessed 3D point cloud data, where each key point set corresponds to a set of neighboring points.

[0029] For each of the generated adaptive neighbor point sets, an attention matrix is ​​obtained through an attention mechanism, and the attention matrix is ​​then multiplied by the adaptive neighbor point set to obtain the final downsampled 3D point cloud data.

[0030] In the embodiments of this disclosure, the formula used in the point cloud downsampling model is as follows:

[0031] f1 = MLP(X) (6)

[0032] f2 = MAX_POOLING(f1) (7)

[0033] f3 = FC(f2) (8)

[0034] f4 = PROJECT(f3, X) (9)

[0035] F5 = ATTENTION(f3, f4) (10)

[0036] Xds =f5·f4 (11)

[0037] In the formulas, equations (6) to (11) represent the operations in the corresponding point cloud downsampling network, equations (6) to (8) represent the operations in the corresponding keypoint extraction network, f1 represents the feature matrix of the 3D point cloud data output by the multilayer perceptron, MLP represents the multilayer perceptron layer, X represents the 3D point cloud data after preprocessing, f2 represents the aggregated features of the 3D point cloud data output by the max pooling layer, MAX_POOLING represents the max pooling layer, f3 represents the set of keypoints output by the fully connected layer, FC represents the fully connected layer, f4 represents the set of neighboring keypoints, PROJECT represents the soft projection layer, f5 represents the attention weights, ATTENTION represents the attention mechanism layer, · represents the dot product operation, X ds This represents the downsampled 3D point cloud data.

[0038] In embodiments of this disclosure, training a neural network-based point cloud downsampling model using a publicly available dataset to obtain a trained neural network-based point cloud downsampling model includes:

[0039] The selected public dataset was divided into a training set and a test set in a 7:3 ratio;

[0040] During training, the total number of epochs was set to 300, the batch size was 8, and Adam was selected as the optimizer. The optimizer employed a decaying learning rate training strategy, with an initial learning rate of 0.08. Every 30 epochs, the learning rate was reduced to 0.8 times its original value. The overall network loss function consisted of the loss function of the downsampling network and the PointNet classification network, as shown below:

[0041] Loss(X,X ds =Loss cls +αLoss ds (12)

[0042] Loss cls =Loss cd +Loss mse (Pred, Label) (13)

[0043]

[0044] In the formula, Loss cls Loss represents the loss function for the downstream classification task. ds Represents the loss function for the downsampling task; Loss cd Indicates Chamfer Distance loss, Loss msedenoted as mean squared loss in the classification task, Label represents the original label, and Pred represents the label predicted by the downstream Pointnet network; the first term on the right side of the equation (14) represents the nearest neighbor loss between the preprocessed 3D point cloud data and the downsampled 3D point cloud data, and the second term on the right side of the equation represents the maximum and minimum loss between the preprocessed 3D point cloud data and the downsampled 3D point cloud data; α and β are weight hyperparameters, with α set to 0.8 and β set to 0.5.

[0045] According to a second aspect of the present disclosure, an end-to-end neural network-based adaptive neighborhood point cloud downsampling classification device is provided, the device comprising:

[0046] The input and preprocessing module is used to input the 3D point cloud data to be classified and to perform preprocessing operations on the 3D point cloud data. The preprocessing operations include: denoising the 3D point cloud data to convert disordered 3D point cloud data into ordered 3D point cloud data, and voxelizing the ordered 3D point cloud data to obtain ordered and regularized 3D point cloud data. The formula is as follows:

[0047] X = Voxel(PCD) (1)

[0048] In the formula, PCD represents the denoised 3D point cloud data, Voxel(·) represents the voxelization operation, and X represents the voxelized 3D point cloud data.

[0049] The sampling module is used to train a neural network-based point cloud downsampling model using a public dataset to obtain a trained neural network-based point cloud downsampling model, and input the ordered and regularized 3D point cloud data into the trained neural network-based point cloud downsampling model to obtain simplified sampled 3D point cloud data.

[0050] The classification module is used to classify the simplified sampled 3D point cloud data using the Pointnet network to obtain the final point cloud classification result.

[0051] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0052] In one embodiment of this disclosure, the above-described method and apparatus, on the one hand, perform preprocessing on the 3D point cloud data to be classified. During preprocessing, the semantic information of the point cloud is effectively preserved, and unnecessary elements in the 3D point cloud data are removed, providing effective 3D point cloud data for downstream classification. On the other hand, the preprocessed 3D point cloud data is input into a pre-trained point cloud downsampling model to obtain downsampled 3D point cloud data. The downsampled 3D point cloud data is then input into the PointNet network for classification to obtain point cloud classification results, thereby improving the accuracy of point cloud classification.

[0053] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0054] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0055] Figure 1 This schematically illustrates a flowchart of the steps of an end-to-end neural network neighborhood point cloud downsampling classification method in an exemplary embodiment of the present disclosure;

[0056] Figure 2 This schematic diagram illustrates a network architecture of a point cloud downsampling model in an exemplary embodiment of the present disclosure.

[0057] Figure 3 This schematically illustrates a framework diagram of the Pointnet network in an exemplary embodiment of the present disclosure;

[0058] Figure 4 The diagram illustrates a block diagram of an end-to-end neural network-based adaptive neighborhood point cloud downsampling classification apparatus according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0059] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0060] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0061] This example implementation first provides a classification method based on end-to-end neural network neighborhood point cloud downsampling. (Reference) Figure 1 As shown, the method may include:

[0062] Step S101: Input the 3D point cloud data to be classified and perform preprocessing operations on the 3D point cloud data; wherein, the preprocessing operations include: performing noise reduction processing on the 3D point cloud data, converting the disordered 3D point cloud data into ordered 3D point cloud data, and performing voxelization processing on the ordered 3D point cloud data to obtain ordered and regular 3D point cloud data, the formula of which is as follows:

[0063] X = Voxel(PCD) (1)

[0064] In the formula, PCD represents the denoised 3D point cloud data, Voxel(·) represents the voxelization operation, and X represents the voxelized 3D point cloud data.

[0065] Step S102: Train the neural network-based point cloud downsampling model using a public dataset to obtain a trained neural network-based point cloud downsampling model. Input the ordered and regularized 3D point cloud data into the trained neural network-based point cloud downsampling model to obtain simplified sampled 3D point cloud data.

[0066] Step S103: Perform point cloud classification on the simplified sampled 3D point cloud data using the Pointnet network to obtain the final point cloud classification result.

[0067] The above method and apparatus, on the one hand, perform preprocessing on the 3D point cloud data to be classified. During preprocessing, the semantic information of the point cloud is effectively preserved, and unnecessary elements in the 3D point cloud data are removed, providing effective 3D point cloud data for downstream classification. On the other hand, the preprocessed 3D point cloud data is input into a pre-trained point cloud downsampling model to obtain downsampled 3D point cloud data. The downsampled 3D point cloud data is then input into the PointNet network for classification to obtain point cloud classification results, thus improving the accuracy of point cloud classification.

[0068] Below, we will refer to Figures 1 to 3 The steps of the method described above in this example embodiment will be explained in more detail.

[0069] In step S101, the 3D point cloud data to be classified is input, and preprocessing operations are performed on the 3D point cloud data. The preprocessing operations include: denoising the 3D point cloud data to convert disordered 3D point cloud data into ordered 3D point cloud data, and voxelizing the ordered 3D point cloud data to obtain ordered and regular 3D point cloud data. The formula is as follows:

[0070] X = Voxel(PCD) (1)

[0071] In the formula, PCD represents the denoised 3D point cloud data, Voxel(·) represents the voxelization operation, and X represents the voxelized 3D point cloud data.

[0072] It is important to understand that during the preprocessing of 3D point cloud data to be classified, the semantic information of the point cloud can be effectively preserved, and unnecessary elements in the 3D point cloud data can be removed, thus providing effective 3D point cloud data for downstream classification.

[0073] In one example, a set of 3D point cloud data to be classified is first input, using {P i |i=1,…,n} represents the 3D point cloud data to be classified, where each point P in the 3D point cloud data to be classified i There are three channels (x, y, z) representing the feature information of the three coordinate axes.

[0074] In step S102, the neural network-based point cloud downsampling model is trained using a public dataset to obtain a trained neural network-based point cloud downsampling model. The ordered and regularized three-dimensional point cloud data is then input into the trained neural network-based point cloud downsampling model to obtain simplified sampled three-dimensional point cloud data.

[0075] It is important to understand that a neural network-based point cloud downsampling model is trained using 3D point cloud data. Then, ordered and regular 3D point cloud data is input into the trained neural network-based point cloud downsampling model to obtain simplified sampled 3D point cloud data. Finally, the number of points in the downsampled 3D point cloud data is controlled to 2048.

[0076] In one embodiment, the point cloud downsampling model includes a keypoint extraction network and an adaptive neighborhood soft projection attention generation network.

[0077] The key point extraction network includes a multi-layer perceptron layer, a pooling layer, and a fully connected layer.

[0078] The adaptive neighborhood soft projection attention generation network includes a soft projection layer and an attention layer.

[0079] In one embodiment, the keypoint extraction network, when processing the preprocessed 3D point data, includes:

[0080] The feature matrix of the 3D point cloud data is extracted through the multilayer perceptron layer;

[0081] Aggregate the aggregated features of the 3D point cloud data using the max pooling layer feature matrix;

[0082] The required set of key points is obtained through the fully connected layer.

[0083] It is important to understand that when aggregating the aggregated features of 3D point cloud data through the feature matrix of the max pooling layer, it also plays a role in dimensionality reduction.

[0084] In one embodiment, when the keypoint set obtained by the keypoint extraction network is input into the adaptive neighborhood soft projection attention generation network, the following is included:

[0085] The size of the neighborhood is adaptively obtained based on the importance of the keypoint set. The keypoint set is then adaptively offset using the following formula:

[0086] X key =MLP(X key )+X key (2)

[0087] In the formula, X key The keypoint set after preprocessing is represented by the keypoint set, which is projected onto the preprocessed 3D point cloud data using soft projection technology, and then an adaptive nearest neighbor clustering method is used, the formula of which is as follows:

[0088] K = K0 × D (3)

[0089]

[0090]

[0091] In the formula, K0 represents the average density of the preprocessed 3D point cloud data, U represents the size of the preprocessed 3D point cloud data region, N0 represents the number of points in the preprocessed 3D point cloud data, D represents the density stretching coefficient, and d represents the anti-local density, which is approximately estimated by kernel density estimation. The set of neighboring points of the keypoint set in the preprocessed 3D point cloud data will be obtained, where each keypoint set corresponds to one set of neighboring points.

[0092] For each of the generated adaptive neighbor point sets, an attention matrix is ​​obtained through an attention mechanism, and the attention matrix is ​​then multiplied by the adaptive neighbor point set to obtain the final downsampled 3D point cloud data.

[0093] In one embodiment, the formula used in the point cloud downsampling model is as follows:

[0094] f1 = MLP(X) (6)

[0095] f2 = PMAX_POOLING(f1) (7)

[0096] f3 = FC(f2) (8)

[0097] f4 = PROHECT(f3, X) (9)

[0098] f5 = ATTENTION(f3, f4) (10)

[0099] X ds =f5·f4 (11)

[0100] In the formulas, equations (6) to (11) represent the operations in the corresponding point cloud downsampling network, equations (6) to (8) represent the operations in the key point extraction network, f1 represents the feature matrix of the 3D point cloud data output by the multilayer perceptron, MLP represents the multilayer perceptron layer, X represents the 3D point cloud data after preprocessing, f2 represents the aggregated features of the 3D point cloud data output by the max pooling layer, MAX_POOLING represents the max pooling layer, f3 represents the set of key points output by the fully connected layer, FC represents the fully connected layer, f4 represents the set of neighboring key points, PROJECT represents the soft projection layer, f5 represents the attention weights, ATTENTION represents the attention mechanism layer, · represents the dot product operation, X ds This represents the downsampled 3D point cloud data.

[0101] It's important to understand that PROJECT primarily performs adaptive nearest neighbor clustering. The input to this layer is the set of keypoints f3 and the preprocessed 3D point cloud data X, and the output is the set of keypoint neighbors f4. ATTENTION represents the attention mechanism layer. The input to this layer is the set of keypoints f3 and the set of keypoint neighbors f4, and the output is the attention weight f5. Finally, the downsampled 3D point cloud data X is obtained by performing a dot product operation between the attention weight f5 and the set of keypoint neighbors f4. ds MLP is short for Multi-layer Perceptron.

[0102] In one embodiment, training the neural network-based point cloud downsampling model using a public dataset to obtain a trained neural network-based point cloud downsampling model includes:

[0103] The selected public dataset was divided into a training set and a test set in a 7:3 ratio;

[0104] During training, the total number of epochs was set to 300, the batch size was 8, and Adam was selected as the optimizer. The optimizer employed a decaying learning rate training strategy, with an initial learning rate of 0.08. Every 30 epochs, the learning rate was reduced to 0.8 times its original value. The overall network loss function consisted of the loss function of the downsampling network and the PointNet classification network, as shown below:

[0105] Loss(X, X) ds =Loss cls +αLoss ds (12)

[0106] Loss cls =Los sc d+Loss mse (Pred, Label) (13)

[0107]

[0108] In the formula, Loss cls Loss represents the loss function for the downstream classification task. ds Represents the loss function for the downsampling task; Loss cd Indicates Chamfer Distance loss, Loss mse denoted as mean squared loss in the classification task, Label represents the original label, and Pred represents the label predicted by the downstream Pointnet network; the first term on the right side of the equation (14) represents the nearest neighbor loss between the preprocessed 3D point cloud data and the downsampled 3D point cloud data, and the second term on the right side of the equation represents the maximum and minimum loss between the preprocessed 3D point cloud data and the downsampled 3D point cloud data; α and β are weight hyperparameters, with α set to 0.8 and β set to 0.5.

[0109] It should be noted that the publicly available dataset is Model40. Chamfer Distance represents the chamfer distance. The training set includes training data, and the test set includes test data. One epoch represents the completion of one forward computation + backpropagation process after all the 3D point cloud data has been fed into the network.

[0110] Once training is complete, the test data from the test set is input into the trained point cloud downsampling model to obtain the classification results of the corresponding test data.

[0111] For the existing Pointnet network, the original input number of the Modelnet40 public dataset is 2048. Comparative experiments were conducted using different sampling rates, and the results are shown in Table 1 below:

[0112] Table 1 shows the results of comparative experiments using different sampling rates.

[0113]

[0114] In step S103, the simplified sampled 3D point cloud data is classified using the Pointnet network to obtain the final point cloud classification result.

[0115] It's important to understand that after obtaining the downsampled 3D point cloud data, it needs to be classified. Specifically, the downsampled 3D point cloud data is input into the PointNet network to classify it and obtain the point cloud classification results. PointNet represents point cloud.

[0116] It is important to understand that the end-to-end neural network-based adaptive neighborhood point cloud downsampling classification is an end-to-end network framework. The downsampling process is integrated with the classification process, which not only improves computational efficiency, but also ensures the accuracy of point cloud classification even when dealing with small 3D point cloud data.

[0117] It should be noted that although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that these steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps. Furthermore, it is readily understood that these steps may be executed synchronously or asynchronously, for example, in multiple modules / processes / threads.

[0118] Furthermore, this example embodiment also provides a classification device based on end-to-end neural network neighborhood point cloud downsampling. (See reference...) Figure 4 As shown, the device 100 may include an input and preprocessing module, a sampling module, and a classification module.

[0119] The input and preprocessing module is used to input the 3D point cloud data to be classified and to perform preprocessing operations on the 3D point cloud data. The preprocessing operations include: denoising the 3D point cloud data to convert disordered 3D point cloud data into ordered 3D point cloud data, and voxelizing the ordered 3D point cloud data to obtain ordered and regular 3D point cloud data. The formula is as follows:

[0120] X = Voxel(PCD) (1)

[0121] In the formula, PCD represents the denoised 3D point cloud data, Voxel(·) represents the voxelization operation, and X represents the voxelized 3D point cloud data.

[0122] The sampling module is used to train a neural network-based point cloud downsampling model using a public dataset to obtain a trained neural network-based point cloud downsampling model, and input the ordered and regularized 3D point cloud data into the trained neural network-based point cloud downsampling model to obtain simplified sampled 3D point cloud data.

[0123] The classification module is used to classify the simplified sampled 3D point cloud data using the Pointnet network to obtain the final point cloud classification result.

[0124] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0125] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. A classification method based on downsampling of neighborhood point clouds using an end-to-end neural network, characterized in that, The method includes: Input the 3D point cloud data to be classified, and perform preprocessing operations on the 3D point cloud data; wherein, the preprocessing operations include: denoising the 3D point cloud data, converting the disordered 3D point cloud data into ordered 3D point cloud data, and performing voxelization on the ordered 3D point cloud data to obtain ordered and regular 3D point cloud data, the formula of which is as follows: In the formula, PCD represents the denoised 3D point cloud data. Represents the voxelization operation. This represents the 3D point cloud data after voxelization. The neural network-based point cloud downsampling model is trained using a public dataset to obtain a trained neural network-based point cloud downsampling model. The ordered and regularized 3D point cloud data is then input into the trained neural network-based point cloud downsampling model to obtain simplified sampled 3D point cloud data. The simplified sampled 3D point cloud data is classified using the Pointnet network to obtain the final point cloud classification result; The point cloud downsampling model includes a key point extraction network and an adaptive neighborhood soft projection attention generation network. The key point extraction network includes a multi-layer perceptron layer, a pooling layer, and a fully connected layer. The adaptive neighborhood soft projection attention generation network includes an adaptive neighborhood soft projection layer and an attention layer; The required set of key points is obtained through the fully connected layer; In the adaptive neighborhood soft projection attention generation network, the input is the key point set obtained by the key point extraction network.

2. The classification method based on end-to-end neural network neighborhood point cloud downsampling according to claim 1, characterized in that, In the key point extraction network, processing the preprocessed 3D point cloud data includes: The feature matrix of the 3D point cloud data is extracted through the multilayer perceptron layer; The aggregated features of the 3D point cloud data are aggregated using the feature matrix of the max pooling layer.

3. The classification method based on end-to-end neural network neighborhood point cloud downsampling according to claim 2, characterized in that, In the adaptive neighborhood soft projection attention generation network, when the keypoint set obtained by the keypoint extraction network is input, it includes: The size of the neighborhood is adaptively obtained based on the importance of the keypoint set. The keypoint set is then adaptively offset using the following formula: In the formula, The keypoint set after preprocessing is represented by a soft projection technique that projects the keypoint set onto the preprocessed 3D point cloud data. An adaptive nearest neighbor clustering method is then used, with the following formula: In the formula, This represents the average density of the preprocessed 3D point cloud data. This indicates the size of the preprocessed 3D point cloud data region. This indicates the number of points in the preprocessed 3D point cloud data. represents the density stretching coefficient, d represents the anti-local density, and will obtain the set of neighboring points of the key point set in the preprocessed 3D point cloud data, where each key point set corresponds to a set of neighboring points; For each generated set of adaptive neighboring points, an attention matrix is ​​obtained through an attention mechanism, and the attention matrix is ​​then multiplied by the set of neighboring points to obtain the final downsampled 3D point cloud data.

4. The classification method based on end-to-end neural network neighborhood point cloud downsampling according to claim 3, characterized in that, The formula used in the point cloud downsampling model is as follows: In the formula, to For the corresponding operations in the point cloud downsampling network, the formula is... to To extract operations from the network corresponding to key points, This represents the feature matrix of the 3D point cloud data output by the multilayer perceptron, where MLP stands for multilayer perceptron layer. This represents the preprocessed 3D point cloud data. This represents the aggregated features of the 3D point cloud data output by the max pooling layer, where MAX_POOLING represents the max pooling layer. This represents the set of key points output by the fully connected layer; FC stands for fully connected layer. The key represents the set of neighboring points, and PROJECT represents the soft projection layer. This represents attention weights, and ATTENTION represents the attention mechanism layer. This represents the dot product operation. This represents the downsampled 3D point cloud data.

5. The classification method based on end-to-end neural network neighborhood point cloud downsampling according to claim 4, characterized in that, The step of training a neural network-based point cloud downsampling model using a publicly available dataset to obtain a trained neural network-based point cloud downsampling model includes: The selected public dataset was divided into a training set and a test set in a 7:3 ratio; During training, the total number of epochs was set to 300, the batch size was 8, and Adam was selected as the optimizer. The optimizer employed a decaying learning rate training strategy, with an initial learning rate of 0.

08. Every 30 epochs, the learning rate was reduced to 0.8 times its original value. The overall network loss function consisted of the loss function of the downsampling network and the PointNet classification network, as shown below: In the formula, This represents the loss function for the downstream classification task. This represents the loss function for the downsampling task; This indicates Chamfer Distance loss. denoted as mean squared error loss in classification task, Label represents the original label, Pred represents the label predicted by the downstream Pointnet network; the first term on the right side of equation (14) represents the nearest neighbor loss between the preprocessed 3D point cloud data and the downsampled 3D point cloud data, and the second term on the right side of equation (14) represents the maximum and minimum loss between the preprocessed 3D point cloud data and the downsampled 3D point cloud data. and For weight hyperparameters, Set to 0.8, Set it to 0.

5.

6. A classification device based on end-to-end neural network neighborhood point cloud downsampling, characterized in that, The device includes: The input and preprocessing module is used to input the 3D point cloud data to be classified and to perform preprocessing operations on the 3D point cloud data. The preprocessing operations include: denoising the 3D point cloud data to convert disordered 3D point cloud data into ordered 3D point cloud data, and voxelizing the ordered 3D point cloud data to obtain ordered and regularized 3D point cloud data. The formula is as follows: In the formula, PCD represents the denoised 3D point cloud data. Represents the voxelization operation. This represents the 3D point cloud data after voxelization. The sampling module is used to train a neural network-based point cloud downsampling model using a public dataset to obtain a trained neural network-based point cloud downsampling model, and input the ordered and regularized 3D point cloud data into the trained neural network-based point cloud downsampling model to obtain simplified sampled 3D point cloud data. The point cloud downsampling model includes a key point extraction network and an adaptive neighborhood soft projection attention generation network. The key point extraction network includes a multi-layer perceptron layer, a pooling layer, and a fully connected layer. The adaptive neighborhood soft projection attention generation network includes an adaptive neighborhood soft projection layer and an attention layer; The required set of key points is obtained through the fully connected layer; In the adaptive neighborhood soft projection attention generation network, the input is the set of key points obtained by the key point extraction network; The classification module is used to classify the simplified sampled 3D point cloud data using the Pointnet network to obtain the final point cloud classification result.

Images