A point cloud feature learning auto-encoding network method based on a voxel grid fusion mechanism

Abstract

The application discloses a point cloud feature learning self-encoding network method based on a voxel grid fusion mechanism, and comprises the following steps: a data set composed of a point cloud data model is proportionally divided into a training set and a test set; a feature learning model based on the voxel grid fusion mechanism is constructed based on data in the training set; the feature learning model is trained to obtain a trained feature learning model; and data in the test set is input into the trained feature learning model to realize feature vector representation of the point cloud data model. The feature learning model based on the voxel grid fusion mechanism is used for feature learning of a three-dimensional shape, and the advantages of voxel and point cloud representation are considered, that is, the problems of low resolution of voxel representation and point cloud disorder are overcome.

Description

Technical Field

[0001] This invention belongs to the field of fully automated products and relates to a point cloud feature learning autoencoder network method based on a voxel grid fusion mechanism. Background Technology

[0002] 3D shape feature representation is one of the most important fundamental research areas in the field of 3D vision. Its strong universality allows for wide applications in areas such as 3D shape generation, shape retrieval, and shape restoration. [1-3] Applications such as...

[0003] Generally, 3D shape feature learning methods employ unsupervised learning to encode 3D models, ensuring that the structural and semantic information of the shape is preserved and compressed into a compact feature vector representation, called the 3D shape codeword. With the explosive growth in the size of 3D models in recent years, traditional feature representation methods have become increasingly inadequate for large-scale data. Furthermore, with the emergence of broader application areas, traditional feature extraction algorithms struggle to achieve wide applicability. This has led to the widespread research into data-driven autoencoder feature learning algorithms. Currently, autoencoder feature extraction algorithms for 3D models mainly fall into two categories: voxel-based and point cloud-based feature extraction algorithms.

[0004] Among them, voxel representation in deep learning is the most direct extension from 2D convolutional neural networks to 3D convolutional neural networks. It discretizes 3D space into regular occupancy grids. Classic works include generative and discriminative volumetric mesh modeling methods. [4] Scannet [5] and Voxnet [6] Method. Additionally, Wu et al. [7] This paper considers introducing adversarial generative mechanisms into 3D model feature learning and adopts a voxel grid representation to more easily incorporate image vision techniques into the algorithm. However, as the resolution of the voxel grid model increases, the number of parameters in the neural network framework increases exponentially, which obviously requires a large amount of computational resources. To address the representation redundancy problem of voxel representation, some alternative methods include octree representation. [8-9] and the representation of kd-trees

[10] An optimization model representation has been proposed. This type of method primarily focuses on improving the efficiency of voxel mesh representation by addressing the data representation itself. Additionally, Li et al.

[11] This paper considers using energy field representation and the idea of ​​a grasper to replace traditional convolution operations, thereby improving the efficiency of convolution operations and making them more suitable for voxel representation. However, such improved algorithms have certain limitations. Due to the complexity of the design method, it is difficult to extend and transfer them to more general feature learning problems.

[0005] In the past three years, deep learning methods based on point cloud representations have been proposed, and they have been widely applied in various fields due to their simplicity, efficiency, flexible representation, and small footprint. Among these, PointNet is a classic framework.

[14] PointNet++

[13] PointCNN

[15] The aforementioned framework represents pioneering work in point cloud deep learning, laying the foundation for the development of deep learning methods in the field of geometric graphics. Regarding unsupervised feature learning, Achlioptas et al.

[12] This paper first proposes an effective point cloud autoencoder generation model for 3D model feature learning. In the encoding stage, the method sorts the input point cloud and uses a one-dimensional convolutional neural network (CNN) to extract features from the point cloud sequence; while in the decoding stage, a three-layer fully connected layer is used to reconstruct the model.

[12] The unsupervised feature learning method was validated using a simple point cloud framework, demonstrating that it can still perform well on the ShapeNet dataset.

[22] The method was migrated to the classification database ModelNet40 / 10 (MN40 / 10). However, this method only uses a simple encoding structure and a fully connected layer for decoding, and this sorting mechanism cannot effectively handle the problem of point cloud disorder. Subsequently, Yang et al.

[16] A more effective autoencoder feature extraction framework is proposed, which completes model reconstruction by folding and deforming a two-dimensional grid. However, this method mainly concerns the model generation problem. While the folding idea improves the decoder's ability to reconstruct the model, it does not solve the problem of the disorder of point clouds. It only extracts global features of the point cloud model through max pooling operation.

[0006] The ideas of Le et al. are quite similar to those of the applicant.

[17] The proposed PointGrid model considers sampling an equal number of point clouds in each voxel grid as the initial input to the model, thus ensuring that high-level features are effectively learned. However, unlike other methods, this is a supervised classification learning problem. Summary of the Invention

[0007] To address the aforementioned problems, the technical solution adopted in this invention is: a point cloud feature learning autoencoder network method based on a voxel grid fusion mechanism, comprising the following steps:

[0008] The dataset consisting of point cloud data models is divided into training and testing sets according to a certain ratio.

[0009] A feature learning model based on a voxel grid fusion mechanism is constructed using data from the training set.

[0010] The feature learning model is trained to obtain a well-trained feature learning model;

[0011] The test set data is input into the trained feature learning model to realize the feature vector representation of the point cloud data model.

[0012] Furthermore, the feature learning model includes a point cloud multi-scale feature extraction module for extracting features from a point cloud data model, a multi-scale feature to voxel grid feature conversion module, and a voxel grid autoencoder feature extraction module that uses the regularization properties of voxel grids to convert the multi-scale features of the point cloud model into voxel structural features.

[0013] The point cloud multi-scale feature extraction module, the multi-scale feature conversion module to voxel mesh feature conversion module, and the voxel mesh autoencoder feature extraction module are cascaded in sequence.

[0014] Furthermore, the feature extraction of the point cloud data model employs a local feature integration method to extract local features of the sampling points in that layer.

[0015] The point cloud multi-scale feature extraction module includes a sampling layer, a grouping layer, a point cloud feature extraction layer, and a max pooling layer; the sampling layer, grouping layer, point cloud feature extraction layer, and max pooling layer are cascaded in sequence.

[0016] The sampling layer is mainly responsible for selecting the sparser convolution center points in the point cloud model; the grouping layer is mainly used to construct the local coordinate system; and finally, the point cloud feature extraction layer extracts the local features of each sample for each local point cloud data through multilayer perceptron and max pooling layer operations.

[0017] Furthermore, the voxel grid autoencoder feature extraction module adopts a local convolution operation at the center of the voxel grid to gather multi-scale point cloud information in space onto the center point of the voxel grid through a learnable kernel fusion mode, thereby fusing important information without destroying the spatial relationship of the point cloud.

[0018] Furthermore, the process of converting the multi-scale features into voxel mesh features is as follows:

[0019] The transformation voxel size is a 15×15×15 volume representation, and the goal is to generate a volume feature representation. Here c f Represents the dimension of volume features;

[0020] First, the center point (c) of each cell in the voxel grid is... i x ,c j y ,c k z ) is defined as a set

[0021] Therefore, for each center point, c i,j,k =(c i x ,c j y ,c k z Obtain different scales within its local neighborhood. n The coordinates of feature points and their point cloud feature representations in (n = 1, 2, 3) and Therefore, for scale l n To obtain a learnable representation of the fusion features:

[0022]

[0023] The fusion features at the same scale are represented as

[0024] Therefore, the multi-scale volume mesh fusion features are obtained through concatenation:

[0025]

[0026] Wherein: F v g It is composed of point cloud features and global features F g Repeated generation; therefore, each cell of the voxel mesh stores multi-scale feature information of the point cloud locality;

[0027] A serial deformation mechanism of three-dimensional volume mesh is adopted to ensure that the volume mesh feature generation model reconstructs the input model.

[0028] Furthermore: the process by which the voxel mesh autoencoding feature extraction module encodes and decodes volumetric features using 3D convolution operations is as follows:

[0029] A volume representation-based 3D convolution and deconvolution operation is employed to achieve a compact 3D codeword extraction process. The input for this stage is a volume feature representation transformed from a point cloud, with a voxel feature representation of size 15×15×15.

[0030] The encoding network consists of four convolutional neural network layers with kernel sizes of 5, 5, 3, and 3. The last layer uses a 1×1 convolution operation to compress the feature map into a 540-dimensional feature representation. Each convolutional layer employs batch normalization and the ELU activation function, which is expressed as:

[0031]

[0032] No pooling or fully connected layers were used in the voxel encoding stage. The extracted 540-dimensional features were input into the decoding module, which employed four deconvolution layers with kernel sizes of 3, 3, 5, and 5, and finally used a tanh activation function.

[0033] This invention presents a point cloud feature learning autoencoder network method based on a voxel mesh fusion mechanism. The provided voxel fusion-based point cloud feature extraction framework combines the advantages of both point cloud feature extraction models and voxel representations, overcoming the disorder problem inherent in point cloud representations while simultaneously extracting point cloud features and ensuring sufficient resolution. This method introduces a learnable voxel feature transformation mechanism to transfer feature representation from point cloud representation to volume representation, and relies on geometric distance constraints to ensure the stability of the transformation. Experimental results show a significant improvement compared to feature extraction using only a basic point cloud framework. Furthermore, compared to some existing cutting-edge unsupervised feature volume computation algorithms, the method in this application exhibits superior performance.

[0034] The objective of this application is to extract compact feature codewords from three-dimensional shapes. This method can effectively overcome the problem of disorder in point clouds.

[0035] This application studies the unsupervised feature extraction problem, and considers directly learning features from point clouds and converting point cloud features to volumetric features, and then implementing feature extraction through an unsupervised autoencoder network. The framework first extracts multi-scale point-by-point features through a point cloud network structure, thus ensuring effective extraction of multi-scale detailed features. Furthermore, a learnable fusion mechanism is used to convert point cloud features into voxel feature representations while preserving spatial local information, and reconstruction constraints are used to ensure that the converted features retain effective model details. Finally, an effective convolution-deconvolution mechanism is used to complete feature extraction and reconstruction. The main contribution of the point cloud feature learning framework based on voxel mesh fusion mechanism proposed in this application is...

[0036] 1) An end-to-end point cloud feature learning model is proposed. The module for converting point cloud features to volume mesh features directly transforms unordered point cloud features into ordered voxel feature representations and extracts feature codewords with certain generalization.

[0037] 2) An effective method for converting point cloud features to volumetric features is proposed. This method effectively considers the spatial location information of point clouds during the point cloud feature fusion process through a locally learnable feature fusion kernel and geometric constraints, and ensures that multi-scale point cloud features are effectively converted into volumetric representation features, while preserving important details of the model.

[0038] 3) The effectiveness of the proposed method was demonstrated through training on the ShapeNet dataset and classification testing of the 3D model on the MN40 / 10 dataset. Furthermore, ablation experiments validated the advantages of the voxel fusion mechanism.

[0039] Voxelized meshes and point cloud representations have wide applications in 3D deep learning and perform well in related fields. However, both representations have inherent advantages and disadvantages: point cloud representations are space-efficient and flexible, but lack structural relationships and are disordered; while voxel representations are redundant, but regular. This application proposes a feature learning model with a voxel mesh fusion mechanism for 3D shape feature learning, combining the advantages of both voxel and point cloud representations, thus overcoming the low resolution of voxel representations and the disorder of point clouds. First, the feature learning model introduces a voxel representation transformation module to transform the multi-scale feature pool extracted from the point cloud network into ordered voxel features. This module uses a local feature fusion strategy and geometric shape loss constraints to supervise the fusion process. Finally, the fused voxel features learn compact shape feature representation codewords through a 3D convolutional network. Experiments show that this method is highly competitive with existing unsupervised learning algorithms in shape classification problems. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a feature learning framework diagram based on the voxel grid fusion mechanism;

[0042] Figure 2 It is a feature autoencoder structure diagram based on voxel representation;

[0043] Figure 3 It is a visualization of the training process;

[0044] Figure 4 This is a visualization of the feature codewords of the model proposed in this application in clustering based on the t-SNE method. Detailed Implementation

[0045] It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of the present invention can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0046] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the present invention or its application or use. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0048] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of the invention. It should also be understood that, for ease of description, the dimensions of the various parts shown in the drawings are not drawn to actual scale. Techniques, methods, and devices known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and devices should be considered part of the specification. In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters in the following figures denote similar items; therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0049] In the description of this invention, it should be understood that the orientation or positional relationship indicated by directional terms such as "front, back, up, down, left, right", "horizontal, vertical, horizontal" and "top, bottom" is generally based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing this invention and simplifying the description. Unless otherwise stated, these directional terms do not indicate or imply that the device or element referred to must have a specific orientation or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on the scope of protection of this invention. The directional terms "inner" and "outer" refer to the inner and outer contours relative to the outline of each component itself.

[0050] For ease of description, spatial relative terms such as "above," "over," "on the upper surface of," "above," etc., are used herein to describe the spatial positional relationship of a device or feature as shown in the figures to other devices or features. It should be understood that spatial relative terms are intended to encompass different orientations in use or operation besides the orientation of the device as described in the figures. For example, if the device in the figures is inverted, a device described as "above" or "above" other devices or structures would subsequently be positioned as "below" or "under" other devices or structures. Thus, the exemplary term "above" can include both "above" and "below." The device may also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatial relative descriptions used herein will be interpreted accordingly.

[0051] Furthermore, it should be noted that the use of terms such as "first" and "second" to define components is merely for the purpose of distinguishing the corresponding components. Unless otherwise stated, the above terms have no special meaning and therefore should not be construed as limiting the scope of protection of this invention.

[0052] An autoencoder network for point cloud feature learning based on a voxel mesh fusion mechanism includes the following steps:

[0053] S1: The dataset consisting of the point cloud data model is divided into training set and test set according to the proportion;

[0054] S2: Construct a feature learning model based on a voxel grid fusion mechanism using data from the training set;

[0055] S3: Train the feature learning model to obtain a trained feature learning model;

[0056] S4: Input the test set data into the trained feature learning model to realize the feature vector representation of the point cloud data model.

[0057] S1 / S2 / S3 / S4 are executed sequentially;

[0058] The feature learning model includes a point cloud multi-scale feature extraction module for extracting features from a point cloud data model, a multi-scale feature to voxel grid feature conversion module, and a voxel grid autoencoder feature extraction module that uses the regularization properties of voxel grids to convert the multi-scale features of the point cloud model into voxel structure features.

[0059] The point cloud multi-scale feature extraction module, the multi-scale feature to voxel mesh feature conversion module, and the voxel mesh autoencoder feature extraction module are cascaded in sequence.

[0060] The feature extraction of the point cloud data model employs a local feature integration method to extract local features of the sampling points in that layer.

[0061] The point cloud multi-scale feature extraction module includes a sampling layer, a grouping layer, a point cloud feature extraction layer, and a max pooling layer; the sampling layer, grouping layer, point cloud feature extraction layer, and max pooling layer are cascaded in sequence.

[0062] The sampling layer is mainly responsible for selecting the sparser convolution center points in the point cloud model; the grouping layer is mainly used to construct the local coordinate system; and finally, the point cloud feature extraction layer extracts the local features of each sample for each local point cloud data through multilayer perceptron and max pooling layer operations.

[0063] The framework process of this application is as follows: Figure 1 As shown. The model mainly consists of three parts: a point cloud multi-scale feature extraction module, a multi-scale feature to voxel grid feature conversion module, and a voxel grid autoencoder feature extraction module. Among them: (1) the extraction of multi-scale features can ensure the preservation of the model's detailed information and global information; (2) the voxelization of point cloud features preserves the fine features of the point cloud and integrates the disordered point cloud features into a regular feature representation. In addition, the method adopts two-stage shape loss constraints in the training stage to ensure the preservation of the model's detailed information and positional information during the point cloud feature to voxel feature conversion process.

[0064] 1.1 Point Cloud Degree Scale Feature Extraction

[0065] like Figure 1 As shown, the input to this network framework is a point cloud representation containing 2048 points. For ease of explanation, we use a matrix to represent the input 3D model point cloud, denoted as X∈R. 2048×3 For feature extraction from point cloud models, PointNet++ is used in this paper.

[13] The framework is as follows: point clouds are sampled layer by layer, and within each layer, a local feature integration method is used to extract local features of the sampled points during feature extraction. The sampling layer is mainly responsible for selecting the sparser convolution center points in the point cloud model; the grouping layer is mainly used to construct a local coordinate system; finally, the point cloud feature extraction layer extracts the local features of each sampled point cloud data using a multilayer perceptron and max pooling operations. In the point cloud network structure of this application, the number of sampling points in each layer is 1024, 348, and 128, respectively. Through layer-by-layer feature compression extraction, this method can extract feature representations of the point cloud model at different scales. As the number of layers increases, more abstract global features are extracted. In this application's method, a three-layer network structure is adopted, and the global features of the model are finally extracted through max pooling operations. Therefore, the different scale feature representations obtained by this point cloud network and the global feature representation of the model constitute a multi-scale feature pool representation, denoted as: and global feature F g ∈R 1×1024 .

[0066] 1.2 Voxel Mesh Feature Generation

[0067] Voxel mesh representation has wide applications in 3D deep learning. This application considers leveraging the regularization properties of voxel meshes to transform multi-scale features of point cloud models into voxel structural features, thereby preserving detailed information while achieving stronger regularity. Considering that multi-scale point cloud feature representation is achieved through multiple downsampling processes, feature maps of different sizes are generated. The feature learning model in this application employs a local convolution operation at the voxel mesh center, aggregating multi-scale point cloud information in space to the voxel mesh center point through a learnable kernel fusion pattern, thus fusing important information without disrupting the spatial relationships of the point cloud.

[0068] Specifically, the default voxel size in this application is a 15×15×15 volume representation, while the goal is to generate a volume feature representation. Here c f This represents the volumetric feature dimension. For ease of description later, the center point (c) of each cell in the voxel grid is first defined. i x ,c j y ,c k z ) is defined as a set Therefore, for each center point, c i,j,k =(c i x ,c j y ,c k zObtain different scales within its local neighborhood. n The coordinates of feature points and their point cloud feature representations in (n = 1, 2, 3) and Therefore, for scale l n This allows us to obtain a learnable representation of the fusion features:

[0069]

[0070] Therefore, we represent the fusion features at the same scale as... Therefore, multi-scale volume mesh fusion features can be obtained through concatenation:

[0071]

[0072] Where F v g It is composed of point cloud features and global features F g Repeated generation is employed. Therefore, each cell of the voxel mesh stores multi-scale feature information of the point cloud's local area. Furthermore, to ensure the preservation of important information during feature transformation, a serial deformation mechanism of the 3D volume mesh is used to guarantee the volume mesh feature generation model. It can accurately reconstruct the input model X.

[0073] In the encoding and decoding stage of the volumetric mesh features, we employ 3D convolutional and deconvolutional operations based on volume representation to achieve a compact 3D codeword extraction process. The input to this stage is a volumetric feature representation transformed from a point cloud, with a voxel feature representation of size 15×15×15. The encoding network consists of four convolutional neural network layers with kernel sizes of 5, 5, 3, and 3, respectively. The last layer uses a 1×1 convolutional operation to compress the feature map into a 540-dimensional feature representation. In each convolutional layer, we employ batch normalization and the ELU activation function. The ELU activation function is expressed as:

[0074]

[0075] Note that this application does not use pooling or fully connected layers in the voxel encoding stage. Immediately afterwards, the extracted 540-dimensional features are input to the decoding module. This application employs four deconvolution layers with kernel sizes of 3, 3, 5, and 5. Finally, a tanh activation function is used. The flowchart of the volume convolution operation is shown below. Figure 2 As shown.

[0076] This application primarily employs two geometric loss functions for training the network model. These functions are used to ensure the preservation of model details during the construction of voxel mesh features from point cloud features and to constrain the consistency of the final autoencoder network reconstruction. Figure 1As shown. For a given input model X, the model generated during the feature transformation stage... and models generated by autoencoder networks Considering that the point cloud generated by the method in this application has a different number of point clouds than the input model, an extended Chamfer Distance (CD) is used to measure the difference between the models:

[0077]

[0078] It is important to note that X and The error between them is the same as in the formula above. Furthermore, considering that the reconstructed point cloud should have a relatively uniform distribution, this application introduces a repulsion loss function to ensure that model points do not cluster in a local area.

[0079]

[0080] Here, K(·) represents the set of k-nearest neighbors, and the distance d(·,·) between two points represents the L2 distance between them. η(r) = -r represents the penalty applied when the k-nearest neighbors are very close. It is a rapid descent function.

[0081] Therefore, by unifying the above loss functions, the overall loss function of the model trained in this application is a joint loss function:

[0082]

[0083]

[0084] The default parameter settings here are: α = 0.7, β = 0.01 and γ = 0.0001.

[0085] The following are the experimental results and analysis.

[0086] 1.1 Database and Experiment Setup

[0087] This network structure uses Tensorflow.

[21] Training will be performed. This section will demonstrate through experiments that the designed network has effective feature extraction capabilities. In the visualization experiments, ShapeNet will be used.

[22] Part dataset

[19] The training dataset contains 16 classes of point cloud models, providing a rich variety of model categories. Additionally, the ShapeNet dataset was used for unsupervised classification testing.

[22] The dataset contains 55 classes and approximately 57,000 models, a large scale that ensures the autoencoder network is adequately trained, improving the generalizability of extracted features. Classification tests were conducted using the ModelNet40 and ModelNet10 datasets. The ModelNet40 dataset contains 40 classes, with 9843 training models and 3991 testing models, while the ModelNet10 dataset contains 10 classes, with 2468 training models and 909 testing models. The classification results on these datasets demonstrate the discriminability of the extracted features. The algorithm in this application runs on an NVIDIA 1080Ti with 11GB of GPU memory.

[0088] 1.2 Visualizing the Training Process

[0089] To clearly illustrate the process by which the autoencoder network learns feature representations in this application, Figure 3 A visualization of the iterative training process is presented, showing the output visualizations of the network at multiple stages. Specifically, the reconstruction results of four different types of models under different iteration cycles are shown. Here, we used the ADMA training strategy to train the network, setting the initial learning rate to 0.001, the batch size to 1, and decreasing the learning rate every 50k iterations with a decay rate of 0.7. We present the results for training for 1, 5, 10, 20, and 40 cycles. It can be seen that as the iteration cycle increases, the model structure and detailed reconstruction are more consistent with the input results. The second column shows the reconstruction results in voxel grid representation, which effectively captures the important structural and detailed information of the model. Furthermore, Figure 2 The red dashed box in the middle shows the fusion result of the two-stage reconstruction after training. The algorithm effectively preserves the model details and overall structure.

[0090] 1.3 Point Cloud Feature Clustering

[0091] This application presents clustering results of feature representations of point cloud models under unsupervised training, using the ShapeNet database.

[22] A feature learning model was trained, and the trained model was used to extract features from the ModelNet10 data model. The t-SNE visualization method was employed.

[20] High-dimensional codewords are embedded into a two-dimensional space. The perplexity parameter, used for visualization, is set to 50 by default. From... Figure 4It can be observed that the feature representations (represented by different colors) learned by the algorithm for different categories can be easily separated, with only a few models showing poor discriminative ability. This may be because the blue-represented "night stand" model and the green "dresser" model in the model set have strong similarities, and the light green-represented "table" model and the pink "desk" model also show similarities. Figure 4 As shown in the red dashed box.

[0092] Figure 4 The feature codewords of the proposed model in this application are visualized in the clustering results based on the t-SNE method.

[0093] 1.4 Transfer Classification Results

[0094] To evaluate the representational ability of the network structure extracted features in this application, similar to references

[12]

[16] , we transferred the autoencoder training model to a classification model library to evaluate the classification ability of the extracted features. First, for the autoencoder network proposed in this application, we used the ShapeNet dataset for training. Here, we used a batch size of 2, and the other parameters were the same as before. In order to improve the diversity of the model, we adopted a strategy of random rotation along the coordinate axis during training, so that the orientation of the input model during the training phase is random. After training, an unsupervised feature extraction autoencoder network can be obtained. Then, through the forward propagation of the trained network, the 540-dimensional feature codewords extracted by the MN40 / 10 model of the classification dataset are calculated. It should be noted here that the autoencoder network used is only trained on ShapeNet and the classification network is not trained. For the extracted features, we trained a shallow classifier using a simple SVM to classify the test models in the ModelNet dataset. Table 2 shows the classification results of related unsupervised models in recent years. As can be observed from the table, the method proposed in this application achieves outstanding classification accuracy on ModelNet40, and also achieves state-of-the-art classification accuracy on ModelNet10. However, since the ModelNet10 dataset is not publicly available, our sampled point cloud results will have some impact on the final results.

[0095] Table 1 Comparison of ablation experiments using the method described in this application.

[0096] method Accuracy (%) (1) Remove the geometric loss constraint in the feature transformation fusion stage 85。5％ (2) Remove the complete voxel fusion module 73。39％ (3) Complete model (the algorithm proposed in this application) 86。6％

[0097] 1.5 Comparison of ablation experiments

[0098] To verify the rationality of the proposed model, this application discusses ablation experiments. The experiments were conducted using the ModelNet40 dataset, employing the unsupervised training strategy described earlier, and utilizing the SVM algorithm for feature extraction training and testing.

[0099] First, to verify the effectiveness of using shape distance to constrain the extracted features during the fusion stage, this application removes the constraint of this loss function during the training stage. The test results are given in Table 1 (1): the test accuracy is somewhat lower than that of the complete model (3) of this application. In addition, this application also removes the fusion module based on the voxel mechanism, degenerating the algorithm into a common autoencoder model that only relies on the point cloud framework itself to extract features. As shown in Table 1 (2), the test accuracy is clearly visible. It can be seen that the result is greatly reduced when relying only on the point cloud model itself and lacking the voxel fusion mechanism. This also verifies the effectiveness of the algorithm framework proposed in this application.

[0100] Table 2 compares the classification accuracy of this application with other unsupervised methods.

[0101] method MN40 MN10 <![CDATA[SPH

[23] ]]> 68。2％ 79。8％ <![CDATA[LFD

[24] ]]> 75。5％ 79。9％ <![CDATA[T-L Network

[25] ]]> 74。4％ - <![CDATA[Vcon-DEA

[26] ]]> 75。5％ 80。5％ <![CDATA[3D-GAN [7] ]]> 83。3％ 91。0％ <![CDATA[Latent-GAN

[12] ]]> 85。7％ 95。3％ <![CDATA[3D-ED-GAN

[26] ]]> 84。3％ 89。2％ Ours 86。6％ 90。1％

[0102] 1.6 Model Limitations and Discussion

[0103] Although the feature extraction framework based on voxel fusion offers a significant improvement over frameworks that only use point clouds for feature extraction, it still suffers from insufficient accuracy and large errors in classification problems, similar to other current algorithms. Furthermore, this model struggles to effectively reconstruct models with rich structures and distinguishes between similar models within a model set. Additionally, the framework proposed in this application does not ideally preserve details of complex input models, proving effective only for simple models.

[0104] This application proposes a point cloud feature extraction framework based on a voxel fusion mechanism, which combines the advantages of point cloud feature extraction models and voxel representations. It overcomes the disorder problem inherent in point cloud representations while simultaneously extracting point cloud features and ensuring sufficient resolution. This method introduces a learnable voxel feature transformation mechanism to transfer feature representation from point cloud representation to volume representation, and relies on geometric distance constraints to ensure the stability of the transformation. Experimental results show a significant improvement compared to feature extraction using only a basic point cloud framework. Furthermore, compared to some existing state-of-the-art unsupervised feature volume computation algorithms, the method in this application demonstrates superior performance.

[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

[0106] [1]Wang W,Huang Q,You S,Yang C,Neumann U。Shape inpainting using

[0107] 3d generative adversarial network and recurrent convolutionalnetworks。In Proceedings of the IEEE International Conference on ComputerVision,2017:2298-2306。

[0108] [2]Stutz D,Geiger A。Learning 3d shape completion under weaksupervision。

[0109] International Journal of Computer Vision。2018:1-20。

[0110] [3]Dai A,Ruizhongtai Qi C,Nieβner M。Shape completion using

[0111] 3d-encoder-predictor cnns and shape synthesis。InProceedings of theIEEE Conference on Computer Vision and Pattern Recognition 2017:5868-5877。

[0112] [4]Brock A,Lim T,Ritchie JM,Weston N。Generative and discriminativevoxel modeling with convolutional neural networks。arXiv preprint arXiv:1608。04236。2016

[0113] [5]Dai A,Chang AX,Savva M,Halber M,Funkhouser T,Nieβner M。

[0114] Scannet:Richly-annotated 3d reconstructions of indoor scenes。InProceedings of the IEEE Conference on Computer Vision and PatternRecognition 2017:5828-5839。

[0115] [6]Maturana D,Scherer S。Voxnet:A 3d convolutional neural network forreal-time object recognition。In2015 IEEE / RSJ International Conference onIntelligent Robots and Systems(IROS)2015:922-928

[0116] [7]Wu J,Zhang C,Xue T,Freeman B,Tenenbaum J。Learning a probabilisticlatent space of object shapes via 3d generative-adversarial modeling。

[0117] InAdvances in neural information processing systems 2016:82-90。

[0118] [8]Wang PS,Liu Y,Guo YX,Sun CY,Tong X。O-cnn:Octree-basedconvolutional neural networks for 3d shape analysis。ACM Transactions onGraphics(TOG)。2017,36(4):1-1。

[0119] [9]Riegler G,Osman Ulusoy A,Geiger A。Octnet:Learning deep 3drepresentations at high resolutions。InProceedings of the IEEE Conference onComputer Vision and Pattern Recognition 2017:3577-3586。

[0120]

[10] Klokov R,Lempitsky V。Escape from cells:Deep kd-networks for therecognition of 3d point cloud models。InProceedings of the IEEE InternationalConference on Computer Vision 2017:863-872。

[0121]

[11] Li Y,Pirk S,Su H,Qi CR,Guibas LJ。Fpnn:Field probing neuralnetworks for 3d data。InAdvances in Neural Information Processing Systems2016:307-315。

[0122]

[12] Achlioptas P,Diamanti O,Mitliagkas I,Guibas L。Representationlearning and adversarial generation of 3d point clouds。arXiv preprint arXiv:1707。02392。2017。

[0123]

[13] Qi CR,Yi L,Su H,Guibas LJ。Pointnet++:Deep hierarchical featurelearning on point sets in a metric space。InAdvances in neural informationprocessing systems 2017:5099-5108。

[0124]

[14] Qi CR,Su H,Mo K,Guibas LJ。Pointnet:Deep learning on point setsfor

[0125] 3d classification and segmentation。InProceedings of the IEEEconference on computer vision and pattern recognition 2017:652-660。

[0126]

[15] Li Y,Bu R,Sun M,Wu W,Di X,Chen B。Pointcnn:Convolution on x-transformed points。InAdvances in neural information processing systems2018:820-830。

[0127]

[16] Yang Y,Feng C,Shen Y,Tian D。Foldingnet:Point cloud auto-encodervia deep grid deformation。InProceedings of the IEEE Conference on ComputerVision and Pattern Recognition 2018:206-215。

[0128]

[17] Le T,Duan Y。Pointgrid:A deep network for 3d shape understanding。

[0129] InProceedings of the IEEE conference on computer vision and patternrecognition 2018:9204-9214。

[0130]

[18] Yu L,Li X,Fu CW,Cohen-Or D,Heng PA。Pu-net:Point cloud upsamplingnetwork。InProceedings of the IEEE Conference on Computer Vision and PatternRecognition 2018:2790-2799。

[0131]

[19] Yi L,Kim VG,Ceylan D,Shen IC,Yan M,Su H,Lu C,Huang Q,Sheffer A,Guibas L。A scalable active framework for region annotation in 3d shapecollections。ACM Transactions on Graphics(TOG)。2016,35(6):1-2。

[0132]

[20] Maaten LV,Hinton G。Visualizing data using t-SNE。Journal ofmachine learning research。2008:2579-605。

[0133]

[21] Abadi M,Barham P,Chen J,Chen Z,Davis A,Dean J,Devin M,Ghemawat S,Irving G,Isard M,Kudlur M。Tensorflow:A system for large-scale machinelearning。In12th{USENIX}Symposium on Operating Systems Design andImplementation({OSDI}16)2016:265-283。

[0134]

[22] Chang AX,Funkhouser T,Guibas L,Hanrahan P,Huang Q,Li Z,SavareseS,Savva M,Song SR,Su Hao and others。Shapenet:An information-rich 3d modelrepository。arXiv preprint arXiv:1512。03012。2015。

[23] Kobbelt L,Schrder P,Kazhdan M,Funkhouser,T,and Rusinkiewicz,S。Rotation invariant sphericalharmonic representation of 3d shape descriptors。Eurographics Symposium onGeometry Processing。2003,43(2):156-164。

[0135]

[24] Chen DY,Tian XP,Shen YT and others。On Visual Similarity Based 3DModel Retrieval。John Wiley and Sons,Ltd。2003:223-232。

[0136]

[25] Girdhar R,Fouhey DF,Rodriguez M and Guptar A。Learning aPredictable and Generative Vector Representation for Objects。2016。

[0137]

[26] Sharma A,Grau O,Fritz M。Vconv-dae:Deep volumetric shape learningwithout object labels。InEuropean Conference on Computer Vision 2016:236-250)。

[0138]

[27] Wang W,Huang Q,You S,Yang C,Neumann U。Shape inpainting using3dgenerative adversarial network and recurrent convolutional networks。InProceedings of the IEEE International Conference on Computer Vision 2017:298-2306。

Claims

1.A point cloud feature learning auto-encoding network method based on a voxel grid fusion mechanism, characterized by: The method comprises the following steps: The data set composed of point cloud data models is divided into a training set and a test set according to a proportion; A feature learning model based on a voxel grid fusion mechanism is constructed based on the data in the training set; The feature learning model is trained to obtain a trained feature learning model; Data in the test set is input into the trained feature learning model to realize feature vector representation of the point cloud data model; The feature learning model comprises a point cloud multi-scale feature extraction module for extracting features of the point cloud data model, a multi-scale feature to voxel grid feature conversion module and a voxel grid auto-encoding feature extraction module for converting multi-scale features of the point cloud model into voxel structure features by using the regularizing characteristics of the voxel grid; The point cloud multi-scale feature extraction module, the multi-scale feature to voxel grid feature conversion module and the voxel grid auto-encoding feature extraction module are sequentially cascaded; The process of converting the multi-scale features into voxel grid features is as follows: The transform converts the vector representation into a volumetric representation The goal is to generate a volumetric representation of the object Here The volumetric representation has a dimensionality First, the center point of each cell in the voxel grid is defined as a set of points defined as a set ; Thus for each center point, the feature point coordinates and point cloud feature representation of the feature points in its local neighborhood are obtained at different scales and and , thus for scale , a learnable fused feature representation is obtained: The fused features in the same scale are represented as ; Thus, the multi-scale voxel grid fusion features are obtained by concatenation: wherein: is a global feature from the point cloud feature is repeatedly generated; thus, the multi-scale feature information of the point cloud is stored in each cell of the voxel grid; A three-dimensional voxel grid concatenation deformation mechanism is adopted to ensure that the voxel grid feature generation model reconstructs the input model. 2.The point cloud feature learning auto-encoding network method based on voxel grid fusion mechanism according to claim 1, characterized in that: The local feature integration method is adopted to extract local features of the sampling points, The point cloud multi-scale feature extraction module comprises a sampling layer, a grouping layer, a point cloud feature extraction layer and a max-pooling layer; the sampling layer, the grouping layer, the point cloud feature extraction layer and the max-pooling layer are sequentially cascaded; The sampling layer is mainly responsible for selecting more sparse convolution center points in the point cloud model; the grouping layer is mainly used to construct a local coordinate system; finally, the point cloud feature extraction layer extracts local features of each sampling point by a multi-layer perception machine and a max-pooling layer operation. 3.The method of claim 1, wherein the method further comprises: The voxel grid auto-encoding feature extraction module adopts a local convolution operation mode of a voxel grid center, and aggregates point cloud multi-scale information in space to the voxel grid center point through a learnable kernel fusion mode, so as to fuse important information without destroying the spatial relationship of the point cloud. 4.The method of claim 1, wherein the method further comprises: The voxel grid auto-encoding feature extraction module uses a 3D convolution operation to encode and decode the voxel features as follows: Three-dimensional convolution operations and deconvolution operations based on volume representation are adopted to realize a compact code word extraction process of three-dimensional shapes. The input of this stage is a point cloud converted volume feature representation, and the input size is a voxel feature representation of ​ The coding network is composed of four layers of convolutional neural networks, and the convolution kernel sizes are 5, 5, 3, and 3 respectively, and the last layer adopts Convolution operation, the feature map is compressed into a 540-dimensional feature representation, in each layer of convolution operation layer, batch normalization operation and ELU activation function are adopted, and the ELU activation function is represented as: In the voxel encoding stage, no pooling operation and full connection operation layer is used, and the extracted 540-dimensional features are input into the decoding module, four layers of deconvolution operation are adopted, the convolution kernel size is 3, 3, 5, 5 respectively, and finally a tanh activation function is adopted.

Images