Geometric invariant three-dimensional model classification algorithm based on smooth secondary loss

Abstract

The invention discloses a geometric invariant three-dimensional model classification algorithm based on smooth secondary loss, and the algorithm comprises the steps: S1, obtaining three-dimensional models of various product parts, carrying out the grid sampling of continuous parameter surfaces and edges in each three-dimensional model, and obtaining discretized parameter surfaces and edges; s2, randomly rotating each discretized edge and parameter surface, and adopting a product part type corresponding to the three-dimensional model, and the discretized parameter surface and edge before and after random rotation as a data set; s3, training a depth classification model by adopting the data set, wherein a loss function of the depth classification model comprises a smooth secondary loss function and a cross entropy loss function which are output based on the model of the parameters before and after rotation; and S4, performing grid sampling on continuous parameter surfaces and edges of the entity three-dimensional model of the to-be-identified product part, and inputting the trained depth classification model to obtain the type of the to-be-identified product part.

Description

Technical Field

[0001] The present invention belongs to the technical field of product part classification, and particularly relates to a geometric invariant three-dimensional model classification algorithm based on smooth quadratic loss. Background Art

[0002] With the rapid development of industrial manufacturing, the types and quantities of parts have shown an explosive growth. During the processes of production, assembly, and supply chain management, a large number of parts need to be accurately classified. However, the traditional manual classification method is inefficient, not only consuming a large amount of time but also being easily affected by human subjective factors, resulting in classification errors.

[0003] The prior art can greatly improve the efficiency and accuracy of three-dimensional part classification by leveraging the powerful data processing capabilities of artificial intelligence, thereby saving valuable time costs; in the three-dimensional model recognition task, model classification methods such as PointNet and PointNet++ have emerged; in Figure 1 Figure 13 shows the classification flowchart of PointNet, which is specifically used to process irregular point cloud data (such as 3D scanned point cloud or LiDAR data). The core idea of PointNet is to independently process the features of each point, and the network processes the entire point cloud by aggregating the features of all points. Each point is first processed through a shared MLP (Multi-Layer Perceptron) to generate the feature vector of that point. Due to the shared MLP, the processing of each point is the same, ensuring that the network is insensitive to the order of points, that is, it does not depend on the arrangement of points.

[0004] However, since the local structure in the point cloud is very important for understanding and recognizing the shape of the object, PointNet performs poorly in dealing with complex and fine-grained structures. After actual tests, it is found that its classification ability for complex industrial parts is very poor, and due to its model's lack of the ability to intervene and train abnormal points, when the training data is unevenly distributed close to the actual training category data volume, its algorithm's robustness is not excellent enough.

[0005] Figure 2 Figure 20 shows the classification flowchart of PointNet++. PointNet++ is an improvement of PointNet, aiming to further enhance the modeling ability of the local structure of point cloud data. Different from PointNet directly processing the entire point cloud, PointNet++ gradually aggregates the local information of the point cloud at different scales through a hierarchical structure. Each layer processes the points in a local area, and by extracting local features, it captures the finer geometric structure in the point cloud. This method can more effectively capture the local geometric information of the point cloud by dividing the point cloud into different local areas and extracting features within each area.

[0006] Although PointNet++ has significant advantages in local feature extraction, it still relies on sampling and aggregation in local regions. In extreme cases, if the density of the point cloud varies greatly, or there are some regions with extremely sparse points, PointNet++ has difficulty effectively extracting and aggregating the features of these sparse regions, which affects the classification results. At the same time, PointNet++ models the local structure in a relatively fine-grained manner, but its robustness to noise and outliers is not very strong. Summary of the Invention

[0007] In view of the above deficiencies in the prior art, the geometric invariant 3D model classification algorithm based on smooth quadratic loss provided by the present invention solves the problem of insufficient classification accuracy of existing 3D model recognition methods for complex 3D shape models.

[0008] In order to achieve the above invention objective, the technical solution adopted by the present invention is as follows:

[0009] Provide a geometric invariant 3D model classification algorithm based on smooth quadratic loss, which includes the steps of:

[0010] S1. Obtain 3D models of various product components, and perform mesh sampling on the continuous parametric surfaces and edges in each 3D model to obtain the discretized parametric surfaces and edges;

[0011] S2. Randomly rotate each discretized edge and parametric surface, and use the product component type corresponding to the 3D model, the discretized parametric surfaces and edges before and after random rotation as the data set;

[0012] S3. Use the data set to train the deep classification model, and the loss function of the deep classification model includes a smooth quadratic loss function and a cross-entropy loss function based on the model outputs of the parameters before and after rotation;

[0013] S4. Perform mesh sampling on the continuous parametric surfaces and edges of the solid 3D model of the product component to be recognized, and then input it into the trained deep classification model to obtain the type of the product component to be recognized.

[0014] Further, the deep classification model includes a CNN model, a graph neural network, and a multi-layer perceptron connected in sequence, and the expression of the loss function of the deep classification model is:

[0015]

[0016] Among them, is the loss function of the deep classification model; is the cross-entropy loss function; is the smooth quadratic loss function; C is the total number of types of product components; y c is the label of the c-th type of product component; Pc is the probability predicted by the deep classification model for the product component belonging to the c-th category; B is the training batch size; β′ is the smoothing degree adjustment parameter; and is the i-th dimensional data output by the graph neural network for the data before and after random rotation of the n-th 3D model; is and is the cosine similarity of.

[0017] Furthermore, the method for randomly rotating each discretized edge and parameter surface includes:

[0018] Randomly select a rotation angle, and calculate the rotation matrices for the discretized edge and parameter surface to rotate along the X, Y, and Z axes of the spatial coordinate system according to the rotation angle:

[0019]

[0020] where R X (θ), R Y(θ) and R Z (θ) are the rotation matrices for the X, Y, and Z axes respectively; θ is the rotation angle;

[0021] For each edge and parameter surface, randomly select a direction among the X, Y, and Z axes as the rotation axis;

[0022] According to the selected rotation axis, act on the three-dimensional geometric coordinates P and normal vector feature N of the parameter surface, and act on the three-dimensional geometric coordinates P and tangent vector feature T of the edge to obtain the rotated features:

[0023] E' fl [P,T] = R l (θ)·E f [P,T]

[0024] S' fl [P,N] = R l (θ)·S f [P,N]

[0025] where R l (θ) is the rotation matrix for the l axis, and l takes values of X, Y, and Z; E f and S f are the discretized edge and parameter surface respectively; E' fl and S' fl are the features of E f and S f rotated along the l axis.

[0026] Further, the CNN model includes a first convolutional layer, a second convolutional layer, a third convolutional layer, an average pooling layer, and a fully connected layer connected in sequence. The input channels, output channels, and convolutional kernel sizes of the first convolutional layer, the second convolutional layer, and the third convolutional layer are (6, 64, 3), (64, 128, 3), and (128, 256, 3) respectively. The input dimension and hidden layer dimension of the fully connected layer are 256 and 64 respectively;

[0027] The discretized parameter surfaces and edges before and after the rotation of each 3D model are fused through multiple channels of the CNN model to obtain the surface structure feature vectors and connecting edge feature vectors before and after the rotation.

[0028] Further, the multi-layer GNN of the graph neural network performs stacked calculations on the information input by each 3D model, enabling the information of each face node to be transmitted between nodes, and obtaining topological features under the global view. The expressions for updating the face node features and edge features of each GNN layer are:

[0029]

[0030] Among them, and are the face node features output by the k-th and k - 1-th GNN layers respectively; σ and φ are the multi-layer perceptron layers for updating the face node features and edge features respectively; is the adjacent face feature of in the k - 1-th GNN layer, and N(u) is the set of all adjacent face features of , where v ∈ N(u); is the feature of the shared edge between and in the k-th GNN layer; is the feature of the shared edge between and output by the k - 1-th GNN layer; is the adjacent face feature of in the k-th GNN layer; β (k) is the feature for marking and distinguishing the face nodes in the k - 1-th GNN layer; γ (k) is the feature for marking and distinguishing the edges in the k - 1-th GNN layer; ⊙ is the dot product.

[0031] Further, the grid sampling of the continuous parameter surfaces and edges in each 3D model includes: sampling the parameter surfaces with a two-dimensional grid on the face nodes and sampling with a one-dimensional grid on each edge.

[0032] Furthermore, the geometric invariant three-dimensional model classification algorithm also includes performing a max pooling operation on the face node features output by the graph neural network using a pooling layer to extract global features, and then inputting the global features into a multi-layer perceptron for classification to obtain the type of the three-dimensional model.

[0033] The beneficial effects of the present invention are as follows: In the face of the demand for complex classification of industrial product parts, this solution starts from the direction of enhancing the rotational invariance of the deep classification model, and focuses on improving the problems of a large number of product categories, unbalanced numbers of products in each category, and relatively complex engineering part model files in the actual production scenario. Specifically: By randomly rotating and augmenting the original three-dimensional training data, this solution can not only introduce additional information into the model training, enrich the feature space that can be learned during the classification model training process, but also make up for the possible lack of data volume in the actual production scenario.

[0034] Based on the original and rotated three-dimensional model data, this solution designs a smooth quadratic loss function at the training level of the deep classification model, which can control the deep classification model to maintain the topological representation consistency of the original and rotated data during the training process, so as to regulate the model's recognition ability of the original features and rotated features, thereby ensuring its high classification consistency under different transformations, and at the same time improving the model's robustness to noise and outliers. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 is the classification flow chart of PointNet in the prior art.

[0036] Figure 2 is the classification flow chart of PointNet++ in the prior art.

[0037] Figure 3 is the overall architecture diagram of the geometric invariant three-dimensional model classification algorithm based on the smooth quadratic loss.

[0038] Figure 4 is the flow chart of the geometric invariant three-dimensional model classification algorithm based on the smooth quadratic loss. DETAILED DESCRIPTION OF THE INVENTION

[0039] The following describes the specific embodiments of the present invention to facilitate those skilled in the art of the present technology to understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art of the present technology, as long as various changes are within the spirit and scope of the present invention defined and determined by the appended claims, these changes are obvious, and all inventions created using the concept of the present invention are within the scope of protection.

[0040] Refer to Figure 4 , Figure 4The flowchart of a geometric invariant 3D model classification algorithm based on a smooth quadratic loss is shown; as Figure 4 shown, the method S includes steps S1 to S4.

[0041] In step S1, 3D models of various product components are obtained, and grid sampling is performed on the continuous parametric surfaces and edges in each 3D model to obtain the discretized parametric surfaces and edges; in this solution, the 3D model is preferably a B-rep format file.

[0042] When performing grid sampling on the parametric surfaces and edges, specifically, a two-dimensional grid is used to sample the parametric surfaces at the surface nodes, and a one-dimensional grid is used to sample each edge. For example, a 10*10 grid is used to sample the original parametric surface, and a 1*10 grid is used for the edges, so that the continuous parametric surfaces and edges are subjected to discretized feature extraction to describe the geometric topology information of each surface and edge.

[0043] The discretized parametric surface includes the coordinates of 3 channels and the normal vectors of 3 channels; the discretized edge includes the coordinates of 3 channels and the tangent vectors of 3 channels.

[0044] In step S2, each discretized edge and parametric surface are randomly rotated, and the product component type corresponding to the 3D model, the discretized parametric surfaces and edges before and after random rotation are used as the data set.

[0045] In an embodiment of the present invention, the method for randomly rotating each discretized edge and parametric surface includes:

[0046] Randomly select a rotation angle, and calculate the rotation matrices for the discretized edges and parametric surfaces to rotate along the X, Y, and Z axes of the space coordinate system according to the rotation angle:

[0047]

[0048] Among them, R X (θ), R Y(θ) and R Z (θ) are the rotation matrices for the X, Y, and Z axes respectively; θ is the rotation angle;

[0049] For each edge and parametric surface, randomly select one direction among the X, Y, and Z axes as the rotation axis;

[0050] According to the selected rotation axis, act on the three-dimensional geometric coordinates P and normal vector feature N of the parametric surface, and act on the three-dimensional geometric coordinates P and tangent vector feature T of the edge to obtain the rotated features:

[0051] E' fl [P,T] = R l (θ)·Ef [P,T]

[0052] S' fl [P,N]=R l (θ)·S f [P,N]

[0053] Among them, R l (θ) is the rotation matrix of the l-axis, and l takes values of X, Y, and Z; E f and S f are respectively the discretized edge and parametric surface; E' fl and S' fl are respectively the features after E f and S f are rotated along the l-axis.

[0054] When this solution performs random rotation data augmentation, it ensures that each 3D model has its rotated corresponding file during the training of the depth classification model, and presents an adjacent relationship in the actual training batch data, which is convenient for the subsequent calculation of the loss function. Since the features of the rotated shape are similar to those of the original shape and produce the same classification result, the model improves the classification accuracy of product parts by minimizing the difference between the two 3D models during the training process.

[0055] In step S3, a dataset is used to train the depth classification model. The loss function of the depth classification model includes a smooth quadratic loss function and a cross-entropy loss function based on the model outputs before and after rotation.

[0056] The depth classification model of this solution includes a CNN model, a graph neural network, and a multi-layer perceptron connected in sequence. As Figure 3 shown, the overall architecture of the geometric invariant 3D model classification algorithm based on smooth quadratic loss provided by this solution consists of three parts. The first part is file preprocessing, including the discretization processing of the 3D model and the processing of the CNN model; at the same time, random rotation is performed based on the original file to obtain the rotated representation, and then all initial topological feature information is fused and expressed through the convolutional neural network layer; the second part is the graph neural network layer, which fuses the features of each face with the edge features around the face, so that each face feature has the information features of adjacent faces and edges, and all face features are uniformly extracted through the pooling layer; the third part is the multi-layer perceptron, which reduces the hidden layer dimension to the number of categories to be classified through the multi-layer perceptron, outputs the final classification result, and at the same time designs a smooth quadratic loss function in this part to control the training process of the classification model.

[0057] During implementation, it is preferred that the expression of the loss function of the depth classification model of this solution is:

[0058]

[0059] Among them, is the loss function of the deep classification model; is the cross-entropy loss function; is the smoothed quadratic loss function; C is the total number of types of product components; y c is the label of the c-th type of product component; P c is the probability that the deep classification model predicts to belong to the c-th type of product component; B is the training batch size; β′ is the smoothing degree adjustment parameter; and are the i-th dimensional data output by the graph neural network of the data before and after random rotation of the n-th 3D model; is and 's cosine similarity.

[0060] The smoothed quadratic loss function of this solution is characterized by a smooth curve and gradient. It imposes penalties on low-similarity features and is more tolerant of high-similarity features, enabling the smoothed quadratic loss function GCL to have better tolerance for abnormal features during the training process and better handle the impact of special data in the training data on the overall training. Through the training constraint of GCL, the randomly rotated features can be tightly aligned with the original corresponding features, thereby enhancing the rotational invariance of the classification model.

[0061] This solution improves the rotational invariance recognition ability of the classification model for complex 3D shapes by first enhancing rotation and then using the smoothed quadratic loss function for constraint, thereby enhancing its generalization performance and robustness in real industrial product classification.

[0062] In step S4, grid sampling is performed on the continuous parametric surfaces and edges of the solid 3D model of the product component to be recognized, and then it is input into the trained deep classification model to obtain the type of the product component to be recognized.

[0063] In implementation, the preferred CNN model of this solution includes a first convolutional layer, a second convolutional layer, a third convolutional layer, an average pooling layer, and a fully connected layer connected in sequence. Its convolutional process is: First convolutional layer CNN(6,64,3) → Second convolutional layer CNN(64,128,3) → Third convolutional layer CNN(128,256,3) → Average pooling layer Pool(1,1) → Fully connected layer FC(256,64), where the function indicates: CNN(input channels, output channels, convolutional kernel size), Pool(1,1) represents the average pooling layer, and FC(input dimension, hidden layer dimension).

[0064] The discretized parameter surfaces and edges before and after the rotation of each 3D model are fused through multiple channels of the CNN model to obtain the surface structure feature vectors and connected edge feature vectors before and after rotation. That is, for each 3D model, two vectors are obtained respectively before and after rotation, one is the feature set of all surfaces as graph nodes, and the other is the set of all edge features.

[0065] After the CNN model initially expresses the topological features of the original 3D model parameters, the topological information of the 3D model needs to be transmitted under the global view next. The graph neural network (GNN) is used to update the face-edge features, and gradually calculate and fuse the features of each face node, including the edge features around the face and the adjacent face node features. Through the stacked calculation of multiple layers of GNN, the topological information of each face node is transmitted between nodes, and the topological features under the global view are gradually learned.

[0066] In the graph neural network, the expressions for updating the face node features and edge features in each layer of the GNN layer are as follows:

[0067]

[0068] Among them, and are the face node features output by the k-th and k-1-th layers of the GNN layer respectively; σ and φ are the multi-layer perceptron layers for updating the face node features and edge features respectively; is the adjacent face feature of in the k-1-th layer of the GNN layer, N(u) is the set of all adjacent face features of , v ∈ N(u); is the feature of the shared edge between and in the k-th layer of the GNN layer; is the feature of the shared edge between and output by the k-1-th layer of the GNN layer; is the adjacent face feature of in the k-th layer of the GNN layer; β (k) is the feature for marking and distinguishing the face nodes in the k-1-th layer of the GNN layer; γ (k) is the feature for marking and distinguishing the edges in the k-1-th layer of the GNN layer; ⊙ is the dot product.

[0069] The multi-layer perceptron layer σ in the graph neural network consists of two fully connected layers to learn the internal latent space features. The algorithm of the graph neural network part uses the adjacent face features and shared edge features of the face node features, and combines the stacked GNN layers by dot product summation and projection respectively, so that the face node feature h uFused with surrounding information, gradually update and learn the global topological features (the graph neural network processing does not change the feature dimension, and the output is [total number of faces, hidden layer dimension]).

[0070] The graph neural network passes through the above and After the stacked iteration update of the two parts of the arithmetic formula, the alternating learning of face-edge features is realized, so that each face node feature gradually obtains a global view. Therefore, this part only needs to output the face node features of the last layer as the final global topological representation result.

[0071] During implementation, this solution preferably further includes using a pooling layer to perform a max-pooling operation on the face node features output by the graph neural network to extract global features, and then inputting the global features into a multi-layer perceptron for classification to obtain the type of the three-dimensional model.

[0072] The classification algorithm of this solution was tested and evaluated with PointNet and PointNet++ on the Solidletters dataset. The classification accuracies of the three algorithms are shown in the following table:

[0073]

[0074] As can be seen from the above table, the classification accuracy of the geometric invariant three-dimensional model classification algorithm based on the smooth quadratic loss of this solution exceeds the accuracies of previous three-dimensional model classification methods such as PointNet and PointNet++.

[0075] In summary, this solution uses the random rotation of the original model in the three-dimensional space coordinates to create new training data, and then combines the smooth quadratic loss function designed in the model training process to reduce the difference in classification and recognition features between the rotated graph and the original graph, so as to improve the accuracy of the existing classification algorithm in the classification tasks of small data volume or complex three-dimensional models.

Claims

1. A geometric invariant 3D model classification algorithm based on smooth quadratic loss, characterized in that, Including the steps: S1. Obtain the 3D models of various product components, and perform grid sampling on the continuous parametric surfaces and edges in each 3D model to obtain the discretized parametric surfaces and edges; S2. Randomly rotate each discretized edge and parametric surface, and use the product component type corresponding to the 3D model, the discretized parametric surfaces and edges before and after random rotation as the data set; S3. Use the data set to train the deep classification model. The loss function of the deep classification model includes a smooth quadratic loss function and a cross-entropy loss function based on the model outputs before and after rotation; S4. Perform grid sampling on the continuous parametric surfaces and edges of the entity 3D model of the product component to be recognized, and then input it into the trained deep classification model to obtain the type of the product component to be recognized.

2. The geometrically invariant three-dimensional model classification algorithm according to claim 1, characterized in that, The deep classification model includes a CNN model, a graph neural network, and a multi-layer perceptron connected in sequence. The expression of the loss function of the deep classification model is: Among them, is the loss function of the depth classification model; is the cross-entropy loss function; is the smoothed quadratic loss function; C is the total number of types of product components; y c is the label of the c-th type of product component; P c is the probability that the depth classification model predicts to belong to the c-th type of product component; B is the training batch size; β′ is the smoothing degree adjustment parameter; and are the i-th dimensional data output by the graph neural network for the data before and after the random rotation of the n-th 3D model; is and 's cosine similarity.

3. The geometric invariant three-dimensional model classification algorithm according to claim 1, characterized in that For each The method of randomly rotating each discretized edge and parametric surface includes: Randomly select the rotation angle, and calculate the rotation matrices for the discretized edges and parametric surfaces to rotate along the X, Y, and Z axes of the spatial coordinate system according to the rotation angle: Among them, R X (θ), R Y(θ) and R Z (θ) are the rotation matrices of the X, Y, and Z axes respectively; θ is the rotation angle; For each edge and parametric surface, randomly select one direction among the X, Y, and Z axes as the rotation axis; According to the selected rotation axis, apply it to the three-dimensional geometric coordinates P and normal vector feature N of the parametric surface, and apply it to the three-dimensional geometric coordinates P and tangent vector feature T of the edge to obtain the rotated features; It is fl [P, T] = R l (θ)·E f [P, T] S' fl [P,N] = R l (θ)·S f [P,N] where, R l (θ) is the rotation matrix of the l-axis, and l takes values of X, Y, and Z; E f and S f are the discretized edge and parametric surface respectively; E' fl and S' fl are the features after E f and S f are rotated along the l-axis.

4. The geometrically invariant three-dimensional model classification algorithm according to claim 1, wherein The CNN model includes a first convolutional layer, a second convolutional layer, a third convolutional layer, an average pooling layer, and a fully connected layer connected in sequence. The input channels, output channels, and convolutional kernel sizes of the first convolutional layer, the second convolutional layer, and the third convolutional layer are (6, 64, 3), (64, 128, 3), and (128, 256, 3) respectively. The input dimension and hidden layer dimension of the fully connected layer are 256 and 64 respectively; The discretized parametric surfaces and edges before and after rotation of each 3D model are fused through multiple channels of the CNN model to obtain the surface structure feature vectors and connecting edge feature vectors before and after rotation.

5. The geometrically invariant three-dimensional model classification algorithm according to claim 2, characterized in that, The multi-layer GNN of the graph neural network performs stacked calculations on the information input by each 3D model, enabling the information of each face node to be transmitted between nodes, and obtaining the topological features under the global view. The expressions for updating the face node features and edge features of each GNN layer are: wherein, and are the face node features output by the k-th and (k - 1)-th layer GNN layers respectively; σ and φ are multi-layer perceptron layers for updating face node features and edge features respectively; is the adjacent face feature of in the (k - 1)-th layer GNN layer, N(u) is the set of all adjacent face features of , v ∈ N(u); is the feature of the shared edge between and in the k-th layer GNN layer; is the feature of the shared edge between and output by the (k - 1)-th layer GNN layer; is the adjacent face feature of in the k-th layer GNN layer; β (k) is the feature for marking and distinguishing face nodes in the (k - 1)-th layer GNN layer; γ (k) is the feature for marking and distinguishing edges in the (k - 1)-th layer GNN layer; ⊙ is the dot product.

6. The geometrically invariant three-dimensional model classification algorithm according to any one of claims 1-5, characterized in that, Performing grid sampling on the continuous parametric surfaces and edges in each 3D model includes: sampling the parametric surface with a two-dimensional grid on the face nodes, and sampling each edge with a one-dimensional grid.

7. The geometrically invariant three-dimensional model classification algorithm according to claim 2, wherein It also includes performing a max-pooling operation on the face node features output by the graph neural network using a pooling layer to extract the global features, and then inputting the global features into the multi-layer perceptron for classification to obtain the type of the 3D model.

Images