3D Model Classification by Fusing View Features and Multi-Branch Networks

Abstract

The present invention proposes a 3D model classification method that fuses view features and a multi-branch network. Two-dimensional views of the 3D model are obtained through projection, and the ConvNeXt network is used to extract view features from them as global features. The Canny algorithm is adopted to extract the contours of the two-dimensional views, and the ResNet improved by attention is used to extract contour features from them as local features. The D1 feature, Hu moment feature, and corner curvature feature of the contour are extracted to form a one-dimensional feature vector as the shape feature, and a one-dimensional convolutional neural network is used to extract the depth shape feature from it. Different weights are assigned to the global feature, local feature, and shape feature for weighted fusion, and then the softmax function is used to obtain the classification prediction probability of the fused feature under each view, and the classification result of the model is obtained through probability voting. The present invention fuses the global feature, local feature, and shape feature, enhancing the representation ability of the 3D model and improving the classification accuracy of the 3D model.

Description

Technical Field:

[0001] The present invention relates to a three-dimensional model classification method integrating view features and a multi-branch network, which has good applications in the field of computer vision. Background Art:

[0002] With the rapid development of computer technology, artificial intelligence has played an important role in the development of various fields. As a popular direction of artificial intelligence, computer vision has now been widely used in various applications in the real world. The main research objects of computer vision include images, three-dimensional models, etc. Among them, the research on three-dimensional models is also a very popular sub-field of computer vision. The research on three-dimensional models mainly involves tasks such as classification, retrieval, and segmentation.

[0003] In recent years, with the rapid development of deep neural networks, using deep learning methods to learn and represent three-dimensional models has become a hot issue in the field of three-dimensional model research. Three-dimensional models can be classified into voxels, point clouds, views, etc. according to different presentation forms. Therefore, at this stage, for different representation methods of three-dimensional models, the related three-dimensional model classification methods can be roughly divided into voxel-based, point cloud-based, and view-based methods. Voxel-based methods usually use dense and regular three-dimensional grids to represent three-dimensional models, and perform operations such as three-dimensional convolution and pooling on them to extract high-order features for learning representation. Point cloud-based methods refer to sampling three-dimensional coordinates from the surface of three-dimensional models to form a three-dimensional point set for learning representation. View-based methods often first render a set of two-dimensional views for each three-dimensional model, and then use a two-dimensional convolutional neural network to extract relevant features. In the research of three-dimensional model classification methods, improving the classification accuracy of the model has always been a very crucial issue. Based on the view classification method, the present invention proposes a three-dimensional model classification method integrating view features and a multi-branch network, which is composed of a multi-branch neural network in parallel, extracts global features, local features, and shape features respectively, and then weights and fuses each feature, further improving the characterization ability of the model's detailed information and the classification accuracy. Summary of the Invention:

[0004] In order to solve the three-dimensional model classification problem in the field of computer vision, the present invention discloses a three-dimensional model classification method integrating view features and a multi-branch network.

[0005] For this reason, the present invention provides the following technical solutions:

[0006] 1. A three-dimensional model classification method integrating view features and a multi-branch network, characterized in that the method includes the following steps:

[0007] Step 1: Extract two-dimensional views from the 3D models in ModelNet10. Place the 3D models in a normalized coordinate system and project a series of views by selecting fixed viewpoints.

[0008] Step 2: Use the Canny algorithm to extract the contours of all views obtained from ModelNet10. Extract the D1 feature, Hu moment feature, and corner curvature feature of the contours to form a one-dimensional shape feature vector.

[0009] Step 3: Construct the training data from the views, contours, and shape feature vectors extracted from the ModelNet10 training model, and construct the test data from the views, contours, and shape feature vectors extracted from the ModelNet10 test model.

[0010] Step 4: Design a multi-branch network model with parallel connections of a ConvNeXt network, an ECA-ResNet network, and a one-dimensional convolutional neural network.

[0011] Step 5: Optimize the multi-branch network using the training data to obtain an optimized multi-branch network model. Use the ConvNeXt branch to extract global features from the views of the test data, use the ECA-ResNet branch to extract local features from the contours of the test data, and use the one-dimensional convolutional neural network branch to extract shape features from the shape vectors of the test data. Assign different weights to the global features, local features, and shape features and perform weighted fusion.

[0012] Step 6: Calculate the classification prediction probability of the fused features using the softmax function, and vote using the classification prediction probabilities of all viewpoints of the 3D model to obtain the classification result of the model.

[0013] 2. The 3D model classification method integrating view features and a multi-branch network according to claim 1, wherein in step 1, the specific steps for extracting two-dimensional views from the 3D models in ModelNet10 are as follows:

[0014] Step 1-1: Preprocess the 3D models in ModelNet10 and place the 3D models in a normalized coordinate system.

[0015] Step 1-2: Place the camera above the model, fix a viewpoint every 60 degrees, and the camera projects to generate a two-dimensional view.

[0016] Step 1-3: Each 3D model obtains six viewpoint views V1, V2, V3, V4, V5, and V6 through projection.

[0017] 3. The 3D model classification method integrating view features and multi-branch network according to claim 1, wherein in step 2, the Canny algorithm is used to extract the contours of all views obtained from ModelNet10, and the D1 feature, Hu moment feature, and corner curvature feature of the extracted contours are combined to form a one-dimensional shape feature vector. The specific steps are as follows:

[0018] Step 2-1: Use the Canny algorithm to extract the contours of the views. The specific steps are as follows:

[0019] Step 2-1-1: Grayscale the view.

[0020] Step 2-1-2: Apply a Gaussian filter to smooth the view. Substitute the horizontal and vertical coordinate indices of the corresponding points in the filter into the Gaussian function to remove view noise. The Gaussian function calculation formula is as follows:

[0021]

[0022] where (x, y) is the point coordinate and σ is the standard deviation.

[0023] Step 2-1-3: Use the Soble operator to calculate the gradient value G x in the horizontal direction and the gradient value G y in the vertical direction, and calculate the gradient value G and gradient direction θ of the smoothed view. The calculation formulas are as follows:

[0024]

[0025]

[0026] Step 2-1-4: Perform edge detection based on the view gradient value. Set the upper threshold and lower threshold, select the edges in the view with gradient values greater than the upper threshold, discard the edges less than the lower threshold, and connect the edges to form contours.

[0027] Step 2-2: Extract the D1 feature of the contour. The specific steps are as follows:

[0028] Step 2-2-1: Calculate the p+q order origin moment of the contour. The calculation formula is as follows:

[0029]

[0030] where ρ(x, y) represents the contour function, R represents the number of rows, and C represents the number of columns.

[0031] Step 2-2-2: Calculate the centroid coordinates (x0, y0) using the zero-order origin moment and first-order origin moment of the contour. The calculation formulas are as follows:

[0032]

[0033] Step 2-2-3 performs random equidistant sampling on the points on the contour. Let the coordinate set of the sampling points be Point = {(x1, y1), …, (x i , y i ), …, (x n , y n )}. Randomly select N points from Point to form the set PD1 = {P1, …, P k , …, P N}. Bins represents the number of intervals, and BinsSize represents the interval length. The calculation formulas are as follows:

[0034] BinsSize = max({dist(P, O)|P ∈ PD1}) / N

[0035] where dist() is the Euclidean distance between two points, O is the centroid of the contour, and max() represents taking the maximum value;

[0036] Step 2-2-4 uses the D1 feature to describe the distance between the randomly sampled points on the contour and the centroid. The calculation of D1_v i is as follows:

[0037] D1_v i = |{P|dist(P, O) ∈ (BinSize * (i - 1), BinSize * i), P ∈ PD1}|

[0038] Step 2-3 extracts the Hu moment features of the contour. The specific steps are as follows:

[0039] Step 2-3-1 calculates the p+q-th order central moment u pq of the contour from the centroid coordinates. The calculation formula is as follows:

[0040]

[0041] Step 2-3-2 normalizes the p+q-th order central moment of the contour using the zero-th order central moment and calculates the p+q-th order normalized central moment η pq of the contour. The calculation formula is as follows:

[0042]

[0043] Step 2-3-3 calculates 7 invariant moment groups of the contour Hu moment through the second-order and third-order normalized central moments of the contour. The calculation formulas are as follows;

[0044] H1 = η 20 + η 02

[0045] H2 = (η 20 - η02 ) + 4×η1 2 1

[0046] H3 = (η 30 - 3×η 12 ) 2 + (3×η 21 - η 03 ) 2

[0047] H4 = (η 30 + η 12 ) 2 + (η 21 + η 03 ) 2

[0048] H5 = (η 30 - 3×η 12 )×(η 30 + η 12 )×[(η 30 + η 12 ) 2 - 3×(η 21 + η 03 ) 2 +

[0049] (3×η 21 - η 03 )×(η 21 + η 03 )×[3×(η 30 + η 12 ) 2 - (η 21 + η 03 ) 2

[0050] H6 = [(η 30 + η 12 ) 2 - (η 12 + η 03 ) 2 ×(η 20 - η 02 ) + 4×η 11 ×(η 30 + η 12 )×(η 21 + η 03 )

[0051] H7 = (3×η 21 - η 03 )×(η 30 + η 12 )×[(η 30 + η​12 ) 2 -3×(η 21 +η 03 ) 2 -

[0052] (3×η 12 -η 30 )×(η 21 +η 03 )×[3×(η 30 +η 12 ) 2 -(η 21 +η 03 ) 2

[0053] Step 2-3-4 combines the calculation results of the 7 invariant moments of the contour Hu moments into a one-dimensional vector as the contour Hu moment feature;

[0054] Step 2-4 extracts the corner curvature feature of the contour. The specific steps are as follows:

[0055] Step 2-4-1 extracts the contour corners. By establishing a detection window, centered on a pixel point, displacing in the horizontal and vertical directions, the points where the gray value changes significantly are the contour corners, and the corner coordinates are extracted from the window;

[0056] Step 2-4-2 randomly samples 6 points from the extracted contour corners, and the coordinates are denoted as (x i ,y i ), i = 1, 2,..., 6. All the sampled points are combined into a two-dimensional array f = [[x1, x2,..., x6], [y1, y2,..., y6]], and the gradients of x i ,y i in the two-dimensional array f are calculated. The calculation formula is as follows:

[0057] x i ′=(x i-1 -x i+1 ) / 2

[0058] y i ′=(y i-1 -y i+1 ) / 2

[0059] x i ″=(x i-1 ′-x i+1 ′) / 2

[0060] y i ″=(y i-1 ′-y i+1 ′) / 2

[0061] Among them, x​i The first - order gradient of x is i the first - order gradient of x i The second - order gradient of x is i the second - order gradient of y i The first - order gradient of y is i the first - order gradient of y i The second - order gradient of y is i the second - order gradient;

[0062] Step 2 - 4 - 3 uses the first - order and second - order gradients of x i , y i to calculate the curvature k of the point (x i , y i ), and the calculation formula is as follows: i (i = 1, 2, …, 6), and the calculation formula is as follows:

[0063]

[0064] Step 2 - 4 - 4 uses the calculated corner curvature feature k i (i = 1, 2, …, 6) to construct a one - dimensional vector as the corner curvature feature;

[0065] Step 2 - 5 performs Log transformation on the Hu - moment feature and the corner curvature feature to make them more conform to the normal distribution, and the calculation formula is as follows:

[0066]

[0067] where a represents the data before Log transformation and b represents the data after Log transformation;

[0068] Step 2 - 6 combines the D1 feature, the Hu - moment feature after Log transformation, and the corner curvature feature after Log transformation into a one - dimensional vector, and then performs normalization processing, and uses the processed feature vector as the shape feature;

[0069] 4. The 3D model classification method integrating view features and multi - branch networks according to claim 1, wherein in the said step 3, the view, contour, and shape feature vectors extracted from ModelNet10 are used to form training data and test data, and the specific steps are as follows:

[0070] Step 3 - 1 uses the view, contour, and shape feature vectors extracted from the 3D models in the ModelNet10 training set to form training data;

[0071] Step 3 - 2 uses the view, contour, and shape feature vectors extracted from the 3D models in the ModelNet10 test set to form test data;

[0072] 5. The 3D model classification method integrating view features and multi-branch network according to claim 1, wherein in step 4, a multi-branch network is designed, and the specific steps are as follows:

[0073] Step 4-1: Determine the depth of the ConvNeXt network, and select ConvNeXt-T with a stack depth B=(3, 3, 9, 3) of 4 ConvNeXt Block blocks;

[0074] Step 4-2: Determine the depth of the ResNet network, select ResNet-34 with a depth of 34 layers, improve ResNet using the ECA attention mechanism, embed the ECA attention module into the basicblock of ResNet, and strengthen the channel features of the input feature map;

[0075] Step 4-3: Construct a one-dimensional convolutional neural network, which consists of two convolutional layers with a convolutional kernel size of 3, two pooling layers, and one fully connected layer;

[0076] Step 4-4: Connect ConvNeXt, ECA-ResNet, and the one-dimensional convolutional neural network in parallel to form a multi-branch network;

[0077] 6. The 3D model classification method integrating view features and multi-branch network according to claim 1, wherein in step 5, the multi-branch network is optimized, and global features, local features, and shape features are extracted using the optimized multi-branch network and weighted and fused. The specific steps are as follows:

[0078] Step 5-1: Optimize the multi-branch network using training data, optimize the ConvNeXt network using the views in the training data, optimize ECA-ResNet using the contours in the training data, and optimize the one-dimensional convolutional neural network using the shape feature vectors in the training data;

[0079] Step 5-2: Extract global features from the views of the test data using the optimized ConvNeXt, and use the output G i ={g1, g2, …, g n} as global features;

[0080] Step 5-3: Extract local features from the contours of the test data using the optimized ECA-ResNet, and use the output T i ={t1, t2, …, t n} as local features;

[0081] Step 5-4: Extract shape features from the shape feature vectors in the test data using the optimized one-dimensional convolutional neural network, and use the output S i ={s1, s2, …, sn} as the shape feature;

[0082] Step 5-5 combines the global feature G i , the local feature T i and the shape feature S i through weighted fusion to obtain F i = {f1, f2, …, f n}, and the calculation formula is as follows:

[0083] F i = α·G i + β·T i + (1 - α - β)·S i

[0084] Among them, α and β are weight coefficients used to adjust the proportion of each branch. After cross-validation, the best results are obtained when α is 0.7 and β is 0.2;

[0085] 7. The 3D model classification method integrating view features and a multi-branch network according to claim 1, wherein in step 6, the softmax function is used to calculate the classification prediction probability of the fused feature under each view, and the classification result of the 3D model is obtained through probability voting. The specific steps are as follows:

[0086] Step 6-1 uses the softmax function to calculate the prediction probability of F i under the 3D model category C j (j = 1, 2,..., 10):

[0087]

[0088] Among them, x j represents the input data of the softmax function;

[0089] Step 6-2 uses the softmax outputs of F1, F2, …, F6 under the 6 views of the 3D model M for probability voting. The probability P i that F j under the i-th view belongs to the class C ji (j = 1, 2,..., 10) is used as the voting value for the corresponding class, and the vote T j obtained by the class C vote (C j | M) is calculated. The class with the most votes is used as the classification result of M. The calculation process is as follows:

[0090] P ji = P(C j | F i )

[0091]

[0092]

[0093] Among them, argmax() is the maximum-taking function, and C u is the classification result of the 3D model M, and u is the label of the classification result.

[0094] Beneficial effects:

[0095] 1. The present invention is a 3D model classification method that combines view features and a multi-branch network. Most view-based 3D model classification methods use neural networks to extract features from the entire view, which has the problem of focusing on global features while ignoring local detail information, and this local detail information is the key to enabling more accurate classification of 3D models. To address this problem, the present invention designs a multi-branch parallel neural network to perform weighted fusion of global features, local features, and shape features, enhancing the model's representation ability, reducing the loss of detail information, and improving the accuracy of 3D model classification.

[0096] 2. The present invention uses the Canny algorithm to extract the view contour, which can better retain the local information of the contour. Extracting the D1 feature, Hu moment feature, and corner curvature feature of the contour map can better describe the shape information of the model and has translational, rotational, and scale invariance. The composed shape features can better supplement the geometric shape detail information missing from the view.

[0097] 3. The present invention uses the attention mechanism to improve ResNet, uses the ECA attention module to enhance the channel features of the input feature map, and does not change the size of the input feature map, reducing the feature loss caused by pooling in the neural network. Embedding the ECA attention module into the ResNet neural network enables it to have better feature extraction ability.

[0098] 4. The present invention uses a softmax classifier to obtain the classification results of each view of the model, and then performs probability voting on the classification results of all views to obtain the final classification category of the model, enhancing the accuracy and generalization of classification.

[0099] 5. The present invention verifies the effect based on the publicly available dataset ModelNet10. The results show that the parallel multi-branch network can better extract the global features, local features, and shape features of the 3D model view. The fused features can enhance the representation ability of the model's detail information and improve the accuracy of 3D model classification. Description of the drawings:

[0100] Figure 1 The test 3D model in the embodiment of the present invention;

[0101] Figure 2Flowchart of fusing view features and multi-branch 3D model classification in the embodiments of the present invention;

[0102] Figure 3 Example diagram of 3D model views in the embodiments of the present invention;

[0103] Figure 4 Example diagram of 3D model contours in the embodiments of the present invention;

[0104] Figure 5 Structural diagram of the ConvNeXt network in the embodiments of the present invention;

[0105] Figure 6 Structural diagram of the ECA-ResNet network in the embodiments of the present invention;

[0106] Figure 7 Structural diagram of the one-dimensional convolutional neural network in the embodiments of the present invention;

[0107] Figure 8 Structural diagram of the multi-branch network in the embodiments of the present invention;

[0108] Figure 9 Prediction process of the multi-branch network in the embodiments of the present invention; Specific implementation manner:

[0109] In order to clearly and completely describe the technical solutions in the embodiments of the present invention, taking a 3D model in the test set of the desk class of the 3D model public dataset ModelNet10 as an example, as Figure 1 shown, tests are carried out, and the present invention is further described in detail in combination with the accompanying drawings in the embodiments.

[0110] The flowchart of the 3D model classification method based on fusing view features and multi-branch network in the embodiments of the present invention is as Figure 2 shown, and includes the following steps:

[0111] 1. The 3D model classification method based on fusing view features and multi-branch network, characterized in that the method includes the following steps:

[0112] Step 1: Extract two-dimensional views from the 3D models in ModelNet10. The specific steps are as follows:

[0113] Step 1-1: Preprocess the 3D models in ModelNet10 and place the 3D models in a normalized coordinate system;

[0114] Step 1-2: Place the camera above the model, fix a viewing angle every 60 degrees, and the camera projects to generate a two-dimensional view;

[0115] In Step 1-3, six perspective views V1, V2, V3, V4, V5, and V6 are obtained by projecting each 3D model. The views extracted from the Figure 1 test model shown are as Figure 3 shown;

[0116] Step 2: Use the Canny algorithm to extract the contours of all views obtained from ModelNet10. The D1 feature, Hu moment feature, and corner curvature feature of the extracted contours are combined to form a one-dimensional shape feature vector. The specific steps are as follows:

[0117] Step 2-1: Use the Canny algorithm to extract the contours of the views. The specific steps are as follows:

[0118] Step 2-1-1: Grayscale the view;

[0119] Step 2-1-2: Apply a Gaussian filter to smooth the view. Substitute the horizontal and vertical coordinate indices of the corresponding points in the filter into the Gaussian function to remove the view noise. The calculation formula of the Gaussian function is as follows:

[0120]

[0121] where (x, y) is the point coordinate and σ is the standard deviation;

[0122] Step 2-1-3: Calculate the gradient value G x in the horizontal direction and the gradient value G y in the vertical direction respectively using the Soble operator. Calculate the gradient value G and the gradient direction θ of the smoothed view. The calculation formulas are as follows:

[0123]

[0124]

[0125] Step 2-1-4: Perform edge detection based on the view gradient value. By setting the upper threshold and the lower threshold, select the edges in the view where the gradient value is greater than the upper threshold, discard the edges where the gradient value is less than the lower threshold, and connect the edges to form a contour. The contour extracted from the Figure 3 view shown is as Figure 4 shown;

[0126] Step 2-2: Extract the D1 feature of the contour. The specific steps are as follows:

[0127] Step 2-2-1: Calculate the p+q order origin moment of the contour. The calculation formula is as follows:

[0128]

[0129] where ρ(x, y) represents the contour function, R represents the number of rows, and C represents the number of columns;

[0130] Step 2-2-2 calculates the centroid coordinates (x0, y0) using the contour zero-order origin moment and the first-order origin moment. The calculation formula is as follows:

[0131]

[0132] In Step 2-2-3, random equidistant sampling is performed on the points on the contour. Let the coordinate set of the sampling points be Point = {(x1, y1), …, (x i , y i ), …, (x n , y n )}. Randomly select N points from Point to form the set PD1 = {P1, …, P k , …, P N}. Bins represents the number of intervals, and BinsSize represents the interval length. The calculation formula is as shown below:

[0133] BinsSize = max({dist(P, O)|P ∈ PD1}) / N

[0134] where dist() is the Euclidean distance between two points, O is the centroid of the contour, and max() represents taking the maximum value;

[0135] In Step 2-2-4, the D1 feature is used to describe the distance between the randomly sampled points on the contour and the centroid. The calculation of D1_v i is as shown below:

[0136] D1_v i = |{P|dist(P, O) ∈ (BinSize * (i - 1), BinSize * i), P ∈ PD1}|

[0137] The D1 feature D1_v Figure 4 extracted from the contour shown in i is as follows:

[0138] D1_v1 = [52, 130, 172, 149, 59, 63, 68, 188, 84, 33]

[0139] D1_v2 = [3, 51, 124, 180, 101, 40, 111, 223, 114, 52]

[0140] D1_v3 = [28, 199, 197, 60, 23, 124, 217, 99, 52, 112]

[0141] D1_v4 = [17, 100, 213, 159, 106, 13, 127, 141, 81, 42]

[0142] D1_v5 = [41, 87, 192, 191, 83, 16, 100, 163, 87, 39]

[0143] D1_v6 = [19, 53, 177, 140, 80, 14, 150, 207, 117, 42]

[0144] Step 2-3 Extract the Hu moment features of the contour. The specific steps are as follows:

[0145] Step 2-3-1 Calculate the (p+q)-order central moment μ of the contour from the centroid coordinates pq , and the calculation formula is as follows:

[0146]

[0147] Step 2-3-2 Normalize the (p+q)-order central moment of the contour using the zero-order central moment, and calculate the (p+q)-order normalized central moment η pq , and the calculation formula is as follows:

[0148]

[0149] Step 2-3-3 Calculate the 7 invariant moment groups of the contour Hu moment through the second-order and third-order normalized central moments of the contour. The calculation formula is as follows;

[0150] H1 = η 20 + η 02

[0151] H2 = (η 20 - η 02 ) + 4 × η1 2 1

[0152] H3 = (η 30 - 3 × η 12 ) 2 + (3 × η 21 - η 03 ) 2

[0153] H4 = (η 30 + η 12 ) 2 + (η 21 + η 03 ) 2

[0154] H5 = (η 30 - 3 × η 12 ) × (η 30 + η 12 ) × [(η 30 + η 12 ) 2-3×(η 21 +η 03 ) 2 +

[0155] (3×η 21 -η 03 )×(η 21 +η 03 )×[3×(η 30 +η 12 ) 2 -(η 21 +η 03 ) 2

[0156] H6=[(η 30 +η 12 ) 2 -(η 12 +η 03 ) 2 ×(η 20 -η 02 )+4×η 11 ×(η 30 +η 12 )×(η 21 +η 03 )

[0157] H7=(3×η 21 -η 03 )×(η 30 +η 12 )×[(η 30 +η 12 ) 2 -3×(η 21 +η 03 ) 2 -

[0158] (3×η 12 -η 30 )×(η 21 +η 03 )×[3×(η 30 +η 12 ) 2 -(η 21 +η 03 ) 2

[0159] Step 2-3-4 forms a one-dimensional vector from the calculation results of the 7 invariant moments of the contour Hu moments as the contour Hu moment feature. The Hu feature Hu_v Figure 4 extracted from the contour shown is as follows: i as follows:

[0160] ​​Hu_v1 = [6.76606323e-04, 4.21745504e-13, 2.96726816e-16, 9.94334602e-15, 4.01

[0161] 864155e-30, 2.41711593e-21, -1.66000791e-29]

[0162] Hu_v2 = [6.86714112e-04, 3.16408999e-12, 4.02669401e-17, 2.70789167e-14, 5.41

[0163] 377350e-30, -1.48228762e-21, 2.77531128e-29]

[0164] Hu_v3 = [6.87721653e-04, 1.77939952e-12, 1.59528426e-17, 1.90077305e-14, 3.19

[0165] 329298e-30, -1.90184352e-20, -9.96780020e-30]

[0166] Hu_v4 = [6.83893207e-04, 1.42370628e-13, 2.82755541e-16, 1.32958145e-14, 1.42876419e-29, 3.57777800e-21, 2.14582527e-29]

[0167] Hu_v5 = [6.87016481e-04, 3.91667937e-12, 3.51694017e-16, 1.59102249e-14, -1.88339022e-29, -2.01073124e-20, 3.25838959e-29]

[0168] Hu_v6 = [6.83878459e-04, 5.67336284e-12, 6.46682041e-16, 2.22495544e-14, -2.90903384e-29, -7.43149490e-21, -7.92251087e-29]

[0169] Step 2-4 Extract the corner curvature features of the contour. The specific steps are as follows:

[0170] Step 2-4-1 Extract the contour corner points. By establishing a detection window, with a pixel point as the center, displace it in the horizontal and vertical directions. The points where the gray value changes significantly are the corner points of the contour, and extract the corner point coordinates from the window;

[0171] Step 2-4-2 Randomly sample 6 points from the extracted contour corner points, and record the coordinates as (x i , y i ), i = 1, 2, …, 6. Form all the sampled points into a two-dimensional array f = [[x1, x2, …, x6], [y1, y2, …, y6]], and calculate the gradients of x i , y i in the two-dimensional array f. The calculation formula is as follows:

[0172] x i ′ = (x i-1 - x i+1 ) / 2

[0173] y i ′ = (y i-1 - y i+1 ) / 2

[0174] x i ″ = (x i-1 ′ - x i+1 ′) / 2

[0175] y i ″ = (y i-1 ′ - y i+1 ′) / 2

[0176] Among them, x i ′ is the first-order gradient of x i , x i ″ is the second-order gradient of x i , y i ′ is the first-order gradient of y i , and y i ″ is the second-order gradient of y i ;

[0177] Step 2-4-3 Use the first-order and second-order gradients of x i , y i to calculate the curvature k i , y i ) (i = 1, 2, …, 6) of the point (x i ). The calculation formula is as follows:

[0178]

[0179] Step 2-4-4 Use the calculated corner point curvature feature k i(i = 1, 2, …, 6) constructs a one-dimensional vector as the corner curvature feature, and extracts the corner curvature feature K_v Figure 4 from the contour shown i as follows:

[0180] K_v1 = [0.01108861, 0.01646123, 0.00914207, 0.00897507, 0.01702089, 0.0106567]

[0182] K_v2 = [0.00035629, 0.00595055, 0.01473849, 0.01002006, 0.00461997, 0.16452911]

[0184] K_v3 = [0.01606139, 0.02003469, 0.00529715, 0.00608346, 0.01710728, 0.00448624]

[0186] K_v4 = [0.000913075237, 0.00631684616, 0.0107563024, 0.0111453515,

[0187] 0.00613555291, 0.000388032455]

[0188] K_v5 = [0.04681764, 0.01169684, 0.00670365, 0.01055543, 0.05118909, 0.00272171]

[0190] K_v6 = [0.00124224, 0.00912105, 0.02976125, 0.00172175, 0.09982274, 0.00284514]

[0192] Step 2-5 performs log transformation on the Hu moment feature and the corner curvature feature to make them more conform to the normal distribution. The calculation formula is as follows:

[0193]

[0194] where a represents the data before log transformation, and b represents the data after log transformation;

[0195] Step 2-6 combines the D1 feature, the log-transformed Hu moment feature, and the log-transformed corner curvature feature into a one-dimensional vector, and then performs normalization processing. The processed feature vector is used as the shape feature;

[0196] From Figure 4 The processed shape feature extracted from the contour shown is denoted as S_v i , where the order of the features is the D1 feature, the Hu feature, and the corner curvature feature, and S_v i is as follows:

[0197] S_v1 = [[0.1446, 0.3614, 0.4782, 0.4142, 0.1640, 0.1751, 0.1890, 0.5227,

[0198] 0.2335, 0.0917, 0.0088, 0.0313, 0.0411, 0.0369, 0.0761, 0.0531, 0.0770, 0.0054, 0.0050, 0.0057, 0.0057, 0.0049, 0.0055]]

[0199] S_v2 = [0.0079, 0.1350, 0.3283, 0.4766, 0.2674, 0.1059, 0.2939, 0.5905,

[0200] 0.3019, 0.1377, 0.0083, 0.0307, 0.0449, 0.0354, 0.0760, 0.0526, 0.0759, 0.0091, 0.0059, 0.0048, 0.0053, 0.0062, 0.0021]

[0201] S_v3 = [0.0672, 0.4777, 0.4729, 0.1440, 0.0552, 0.2977, 0.5209, 0.2377,

[0202] 0.1248, 0.2689, 0.0076, 0.0285, 0.0399, 0.0325, 0.0702, 0.0468, 0.0687, 0.0043, 0.0041, 0.0055, 0.0053, 0.0042, 0.0056]

[0203] S_v4 = [0.0456, 0.2681, 0.5710, 0.4263, 0.2842, 0.0349, 0.3405, 0.3780,

[0204] 0.2172,0.1126,0.0084,0.0311,0.0394,0.0352,0.0741,0.0519,0.0725,0.0108,0.0059,0.0053,0.0052,0.0059,0.0091

[0205] S_v5 = [0.1105,0.2344,0.5172,0.5145,0.2236,0.0431,0.2694,0.4391,

[0206] 0.2344,0.1051,0.0085,0.0295,0.0395,0.0357,0.0736,0.0506,0.0739,0.0036,0.0052,0.0059,0.0053,0.0035,0.0069

[0207] S_v6 = [0.0501,0.1397,0.4667,0.3691,0.2109,0.0369,0.3955,0.5458,

[0208] 0.3085,0.1107,0.0083,0.0281,0.0382,0.0345,0.0713,0.0492,0.0713,0.0077,0.0054,0.0040,0.0073,0.0026,0.0067

[0209] Step 3: Construct the training data and test data from the view, contour, and shape feature vectors extracted from ModelNet10. The specific steps are as follows:

[0210] Step 3-1: Construct the training data from the view, contour, and shape feature vectors extracted from the 3D models in the ModelNet10 training set;

[0211] Step 3-2: Construct the test data from the view, contour, and shape feature vectors extracted from the 3D models in the ModelNet10 test set;

[0212] Step 4: Design a multi-branch network. The specific steps are as follows:

[0213] Step 4-1: Determine the depth of the ConvNeXt network, and select ConvNeXt-T with a stacked depth B=(3,3,9,3) of 4 ConvNeXt Block blocks, as Figure 5 shown;

[0214] Step 4-2: Determine the depth of the ResNet network, select ResNet-34 with a depth of 34 layers, and improve ResNet using the ECA attention mechanism, asFigure 6 As shown, the ECA attention module is embedded into the basicblock of ResNet to enhance the channel features of the input feature map;

[0215] Step 4-3 constructs a one-dimensional convolutional neural network, which consists of two convolutional layers with a convolutional kernel size of 3, two pooling layers, and one fully connected layer, as Figure 7 shown;

[0216] Step 4-4 connects ConvNeXt, ECA-ResNet, and the one-dimensional convolutional neural network in parallel to form a multi-branch network, as Figure 8 shown;

[0217] Step 5: Optimize the multi-branch network, use the optimized multi-branch network to extract global features, local features, and shape features, and perform weighted fusion on them, as Figure 9 shown. The specific steps are as follows:

[0218] Step 5-1 uses the training data to optimize the multi-branch network, optimizes the ConvNeXt network using the views in the training data, optimizes the ECA-ResNet using the contours in the training data, and optimizes the one-dimensional convolutional neural network using the shape feature vectors in the training data;

[0219] Step 5-2 uses the optimized ConvNeXt to extract global features from the views of the test data, and takes the output G i ={g1, g2,..., g n} as the global features. The global features extracted from the Figure 3 views shown are as follows:

[0220] G1 = tensor([0.1060, -1.9721, -4.2902, 6.4690, 0.1252, 3.3780, -0.0697, 3.1018,

[0221] -0.8460, -3.1311])

[0222] G2 = tensor([-1.7427, -2.4591, -2.6067, 7.8893, -0.8068, 1.6520, -0.5407,

[0223] 1.0018, 0.1544, -3.1839])

[0224] G3 = tensor([0.2592, -3.8880, -6.0772, 7.1526, -0.3528, 4.5998, -1.4472,

[0225] 2.4112,-0.3753,-3.9285])

[0226] G4 = tensor([-0.1329,-0.6411,-4.5287,5.7778,-1.0014,3.1769,-1.1457,

[0227] 5.4012,-0.7553,-3.6501])

[0228] G5 = tensor([-0.5851,-3.1796,-2.4710,8.9053,-2.4974,2.0448,-1.2406,

[0229] 1.6773,-1.2830,-2.4821])

[0230] G6 = tensor([0.6809,-5.5116,-7.1551,9.2278,0.9499,5.6715,-2.0124,

[0231] 1.4039,-0.7365,-4.4241])

[0232] Step 5-3 uses the optimized ECA-ResNet to extract local features from the contours of the test data, and takes the output T i = {t1,t2,…,t n} of the fully connected layer as local features. The local features extracted from the Figure 4 contours shown are as follows:

[0233] T1 = tensor([-8.2020,-1.5831,-3.9442,3.6607,-4.2069,-7.2075,-3.2682,-2.7029,

[0234] -0.4619,-9.7197])

[0235] T2 = tensor([-8.4854,-0.6908,-6.7370,2.9918,-5.1717,-6.1493,-3.8799,-4.2624,

[0236] -0.2825,-10.6619])

[0237] T3 = tensor([-7.3523,1.2310,-5.4584,1.2777,-5.9076,-6.8348,-4.1034,-2.7434,

[0238] -2.3887, -10.8623])

[0239] T4 = tensor([-7.9083, -0.7229, -3.7220, 1.1410, -4.7686, -7.8187, -4.0383,

[0240] -0.3075, -2.5556, -9.8964])

[0241] T5 = tensor([-8.9364, -0.4917, -6.0460, 2.6376, -6.0707, -6.2793, -3.9022, -4.2192,

[0242] -0.5323, -10.7659])

[0243] T6 = tensor([-8.3366, -0.2198, -5.7498, 0.6543, -4.4407, -3.8683, -2.2905, -3.6979, 0.0

[0244] 692, -11.7679])

[0245] Step 5 - 4 uses the optimized one - dimensional convolutional neural network to extract shape features from the shape feature vectors in the test data, and takes the output S i = {s1, s2, …, s n} as the shape features. The shape features extracted from the contour shown in Figure 4 are as follows:

[0246] S1 = tensor([-8.1567, -2.5075, -4.9340, 2.5293, -3.6304, -5.8005, -1.1770, -6.1888,

[0247] -1.1127, -6.9648])

[0248] S2 = tensor([-7.3290, -3.1018, -8.0959, 1.9795, -2.3389, -3.3332, -2.5521, -4.7456,

[0249] -4.9350, -7.5361])

[0250] S3 = tensor([-7.1470, -3.0540, -9.2471, 3.4491, -2.4548, -2.4936, -2.4421, -4.7526,

[0251] -3.9558,-8.9004])

[0252] S4 = tensor([-9.2072,-3.9927,-5.7161,4.1392,-3.9578,-5.3168,-2.8656,-6.3504,

[0253] -1.5072,-7.0379])

[0254] S5 = tensor([-8.4480,-2.8269,-8.6722,3.3568,-3.4005,-2.7898,-2.9282,-4.8846,

[0255] -4.9796,-9.0730])

[0256] S6 = tensor([-6.6904,-1.2853,-8.7850,2.5830,-2.6505,-3.6769,-2.8948,-4.6780,

[0257] -3.3433,-7.1355])

[0258] Step 5-5 performs weighted fusion on the global feature G i , the local feature T i and the shape feature S i to obtain F i = {f1,f2,…,f n}, and the calculation method is as follows:

[0259] F i = α·G i + β·T i + (1-α-β)·S i

[0260] where α and β are weight coefficients used to adjust the proportion of each branch. After cross-validation, taking α = 0.7 and β = 0.2 gives the best results. The fused features F1,F2,…,F6 extracted from the Figure 1 shown test model under 6 perspectives are as follows:

[0261] F1 = tensor([-2.3819,-1.9478,-4.2854,5.5134,-1.1168,0.3430,-0.8201,1.0118,

[0262] -0.7958,-4.8322])

[0263] F2 = tensor([-3.6499, -2.1697, -3.9817, 6.3188, -1.8330, -0.4068, -1.4097, -0.6258,

[0264] -0.4419, -5.1147])

[0265] F3 = tensor([-2.0037, -2.7808, -6.2704, 5.6073, -1.6740, 1.6035, -2.0779, 0.6639,

[0266] -1.1360, -5.8124])

[0267] F4 = tensor([-2.5954, -0.9926, -4.4861, 4.6866, -2.0505, 0.1284, -1.8962, 3.0843,

[0268] -1.1905, -5.2381])

[0269] F5 = tensor([-3.0417, -2.6068, -3.8061, 7.0969, -3.3024, -0.1035, -1.9417, -0.1582,

[0270] -1.5025, -4.7979])

[0271] F6 = tensor([-1.8597, -4.0306, -7.0370, 6.8486, -0.4883, 2.8287, -2.1563, -0.2246,

[0272] -0.8360, -6.1640])

[0273] Step 6: Calculate the classification prediction probabilities of the fused features from each perspective using the softmax function, and obtain the classification result of the 3D model through probability voting. The specific steps are as follows:

[0274] Step 6-1 Calculate F i The prediction probability under the 3D model category C j (j = 1, 2,..., 10):

[0275]

[0276] where x j represents the input data of the softmax function;

[0277] FromFigure 1 The softmax output result P of the fused features from 6 perspectives extracted from the shown test model Vi is as follows:

[0278] P V1 = tensor([1.5880e-03, 1.9877e-04, 1.9570e-05, 9.2094e-01, 1.6187e-03,

[0279] 4.1865e-02, 1.3321e-03, 3.1760e-02, 6.1292e-04, 6.2374e-05])

[0280] P V2 = tensor([6.5333e-05, 3.1917e-05, 2.7537e-05, 9.9608e-01, 1.6657e-04,

[0281] 1.9473e-03, 2.1735e-04, 1.0164e-03, 4.3555e-04, 1.5461e-05])

[0282] P V3 = tensor([9.3158e-04, 1.4728e-05, 1.6496e-06, 9.1835e-01, 5.0520e-04,

[0283] 7.1503e-02, 1.6911e-04, 8.0133e-03, 4.9392e-04, 1.4143e-05])

[0284] P V4 = tensor([1.5327e-03, 9.2194e-04, 1.8895e-05, 5.6545e-01, 6.4302e-04,

[0285] 4.1963e-02, 5.5663e-04, 3.8804e-01, 8.2248e-04, 4.5492e-05])

[0286] P V5 = tensor([7.5425e-05, 5.6325e-06, 1.1440e-05, 9.9804e-01, 1.1143e-05,

[0287] 1.0463e-03, 3.9157e-05, 7.2448e-04, 3.7533e-05, 1.1314e-05])

[0288] P V6 = tensor([1.8860e-04, 3.8561e-07, 7.4545e-08, 9.7139e-01, 2.4680e-04,

[0289] 2.7729e-02, 1.2760e-05, 3.8865e-04, 4.5705e-05, 1.1441e-06])

[0290] P 11 = 1.5880e-03, P 21 = 1.9877e-04, P 31 = 1.9570e-05, P 41 = 9.2094e-01, P 51 = 1.6187e-03, P 61 = 4.1865e-02, P 71 = 1.3321e-03, P 81 = 3.1760e-02, P 91 = 6.1292e-04, P 10,1 = 6.2374e-05

[0291] P 12 = 6.5333e-05, P 22 = 3.1917e-05, P 32 = 2.7537e-05, P 42 = 9.9608e-01, P 52 = 1.6657e-04, P 62 = 1.9473e-03, P 72 = 2.1735e-04, P 82 = 1.0164e-03, P 92 = 4.3555e-04, P 10,2 = 1.5461e-05

[0292] P 13 = 9.3158e-04, P 23 = 1.4728e-05, P 33 = 1.6496e-06, P 43 = 9.1835e-01, P 53 = 5.0520e-04, P 63 = 7.1503e-02, P 73 = 1.6911e-04, P 83 = 8.0133e-03, P 93= 4.9392e-04, P 10,3 = 1.4143e-05

[0293] P 14 = 1.5327e-03, P 24 = 9.2194e-04, P 34 = 1.8895e-05, P 44 = 5.6545e-01, P 54 = 6.4302e-04, P 64 = 4.1963e-02, P 74 = 5.5663e-04, P 84 = 3.8804e-01, P 94 = 8.2248e-04, P 10,4 = 4.5492e-05

[0294] P 15 = 7.5425e-05, P 25 = 5.6325e-06, P 35 = 1.1440e-05, P 45 = 9.9804e-01, P 55 = 1.1143e-05, P 65 = 1.0463e-03, P 75 = 3.9157e-05, P 85 = 7.2448e-04, P 95 = 3.7533e-05, P 10,5 = 1.1314e-05

[0295] P 16 = 1.8860e-04, P 26 = 3.8561e-07, P 36 = 7.4545e-08, P 46 = 9.7139e-01, P 56 = 2.4680e-04, P 66 = 2.7729e-02, P 76 = 1.2760e-05, P 86 = 3.8865e-04, P 96 = 4.5705e-05, P 10,6 = 1.1441e-06

[0296] Step 6-2 Use the softmax outputs of F1, F2, …, F6 from 6 perspectives of the 3D model M for probability voting, and for the F at the i-th perspective i belongs to class C jThe probability P for (j = 1, 2,..., 10) ji Calculate C as the voting value for the corresponding class j The votes T obtained by the class vote (C j |M), take the class with the most votes as the classification result of M, and the calculation process is as follows:

[0297] P ji = P(C j |F i )

[0298]

[0299] Figure 1 The voting calculation process of the test model shown is as follows:

[0300] T vote T(C1|desk) = 1.5880e - 03 + 6.5333e - 05 + 9.3158e - 04 + 1.5327e - 03 + 7.5425e - 05 + 1.

[0301] 8860e - 04 = 0.0044

[0302] T vote T(C2|desk) = 1.9877e - 04 + 3.1917e - 05 + 1.4728e - 05 + 9.2194e - 04 + 5.6325e - 06 + 3.

[0303] 8561e - 07 = 0.0012

[0304] T vote T(C3|desk) = 1.9570e - 05 + 2.7537e - 05 + 1.6496e - 06 + 1.8895e - 05 + 1.1440e - 05 + 7.

[0305] 4545e - 08 = 0.00008

[0306] T vote T(C4|desk) = 9.2094e - 01 + 9.9608e - 01 + 9.1835e - 01 + 5.6545e - 01 + 9.9804e - 01 + 9.

[0307] 7139e - 01 = 4.4519

[0308] T vote T(C5|desk) = 1.6187e - 03 + 1.6657e - 04 + 5.0520e - 04 + 6.4302e - 04 + 1.1143e - 05 + 2.

[0309] 4680e-04 = 0.0032

[0310] T vote (C6|desk) = 4.1865e-02 + 1.9473e-03 + 7.1503e-02 + 4.1963e-02 + 1.0463e-03 + 2.

[0311] 7729e-02 = 0.1861

[0312] T vote (C7|desk) = 1.3321e-03 + 2.1735e-04 + 1.6911e-04 + 5.5663e-04 + 3.9157e-05 + 1.

[0313] 2760e-05 = 0.0023

[0314] T vote (C8|desk) = 3.1760e-02 + 1.0164e-03 + 8.0133e-03 + 3.8804e-01 + 7.2448e-04 + 3.

[0315] 8865e-04 = 0.4299

[0316] T vote (C9|desk) = 6.1292e-04 + 4.3555e-04 + 4.9392e-04 + 8.2248e-04 + 3.7533e-05 + 4.

[0317] 5705e-05 = 0.0024

[0318] T vote (C 10 (|desk) = 6.2374e-05 + 1.5461e-05 + 1.4143e-05 + 4.5492e-05 + 1.1314e-05 + 1.

[0319] 1441e-06 = 0.0001

[0320] argmaxT vote (C4|desk) = 4.4519

[0321] Obtained by voting Figure 1 The final classification result of the shown test model is the 4th class, that is, the desk class, and the classification is correct. Implementing the present invention on the test data, the classification accuracy rate is as follows:

[0322] Classification accuracy rate = 93.5%

[0323] The 3D model classification method integrating view features and multi-branch networks implemented by the embodiments of the present invention designs a multi-branch parallel neural network on the basis of the view-based 3D model classification method. It uses the ConvNeXt network to extract global features, uses the ECA-ResNet and one-dimensional convolutional neural network to extract local features and shape features, and performs weighted fusion of the global features, local features and shape features to enhance the representational ability of the features for the model and solve the problem of ignoring view detail information. Experiments prove that this method has good results for 3D model classification.

[0324] The above is a detailed introduction to the embodiments of the present invention in conjunction with the accompanying drawings. The specific implementation manners herein are only used to help understand the method of the present invention. For those of ordinary skill in the art, according to the idea of the present invention, various changes and modifications can be made within the specific implementation manners and application scope. Therefore, the present specification should not be construed as a limitation to the present invention.

Claims

1. A three-dimensional model classification method that fuses view features and a multi-branch network, characterized in that, The method includes the following steps: Step 1: Extract two-dimensional views from the 3D models of ModelNet10. Place the 3D models in a normalized coordinate system, and project a series of views by selecting fixed viewpoints; Step 2: Use the Canny algorithm to extract the contours of all the views obtained from ModelNet10. Extract the D1 feature, Hu moment feature, and corner curvature feature of the contours to form a one-dimensional shape feature vector; Step 3: Construct training data from the views, contours, and shape feature vectors extracted from the ModelNet10 training model, and construct test data from the views, contours, and shape feature vectors extracted from the ModelNet10 test model; Step 4: Design a multi-branch network model with a parallel connection of a ConvNeXt network, an ECA-ResNet network, and a one-dimensional convolutional neural network; Step 5: Optimize the multi-branch network using the training data to obtain an optimized multi-branch network model. Use the ConvNeXt branch to extract global features from the views of the test data, use the ECA-ResNet branch to extract local features from the contours of the test data, use the one-dimensional convolutional neural network branch to extract shape features from the shape vectors of the test data, assign different weights to the global features, local features, and shape features, and perform weighted fusion; Step 6: Use the softmax function to calculate the classification prediction probability of the fused features, and vote using the classification prediction probabilities of all viewpoints of the 3D model to obtain the classification result of the model.

2. The 3D model classification method integrating view features and multi-branch network according to claim 1, characterized in that In the said Step 1, the specific steps for extracting two-dimensional views from the 3D models of ModelNet10 are as follows: Step 1-1: Preprocess the 3D models in ModelNet10 and place the 3D models in a normalized coordinate system; Step 1-2: Place the camera above the model, fix a viewpoint every 60 degrees, and the camera projects to generate a two-dimensional view; Step 1-3: Each 3D model obtains six-viewpoint views V1, V2, V3, V4, V5, and V6 through projection.

3. The three-dimensional model classification method integrating view features and a multi-branch network according to claim 1, characterized in that In the said Step 2, the specific steps for using the Canny algorithm to extract the contours of all the views obtained from ModelNet10, and extracting the D1 feature, Hu moment feature, and corner curvature feature of the contours to form a one-dimensional shape feature vector are as follows: Step 2-1: Use the Canny algorithm to extract the contours of the views. The specific steps are as follows: Step 2-1-1: Perform grayscale processing on the views; Step 2-1-2: Apply a Gaussian filter to smooth the views. Substitute the horizontal and vertical coordinate indices of the corresponding points in the filter into the Gaussian function to remove the view noise. The Gaussian function calculation formula is as follows: where (x, y) is the point coordinate and σ is the standard deviation; Step 2-1-3 uses the Soble operator to calculate the gradient value G in the horizontal direction x and the gradient value G in the vertical direction y , and calculates the gradient value G and the gradient direction θ of the smoothed view. The calculation formulas are as follows: Step 2-1-4: Perform edge detection based on the view gradient values. Set the upper threshold and the lower threshold, select the edges in the view with gradient values greater than the upper threshold, discard the edges less than the lower threshold, and connect the edges to form contours; Step 2-2: Extract the D1 feature of the contours. The specific steps are as follows: Step 2-2-1: Calculate the p+q-th order origin moment of the contours. The calculation formula is as follows: Among them, ρ(x, y) represents the contour function, R represents the number of rows, and C represents the number of columns; In step 2-2-2, the centroid coordinates (x0, y0) are calculated using the zeroth-order and first-order moments of the contour about the origin, and the calculation formula is as follows: Step 2-2-3 performs random equidistant sampling on the points on the contour. Let the coordinate set of the sampling points be Point = {(x1, y1), …, (x i , y i ), …, (x n , y n )}. Randomly select N points from Point to form the set PD1 = {P1, …, P k , …, P N}. Bins represents the number of intervals, and BinsSize represents the interval length. The calculation formula is as follows: BinsSize = max({dist(P, O)|P ∈ PD1}) / N Among them, dist() is the Euclidean distance between two points, O is the centroid of the contour, and max() represents taking the maximum value; Step 2-2-4 uses D1 to characterize the distance between randomly sampled points on the contour and the centroid, and the calculation of D1_v i is as follows: D1_v i = |{P | dist(P, O) ∈ (BinSize * (i - 1), BinSize * i), P ∈ PD1}| In step 2-3, the Hu moment features of the contour are extracted. The specific steps are as follows: Step 2-3-1 calculates the central moment μ of order p+q of the contour from the centroid coordinates pq , and the calculation formula is as follows: Step 2-3-2 normalizes the (p+q)-th central moment of the contour using the zero-th central moment, and calculates the (p+q)-th normalized central moment η of the contour pq , and the calculation formula is as follows: In step 2-3-3, seven invariant moment groups of the contour Hu moment are calculated through the second-order and third-order normalized central moments of the contour. The calculation formula is as follows; H1 = η 20 + η 02 H3 = (η 30 - 3×η 12 ) 2 +(3×η 21 - η 03 ) 2 H4 = (η 30 + η 12 ) 2 +(η 21 + η 03 ) 2 H5 = (η 30 - 3×η 12 )×(η 30 + η 12 )×[(η 30 + η 12 ) 2 - 3×(η 21 + η 03 ) 2 + (3×η 21 -η 03 )×(η 21 +η 03 )×[3×(η 30 +η 12 ) 2 -(η 21 +η 03 ) 2 ​ H6 = [(η 30 + η 12 ) 2 -(η 12 + η 03 ) 2 × (η 20 - η 02 ) + 4 × η 11 × (η 30 + η 12 ) × (η 21 + η 03 ) H7 = (3 × η 21 - η 03 ) × (η 30 + η 12 ) × [(η 30 + η 12 ) 2 - 3 × (η 21 + η 03 ) 2 - (3×η 12 -η 30 )×(η 21 +η 03 )×[3×(η 30 +η 12 ) 2 -(η 21 +η 03 ) 2 ​ In step 2-3-4, the calculation results of the seven invariant moments of the contour Hu moment are formed into a one-dimensional vector, which is used as the contour Hu moment feature; In step 2-4, the corner curvature features of the contour are extracted. The specific steps are as follows: In step 2-4-1, the contour corners are extracted. By establishing a detection window, with a pixel point as the center, displacing in the horizontal and vertical directions, the points where the gray value changes significantly are the corners of the contour, and the corner coordinates are extracted from the window; Step 2-4-2 randomly samples 6 points from the extracted contour corner points, and the coordinates are denoted as (x i , y i ), where i = 1, 2, …, 6. All the sampled points are formed into a two-dimensional array f = [[x1, x2, …, x6], [y1, y2, …, y6]], and calculate the gradients of x i , y i in the two-dimensional array f. The calculation formula is as follows: where x i ′ is the first-order gradient of x i , and x i ″ is the second-order gradient of x i . y i ′ is the first-order gradient of y i , and y i ″ is the second-order gradient of y i ; Step 2-4-3 uses x i , y i to calculate the curvature k i , y i ) of the point (x i (i = 1, 2, …, 6), and the calculation formula is as follows: Step 2-4-4 uses the calculated corner curvature feature k i (i = 1, 2, …, 6) to construct a one-dimensional vector as the corner curvature feature; In step 2-5, the Hu moment features and corner curvature features are logarithmically transformed to make them more conform to the normal distribution. The calculation formula is as follows: Among them, a represents the data before logarithmic transformation, and b represents the data after logarithmic transformation; In step 2-6, the D1 feature, the logarithmically transformed Hu moment feature, and the logarithmically transformed corner curvature feature are combined into a one-dimensional vector, and then normalized. The processed feature vector is used as the shape feature.

4. The 3D model classification method integrating view features and multi-branch network according to claim 1, characterized in that, In step 3, the view, contour, and shape feature vectors extracted from ModelNet10 are used to form the training data and test data. The specific steps are as follows: In step 3-1, the view, contour, and shape feature vectors extracted from the 3D models in the ModelNet10 training set are used to form the training data; In step 3-2, the view, contour, and shape feature vectors extracted from the 3D models in the ModelNet10 test set are used to form the test data.

5. The 3D model classification method integrating view features and multi-branch network according to claim 1, wherein In step 4, a multi-branch network is designed. The specific steps are as follows: In step 4-1, the depth of the ConvNeXt network is determined, and ConvNeXt-T with a stacked depth B = (3, 3, 9, 3) of 4 ConvNeXt Block blocks is selected; In step 4-2, the depth of the ResNet network is determined, and ResNet-34 with a depth of 34 layers is selected. The ECA attention mechanism is used to improve ResNet, and the ECA attention module is embedded into the basicblock of ResNet to enhance the channel features of the input feature map; In step 4-3, a one-dimensional convolutional neural network is constructed. The network consists of two convolutional layers with a convolutional kernel size of 3, two pooling layers, and one fully connected layer; In step 4-4, ConvNeXt, ECA-ResNet, and the one-dimensional convolutional neural network are connected in parallel to form a multi-branch network.

6. The three-dimensional model classification method integrating view features and multi-branch network according to claim 1, characterized in that In step 5, the multi-branch network is optimized. The optimized multi-branch network is used to extract global features, local features, and shape features, and they are weighted and fused. The specific steps are as follows: Step 5-1 Optimize the multi-branch network using the training data. Optimize the ConvNeXt network using the views in the training data, optimize the ECA-ResNet using the contours in the training data, and optimize the one-dimensional convolutional neural network using the shape feature vectors in the training data; Step 5-2 uses the optimized ConvNeXt to extract global features from the views of the test data, and the output G i = {g1, g2, …, g n} of the fully connected layer is used as the global feature; Step 5-3 uses the optimized ECA-ResNet to extract local features from the contours of the test data, and takes the output T i ={t1, t2, …, t n} of the fully connected layer as the local features; Step 5-4 uses an optimized one-dimensional convolutional neural network to extract shape features from the shape feature vectors in the test data, and takes the output S i ={s1, s2, …, s n} of the fully connected layer as the shape features; Step 5-5 performs weighted fusion on the global feature G i , the local feature T i and the shape feature S i to obtain F i = {f1, f2, …, f n}, and the calculation formula is as follows: F i = α·G i + β·T i +(1 - α - β)·S i Among them, α and β are weight coefficients used to adjust the proportion of each branch. After cross-validation, the best results are obtained when α is taken as 0.7 and β is taken as 0.

2.

7. The 3D model classification method integrating view features and multi-branch network according to claim 1, wherein In the said Step 6, use the softmax function to calculate the classification prediction probability of the fused features under each perspective, and obtain the classification result of the 3D model through probability voting. The specific steps are as follows: Step 6-1 Calculate F using the softmax function i The predicted probability under the three-dimensional model category C j (j = 1, 2,..., 10): where x j represents the input data of the softmax function; Step 6-2 performs probability voting using the softmax outputs of F1, F2, …, F6 under six perspectives of the three-dimensional model M, and takes the probability P i belonging to class C j (j = 1, 2, ..., 10) as the voting value for the corresponding class, calculates the vote T ji obtained by class C j (C vote (C j |M), and takes the class with the most votes as the classification result of M. The calculation process is as follows: P ji = P(C j |F i ) Among them, argmax() is the maximum-taking function, and C u is the classification result of the three-dimensional model M, and u is the label of the classification result.

Images