Three-dimensional model classification method based on feature fusion and voting mechanism

Abstract

The application relates to a three-dimensional model classification method based on feature fusion and a voting mechanism. The application first pre-processes training models and test models in a three-dimensional model data set to obtain a group of two-dimensional views representing the three-dimensional models, and obtains low-level shape features of the views, namely D1, D2, D3, Zernike moments, Fourier descriptors and KSP features, through calculation. RegNet design space is used to realize extraction of high-level semantic features of the views. Concat is used to fuse the low-level shape features and the high-level semantic features of the views obtained from the training models to obtain training data. Concat is used to fuse the low-level shape features and the high-level semantic features of the views obtained from the test models to obtain test data. Then, the RegNet design space and LSTM are optimized by using the training data, discriminative features are extracted from the fused features of the test data by using the optimized RegNet design space and LSTM, and the discriminative features are input into Softmax, a voting algorithm is used to vote the probability distribution output by Softmax, and the classification result of the test data is obtained. The application realizes good classification of the three-dimensional models and more accurately predicts the real categories of the three-dimensional models.

Description

Technical Field

[0001] The present invention relates to a three-dimensional model classification method based on feature fusion and voting mechanism, which has good application in the field of three-dimensional model classification. Background Art

[0002] In recent years, with the continuous development of science and technology and the continuous improvement of 3D model acquisition equipment, the process of generating 3D models has become easier. Since the objective world is composed of three-dimensional objects, 3D models occupy a large part of this objective world, and therefore the application of 3D models has become increasingly widespread. Currently, 3D models have been widely used in various fields such as industrial design, virtual reality, autonomous driving, and 3D reconstruction. The quality of 3D model classification to a certain extent affects practical applications. Therefore, how to efficiently and reasonably classify 3D models has become a hot topic of research for many scholars. Currently, 3D model classification methods can be divided into point cloud-based methods, voxel-based methods, and view-based methods based on the different representation forms of extracted 3D model feature descriptors. View-based classification methods can fully utilize the superior performance of convolutional neural networks to obtain 3D model view features, achieving better classification results. Therefore, view-based 3D model classification is a key focus of future 3D model classification research.

[0003] The view-based classification method first uses a virtual camera to perform a two-dimensional geometric projection on the three-dimensional model to obtain a set of two-dimensional views that represent the model; then a convolutional neural network is used to obtain high-level semantic features of the view; finally, a pooling operation is used to achieve feature fusion between views to complete the three-dimensional model classification. However, the existing projection method will produce a large number of redundant views, and the fusion of features between multiple views may cause the submergence and confusion of beneficial features. In addition, the two-dimensional view can only represent partial information on the surface of the three-dimensional model, and to a certain extent, it is difficult to capture the overall shape information of the model. In response to the above problems, the present invention first uses the view low-level shape features (LSF) to represent the overall shape of the three-dimensional model; then the LSF is fused with the view high-level semantic features (HSF) through Concat, and the fused features are denoised and optimized using LSTM; finally, a voting mechanism is used to reduce the redundancy of the view set to complete the three-dimensional model classification. Summary of the Invention

[0004] In order to solve the problems existing in the field of 3D model classification, the present invention proposes a 3D model classification method based on feature fusion and voting mechanism.

[0005] To this end, the present invention provides the following technical solutions.

[0006] A 3D model classification method based on feature fusion and voting mechanism, the method comprising the following steps:

[0007] Step 1: Preprocess the 3D model dataset to obtain a set of 2D views representing the 3D model.

[0008] Step 2: Apply the shape function of the 3D model to the 2D view. Use the D1 shape function to describe the distance between the center point of the 2D view and a random point on the boundary. Use the D2 shape function to describe the distance between two random points on the boundary of the view. Use the D3 shape function to describe the square root of the area formed by three random points on the boundary of the view.

[0009] Step 3: Extract the Zernike moment features of the 2D view and normalize them to the range of (0, 1) using the standard moment method;

[0010] Step 4: Extract the contour features of the 2D view, use Fourier descriptors to represent the view contour information, and further normalize the obtained Fourier descriptors to obtain a feature vector sequence that is invariant to translation, scaling, and rotation.

[0011] Step 5: Extract local texture features of the view, use the KSP feature vector to represent the view texture information, and concatenate the KSP feature with the D1, D2, D3, Zernike moment features and Fourier features obtained in steps 2, 3, and 4 to obtain the LSF;

[0012] Step 6: Extract the HSF of the view based on the RegNet design space. Use Concat to fuse the LSF obtained in step 5 with the HSF to obtain a joint descriptor of the 3D model. Use the 3D model training set after feature fusion in the 3D model dataset as training data, and use the 3D model test set after feature fusion in the 3D model dataset as test data.

[0013] Step 7: Use the training data to optimize the RegNet design space and LSTM. Use the optimized RegNet design space and LSTM to extract discriminant features from the fused features of the test data, and then input them into Softmax. Use the voting algorithm to vote on the probability distribution of Softmax output to obtain the classification results of the test data.

[0014] Specifically, in step 1, the three-dimensional model data set is preprocessed, and the specific steps are as follows:

[0015] Step 1-1: Construct the rendering background of the 3D model, set 8 point light sources at different angles, and use the Phong reflection model to realize the 2D geometric projection of the 3D model;

[0016] In step 1-2, based on the rendering background constructed in step 1-1, six virtual cameras are evenly placed on a circle with a 30° angle to the horizontal plane, diagonally above the center of the 3D model. The virtual cameras are pointed toward the center of the model, and a 2D view is generated every 60° along the circumference.

[0017] Step 1-3 uses a set of acquired two-dimensional views to represent a three-dimensional model, and constructs a multi-view representation set V(M) of the model = {v l ,1≤l≤6}, where v l Represents a 2D view of a given 3D model M.

[0018] Specifically, in step 2, the extraction of D1, D2 and D3 is carried out as follows:

[0019] Step 2-1 Sample N points from the view boundary and construct a set PD1 = {p1, p2, ..., p N}, the set of D1 shape distribution is {D1 _w1 ,…,D1 _wi ,…,D1 _wBins}, where D1 _wi It is the statistic within the interval (BSize*(i-1), BSize*i], Bins is the number of intervals, BSize is the interval length, D1 _wi is defined as follows:

[0020] D1 _wi ={p|dist(p,O)∈(BSize*(i-1),BSize*i],p∈PD1}

[0021] Here BSize = max({dist(p,O)|p∈PD1}) / N, dist(·) represents the Euclidean distance between two points;

[0022] Step 2-2 Sample N point pairs from the view boundary and construct a set PD2 = {(p1,q1),(p2,q2),…,(p N ,q N )}, the set of D2 shape distributions is {D2 _w1 ,…,D2 _wi ,…,D2 _wBins}, where D2 _wi is the statistic within (BSize*(i-1), BSize*i], D2 _wi is defined as follows:

[0023] D2 _wi ={(p,q)|dist(p,q)∈(BSize*(i-1),BSize*i],(p,q)∈PD2}

[0024] Here BSize = max({dist(p,q)|(p,q)∈PD2}) / N;

[0025] Step 2-3 samples N triplet points from the view boundary and constructs the set PD3 = {(p1,q1,r1),(p2,q2,r2),…,(p N ,q N ,r N )}, the set of D3 shape distributions is {D3 _w1 ,…,D3 _wi ,…,D3 _wBins}, where D3 _wi is the statistic within (BSize*(i-1), BSize*i], D3 _wi is defined as follows:

[0026] D3 _wi ={(p,q,r)|Q∈(BSize*(i-1),BSize*i],(p,q,r)∈PD3}

[0027] Here BSize = max({H(p,q,r)|(p,q,r)∈PD3}) / N, where H(·) is used to calculate the area Q of the triangle (p,q,r), and Q is defined as follows:

[0028]

[0029] Among them, l1=dist(p,q), l2=dist(p,r), l3=dist(q,r), a=(l1+l2+l3) / 2.

[0030] Specifically, in step 3, the extraction of the Zernike moment features of the view is carried out in the following steps:

[0031] Step 3-1 Zernike moments are orthogonal complex moments based on Zernike polynomials, which are composed of a set of complex polynomials V nm (r,θ) is expressed as follows:

[0032] V nm (x,y)=V nm (r,θ)=R nm (r)e jmθ

[0033] Where r=(x 2 +y 2 ) 1 / 2 ≤1, θ=arctan(y / x), (r,θ) is the polar coordinate of (x,y) on the unit circle, n-|m| is an even number and |m|≤n, Rnm (r) is a radial polynomial, which is expressed as follows:

[0034]

[0035] Step 3-2 constructs a polar coordinate system based on the view center, and then maps the normalized coordinates of the pixel points to the inside of the unit circle. For a two-dimensional view f(x,y), its Zernike moment is defined as follows:

[0036]

[0037] Where * represents conjugate complex number;

[0038] Specifically, in step 4, the Fourier descriptor of the view is extracted, and the specific steps are:

[0039] Step 4-1 View center point (x m ,y m ) is defined as follows:

[0040]

[0041] Where (x i ,y i ) is a pixel point on the view contour, and i=1,2,…,N;

[0042] Step 4-2 The center distance function is defined as follows:

[0043]

[0044] Where f(i) is the value of the view in the spatial domain, and the discrete Fourier transform is expressed as follows:

[0045]

[0046] Where u=0,1,…,N-1;

[0047] Step 4-3 normalizes F(u):

[0048]

[0049] Specifically, in step 5, the KSP feature vector is used to represent the view texture information, and the KSP feature is concatenated with D1, D2, D3, Zernike moment feature and Fourier feature to obtain LSF. The specific steps are:

[0050] Step 5-1 defines a 3×3 window in the view. Set the pixels whose grayscale values ​​are greater than or equal to the threshold K to 1, and set the pixels whose grayscale values ​​are less than the threshold K to 0. K is defined as follows:

[0051]

[0052] where max 3×3 、min 3×3 They represent the maximum grayscale value and the minimum grayscale value in a 3×3 window, respectively, and α is a scaling factor;

[0053] In step 5-2, count the number of pixels set to 1 among these 9 pixels. Use the square of this number to represent the KSP code of the view in the 3×3 window. Traverse all pixels in the entire view to obtain the KSP code of the view.

[0054] Step 5-3 uses the probability density of each interval in the KSP code as the KSP feature, and concatenates the KSP feature with the D1, D2, D3, Zernike moment feature and Fourier feature obtained in steps 2, 3, and 4 to obtain the LSF;

[0055] Specifically, in step 6, Concat is used to fuse the HSF and LSF features to obtain training data and test data. The specific steps are as follows:

[0056] Step 6-1: Input the 2D view set of the 3D model into the RegNet design space to extract HSF;

[0057] In step 6-2, the LSF obtained in step 5-3 and the HSF obtained in step 6-1 are fused using Concat. The 3D model training set after feature fusion in the 3D model dataset is used as training data, and the 3D model test set after feature fusion in the 3D model dataset is used as test data.

[0058] Specifically, in step 7, the training data is used to optimize the RegNet design space and LSTM, and the optimized RegNet design space and LSTM are used to extract discriminant features from the fusion features of the test data, which are then input into Softmax. The voting algorithm is used to vote on the probability distribution of the Softmax output to obtain the classification result of the test data. The specific steps are:

[0059] Step 7-1: Optimize the RegNet design space and LSTM using training data.

[0060] Step 7-2 uses the optimized RegNet design space and LSTM to extract discriminant features from the fusion features of the test data and inputs them into Softmax to obtain the probability distribution p lj , where l = 1, 2, ..., 6, j = 0, 1, ..., c-1, c is the number of 3D model categories;

[0061] Step 7-3p ljThe Shannon entropy calculation process is as follows:

[0062]

[0063] Shannon entropy H l The smaller it is, the higher the reliability of the Softmax is, the more accurate the classification result is, and the larger the weight corresponding to the Softmax is. According to the normalized exponential function, the Shannon entropy is converted into the Softmax weight w l :

[0064]

[0065] Step 7-4 is to expand the probability distribution p lj The discreteness will be greater than the threshold k p lj Put it into the set T with weight kw l The definition is as follows:

[0066]

[0067] Among them, maxset(T) represents selecting the maximum value from the set T;

[0068] Step 7-5 Calculate the weighted probability distribution Q lj :

[0069]

[0070] Adding the columns of Q gives us vector V, as shown below:

[0071]

[0072] Step 7-6 For a given 3D model M, the category determination process is as follows:

[0073]

[0074] Beneficial effects:

[0075] The present invention is a three-dimensional model classification method based on feature fusion and voting mechanism.

[0076] 1. This paper extracts high-level semantic features (HSFs) from 3D model views based on the RegNet design space. Unlike traditional neural networks that rely heavily on expert prior knowledge, the neural networks designed using the RegNet design space have strong generalization and interpretability while maintaining low complexity.

[0077] 2. Due to modal differences between 2D views and 3D models, a single HSF can only represent partial model information and cannot capture the overall shape. To more comprehensively represent 3D model information, we use Concat to fuse the view's low-level shape features (LSF) and HSF, leveraging their complementary strengths. This creates a joint descriptor for the 3D model, and then optimizes the fused features using LSTM.

[0078] 3. The two-dimensional view set obtained by the existing method will generate a large number of redundant views, and each view is relatively independent. To address the above problem, the present invention uses a voting mechanism to vote on the classification results of each view to avoid interference from poor views, reduce view set redundancy, and improve the classification ability of the three-dimensional model.

[0079] 4. This invention was applied to the publicly available 3D model dataset ModelNet10, achieving high classification accuracy. Furthermore, the concept of this invention aligns with basic human cognition: if two 3D models appear roughly the same from multiple different perspectives, then they are considered similar. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 An example diagram of a three-dimensional model sofa in an embodiment of the present invention.

[0081] Figure 2 Flowchart of the three-dimensional model classification method in an embodiment of the present invention.

[0082] Figure 3 Schematic diagram of three-dimensional model projection in an embodiment of the present invention.

[0083] Figure 4 A three-dimensional model multi-view set in an embodiment of the present invention.

[0084] Figure 5 Schematic diagram of a two-dimensional shape function in an embodiment of the present invention.

[0085] Figure 6 Diagram of the RegNet design space in an embodiment of the present invention. DETAILED DESCRIPTION

[0086] In order to clearly and completely describe the technical solutions in the embodiments of the present invention, Figure 1 Taking the three-dimensional model of the sofa category in the ModelNet10 test set as an example, the present invention is further described in detail in combination with the ModelNet10 test set.

[0087] The flowchart of the three-dimensional model classification method based on feature fusion and voting mechanism in the embodiment of the present invention is as follows: Figure 2 As shown, the following steps are included.

[0088] Step 1: Preprocess the dataset ModelNet10, the specific steps are as follows:

[0089] Step 1-1: Construct a rendering background of the three-dimensional model, set 8 different angle point light sources, and realize the two-dimensional geometric projection of the three-dimensional model by using the Phong reflection model, the projection process of the three-dimensional model is as shown in Figure 3 ;

[0090] Step 1-2: Based on the rendering background constructed in step 1-1, 6 virtual cameras are uniformly placed on the circle with an angle of 30° between the center of the three-dimensional model and the horizontal plane, the angle of the virtual camera points to the center of the model, and a two-dimensional view is generated every 60° along the circle;

[0091] Step 1-3: Use a set of two-dimensional views obtained to represent the three-dimensional model M, and construct a multi-view representation set V(M) = {v l ,1≤l≤6} of the three-dimensional model M, where v l represents a two-dimensional view of a given three-dimensional model M, and the two-dimensional view set extracted from the three-dimensional model instance shown in Figure 1 is as shown in Figure 4 ;

[0092] Step 2: Extract D1, D2 and D3, the specific steps are as follows:

[0093] Step 2-1: The two-dimensional shape functions D1, D2 and D3 are as shown in Figure 5 , sample N points from the view boundary to construct the set PD1 = {p1, p2, …, p N}, the set of D1 shape distributions is {D1 _w1 ,…,D1 _wi ,…,D1 _wBins}, where D1 _wi is the statistic quantity in the interval (BSize*(i-1), BSize*i], Bins is the interval number, BSize is the interval length, and D1 _wi is defined as follows:

[0094] D1 _wi = {p | dist(p, O) ∈ (BSize*(i-1), BSize*i], p ∈ PD1}

[0095] Here BSize = max({dist(p, O) | p ∈ PD1}) / N, dist(·) represents the Euclidean distance between two points, and the D1 feature extracted from the multi-view set shown in Figure 4 is as follows:

[0096] D1=[[0.0400,0.1202,0.0921,…],[0.0150,0.0470,0.1281,…],[0.0120,0.0831,0.1252 ,…],[0,0.0360,0.1372,…],[0.0370,0.1131,0.1441,…],[0.0200,0.0470,0.1401,…]];

[0097] Step 2-2 Sample N point pairs from the view boundary and construct a set PD2 = {(p1,q1),(p2,q2),…,(p N ,q N )}, the set of D2 shape distributions is {D2 _w1 ,…,D2 _wi ,…,D2 _wBins}, where D2 _wi is the statistic within (BSize*(i-1), BSize*i], D2 _wi is defined as follows:

[0098] D2 _wi ={(p,q)|dist(p,q)∈(BSize*(i-1),BSize*i],(p,q)∈PD2}

[0099] Here BSize=max({dist(p,q)|(p,q)∈PD2}) / N, from Figure 4 The D2 features extracted from the multi-view set shown are as follows:

[0100] D2=[[0.0640,0.0900,0.1131,…],[0.0540,0.0600,0.1081,…],[0.0540,0.0670,0.1231,… ],[0.0590,0.0790,0.1341,…],[0.0590,0.0730,0.1041,…],[0.0560,0.0560,0.0980,…]];

[0101] Step 2-3 samples N triplet points from the view boundary and constructs the set PD3 = {(p1,q1,r1),(p2,q2,r2),…,(p N ,q N ,r N )}, the set of D3 shape distributions is {D3 _w1 ,…,D3 _wi ,…,D3 _wBins}, where D3 _wi is the statistic within (BSize*(i-1), BSize*i], D3 _wiis defined as follows:

[0102] D3 _wi ={(p,q,r)|Q∈(BSize*(i-1),BSize*i],(p,q,r)∈PD3}

[0103] Here BSize = max({H(p,q,r)|(p,q,r)∈PD3}) / N, where H(·) is used to calculate the area Q of the triangle (p,q,r), and Q is defined as follows:

[0104]

[0105] Where l1=dist(p,q), l2=dist(p,r), l3=dist(q,r), a=(l1+l2+l3) / 2, from Figure 4 The D3 features extracted from the multi-view set shown are as follows:

[0106] D3=[[0.4028,0.2234,0.1252,…],[0.2962,0.2052,0.1641,…],[0.2762,0.2322,0.1571,… ],[0.3413,0.2102,0.1651,…],[0.3630,0.2030,0.1480,…],[0.2912,0.2232,0.1801,…]];

[0107] Step 3: Extract the Zernike moment features of the view. The specific steps are as follows:

[0108] Step 3-1 Zernike moments are orthogonal complex moments based on Zernike polynomials, which are composed of a set of complex polynomials V nm (r,θ) is expressed as follows:

[0109] V nm (x,y)=V nm (r,θ)=R nm (r)e jmθ

[0110] Where r=(x 2 +y 2 ) 1 / 2 ≤1, θ=arctan(y / x), (r,θ) is the polar coordinate of (x,y) on the unit circle, n-|m| is an even number and |m|≤n, R nm (r) is a radial polynomial, which is expressed as follows:

[0111]

[0112] Step 3-2 constructs a polar coordinate system based on the view center, and then maps the normalized coordinates of the pixel points to the inside of the unit circle. For a two-dimensional view f(x,y), its Zernike moment is defined as follows:

[0113]

[0114] Among them, * represents the conjugate complex number, from Figure 4 The Zernike moment features extracted from the multi-view set shown are as follows: Zernike = [[-0.2531, 0.3724, 0.1689, …], [0.0248, -0.0462, 0.1748, …], [0.0405, -0.0787, 0.1983, …], [-0.2843, 0.0292, 0.2867, …], [-0.0048, 0.1628, -0.3511, …], [0.1548, 0.2047, -0.0067, …]];

[0115] Step 4: Extract the Fourier descriptor of the view. The specific steps are:

[0116] Step 4-1 View center point (x m ,y m ) is defined as follows:

[0117]

[0118] Where (x i ,y i ) is a pixel point on the view contour, and i=1,2,…,N;

[0119] Step 4-2 The center distance function is defined as follows:

[0120]

[0121] Where f(i) is the value of the view in the spatial domain, and the discrete Fourier transform is expressed as follows:

[0122]

[0123] Where u=0,1,…,N-1;

[0124] Step 4-3 normalizes F(u):

[0125]

[0126] from Figure 4 The Fourier features extracted from the multi-view set shown are as follows:

[0127] Fourier=[[0.2037,0.1139,0.0629,…],[0.1961,0.0891,0.1138,…],[0.1897,0.0948,0.0849 ,…],[0.1931,0.1083,0.0768,…],[0.2022,0.0969,0.0858,…],[0.1905,0.0939,0.1004,…]];

[0128] Step 5: Use the KSP feature vector to represent the view texture information, and concatenate the KSP feature with D1, D2, D3, Zernike moment feature and Fourier feature to obtain LSF. The specific steps are as follows:

[0129] Step 5-1 defines a 3×3 window in the view. Set the pixels whose grayscale values ​​are greater than or equal to the threshold K to 1, and set the pixels whose grayscale values ​​are less than the threshold K to 0. K is defined as follows:

[0130]

[0131] where max 3×3 、min 3×3 They represent the maximum grayscale value and the minimum grayscale value in a 3×3 window, respectively, and α is a scaling factor;

[0132] Step 5-2 counts the number of pixels set to 1 in these 9 pixels, and uses the square of this value to represent the KSP code of the view in the 3×3 window. Traverse all pixels of the entire view to obtain the KSP code of this view; Step 5-3 uses the probability density of each interval in the KSP code as the KSP feature, from Figure 4 The KSP features extracted from the multi-view set shown are as follows:

[0133] KSP=[[0.00331,0.00065,0.00066,…],[0.00310,0.00108,0.00132,…],[0.00343,0.00131,0.00152,…],[0.00407,0.00095,0.00092,…],[0.00329,0.00124,0.00157,…],[0.00290,0.00100,0.00117,…]], and concatenate the KSP feature with the D1, D2, D3, Zernike moment feature and Fourier feature obtained in steps 2, 3 and 4 to obtain LSF. Figure 4The LSF extracted from the multi-view set shown is as follows: LSF = [[0.2037, 0.1139, 0.0629, …, 0.0400, 0.1202, 0.0921, …, 0.0640, 0.0900, 0.1131, …, 0.4028, 0.2234, 0.1252, …, -0.2531, 0.3724, 0.1689, …, 0.00331, 0.00065, 0.00066, …], [0.1961, 0.0891, 0.1138, …, 0.0150, 0.0470, 0.1281, …, 0.0540, 0.0600,0.1081,…,0.2962,0.2052,0.1641,…,0.0248,-0.0462,0.1748,…,0.00310,0.00108,0.00132,…],[0.1897,0.0948,0.0849,…,0.0120,0.0831,0.1252,…,0.0540,0.0670,0.1231,…,0.2762,0.2322,0.1571,…,0.0405,-0.0787,0.1983,…,0.00343,0.0013 1,0.00152,…],[0.1931,0.1083,0.0768,…,0,0.0360,0.1372,…,0.0590,0.0790,0.1341,…,0.3413,0.2102,0.1651,…,-0.2843,0.0292,0.2867,…,0.00407,0.00095,0.00092,…],[0.2022,0.0969,0.0858,…,0.0370,0.1131,0.1441,…,0.0590,0.0730,0.1041,… ,0.3630,0.2030,0.1480,…,-0.0048,0.1628,-0.3511,…,0.00329,0.00124,0.00157,…],[0.1905,0.0939,0.1004,…,0.0200,0.0470,0.1401,…,0.0560,0.0560,0.0980,…,0.2912,0.2232,0.1801,…,0.1548,0.2047,-0.0067,…,0.00290,0.00100,0.00117,…]];

[0134] Step 6: Use Concat to fuse the HSF and LSF features to obtain training data and test data. The specific steps are as follows:

[0135] Step 6-1 Input the 2D view set of the 3D model into the RegNet design space to extract HSF. The RegNet design space is as follows: Figure 6 As shown, from Figure 4 The HSF extracted from the multi-view set shown is as follows:

[0136] HSF=[[2.1087,1.0969,0.05534,…],[2.9560,0.2435,0.0056,…],[0.7929,0.4999, 0,…],[1.1774,0.5657,0,…],[1.3218,0.2720,0,…],[1.7902,0.7559,0.0034,…]];

[0137] Step 6-2 uses Concat to fuse the LSF obtained in step 5-3 with the HSF obtained in step 6-1. Figure 4 The fusion features extracted from the multi-view set shown are as follows:

[0138] JD=[[2.1087,1.0969,0.05534,…,0.2037,0.1139,0.0629,…,0.0400,0.1202,0.0921,…,0.0640,0.0900,0.1131,…,0.4028,0.2234,0.1252,…,-0.2531,0.3724,0.1689,…,0.00331,0.00065,0.00066,…],[2.9560,0.2435,0.0056,…,0.1961,0.0891,0.1138,…,0.0150,0.0470,0.1281,…,0.0540,0.0600,0.1081,…,0.2962,0.2052,0.1641,…,0.0248,-0.0462,0.1748,…,0.00310,0.00108,0.00132,…],[0.7929,0.4999,0,…,0.1897,0.0948,0.0849,…,0.0120,0.0831,0.1252,…,0.0540,0.0670,0.1231,…,0.2762,0.2322,0.1571,…,0.0405,-0.0787,0.1983,…,0.00343,0.00131,0.00152,…],[1.1774,0.5657,0,…,0.1931,0.1083,0.0768,…,0,0.0360,0.1372,…,0.0590,0.0790,0.1341,…,0.3413,0.2102,0.1651,…,-0.2843,0.0292,0.2867,…,0.00407,0.00095,0.00092,…],[1.3218,0.2720,0,…,0.2022,0.096,0.0858,…,0.0370,0.1131,0.1441,…,0.0590,0.0730,0.1041,…,0.3630,0.2030,0.1480,…,-0.0048,0.1628,-0.3511,…,0.00329,0.00124,0.00157,…],[1.7902,0.7559,0.0034,…,0.1905,0.0939,0.1004,…,0.0200,0.0470,0.1401,…,0.0560,0.0560,0.0980,…,0.2912,0.2232,0.1801,…,0.1548,0.2047,-0.0067,…,0.00290,0.00100,0.00117,…]], the 3D model training set after feature fusion in ModelNet10 is used as training data, and the 3D model test set after feature fusion in ModelNet10 is used as test data.

[0139] Step 7: Use the training data to optimize the RegNet design space and LSTM. Use the optimized RegNet design space and LSTM to extract discriminant features from the fused features of the test data, then input them into Softmax. Use the voting algorithm to vote on the probability distribution of the Softmax output to obtain the classification results of the test data. The specific steps are as follows:

[0140] Step 7-1: Optimize the RegNet design space and LSTM using training data.

[0141] Step 7-2 uses the optimized RegNet design space and LSTM to extract discriminant features from the fusion features of the test data and inputs them into Softmax to obtain the probability distribution p lj , where l = 1, 2, …, 6, j = 0, 1, …, c-1, c is the number of 3D model categories, and the value of c is 10;

[0142] Step 7-3p lj The Shannon entropy calculation process is as follows:

[0143]

[0144] Shannon entropy H l The smaller it is, the higher the reliability of the Softmax is, the more accurate the classification result is, and the larger the weight corresponding to the Softmax is. According to the normalized exponential function, the Shannon entropy is converted into the Softmax weight w l :

[0145]

[0146] Step 7-4 is to expand the probability distribution p lj The discreteness will be greater than the threshold k p lj Put it into the set T, k = 0.3, and the weight kw l The definition is as follows:

[0147]

[0148] Among them, maxset(T) represents selecting the maximum value from the set T;

[0149] Step 7-5 Calculation Figure 4 The weighted probability distribution of multiple view sets is shown:

[0150] Q=[[0.00093,0.001,0.00013,0.00007,0,0,0,0.99766,0,0.00017],[0.00081,0.00219,0.00004,0.00035,0,0.0.00001,0, 0.99630,0.00001,0.00025],[0.05356,0.00572,0.00005,0.00014,0,0.00003,0,0.93652,0.00007,0.00387],[0.03242, 0.00644,0.00028,0.00003,0,0,0,0.95781,0.00002,0.00298],[0.00227,0.00568,0.00038,0.00009,0,0.00001,0,0.99120,0.00001,0.00033],[0.00038,0.00391,0.00042,0.00061,0,0.00002,0,0.99346,0,0.00119]], and we can get the vector V by adding the columns of Q as follows:

[0151] V=[0.09037,0.02494,0.00130,0.00129,0,0.00007,0,5.8729,0.00011,0.00879];

[0152] Step 7-6 Figure 1 The category determination process for the 3D model instance shown is as follows:

[0153]

[0154] V7=5.8729 is the largest, the 7th category is sofa, Figure 1 The category of the three-dimensional model instance shown is sofa;

[0155] The present invention is implemented in the 3D model test set in ModelNet10, and the classification accuracy is 862 / 908=94.93%.

[0156] The three-dimensional model classification method based on feature fusion and voting mechanism implemented in the embodiment of the present invention adopts feature fusion and voting mechanism to achieve a high classification accuracy.

[0157] The above is a detailed description of the embodiments of the present invention in conjunction with the accompanying drawings. The specific embodiments herein are only intended to help understand the method of the present invention. For those skilled in the art, according to the concept of the present invention, changes and modifications may be made in the specific embodiments and application scope. Therefore, this specification should not be understood as limiting the present invention.

Claims

1. A 3D model classification method based on feature fusion and voting mechanism, characterized in that: The method comprises the following steps: Step 1: Preprocess the 3D model dataset to obtain a set of 2D views representing the 3D model. Step 2: Apply the shape function of the 3D model to the 2D view. Use the D1 shape function to describe the distance between the center point of the 2D view and a random point on the boundary. Use the D2 shape function to describe the distance between two random points on the boundary of the view. Use the D3 shape function to describe the square root of the area formed by three random points on the boundary of the view. Step 3: Extract the Zernike moment features of the 2D view and normalize them to the range of (0, 1) using the standard moment method; Step 4: Extract the contour features of the 2D view, use Fourier descriptors to represent the view contour information, and further normalize the obtained Fourier descriptors to obtain a feature vector sequence that is invariant to translation, scaling, and rotation. Step 5: Extract the local texture features of the view, use the KSP feature vector to represent the view texture information, and concatenate the KSP feature with the D1, D2, D3, Zernike moment features and Fourier features obtained in steps 2, 3, and 4 to obtain the low-level shape feature LSF. Step 5-1 defines a 3×3 window in the view. Set the pixels whose grayscale values ​​are greater than or equal to the threshold K to 1, and set the pixels whose grayscale values ​​are less than the threshold K to 0. K is defined as follows: Among them, max3×3 and min3×3 represent the maximum grayscale value and the minimum grayscale value in the 3×3 window respectively, and α is a scaling factor; In step 5-2, count the number of pixels set to 1 among these 9 pixels. Use the square of this number to represent the KSP code of the view in the 3×3 window. Traverse all pixels in the entire view to obtain the KSP code of the view. Step 5-3 uses the probability density of each interval in the KSP code as the KSP feature, and concatenates the KSP feature with the D1, D2, D3, Zernike moment feature and Fourier feature obtained in steps 2, 3, and 4 to obtain the LSF; Step 6: Extract the high-level semantic features (HSF) of the view based on the RegNet design space. Use Concat to fuse the LSF obtained in step 5 with the HSF to obtain a joint descriptor of the 3D model. Use the 3D model training set after feature fusion in the 3D model dataset as training data, and use the 3D model test set after feature fusion in the 3D model dataset as test data. Step 7: Use the training data to optimize the RegNet design space and LSTM. Use the optimized RegNet design space and LSTM to extract discriminant features from the fused features of the test data, and then input them into Softmax. Use the voting algorithm to vote on the probability distribution of Softmax output to obtain the classification results of the test data.

2. A three-dimensional model classification method based on feature fusion and voting mechanism according to claim 1, characterized in that: In step 1, the three-dimensional model data set is preprocessed, and the specific steps are as follows: Step 1-1: Construct the rendering background of the 3D model, set 8 point light sources at different angles, and use the Phong reflection model to realize the 2D geometric projection of the 3D model; In step 1-2, based on the rendering background constructed in step 1-1, six virtual cameras are evenly placed on a circle with a 30° angle to the horizontal plane, diagonally above the center of the 3D model. The virtual cameras are pointed toward the center of the model, and a 2D view is generated every 60° along the circumference. Step 1-3 uses a set of acquired two-dimensional views to represent a three-dimensional model, and constructs a multi-view representation set V(M) of the model = {v l ,1≤l≤6}, where v l Represents a 2D view of a given 3D model M.

3. The three-dimensional model classification method based on feature fusion and voting mechanism according to claim 1 is characterized in that: In step 2, the extraction of D1, D2 and D3 is carried out in the following steps: Step 2-1 Sample N points from the view boundary and construct a set PD1 = {p1, p2, ..., p N }, the set of D1 shape distribution is {D1 _w1 ,…,D1 _wi ,…,D1 _wBins }, where D1 _wi It is the statistic within the interval (BSize*(i-1), BSize*i], Bins is the number of intervals, BSize is the interval length, D1 _wi is defined as follows: D1 _wi ={p|dist(p,O)∈(BSize*(i-1),BSize*i],p∈PD1} Here BSize = max({dist(p,O)|p∈PD1}) / N, dist(·) represents the Euclidean distance between two points; Step 2-2 Sample N point pairs from the view boundary and construct a set PD2 = {(p1,q1),(p2,q2),…,(p N ,q N )}, the set of D2 shape distributions is {D2 _w1 ,…,D2 _wi ,…,D2 _wBins }, where D2 _wi is the statistic within (BSize*(i-1), BSize*i], D2 _wi is defined as follows: D2 _wi ={(p,q)|dist(p,q)∈(BSize*(i-1),BSize*i],(p,q)∈PD2} Here BSize = max({dist(p,q)|(p,q)∈PD2}) / N; Step 2-3 samples N triplet points from the view boundary and constructs the set PD3 = {(p1,q1,r1),(p2,q2,r2),…,(p N ,q N ,r N )}, the set of D3 shape distributions is {D3 _w1 ,…,D3 _wi ,…,D3 _wBins }, where D3 _wi is the statistic within (BSize*(i-1), BSize*i], D3 _wi is defined as follows: D3 _wi ={(p,q,r)|Q∈(BSize*(i-1),BSize*i],(p,q,r)∈PD3} Here BSize = max({H(p,q,r)|(p,q,r)∈PD3}) / N, where H(·) is used to calculate the area Q of the triangle (p,q,r), and Q is defined as follows: Among them, l1=dist(p,q), l2=dist(p,r), l3=dist(q,r), a=(l1+l2+l3) / 2.

4. The three-dimensional model classification method based on feature fusion and voting mechanism according to claim 1 is characterized in that: In step 3, the extraction of the Zernike moment features of the view is carried out in the following specific steps: Step 3-1 Zernike moments are orthogonal complex moments based on Zernike polynomials, which are composed of a set of complex polynomials V nm (r,θ) is expressed as follows: V nm (x,y)V nm (r,θ))R nm (bid jmθ Where r=(x 2 +y 2 ) 1 / 2 ≤1, θ=arctan(y / x), (r,θ) is the polar coordinate of (x,y) on the unit circle, n-|m| is an even number and |m|≤n, R nm (r) is a radial polynomial, which is expressed as follows: Step 3-2 constructs a polar coordinate system based on the view center, and then maps the normalized coordinates of the pixel points to the inside of the unit circle. For a two-dimensional view f(x,y), its Zernike moment is defined as follows: Where * represents a conjugate complex number.

5. The three-dimensional model classification method based on feature fusion and voting mechanism according to claim 1 is characterized in that: In step 4, the Fourier descriptor of the view is extracted, and the specific steps are: Step 4-1 View center point (x m ,y m ) is defined as follows: Where (x i ,y i ) is a pixel point on the view contour, and i=1,2,…,N; Step 4-2 The center distance function is defined as follows: Where f(i) is the value of the view in the spatial domain, and the discrete Fourier transform is expressed as follows: Where u=0,1,…,N-1; Step 4-3 normalizes F(u):

6. The three-dimensional model classification method based on feature fusion and voting mechanism according to claim 1 is characterized in that: In step 6, Concat is used to fuse the HSF and LSF features to obtain training data and test data. The specific steps are as follows: Step 6-1: Input the 2D view set of the 3D model into the RegNet design space to extract HSF; In step 6-2, Concat is used to fuse the LSF obtained in step 5-3 and the HSF obtained in step 6-1, and the 3D model training set after feature fusion in the 3D model dataset is used as training data, and the 3D model test set after feature fusion in the 3D model dataset is used as test data.

7. The three-dimensional model classification method based on feature fusion and voting mechanism according to claim 1 is characterized in that: In step 7, the training data is used to optimize the RegNet design space and LSTM, and the optimized RegNet design space and LSTM are used to extract discriminant features from the fusion features of the test data, which are then input into Softmax. The voting algorithm is used to vote on the probability distribution of the Softmax output to obtain the classification result of the test data. The specific steps are as follows: Step 7-1: Optimize the RegNet design space and LSTM using training data. Step 7-2 uses the optimized RegNet design space and LSTM to extract discriminant features from the fusion features of the test data and inputs them into Softmax to obtain the probability distribution p lj , where l = 1, 2, ..., 6, j = 0, 1, ..., c-1, c is the number of 3D model categories; Step 7-3 p lj The Shannon entropy calculation process is as follows: Shannon entropy H l The smaller it is, the higher the reliability of the Softmax is, the more accurate the classification result is, and the larger the weight corresponding to the Softmax is. According to the normalized exponential function, the Shannon entropy is converted into the Softmax weight w l : Step 7-4 is to expand the probability distribution p lj The discreteness will be greater than the threshold k p lj Put it into the set T with weight kw l The definition is as follows: Among them, maxset(T) represents selecting the maximum value from the set T; Step 7-5 Calculate the weighted probability distribution Q lj : Adding the columns of Q gives us vector V, as shown below: Step 7-6 For a given 3D model M, the category determination process is as follows:

Images