Multi-view three-dimensional model classification method based on feature fusion and model stacking

Abstract

The invention relates to a multi-view three-dimensional model classification method based on feature fusion and model stacking. The method comprises the following steps: firstly, projecting a three-dimensional model to obtain a two-dimensional view set by using a fixed visual angle projection mode; selecting random sampling points on the view, and calculating shape descriptors D1, D2 and D3 of the view; the method comprises the following steps: acquiring view features from a view by using HorNet, fusing view semantic features and shape distribution features, using a CatBoost model and a random forest model as base models, optimizing the CatBoost model and the random forest model by using fusion features of training data, and performing classification prediction on the view of a three-dimensional model by using the fusion features of the training data. A random forest is used as a meta-model, and prediction results of a base model CatBoost model and the random forest model are used as new features to optimize the meta-model random forest model. And finally, classifying different views by using the optimized meta-model, synthesizing prediction results of the views through a voting algorithm, and finally realizing classification of the three-dimensional model. The method has a good three-dimensional model classification effect, and can more accurately solve the problem of three-dimensional model classification.

Description

Technical field:

[0001] The present invention relates to a multi-view method based on feature fusion and model stacking. Figure 3 Dimensional model classification method. Background technology:

[0002] In recent years, with the emergence of small 3D sensors and the continuous development of computer vision, the process of generating 3D models has become more convenient, and their number has increased exponentially. A 3D model is a digital representation of an object or scene with a 3D spatial structure generated by a computer, usually composed of vertices, edges, and faces. They are widely used in games, animation, architectural design, industrial design and other fields, and can simulate the shape, texture and material of real objects. With the advancement of technology, 3D modeling tools are becoming increasingly popular, and 3D models have become a popular multimedia data type in recent years. The quality of 3D model classification affects the practical application of 3D models. How to efficiently classify and manage 3D models has become a hot topic for many scholars.

[0003] With the rapid development of deep learning, convolutional neural networks (CNN) have achieved good results in the field of semantic segmentation and classification of two-dimensional images, and also provided new ideas for three-dimensional model classification. The view-based three-dimensional model classification algorithm replaces the three-dimensional model with a set of views from different angles. Compared with other three-dimensional model classification methods, it not only reduces the amount of network calculation, but also makes the global feature extraction of the three-dimensional model more perfect. The present invention uses the HorNet convolutional neural network to extract view semantic features, and combines the Concat method to deeply fuse the view semantic features with the shape distribution features. Using the CatBoost model and the random forest model as the base model, the CatBoost model and the random forest model are optimized using the fusion features of the training data, and the fusion features of the training data are used to classify and predict the three-dimensional model views. Using the random forest as the meta-model, the prediction results of the base model CatBoost model and the random forest model are used as new features to optimize the meta-model random forest model. Finally, the optimized meta-model is used to classify different views, and the prediction results of each view are integrated through the voting algorithm to finally achieve the classification of the three-dimensional model. This innovative method significantly improves the classification accuracy of the three-dimensional model. Summary of the invention:

[0004] In order to solve the problem of 3D model classification in the field of computer vision, the present invention discloses a multi-viewing Figure 3 Dimensional model classification method.

[0005] To this end, the present invention provides the following technical solutions:

[0006] 1. A multi-view algorithm based on feature fusion and model stacking Figure 3 The method for classifying dimensional models is characterized in that the method mainly comprises the following steps:

[0007] Step 1: Project the 3D model in Modelnet10 to obtain a 2D view set.

[0008] Step 2: Use the D1 descriptor to describe the distance between the centroid of the 2D view and a random sampling point on the boundary, use the D2 descriptor to describe the distance between two random sampling points on the boundary of the 2D view, and use the D3 descriptor to describe the square root of the area of ​​the triangle formed by three random sampling points on the boundary of the 2D view.

[0009] Step 3: Extract the two-dimensional view of the three-dimensional model in Modelnet10, and extract the D1, D2, and D3 shape descriptors of the two-dimensional view to form training data and test data.

[0010] Step 4: Extract the semantic features of the view based on the HorNet design space, use Concat to fuse the shape distribution features obtained in step 3 with the view semantic features to obtain a joint descriptor of the 3D model, use the 3D model training set after feature fusion in the 3D model dataset as training data, and use the 3D model test set after feature fusion in the 3D model dataset as test data.

[0011] Step 5: Use the training data to optimize the CatBoost model, extract the discriminant features from the fused features of the training data, input them into the CatBoost model for inference, and obtain the prediction results of the training data.

[0012] Step 6: Use the training data to optimize the random forest model, extract the discriminant features from the fused features of the training data and input them into the random forest model for inference to obtain the prediction results of the training data.

[0013] Step 7: Extract the prediction results of the base model CatBoost model and random forest model on the training data as features and input them into the meta-model random forest model for training. Use the prediction results of the base model on the test data for inference to obtain the prediction results of the test data.

[0014] Step 8: Use the voting algorithm to vote on the view prediction results output by the stacked model to obtain the classification results of the 3D model test data.

[0015] 2. The multi-view method based on feature fusion and model stacking according to claim 1 Figure 3 The three-dimensional model classification method is characterized in that, in step 1, the three-dimensional model is preprocessed, and the specific steps are:

[0016] Step 1-1 constructs a rendering background of the 3D model, fixes the 3D model at the center of the virtual sphere, and selects a fixed angle and amount for projection;

[0017] In step 1-2, based on the rendering background constructed in step 1-1, a virtual camera is placed on a circle 30° above the center of the 3D model. The camera rotates one circle with each step of 60° to project a set of 2D views, which are recorded as V1, V2, V3, V4, V5 and V6.

[0018] 3. The multi-view method based on feature fusion and model stacking according to claim 1 Figure 3 The dimensional model classification method is characterized in that in step 2, the shape distribution features D1, D2, and D3 of the view are extracted, and the specific steps are:

[0019] Step 2-1 uses the D1 descriptor to describe the distance between the centroid of the two-dimensional view and the random sampling points on the boundary. The specific steps are:

[0020] Step 2-1-1 Sample the edge of each 3D model projection view and put all the obtained pixels into PV = {(a 1 ,b 1 ),...,(a i ,b i ),...,(a n ,b n )} set, where (a i ,b i ) are the coordinates of the random sampling points;

[0021] Step 2-1-2 Extract pixels from PV, where N is the total number of pixels, and record them in the set PD1 = {m i1 ,...,m ik ,...,m iN}middle;

[0022] Step 2-1-3 The two-dimensional D1 shape distribution feature set is {D1 _1 ,...,D1 _i ,...,D1 _NumSize}, where D1 _i It is expressed as statistical information within the interval (NumSize*(i-1), NumSize*i), where Num represents the number of statistical intervals and NumSize is the length of the statistical interval, as shown below:

[0023] NumSize=max({dist(m,O)|m∈PD1}) / N

[0024] Among them, dist() represents the Euclidean distance between two points, O represents the centroid of the two-dimensional view, max() represents the maximum value, D1 _i The definition is as follows:

[0025] D1 _i =|{m│dist(m,O)∈(NumSize*(i-1),NumSize*i),m∈PD1}|;

[0026] Step 2-2 uses the D2 descriptor to describe the distance between two random sampling points on the boundary of the two-dimensional view. The specific steps are:

[0027] Step 2-2-1 Extract pixel pairs from PV and record them in PD2={(m i1 ,n i1 ),...,(m ik ,n ik ),...,(m iN ,n iN )} set, N is the total number of pixel pairs;

[0028] Step 2-2-2 The two-dimensional D2 shape distribution feature set is represented as {D2 _1 ,...,D2 _i ,...,D2 _NumSize}, where D2 _i It is expressed as statistical information within the interval (NumSize*(i-1), NumSize*i). The dist() function represents the Euclidean distance between two points. NumSize is the length of the statistical interval, as shown below:

[0029] NumSize=max({dist(x,y)|(x,y)∈PD2}) / N;

[0030] Step 2-2-3 uses the D2 descriptor to describe the distance between two random sampling points on the boundary of the two-dimensional view, D 2 _ i The calculation formula is as follows:

[0031] D2 _i =|{(x,y)|dist(x,y)∈(NumSize*(i-1),NumSize*i),(x,y)∈PD2}|;

[0032] Step 2-3 uses the D3 descriptor to describe the square root of the area of ​​the triangle formed by three random sampling points on the boundary of the two-dimensional view. The specific steps are:

[0033] Step 2-3-1 Extract the sampling pixel point triplet from PV and record it in PD3={(m i1,n i1 ,q i1 ),...,(m ik ,n ik ,q ik ),...,(m iN ,n iN ,q iN )} set, N is the total number of pixel triplets;

[0034] Step 2-3-2 The two-dimensional D3 shape distribution feature set is {D3 _1 ,...,D3 _i ,...,D3 _NumSize}, D3 _i It is expressed as statistical information within the interval (NumSize*(i-1), NumSize*i), where NumSize is the length of the statistical interval, as shown below:

[0035] NumSize=max({herson(x,y,z)|(x,y,z)∈PD3}) / N

[0036] Among them, herson(x,y,z) represents the Heron formula used to calculate the area of ​​the triangle formed by the three pixel points x, y and z. The calculation formula is as follows:

[0037]

[0038] s=(l 1 +l 2 +l 3 ) / 2

[0039] Among them, l 1 = dist(x,y),l 2 = dist(y,z),l 3 = dist(x,z);

[0040] Step 2-3-3 uses the D3 descriptor to represent the square root of the area of ​​the triangle formed by three random sampling points on the boundary of the two-dimensional view, D3_ i The calculation formula is as follows:

[0041] D3 _i =|{(x,y,z)|herson(x,y,z)∈NumSize*(i-1),NumSize*i),(x,y,z)∈PD3}|;

[0042] 4. The multi-view method based on feature fusion and model stacking according to claim 1 Figure 3The method for classifying a dimensional model is characterized in that, in step 3, training data and test data are extracted from ModelNet10, and the specific steps are as follows:

[0043] Step 3-1 projects the 3D training model in ModelNet10 to obtain a 2D view, extracts the D1, D2, and D3 descriptors of the 2D view, and forms training data with the 2D view;

[0044] Step 3-2 projects the three-dimensional test model in ModelNet10 to obtain a two-dimensional view, extracts the D1, D2, and D3 descriptors of the two-dimensional view, and forms test data with the two-dimensional view.

[0045] 5. The multi-view method based on feature fusion and model stacking according to claim 1 Figure 3 The method for classifying three-dimensional models is characterized in that, in step 4, a joint descriptor is obtained by fusing shape distribution features and view semantic features, a three-dimensional model training set after feature fusion in the three-dimensional model data set is used as training data, and a three-dimensional model test set after feature fusion in the three-dimensional model data set is used as test data, and the specific steps are as follows:

[0046] Step 4-1: Input the 2D view set of the 3D model into the HorNet design space to extract the view semantic features;

[0047] In step 4-2, Concat is used to fuse the shape distribution features obtained in step 3 with the view semantic features obtained in step 4-1, and the 3D model training set after feature fusion in the 3D model dataset is used as training data, and the 3D model test set after feature fusion in the 3D model dataset is used as test data.

[0048] 6. The multi-view method based on feature fusion and model stacking according to claim 1 Figure 3 The method for classifying a dimensional model is characterized in that in step 5, the training data is used to optimize the CatBoost model, discriminant features are extracted from the fusion features of the training data and input into the CatBoost model for reasoning to obtain the prediction results of the training data, and the specific steps are as follows:

[0049] Step 5-1: Optimize the CatBoost model using training data;

[0050] Step 5-2: Use the HorNet design space to extract discriminant features from the fusion features of the training data and input them into the optimized CatBoost model to obtain the i-th view V i The classification prediction y of the fusion feature i , the calculation process is as follows:

[0051]

[0052] Among them, F(X i ) is the CatBoost model for the input view V i The fusion feature X i The mapping function is , M is the total number of trees in the model, γ m is the output weight of the mth tree after learning rate adjustment, T m (X i ) is the mth decision tree for view V i The fusion feature X i Output:

[0053] Step 5-3 The prediction probability calculation process of the CatBoost model output is as follows:

[0054]

[0055] Where P(y i =k|X i ) is a given feature X i When the view V i The probability of belonging to category k, y ik is the CatBoost model for the view V i In the logits output of category k, K is the total number of categories;

[0056] Step 5-4 makes the final classification decision based on the predicted probability, and the calculation formula is as follows:

[0057]

[0058] Among them, Y i is the view V i The final classification result.

[0059] 7. The multi-view method based on feature fusion and model stacking according to claim 1 Figure 3 The dimensional model classification method is characterized in that in step 6, the training data is used to optimize the random forest model, and the discriminant features are extracted from the fusion features of the training data and input into the random forest model for reasoning to obtain the prediction results of the training data. The specific steps are:

[0060] Step 6-1 optimizes the random forest model using training data;

[0061] Step 6-2 uses the HorNet design space to extract discriminant features from the fusion features of the training data and inputs them into the optimized random forest model. Each tree in the random forest predicts the input view fusion feature X. The calculation process is as follows:

[0062] y t =Tree t(x),t=1,2,...,T

[0063] where y t represents the predicted category of the t-th tree, and T represents the total number of trees;

[0064] Step 6-3 calculates the number of view fusion features X in all decision tree predictions for k categories. The calculation process is as follows:

[0065]

[0066] Where Vote(k) is the voting function, I() is the indicator function, when the prediction result y t = k, I(y t =k) ​​= 1, otherwise, I(y t = k) = 0;

[0067] Step 6-4 uses the voting method to determine the final category of the view fusion feature X. The calculation process is as follows:

[0068]

[0069] 8. The multi-view method based on feature fusion and model stacking according to claim 1 Figure 3 The method for classifying dimensional models is characterized in that in step 7, the prediction results of the base model CatBoost model and the random forest model on the training data are extracted as features to input the meta-model random forest model for training, and the prediction results of the base model of the test data are used for reasoning to obtain the prediction results of the test data. The specific steps are as follows:

[0070] Step 7-1 uses the base model CatBoost model and the random forest model to classify and predict the view fusion feature X of the training data respectively, and combines the prediction results into a new feature vector as shown below:

[0071] y new , k=M k (x)

[0072] Among them, M k represents the base model, y new , k represents the base model M k For the prediction results of the view fusion feature X, the prediction results are combined into a new feature vector as shown below:

[0073]

[0074] Step 7-2 Select random forest as the meta-model and use the new feature vector x new Optimize metamodel;

[0075] Step 7-3 uses the base model CatBoost model and the random forest model to classify and predict the view fusion feature X of the test data respectively, and combines the prediction results into a new feature vector as shown below:

[0076] y′ new , k=M k (X)

[0077] Among them, M k represents the base model, y' new , k represents the base model M k The prediction results of the fusion feature X of the test data view are combined into a new feature vector as shown below:

[0078]

[0079] Step 7-4: The new feature vector x' new Input the trained meta-model random forest model to make the final prediction as follows:

[0080] y=M meta (x′ new )

[0081] Among them, M meta Represents the metamodel.

[0082] 9. The multi-view method based on feature fusion and model stacking according to claim 1 Figure 3 The method for classifying three-dimensional models is characterized in that in step 8, a voting algorithm is used to vote on the view prediction results output by the stacking model to obtain the classification results of the three-dimensional model test data, and the specific steps are:

[0083] Step 8-1 The two-dimensional view set obtained by projecting the three-dimensional model A is V(A)={V1, V2, ..., V6};

[0084] Step 8-2 Classification prediction result y of the projection view Vn of the 3D model A n The number of occurrences is counted, and the category with the most votes is the final classification result of the 3D model. The process is as follows:

[0085]

[0086] Where I() represents the indicator function. When f n When (x) = y, I(f n (x)=y)=1, otherwise, I(f n (x)=y)=0.

[0087] Beneficial effects:

[0088] The present invention provides a multi-view method based on feature fusion and model stacking. Figure 3 Dimensional model classification method.

[0089] 1. The present invention improves the three-dimensional shape distribution feature to make it applicable to two-dimensional views, and uses the shape distribution function to describe the shape information of the three-dimensional model view. For the geometric shape of the three-dimensional model, the present invention integrates the view feature and the shape feature to comprehensively describe the three-dimensional model. The shape feature of the three-dimensional model is obtained by fusing the D1 feature, the D2 feature, and the D3 feature, and has good rotation robustness, and can well express the geometric shape characteristics of the three-dimensional model.

[0090] 2. The present invention integrates view semantic features and shape distribution features. Compared with the method of using only view features to classify three-dimensional models, the method of integrating view semantic features and shape distribution features can present model information more comprehensively from multiple angles, complement each other's advantages, enrich and ensure the description ability of the three-dimensional model, and improve the accuracy of three-dimensional model classification.

[0091] 3. The present invention introduces model stacking, which can effectively reduce the risk of overfitting, improve the generalization ability of the model, and greatly improve the training speed. The present invention uses this model to perform classification prediction and decision-making on the joint descriptor of the three-dimensional model, thereby improving the classification accuracy of the three-dimensional model.

[0092] 4. The present invention verifies the proposed 3D model classification method using the public dataset ModelNet10 model library, and achieves a high classification accuracy. The results show that the classification method proposed by the present invention is effective. Description of the drawings:

[0093] Figure 1 is a bathtub instance of a three-dimensional model to be classified in an embodiment of the present invention;

[0094] Figure 2 A flowchart of three-dimensional model classification in an embodiment of the present invention;

[0095] Figure 3 Schematic diagram of different angle views of a three-dimensional model to be classified obtained by a virtual camera in an embodiment of the present invention;

[0096] Figure 4 These are the three shape features D1, D2, and D3 in the embodiments of the present invention; Specific implementation method:

[0097] In order to make the technical solution in the embodiment of the present invention clearly and specifically described, a 3D model in the bathtub class of the test set of the 3D model public data set ModelNet10 is taken as an example. Figure 1As shown, tests are carried out, and the present invention is further described in detail in conjunction with the drawings in the embodiments.

[0098] The present invention implements multi-view based on feature fusion and model stacking Figure 3 Dimensional model classification method framework, such as Figure 2 As shown, the following steps are included:

[0099] Step 1 projects the 3D model in Modelnet10 to obtain a 2D view set. The specific steps are as follows:

[0100] Step 1-1 constructs a rendering background of the 3D model, fixes the 3D model at the center of the virtual sphere, and selects a fixed angle and amount for projection;

[0101] In step 1-2, based on the rendering background constructed in step 1-1, a virtual camera is placed on a circle 30° above the center of the 3D model. The camera rotates one circle with each step of 60° to project a set of 2D views, which are recorded as V1, V2, V3, V4, V5 and V6.

[0102] Step 2 extracts the D1, D2, and D3 features of the 3D model view. The specific steps are as follows:

[0103] Step 2-1 uses the D1 descriptor to describe the distance between the centroid of the two-dimensional view and the random sampling points on the boundary, such as Figure 4 As shown, the specific steps are:

[0104] Step 2-1-1 Sample the edge of each 3D model projection view and put all the obtained pixels into PV = {(a 1 ,b 1 ),...,(a i ,b i ),...,(a n ,b n )} set, where (a i ,b i ) are the coordinates of the random sampling points;

[0105] Step 2-1-2 Extract pixels from PV, where N is the total number of pixels, and record them in the set PD1 = {m i1 ,...,m ik ,...,m iN}middle;

[0106] Step 2-1-3 The two-dimensional D1 shape distribution feature set is {D1 _1 ,...,D1 _i ,...,D1 _NumSize}, where D1 _iIt is expressed as statistical information within the interval (NumSize*(i-1), NumSize*i), where Num represents the number of statistical intervals and NumSize is the length of the statistical interval, as shown below:

[0107] NumSize=max({dist(m,O)|m∈PD1}) / N,

[0108] Among them, dist() represents the Euclidean distance between two points, O represents the centroid of the two-dimensional view, max() represents the maximum value, D1 _i The definition is as follows:

[0109] D1 _i =|{m│dist(m,O)∈(NumSize*(i-1),NumSize*i),m∈PD1}|,

[0110] from Figure 1 Extract D1 feature from the view of the three-dimensional model shown, D1 = [[0.023809524, 0.05357143, 0.3690476, 0.21726191, 0.20238096, 0.005952381, 0.35119048, 0.89880955, 0.4940476, 0.20833333]…];

[0111] Step 2-2 uses the D2 descriptor to describe the distance between two random sampling points on the boundary of the two-dimensional view, such as Figure 4 As shown, the specific steps are:

[0112] Step 2-2-1 Extract pixel pairs from PV and record them in PD2={(m i1 ,n i1 ),...,(m ik ,n ik ),...,(m iN ,n iN )} set, N is the total number of pixel pairs;

[0113] Step 2-2-2 The two-dimensional D2 shape distribution feature set is represented as {D2 _1 ,...,D2 _i ,...,D2 _NumSize}, where D2 _i It is expressed as statistical information within the interval (NumSize*(i-1), NumSize*i). The dist() function represents the Euclidean distance between two points. NumSize is the length of the statistical interval, as shown below:

[0114] NumSize=max({dist(x,y)|(x,y)∈PD2}) / N;

[0115] Step 2-2-3 uses the D2 descriptor to describe the distance between two random sampling points on the boundary of the two-dimensional view, D 2 _ i The calculation formula is as follows:

[0116] D2 _i =|{(x,y)|dist(x,y)∈(NumSize*(i-1),NumSize*i),(x,y)∈PD2}|,

[0117] from Figure 1 Extract D2 features from the view of the three-dimensional model shown, D2 = [[0.15476191, 0.21428572, 0.3184524, 0.4642857, 0.44047618, 0.5, 0.19940476, 0.3392857, 0.14583333, 0.04761905]…];

[0118] Step 2-3 uses the D3 descriptor to describe the square root of the area of ​​the triangle formed by three random sampling points on the boundary of the two-dimensional view, such as Figure 4 As shown, the specific steps are:

[0119] Step 2-3-1 Extract the sampling pixel point triplet from PV and record it in PD3={(m i1 ,n i1 ,q i1 ),...,(m ik ,n ik ,q ik ),...,(m iN ,n iN ,q iN )} set, N is the total number of pixel triplets;

[0120] Step 2-3-2 The two-dimensional D3 shape distribution feature set is {D3 _1 ,...,D3 _i ,...,D3 _NumSize}, D3 _i It is expressed as statistical information within the interval (NumSize*(i-1), NumSize*i), where NumSize is the length of the statistical interval, as shown below:

[0121] NumSize=max({herson(x,y,z)|(x,y,z)∈PD3}) / N

[0122] Among them, herson(x,y,z) represents the Heron formula used to calculate the area of ​​the triangle formed by the three pixel points x, y and z. The calculation formula is as follows:

[0123]

[0124] s=(l 1 +l 2 +l 3 ) / 2

[0125] Among them, l 1 = dist(x,y),l 2 = dist(y,z),l 3 = dist(x,z);

[0126] Step 2-3-3 uses the D3 descriptor to represent the square root of the area of ​​the triangle formed by three random sampling points on the boundary of the two-dimensional view, D3_ i The calculation formula is as follows:

[0127] D3 _i =|{(x,y,z)|herson(x,y,z)∈NumSize*(i-1),NumSize*i),(x,y,z)∈PD3}|

[0128] from Figure 1 Extract D3 features from the 3D model view shown, D3 = [[1, 0.67261904, 0.49702382, 0.2857143, 0.20238096, 0.065476194, 0.041666668, 0.014880952, 0.04464286, 0]…];

[0129] Step 3 extracts training data and test data from ModelNet10. The specific steps are:

[0130] Step 3-1 projects the 3D training model in ModelNet10 to obtain a 2D view, extracts the D1, D2, and D3 descriptors of the 2D view, and forms training data with the 2D view;

[0131] Step 3-2 projects the three-dimensional test model in ModelNet10 to obtain a two-dimensional view, extracts the D1, D2, and D3 descriptors of the two-dimensional view, and forms test data with the two-dimensional view;

[0132] Step 4 fuses the shape distribution features and the view semantic features to obtain a joint descriptor, uses the 3D model training set after feature fusion in the 3D model dataset as training data, and uses the 3D model test set after feature fusion in the 3D model dataset as test data. The specific steps are as follows:

[0133] Step 4-1 Input the 2D view set of the 3D model into the HorNet design space to extract the view semantic features. Figure 3 The view semantic features extracted from the obtained view set are as follows:

[0134] VF=[[0.2785431,0.1589038,1.5806426,…],[0.4402559,-0.05467467,1.4841751,…],[0.26817697,0.19082978,1.5656699,… ],[0.33223772,0.16727105,1.5979155,…],[0.34325728,0.12829812,1.618401,…],[0.41015503,0.17737564,1.6781431…]]

[0135] Step 4-2 uses Concat to fuse the shape distribution features obtained in step 3 with the view semantic features obtained in step 4-1, and uses the 3D model training set after feature fusion in the 3D model dataset as training data, and uses the 3D model test set after feature fusion in the 3D model dataset as test data. Figure 3 The joint descriptor obtained by fusing the features extracted from the acquired view set is as follows:

[0136] MF=[[0.2785431,0.1589038,1.5806426,…,0.83333,0.58333,0.402298,…,0.114942,0 .186781,0.373563,…,0,0,0.244253,…],[0.44025,-0.054674,1.48417,…,0.145118,0. 245382,0.414248,…,0.114942,0.186781,0.373563,…,0.0237467,1,0.548812,…],[0.268176,0.190829,1.56566,…,0.145118,0.245382,0.414248,…,0.04310,0.83333,0.58 333…],[0.332237,0.167271,1.59791,…,0.1396,0.17948,0.3732,…,0.22691,0.145118,0.26385,…,0.02374,1,0.54881…],[0.34325,0.12829,1.61840,…,0,0.40830,0.3944 6,…,0.21652,0.21937,0.27065,…0.037037,1,0.62393…],[0.41015,0.17737,1.67814,…,0.13521,0.28169,0.3718,…,0.35294,0.1833,0.30449,…,0.05882,1,0.73010,…]].

[0137] Step 5 uses the training data to optimize the CatBoost model, extracts discriminant features from the fusion features of the training data and inputs them into the CatBoost model for inference to obtain the prediction results of the training data. The specific steps are:

[0138] Step 5-1: Optimize the CatBoost model using training data;

[0139] Step 5-2: Use the HorNet design space to extract discriminant features from the fusion features of the training data and input them into the optimized CatBoost model to obtain the i-th view V i The classification prediction y of the fusion feature i , the calculation process is as follows:

[0140]

[0141] Among them, F(X i ) is the CatBoost model for the input view V i The fusion feature Xi The mapping function is , M is the total number of trees in the model, γ m is the output weight of the mth tree after learning rate adjustment, T m (X i ) is the mth decision tree for view V i The fusion feature X i Output:

[0142] Step 5-3 The prediction probability calculation process of the CatBoost model output is as follows:

[0143]

[0144] Where P(y i =k|X i ) is a given feature X i When the view V i The probability of belonging to category k, y ik is the CatBoost model for the view V i In the logits output of category k, K is the total number of categories;

[0145] Figure 1 The predicted probabilities for the set of views of the shown 3D model are as follows:

[0146] P=[[0.93190,0.01491,0.00554,0.00887,0.00894,0.00500,0.00834,0 .00528,0.00537,0.00586],[0.93056,0.01558,0.00557,0.00890,0.009 03,0.00509,0.00849,0.00540,0.00545,0.00594],[0.92960,0.01481,0.00590,0.00920,0.00923,0.00519,0.00868,0.00551,0.00572,0.00615 ],[0.92687,0.01379,0.00623,0.00966,0.00983,0.00561,0.00924,0.00578,0.00627,0.00672],[0.92615,0.01426,0.00646,0.01001,0.01002 ,0.00552,0.00913,0.00588,0.00602,0.00656],[0.92245,0.01500,0.00635,0.01043,0.01047,0.00580,0.00975,0.00618,0.00655,0.00702]]

[0147] Step 5-4 makes the final classification decision based on the predicted probability, and the calculation formula is as follows:

[0148]

[0149] Among them, Y i is the view V i The final classification result.

[0150] Step 6 uses the training data to optimize the random forest model, extracts discriminant features from the fusion features of the training data and inputs them into the random forest model for inference to obtain the prediction results of the training data. The specific steps are:

[0151] Step 6-1 optimizes the random forest model using training data;

[0152] Step 6-2 uses the HorNet design space to extract discriminant features from the fusion features of the training data and inputs them into the optimized random forest model. Each tree in the random forest predicts the input view fusion feature X. The calculation process is as follows:

[0153] y t =Tree t (x),t=1,2,...,T

[0154] where y t represents the predicted category of the t-th tree, and T represents the total number of trees;

[0155] Step 6-3 calculates the number of view fusion features X in all decision tree predictions for k categories. The calculation process is as follows:

[0156]

[0157] Where Vote(k) is the voting function, I() is the indicator function, when the prediction result y t = k, I(y t =k) ​​= 1, otherwise, I(y t = k) = 0,

[0158] The number of decision trees T in the random forest is 100. Figure 1 The prediction results for the shown set of 3D model views are shown below:

[0159] V=[[52,48,0,0,0,0,0,0,0,0],[100,0,0,0,0,0,0,0,0,0],[98,2,0,0,0,0,0,0,0,0] ,[100,0,0,0,0,0,0,0,0,0],[100,0,0,0,0,0,0,0,0,0],[99,1,0,0,0,0,0,0,0,0]];

[0160] Step 6-4 uses the voting method to determine the final category of the view fusion feature X. The calculation process is as follows:

[0161]

[0162] Figure 1 The prediction results for the shown set of 3D model views are shown below:

[0163] y=[0,0,0,0,0,0].

[0164] Step 7: Extract the prediction results of the base model CatBoost model and the random forest model on the training data as features to input the meta-model random forest model for training. Use the base model prediction results of the test data for inference to obtain the prediction results of the test data. The specific steps are as follows:

[0165] Step 7-1 uses the base model CatBoost model and the random forest model to classify and predict the view fusion feature X of the training data respectively, and combines the prediction results into a new feature vector as shown below:

[0166] y new , k=M k (x)

[0167] Among them, M k represents the base model, y new , k represents the base model M k The prediction results of the view fusion feature X are combined into a new feature vector as shown below:

[0168]

[0169] Step 7-2 Select random forest as the meta-model and use the new feature vector x new Optimize metamodel;

[0170] Step 7-3 uses the base model CatBoost model and the random forest model to classify and predict the view fusion feature X of the test data respectively, and combines the prediction results into a new feature vector as shown below:

[0171] y′ new , k=M k (X)

[0172] Among them, M k represents the base model, y' new , k represents the base model M k The prediction results of the fusion feature X are combined into a new feature vector for the test data view, as shown below:

[0173]

[0174] Figure 1 The new feature vector for the view set of the shown 3D model looks like this:

[0175] x' new = [[[0, 1], [0, 2]], [[0, 1], [0, 2]], [[0, 1], [0, 2]], [[0, 1], [0, 2]], [[0, 1], [0, 2]], [[0, 1], [0, 2]], [[0, 1], [0, 2]]];

[0176] Step 7-4: The new feature vector x' new Input the trained meta-model random forest model to make the final prediction as follows:

[0177] y=M meta (x′ new )

[0178] Among them, M meta Represents the metamodel.

[0179] Step 8 uses a voting algorithm to vote on the view prediction results output by the stacking model to obtain the classification results of the 3D model test data. The specific steps are as follows:

[0180] Step 8-1 The two-dimensional view set obtained by projecting the three-dimensional model A is V(A)={V1, V2, ..., V6};

[0181] Step 8-2 Classification prediction result y of the projection view Vn of the 3D model A n The number of occurrences is counted, and the category with the most votes is the final classification result of the 3D model. The process is as follows:

[0182]

[0183] Where I() represents the indicator function. When f n When (x) = y, I(f n (x)=y)=1, otherwise, I(f n (x) = y) = 0,

[0184] Figure 1 The classification result of the 3D model shown is the bathtub class, and the classification accuracy of the test data set is as follows:

[0185] Classification accuracy = 121 / 127≈95.28%.

[0186] The above is a detailed description of the embodiments of the present invention in conjunction with the accompanying drawings, and the specific implementation methods herein are only used to help understand the method of the present invention. For those of ordinary skill in the art, according to the idea of ​​the present invention, changes and modifications can be made in the specific implementation methods and application scopes, so the present invention should not be understood as limiting the present invention.

Claims

1. A multi-view 3D model classification method based on feature fusion and model stacking, characterized in that: The method mainly includes the following steps: Step 1: Project the 3D model in Modelnet10 to obtain a 2D view set. Step 2: Use the D1 descriptor to describe the distance between the centroid of the 2D view and a random sampling point on the boundary, use the D2 descriptor to describe the distance between two random sampling points on the boundary of the 2D view, and use the D3 descriptor to describe the square root of the area of ​​the triangle formed by three random sampling points on the boundary of the 2D view. Step 3: Extract the two-dimensional view of the three-dimensional model in Modelnet10, and extract the D1, D2, and D3 shape descriptors of the two-dimensional view to form training data and test data. Step 4: Extract the semantic features of the view based on the HorNet design space, use Concat to fuse the shape distribution features obtained in step 3 with the view semantic features to obtain a joint descriptor of the 3D model, use the 3D model training set after feature fusion in the 3D model dataset as training data, and use the 3D model test set after feature fusion in the 3D model dataset as test data. Step 5: Use the training data to optimize the CatBoost model, extract the discriminant features from the fused features of the training data, input them into the CatBoost model for inference, and obtain the prediction results of the training data. Step 6: Use the training data to optimize the random forest model, extract the discriminant features from the fused features of the training data and input them into the random forest model for inference to obtain the prediction results of the training data. Step 7: Extract the prediction results of the base model CatBoost model and random forest model on the training data as features and input them into the meta-model random forest model for training. Use the prediction results of the base model on the test data for inference to obtain the prediction results of the test data. Step 8: Use the voting algorithm to vote on the view prediction results output by the stacked model to obtain the classification results of the 3D model test data.

2. The multi-view 3D model classification method based on feature fusion and model stacking according to claim 1, characterized in that: In step 1, the three-dimensional model is preprocessed, and the specific steps are: Step 1-1 constructs a rendering background of the 3D model, fixes the 3D model at the center of the virtual sphere, and selects a fixed angle and amount for projection; In step 1-2, based on the rendering background constructed in step 1-1, a virtual camera is placed on a circle 30° above the center of the 3D model. The camera rotates one circle with each step of 60° to project a set of 2D views, which are recorded as V1, V2, V3, V4, V5 and V6.

3. The multi-view 3D model classification method based on feature fusion and model stacking according to claim 1, characterized in that: In step 2, the shape distribution features D1, D2, and D3 of the view are extracted, and the specific steps are as follows: Step 2-1 uses the D1 descriptor to describe the distance between the centroid of the two-dimensional view and the random sampling points on the boundary. The specific steps are: Step 2-1-1 Sample the edge of each 3D model projection view and put all the obtained pixels into PV = {(a1,b1),...,(a i ,b i ),...,(a n ,b n )} set, where (a i ,b i ) are the coordinates of the random sampling points; Step 2-1-2 Extract pixels from PV, where N is the total number of pixels, and record them in the set PD1 = {m i1 ,...,m ik ,...,m iN }middle; Step 2-1-3 The two-dimensional D1 shape distribution feature set is {D1 _1 ,...,D1 _i ,...,D1 _NumSize }, where D1 _i It is expressed as statistical information within the interval (NumSize*(i-1), NumSize*i), where Num represents the number of statistical intervals and NumSize is the length of the statistical interval, as shown below: NumSize=max({dist(m,O)|m∈PD1}) / N Among them, dist() represents the Euclidean distance between two points, O represents the centroid of the two-dimensional view, max() represents the maximum value, D1 _i The definition is as follows: D1 _i =|{m│dist(m,O)∈(NumSize*(i-1),NumSize*i),m∈PD1}|; Step 2-2 uses the D2 descriptor to describe the distance between two random sampling points on the boundary of the two-dimensional view. The specific steps are: Step 2-2-1 Extract pixel pairs from PV and record them in PD2={(m i1 ,n i1 ),...,(m ik ,n ik ),...,(m iN ,n iN )} set, N is the total number of pixel pairs; Step 2-2-2 The two-dimensional D2 shape distribution feature set is represented as {D2 _1 ,...,D2 _i ,...,D2 _NumSize }, where D2 _i It is expressed as statistical information within the interval (NumSize*(i-1), NumSize*i). The dist() function represents the Euclidean distance between two points. NumSize is the length of the statistical interval, as shown below: NumSize=max({dist(x,y)|(x,y)∈PD2}) / N; Step 2-2-3 uses the D2 descriptor to describe the distance between two random sampling points on the boundary of the two-dimensional view, D2_ i The calculation formula is as follows: D2 _i =|{(x,y)|dist(x,y)∈(NumSize*(i-1),NumSize*i),(x,y)∈PD2}|; Step 2-3 uses the D3 descriptor to describe the square root of the area of ​​the triangle formed by three random sampling points on the boundary of the two-dimensional view. The specific steps are: Step 2-3-1 Extract the sampling pixel point triplet from PV and record it in PD3={(m i1 ,n i1 ,q i1 ),...,(m ik ,n ik ,q ik ),...,(m iN ,n iN ,q iN )} set, N is the total number of pixel triplets; Step 2-3-2 The two-dimensional D3 shape distribution feature set is {D3 _1 ,...,D3 _i ,...,D3 _NumSize }, D3 _i It is expressed as statistical information within the interval (NumSize*(i-1), NumSize*i), where NumSize is the length of the statistical interval, as shown below: NumSize=max({herson(x,y,z)|(x,y,z)∈PD3}) / N Among them, herson(x,y,z) represents the Heron formula used to calculate the area of ​​the triangle formed by the three pixel points x, y and z. The calculation formula is as follows: s＝(l1+l2+l3) / 2 Among them, l1=dist(x,y), l2=dist(y,z), l3=dist(x,z); Step 2-3-3 uses the D3 descriptor to represent the square root of the area of ​​the triangle formed by three random sampling points on the boundary of the two-dimensional view, D3_ i The calculation formula is as follows: D3 _i =|{(x,y,z)|herson(x,y,z)∈NumSize*(i-1),NumSize*i),(x,y,z)∈PD3}|。 4. The multi-view 3D model classification method based on feature fusion and model stacking according to claim 1, characterized in that: In step 3, training data and test data are extracted from ModelNet10. The specific steps are as follows: Step 3-1 projects the 3D training model in ModelNet10 to obtain a 2D view, extracts the D1, D2, and D3 descriptors of the 2D view, and forms training data with the 2D view; Step 3-2 projects the three-dimensional test model in ModelNet10 to obtain a two-dimensional view, extracts the D1, D2, and D3 descriptors of the two-dimensional view, and forms test data with the two-dimensional view.

5. The multi-view 3D model classification method based on feature fusion and model stacking according to claim 1, characterized in that: In step 4, the shape distribution feature and the view semantic feature are fused to obtain a joint descriptor, and the 3D model training set after feature fusion in the 3D model dataset is used as training data, and the 3D model test set after feature fusion in the 3D model dataset is used as test data. The specific steps are: Step 4-1: Input the 2D view set of the 3D model into the HorNet design space to extract the view semantic features; In step 4-2, Concat is used to fuse the shape distribution features obtained in step 3 with the view semantic features obtained in step 4-1, and the 3D model training set after feature fusion in the 3D model dataset is used as training data, and the 3D model test set after feature fusion in the 3D model dataset is used as test data.

6. The multi-view 3D model classification method based on feature fusion and model stacking according to claim 1, characterized in that In step 5, the training data is used to optimize the CatBoost model, and discriminant features are extracted from the fusion features of the training data and input into the CatBoost model for reasoning to obtain the prediction results of the training data. The specific steps are: Step 5-1: Optimize the CatBoost model using training data; Step 5-2: Use the HorNet design space to extract discriminant features from the fusion features of the training data and input them into the optimized CatBoost model to obtain the i-th view V i The classification prediction y of the fusion feature i , the calculation process is as follows: Among them, F(X i ) is the CatBoost model for the input view V i The fusion feature X i The mapping function is , M is the total number of trees in the model, γ m is the output weight of the mth tree after learning rate adjustment, T m (X i ) is the mth decision tree for view V i The fusion feature X i Output: Step 5-3 The prediction probability calculation process of the CatBoost model output is as follows: Where P(y i =k|X i ) is a given feature X i When the view V i The probability of belonging to category k, y ik is the CatBoost model for the view V i In the logits output of category k, K is the total number of categories; Step 5-4 makes the final classification decision based on the predicted probability, and the calculation formula is as follows: Among them, Y i is the view V i The final classification result.

7. The multi-view 3D model classification method based on feature fusion and model stacking according to claim 1, characterized in that In step 6, the training data is used to optimize the random forest model, and the discriminant features are extracted from the fusion features of the training data and input into the random forest model for reasoning to obtain the prediction results of the training data. The specific steps are: Step 6-1 optimizes the random forest model using training data; Step 6-2 uses the HorNet design space to extract discriminant features from the fusion features of the training data and inputs them into the optimized random forest model. Each tree in the random forest predicts the input view fusion feature X. The calculation process is as follows: y t =Tree t (x),t=1,2,...,T where y t represents the predicted category of the t-th tree, and T represents the total number of trees; Step 6-3 calculates the number of view fusion features X in all decision tree predictions for k categories. The calculation process is as follows: Where Vote(k) is the voting function, I() is the indicator function, when the prediction result y t = k, I(y t =k) ​​= 1, otherwise, I(y t = k) = 0; Step 6-4 uses the voting method to determine the final category of the view fusion feature X. The calculation process is as follows:

8. The multi-view 3D model classification method based on feature fusion and model stacking according to claim 1, characterized in that In step 7, the prediction results of the base model CatBoost model and the random forest model on the training data are extracted as features to input the meta-model random forest model for training, and the prediction results of the base model of the test data are used for reasoning to obtain the prediction results of the test data. The specific steps are: Step 7-1 uses the base model CatBoost model and the random forest model to classify and predict the view fusion feature X of the training data respectively, and combines the prediction results into a new feature vector as shown below: y new ,k=M k (x) Among them, M k represents the base model, y new , k represents the base model M k For the prediction results of the view fusion feature X, the prediction results are combined into a new feature vector as shown below: Step 7-2 Select random forest as the meta-model and use the new feature vector x new Optimize metamodel; Step 7-3 uses the base model CatBoost model and the random forest model to classify and predict the view fusion feature X of the test data respectively, and combines the prediction results into a new feature vector as shown below: y′ new ,k=M k (X) Among them, M k represents the base model, y' new , k represents the base model M k The prediction results of the fusion feature X of the test data view are combined into a new feature vector as shown below: Step 7-4: New feature vector x' new Input the trained meta-model random forest model to make the final prediction as follows: y=M meta (x′ new ) Among them, M meta Represents the metamodel.

9. The multi-view 3D model classification method based on feature fusion and model stacking according to claim 1, characterized in that In step 8, a voting algorithm is used to vote on the view prediction results output by the stacking model to obtain the classification results of the 3D model test data. The specific steps are as follows: Step 8-1 The two-dimensional view set obtained by projecting the three-dimensional model A is V(A) = {V1, V2, ..., V6}; Step 8-2 Classification prediction result y of the projection view Vn of the 3D model A n The number of occurrences is counted, and the category with the most votes is the final classification result of the 3D model. The process is as follows: Among them, I() represents the indicator function. When f n When (x) = y, I(f n (x)=y)=1, otherwise, I(f n (x)=y)=0.

Images