A zero-shot classification method for 3D models

Abstract

The invention discloses a zero-shot classification method for three-dimensional models, comprising: S1, constructing a zero-shot three-dimensional model data set; S2, taking the multi-view representation of the three-dimensional model in the zero-shot three-dimensional model data set as input, and using a deep ensemble learning sub-network to extract the visual features of the complex three-dimensional model; S3, inputting the visual features of the three-dimensional model into a semantic manifold embedding sub-network, effectively capturing the visual association and semantic association between known categories and unknown categories; S4, inputting the three-dimensional model of the unknown category in the zero-shot three-dimensional model data set, extracting its visual and semantic feature information through a deep ensemble learning sub-network and a semantic manifold embedding sub-network, using a cosine distance metric function to find the best semantic label for the unknown class, and completing the classification and recognition of the unknown category. The invention can realize the effective recognition of unknown categories and achieve good classification accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of computer graphics, computer vision and intelligent recognition, and in particular to a zero-sample classification method for three-dimensional models. Background Art

[0002] Zero-shot 3D model classification is very important for understanding and analyzing 3D shapes. Currently, with the continuous development of deep learning, traditional 3D model classification and recognition based on closed sets has achieved a high level. At the same time, the rapid development of computer graphics and computer-aided design technology has led to an exponential growth in the number of 3D models. The 3D models collected and used for training cannot contain all categories. In order to enable deep learning networks to effectively identify 3D models in unknown classes, it is necessary to study 3D model classification methods for zero-shot learning. Summary of the invention

[0003] The purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art. Aiming at the problem that the current zero-sample classification method has poor ability in the field of three-dimensional models, a zero-sample classification method for three-dimensional models is proposed, which can effectively identify new categories that have not been seen in the training process and complete the classification and recognition of unknown categories.

[0004] To achieve the above purpose, the technical solution provided by the present invention is: a zero-shot classification method for three-dimensional models, which constructs a zero-shot three-dimensional model data set to identify new classes that have not been seen in the training process, and proposes a deep learning network for zero-shot three-dimensional model classification. The deep learning network uses two convolutional neural networks with inconsistent structures, one of which is combined with an ensemble learning method to effectively extract visual feature information of the three-dimensional model, which is called a deep ensemble learning subnetwork, and the other convolutional neural network is combined with a manifold learning method to capture the correlation between visual features and semantic features of unknown categories that have not been seen and trained known categories, which is called a semantic manifold embedding subnetwork. Through the deep learning network, the classification and recognition of unknown categories are completed; it includes the following steps:

[0005] S1. Study the semantic correlation between existing 3D model data, build a zero-shot 3D model dataset, and provide a data benchmark for the zero-shot 3D model classification task;

[0006] S2, taking the multi-view representation of the 3D model in the zero-shot 3D model dataset as input, and using the deep ensemble learning sub-network to extract the visual features of the complex 3D model;

[0007] S3, input the visual features of the 3D model into the semantic manifold embedding subnetwork, and use the label vector containing rich semantic information in the zero-shot 3D model dataset as a guide to effectively capture the visual and semantic associations between known and unknown categories, and complete the training of the deep learning network;

[0008] S4. Input the unknown category 3D models in the zero-sample 3D model dataset into the trained deep learning network, extract their visual and semantic feature information through the deep integrated learning subnetwork and the semantic manifold embedding subnetwork, use the cosine distance measurement function to find the best semantic label for the unknown class, and complete the classification and recognition of the unknown category.

[0009] Further, in step S1, the zero-sample 3D model classification task aims to achieve classification and recognition of unknown categories, and the constructed zero-sample 3D model dataset includes a data set, a label set, and an attribute set;

[0010] The data set and label set refer to the closely related non-rigid 3D data sets Shrec2014 and Shrec2015 as data sources, and 41 classes of 1677 non-rigid 3D models with clear shapes and textures are selected. According to the principle of zero-sample data set construction, they are divided into non-overlapping training sets and test sets, where the training set contains 33 classes and 1493 3D models; the test set contains 8 classes and 184 3D models;

[0011] The attribute set refers to a set based on a label set, using a 300-dimensional Word2Vec word vector as an attribute of a three-dimensional model, that is, a semantic label vector, to complete the construction of the attribute set.

[0012] Further, in step S2, the visual representation of the three-dimensional model is constructed with multiple views as input, and the deep ensemble learning subnetwork includes a group of base learners and an ensemble learner, the base learner is composed of a convolutional neural network, and its main purpose is to complete the initial decision for each view in the multi-view representation of the three-dimensional model to form an initial view representation; the ensemble learner includes an integration layer and a fully connected layer, and the initial view representation obtained by the base learner is integrated to form an overall representation of the three-dimensional model; to ensure the integrity of the data, the splicing operation is used to complete the view feature integration, and the fully connected layer is used to map it into a 300-dimensional overall feature;

[0013] In order to effectively train the deep ensemble sub-network, the loss function L of the base learner is constructed i And the loss function L of the ensemble learner e , respectively solved by the following formulas:

[0014]

[0015]

[0016] Where N is the number of views, K is the number of categories, and p{l i = j} function means that when the true label of the i-th sample is j, it takes the value 1, otherwise it takes the value 0; the function λ{l i =j} means that when the true label of the i-th sample is j, the value is 1, otherwise it is -1; b ij represents the classification prediction value of the i-th view in the j-th dimension;

[0017] The overall classification loss L of the deep ensemble learning sub-network is solved by the following formula:

[0018]

[0019] Where, L i is the loss of the i-th base learner, L e is the loss of the ensemble learner, α and β represent the proportion of the base learner loss and the ensemble learner loss in the entire loss function.

[0020] Further, in step S3, guided by the semantic label vector of the 3D model data, a semantic manifold embedding subnetwork is constructed. First, the semantic label vector in the zero-sample 3D model dataset is used as input, and the 300-dimensional semantic features of this type of label are constructed using a fully connected layer. Then, based on the semantic features, an adversarial loss L between the visual features and the semantic features is constructed. D , complete the alignment of visual features to semantic features, against the loss L D Solved by the following formula:

[0021]

[0022] Where N′ is the number of samples, which is consistent with the number of 3D model categories, and a i is the i-th semantic feature, p i For the i-th visual feature, l0() and l1() are solved by the following formula:

[0023] l t (e) = tlog(f(R λ (e)))+1-t)log(1-f(R λ (e)))

[0024] Where f() is a set of multi-layer perceptrons, which aims to realize the mapping from feature space to binary classification; e is a visual feature or a semantic feature; t is 0 or 1, which is recorded as 0 when the input is a visual feature and 1 when the input is a semantic feature. In order to make the feature distribution of the visual feature domain and the semantic feature domain embedded in the same space as close as possible, a gradient reversal layer is added to the semantic manifold embedding subnetwork, which is defined as Rλ (), which is the identity transformation R in forward propagation λ (x) = x, and it will be multiplied by -λ during back propagation. λ is a hyperparameter, which changes from 0 to 1 during the training process and plays a role in reverse learning. t (e) When the minimum value is taken, the semantic manifold embedding subnetwork cannot distinguish whether the data comes from the visual feature domain or the semantic feature domain, making the distribution of the two domains closer;

[0025] Through the semantic manifold embedding subnetwork, the independent visual features between classes are embedded into the semantic feature space that is interrelated between classes, indirectly capturing the association between unknown categories and known categories.

[0026] Furthermore, in step S4, the unknown class 3D model in the zero-sample 3D model dataset is input, and its visual features can be obtained based on the deep ensemble learning subnetwork, and then embedded into the semantic feature space using the semantic manifold embedding subnetwork, and the cosine distance is used to find the semantic label closest to it, thereby completing the classification and recognition of the unknown category.

[0027] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0028] 1. The present invention proposes a zero-sample 3D model dataset to provide a data benchmark for the zero-sample 3D model classification task.

[0029] 2. The present invention proposes a deep learning network for zero-shot 3D model classification. It uses a deep ensemble learning subnetwork to extract the visual features of complex 3D models, and constructs a semantic manifold embedding subnetwork guided by a label vector containing rich semantic information, effectively capturing the visual and semantic associations between different classes to complete zero-shot classification.

[0030] 3. The correlation and visualization analysis on the zero-sample 3D model dataset verified the rationality of the dataset; the comparative experiments on the zero-sample 3D model dataset verified the effectiveness of the present invention; the comparative experiments on other datasets further demonstrated the universality of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1This is the word cloud visualization of the zero-sample 3D model dataset proposed in this invention and some 3D model instance diagrams. The word cloud visualization contains 41 categories of words, including ant, bear, bee, bird, bull, butterfly, camel, cat, centaur, chick, cow, crab, crocodile, deer, dinosaur, dog, dolphin, duck, elephant, fish, frog, giraffe, gorilla, horse, kangaroo, man, mantaRay, mermaid, etc. Aid, monkey, mouse, octopus, pig, rabbit, robot, santa, shark, sheep, snake, spider, tortoise, and woman respectively represent ant, bear, bee, bird, bull, butterfly, camel, cat, centaur, chicken, cow, crab, crocodile, deer, dinosaur, dog, penguin, duck, elephant, fish, frog, giraffe, gorilla, horse, kangaroo, man, manta ray, mermaid, monkey, mouse, octopus, pig, rabbit, robot, Santa Claus, shark, sheep, snake, spider, tortoise, and woman. Subsequent words of this type have the same meaning.

[0032] Figure 2 This is a word vector visualization diagram of the zero-sample 3D model dataset proposed in this invention.

[0033] Figure 3 This is the overall framework diagram of the zero-shot classification method proposed in the present invention, where Word2vec is the word vector embedding.

[0034] Figure 4 This is the overall structure diagram of the deep learning network for zero-sample 3D model classification proposed in the present invention, where CNN is a convolutional neural network.

[0035] Figure 5 Predict label probability map for a single class.

[0036] Figure 6 Visualize the semantic relevance of unknown classes.

[0037] Figure 7 Visualize the features of unknown classes. DETAILED DESCRIPTION

[0038] The present invention will be further described below in conjunction with specific embodiments.

[0039] This embodiment provides a zero-sample classification method for three-dimensional models, constructs a zero-sample three-dimensional model dataset, proposes a deep learning network for zero-sample three-dimensional model classification, and realizes multi-view Figure 3Firstly, the semantic correlation between traditional 3D model data is studied, and a zero-shot 3D model dataset is constructed, which contains 1677 non-rigid 3D models of 41 relevant classes, providing a data benchmark for the zero-shot 3D model classification task. Figure 1 The word cloud visualization of the zero-sample 3D model dataset and some 3D model examples are shown. In the word cloud visualization, the size of the word indicates the number of models in this category. Figure 2 A visualization of the word vectors of the dataset is shown.

[0040] For the zero-shot 3D model classification task, a deep learning network for zero-shot 3D model classification is proposed, such as Figure 3 As shown in , the network takes the multi-view representation of data in the zero-shot 3D model dataset as input, uses an end-to-end deep learning network to extract its visual features, and establishes the association between unknown and known classes based on semantic manifold embedding to achieve zero-shot classification. Specifically, Figure 4 As shown, the deep learning network uses two convolutional neural networks with inconsistent structures. One of the convolutional neural networks combines the ensemble learning method to effectively extract the visual feature information of the three-dimensional model, which is called the deep ensemble learning subnetwork, and the other convolutional neural network combines the manifold learning method to capture the correlation between the visual features and semantic features of unknown categories that have not been seen and the trained known categories, which is called the semantic manifold embedding subnetwork. Among them, the deep ensemble learning subnetwork includes a set of base learners and an ensemble learner, which are combined to obtain the visual representation of the three-dimensional model; the semantic manifold embedding subnetwork is guided by the semantic label vector to complete the mapping of the visual representation to the semantic manifold space, capture the semantic correlation between classes, and support the classification of unknown classes. It includes the following steps:

[0041] 1) Views of a 3D model from different perspectives have different representations of its overall information. Therefore, multiple views of a 3D model should have different weights. Views from all angles should not be weighted equally. Figure 1 In order to fully consider the impact of each view on the classification results, a deep ensemble learning subnetwork consisting of a set of base learners and an ensemble learner was constructed by simulating the ensemble learning method. The initial classification decision was made for each view, and multiple initial decisions were combined to form the final decision to represent the composite features of the 3D model. At the same time, the corresponding loss function was designed to better extract the visual features of the complex 3D model.

[0042] The base learner is composed of a convolutional neural network, and its main purpose is to complete the initial decision for each view in the multi-view representation of the three-dimensional model. In terms of the selection of the convolutional neural network, considering that the view data contains both the overall contour information of the three-dimensional model and the local geometric detail information, and unlike the two-dimensional image, the view does not have rich color information, and a deeper deep learning network needs to be constructed. Therefore, in order to effectively capture the above feature information and not cause problems such as gradient explosion and gradient disappearance due to the excessive number of network layers, the base learner adopts a residual neural network, introduces a residual module, and adds cross-layer connections to the network, so that when the number of network layers is deepened, there will be no network degradation problems. The ensemble learner includes an integration layer and a fully connected layer. The main purpose is to integrate the base learner to obtain the initial decision of the view, form the final decision on the three-dimensional model, and characterize the composite features of the three-dimensional model. In general, the integration method mainly includes two operations: pooling and splicing. Compared with pooling, the advantage of the splicing operation is that there is no need for dimensionality reduction, so the complete initial decision information can be retained to the greatest extent. Therefore, the present invention uses the splicing operation to complete the integration of view features and maps them to 300-dimensional overall features through the fully connected layer.

[0043] In order to effectively train the deep ensemble sub-network, the loss function L of the base learner is constructed i And the loss function L of the ensemble learner e , respectively solved by the following formulas:

[0044]

[0045]

[0046] Where N is the number of views, K is the number of categories, and p{l i = j} function means that when the true label of the i-th sample is j, it takes the value 1, otherwise it takes the value 0; the function λ{l i =j} means that when the true label of the i-th sample is j, the value is 1, otherwise it is -1; b ij represents the classification prediction value of the i-th view in the j-th dimension.

[0047] Then the overall classification loss L of the deep ensemble learning sub-network can be solved by the following formula:

[0048]

[0049] Where, L i is the loss of the i-th base learner, L e is the loss of the ensemble learner, α and β represent the proportion of the base learner loss and the ensemble learner loss in the entire loss function.

[0050] 2) Guided by the semantic label vector of the data, a semantic manifold embedding subnetwork is constructed to learn the mapping from visual domain features to attribute domain features and capture the semantic correlation between classes.

[0051] First, the semantic label vector in the zero-shot 3D model dataset is used as input, and the 300-dimensional semantic features of this type of label are constructed using the fully connected layer. Then, based on the semantic features, the adversarial loss L between the visual feature domain and the semantic feature domain is constructed. D , complete the alignment of visual features to semantic features, against the loss L D It can be solved by the following formula:

[0052]

[0053] Where N′ is the number of samples, which is consistent with the number of 3D model categories, and a i is the i-th semantic feature, p i is the i-th visual feature, l0() and l1() can be solved by the following formula:

[0054] l t (e) = tlog(f(R λ (e)))+(1-t)log(1-f(R λ (e)))

[0055] In the formula, f() is a set of multi-layer perceptrons, which aims to realize the mapping from feature space to binary classification; e is a visual feature or a semantic feature; t is 0 or 1, which is recorded as 0 when the input is a visual feature and 1 when the input is a semantic feature. In order to make the feature distribution of the visual feature domain and the semantic feature domain embedded in the same space as close as possible, a gradient reversal layer is added to the semantic manifold embedding subnetwork, which is defined as R λ (). This function is the identity transformation R in forward propagation λ (x) = x, and it will be multiplied by -λ during back propagation. λ is a hyperparameter that changes from 0 to 1 during the training process and plays a role in reverse learning. Then l t (e) When it takes a minimum value, the network cannot distinguish whether the data comes from the visual feature domain or the semantic feature domain, making the distribution of the two domains closer.

[0056] Through the semantic manifold embedding subnetwork, the independent visual features between classes are embedded into the semantic feature space that is interrelated between classes, indirectly capturing the association between unknown categories and known categories.

[0057] 3) The unknown 3D model in the constructed zero-sample 3D model dataset is input, and its visual features can be obtained based on the deep ensemble learning sub-network. Then, the semantic manifold embedding sub-network is used to embed it into the semantic feature space, and the cosine distance is used to find the semantic label closest to it to complete the recognition of the unknown category.

[0058] Experimental configuration: The hardware environment of this experiment is Intel Core i7 2600k+Tesla V100 32GB+16GBRAM, and the software environment is Windows 10x64+CUDA 10.0+CuDNN 7.1+Pytorch 1.1.0+Python3.6+Matlab.

[0059] Dataset: In the experiment, the zero-sample 3D model dataset is used as a benchmark to verify the effectiveness of the algorithm. Table 1 gives the basic information of the dataset. At the same time, the Ali dataset is used to verify the universality of the algorithm. In the Ali dataset, 30 classes in ModelNet40 are used as training sets, and 10 classes in ModelNet10, 14 classes in McGill, and 30 classes in Shrec2015 are used as test sets. Table 2 gives the basic information of the dataset.

[0060] Table 3 shows the experimental results on the zero-shot 3D model dataset, where ZS3D-Net represents the proposed zero-shot classification method. Table 4 shows the number of single-category models, correlation coefficients, and classification accuracy of unknown categories used for testing on the constructed zero-shot 3D model dataset. Table 5 shows the comparative experiments on the Ali dataset.

[0061] Table 1 Basic information of zero-sample 3D model dataset

[0062]

[0063]

[0064] Table 2 Basic information of Ali dataset

[0065]

[0066] Table 3 Experimental results on zero-shot 3D model dataset

[0067] method Accuracy ALE 36.7 DeViSE 38.1 ESZSL 29.3 SAE 18.8 SJE 32.2 ZS3D-Net(MVCNN) 47.8 ZS3D-Net 58.6

[0068] Table 4 Number of single-category models, correlation coefficients and classification accuracy

[0069] category Number of models Correlation coefficient Accuracy shark 58 0.358 84.6 Robot 24 0.223 81.5 woman 24 0.216 75.4 centaur 24 0.193 17.3 cow 22 0.341 7.0 giraffe 25 0.392 0.4 monkey 4 0.412 1.1 sheep 3 0.315 0.1

[0070] Table 5 Comparative experimental results on the Ali dataset

[0071] method ModelNet10 McGill Shrec2015 ZSLPC 28.0 10.7 5.2 MHPC 33.9 12.5 <![CDATA[ 6.2 ]]> f-CLSWGAN 20.7 10.2 5.2 CADA-VAE 23.0 10.7 6.2 ZS3D-Net(MVCNN) 22.7 <![CDATA[ 13.3 ]]> 5.9 ZS3D-Net <![CDATA[ 30.0 ]]> 15.1 6.7

[0072] As shown in Table 3, the proposed ZS3D-Net achieved a classification accuracy of 58.6%, which is significantly better than the existing open source zero-shot classification model, fully demonstrating the effectiveness of this method. Figure 5 , Figure 6 and Figure 7 , we can see that: (1) The number of models has a great impact on the classification accuracy of unknown categories. Overall, shark, which has the largest number of models, has the highest classification accuracy, reaching 84.6%; while monkey and sheep, which have the smallest number of models, have classification accuracy rates less than 1.5%. This is because the deep learning network causes less loss when misclassifying small class samples, so its learning results tend to ensure the classification accuracy of large class samples. Figure 5 This is also illustrated by the fact that the cow and giraffe in the middle class are most often misclassified as shark. (2) The semantic relevance between unknown and known categories in the zero-sample 3D model dataset constructed by the present invention is relatively balanced, so it has little impact on classification accuracy. (3) Comparing the unknown categories, such as Figure 6 The semantic relevance shown and Figure 7 It can be found from the feature correlation shown that the latter has a greater impact on the classification results of unknown categories. For example, centaur and woman are relatively close in visual distribution, but far apart in semantic association. Centaur is most often misclassified in woman during classification. At the same time, as shown in Table 5, the zero-shot classification method proposed in the present invention achieves the highest classification accuracy on the dataset McGill and Shrec2015; and the second best classification accuracy on ModelNet10. This illustrates the universality of the present invention to a certain extent.

[0073] Existing zero-shot classification methods have poor effects in the field of three-dimensional model classification. The present invention designs and constructs a zero-shot three-dimensional model dataset for the zero-shot three-dimensional model classification task, which has good interpretability and generalization, and provides a benchmark for the development of zero-shot three-dimensional classification work. At the same time, a deep learning network for zero-shot three-dimensional model classification is proposed. Through the deep integrated learning subnetwork and the semantic manifold embedding subnetwork, the correlation between the visual features and semantic features of unknown and known classes is fully captured, and the effective recognition of unknown categories is achieved. Good classification accuracy is achieved, which is worthy of promotion.

[0074] The above-described embodiments are only preferred embodiments of the present invention and are not intended to limit the scope of implementation of the present invention. Therefore, all changes made according to the shape and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A zero-shot classification method for three-dimensional models, characterized in that: In order to identify new categories that have not been seen during the training process, this method constructs a zero-shot 3D model dataset and proposes a deep learning network for zero-shot 3D model classification. The deep learning network uses two convolutional neural networks with inconsistent structures. One of the convolutional neural networks combines the ensemble learning method to effectively extract the visual feature information of the 3D model. It is called the deep ensemble learning subnetwork. The other convolutional neural network combines the manifold learning method to capture the correlation between the visual features and semantic features of unknown categories that have not been seen and the trained known categories. It is called the semantic manifold embedding subnetwork. Through this deep learning network, the classification and recognition of unknown categories are completed. It includes the following steps: S1. Study the semantic correlation between existing 3D model data, build a zero-shot 3D model dataset, and provide a data benchmark for the zero-shot 3D model classification task; S2, taking the multi-view representation of the 3D model in the zero-shot 3D model dataset as input, and using the deep ensemble learning sub-network to extract the visual features of the complex 3D model; S3, input the visual features of the 3D model into the semantic manifold embedding subnetwork, and use the label vector containing rich semantic information in the zero-shot 3D model dataset as a guide to effectively capture the visual and semantic associations between known and unknown categories, and complete the training of the deep learning network; Guided by the semantic label vector of the 3D model data, a semantic manifold embedding subnetwork is constructed. First, the semantic label vector in the zero-sample 3D model dataset is used as input, and the 300-dimensional semantic features of this type of label are constructed using the fully connected layer. Then, the adversarial loss L between the visual features and the semantic features is constructed based on the semantic features. D , complete the alignment of visual features to semantic features, against the loss L D Solved by the following formula: Where N′ is the number of samples, which is consistent with the number of 3D model categories, and a i is the i-th semantic feature, p i is the i-th visual feature, l0( ) and l1( ) are solved by the following formula: l t (e)(tlog(f(R λ (e)))+(1-t)log(1-f(R λ (e))) Where f( ) is a set of multi-layer perceptrons, which aims to realize the mapping from feature space to binary classification; e is a visual feature or a semantic feature; t is 0 or 1, which is recorded as 0 when the input is a visual feature and 1 when the input is a semantic feature. In order to make the feature distribution of the visual feature domain and the semantic feature domain embedded in the same space as close as possible, a gradient reversal layer is added to the semantic manifold embedding subnetwork, which is defined as R λ ( ), which is the identity transformation R in forward propagation λ (x) = x, and it will be multiplied by -λ during back propagation. λ is a hyperparameter that changes from 0 to 1 during training and plays a role in reverse learning. t (e) When the minimum value is taken, the semantic manifold embedding subnetwork cannot distinguish whether the data comes from the visual feature domain or the semantic feature domain, making the distribution of the two domains closer; Through the semantic manifold embedding subnetwork, the independent visual features between classes are embedded into the semantic feature space that is interrelated between classes, indirectly capturing the association between unknown categories and known categories; S4. Input the unknown category 3D models in the zero-sample 3D model dataset into the trained deep learning network, extract their visual and semantic feature information through the deep integrated learning subnetwork and the semantic manifold embedding subnetwork, use the cosine distance measurement function to find the best semantic label for the unknown class, and complete the classification and recognition of the unknown category.

2. The zero-sample classification method for three-dimensional models according to claim 1, characterized in that: In step S1, the zero-sample 3D model classification task aims to achieve classification and recognition of unknown categories. The constructed zero-sample 3D model dataset includes a data set, a label set, and an attribute set. The data set and label set refer to the closely related non-rigid 3D data sets Shrec2014 and Shrec2015 as data sources, and 41 classes of 1677 non-rigid 3D models with clear shapes and textures are selected. According to the principle of zero-sample data set construction, they are divided into non-overlapping training sets and test sets, where the training set contains 33 classes and 1493 3D models; the test set contains 8 classes and 184 3D models; The attribute set refers to the construction of the attribute set based on the label set, using the 300-dimensional Word2Vec word vector as the attribute of the three-dimensional model, that is, the semantic label vector.

3. The zero-sample classification method for three-dimensional models according to claim 1, characterized in that: In step S2, the visual representation of the three-dimensional model is constructed with multiple views as input. The deep ensemble learning subnetwork includes a group of base learners and an ensemble learner. The base learner is composed of a convolutional neural network, and its main purpose is to complete the initial decision for each view in the multi-view representation of the three-dimensional model to form an initial view representation; the ensemble learner includes an integration layer and a fully connected layer, and forms an overall representation of the three-dimensional model by integrating the initial view representation obtained by the base learner; to ensure the integrity of the data, the view feature integration is completed by using a splicing operation, and the fully connected layer is used to map it into a 300-dimensional overall feature; In order to effectively train the deep ensemble sub-network, the loss function L of the base learner is constructed i And the loss function L of the ensemble learner e , respectively solved by the following formulas: Where N is the number of views, K is the number of categories, and p{l i = j} function means that when the true label of the i-th sample is j, it takes the value 1, otherwise it takes the value 0; the function λ{l i =j} means that when the true label of the i-th sample is j, the value is 1, otherwise it is -1; b ij represents the classification prediction value of the i-th view in the j-th dimension; The overall classification loss L of the deep ensemble learning sub-network is solved by the following formula: Where, L i is the loss of the i-th base learner, L e is the loss of the ensemble learner, α and β represent the proportion of the base learner loss and the ensemble learner loss in the entire loss function.

4. The zero-sample classification method for three-dimensional models according to claim 1, characterized in that: In step S4, the unknown 3D model in the zero-sample 3D model dataset is input, and its visual features can be obtained based on the deep ensemble learning sub-network, and then The semantic manifold embedding subnetwork is used to embed it into the semantic feature space, and the cosine distance is used to find the semantic label closest to it to complete the classification and recognition of unknown categories.

Images