3D object identification and retrieval method based on graph attention network
Through the method based on graph attention network, combining convolutional neural network and multi-layer perceptron module, a connection graph is established for feature fusion, which solves the problem of failing to effectively utilize view features and de-redundant information in the existing technology, and improves the problem of The accuracy of 3D object recognition and the stability of the recognition model.
Patent Information
- Application Number
- CN202411211790.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2024-12-06
AI Technical Summary
The existing 3D model recognition method fails to effectively combine the local and global features of the view, does not consider the geometric position relationship between different views, does not perform multi-view information de-redundancy, and does not handle data imbalance, resulting in low recognition accuracy. .
Using a graph attention network method, through convolutional neural network, graph attention network, global pooling module and multi-layer perceptron module, a recognition model is built, the similarity of viewing angle images is calculated, and the connection diagram is established for feature fusion to reduce the Redundant information, making full use of the connections between multi-view diagrams.
The accuracy of 3D object recognition is improved, and the useful information is paid attention to through the dynamic attention mechanism, local and global feature expression is enhanced, view redundant information input is reduced, and data imbalance problem is improved.
Smart Images

Figure CN119091432A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of 3D object recognition technology, and in particular to a 3D object recognition and retrieval method based on a graph attention network. Background Art
[0002] With the development and maturity of three-dimensional acquisition technologies such as laser scanning technology, parallax method, and multi-view reconstruction, 3D model recognition and retrieval have become one of the important research directions in the field of computer vision. Today, the mainstream methods are divided into three types: voxel-based methods, point cloud-based methods, and view-based methods. Since the computational complexity of the view-based method is much smaller than that of the voxel-based and point cloud-based methods, the present invention uses a view-based method to recognize and retrieve 3D models. Unlike the voxel-based or point cloud-based methods, the view-based method does not directly process the three-dimensional model, but first projects the three-dimensional object into a 2D image under multiple perspectives through rendering software. Then, the 3D model recognition problem is converted into processing on a two-dimensional image, which avoids complex calculations directly on the three-dimensional data. With the development of transfer learning, the use of convolutional neural networks that have been trained on large-scale data sets such as ImageNet can accelerate the convergence speed of the model and enable the model to reach the optimal solution faster. However, the existing 3D model recognition methods still have the following major problems: they do not combine the local features and global features of the view for judgment, do not consider the geometric position relationship between different views, do not perform redundancy operations on multi-view information, do not consider that different perspectives can provide different amounts of useful information for 3D model recognition, and fail to consider the problem of data imbalance.
[0003] In the prior art, among the 3D model recognition and retrieval methods, the method based on graph convolutional neural network has achieved excellent performance, which includes the following steps: inputting multi-view images into a shared convolutional neural network to obtain two-dimensional view features of multiple views; using the KNN algorithm to select the closest viewpoints for connection to construct a view-graph structure with multiple views as graph nodes; fusing the connected viewpoint features through the graph convolution module; introducing the non-local message passing module to fuse all viewpoint features; gradually coarsening the view-graph structure based on the farthest point sampling; obtaining the output vector of the model through the multi-layer perceptron network, and sorting and retrieving based on the similarity of the output vector. However, although this method integrates local features and global features and takes into account the geometric position relationship between different views, it introduces redundant information of multiple views, does not take into account that the amount of useful information that different viewpoints can provide for 3D model recognition is different, and does not take into account the problem of data imbalance. Summary of the invention
[0004] The primary purpose of the present invention is to overcome the problems existing in the prior art and provide a 3D object recognition and retrieval method based on a graph attention network. The present invention can reduce the redundant information of multiple views and fully consider the connection between multi-view images to improve the accuracy of 3D object recognition.
[0005] In order to achieve the above objectives, the present invention provides a 3D object recognition and retrieval method based on a graph attention network, comprising: S1: Acquire multiple images of multiple 3D objects at different viewing angles and labels corresponding to each viewing angle image, preprocess all viewing angle images based on the labels to obtain a data set, and divide the data set into a training set and a validation set in proportion; S2: constructing a recognition model, and using the training set to reversely iterate the recognition model to obtain an optimal recognition model, wherein the recognition model includes a convolutional neural network, a graph attention network, a global pooling module, and a multi-layer perceptron module; S3: Input multiple different-view images of the 3D object to be identified into the optimal recognition model to obtain a final vector, and determine the recognition and retrieval results based on the final vector.
[0006] Furthermore, the preprocessing described in S1 specifically includes: S1.1: classify all view images according to the labels, and resample the view images of each label; S1.2: After resampling, the view images in each label are normalized according to the channel dimension; S1.3: Perform a random affine transformation on the normalized view image to obtain a data set, and divide the data set into a training set and a validation set in proportion, wherein the ratio is 8:2, and the affine transformation includes at least one of random scaling, random translation, and random rotation.
[0007] Further, the reverse iterative training of the recognition model using the training set in S2 to obtain the optimal recognition model includes: S2.1: Input the training set into the convolutional neural network to obtain the primary features of each perspective image vector; S2.2: Calculate the similarity between the primary feature vector of each view image and the primary feature vectors of other view images, and sort the similarities to obtain a sorting result; S2.3: Based on the sorting result, a connection graph with primary feature vectors as nodes is established, wherein the connection graph includes at least one of a nearest connection graph, a farthest connection graph and a full connection graph; S2.4: Input the connection graph into the graph attention network to obtain local feature descriptors and global feature descriptors; S2.5: performing global pooling processing on the local feature descriptor and the global feature descriptor, and inputting the processing result into a multi-layer perceptron to obtain an output vector; S2.6: Convert the label corresponding to each perspective image into a one-hot encoding, calculate the loss function of the output vector and the one-hot encoding, and update the parameters of the recognition model using a stochastic gradient descent method; S2.7: Repeat steps S2.1 to S2.6 until the number of training times reaches the specified number, and the recognition model with the highest accuracy on the validation set is taken as the optimal recognition model.
[0008] Furthermore, the convolutional neural network described in S2.1 is a ResNet34 neural network.
[0009] Further, the method of calculating the similarity in S2.2 is as follows:
[0010] in, For the The primary feature vector corresponding to the image of each viewing angle is For the The primary feature vector corresponding to the image of each viewing angle is is the primary feature vector The L2 norm of is the primary feature vector The L2 norm of .
[0011] Furthermore, S2.3 is specifically: Based on the sorting result, the primary feature vector corresponding to each perspective image is connected with the primary feature vectors corresponding to the k perspective images with the highest similarity to obtain a nearest connection graph; the primary feature vector corresponding to each perspective image is connected with the primary feature vectors corresponding to the k perspective images with the lowest similarity to obtain a farthest connection graph; the primary feature vector corresponding to each perspective image is connected with the primary feature vectors corresponding to all perspective images to obtain a fully connected graph, wherein the value of k is determined by the number of perspectives of the perspective image obtained in S1.
[0012] Furthermore, the value of k is determined by the number of viewpoints from which the viewpoint image is obtained in S1. Specifically, if the viewpoint image is obtained from 12 different viewpoints in S1, k is 2; if the viewpoint image is obtained from 20 different viewpoints in S1, k is 3.
[0013] Furthermore, S2.4 includes: Inputting the nearest connection graph and the farthest connection graph into a graph attention network to obtain a first fused feature, performing average pooling and concatenation operations on the first fused feature to obtain a local feature descriptor; The fully connected graph is input into a graph attention network to obtain a second fused feature, and average pooling and concatenation operations are performed on the second fused feature to obtain a global feature descriptor.
[0014] Further, the loss function described in S2.6 is as follows:
[0015]
[0016] in, The output classification score after inputting the multilayer perceptron into the nearest connection graph, The output classification score after the farthest connection graph is input into the multilayer perceptron, Output classification score after inputting the multilayer perceptron into the fully connected graph, For labels, is the hyperparameter weight for different connection graphs, and in the 12-view loop configuration is , in the 20-view configuration , is a hyperparameter, the default value is 2. To identify the probability value predicted by the model, is the category weight coefficient, The value of is inversely proportional to the number of categories.
[0017] Further, determining the recognition and retrieval results based on the final vector specifically includes: For the recognition task, the index corresponding to the largest element in the final vector is used as the recognition category; For the retrieval task, the cosine similarity between the output vectors corresponding to each 3D object to be identified is calculated, and the 3D objects below the threshold are filtered based on a preset threshold, and the 3D models above the threshold are taken as retrieval results.
[0018] Compared with the prior art, the present invention has the following beneficial effects: The present invention establishes a recognition model including a convolutional neural network, a graph attention network, a global pooling module and a multi-layer perceptron module. The multi-perspective images of the 3D object are first input into the convolutional neural network for preliminary feature extraction to obtain the primary feature vector corresponding to each perspective image, and then the similarity between the primary feature vectors of each perspective image is calculated, and a connection graph is constructed based on the similarity. Among them, the nearest connection graph focuses on the learning of information on the geometric position of the 3D object, the farthest connection graph strengthens the expression of differentiated information and reduces the input of redundant information of the view, and the fully connected graph is used to update the global information of the 3D object. The connection graph is further subjected to feature fusion through a graph neural network with a dynamic attention mechanism, which fully considers the connection between multiple views and improves the accuracy of 3D object recognition. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 is a flowchart of a 3D object recognition and retrieval method based on a graph attention network according to embodiment 1 of the present invention; Figure 2 1 is a network architecture diagram of ResNet34 according to Embodiment 1 of the present invention; Figure 3 is a structural diagram of residual block 1 in ResNet34 of embodiment 1 of the present invention; Figure 4 is a structural diagram of the residual block 2 in ResNet34 of Embodiment 1 of the present invention; Figure 5 is a structural diagram of a graph attention module according to embodiment 1 of the present invention; Figure 6 This is a framework diagram of a 3D object recognition and retrieval method based on a graph attention network in Example 1 of the present invention. DETAILED DESCRIPTION
[0020] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.
[0021] In the description of the present invention, it should be noted that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.
[0022] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0023] Furthermore, in the description of the present invention, unless otherwise specified, “plurality” means two or more.
[0024] Example 1 like Figure 1 As shown, a 3D object recognition and retrieval method based on a graph attention network in a preferred embodiment of the present invention includes: S1: Acquire multiple images of multiple 3D objects at different viewing angles and labels corresponding to each viewing angle image, preprocess all viewing angle images based on the labels to obtain a data set, and divide the data set into a training set and a validation set in proportion; In one embodiment, S1 specifically includes: S1.1: Classify all view images according to the labels, and resample the view images of each label to ensure that the number of view images of each category is the same; S1.2: After resampling, the view images in each label are normalized according to the channel dimension; S1.3: Perform a random affine transformation on the normalized view image to obtain a data set, and divide the data set into a training set and a validation set in proportion, wherein the ratio is 8:2, and the affine transformation includes at least one of random scaling, random translation, and random rotation.
[0025] S2: constructing a recognition model, and using the training set to reversely iterate the recognition model to obtain an optimal recognition model, wherein the recognition model includes a convolutional neural network, a graph attention network, a global pooling module, and a multi-layer perceptron module; In one embodiment, S2 includes: S2.1: Input the training set into the convolutional neural network to obtain the primary features of each perspective image Vector, where the convolutional neural network is the ResNet34 neural network. The network architecture diagram of the ResNet34 neural network is shown in Figure 2 , ResNet34 introduces two different residual blocks. In this embodiment, they are residual block 1 and residual block 2. The residual block structure is as follows: Figure 3 and 4 As shown in Figure 1, a basic residual block consists of a convolutional layer, batch normalization, activation function, and skip connection. The residual block directly transfers shallow information from one layer to a deep layer through skip connections. The mathematical expression of the residual block is:
[0026] in, represents the output of the residual block, is a nonlinear activation function, is the residual function, is the input of the shallow network, Represents the weight learning parameters of the convolutional layer in the residual block; the ResNet34 network is first pre-trained on the ImageNet dataset to obtain rich feature representation of 2D images, and then migrated to the 3D dataset for fine-tuning.
[0027] S2.2: Calculate the similarity between the primary feature vector of each view image and the primary feature vectors of other view images, and sort the similarities to obtain a sorting result; In one embodiment, the similarity is calculated in S2.2 as follows:
[0028] in, For the The primary feature vector corresponding to the image of each viewing angle is For the The primary feature vector corresponding to the image of each viewing angle is is the primary feature vector The L2 norm of is the primary feature vector The L2 norm of .
[0029] S2.3: Based on the sorting result, a connection graph with the primary feature vectors as nodes is established, and the connection graph includes at least one of a nearest connection graph, a farthest connection graph, and a full connection graph; In one embodiment, S2.3 is specifically: Based on the sorting results, the primary feature vector corresponding to each view image is connected with the primary feature vectors corresponding to the k view images with the highest similarity to obtain the nearest connection graph; the primary feature vector corresponding to each view image is connected with the primary feature vectors corresponding to the k view images with the lowest similarity to obtain the farthest connection graph; the primary feature vector corresponding to each view image is connected with the primary feature vectors corresponding to all view images to obtain the fully connected graph. The nearest connection graph and the farthest connection graph are used to capture the local structure of the 3D object. The nearest connection graph focuses on the learning of 3D geometric position information. The farthest connection graph strengthens the expression of differentiated information and reduces the input of redundant information of the view. The fully connected graph is used to update the global information of the 3D object. The view-graph structure is further feature-fused through a hierarchical graph attention network. This graph network can adaptively focus on views that provide more useful information and fully consider the connection between multiple views. The value of k is determined by the number of views from which the view images are obtained in S1.
[0030] Specifically, the value of k is determined by the number of perspectives from which the perspective image is obtained in S1, specifically: if the perspective image is obtained from 12 different perspectives in S1, k is 2; if the perspective image is obtained from 20 different perspectives in S1, k is 3.
[0031] S2.4: Input the connection graph into the graph attention network to obtain local feature descriptors and global feature descriptors; In one embodiment, the nearest connection graph and the farthest connection graph are input into a graph attention network to obtain a first fused feature, and average pooling and concatenation operations are performed on the first fused feature to obtain a local feature descriptor; The fully connected graph is input into a graph attention network to obtain a second fused feature, and average pooling and concatenation operations are performed on the second fused feature to obtain a global feature descriptor.
[0032] Specifically, the graph attention network includes three layers of graph attention modules, such as Figure 5 As shown in the figure, the graph attention module includes: linear transformation layer, activation function, batch normalization layer, multi-head attention module, and its calculation process is as follows:
[0033]
[0034]
[0035]
[0036]
[0037] The input of the graph attention network is a connection graph , each viewpoint on the connection graph will be connected to other The viewpoint is called the central node on the graph. , and the central node Connected The viewpoints are called neighbor nodes on the graph , the characteristics of the central node on the graph and neighbor nodes Features After linear transformation, batch normalization and nonlinear activation, the central node is obtained From neighbor nodes The new feature representation learned , then Input the multi-head attention module to get the central node Output features after graph attention network , where and Respectively represent the central node and neighbor nodes The learnable parameter matrix for linear transformation, symbol represents the concatenation of two vectors. is a learnable vector, is a nonlinear activation function, function Will and Multiply element by element to get a central node With neighbor nodes The attention score between , through the function Attention score Perform normalization to get the normalized attention score , the normalized attention score With neighbor nodes New feature representation Multiply and pair The sum of all neighbors of Nonlinear activation function to get the output of single-head attention network , It will adaptively learn useful feature representations from other neighbor nodes according to the size of the normalized attention score, symbol Concatenate the single-head attention output features to get the central node The output of the multi-head attention network , the graph attention network has three layers, and the output of each layer of graph attention module All the visual features are averaged and concatenated to obtain local or global feature descriptors. , and its calculation formula is: .
[0038] S2.5: performing global pooling processing on the local feature descriptor and the global feature descriptor, and inputting the processing result into a multi-layer perceptron to obtain an output vector; In one embodiment, the local feature descriptor and the global feature descriptor are obtained by a three-layer multi-layer perceptron. The output features of the recognition model are obtained. The output dimension of the three-layer multilayer perceptron is [512, 512, K]. is the number of categories in the dataset.
[0039] S2.6: Convert the label corresponding to each perspective image into a one-hot encoding, calculate the loss function of the output vector and the one-hot encoding, and update the parameters of the recognition model using a stochastic gradient descent method; In one embodiment, the loss function in S2.6 is as follows:
[0040]
[0041] in, The output classification score after inputting the multilayer perceptron into the nearest connection graph, The output classification score after the farthest connection graph is input into the multilayer perceptron, Output classification score after inputting the multilayer perceptron into the fully connected graph, For labels, is the hyperparameter weight for different connection graphs, and in the 12-view loop configuration is , in the 20-view configuration , is a hyperparameter, the default value is 2. To identify the probability value predicted by the model, is the category weight coefficient, The value of is inversely proportional to the number of categories.
[0042] S2.7: Repeat steps S2.1 to S2.6 until the number of training times reaches the specified number, and the recognition model with the highest accuracy on the validation set is taken as the optimal recognition model.
[0043] In one embodiment, the prescribed number of times is 40 times. As the model is continuously trained, the loss gradually decreases, and the model gradually converges to a local optimal model.
[0044] S3: Input multiple different-view images of the 3D object to be identified into the optimal recognition model to obtain a final vector, and determine the recognition and retrieval results based on the final vector.
[0045] In one embodiment, it specifically includes: For the recognition task, the index corresponding to the largest element in the final vector is used as the recognition category; For the retrieval task, the cosine similarity between the output vectors corresponding to each 3D object to be identified is calculated, and the 3D objects below the threshold are filtered based on a preset threshold, and the 3D models above the threshold are taken as retrieval results.
[0046] Example 2 This embodiment is based on a 3D object recognition and retrieval method based on a graph attention network in Embodiment 1, and the method proposed in the present invention is evaluated using ModelNet40, ModelNet10 and ShapeNetCore55 datasets. ModelNet40 consists of 12,311 3D objects in 40 categories, which are divided into 9,843 objects as training sets and 2,468 objects as test sets. ModelNet10 is a subset of ModelNet40, containing 4,899 CAD models in 10 categories, and the ModelNet dataset has two multi-view rendering versions, namely 12 views of the cycle and 20 views of the dodecahedron. ShapeNetCore55 is a challenging benchmark dataset in the field of 3D shape retrieval, containing 51,162 3D models in 55 categories, which are further subdivided into 203 subcategories, of which 35,764 (70%) objects are used as training sets, 5,133 (10%) objects are used as validation sets, and 10,265 (20%) objects are used as test sets.
[0047] The method (Ours) proposed in this embodiment is compared with the voxel-based method, the point cloud-based method, and the view-based method on the ModelNet40 and ModelNet10 datasets, as shown in Table 1: Table 1
[0048] Table 1 compares the experimental results of different representative methods on ModelNet10 / 40, including VRNEnsemble, VoxNet, LP-3DCNN, PointNet, PointASNL, DGCNN, PCNN, 3D2SeqViews, CAR-Net, MHBN, GVCNN, HMVCN, MVT-small, Relation Network, RotationNet, -MLP-36, View-GCN, View-GCN++, VFMVAC, where the symbol - indicates that this indicator is not provided in the original paper. These methods are divided into three input formats: voxel, point cloud, and view. Basically, the view-based methods achieve better performance than the point cloud or voxel-based methods. For 20 views, the present invention achieves the highest overall accuracy (Overall) and class accuracy (Class) on ModelNet40, reaching the state-of-the-art level. 12 views provide less information than 20 views, so the classification performance on 12 views is generally lower than that on 20 views. In the test on the 12-view loop configuration, the model also achieved very high accuracy on ModelNet10 / 40.
[0049] The method (Ours) proposed in this embodiment is compared with the voxel-based method, the point cloud-based method and the view-based method. The comparison results on the ShapenetCore55 dataset are shown in Table 2: Table 2
[0050] The five indicators of P@N, R@N, F1@N, mAP and NDCG are used to measure the retrieval performance of the ShapenetCore55 dataset, where N represents the actual number of retrieved 3D objects for each query. The above five indicators can be divided into two versions: macro average and micro average. The micro average gives each sample the same weight, which will result in a larger number of categories contributing more to the overall performance. The macro average gives each category the same weight, regardless of the difference in the number of samples, which will result in the performance of each category being equally important to the overall performance, and is more suitable for situations where there is a significant imbalance in the number of different categories. The present invention is compared with advanced models such as ZFDR, DeepVoxNet, DLAN, GIFT, REVGG, MVCNN, RotationNet, View-GCN, View-GCN++, and CAR-Net. It can be seen from Table 2 that among the ten indicators, two have reached the best level and seven have reached the second best level. This result is very competitive compared to the experimental results of other methods.
[0051] In summary, an embodiment of the present invention provides a 3D object recognition and retrieval method based on a graph attention network, which establishes a recognition model including a convolutional neural network, a graph attention network, a global pooling module and a multi-layer perceptron module. The multi-perspective images of the 3D object are first input into the convolutional neural network for preliminary feature extraction to obtain the primary feature vector corresponding to each perspective image, and then the similarity between the primary feature vectors of each perspective image is calculated, and a connection graph is constructed based on the similarity, wherein the nearest connection graph focuses on learning the information of the geometric position of the 3D object, the farthest connection graph strengthens the expression of differentiated information and reduces the input of redundant information of the view, and the fully connected graph is used to update the global information of the 3D object. The connection graph is further feature fused through a graph neural network with a dynamic attention mechanism, which fully considers the connection between multiple views and improves the accuracy of 3D object recognition.
[0052] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and substitutions can be made without departing from the technical principles of the present invention. These improvements and substitutions should also be regarded as the scope of protection of the present invention.
Claims
1. A 3D object recognition and retrieval method based on graph attention network, characterized in that: The method comprises the following steps: S1: Acquire multiple images of multiple 3D objects at different viewing angles and labels corresponding to each viewing angle image, preprocess all viewing angle images based on the labels to obtain a data set, and divide the data set into a training set and a validation set in proportion; S2: constructing a recognition model, and using the training set to reversely iterate the recognition model to obtain an optimal recognition model, wherein the recognition model includes a convolutional neural network, a graph attention network, a global pooling module, and a multi-layer perceptron module; S3: Input multiple different-view images of the 3D object to be identified into the optimal recognition model to obtain a final vector, and determine the recognition and retrieval results based on the final vector.
2. According to claim 1, a 3D object recognition and retrieval method based on graph attention network is characterized in that: The preprocessing described in S1 specifically includes: S1.1: classify all view images according to the labels, and resample the view images of each label; S1.2: After resampling, the view images in each label are normalized according to the channel dimension; S1.3: Perform a random affine transformation on the normalized view image to obtain a data set, and divide the data set into a training set and a validation set in proportion, wherein the ratio is 8:2, and the affine transformation includes at least one of random scaling, random translation, and random rotation.
3. The 3D object recognition and retrieval method based on graph attention network according to claim 1, It is characterized in that The reverse iterative training of the recognition model using the training set in S2 to obtain the optimal recognition model includes: S2.1: Input the training set into the convolutional neural network to obtain the primary features of each perspective image vector; S2.2: Calculate the similarity between the primary feature vector of each view image and the primary feature vectors of other view images, and sort the similarities to obtain a sorting result; S2.3: Based on the sorting result, a connection graph with primary feature vectors as nodes is established, wherein the connection graph includes at least one of a nearest connection graph, a farthest connection graph and a full connection graph; S2.4: Input the connection graph into the graph attention network to obtain local feature descriptors and global feature descriptors; S2.5: performing global pooling processing on the local feature descriptor and the global feature descriptor, and inputting the processing result into a multi-layer perceptron to obtain an output vector; S2.6: Convert the label corresponding to each perspective image into a one-hot encoding, calculate the loss function of the output vector and the one-hot encoding, and update the parameters of the recognition model using a stochastic gradient descent method; S2.7: Repeat steps S2.1 to S2.6 until the number of training times reaches the specified number, and the recognition model with the highest accuracy on the validation set is taken as the optimal recognition model.
4. The 3D object recognition and retrieval method based on graph attention network according to claim 3, It is characterized in that The convolutional neural network described in S2.1 is a ResNet34 neural network.
5. The 3D object recognition and retrieval method based on graph attention network according to claim 3, It is characterized in that The method for calculating similarity in S2.2 is as follows: in, For the The primary feature vector corresponding to the image of each viewing angle is For the The primary feature vector corresponding to the image of each viewing angle is is the primary feature vector The L2 norm of is the primary feature vector The L2 norm of .
6. The 3D object recognition and retrieval method based on graph attention network according to claim 3, characterized in that: S2.3 specifically: Based on the sorting result, the primary feature vector corresponding to each perspective image is connected with the primary feature vectors corresponding to the k perspective images with the highest similarity to obtain a nearest connection graph; the primary feature vector corresponding to each perspective image is connected with the primary feature vectors corresponding to the k perspective images with the lowest similarity to obtain a farthest connection graph; the primary feature vector corresponding to each perspective image is connected with the primary feature vectors corresponding to all perspective images to obtain a fully connected graph, wherein the value of k is determined by the number of perspectives of the perspective image obtained in S1.
7. The 3D object recognition and retrieval method based on graph attention network according to claim 3, characterized in that: The value of k is determined by the number of viewpoints from which the viewpoint image is obtained in S1. Specifically, if the viewpoint image is obtained from 12 different viewpoints in S1, k is 2; if the viewpoint image is obtained from 20 different viewpoints in S1, k is 3.
8. The 3D object recognition and retrieval method based on graph attention network according to claim 3, characterized in that: S2.4 includes: Inputting the nearest connection graph and the farthest connection graph into a graph attention network to obtain a first fused feature, performing average pooling and concatenation operations on the first fused feature to obtain a local feature descriptor; The fully connected graph is input into a graph attention network to obtain a second fused feature, and average pooling and concatenation operations are performed on the second fused feature to obtain a global feature descriptor.
9. The 3D object recognition and retrieval method based on graph attention network according to claim 3, characterized in that: The loss function described in S2.6 is as follows: in, The output classification score after inputting the multilayer perceptron into the nearest connection graph, The output classification score after the farthest connection graph is input into the multilayer perceptron, Output classification score after inputting the multilayer perceptron into the fully connected graph, For labels, is the hyperparameter weight for different connection graphs, and in the 12-view loop configuration is , in the 20-view configuration , is a hyperparameter, the default value is 2. To identify the probability value predicted by the model, is the category weight coefficient, The value of is inversely proportional to the number of categories.
10. The 3D object recognition and retrieval method based on graph attention network according to claim 1, characterized in that: The determining of the recognition and retrieval results based on the final vector specifically includes: For the recognition task, the index corresponding to the largest element in the final vector is used as the recognition category; For the retrieval task, the cosine similarity between the output vectors corresponding to each 3D object to be identified is calculated, and the 3D objects below the threshold are filtered based on a preset threshold, and the 3D models above the threshold are taken as retrieval results.
Citation Information
Cited By
Three-dimensional model retrieval method based on multi-ring view and hidden class mining
CN119441528A
