Three-dimensional model retrieval method based on shape perception enhancement

Abstract

The invention discloses a three-dimensional model retrieval method based on shape perception enhancement, and the method comprises the following steps: obtaining and processing a real picture, obtaining the multi-scale features of the real picture, inputting the obtained data into a dynamic sparse feature correction module for processing, obtaining a fusion feature, and carrying out the recognition of the fusion feature. Performing random initialization in a spherical spatial feature aggregation module to generate spherical spatial features; inputting the fused features into a spherical spatial feature aggregation module for processing to obtain spherical vectors, establishing a matching relationship between the spherical spatial features and the spherical vectors, calculating similarities by adopting cosine measurement and sorting, and selecting the first 10 spherical spatial features with highest option similarities as the second 10 spherical spatial features with highest option similarities; and outputting the category labels of the three-dimensional retrieval model corresponding to the first 10 spherical spatial features with the highest similarity as a final three-dimensional model retrieval result. According to the method, the feature extraction of the target shape in the real picture is more accurate through the three-dimensional retrieval model, and the robustness of the three-dimensional retrieval model facing diversified real scenes is remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional model retrieval, and in particular relates to a three-dimensional model retrieval method based on shape perception enhancement. Background Art

[0002] In real-world scenarios, real images are widely used for retrieval of 3D models due to their ease of acquisition and rich information. However, current technologies still face significant challenges in handling the matching task between real images and 3D models. Real images typically contain complex backgrounds, lighting changes, and diverse colors and textures. These non-geometric attributes often interfere with the model's judgment of shape structure during feature extraction. When the appearance of an object in an image is significantly different from that of a 3D model, it often cannot be accurately matched even if its geometric shape remains the same. This reflects the inadequacy of existing methods in distinguishing between object structure and appearance information.

[0003] Existing 3D model retrieval methods rely heavily on prior knowledge during model learning and lack efficient automatic adaptability, particularly in large-scale, diverse data environments, exhibiting low training efficiency and generalization performance. In recent years, a novel contrastive learning framework has been proposed that effectively narrows the modality gap between 2D images and 3D models by jointly optimizing instance-level and category-level contrastive learning tasks, improving the accuracy of cross-modal retrieval. A novel attention mechanism has been introduced to improve 3D shape segmentation and retrieval, more accurately capturing shape features. A deep 3D shape retrieval method employs hierarchical attention and metric learning, focusing on key shape components for precise retrieval.

[0004] The ability to discern fine-grained structural differences is also a weak point in current technology. While existing methods can perform basic classification tasks at the broad category level, they exhibit instability when dealing with models with subtle differences within categories. For example, a slight change in the curve of a chair back or the structure of an armrest can cause the system to misclassify an object as belonging to a different category. This lack of a hierarchical representation of feature modeling limits its effectiveness in high-precision retrieval tasks.

[0005] In the face of the above problems, we need an image encoding and matching method that can focus on extracting the essential features of the target shape and has strong robustness and fine-grained discrimination capabilities, so as to improve the accuracy and reliability of the three-dimensional model retrieval system driven by real images in practical applications. Summary of the Invention

[0006] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides a three-dimensional model retrieval method based on shape perception enhancement, which solves the problems mentioned in the background art.

[0007] To achieve the above-mentioned object, the present invention provides the following technical solution: a three-dimensional model retrieval method based on shape perception enhancement, comprising the following steps: Step S1: Get a real picture; Step S2: constructing a three-dimensional retrieval model; the three-dimensional retrieval model includes a query encoder, a dynamic sparse feature correction module, and a spherical space feature aggregation module; Step S3: extract the real image through the query encoder to obtain multi-scale features of the real image; Step S4: Input the multi-scale features of the real image into the dynamic sparse feature correction module for processing to obtain fusion features; Step S5: randomly initialize and generate spherical space features in the spherical space feature aggregation module; input the fused features into the spherical space feature aggregation module for processing to obtain spherical vectors, and establish a matching relationship between the spherical space features and the spherical vectors; Step S6: Use the cosine measurement method to calculate the similarity of the matching relationship between the spherical space features and the spherical vectors, sort the calculated similarity scores of the spherical space features and the spherical vectors in descending order to obtain the sorting results, select the top 10 spherical space features with the highest similarity from the sorting results, and output the category labels of the three-dimensional retrieval models corresponding to the top 10 spherical space features with the highest similarity as the final three-dimensional model retrieval results.

[0008] Furthermore, the specific process of step S3 is: The real image is extracted by querying the encoder to obtain the multi-scale features of the real image; The query encoder consists of a base network, a shape overview network, a shape focus network, and an output network; The basic network consists of a first embedding feature, a first basic convolution block, a second embedding feature, a second basic convolution block, a third embedding feature, and a third basic convolution block; The shape overview network consists of the fourth embedding feature, the fourth basic convolutional block, and the first prior knowledge module; The shape focusing network consists of the first dynamic convolutional layer, the fifth embedding feature, and the second dynamic convolutional layer; The output network includes a second prior knowledge module; the real image is sequentially input into the first embedded feature, the first basic convolution block, the second embedded feature, the second basic convolution block, and the third embedded feature in the basic network for processing to obtain the output of the third embedded feature, and the output of the third embedded feature is input into the third basic convolution block for processing to obtain the output of the third basic convolution block, i.e., the low-level and mid-level features of the real image; The low-level and mid-level features of the real image are sequentially input into the fourth embedding feature and the fourth basic convolution block in the shape overview network for processing, obtaining the output of the fourth basic convolution block. The output of the fourth basic convolution block is then input into the first priori knowledge module for processing, obtaining the output of the first priori knowledge module, i.e., the real image context feature. The real image context features are sequentially input into the first dynamic convolution layer and the fifth embedded features in the shape focusing network for processing to obtain the output of the fifth embedded features. The real image context features and the output of the fifth embedded features are input into the second dynamic convolution layer for processing to obtain the output of the second dynamic convolution layer. The output of the second dynamic convolutional layer is input into the second prior knowledge module in the output network for processing to obtain the output of the second prior knowledge module; that is, the multi-scale features of the real image.

[0009] Furthermore, the specific process of step S4 is as follows: The multi-scale features of the real image are processed through the dynamic sparse feature correction module to obtain fusion features; The dynamic sparse feature correction module consists of the first layer normalization, residual feature perception module, second layer normalization, adaptive weight module and sparse feature perception module; First, the multi-scale features of the real image are input into the first layer of normalization for processing to obtain the normalized output of the first layer. The normalized output of the first layer is input into the residual feature perception module for processing to obtain the output of the residual feature perception module, that is, the final class-specific residual attention feature; The final class-specific residual attention feature is added to the multi-scale feature of the real image to obtain the output of the first addition, the output of the first addition is input to the second layer normalization for processing, and the normalized output of the second layer is obtained. The normalized output of the second layer is input to the sparse feature perception module for processing to obtain the output of the sparse feature perception module; The final class-specific residual attention feature is input into the adaptive weight module for processing to obtain the output of the adaptive weight module, i.e., the adaptive weight module fusion feature; The output of the sparse feature perception module is added to the fusion feature of the adaptive weight module to obtain a second added output, and the second added output, the output of the sparse feature perception module and the first added output are added together to obtain a fusion feature.

[0010] Furthermore, the processing process of the residual feature perception module is as follows: The normalized output of the first layer is input into the third layer of the residual feature perception module to obtain the feature tensor x; The feature tensor is processed by the fully connected layer as a classifier Processing is performed to generate the raw classification scores for each category, which are represented by: ; Where, represents the original classification score; Indicates that the input feature tensor x is input into the classifier for calculation; Calculate the class-specific attention score based on the original classification score, expressed as: ; Where, Indicates the Class and class-specific attention scores; represents the exponential function; The temperature parameter for controlling fractional sharpness; Indicates the The transpose of the class-specific eigenvectors at each spatial location; Indicates the The classifier weight vector for each category; Indicates the The transpose of the class-specific eigenvectors at each spatial location; is transposed; is the dot product operation; Based on the The class-specific attention scores generate class-specific feature vectors, representing: ; Where, represents the class-specific feature vector of class i; represents the class-specific attention score of the i-th class at spatial position k; Based on the The class-specific feature vectors at each spatial location are used to calculate the global average pooling feature, which is expressed as: ; Where, Represents the global average pooling feature; The global average pooling feature and the class-specific feature vector are combined to obtain the final class-specific residual attention feature, which is expressed as: ; Where, represents the final class-specific residual attention feature; represents a learnable scalar parameter.

[0011] Furthermore, the processing process of the sparse feature perception module is as follows: The normalized output of the second layer is input into the sparse feature perception module and first processed by 1×1 convolution and 3×3 convolution to generate Q, L, and V; where Q, L, and V are query matrix, key matrix, and value matrix respectively; Calculate the attention matrix based on Q and L, which means: ; Where, represents the attention matrix; is the dimension of the feature vector; The maximum attention score in the retained attention matrix is ​​selected through the selection mechanism, which is expressed as: ; Where, Represents the element in row i and column j of the sparse attention matrix; Represents the attention matrix The jth row of After sorting the attention scores of the j-th row, select the top k largest attention scores; is the element in row i and column j in the attention matrix; Will and V are processed through the activation function to obtain the output of the sparse feature perception module, which is expressed as: ; Where, Represents the output of the sparse feature perception module; Represents the activation function.

[0012] Furthermore, the processing of the adaptive weight module is as follows: The final class-specific residual attention features are input into the adaptive weight module for processing to obtain the adaptive weight module fusion features; The adaptive weight module includes a deep feature pyramid module and a shallow feature pyramid module; The final class-specific residual attention features are input into the deep feature pyramid module and the shallow feature pyramid module for processing, respectively, to obtain the output of the deep feature pyramid module and the output of the shallow feature pyramid module; Add the output of the deep feature pyramid module and the output of the shallow feature pyramid module to obtain the adaptive weight module fusion feature, which is expressed as: ; Where, Represents the adaptive weight module fusion feature of the i-th layer; Represents the output of the i-th layer deep feature pyramid module; Represents the output of the i-th layer shallow feature pyramid module; Represents the weight of adaptive learning.

[0013] Furthermore, the specific process of step S5 is as follows: Random initialization in the spherical space feature aggregation module spherical spatial features, feature dimensions ; Randomly initialized And the feature dimension D form a spherical matrix ,in ; For the Spherical space features, For the A spherical space feature is a dimension real vector; The fused features are mapped into the spherical space feature aggregation module for processing using the predictor network, which is represented as: ; Where, is the eigenvector after mapping, A function representing the predictor network; represents the parameters of the predictor network; It is a fusion feature; Afterwards, the mapped feature vector Perform normalization to obtain the i-th spherical vector ; ; is the total number of spherical vectors; At the same time, the first spherical vectors Stacked by rows, each spherical vector is a row of the matrix to form a batch sample feature matrix ; Based on the i-th spherical vector Calculate the average value of the current batch feature, which means: ; Where, is the average value of the features of the tth batch; represents the i-th spherical vector in the t-th batch; Use the predictor network to average the features of batch t Map it to the spherical space feature aggregation module for processing and calculate the spherical space features, which are expressed as: ; Where, Indicates that at the t+1 iteration Spherical space features; Indicates the tth iteration Spherical space features; represents the momentum parameter; based on and batch sample feature matrix Calculate the similarity matrix, which means: ; Where, represents the sample-prototype similarity matrix; based on right and Perform contrastive learning optimization and obtain contrastive learning optimization results, which means: ; Where, represents the natural logarithm function; represents the temperature parameter; is the jth spherical space feature; is the total number of spherical space features; And based on the contrastive learning optimization results, and matching relationship.

[0014] Further, the three-dimensional retrieval model includes a rendering encoder; The 3D model image is input into the rendering encoder for extraction to obtain the multi-scale features of the rendered image.

[0015] Furthermore, the 3D retrieval model is optimized through a comprehensive loss function, which includes instance-level contrast loss, category-level contrast loss, and spherical space aggregation loss. Instance-level contrastive loss, which means: ; Where, represents instance-level contrast loss; Represents the multi-scale features of the rendered image corresponding to the i-th spherical vector; Represents the multi-scale features of the y-th rendered image; Category-level contrastive loss, expressed as: ; Where, represents the category-level contrast loss; is the category label; Represents the i-th spherical vector Category label ; The feature vector representing the class label e; Spherical space aggregation loss, expressed as: ; Where, represents the spherical space aggregation loss; is the jth spherical space feature; Comprehensive loss function ,express: ;

[0016] Where, is the category-level contrast loss in the comprehensive loss function The weight coefficient of .

[0017] Furthermore, the specific process of step S1 is: Use the public Pix3D dataset and Stanford Cars dataset; The Pix3D dataset contains real images and 3D model images of chairs and tables; the Stanford Cars dataset contains real images and 3D model images of car models; Extract real images and 3D model images from the Pix3D dataset or the Stanford Cars dataset as query data for input into the 3D retrieval model for processing.

[0018] Compared with the existing technology, the present invention has the following beneficial effects: (1) This invention combines a query encoder and a rendering encoder with dynamic convolution to efficiently capture multi-scale features of real images and 3D model views, ensuring accurate extraction of key area features. The shape focusing network in the query encoder uses dynamic convolution layers to focus on target shape details, effectively reducing the interference of non-geometric attributes such as complex backgrounds and lighting changes on feature extraction. Compared with traditional methods, the feature extraction of target shapes in real images is more accurate, significantly improving the robustness of the 3D retrieval model in the face of diverse real-world scenarios and ensuring the accuracy of subsequent retrieval.

[0019] (2) The present invention inputs the multi-scale features and fusion features of the real image into the spherical space feature aggregation module for processing, establishes a cross-modal matching relationship between the fusion features and the spherical space features, and uses the cosine metric to sort the matching relationship to obtain the retrieval results. This cross-modal matching method fully considers the differences and connections between the real image and the three-dimensional model view in the feature space. Through the optimization processing of the spherical space feature aggregation module, it effectively narrows the modal gap between the two-dimensional image and the three-dimensional model. Compared with traditional cross-modal matching methods, it can more accurately measure the similarity between the two, thereby improving the accuracy of cross-modal retrieval, providing a more reliable solution for three-dimensional model retrieval based on real images, and in practical applications, it can more accurately find three-dimensional models that match real images for users. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flow chart of the method of the present invention.

[0021] Figure 2 It is a frame diagram of the query encoder of the present invention.

[0022] Figure 3 This is a framework diagram of the dynamic sparse feature correction module of the present invention. DETAILED DESCRIPTION

[0023] like Figure 1 As shown, the present invention provides a technical solution: a three-dimensional model retrieval method based on shape perception enhancement, comprising the following steps: Step S1: Get a real picture; Step S2: constructing a three-dimensional retrieval model; the three-dimensional retrieval model includes a query encoder, a dynamic sparse feature correction module, and a spherical space feature aggregation module; Step S3: extract the real image through the query encoder to obtain multi-scale features of the real image; Step S4: Input the multi-scale features of the real image into the dynamic sparse feature correction module for processing to obtain fusion features; Step S5: randomly initialize and generate spherical space features in the spherical space feature aggregation module; input the fused features into the spherical space feature aggregation module for processing to obtain spherical vectors, and establish a matching relationship between the spherical space features and the spherical vectors; Step S6: Use the cosine measurement method to calculate the similarity of the matching relationship between the spherical space features and the spherical vectors, sort the calculated similarity scores of the spherical space features and the spherical vectors in descending order to obtain the sorting results, select the top 10 spherical space features with the highest similarity from the sorting results, and output the category labels of the three-dimensional retrieval models corresponding to the top 10 spherical space features with the highest similarity as the final three-dimensional model retrieval results.

[0024] The specific process of step S1 is as follows: We selected the Pix3D dataset and the Stanford Cars dataset, which are widely recognized public datasets in the field of 3D models. The Pix3D dataset contains a variety of object categories (such as chairs and tables), providing real images and 3D model images; the Stanford Cars dataset contains 196 vehicle categories, providing real images and 3D model images; Extract real images and 3D model images from the Pix3D dataset or the Stanford Cars dataset as query data for input into the 3D retrieval model for processing; It should be noted that the processing procedures for the Pix3D dataset and the Stanford Cars dataset are the same.

[0025] The specific process of step S3 is as follows: like Figure 2 As shown in the figure, the real image is extracted through the query encoder to obtain the multi-scale features of the real image; the query encoder is built based on ResNet50; The query encoder consists of a base network, a shape overview network, a shape focus network, and an output network; The basic network consists of a first embedding feature, a first basic convolution block, a second embedding feature, a second basic convolution block, a third embedding feature, and a third basic convolution block; The shape overview network consists of the fourth embedding feature, the fourth basic convolutional block, and the first prior knowledge module; The shape focusing network consists of the first dynamic convolutional layer, the fifth embedding feature, and the second dynamic convolutional layer; The output network includes a second prior knowledge module; the real image is sequentially input into the first embedded feature, the first basic convolution block, the second embedded feature, the second basic convolution block, and the third embedded feature in the basic network for processing to obtain the output of the third embedded feature, and the output of the third embedded feature is input into the third basic convolution block for processing to obtain the output of the third basic convolution block, i.e., the low-level and mid-level features of the real image; The low-level and mid-level features of the real image are sequentially input into the fourth embedding feature and the fourth basic convolution block in the shape overview network for processing, obtaining the output of the fourth basic convolution block. The output of the fourth basic convolution block is then input into the first priori knowledge module for processing, obtaining the output of the first priori knowledge module, i.e., the real image context feature. The real image context features are sequentially input into the first dynamic convolution layer and the fifth embedded features in the shape focusing network for processing to obtain the output of the fifth embedded features. The real image context features and the output of the fifth embedded features are input into the second dynamic convolution layer for processing to obtain the output of the second dynamic convolution layer. The output of the second dynamic convolutional layer is input into the second prior knowledge module in the output network for processing to obtain the output of the second prior knowledge module; that is, the multi-scale features of the real image.

[0026] Wherein, the three-dimensional retrieval model includes a rendering encoder; Input the 3D model image into the rendering encoder for extraction to obtain multi-scale features of the rendered image; Among them, the rendering encoder is built based on ResNet18. The rendering encoder has the same processing method as the query encoder. The rendering encoder is also composed of a base network, a shape overview network, a shape focus network and an output network. However, the rendering encoder has lower computational complexity and is suitable for processing multi-view data. It also extracts low-level and mid-level features of the 3D model image through the base network, the shape overview network, the shape focus network and the output network, and uses dynamic convolution layers to finely perceive the low-level and mid-level features. In addition, both the query encoder and the rendering encoder use dynamic convolution layers to improve the feature expression ability, so that the rendering encoder and the query encoder can better capture important information in real images and 3D model images, thus laying the foundation for subsequent feature fusion and matching tasks.

[0027] The specific process of step S4 is as follows: like Figure 3 As shown in the figure, the multi-scale features of the real image are processed by the dynamic sparse feature correction module to obtain fusion features; the dynamic sparse feature correction module is built based on the Transformer framework; The dynamic sparse feature correction module consists of the first layer normalization, residual feature perception module, second layer normalization, adaptive weight module and sparse feature perception module; First, the multi-scale features of the real image are input into the first layer of normalization for processing to obtain the normalized output of the first layer. The normalized output of the first layer is input into the residual feature perception module for processing to obtain the output of the residual feature perception module, that is, the final class-specific residual attention feature; The final class-specific residual attention feature is added to the multi-scale feature of the real image to obtain the output of the first addition, the output of the first addition is input to the second layer normalization for processing, and the normalized output of the second layer is obtained. The normalized output of the second layer is input to the sparse feature perception module for processing to obtain the output of the sparse feature perception module; The final class-specific residual attention feature is input into the adaptive weight module for processing to obtain the output of the adaptive weight module, i.e., the adaptive weight module fusion feature; The output of the sparse feature perception module is added to the fusion feature of the adaptive weight module to obtain a second added output, and the second added output, the output of the sparse feature perception module and the first added output are added together to obtain a fusion feature.

[0028] Among them, the processing process of the residual feature perception module is: The normalized output of the first layer is input into the third layer of the residual feature perception module to obtain the feature tensor x; Through the fully connected layer (1×1 convolution) as the classifier FC pair feature tensor Processing is performed to generate the raw classification scores for each category, which are represented by: ; Where, represents the original classification score; Indicates that the input feature tensor x is input into the classifier FC for calculation; Calculate the class-specific attention score based on the original classification score, expressed as: ; Where, Indicates the Class and class-specific attention scores; represents the exponential function; The temperature parameter for controlling fractional sharpness; Indicates the The transpose of the class-specific eigenvectors at each spatial location; Indicates the The classifier weight vector for each category; Indicates the The transpose of the class-specific eigenvectors at each spatial location; is transposed; is the dot product operation; Based on the The class-specific attention scores generate class-specific feature vectors, representing: ; Where, represents the class-specific feature vector of class i; represents the class-specific attention score of the i-th class at spatial position k; Based on the The class-specific feature vectors at each spatial location are used to calculate the global average pooling feature, which is expressed as: ; Where, Represents the global average pooling feature; The global average pooling feature and the class-specific feature vector are combined to obtain the final class-specific residual attention feature, which is expressed as: ; Where, represents the final class-specific residual attention feature; represents a learnable scalar parameter.

[0029] Among them, the processing process of the sparse feature perception module is: The normalized output of the second layer is input into the sparse feature perception module and first processed by 1×1 convolution and 3×3 convolution to generate Q, L, and V; where Q, L, and V are the query matrix, key matrix, and value matrix respectively; Calculate the attention matrix based on Q and L, which means: ; Where, represents the attention matrix; is the dimension of the feature vector; The maximum attention score in the retained attention matrix is ​​selected through the selection mechanism, which is expressed as: ; Where, Represents the element in row i and column j of the sparse attention matrix; Represents the attention matrix The jth row of After sorting the attention scores of the j-th row, select the top k largest attention scores; is the element in row i and column j in the attention matrix; Will and V are processed through the activation function to obtain the output of the sparse feature perception module, which is expressed as: ; Where, Represents the output of the sparse feature perception module; Represents the activation function.

[0030] Among them, the processing process of the adaptive weight module is: The final class-specific residual attention features are input into the adaptive weight module for processing to obtain the adaptive weight module fusion features; The adaptive weight module includes a deep feature pyramid module and a shallow feature pyramid module; The final class-specific residual attention features are input into the deep feature pyramid module and the shallow feature pyramid module for processing, respectively, to obtain the output of the deep feature pyramid module and the output of the shallow feature pyramid module; Add the output of the deep feature pyramid module and the output of the shallow feature pyramid module to obtain the adaptive weight module fusion feature, which is expressed as: ; Where, Represents the adaptive weight module fusion feature of the i-th layer; Represents the output of the i-th layer deep feature pyramid module; Represents the output of the i-th layer shallow feature pyramid module; Represents the weight of adaptive learning.

[0031] The specific process of step S5 is as follows: Random initialization in the spherical space feature aggregation module spherical spatial features, feature dimensions ; Randomly initialized And the feature dimension D form a spherical matrix ,in ; For the Spherical space features, For the A spherical space feature is a dimension real vector; The fused features are mapped into the spherical space feature aggregation module for processing using the predictor network, which is represented as: ; Where, is the eigenvector after mapping, A function representing the predictor network; represents the parameters of the predictor network; is the fusion feature; then, the mapped feature vector Perform normalization to obtain the i-th spherical vector ; ; is the total number of spherical vectors; At the same time, the first spherical vectors Stacked by rows, each spherical vector is a row of the matrix to form a batch sample feature matrix ; Based on the i-th spherical vector Calculate the average value of the current batch feature, which means: ; Where, is the average value of the features of the tth batch; represents the i-th spherical vector in the t-th batch; Use the predictor network (the predictor network is composed of a multi-layer perceptron MLP) to average the features of the tth batch Map it to the spherical space feature aggregation module for processing and calculate the spherical space features, which are expressed as: ; Where, Indicates that at the t+1 iteration Spherical space features; Indicates the tth iteration Spherical space features; represents the momentum parameter, with a value of 0.99; based on and batch sample feature matrix Calculate the similarity matrix, which means: ; Where, represents the sample-prototype similarity matrix; based on right and Perform contrastive learning optimization and obtain contrastive learning optimization results, which means: ; Where, represents the natural logarithm function; represents the temperature parameter; is the jth spherical space feature; is the total number of spherical space features; And based on the contrastive learning optimization results, and matching relationship.

[0032] The training optimizes the 3D retrieval model through a comprehensive loss function, which includes instance-level contrast loss, category-level contrast loss, and spherical space aggregation loss. Instance-level contrastive loss, which means: ; Where, represents instance-level contrast loss; Represents the multi-scale features of the rendered image corresponding to the i-th spherical vector; Represents the multi-scale features of the y-th rendered image (including positive and negative samples); Category-level contrastive loss, expressed as: ; Where, represents the category-level contrast loss; is the category label; Represents the i-th spherical vector Category label ; The feature vector representing the class label e; Spherical space aggregation loss, expressed as: ; Where, represents the spherical space aggregation loss; is the jth spherical space feature; Comprehensive loss function ,express: ;

[0033] Where, is the category-level contrast loss in the comprehensive loss function The weight coefficient of .

[0034] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A three-dimensional model retrieval method based on shape perception enhancement, characterized in that: The steps include: Step S1: Get a real picture; Step S2: constructing a three-dimensional retrieval model; the three-dimensional retrieval model includes a query encoder, a dynamic sparse feature correction module, and a spherical space feature aggregation module; Step S3: extract the real image through the query encoder to obtain multi-scale features of the real image; Step S4: Input the multi-scale features of the real image into the dynamic sparse feature correction module for processing to obtain fusion features; Step S5: randomly initialize and generate spherical space features in the spherical space feature aggregation module; input the fused features into the spherical space feature aggregation module for processing to obtain spherical vectors, and establish a matching relationship between the spherical space features and the spherical vectors; Step S6: Use the cosine measurement method to calculate the similarity of the matching relationship between the spherical space features and the spherical vectors, sort the calculated similarity scores of the spherical space features and the spherical vectors in descending order to obtain the sorting results, select the top 10 spherical space features with the highest similarity from the sorting results, and output the category labels of the three-dimensional retrieval models corresponding to the top 10 spherical space features with the highest similarity as the final three-dimensional model retrieval results.

2. The 3D model retrieval method based on shape perception enhancement according to claim 1, characterized in that: The specific process of step S3 is: The real image is extracted by querying the encoder to obtain the multi-scale features of the real image; The query encoder consists of a base network, a shape overview network, a shape focus network, and an output network; The basic network consists of a first embedding feature, a first basic convolution block, a second embedding feature, a second basic convolution block, a third embedding feature, and a third basic convolution block; The shape overview network consists of the fourth embedding feature, the fourth basic convolutional block, and the first prior knowledge module; The shape focusing network consists of the first dynamic convolutional layer, the fifth embedding feature, and the second dynamic convolutional layer; The output network includes a second prior knowledge module; The real picture is sequentially input into the first embedded feature, the first basic convolution block, the second embedded feature, the second basic convolution block, and the third embedded feature in the basic network for processing, and the output of the third embedded feature is obtained. The output of the third embedded feature is input into the third basic convolution block for processing, and the output of the third basic convolution block is obtained, that is, the low-level and mid-level features of the real picture; The low-level and mid-level features of the real image are sequentially input into the fourth embedding feature and the fourth basic convolution block in the shape overview network for processing, obtaining the output of the fourth basic convolution block. The output of the fourth basic convolution block is then input into the first priori knowledge module for processing, obtaining the output of the first priori knowledge module, i.e., the real image context feature. The real image context features are sequentially input into the first dynamic convolution layer and the fifth embedded features in the shape focusing network for processing to obtain the output of the fifth embedded features. The real image context features and the output of the fifth embedded features are input into the second dynamic convolution layer for processing to obtain the output of the second dynamic convolution layer. The output of the second dynamic convolutional layer is input into the second prior knowledge module in the output network for processing to obtain the output of the second prior knowledge module; that is, the multi-scale features of the real image.

3. The 3D model retrieval method based on shape perception enhancement according to claim 2, characterized in that: The specific process of step S4 is: The multi-scale features of the real image are processed through the dynamic sparse feature correction module to obtain fusion features; The dynamic sparse feature correction module consists of the first layer normalization, residual feature perception module, second layer normalization, adaptive weight module and sparse feature perception module; First, the multi-scale features of the real image are input into the first layer of normalization for processing to obtain the normalized output of the first layer. The normalized output of the first layer is input into the residual feature perception module for processing to obtain the output of the residual feature perception module, that is, the final class-specific residual attention feature; The final class-specific residual attention feature is added to the multi-scale feature of the real image to obtain the output of the first addition, the output of the first addition is input to the second layer normalization for processing, and the normalized output of the second layer is obtained. The normalized output of the second layer is input to the sparse feature perception module for processing to obtain the output of the sparse feature perception module; The final class-specific residual attention feature is input into the adaptive weight module for processing to obtain the output of the adaptive weight module, i.e., the adaptive weight module fusion feature; The output of the sparse feature perception module is added to the fusion feature of the adaptive weight module to obtain a second added output, and the second added output, the output of the sparse feature perception module and the first added output are added together to obtain a fusion feature.

4. The 3D model retrieval method based on shape perception enhancement according to claim 3, characterized in that: The processing process of the residual feature perception module is: The normalized output of the first layer is input into the third layer of the residual feature perception module to obtain the feature tensor x; The feature tensor is processed by the fully connected layer as a classifier Processing is performed to generate the raw classification scores for each category, which are represented by: ； Where, represents the original classification score; Indicates that the input feature tensor x is input into the classifier for calculation; Calculate the class-specific attention score based on the original classification score, expressed as: ； Where, Indicates the Class and class-specific attention scores; represents the exponential function; The temperature parameter for controlling fractional sharpness; Indicates the The transpose of the class-specific eigenvectors at each spatial location; Indicates the The classifier weight vector for each category; Indicates the The transpose of the class-specific eigenvectors at each spatial location; is transposed; is the dot product operation; Based on the The class-specific attention scores generate class-specific feature vectors, representing: ； Where, represents the class-specific feature vector of class i; represents the class-specific attention score of the i-th class at spatial position k; Based on the The class-specific feature vectors at each spatial location are used to calculate the global average pooling feature, which is expressed as: ； Where, Represents the global average pooling feature; The global average pooling feature and the class-specific feature vector are combined to obtain the final class-specific residual attention feature, which is expressed as: ； Where, represents the final class-specific residual attention feature; represents a learnable scalar parameter.

5. The three-dimensional model retrieval method based on shape perception enhancement according to claim 4, characterized in that: The processing process of the sparse feature perception module is as follows: The normalized output of the second layer is input into the sparse feature perception module and first processed by 1×1 convolution and 3×3 convolution to generate Q, L, V; Where Q, L, and V are query matrix, key matrix, and value matrix respectively; Calculate the attention matrix based on Q and L, which means: ； Where, represents the attention matrix; is the dimension of the feature vector; The maximum attention score in the retained attention matrix is ​​selected through the selection mechanism, which is expressed as: ； Where, Represents the element in row i and column j of the sparse attention matrix; Represents the attention matrix The jth row of After sorting the attention scores of the j-th row, select the top k largest attention scores; is the element in row i and column j in the attention matrix; Will and V are processed through the activation function to obtain the output of the sparse feature perception module, which is expressed as: ； Where, Represents the output of the sparse feature perception module; Represents the activation function.

6. The three-dimensional model retrieval method based on shape perception enhancement according to claim 5, characterized in that: The processing process of the adaptive weight module is: The final class-specific residual attention features are input into the adaptive weight module for processing to obtain the adaptive weight module fusion features; The adaptive weight module includes a deep feature pyramid module and a shallow feature pyramid module; The final class-specific residual attention features are input into the deep feature pyramid module and the shallow feature pyramid module for processing, respectively, to obtain the output of the deep feature pyramid module and the output of the shallow feature pyramid module; Add the output of the deep feature pyramid module and the output of the shallow feature pyramid module to obtain the adaptive weight module fusion feature, which is expressed as: ； Where, Represents the adaptive weight module fusion feature of the i-th layer; Represents the output of the i-th layer deep feature pyramid module; Represents the output of the i-th layer shallow feature pyramid module; Represents the weight of adaptive learning.

7. The three-dimensional model retrieval method based on shape perception enhancement according to claim 6, characterized in that: The specific process of step S5 is: Random initialization in the spherical space feature aggregation module spherical spatial features, feature dimensions ; Randomly initialized And the feature dimension D form a spherical matrix ,in ; For the Spherical space features, For the A spherical space feature is a dimension real vector; The fused features are mapped into the spherical space feature aggregation module for processing using the predictor network, which is represented as: ； Where, is the eigenvector after mapping, A function representing the predictor network; represents the parameters of the predictor network; It is a fusion feature; Afterwards, the mapped feature vector Perform normalization to obtain the i-th spherical vector ; ; is the total number of spherical vectors; At the same time, the first spherical vectors Stacked by rows, each spherical vector is a row of the matrix to form a batch sample feature matrix ; Based on the i-th spherical vector Calculate the average value of the current batch feature, which means: ； Where, is the average value of the features of the tth batch; represents the i-th spherical vector in the t-th batch; Use the predictor network to average the features of batch t Map it to the spherical space feature aggregation module for processing and calculate the spherical space features, which are expressed as: ； Where, Indicates that at the t+1 iteration Spherical space features; Indicates the tth iteration Spherical space features; represents the momentum parameter; based on and batch sample feature matrix Calculate the similarity matrix, which means: ； Where, represents the sample-prototype similarity matrix; based on right and Perform contrastive learning optimization and obtain contrastive learning optimization results, which means: ； Where, represents the natural logarithm function; represents the temperature parameter; is the jth spherical space feature; is the total number of spherical space features; And based on the contrastive learning optimization results, and matching relationship.

8. The three-dimensional model retrieval method based on shape perception enhancement according to claim 7, characterized in that: The three-dimensional retrieval model includes a rendering encoder; The 3D model image is input into the rendering encoder for extraction to obtain the multi-scale features of the rendered image.

9. The three-dimensional model retrieval method based on shape perception enhancement according to claim 8, characterized in that: Optimize the 3D retrieval model through a comprehensive loss function, which includes instance-level contrast loss, category-level contrast loss, and spherical space aggregation loss; Instance-level contrastive loss, which means: ； Where, represents instance-level contrast loss; Represents the multi-scale features of the rendered image corresponding to the i-th spherical vector; Represents the multi-scale features of the y-th rendered image; Category-level contrastive loss, expressed as: ； Where, represents the category-level contrast loss; is the category label; Represents the i-th spherical vector Category label ; The feature vector representing the class label e; Spherical space aggregation loss, expressed as: ； Where, represents the spherical space aggregation loss; is the jth spherical space feature; Comprehensive loss function ,express: ; Where, is the category-level contrast loss in the comprehensive loss function The weight coefficient of .

10. The three-dimensional model retrieval method based on shape perception enhancement according to claim 9, characterized in that: The specific process of step S1 is: Use the public Pix3D dataset and Stanford Cars dataset; The Pix3D dataset contains real images and 3D model images of chairs and tables; the Stanford Cars dataset contains real images and 3D model images of car models; Extract real images and 3D model images from the Pix3D dataset or the Stanford Cars dataset as query data for input into the 3D retrieval model for processing.

Images