Three-dimensional model retrieval method and system based on high-order adjacency relation constraint

Abstract

The invention provides a three-dimensional model retrieval method and system based on high-order adjacency relation constraint, and belongs to the technical field of three-dimensional model retrieval, and the method comprises the steps: obtaining a to-be-queried three-dimensional model, rendering an illumination view and a depth view of the to-be-queried three-dimensional model, and carrying out the channel splicing of the illumination view and the depth view to form multi-channel information; based on multi-channel information, adopting a feature extraction model to extract initial features to obtain initial feature descriptors of all channels; performing initial grouping on the multiple views based on the cosine similarity of the initial feature descriptors among the channels; representing the structural correlation degree of each view by adopting a high-order adjacency relation matrix, optimizing the initial grouping based on the high-order adjacency relation matrix to obtain a final grouping result, and obtaining a final feature descriptor based on the final grouping result; and performing similarity measurement on the final feature descriptors and model features in a feature database to obtain a three-dimensional model retrieval result. According to the method, the retrieval stability and accuracy of the three-dimensional model are improved.

Description

Technical Field

[0001] This invention belongs to the field of 3D model retrieval technology, and particularly relates to a 3D model retrieval method and system based on high-order adjacency constraints. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] 3D model retrieval technology has broad application prospects in fields such as virtual reality, industrial manufacturing, and autonomous driving. The core task of 3D model retrieval is to quickly and accurately retrieve similar models from a 3D model database for a user-input model to be retrieved.

[0004] Based on the different representations of 3D models, existing 3D model retrieval methods can be mainly divided into four categories: point cloud-based, voxel-based, mesh-based, and view-based methods. Point cloud-based methods typically utilize deep learning to analyze point cloud data to extract its geometric structural features. These methods offer high flexibility and can adapt to irregular point cloud distributions. However, feature extraction faces significant challenges due to the sparsity and disorder of point cloud data. Voxel-based methods use regular voxel meshes to represent 3D models and employ 3D convolutional neural networks to extract features. These methods can preserve the spatial structural information of the model relatively completely, resulting in high retrieval accuracy. However, as voxel resolution increases, the space and time complexity of these methods also increase significantly, limiting their practical application scale and efficiency. Mesh-based methods extract model features by analyzing the surface mesh data of the model. These methods can capture the continuity and topological structure of the model surface well, but are highly dependent on mesh quality. View-based methods project the 3D model from multiple perspectives to generate a series of 2D images, using existing image processing networks to extract features from the multi-view images, thereby achieving model retrieval. With the development of deep learning and image processing technologies, such methods have demonstrated excellent retrieval performance and good engineering feasibility. However, existing methods often struggle to fully express the structural and semantic information of 3D models during view generation, and the potential relationships between views are not fully explored, thus limiting the accuracy of model retrieval.

[0005] Existing methods for generating rendering views of 3D models are prone to missing geometric and spatial feature information, making it difficult to fully reflect the shape and semantic features of 3D models. In addition, existing multi-view grouping feature fusion methods usually adopt direct grouping, which fails to effectively mine the potential high-order structural relationships between views. Furthermore, traditional pooling operations can cause the loss of key features, resulting in limited model feature expression capabilities and thus affecting retrieval performance. Summary of the Invention

[0006] To overcome the shortcomings of the prior art, this invention proposes a 3D model retrieval method and system based on high-order adjacency constraints. The method uses a lighting view and a depth view to represent the 3D model, and mines the potential relationships between views based on a view double-grouping strategy, thereby enhancing the robustness and discriminativeness of the 3D model feature representation and further improving the stability and accuracy of 3D model retrieval.

[0007] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: In a first aspect, the present invention discloses a three-dimensional model retrieval method based on higher-order adjacency relation constraints, comprising: Obtain the 3D model to be queried, render the lighting view and depth view of the 3D model to be queried, and stitch the lighting view and depth view into multi-channel information. Based on the multi-channel information, an initial feature descriptor for each channel is obtained by using a feature extraction model to extract initial features. Initial grouping of multiple views is performed based on the cosine similarity of the initial feature descriptors between each channel; A high-order adjacency matrix is ​​used to characterize the structural correlation of each view. The initial grouping is optimized based on the high-order adjacency matrix to obtain the final grouping result. The final feature descriptor is obtained based on the final grouping result. The final feature descriptor is compared with the model features in the feature database to obtain the 3D model retrieval results.

[0008] Secondly, this invention discloses a three-dimensional model retrieval system based on high-order adjacency relation constraints, comprising: The data acquisition module is used to acquire the 3D model to be queried, render the lighting view and depth view of the 3D model to be queried, and stitch the lighting view and depth view into multi-channel information. The feature extraction module is used to extract initial features based on the multi-channel information using a feature extraction model to obtain initial feature descriptors for each channel; The initial grouping module is used to initially group multiple views based on the cosine similarity of the initial feature descriptors between each channel; The optimization grouping module is used to characterize the structural correlation of each view using a high-order adjacency relation matrix, optimize the initial grouping based on the high-order adjacency relation matrix to obtain the final grouping result, and obtain the final feature descriptor based on the final grouping result; The model retrieval module is used to measure the similarity between the final feature descriptor and the model features in the feature database to obtain the 3D model retrieval results.

[0009] Thirdly, the present invention discloses an electronic device, including a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when run by the processor, complete the steps of the above-mentioned three-dimensional model retrieval method based on higher-order adjacency constraints.

[0010] Fourthly, the present invention discloses a computer-readable storage medium for storing computer instructions, which, when executed by a processor, complete the steps of the above-mentioned three-dimensional model retrieval method based on higher-order adjacency constraints.

[0011] Fifthly, the present invention discloses a computer program product including executable instructions stored in a computer-readable storage medium; when a processor of an electronic device reads the executable instructions from the computer-readable storage medium and executes the executable instructions, the above-mentioned three-dimensional model retrieval method based on higher-order adjacency constraints is implemented.

[0012] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention addresses the shortcomings of traditional multi-view feature fusion methods that fail to fully explore the potential relationships between views. It introduces a high-order adjacency modeling mechanism. By constructing a multi-order adjacency matrix, it explores the deep structural connections between multiple views, enabling feature interaction and fusion across view groups based on initial grouping. This improves the overall consistency and local sensitivity of the model's feature representation, thereby enhancing the expressiveness and robustness of the model descriptor.

[0013] To further improve the aggregation effect of feature representation, this invention designs a view dual-grouping strategy. This strategy first initially groups multi-channel views based on the cosine similarity between view features. Then, it optimizes and adjusts the grouping results based on higher-order adjacency relationships, guiding the views to aggregate on a more reasonable structural basis. Features within each group are fused to form group-level descriptors, and all group-level features are then uniformly aggregated to generate the final model descriptor. This strategy effectively improves the global expressive power of the model descriptor, contributing to improved accuracy and stability of 3D model retrieval.

[0014] This invention fully utilizes the advantages of depth images in expressing the geometry and spatial structure of 3D models. It combines lighting views and depth views from multiple perspectives and fuses them in the channel dimension, significantly enhancing the completeness and discriminative ability of view information, thereby effectively overcoming the problem that existing rendering views lose structural information.

[0015] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0016] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0017] Figure 1 This is a flowchart of the three-dimensional model retrieval method based on higher-order adjacency constraints as described in Embodiment 1 of the present invention.

[0018] Figure 2 This is a framework diagram of the three-dimensional model retrieval method based on higher-order adjacency constraints described in Embodiment 1 of the present invention. Detailed Implementation

[0019] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0020] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0021] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0022] Example 1 In one or more embodiments, to address the two core problems in 3D model retrieval—namely, the loss of 3D model structural information due to rendered views and the ineffective mining and modeling of potential relationships between views—a 3D model retrieval method based on high-order adjacency constraints is disclosed. Figures 1-2 As shown, it includes the following steps: Step S1: Obtain the 3D model to be queried, render the lighting view and depth view of the 3D model to be queried, and stitch the lighting view and depth view into multi-channel information. Step S1-1: Obtain the 3D model to be queried; Step S1-2: Set up the virtual camera and generate the view; Step S1-2-1: Set up one or more virtual cameras in 3D space, placing them around the 3D model. Typically, the cameras are evenly distributed around the center point of the model to ensure coverage of the entire 3D model from multiple directions.

[0023] Step S1-2-2: Virtual cameras capture images of the 3D model from their respective positions, generating 2D images. Specifically, each camera's viewpoint captures a specific side or local detail of the model.

[0024] In this embodiment, the method for obtaining multiple views involves placing the 3D model at the center, with multiple virtual cameras positioned around a regular circle at a 30° elevation angle. The azimuth angles of the virtual cameras are spaced 30° apart, thereby generating 12 views, including the view... To view .

[0025] Step S1-3: Based on the process of generating views in step S1-2, generate multiple lighting views and multiple depth views of the 3D model. Input the lighting multi-view obtained from the rendered 3D model. Depth Multiview ;in, Indicates the first The model of the first A lighting view from multiple perspectives Indicates the first The model of the first A depth view from multiple perspectives.

[0026] Specifically, in the view generation stage, a fixed light source is used to render the 3D model, obtaining a lighting view that reflects the model's outline and surface texture information. Simultaneously, by calculating the distance from the virtual camera to the model's surface, a depth view that expresses geometric structure and spatial depth information is generated.

[0027] Steps S1-4: Multi-channel information fusion; Lighting Multiview and depth multiview Multi-channel information is generated by stitching and merging along the channel dimension. ;in, Indicates the first The model of the first Multi-channel information from multiple perspectives.

[0028] This approach generates two types of views from multiple fixed perspectives and fuses them along the channel dimension to construct a multi-channel view representation. This allows for the consideration of both the shape and contour features and depth structure information of the model within the same input space. This strategy effectively alleviates the problems of limited view information representation and insufficient shape description in traditional methods.

[0029] Step S2: Based on the multi-channel information, use a feature extraction model to extract initial features to obtain the initial feature descriptors for each channel; Extracting and fusing multi-view features: multi-channel information after channel stitching and fusion. The data is fed into a convolutional neural network for initial feature extraction, resulting in initial feature descriptors. ;in, Indicates the first The model of the first Initial feature descriptors for each perspective.

[0030] The convolutional neural network can adopt network structures such as ResNet50, and there is no limitation here.

[0031] Step S3: Perform initial grouping of the multi-view based on the cosine similarity of the initial feature descriptors between each channel; Initial grouping based on cosine similarity. For the ... Multichannel descriptor Calculate its relationship with The cosine similarity between channels is used to obtain a cosine similarity score between that channel and other channels. Based on this score, the multiple views are grouped. .

[0032] Specifically, the cosine similarity of multiple views is mapped to an interval of 0 to 1, where this interval is divided into 3 groups on average. Each view is then mapped to a specific interval according to its cosine similarity, thus obtaining the initial grouping. ;in Indicates the first The preliminary grouping results of the model, the first The formula for calculating the cosine similarity between multi-channel information and other channel information is as follows: (1) In the formula, Let be the cosine similarity, and n be the number of other channels.

[0033] Step S4: Use a high-order adjacency matrix to characterize the structural correlation of each view, optimize the initial grouping based on the high-order adjacency matrix to obtain the final grouping result, and obtain the final feature descriptor based on the final grouping result; Step S4-1: Optimize grouping results based on higher-order adjacency relations and mine potential structural associations among multiple views. This embodiment introduces a higher-order adjacency relation matrix to characterize the potential structural associations between different views, and in the initial grouping... The feature fusion process of multiple views is optimized based on this. Specifically, firstly, a multi-order adjacency relation matrix is ​​defined. , used to indicate the first The view and the first The view in the first The degree of structural correlation of order is calculated using the following formula: (2) In the formula, Here, s is the distance scaling parameter, and s-nearest neighbor relationship refers to the result obtained by the k-nearest neighbor algorithm. Indicates the first The and the first The view in the first The distance relationship under the order is defined as follows: (3) In the formula, Let i be the vector in the i-th row of the k-1 order relation matrix. Let be the vector in the j-th row of the k-1 order relation matrix.

[0034] In this embodiment, a second-order adjacency relationship is mainly adopted (i.e., Structural modeling is performed to fully explore the indirect relationships between views. Next, based on the second-order adjacency matrix... Optimize view grouping. The specific steps are as follows: Iterate through the initial groups. The degree of structural correlation between each pair of views in ( If the structural correlation between the two is greater than a set threshold, i.e. Then the first Zhang View and the First There must be a significant correlation between the views; otherwise, the first view is considered to be... Zhang View and the First There is no significant correlation between the tensor views; if the structural correlation is greater than a set threshold and the initial groupings do not belong to the same group, i.e. and and If they did not originally belong to the same group, then... and Each group is then placed into its corresponding subgroup. Through these steps, the optimized final grouping result can be obtained. The fusion result includes initial grouping structure information and higher-order adjacency relationships.

[0035] This embodiment optimizes the grouping results through higher-order adjacency relationships, thereby enhancing the ability to model local and global structural information in multi-view feature representation.

[0036] Step S4-2: Perform within-group average pooling on the final grouping results to obtain level feature descriptors, and then perform average pooling on the group-level feature descriptors to obtain the final feature descriptors. ;in, Indicates the first The final feature descriptor of each model.

[0037] Step S5: Perform a similarity measurement between the final feature descriptor and the model features in the feature database to obtain the 3D model retrieval results; Specifically, the final feature descriptor The features of the model to be retrieved are input into the feature database for similarity measurement, and the retrieval results are output based on the feature with the highest similarity. All feature descriptors are pre-stored in the feature database.

[0038] This embodiment introduces a two-stage view grouping strategy. The first stage performs preliminary grouping based on the cosine similarity between views. The second stage optimizes and reconstructs the initial grouping results by combining higher-order adjacency relationships, thereby realizing the mining and reorganization of potential structural relationships between views. Through this mechanism, not only is the structural consistency in the multi-view feature fusion process improved, but the sensitivity of the model descriptor to 3D shape differences is also significantly enhanced.

[0039] The three-dimensional model retrieval method proposed in this invention can significantly improve the accuracy and stability of retrieval results, and has strong practical value and broad application prospects.

[0040] Example 2 In one or more embodiments, a 3D model retrieval system based on high-order adjacency constraints is disclosed, specifically including: The data acquisition module is used to acquire the 3D model to be queried, render the lighting view and depth view of the 3D model to be queried, and stitch the lighting view and depth view into multi-channel information. The feature extraction module is used to extract initial features based on the multi-channel information using a feature extraction model to obtain initial feature descriptors for each channel; The initial grouping module is used to initially group multiple views based on the cosine similarity of the initial feature descriptors between each channel; The optimization grouping module is used to characterize the structural correlation of each view using a high-order adjacency relation matrix, optimize the initial grouping based on the high-order adjacency relation matrix to obtain the final grouping result, and obtain the final feature descriptor based on the final grouping result; The model retrieval module is used to measure the similarity between the final feature descriptor and the model features in the feature database to obtain the 3D model retrieval results.

[0041] Example 3 This embodiment provides an electronic device, including a memory and a processor, as well as computer instructions stored in the memory and running on the processor. When the computer instructions are executed by the processor, they complete the steps of the above-described three-dimensional model retrieval method based on higher-order adjacency constraints.

[0042] Example 4 This embodiment provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, complete the steps of the above-described 3D model retrieval method based on higher-order adjacency constraints.

[0043] Example 5 This embodiment provides a computer program product including executable instructions, which are computer instructions; the executable instructions are stored in a computer-readable storage medium. When the processor of an electronic device reads the executable instructions from the computer-readable storage medium and executes the executable instructions, the electronic device performs the 3D model retrieval method based on higher-order adjacency constraints provided in this embodiment.

[0044] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0045] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0046] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment, whereby a series of operational steps are performed to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0047] The descriptions of each embodiment in the above embodiments have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0048] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for 3D model retrieval based on high-order adjacency relation constraints, characterized in that, The method comprises the following steps: acquiring a three-dimensional model to be queried, rendering an illumination view and a depth view of the three-dimensional model to be queried, and channel splicing the illumination view and the depth view into multi-channel information; based on the multi-channel information, using a feature extraction model to extract initial features to obtain initial feature descriptors of each channel; initially grouping multiple views based on the cosine similarity of the initial feature descriptors between each channel; using a high-order adjacency relation matrix to represent the structural correlation degree of each view, optimizing the initial grouping based on the high-order adjacency relation matrix to obtain a final grouping result, and obtaining a final feature descriptor based on the final grouping result; measuring the similarity between the final feature descriptor and the model features in a feature database to obtain a three-dimensional model retrieval result.

2. The 3D model retrieval method based on high-order adjacency relation constraint according to claim 1, wherein, The initial grouping of multiple views based on the cosine similarity of the initial feature descriptors between each channel is specifically as follows: mapping the cosine similarity of multiple views to an interval of 0 to 1, dividing the interval into several groups on average, and mapping each view cosine similarity to a specific interval according to the size of the cosine similarity to obtain a preliminary grouping.

3. The 3D model retrieval method based on high-order adjacency constraints as described in claim 1, characterized in that, The high-order adjacency relation matrix is used to represent the structural correlation degree of each view, and is specifically as follows: The adjacency relation matrix is ​​used to represent the first... The view and the first The view in the first The degree of structural correlation of order is calculated using the following formula: wherein is a multi-order adjacency relation matrix is a distance scaling parameter, denotes the distance relation between the th view and the th view at the th order, the distance relation being as follows: wherein is the jth initial feature descriptor, is the jth initial feature descriptor, is the jth initial feature descriptor, is the ith row vector in the k-1 order relation matrix, is the jth row vector in the k-1 order relation matrix.

4. The method of claim 1, wherein the high-order adjacency relation constraint is defined as: ###0002### where N is the number of the neighbor relations, and the neighbor relations are defined as: ###0003### The final grouping result obtained by optimizing the initial grouping based on the high-order adjacency relation matrix is specifically as follows: traversing the feature structural correlation degree of each pair of views in the initial grouping, if the structural correlation degree is greater than a set threshold, it is determined that there is significant correlation between the two views, otherwise it is determined that there is no significant correlation between the two views; if the structural correlation degree is greater than the set threshold and the initial grouping does not belong to the same group, then the two views are respectively included in the corresponding grouping.

5. The method of claim 1, wherein the high-order adjacency relation constraint is defined as: ###0002### where N is the number of the neighbor relations, and the neighbor relations are defined as: ###0003### The final feature descriptor obtained based on the final grouping result is specifically as follows: performing in-group average pooling on the final grouping result to obtain group-level feature descriptors, and then performing average pooling on the group-level feature descriptors to obtain the final feature descriptor.

6. The method of claim 1, wherein the high-order adjacency relation constraint is defined as: ###0002### where N is the number of the neighbor relations, and the neighbor relations are defined as: ###0003### The illumination view and the depth view of the three-dimensional model to be queried are rendered, and are specifically as follows: A fixed light source is used to render the three-dimensional model to obtain an illumination view reflecting the contour and surface texture information of the model; at the same time, a depth view expressing the geometric structure and spatial depth information is generated by calculating the distance from a virtual camera to the model surface.

7. A three-dimensional model search system based on high-order adjacency relation constraint, characterized by, The method comprises the following steps: A data acquisition module is configured to acquire a three-dimensional model to be queried, render an illumination view and a depth view of the three-dimensional model to be queried, and channel splice the illumination view and the depth view into multi-channel information; A feature extraction module is configured to extract initial features based on the multi-channel information using a feature extraction model to obtain initial feature descriptors of each channel; An initial grouping module is configured to initially group multiple views based on the cosine similarity of the initial feature descriptors between each channel; An optimized grouping module is configured to use a high-order adjacency relation matrix to represent the structural correlation degree of each view, optimize the initial grouping based on the high-order adjacency relation matrix to obtain a final grouping result, and obtain a final feature descriptor based on the final grouping result; A model retrieval module is configured to measure the similarity between the final feature descriptor and the model features in a feature database to obtain a three-dimensional model retrieval result.

8. An electronic device, comprising: A computer program product comprising executable instructions stored in a computer readable storage medium, wherein the executable instructions, when executed by a processor of an electronic device, implement the method of any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, A computer program product comprising executable instructions stored in a computer readable storage medium, wherein the executable instructions, when executed by a processor of an electronic device, implement the method of any one of claims 1-6.

10. A computer program product, characterised in that, The computer program product comprises executable instructions stored in a computer readable storage medium; The computer program product comprises executable instructions stored in a computer readable storage medium;

Images