Multi-view three-dimensional model retrieval method and system based on dual-parameter fusion network

Through the multi-view three-dimensional model retrieval method of dual-parameter fusion network, the problem of missing key features in the multi-view feature aggregation process is solved, the accuracy and reliability of three-dimensional model retrieval is improved, and more accurate feature description and classification recognition are achieved.

CN119089000AActive Publication Date: 2024-12-06UNIV OF JINAN

Patent Information

Application Number
CN202411271459.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-11
Publication Date
2024-12-06
Estimated Expiration
2044-09-11

AI Technical Summary

Technical Problem

The existing three-dimensional model retrieval methods based on multi-view have large amounts of calculations, low feature aggregation effect, and are prone to losing key features during the aggregation process, resulting in low retrieval accuracy.

Method used

Using a dual-parameter fusion network method, by acquiring multi-views (lighting, depth, thickness) and performing feature fusion and weighting, the spatial attention mechanism and full connection layer improve the accuracy and importance of feature descriptors, and realize multiple Level feature fusion to improve retrieval accuracy.

Benefits of technology

It improves the accuracy and reliability of three-dimensional model retrieval, reduces the number of updates of feature importance, enhances the feature aggregation effect, and improves the accuracy of classification recognition.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119089000A_ABST
    Figure CN119089000A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of three-dimensional model retrieval, and provides a multi-view three-dimensional model retrieval method and system based on a dual-parameter fusion network, and the method comprises the following steps: obtaining multiple views corresponding to a to-be-retrieved three-dimensional model; fusing the illumination multi-view and the depth multi-view, extracting features, adaptively setting weight parameters based on the similarity of the features, and adjusting the feature importance to obtain a first layer of feature descriptors; feature extraction is carried out on the thickness multi-view, the similarity between features is calculated, the view with the maximum similarity is taken out and processed through a space attention mechanism, and a second-layer feature descriptor is obtained; the first layer of feature descriptors and the second layer of feature descriptors are subjected to weighted fusion based on the feature scores of the feature descriptors, and classification categories of the three-dimensional model are obtained through a full connection layer; according to the method, the feature aggregation effect between multi-view three-dimensional model retrieval can be improved, key features lost in the multi-view feature aggregation process are reduced, and the retrieval precision is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This disclosure relates to the technical field of 3D model retrieval, specifically, to a multi-view model retrieval method based on a dual-parameter fusion network. Figure 3 Dimensional model retrieval methods and systems. Background Technology

[0002] The statements in this section are merely background information relating to this disclosure and do not necessarily constitute prior art.

[0003] 3D model retrieval has garnered significant attention in fields such as computer vision, computer graphics, and virtual reality, with broad applications including autonomous driving, medical imaging, 3D printing, and scene reconstruction. In 3D model retrieval, feature extractors are used to acquire features of both the retrieval object and the 3D model, and then these features are compared to assess their similarity. When a retrieval object is provided, the goal of 3D model retrieval is to find matching 3D models and obtain the corresponding search results. In 3D model retrieval, a 3D model refers to a three-dimensional representation of a real physical object or virtual object used for retrieval and identification. For example, a 3D model can be 3D data of an object obtained through scanning, modeling, or measurement, including geometric shapes and surface details; a specific 3D model can be a 3D model drawing created using drafting software such as CAD.

[0004] In 3D model retrieval research, based on the different data formats required to represent the model, methods can be categorized into four types: point cloud-based, voxel-based, multi-view-based, and mesh-based methods. Point cloud-based 3D model retrieval utilizes deep learning techniques to extract 3D model features from point cloud data due to its irregular and disordered nature. Voxel-based 3D model retrieval approximates the 3D model using voxel sets, employing 3D convolutional neural networks to extract voxel data. Multi-view-based 3D model retrieval uses virtual cameras to acquire multi-view projections of the 3D model, transforming it into multiple 2D views. Multi-view-based 3D model retrieval leverages mature image recognition frameworks, thus it is considered a high-performance method currently available.

[0005] The inventors discovered in their research that existing multi-view-based 3D model retrieval methods suffer from problems such as high computational load due to the large number of views, low feature aggregation effect, loss of key features during aggregation, and excessive feature loss during multi-view feature aggregation. Summary of the Invention

[0006] To address the aforementioned problems, this disclosure proposes a multi-view method based on a dual-parameter fusion network. Figure 3 Multidimensional model retrieval methods and systems to improve multi-view Figure 3The feature aggregation effect between 3D models is improved, reducing the loss of key features during multi-view feature aggregation and improving retrieval accuracy. A more effective multi-view feature aggregation method is proposed for 3D model retrieval, thereby improving the accuracy of 3D model retrieval and making the retrieval results more accurate and reliable.

[0007] To achieve the above objectives, the present disclosure adopts the following technical solution:

[0008] The first aspect of this disclosure provides a multi-view based on a dual-parameter fusion network. Figure 3 The dimensional model retrieval method includes the following steps:

[0009] Obtain multiple views corresponding to the 3D model to be retrieved, including lighting multiple views, depth multiple views and thickness multiple views;

[0010] The illumination multi-view and depth multi-view are fused and features are extracted. The weight parameters are adaptively set based on the similarity of the features, and the feature importance is adjusted to obtain the first layer feature descriptor.

[0011] Feature extraction is performed on the thickness multi-view and the similarity between features is calculated. The view with the highest similarity is extracted and processed through a spatial attention mechanism to obtain the second layer feature descriptor.

[0012] The first-layer feature descriptors and the second-layer feature descriptors are weighted and fused based on the feature scores of the feature descriptors, and then passed through a fully connected layer to obtain the classification category of the 3D model.

[0013] The second aspect of this disclosure provides a multi-view network based on dual-parameter fusion networks. Figure 3 Dimensional model retrieval system, including:

[0014] Data acquisition module: configured to acquire multiple views corresponding to the 3D model to be retrieved, including lighting multiple views, depth multiple views and thickness multiple views;

[0015] The first feature fusion module is configured to fuse multiple lighting views and multiple depth views and extract features, adaptively set weight parameters based on feature similarity, and adjust feature importance to obtain the first layer feature descriptor.

[0016] The second feature fusion module is configured to extract features from the thickness multi-view and calculate the similarity between features. The view with the highest similarity is extracted and processed through a spatial attention mechanism to obtain the second layer feature descriptor.

[0017] The classification and category recognition module is configured to perform weighted fusion of the feature scores of the first-layer feature descriptors and the second-layer feature descriptors, and then obtain the classification category of the 3D model through a fully connected layer.

[0018] An electronic device includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor. When executed by the processor, the computer instructions perform the multi-view operation based on a dual-parameter fusion network as described in the first aspect of this disclosure. Figure 3 Steps in the dimensional model retrieval method.

[0019] A computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the multi-view operation based on a dual-parameter fusion network as described in the first aspect of this disclosure. Figure 3 Steps in the dimensional model retrieval method.

[0020] Compared with the prior art, the beneficial effects of this disclosure are as follows:

[0021] In this disclosure, different parallel processing branches are used for different views, and lighting multi-view... Depth Multiview A processing branch is employed, which adaptively obtains weight parameters based on similarity to identify the importance of features; simultaneously, the thickness multi-view is processed. A separate branch processing method is adopted; the two processing branches are fused based on similarity scores and weighted fusion to further identify the importance of features; the importance parameters are automatically updated twice to extract features with higher importance as features for judgment and recognition, thereby improving the accuracy of classification and retrieval; at the same time, feature fusion is performed in different processing branches, and features in each branch are also fused to achieve multi-level feature fusion, which further improves the accuracy of 3D model classification.

[0022] The advantages of this disclosure, as well as its additional advantages, will be described in detail in the following specific embodiments. Attached Figure Description

[0023] The accompanying drawings, which form part of this disclosure, are used to provide a further understanding of this disclosure. The illustrative embodiments of this disclosure and their descriptions are used to explain this disclosure and do not constitute a limitation thereof.

[0024] Figure 1 This is a flowchart of the three-dimensional model retrieval method according to Embodiment 1 of this disclosure;

[0025] Figure 2 This is a schematic diagram of the retrieval method based on the constructed retrieval model based on a dual-parameter fusion network according to Embodiment 1 of this disclosure;

[0026] Figure 3 This is a schematic diagram of the virtual camera settings in the multi-view acquisition method of Embodiment 1 of this disclosure; Detailed Implementation

[0027] The present disclosure will be further described below with reference to the accompanying drawings and embodiments.

[0028] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of this disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0029] It should be noted that the terminology used herein is for descriptive purposes only and is not intended to limit the exemplary embodiments according to this disclosure. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof. It should be noted that, without conflict, the various embodiments and features within those embodiments can be combined with each other. The embodiments will now be described in detail with reference to the accompanying drawings.

[0030] Example 1

[0031] In one or more of the technical solutions disclosed in the embodiments, such as Figures 1 to 2 As shown, multi-view based on dual-parameter fusion network Figure 3 The dimensional model retrieval method includes the following steps:

[0032] Step 1: Obtain multiple views corresponding to the 3D model to be retrieved, including lighting multiple views. Depth Multiview With thickness multi-view

[0033] Step 2: View multiple lighting perspectives Depth Multiview The system performs fusion and feature extraction, adaptively sets weight parameters based on feature similarity, and adjusts feature importance to obtain the first-layer feature descriptor.

[0034] Step 3: Perform multiple views of the thickness. Perform feature extraction and calculate the similarity between features, then extract the view g with the highest similarity. i The second-layer feature descriptor is obtained by processing through the spatial attention mechanism c1.

[0035] Step 4: First-level feature descriptor With the second-level feature descriptor Weighted fusion is performed based on feature scores, and then passed through a fully connected layer. Output the classification category of the 3D model.

[0036] In this embodiment, different parallel processing branches are used for different views to handle multiple lighting views. Depth Multiview A processing branch is employed, which adaptively obtains weight parameters based on similarity to identify the importance of features; simultaneously, the thickness multi-view is processed. A separate branch processing method is adopted; the two processing branches are fused based on similarity scores and weighted fusion is performed to further identify the importance of features; the importance parameters are automatically updated twice, which can extract features with higher importance as the features used for judgment and recognition, thereby improving the accuracy of classification and retrieval; at the same time, feature fusion is performed in different processing branches, and the features of each branch are also fused to achieve multi-level feature fusion, which further improves the accuracy of 3D model classification.

[0037] In step 1, a view refers to a two-dimensional image obtained by taking pictures of a three-dimensional model from different angles; this method generates a series of two-dimensional views, i.e., image projections, by simulating a virtual camera to take pictures of a three-dimensional model from multiple angles.

[0038] For each 3D model, multiple views are generated using different perspectives, lighting conditions, or projection methods. The perspective can be different rotation angles of the model or viewing angles.

[0039] Optionally, the view acquisition process may include the following steps:

[0040] Step 11, Virtual Camera Setup: Set up one or more virtual cameras in 3D space, placing them around the 3D model. Typically, the cameras are evenly distributed around the center point of the model to ensure coverage of the entire 3D model from multiple directions.

[0041] Step 12, View Generation: Virtual cameras capture images of the 3D model from their respective positions, generating 2D images;

[0042] Specifically, each camera viewpoint captures a specific side or local detail of the model.

[0043] In this embodiment, the method for obtaining multiple views involves placing the 3D model at the center, with multiple virtual cameras positioned within a regular circle at a 30° elevation angle. The azimuth angles of the virtual cameras are spaced 30° apart, thereby generating 12 views C1 to C12. Figure 3 As shown;

[0044] Multiple views of lighting Depth Multiview and thickness multi-view In this context, i in the subscripts and superscripts represents the i-th view, and m represents the m-th category, which is the classification category of the 3D model.

[0045] The obtained multi-views are feature representations of the 3D model. The views are 2D images, and image recognition and deep learning technologies can be directly applied to extract the features of the views in order to achieve the retrieval and recognition of the 3D model.

[0046] In step 2, multiple views of the lighting are generated. Depth Multiview The system performs fusion and feature extraction, adaptively sets weight parameters based on feature similarity, and adjusts feature importance to obtain the first-layer feature descriptor. like Figure 2 As shown, the upper processing branch in the diagram implements the following steps:

[0047] Step 21, input multiple lighting views and depth multiview Channel blending yields a multi-view image with lighting depth blending.

[0048] In step 21, the input lighting multi-view is processed. and depth multiview Channel fusion, specifically, uses a direct stitching method to stitch the channels of the lighting view and the depth view along the channel dimension. If the lighting view is 3D data with shape (H, W, C1) and the depth view is 3D data with shape (H, W, C2), then the stitched result is (H, W, C1+C2). This method preserves the original information of each channel.

[0049] In the three-dimensional data (H, W, C1), the meaning of each dimension is as follows:

[0050] H (Height): Represents the height of the image or view, i.e., the number of pixels in the vertical direction.

[0051] W (Width): Represents the width of the image or view, i.e., the number of pixels in the horizontal direction.

[0052] C(Channels): Represents the number of channels in an image or view.

[0053] In multi-view or feature extraction scenarios, H and W represent the height and width of the image, respectively, and C represents the number of channels of the image or feature; channels can contain color information, depth information, or other feature information, such as lighting features, texture features, etc.

[0054] In the steps described above in this embodiment, channel fusion can integrate data from different sources, improving the overall feature representation capability. In multi-view scenarios, channel fusion can integrate lighting and depth features to obtain a two-dimensional view containing more environmental information, thus enabling more accurate understanding and processing of the scene.

[0055] Step 22: Fuse lighting depth into multiple views The layers are fed into a convolutional neural network (FCN) for feature extraction to obtain initial feature descriptors for illumination and depth multi-view. Similarity calculations are then performed to obtain the illumination-depth fusion multi-view similarity matrix. A multi-view image with lighting depth fusion is obtained by summing similarity scores. Multi-view similarity score

[0056] Specifically, lighting depth fusion multi-view Multi-view similarity score The method for determining this includes the following steps:

[0057] Step 221: Initialize feature descriptors for all illumination depth multiviews. Perform standardization processing to normalize each vector into a unit vector;

[0058] Specifically, standardization can be achieved by dividing by their respective norms, that is:

[0059]

[0060] Step 222: Construct a similarity matrix The elements in the i-th row and j-th column of the matrix represent feature descriptors. and The similarity value;

[0061] Step 223: Calculate the initial feature descriptors for any two lighting depth multi-view images. and The similarity values ​​are then filled into the similarity matrix. The corresponding position is obtained;

[0062] Optionally, similarity calculation can be performed by calculating the cosine similarity between two feature descriptors, as shown in the following formula:

[0063]

[0064] Similarity matrix The closer the value in the value is to 1, the more similar the two view features are; the closer it is to 0, the less similar they are.

[0065] Step 224: By analyzing the similarity matrix The overall multi-view similarity score is obtained by summing the results row by row or column by column and element by element. This score reflects the consistency of the fused multi-view in the feature space;

[0066] Taking the first view C1 out of 12 views as an example, the similarity score of the first view C1 is calculated by summing the similarity scores of C1 with the other 11 views. Similarity matrix The matrix is ​​symmetric, and the total similarity score of the first view is obtained by summing each row or column.

[0067] The above methods effectively utilize cosine similarity to evaluate the similarity of fused multi-views and aggregate this information into an overall similarity score to support subsequent decision-making or analysis.

[0068] Step 23: Fuse multiple views based on illumination depth Multi-view similarity score Generate a weight matrix based on the score.

[0069] Step 24: Fuse the illumination and depth into the multi-view initial feature descriptor. With weight matrix Multiplication yields multi-view features Based on the weight matrix Size will affect multi-view features The first-layer feature descriptors are obtained by fusing features from each group separately.

[0070] Further technical solutions involve fusing multiple views based on illumination depth. Multi-view similarity score Based on similarity score Size generates weight matrix Specifically, regarding the obtained similarity score Find the minimum similarity score The minimum similarity score is subtracted from each similarity score and used as the weight parameter, i.e.:

[0071]

[0072] This embodiment improves accuracy by performing dual-parameter weighted fusion of multi-view feature descriptors and multi-level feature fusion. The method takes multi-channel view data as input, first measures the similarity between multiple views before feature weighting, and then obtains the similarity matrix between multiple views. Subsequently, the weight matrix is ​​updated based on the similarity difference between views, and each view is weighted according to the updated weights. This allows the correlation between views to be taken into account during feature aggregation, thereby improving the multi-view feature aggregation effect.

[0073] In step 3, the thickness multi-view is processed. Perform feature extraction and calculate the similarity between features, then extract the view g with the highest similarity. i The second-layer feature descriptor is obtained by processing through the spatial attention mechanism c1. Here are the steps:

[0074] Step 31: Obtain the thickness multi-view Layering is performed using a convolutional neural network (FCN) to extract features, resulting in initial feature descriptors for the thickness multi-view. Similarity was calculated based on Pearson correlation to obtain the thickness multi-view similarity matrix. Thickness multi-view obtained by similarity summation Multi-view similarity score

[0075] This step involves constructing a similarity matrix and calculating the initial feature descriptors for pairwise thickness multiviews. The similarity is calculated by summing the similarities to obtain a multi-view similarity score. The method is the same as step 22, and will not be repeated in this step;

[0076] Step 32: Obtain the thickness multi-view Multi-view similarity score Extract the view g with the highest similarity i The second-layer feature descriptor is obtained through the spatial attention mechanism c1.

[0077] Specifically, extract the view g with the highest similarity. i The second-layer feature descriptor is obtained through the spatial attention mechanism c1. The steps include:

[0078] Step 321: Process the view with the highest similarity through a convolutional layer to obtain query, key and value features. Generate a spatial attention matrix through key-value pairs. This matrix captures discriminative details within the view by exploring the spatial relationship between any two pixels in the features.

[0079] Step 322: Weight the importance of each view feature in the spatial branch using the spatial attention matrix, sum the original input and the result to generate a spatial attention descriptor, and aggregate the view features using the max pooling function to obtain a compact spatial shape descriptor.

[0080] Step 4: Transfer the first-layer feature descriptors and the second-level feature descriptor Feature scoring based on feature descriptors Weighted fusion is performed to obtain the final multi-view feature descriptor; specifically, the weighted fusion method includes the following steps:

[0081] Step 41: Transfer the first-layer feature descriptor and the second-level feature descriptor Feature scores are calculated using the Sigmoid function, and the score calculation formula is as follows:

[0082]

[0083]

[0084] in, It is the first-level feature descriptor The score, It is a second-level feature descriptor The score;

[0085] Step 42: Multiply the two-layer feature descriptors by their corresponding scores to obtain two weighted feature descriptors. Then, calculate the two-layer feature descriptors J. 加 With K 加 The data is then stitched and fused together, and the resulting 3D model category is output through a fully connected layer. The calculation process of the weighted feature descriptor can be represented as follows:

[0086]

[0087]

[0088] In this step, the weights of the weighted calculation are adjusted by using feature scores, which can further highlight the important views to improve the accuracy of identifying the classification categories of 3D models.

[0089] The final 3D model feature descriptor passes through a fully connected layer. Output the classification category of its 3D model.

[0090] One feasible technical solution is that the above steps can be achieved by constructing a retrieval model based on a dual-parameter fusion network, such as... Figure 2 As shown, it includes a first branch network, a second branch network, a weighted fusion module, and a fully connected layer;

[0091] The first branch network includes a channel fusion module, a first convolutional neural network (FCN), a first similarity calculation module, a weight update module, and a pooling module connected in sequence.

[0092] The second branch network includes a second convolutional neural network (FCN), a second similarity calculation module, and a spatial attention module connected in sequence.

[0093] The first branch network is configured to perform step 2, outputting the first-layer feature descriptor. That is, V1, specifically:

[0094] The channel fusion module is used to process the input lighting multiview. and depth multiview And perform channel fusion to obtain a multi-view image with illumination depth fusion. Feature extraction is performed using a first convolutional neural network (FCN) to obtain the initial feature descriptor for the illumination-depth fusion multi-view. The first similarity calculation module uses cosine similarity to calculate similarity, obtaining multiple scores Score1, Score2, Score3...Scorei to construct a score matrix.

[0095] The weight update module is configured to be based on the similarity score matrix. Update the weights to obtain the corresponding weight matrix

[0096] The pooling module is configured to be based on the updated weight matrix. Size will affect multi-view features Group pooling is used to fuse features separately to obtain the first-layer feature descriptors.

[0097] Among them, the pooling module of the first branch network adopts group pooling, which divides the channels of the feature map into several groups and then performs pooling operation on each group separately.

[0098] The second branch network is configured to perform step 3, outputting the second-layer feature descriptor. That is, V2, specifically:

[0099] The second convolutional neural network (FCN) is used to process multiple thickness views. Perform feature extraction;

[0100] The second similarity calculation module is used to calculate similarity using Pearson correlation to obtain the thickness multi-view similarity matrix.

[0101] The spatial attention module is used to extract the view g with the highest similarity. i The second-layer feature descriptor is obtained through the spatial attention mechanism c1.

[0102] Weighted fusion module, such as Figure 2 As shown, it is used to connect the first layer feature descriptors. Second-level feature descriptor The feature descriptor obtained in the first process and the feature descriptor obtained in the second process Feature scoring based on feature descriptors Weighted fusion is performed to obtain the final multi-view feature descriptor.

[0103] The final 3D model feature descriptor passes through a fully connected layer. Output the classification category of its 3D model. For example... Figure 2 As shown, the categories can include: airplanes, plants, cars, cups, etc.

[0104] The method proposed in this embodiment is used to perform dual-parameter weighting on multiple views of a 3D model, solving the problem of losing key features during feature aggregation of multiple views of a 3D model, improving the retrieval accuracy of 3D models, helping to overcome the shortcomings of previous methods, improving the correlation between multiple views of a 3D model, and assigning greater weight to key views and key features. Furthermore, this method has a reliable design principle, simple structure, and very broad application prospects.

[0105] This method focuses on dual-parameter feature weighting and a multi-level feature fusion network to reduce the loss of key features during multi-view feature fusion, continuously adjusting these features to improve the accuracy of retrieval of 3D models. Direct feature fusion often results in the loss of many key features; however, the dual-parameter weighting method weights key features at different stages, thereby reducing the loss of key features. Furthermore, multi-view weighted fusion after calculating multi-view similarity allows the fusion to consider the correlation between different viewpoints, thus making the relationships between multiple views closer. The 3D model retrieval method in this embodiment will result in more accurate and reliable results, improving the robustness of the model.

[0106] Example 2

[0107] Based on Example 1, this example provides a multi-view based on a dual-parameter fusion network. Figure 3 Dimensional model retrieval system, including:

[0108] Data acquisition module: configured to acquire multiple views corresponding to the 3D model to be retrieved, including lighting multiple views, depth multiple views and thickness multiple views;

[0109] The first feature fusion module is configured to fuse multiple lighting views and multiple depth views and extract features, adaptively set weight parameters based on feature similarity, and adjust feature importance to obtain the first layer feature descriptor.

[0110] The second feature fusion module is configured to extract features from the thickness multi-view and calculate the similarity between features. The view with the highest similarity is extracted and processed through a spatial attention mechanism to obtain the second layer feature descriptor.

[0111] The classification and category recognition module is configured to perform weighted fusion of the feature scores of the first-layer feature descriptors and the second-layer feature descriptors, and then obtain the classification category of the 3D model through a fully connected layer.

[0112] It should be noted that each module in this embodiment corresponds one-to-one with each step in embodiment 1, and their specific implementation process is the same, so it will not be repeated here.

[0113] Example 3

[0114] Based on Embodiment 1, this embodiment provides an electronic device, including a memory and a processor, as well as computer instructions stored in the memory and running on the processor. When the processor executes the computer instructions, it performs the multi-view function based on a dual-parameter fusion network as described in Embodiment 1. Figure 3 Steps in the dimensional model retrieval method.

[0115] Example 4

[0116] Based on Embodiment 1, this embodiment provides a computer-readable storage medium for storing computer instructions. When these computer instructions are executed by a processor, they complete the multi-view function based on a dual-parameter fusion network as described in Embodiment 1. Figure 3 Steps in the dimensional model retrieval method.

[0117] The foregoing description is merely a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Those skilled in the art will readily appreciate that various modifications and variations are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present disclosure shall be included within the scope of protection of the present disclosure.

[0118] While the specific embodiments of this disclosure have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of this disclosure. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of this disclosure are still within the scope of protection of this disclosure.

Claims

1. A multi-view 3D model retrieval method based on a dual parameter fusion network, characterized in that: The steps include: Obtaining multiple views corresponding to the 3D model to be retrieved, including illumination multi-view, depth multi-view and thickness multi-view; The illumination multi-view and depth multi-view are fused and features are extracted. The weight parameters are adaptively set based on the similarity of the features, and the feature importance is adjusted to obtain the first-layer feature descriptor. Feature extraction is performed on multiple thickness views and the similarity between features is calculated. The view with the maximum similarity is taken out and processed through the spatial attention mechanism to obtain the second-layer feature descriptor. The first-layer feature descriptors and the second-layer feature descriptors are weightedly fused based on the feature scores of the feature descriptors, and the classification category of the three-dimensional model is obtained through a fully connected layer.

2. The multi-view 3D model retrieval method based on dual parameter fusion network according to claim 1, characterized in that: The method for determining the first-layer feature descriptor comprises the following steps: The input illumination multi-view and depth multi-view channels are fused to obtain illumination depth fusion multi-view; The illumination depth fusion multi-view is layered and input into the convolutional neural network for feature extraction to obtain the illumination depth multi-view initial feature descriptor, and the similarity is calculated to obtain the illumination depth fusion multi-view similarity matrix. The multi-view similarity score of the illumination depth fusion multi-view is obtained by adding the similarities; According to the multi-view similarity scores of the multi-view fusion of illumination depth, a weight matrix is ​​generated according to the score size; The illumination depth fusion multi-view initial feature descriptor is multiplied by the weight matrix to obtain the multi-view feature. The multi-view features are grouped according to the size of the weight matrix, and feature fusion is performed separately to obtain the first-layer feature descriptor.

3. The multi-view 3D model retrieval method based on dual parameter fusion network as claimed in claim 2, characterized in that: The cosine similarity is used to calculate the similarity of the obtained illumination depth multi-view initial feature descriptors; Alternatively, a method for determining a multi-view similarity score of illumination depth fusion multi-views comprises the following steps: Normalize all illumination depth multi-view initial feature descriptors and normalize each vector to a unit vector; Construct a similarity matrix, where the elements in the i-th row and j-th column of the matrix represent feature descriptors and Similarity value of ; Compute the initial feature descriptor of any two illumination depth multi-views and The similarity value of is filled in the corresponding position of the similarity matrix; By similarity matrix Perform element-by-element summation row by row or column by column to obtain the overall multi-view similarity score.

4. The multi-view 3D model retrieval method based on dual parameter fusion network as claimed in claim 2, characterized in that: The method for generating the weight matrix is ​​as follows: specifically, for the obtained similarity scores, a minimum similarity score is found, and each similarity score is subtracted from the minimum similarity score value to take the result as a weight parameter.

5. The multi-view 3D model retrieval method based on dual parameter fusion network as claimed in claim 1, characterized in that: The second layer feature descriptor The method for determining comprises the following steps: The thickness multi-views are layered and feature extracted through a convolutional neural network to obtain the initial feature descriptors of the thickness multi-views. The similarity is calculated based on the Pearson correlation to obtain the thickness multi-view similarity matrix. The multi-view similarity score of the thickness multi-view is obtained by adding the similarities. The multi-view similarity scores of the thick multi-views are taken out, and the second-layer feature descriptor is obtained through the spatial attention mechanism.

6. The multi-view 3D model retrieval method based on dual parameter fusion network as claimed in claim 5, characterized in that: The first layer feature descriptors and the second layer feature descriptors are weightedly fused based on the feature scores of the feature descriptors. Specifically, the steps include: The first layer feature descriptor And the second layer feature descriptor The feature scores are calculated using the Sigmoid function respectively; The two layers of feature descriptors are multiplied with the corresponding two layers of feature descriptor scores to obtain two layers of weighted feature descriptors respectively; then the two layers of feature descriptors are concatenated and fused.

7. The multi-view 3D model retrieval method based on dual parameter fusion network according to claim 1, characterized in that: It also includes a process of constructing a retrieval model temporal multi-view three-dimensional model retrieval method based on a dual parameter fusion network, including a first branch network, a second branch network, a weighted fusion module, and a fully connected layer; The first branch network includes a channel fusion module, a first convolutional neural network, a first similarity calculation module, a weight updating module and a pooling module which are connected in sequence; The second branch network includes a second convolutional neural network, a second similarity calculation module and a spatial attention module which are connected in sequence.

8. A multi-view 3D model retrieval system based on dual parameter fusion network, characterized in that: include: A data acquisition module is configured to acquire multiple views corresponding to the three-dimensional model to be retrieved, including illumination multi-views, depth multi-views and thickness multi-views; The first feature fusion module is configured to fuse the illumination multi-view and the depth multi-view and extract features, adaptively set weight parameters based on the similarity of features, and adjust the feature importance to obtain the first layer feature descriptor; The second feature fusion module is configured to extract features from multiple thickness views and calculate the similarity between the features, extract the view with the maximum similarity and process it through the spatial attention mechanism to obtain the second-layer feature descriptor; Classification category recognition module: It is configured to perform weighted fusion of the first-layer feature descriptors and the second-layer feature descriptors based on the feature scores of the feature descriptors, and obtain the classification category of the three-dimensional model through the fully connected layer.

9. An electronic device, characterized in that: The method comprises a memory and a processor and computer instructions stored in the memory and executed on the processor. When the computer instructions are executed by the processor, the steps in the multi-view three-dimensional model retrieval method based on a dual parameter fusion network as described in any one of claims 1 to 7 are completed.

10. A computer-readable storage medium, characterized in that: Used to store computer instructions, which, when executed by a processor, complete the steps of the multi-view three-dimensional model retrieval method based on a dual parameter fusion network as described in any one of claims 1 to 7.

Citation Information

Patent Citations

  • Three-dimensional model retrieval method of multi-view neural network based on global feature capture aggregation

    CN110457515A

  • Multi-view three-dimensional model retrieval method and system based on pairing depth feature learning

    CN111382300A

  • Multi-view three-dimensional model retrieval method based on multi-level view association convolutional network

    CN111797269A

  • Haze prediction method based on feature enhancement ConvLSTM

    CN112990531A

  • Three-dimensional image printing method and device, equipment and storage medium

    CN118205201A

Cited By

  • Three-dimensional model retrieval method based on multi-view aggregation

    CN120067376A

  • A three-dimensional model retrieval method based on multi-view aggregation

    CN120067376B

  • Three-dimensional live-action geological surveying and mapping method and device, storage medium and equipment

    CN120563753A