Multi-view three-dimensional model retrieval method and system based on dual parameter fusion network

Abstract

The present disclosure relates to the technical field of three-dimensional model retrieval, and proposes a multi-view three-dimensional model retrieval method and system based on a dual parameter fusion network, including the following steps: obtaining a multi-view corresponding to a three-dimensional model to be retrieved; fusing and extracting features from the light multi-view and the depth multi-view, adaptively setting a weight parameter based on the similarity of the features, adjusting the feature importance to obtain a first layer feature descriptor; extracting features from the thickness multi-view and calculating the similarity between the features, taking out the view with the maximum similarity and processing it through a spatial attention mechanism to obtain a second layer feature descriptor; and weighting and fusing the first layer feature descriptor and the second layer feature descriptor based on the feature scores of the feature descriptors, and obtaining the classification category of the three-dimensional model through a full connection layer; the present disclosure can improve the feature aggregation effect between multi-view three-dimensional model retrieval, reduce the loss of key features in the multi-view feature aggregation process, and improve the retrieval precision.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of three-dimensional model retrieval, in particular, to a multi-view three-dimensional model retrieval method and system based on a dual parameter fusion network. Figure Three TECHNICAL FIELD BACKGROUND

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute the prior art.

[0003] Three-dimensional model retrieval is of great concern in the fields of computer vision, computer graphics and virtual reality, and has a wide range of applications, including autonomous driving, medical imaging, 3D printing and scene reconstruction. In the process of three-dimensional model retrieval, a feature extractor is used to obtain the features of the retrieval object and the three-dimensional model, respectively, and then the similarity between them is evaluated by comparing these features. When a retrieval object is provided, the goal of three-dimensional model retrieval is to find a three-dimensional model that matches it and obtain the corresponding retrieval result. In three-dimensional model retrieval, a three-dimensional model refers to a three-dimensional representation of a real physical object or a virtual object for retrieval and identification. For example, a three-dimensional model can be a three-dimensional data of an object obtained by scanning, modeling or measurement, containing geometric shapes, surface details, etc.; a specific three-dimensional model can be a three-dimensional model graph drawn by a drawing software such as CAD.

[0004] In the research of three-dimensional model retrieval, according to the different data formats required to represent the model, it can be divided into four methods: point cloud-based, voxel-based, multi-view-based and mesh-based methods. Point cloud-based three-dimensional model retrieval uses deep learning techniques to extract three-dimensional model features from point cloud data, which has irregular and unordered characteristics. Voxel-based three-dimensional model retrieval uses a set of voxel units to approximate the representation of a three-dimensional model, and uses a 3D convolutional neural network to extract voxel data. Multi-view-based three-dimensional model retrieval uses virtual cameras to obtain view projections of a three-dimensional model from multiple perspectives, converting the three-dimensional model into multiple two-dimensional views. Using multi-view-based three-dimensional model retrieval can take advantage of mature image recognition frameworks, so multi-view-based three-dimensional model retrieval methods are considered to be the best performing methods at present.

[0005] The inventors found in their research that existing multi-view-based three-dimensional model retrieval has the problems of large amount of calculation caused by multiple views, low feature aggregation effect, loss of key features in the aggregation process, and excessive loss of features in multi-view feature aggregation. SUMMARY

[0006] To solve the above problems, the present disclosure provides a multi-view three-dimensional model retrieval method and system based on a dual parameter fusion network, which improves the multi-view three-dimensional model retrieval performance. Figure Three SUMMARY Figure ThreeThe feature aggregation effect between the multi-view model retrieval is reduced, the loss of key features in the multi-view feature aggregation process is reduced, and the retrieval precision is improved. An effective multi-view feature aggregation method is proposed for three-dimensional model retrieval, so as to improve the three-dimensional model retrieval accuracy, and make the retrieval result more accurate and reliable.

[0007] In order to achieve the above object, the present disclosure adopts the following technical solutions:

[0008] The first aspect of the present disclosure provides a multi-view Figure Three The multi-view model retrieval method comprises the following steps:

[0009] Obtaining the multi-view corresponding to the three-dimensional model to be retrieved, including the illumination multi-view, the depth multi-view and the thickness multi-view;

[0010] Fusing the illumination multi-view and the depth multi-view and extracting features, adaptively setting weight parameters based on the similarity of the features, adjusting the feature importance to obtain the first layer feature descriptor;

[0011] Extracting features from the thickness multi-view and calculating the similarity between the features, taking out the view with the maximum similarity and processing it through the spatial attention mechanism to obtain the second layer feature descriptor;

[0012] The first layer feature descriptor and the second layer feature descriptor are weighted and fused based on the feature scores of the feature descriptors, and the classification category of the three-dimensional model is obtained through the full connection layer.

[0013] The second aspect of the present disclosure provides a multi-view Figure Three The multi-view model retrieval system comprises:

[0014] The data acquisition module is configured to obtain the multi-view corresponding to the three-dimensional model to be retrieved, including the illumination multi-view, the depth multi-view and the thickness multi-view;

[0015] The first feature fusion module is configured to fuse the illumination multi-view and the depth multi-view and extract features, adaptively set weight parameters based on the similarity of the features, adjust the feature importance to obtain the first layer feature descriptor;

[0016] The second feature fusion module is configured to extract features from the thickness multi-view and calculate the similarity between the features, take out the view with the maximum similarity and process it through the spatial attention mechanism to obtain the second layer feature descriptor;

[0017] The classification category identification module is configured to weight and fuse the first layer feature descriptor and the second layer feature descriptor based on the feature scores of the feature descriptors, and obtain the classification category of the three-dimensional model through the full connection layer.

[0018] An electronic device includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor. When executed by the processor, the computer instructions perform the multi-view operation based on a dual-parameter fusion network as described in the first aspect of this disclosure. Figure Three The steps in the dimensional model retrieval method.

[0019] A computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the multi-view operation based on a dual-parameter fusion network as described in the first aspect of this disclosure. Figure Three The steps in the dimensional model retrieval method.

[0020] Compared with the prior art, the beneficial effects of this disclosure are as follows:

[0021] In this disclosure, different parallel processing branches are used for different views, and lighting multi-view... Depth Multiview A processing branch is employed, which adaptively obtains weight parameters based on similarity to identify the importance of features; simultaneously, the thickness multi-view is processed. A separate branch processing method is adopted; the two processing branches are fused based on similarity scores and weighted fusion to further identify the importance of features; the importance parameters are automatically updated twice to extract features with higher importance as features for judgment and recognition, thereby improving the accuracy of classification and retrieval; at the same time, feature fusion is performed in different processing branches, and features in each branch are also fused to achieve multi-level feature fusion, which further improves the accuracy of 3D model classification.

[0022] The advantages of this disclosure, as well as its additional advantages, will be described in detail in the following specific embodiments. Attached Figure Description

[0023] The accompanying drawings, which form part of this disclosure, are used to provide a further understanding of this disclosure. The illustrative embodiments of this disclosure and their descriptions are used to explain this disclosure and do not constitute a limitation thereof.

[0024] Figure 1 This is a flowchart of the three-dimensional model retrieval method according to Embodiment 1 of this disclosure;

[0025] Figure 2 This is a schematic diagram of the retrieval method based on the constructed retrieval model based on a dual-parameter fusion network according to Embodiment 1 of this disclosure;

[0026] Figure 3 This is a schematic diagram of the virtual camera settings in the multi-view acquisition method of Embodiment 1 of this disclosure; Detailed Implementation

[0027] The present disclosure will be further described below in conjunction with the accompanying drawings and embodiments.

[0028] It should be noted that the following detailed description is exemplary in nature and is intended to provide further description of the present disclosure. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.

[0029] It should be noted that the terms used herein are only intended to describe specific embodiments and are not intended to limit exemplary embodiments according to the present disclosure. As used herein, the singular form is intended to include the plural form unless the context clearly indicates otherwise, and it should also be understood that when the terms "comprise" and / or "include" are used in the specification, they indicate the presence of the features, steps, operations, devices, components and / or combinations thereof. It should be noted that the various embodiments and features in the present disclosure can be combined with each other without conflict, and the embodiments will be described in detail below with reference to the accompanying drawings.

[0030] Embodiment 1

[0031] In the technical solutions disclosed in one or more embodiments, as shown in Figures 1-2 A multi-view Figure Three dimensional model retrieval method based on a dual parameter fusion network, comprising the following steps:

[0032] Step 1, obtaining the corresponding multi-view of the three-dimensional model to be retrieved, including illumination multi-view depth multi-view and thickness multi-view

[0033] Step 2, fusing and extracting features from the illumination multi-view depth multi-view , setting weight parameters based on the similarity of the features, adjusting the importance of the features to obtain first layer feature descriptors

[0034] Step 3, extracting features from the thickness multi-view and calculating the similarity between the features, taking out the view g with the maximum similarity i through spatial attention mechanism c1 processing, to obtain second layer feature descriptors

[0035] Step 4, the first layer feature descriptors and the second layer feature descriptors are respectively weighted and fused based on the feature scores, and the output is obtained through a fully connected layer to obtain the classification category of the three-dimensional model.

[0036] In this embodiment, different parallel processing branches are used for different views, and the light multi-view Depth multi-view is processed using one processing branch, and the weight parameter is adaptively obtained based on similarity, so as to identify the importance of the feature; meanwhile, the thickness multi-view is processed using a separate branch; the two processing branches are fused and weighted based on the similarity score, and the importance of the feature is further identified; the importance parameter is automatically updated twice, and the feature with higher importance can be extracted as the feature for judgment and recognition, so as to improve the accuracy of classification and retrieval; meanwhile, feature fusion is performed in different processing branches, and the features of each branch are also fused, multi-level feature fusion is realized, and the accuracy of three-dimensional model classification is further improved.

[0037] In step 1, the view refers to a two-dimensional image obtained by shooting a three-dimensional model from different angles; this method generates a series of two-dimensional views, i.e., image projections, by simulating a virtual camera shooting a three-dimensional model from multiple angles;

[0038] For each three-dimensional model, multiple views are generated using different viewing angles, lighting conditions, or projection methods. The viewing angle can be a different rotation angle or observation angle of the model.

[0039] Optionally, the view acquisition process can include the following steps:

[0040] Step 11, virtual camera setting: one or more virtual cameras are set in the three-dimensional space, and these cameras are placed around the three-dimensional model. Usually, the cameras are uniformly distributed around the center point of the model, to ensure that the entire three-dimensional model can be covered from multiple directions;

[0041] Step 12, view generation: the virtual camera shoots the three-dimensional model from its own position to generate a two-dimensional image;

[0042] Specifically, each camera viewing angle captures a specific side or local detail of the model.

[0043] In this embodiment, the multi-view acquisition method is to place the three-dimensional model at the center, and multiple virtual cameras are located at a regular circle with an elevation angle of 30°. The azimuth angle of the virtual camera is spaced apart at an interval of 30°, thereby generating 12 views C1 to C12, as shown in Figure 3 ;

[0044] Light multi-view Depth multi-view and thickness multi-view wherein i in the superscript and subscript represents the i-th view, m represents the m-th category, and the three-dimensional model is a classification category.

[0045] The obtained multi-view is a feature representation of the three-dimensional model, and the view is a two-dimensional image. Image recognition and deep learning techniques can be directly applied to extract features of the view to realize retrieval and recognition of the three-dimensional model.

[0046] In step 2, the light multi-view Depth multi-view is fused and features are extracted. The weight parameters are adaptively set based on the similarity of the features, and the importance of the features is adjusted to obtain the first layer feature descriptor. As Figure 2 shown, in the upper processing branch in the figure, the following steps are implemented:

[0047] In step 21, the input light multi-view and depth multi-view are fused to obtain a light-depth fused multi-view.

[0048] In step 21, the input light multi-view and depth multi-view are fused. Specifically, the channel fusion method is used, and the light view and the depth view are directly spliced in the channel dimension. If the light view is three-dimensional data with a shape of (H, W, C1) and the depth view is three-dimensional data with a shape of (H, W, C2), the spliced result is (H, W, C1+C2). This method preserves the original information of each channel.

[0049] In the three-dimensional data (H, W, C1), each dimension has the following meanings:

[0050] H (Height, height): represents the height of the image or view, that is, the number of pixels in the vertical direction.

[0051] W (Width, width): represents the width of the image or view, that is, the number of pixels in the horizontal direction.

[0052] C (Channels, number of channels): represents the number of channels of the image or view.

[0053] In the context of multi-view or feature extraction, H and W represent the height and width of the image respectively, and C represents the number of channels of the image or feature. The channel can contain color information, depth information or other feature information such as light feature, texture feature, etc.

[0054] In the above steps in this embodiment, the channel fusion can integrate data from different sources and improve the overall feature expression capability. In the multi-view case, the channel fusion can integrate the features of light and depth, thereby obtaining a two-dimensional view containing more environmental information, which can more accurately understand and process the scene.

[0055] Step 22, fusing the light depth multi-view Layered input to the convolutional neural network (FCN) for feature extraction, to obtain the light depth multi-view initial feature descriptor And similarity calculation, to obtain the light depth fusion multi-view similarity matrix Through the similarity of the light depth fusion multi-view Multi-view similarity score

[0056] Specifically, the light depth fusion multi-view Multi-view similarity score The determination method, comprising the following steps:

[0057] Step 221, standardizing all light depth multi-view initial feature descriptors Each vector is normalized to a unit vector;

[0058] Specifically, the standardization can be realized by dividing the respective norm, that is:

[0059]

[0060] Step 222, constructing a similarity matrix The element of the i-th row and the j-th column in the matrix represents the similarity value of the feature descriptor And ;

[0061] Step 223, calculating the similarity value of any two light depth multi-view initial feature descriptors And Fill in the corresponding position of the similarity matrix , to obtain

[0062] Optionally, the similarity calculation can calculate the cosine similarity of two feature descriptors, and the formula is as follows:

[0063]

[0064] The value in the similarity matrix The closer to 1, the more similar the two view features are; the closer to 0, the more dissimilar.

[0065] Step 224, by performing element-wise addition on the similarity matrix Row or column by row or column, to obtain the overall multi-view similarity score This score reflects the consistency of the fused multi-view in the feature space;

[0066] Taking a first view C1 in 12 views as an example, the similarity score of the first view C1 is calculated by calculating the similarity of C1 with other 11 views, and then the similarity with the 11 views is added to obtain The similarity matrix is a symmetric matrix, and the row or column sum is the total similarity score of the first view;

[0067] The above method effectively utilizes the cosine similarity to evaluate the similarity of the fused multi-view, and aggregates the information into a whole similarity score to support subsequent decision or analysis;

[0068] Step 23, according to the light depth fused multi-view The multi-view similarity score According to the size of the score to generate a weight matrix

[0069] Step 24, multiplying the light depth fused multi-view initial feature descriptor With the weight matrix To get the multi-view feature According to the size of the weight matrix The multi-view feature Grouping respectively to get the first layer feature descriptor

[0070] Further technical solutions, according to the light depth fused multi-view The multi-view similarity score According to the size of the similarity score Generate a weight matrix Specifically, for the obtained similarity score Find the smallest similarity score Subtract the smallest similarity score value from each similarity score as a weight parameter, that is:

[0071]

[0072] The embodiment improves the accuracy by double parameter weighted fusion of multi-view feature descriptors and multi-level feature fusion. The method inputs multi-channel view data, first measures the similarity between multi-views before feature weighting, thereby obtaining a similarity matrix between multi-views, then updates a weight matrix according to the similarity difference between views, and weights each view according to the updated weight, so that the correlation between views is considered when aggregating features, thereby improving the multi-view feature aggregation effect.

[0073] In step 3, the thickness multi-view Feature extraction is performed and the similarity between features is calculated, and the view g with the maximum similarity is taken out i The second layer feature descriptor is obtained through spatial attention mechanism c1 processing The steps are as follows:

[0074] Step 31, the thickness multi-view Feature extraction is performed by a convolutional neural network (FCN), and the initial feature descriptor of the thickness multi-view is obtained The similarity is calculated based on the Pearson correlation, and the thickness multi-view similarity matrix is obtained The thickness multi-view The multi-view similarity score is obtained by adding the similarity

[0075] In this step, the similarity matrix is constructed, and the similarity between the initial feature descriptors of the two thickness multi-views is calculated The multi-view similarity score is obtained by adding the similarity The method and steps of step 22 are the same, and this step will not be described again;

[0076] Step 32, the thickness multi-view The multi-view similarity score is obtained The view g with the maximum similarity is taken out i The second layer feature descriptor is obtained through spatial attention mechanism c1

[0077] Specifically, the view g with the maximum similarity is taken out i The second layer feature descriptor is obtained through spatial attention mechanism c1 The steps include:

[0078] Step 321, the view with the maximum similarity is processed through the convolutional layer to obtain the query (query), key (key) and value (value) features, and a spatial attention matrix is generated through the key-value pair. The matrix captures the distinguishing details within the view by exploring the spatial relationship between any two pixels in the feature;

[0079] Step 322, the spatial attention matrix is used to weight the importance of each view feature in the spatial branch, and the original input is added to the result to generate a spatial attention descriptor. The view features are aggregated using the max pooling function to obtain a compact spatial shape descriptor

[0080] Step 4, the first layer feature descriptor and the second layer feature descriptor The feature score based on the feature descriptor The weighted fusion is performed to obtain a final multi-view feature descriptor; specifically, the weighted fusion method includes the following steps:

[0081] Step 41, the first layer feature descriptor and the second layer feature descriptor are respectively subjected to feature score calculation through a Sigmoid function, and the score calculation formula is as follows:

[0082]

[0083]

[0084] wherein, is the score of the first layer feature descriptor , is the score of the second layer feature descriptor ;

[0085] Step 42, the two layer feature descriptors are multiplied by the corresponding two layer feature descriptor scores to obtain two layer weighted feature descriptors respectively. Then, the two layer feature descriptors J 加 and K 加 are spliced and fused, and a three-dimensional model category is output through a full connection layer; the calculation process of the weighted feature descriptor can be represented as follows:

[0086]

[0087]

[0088] In this step, the weight of the weighted calculation is adjusted through the feature score, which can further highlight the important view to improve the accuracy of recognizing the three-dimensional model classification category.

[0089] The final three-dimensional model feature descriptor is output through a full connection layer , and the three-dimensional model classification category thereof is obtained.

[0090] One of the technical solutions that can be implemented is that the above steps can be implemented through a retrieval model based on a dual parameter fusion network, as shown in Figure 2 , which includes a first branch network, a second branch network, a weighted fusion module and a full connection layer.

[0091] The first branch network includes a channel fusion module, a first convolutional neural network (FCN), a first similarity calculation module, a weight updating module and a pooling module connected in sequence.

[0092] The second branch network includes a second convolutional neural network (FCN), a second similarity calculation module and a spatial attention module connected in sequence.

[0093] The first branch network is configured to perform the steps of step 2, output the first layer feature descriptor That is, V1, specifically:

[0094] The channel fusion module is configured to perform channel fusion on the input light illumination multi-view and depth multi-view to obtain a light depth fusion multi-view The feature extraction is performed by a first convolutional neural network (FCN) to obtain an initial feature descriptor of the light depth fusion multi-view The first similarity calculation module calculates the similarity by cosine similarity to obtain a plurality of scores Score1, Score2, Score3, …, Scorei to construct a score matrix

[0095] The weight updating module is configured to update the weights based on the similarity score matrix to obtain a corresponding weight matrix

[0096] The pooling module is configured to perform group pooling based on the updated weight matrix to obtain a multi-view feature The group pooling is performed to obtain a first layer feature descriptor

[0097] The pooling module of the first branch network adopts group pooling (Group Pooling), which divides the channels of the feature map into several groups, and then performs a pooling operation on each group.

[0098] The second branch network is configured to perform the steps of step 3, output the second layer feature descriptor That is, V2, specifically:

[0099] The second convolutional neural network (FCN) is configured to perform feature extraction on the thickness multi-view ;

[0100] The second similarity calculation module is configured to calculate the similarity by Pearson correlation to obtain a thickness multi-view similarity matrix

[0101] The spatial attention module is configured to obtain the maximum similarity view g i by the spatial attention mechanism c1 to obtain the second layer feature descriptor

[0102] The weighted fusion module, as shown in Figure 2 is configured to connect the first layer feature descriptor Second-level feature descriptor The feature descriptor obtained in the first process and the feature descriptor obtained in the second process Feature scoring based on feature descriptors Weighted fusion is performed to obtain the final multi-view feature descriptor.

[0103] The final 3D model feature descriptor passes through a fully connected layer. Output the classification category of its 3D model. For example... Figure 2 As shown, the categories can include: airplanes, plants, cars, cups, etc.

[0104] The method proposed in this embodiment is used to perform dual-parameter weighting on multiple views of a 3D model, solving the problem of losing key features during feature aggregation of multiple views of a 3D model, improving the retrieval accuracy of 3D models, helping to overcome the shortcomings of previous methods, improving the correlation between multiple views of a 3D model, and assigning greater weight to key views and key features. Furthermore, this method has a reliable design principle, simple structure, and very broad application prospects.

[0105] This method focuses on dual-parameter feature weighting and a multi-level feature fusion network to reduce the loss of key features during multi-view feature fusion, continuously adjusting these features to improve the accuracy of retrieval of 3D models. Direct feature fusion often results in the loss of many key features; however, the dual-parameter weighting method weights key features at different stages, thereby reducing the loss of key features. Furthermore, multi-view weighted fusion after calculating multi-view similarity allows the fusion to consider the correlation between different viewpoints, thus making the relationships between multiple views closer. The 3D model retrieval method in this embodiment will result in more accurate and reliable results, improving the robustness of the model.

[0106] Example 2

[0107] Based on Example 1, this example provides a multi-view based on a dual-parameter fusion network. Figure Three Dimensional model retrieval system, including:

[0108] Data acquisition module: configured to acquire multiple views corresponding to the 3D model to be retrieved, including lighting multiple views, depth multiple views and thickness multiple views;

[0109] The first feature fusion module is configured to fuse multiple lighting views and multiple depth views and extract features, adaptively set weight parameters based on feature similarity, and adjust feature importance to obtain the first layer feature descriptor.

[0110] The second feature fusion module is configured to perform feature extraction on the thickness multi-view and calculate the similarity between the features, take out the view with the maximum similarity and process it through a spatial attention mechanism to obtain a second layer feature descriptor.

[0111] The classification category recognition module is configured to perform weighted fusion of the first layer feature descriptor and the second layer feature descriptor based on the feature scores of the feature descriptors, and obtain the classification category of the three-dimensional model through a full connection layer.

[0112] It should be noted that each module in the embodiment corresponds to each step in Embodiment 1 one by one, and the specific implementation process is the same, which will not be repeated here.

[0113] Embodiment 3

[0114] Based on Embodiment 1, this embodiment provides an electronic device, which includes a memory and a processor, and computer instructions stored in the memory and running on the processor, when the computer instructions are run by the processor, the steps of the multi-view model retrieval method based on the dual parameter fusion network in Embodiment 1 are completed. Figure Three

[0115] Embodiment 4

[0116] Based on Embodiment 1, this embodiment provides a computer readable storage medium for storing computer instructions, when the computer instructions are executed by the processor, the steps of the multi-view model retrieval method based on the dual parameter fusion network in Embodiment 1 are completed. Figure Three

[0117] The above only describes the preferred embodiments of the present disclosure and is not intended to limit the present disclosure. For those skilled in the art, the present disclosure can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

[0118] The above describes the specific embodiments of the present disclosure in conjunction with the accompanying drawings, but is not intended to limit the protection scope of the present disclosure. Those skilled in the art should understand that various modifications or changes made on the basis of the technical solutions of the present disclosure without creative labor are still within the protection scope of the present disclosure.​​

Claims

1. A method for multi-view 3D model retrieval based on a dual parameter fusion network, characterized in that, The method comprises the following steps: Obtaining the multi-view corresponding to the three-dimensional model to be searched, including the light multi-view, the depth multi-view and the thickness multi-view; Fusing the light multi-view and the depth multi-view and extracting features, adaptively setting weight parameters based on the similarity of the features, adjusting the importance of the features to obtain the first layer feature descriptor; Extracting features from the thickness multi-view and calculating the similarity between the features, taking out the view with the maximum similarity and processing it through the spatial attention mechanism to obtain the second layer feature descriptor; Weighted fusion of the first layer feature descriptor and the second layer feature descriptor based on the feature score of the feature descriptor, and obtaining the classification category of the three-dimensional model through the full connection layer; First layer feature descriptor A determination method, comprising the following steps: Fusing the input light multi-view and depth multi-view channels to obtain light depth fusion multi-view; Layered input of the light depth fusion multi-view into the convolutional neural network for feature extraction to obtain the light depth multi-view initial feature descriptor, and similarity calculation to obtain the light depth fusion multi-view similarity matrix, and the light depth fusion multi-view multi-view similarity score is obtained by adding the similarity; According to the light depth fusion multi-view multi-view similarity score, a weight matrix is generated according to the score size; Multiplying the light depth fusion multi-view initial feature descriptor and the weight matrix to obtain the multi-view feature, grouping the multi-view feature according to the weight matrix size, and respectively performing feature fusion to obtain the first layer feature descriptor; Second layer feature descriptor A determination method of the second layer feature descriptor, comprising the following steps: The thickness multi-view is layered to extract features by a convolutional neural network to obtain initial feature descriptors of the thickness multi-view, similarity is calculated based on Pearson correlation to obtain a thickness multi-view similarity matrix; and a multi-view similarity score of the thickness multi-view is obtained by adding the similarity Taking the view with the maximum similarity of the thickness multi-view multi-view similarity score through the spatial attention mechanism to obtain the second layer feature descriptor. 2.The multi-view 3D model retrieval method based on dual parameter fusion network of claim 1, wherein: The cosine similarity is used for the obtained light depth multi-view initial feature descriptor to calculate the similarity. The method for determining the multi-view similarity score of the light depth fusion multi-view comprises the following steps: Standardizing all light depth multi-view initial feature descriptors, and normalizing each vector to a unit vector; A similarity matrix is constructed, where the element in the i-th row and j-th column of the matrix represents the similarity value of the feature descriptor with the feature descriptor Computing initial feature descriptors for any two light depth multi-views with a similarity value and filling in the corresponding position of the similarity matrix of By performing a row-wise or column-wise element-wise summation of the similarity matrix a global multi-view similarity score is obtained. 3.The multi-view 3D model retrieval method based on dual parameter fusion network of claim 1, wherein: The generation method of the weight matrix, specifically, for the obtained similarity score, find the smallest similarity score, subtract the smallest similarity score value from each similarity score as the weight parameter. 4.The multi-view 3D model retrieval method based on dual parameter fusion network of claim 1, wherein: The first layer feature descriptor and the second layer feature descriptor are weighted and fused based on the feature score of the feature descriptor, specifically, comprising the following steps: the first layer of feature descriptors and the second layer of feature descriptors are computed by a function of feature scores, respectively; Multiply the two layer feature descriptors and the corresponding two layer feature descriptor scores to obtain two layer weighted feature descriptors respectively; then splice and fuse the two layer feature descriptors. 5.The multi-view 3D model retrieval method based on dual parameter fusion network of claim 1, wherein, It also includes a method for searching time multi-view three-dimensional model based on a double parameter fusion network search model, including a first branch network, a second branch network, a weighted fusion module and a full connection layer; The first branch network comprises a channel fusion module, a first convolutional neural network, a first similarity calculation module, a weight updating module and a pooling module connected in sequence; The second branch network comprises a second convolutional neural network, a second similarity calculation module and a spatial attention module connected in sequence.

6. A multi-view 3D model retrieval system based on a dual parameter fusion network, characterized in that, It includes: The data acquisition module is configured to obtain the multi-view corresponding to the three-dimensional model to be searched, including the light multi-view, the depth multi-view and the thickness multi-view; The first feature fusion module is configured to fuse and extract features of the light multi-view and the depth multi-view, set a weight parameter based on the similarity of the features, adjust the feature importance to obtain a first layer feature descriptor; The second feature fusion module is configured to extract features of the thickness multi-view and calculate the similarity between the features, take out the view with the maximum similarity and process it through a spatial attention mechanism to obtain a second layer feature descriptor; The classification category recognition module is configured to fuse the first layer feature descriptor and the second layer feature descriptor based on the feature scores of the feature descriptors, and obtain the classification category of the three-dimensional model through a fully connected layer; First layer feature descriptor A determination method, comprising the following steps: The input light multi-view and depth multi-view channels are fused to obtain a light depth fusion multi-view; The light depth fusion multi-view is input into a convolutional neural network in layers to obtain initial feature descriptors of the light depth multi-view, and the similarity is calculated to obtain a light depth fusion multi-view similarity matrix, and the multi-view similarity score of the light depth fusion multi-view is obtained by adding the similarity; According to the multi-view similarity score of the light depth fusion multi-view, a weight matrix is generated according to the score size; The initial feature descriptors of the light depth fusion multi-view are multiplied by the weight matrix to obtain multi-view features, and the multi-view features are grouped according to the size of the weight matrix, and the first layer feature descriptors are obtained by feature fusion respectively; Second layer feature descriptor A determination method of the second layer feature descriptor, comprising the following steps: The thickness multi-view is layered to extract features by a convolutional neural network to obtain initial feature descriptors of the thickness multi-view, similarity is calculated based on Pearson correlation to obtain a thickness multi-view similarity matrix; and a multi-view similarity score of the thickness multi-view is obtained by adding the similarity The multi-view similarity score of the thickness multi-view is taken out, and the second layer feature descriptor is obtained through the spatial attention mechanism.

7. An electronic device, comprising: The computer instructions are run by the processor to complete the steps of the multi-view three-dimensional model retrieval method based on the dual parameter fusion network in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, A computer program product for storing computer instructions, which are executed by a processor to complete the steps of the multi-view three-dimensional model retrieval method based on the dual parameter fusion network in any one of claims 1-5.

Images