CAD Model Retrieval Method Based on Robust Visual Representation and Source Semantic Graph
By jointly training a shared visual network and a category prediction layer, and combining the source semantic graph for structural correction, the problems of modal differences and image completion errors in sketch and CAD model retrieval are solved, thereby improving retrieval accuracy and efficiency.
Patent Information
- Application Number
- CN202611310422.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-27
- Publication Date
- 2026-11-17
AI Technical Summary
In existing sketch-to-CAD model retrieval methods, there are modal differences between sketches and CAD rendered images, resulting in poor retrieval performance. Furthermore, existing methods increase model parameters or introduce image completion errors, making it difficult to effectively utilize the source semantic graph for structural correction.
A shared visual network and a category prediction layer are jointly trained. Through multi-positive sample contrast learning and representation consistency loss before and after occlusion, the visual-category joint basic score is calculated. When the confidence is low, the source semantic graph is used to perform structural residual correction, and the similarity between the query structure vector and the CAD model structure vector is directly obtained.
It improves the accuracy and efficiency of sketch-to-CAD model retrieval, avoids image completion errors, uses source semantic graphs for structural correction only when necessary, and reduces computational load and model complexity.
Smart Images

Figure CN123116375A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the fields of computer-aided design, 3D model retrieval and intelligent manufacturing technology, and in particular relates to a CAD model retrieval method based on robust visual representation and source semantic graph. Background Technology
[0002] Mechanical design, tooling design, mold design, and the design of non-standard automated equipment accumulate a large number of STEP format 3D CAD models. When designing new parts, designers usually want to quickly find reusable parts based on natural language descriptions, sketch outlines, or existing models to reduce redundant modeling and verification.
[0003] Existing sketch-to-CAD model retrieval methods typically render each CAD model into multiple 2D views, then learn common representations of the sketch and CAD views through a cross-modal network, and return candidate models based on similarity ranking. There are significant differences between sketches and rendered CAD images. Sketches have sparse lines, unstable drawing scale and perspective, and may miss holes, slots, or local contours; rendered CAD images are more regular and complete. To adapt to the two types of inputs separately, independent query encoders and CAD encoders can be set, but this significantly increases the number of model parameters. If both types of inputs use the exact same feature modulation method, it is difficult to compensate for the modal differences between them.
[0004] When a sketch has local gaps, existing methods often first complete the sketch through image inpainting or generative modeling, and then input the completed result into the retrieval network. This concatenated approach requires additional training of the generative network and propagates the completion error to subsequent sorting. The visual quality of the completed image is not entirely consistent with the category and shape discrimination capabilities required for CAD model retrieval. Therefore, it is necessary to bypass the pixel-level completion process and directly constrain the representation of the complete and damaged sketches in the CAD model retrieval space.
[0005] Two-dimensional visual features alone are insufficient to clearly express the geometric face adjacency, surface attributes, and manufacturing features in CAD models. Source semantic graphs can organize B-Rep geometric faces, surface fragments, and fragment source relationships, providing supplementary information for visually similar but structurally different candidates. However, calculating the structural similarity between each query and the entire library not only increases computational load but may also interfere with the originally clear visual ranking when the structural representation does not adequately correspond to the query intent. Therefore, it is necessary to determine whether to enable structural correction based on the confidence level of the visual ranking. Summary of the Invention
[0006] The purpose of this invention is to address the problems existing in the prior art and provide a CAD model retrieval method based on robust visual representation and source semantic graph. This method organizes the node information, surface fragment information, and their source associations of candidate CAD models into a source semantic graph, and uses this graph to obtain the structure vector of the candidate CAD model. When the confidence of the visual-category basic ranking is low, structural residual correction is performed on the basic retrieval score. To achieve the above-mentioned objective, this invention specifically adopts the following technical solution:
[0007] In a first aspect, the present invention provides a CAD model retrieval method based on robust visual representation and source semantic graph, comprising the following steps:
[0008] S1. Obtain a query set consisting of two-dimensional query images and a model library consisting of candidate CAD models. Each candidate CAD model represents a three-dimensional model of a mechanical part. Each query image is used to describe a mechanical part. Multi-view images and geometric structure data are associated with each candidate CAD model. At the same time, candidate CAD models that are related to or unrelated to the query images are recorded.
[0009] S2. Form complete query and damaged query from the same query image, and perform multi-positive sample contrast learning on the candidate CAD models related to the query image for complete query and damaged query respectively. The joint retrieval loss, the representation consistency loss before and after occlusion loss and the category supervision loss are weighted and summed to form the total loss. Based on minimizing the total loss, the shared visual network and the category prediction layer are jointly trained.
[0010] S3. Based on the trained shared visual network and category prediction layer, calculate the visual-category joint basic score between the query image to be retrieved and each candidate CAD model in the model library, and sort the candidate CAD models in descending order according to the visual-category joint basic score, thereby forming the basic ranking of the candidate CAD models, which is used to determine whether to enable source semantic graph correction.
[0011] S4. The difference between the two highest visual-category joint basic scores is used as the ranking interval for the query image to be retrieved: If the ranking interval reaches the preset gate threshold, the visual-category joint basic score is used as the final retrieval score of the candidate CAD model; if the ranking interval does not reach the preset gate threshold, the query structure vector is obtained based on the query visual vector, and the CAD model structure vector is obtained based on the constructed source semantic graph. The similarity between the query structure vector and the CAD model structure vector is calculated and used as the structural similarity. The structural similarity is weighted with bounded structural residual weights to form a correction term, which is used to correct the visual-category joint basic score. The corrected score is used as the final retrieval score of the candidate CAD model.
[0012] S5. Arrange all candidate CAD models in descending order of their final search scores to obtain the final ranking and complete the search.
[0013] Based on the above scheme, each step can be implemented in the following preferred manner.
[0014] As a preferred embodiment of the first aspect above, in S2, the specific process of jointly training the shared visual network and the category prediction layer is as follows:
[0015] S21. First, obtain a query image and use it as the complete query. Then, scale the query image to a preset size and randomly set a white rectangular occlusion area within the scaled image. The occluded image will then be used as the damaged query.
[0016] S22. The multi-view images of candidate CAD models related to a query image are taken as relevant multi-view images. A quadruple consisting of the complete query, the damaged query, a view from the relevant multi-view images, and the category label of the query image corresponding to the same query image is used as a sample in the training batch. A shared visual network with modal identity encodes the complete query, the damaged query, and the single view in the sample, respectively, to obtain the corresponding complete query visual vector. Damaged query visual vector Visual vectors of a single view The visual vector of a single view is used as the CAD visual vector of the candidate CAD model. ;in This refers to the sample index in the training batch. This represents the number of samples in the training batch. For view index;
[0017] S23. Calculate the representation consistency loss before and after occlusion based on the complete query visual vector and the damaged query visual vector. Calculate the category supervision loss using the category prediction layer. Calculate the joint retrieval loss based on two similarity matrices. The first similarity matrix is used to represent the similarity between the complete query visual vector and the CAD visual vector, and the second similarity matrix is used to represent the similarity between the damaged query visual vector and the CAD visual vector. Calculate the total loss to update the shared visual network and the category prediction layer.
[0018] As a preferred embodiment of the first aspect mentioned above, in S22, the processing flow of the shared visual network is as follows: the complete query, the damaged query, or the single view is used as the input of the shared visual network. First, it is processed by the shared visual backbone to obtain shape features, and then processed by the shared projection network to obtain projected visual features. These features are then input into the shared residual adapter to obtain adapted features and perform L2 normalization, thereby obtaining the visual vector of the complete query, the visual vector of the damaged query, or the visual vector of the single view.
[0019] As a preferred embodiment of the first aspect above, the shared projection network is composed of a first linear layer, a GELU nonlinear activation layer, and a second linear layer cascaded in sequence, used to map the shape feature to a unified low-dimensional retrieval space, thereby obtaining the projected visual feature.
[0020] The shared residual adapter includes hierarchical normalization. Dimensionality reduction linear layer, GELU nonlinear activation layer The calculation of the linear layer of increasing dimension is as follows:
[0021]
[0022] In the formula, The projected visual features output by the shared projection network; For learnable residual scaling parameters; , and , These are the parameters for the dimension-reducing linear layer and the dimension-increasing linear layer, respectively. It is the sigmoid activation function; Modal indexes for the input of a shared visual network; when using a shared visual network to encode complete or damaged queries, To query modal identity vectors When using a shared visual network to encode the view, CAD modal identity vector ; Features after adaptation.
[0023] As a preferred embodiment of the first aspect above, the joint retrieval loss is specifically calculated as follows:
[0024] S231. Calculate the first A complete query visual vector With the CAD visual vectors Similarity between This forms the first similarity matrix. Calculate the first A damaged query visual vector With the CAD visual vectors Similarity between This forms the second similarity matrix. ;in, A learnable temperature scaling factor; For transpose;
[0025] S232. Construct a multi-positive sample label matrix based on the class labels of each sample in the training batch. When the first The query image of the first sample and the first When the category labels of individual views of samples are the same, the position in the multi-positive sample label matrix value at ,otherwise ;
[0026] S233. Calculate the multi-positive sample loss for a complete query in the CAD direction. Damaged queries reveal multiple positive sample loss in the CAD direction. Multiple positive sample loss from CAD to complete query direction CAD to damaged query direction multi-positive sample loss This allows us to calculate the retrieval loss for the complete query. and retrieval loss of damaged queries The average of the two retrieval losses is used to construct the joint retrieval loss. :
[0027]
[0028]
[0029]
[0030]
[0031]
[0032]
[0033]
[0034] in, Position in the multi-positive sample label matrix The value at; For the first The complete query visual vector and the first Similarity between CAD visual vectors; For the first A damaged query visual vector With the Similarity between CAD visual vectors.
[0035] As a preferred embodiment of the first aspect mentioned above, in S3, the image to be retrieved is... With a candidate CAD model Visual-Category Joint Basic Score Specifically:
[0036]
[0037] in, The query visual vector is obtained by encoding the query image to be retrieved from a pre-trained shared visual network; The CAD visual vector of the candidate CAD model is obtained by encoding all views in the multi-view image through a trained shared visual network and fusing the visual vectors of each view. The class probability vector is obtained by processing the visual vector of the query through a trained class prediction layer and the Softmax function; The class probability vector is obtained by processing the CAD visual vector through a trained class prediction layer and the Softmax function. These are non-negative category semantic weights.
[0038] As a preferred embodiment of the first aspect above, in S4, the query structure vector and the CAD model structure vector are obtained in the following way:
[0039] S41. First, construct the source semantic graph of the candidate CAD model. The source semantic graph includes a node feature matrix. Node adjacency matrix Fragment feature matrix and source association matrix , The number of nodes; This refers to the number of surface fragments or proxy fragments.
[0040] S42. Input the source semantic graph into the trained source semantic graph encoder. First, the node encoder encodes the node information to obtain the node representation, and the fragment encoder encodes the fragment information to obtain the fragment representation. Then, the source constraint decoder performs constrained information fusion on the node representation and fragment representation based on the source association between fragments and nodes to obtain the initial CAD model structure vector.
[0041] S43. The trained structural residual adaptation network performs residual mapping on the initial CAD model structure vector. First, the initial CAD model structure vector is layer normalized, and then processed through the third and fourth linear layers to obtain a residual correction vector with the same dimension as the initial CAD model structure vector. After adjusting the initial CAD model structure vector using the residual correction vector, the final CAD model structure vector is obtained. ;in, A learnable scalar shared by all candidate CAD models;
[0042] S44. The trained query structure mapping network first linearly maps the query visual vector into a vector, which is then activated by GELU and passed through a linear layer and normalized by L2 to obtain the query structure vector.
[0043] As a preferred embodiment of the first aspect, in S41, for candidate CAD models where only triangular meshes can be obtained but the source association between B-Rep geometric surfaces and triangular facets cannot be obtained, the triangular meshes are first translated to a point with the center of the candidate CAD model's bounding box as the origin, and then scaled according to the diagonal length of the bounding box. After removing degenerate triangular facets, the normalized three-dimensional centroid coordinates, unit normal vector, and plane offset of each remaining triangular facet are calculated, and the unit normal vector is made to point away from the center of the bounding box. The three components of the unit normal vector are multiplied by the first quantization factor and rounded, and the plane offset is multiplied by the second quantization factor and rounded. Triangular facets with the same four quantization results are grouped into the same proxy plane group. The areas of the triangular facets in each proxy plane group are summed to obtain the area of the proxy plane group. A predetermined number of proxy plane groups are retained in descending order of size, and each retained proxy plane group is used as a proxy entity to construct a node in the source semantic graph. The normalized 3D centroid coordinates of each triangular facet within each proxy plane group are weighted and averaged to obtain the corresponding proxy entity centroid. The four nearest neighbors of each node are determined based on the Euclidean distance between the centroids of each proxy entity. When any node belongs to the four nearest neighbors of another node, a bidirectional adjacency relationship is established between them. The five-dimensional features of each node consist of the proportion of the area of the corresponding proxy plane group to the total area of all retained proxy plane groups, the three coordinates of the proxy entity centroid, and the surface type code, which is uniformly set to 0. The unit normal vectors of each triangular facet within the proxy plane group are weighted and averaged, and the weighted average result is normalized again to obtain a new unit normal vector. And according to the centroid of the agent entity Calculate the new plane offset Each row of the fragment feature matrix represents a four-dimensional feature of a proxy fragment, consisting of the three components of the new unit normal vector and the new plane offset, approximating the four-dimensional feature of the surface fragment, and the source correlation matrix is the identity matrix.
[0044] As a preferred embodiment of the first aspect above, in S42, fusing the node representation and fragment representation using the source-constrained decoder specifically involves: fusing the node representation... By querying the projection matrix , to represent the fragment Projected through the key matrix respectively Sum projection matrix Calculate the first one according to the following formula The node is the first Attention weight of each fragment , thus calculating the first fragment context of each node and source enhancement And by enhancing the representation of the source of each node, the initial CAD model structure vector is calculated. :
[0045]
[0046]
[0047]
[0048]
[0049] in, Position in the source correlation matrix The value at; The dimension of the structure space; Indicates splicing; , For parameters.
[0050] As a preferred embodiment of the first aspect mentioned above, in S4, the bounded structure residual weights The calculation method is as follows:
[0051]
[0052]
[0053] in, This represents the upper limit of the structural residual amplitude. The upper limit of the pre-set structural residual weight; , and These are learnable parameters; This is the weighting change coefficient; To prevent A positive number set to zero; It is the sigmoid activation function; This is the softplus activation function; The sorting interval for the images to be retrieved.
[0054] In a second aspect, the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, enables the CAD model retrieval method based on robust visual representation and source semantic graph as described in any of the solutions of the first aspect above.
[0055] Thirdly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the CAD model retrieval method based on robust visual representation and source semantic graph as described in any of the solutions of the first aspect above.
[0056] Fourthly, the present invention provides a computer electronic device, which includes a memory and a processor;
[0057] The memory is used to store computer programs;
[0058] The processor is configured to, when executing the computer program, implement the CAD model retrieval method based on robust visual representation and source semantic graph as described in any of the solutions of the first aspect above.
[0059] Compared with the prior art, the present invention has the following advantages:
[0060] This invention uses hand-drawn sketches, outlines, or engineering line drawings as query images, searches for similar CAD models in a model library, and performs source semantic graph correction on query images with low confidence. Compared to methods that set separate query and CAD backbones, this invention only distinguishes modal identities in a shared residual adapter, thus eliminating the need to copy the complete visual backbone. Local occlusion training directly affects the ranking space of the queried CAD model, avoiding the introduction of image completion networks and their generation errors. Category semantic terms are used to supplement simple appearance similarity, while the source semantic graph only participates in the calculation when there is ambiguity in the basic ranking. For query images with high confidence, the structural similarity calculation between the query image and the CAD model is skipped directly, and the basic score and basic ranking do not change due to structural branches. Attached Figure Description
[0061] Figure 1 This is a flowchart of the method of the present invention;
[0062] Figure 2 This is a schematic diagram of the shared visual network composition of the present invention;
[0063] Figure 3 This is a schematic diagram of the source semantic graph encoder of the present invention;
[0064] Figure 4 This is a schematic diagram of a computer electronic device provided by the present invention. Detailed Implementation
[0065] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. Technical features in the various embodiments of the present invention can be combined accordingly without mutual conflict.
[0066] In the description of this invention, it should be understood that the terms "first" and "second" are used only for descriptive purposes and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include at least one of those features.
[0067] like Figure 1 As shown, in a preferred embodiment of the present invention, the CAD model retrieval method based on robust visual representation and source semantic graph includes the following steps S1 to S5. The specific implementation process of each step is described in detail below.
[0068] S1. Obtain a query set consisting of two-dimensional query images and a model library consisting of candidate CAD models. Each candidate CAD model represents a three-dimensional model of a mechanical part. Each query image is used to describe a mechanical part. Multi-view images and geometric structure data are associated with each candidate CAD model. At the same time, candidate CAD models that are related to or unrelated to the query images are recorded.
[0069] It should be noted that in step S1 of this invention, after obtaining the query image, this embodiment establishes a unique query identifier for each query image and records the category label and data partitioning marker of the query image. The query image includes hand-drawn sketches, partially missing sketches, outline drawings, or engineering line drawings. Its category label refers to the mechanical component described by the query image, such as a screw or flange. The data partitioning marker is used to divide the query set into an optimization query set, a validation query set, and a test query set. The test query set does not participate in network parameter updates, training round selection, or gating threshold determination.
[0070] It should be noted that, in step S1 of this invention, similar to the aforementioned process, it is still necessary to record the unique identifier and category label of the candidate CAD model. The category label refers to the mechanical part represented by the candidate CAD model. The geometric structure data associated with the candidate CAD model includes STEP or other B-Rep data, triangular mesh data, or geometric topology data exported from the CAD system. For the generation of multi-view images, the candidate CAD model is first normalized in position and scale so that its geometric center coincides with the origin of the preset coordinate system and is within a unified scale range. Then, 12 fixed viewing directions are set around the candidate CAD model, and virtual cameras are established facing its geometric center. The candidate CAD model is geometrically projected onto a two-dimensional imaging plane, and 12 views are generated as multi-view images under unified imaging size, background, and rendering parameters. These views collectively represent a library entry and are not considered as multiple independent candidates.
[0071] It should be noted that in step S1 of this invention, when the category label of the query image is the same as the category label of the candidate CAD model, the candidate CAD model is marked as a relevant candidate, i.e., related to the query image; when the category label of the query image is different from the category label of the candidate CAD model, the candidate CAD model is marked as an irrelevant candidate, i.e., unrelated to the query image. As can be seen from the above process, the same query image can correspond to multiple relevant candidates within the same category, and the above-mentioned correlation relationship is a semantic correspondence in retrieval, not requiring the query image and a certain view of the candidate CAD model to have the same camera pose.
[0072] It should be noted that step S1 of this invention also includes preprocessing the multi-view images of the two-dimensional query image and the candidate CAD model. During the model training phase, the image is converted to a three-channel RGB image, the shorter side of the image is scaled to 256 pixels, and then randomly cropped to 224×224 pixels. A random rotation of no more than 8 degrees, a translation of no more than 4% of the image size in each direction, and a scaling factor of 0.95 to 1.05 times are applied. During the retrieval and testing phase, the shorter side of the image is scaled to 256 pixels, and then the image is cropped to 224×224 pixels at the center. The cropped image is normalized according to the RGB three-channel means of 0.485, 0.456, and 0.406, and the standard deviations of 0.229, 0.224, and 0.225.
[0073] S2. Form complete and damaged queries from the same query image, and perform multi-positive sample contrast learning on the candidate CAD models related to the query image for both complete and damaged queries. The joint retrieval loss, the representation consistency loss before and after occlusion loss, and the category supervision loss are weighted and summed to form the total loss. Based on minimizing the total loss, the shared visual network and the category prediction layer are jointly trained.
[0074] It should be noted that, in step S2 of this invention, the specific process of jointly training the shared visual network and the category prediction layer is as follows:
[0075] S21. First, obtain a query image and use it as the complete query. Then, scale the query image to a preset size and randomly set a white rectangular occlusion area within the scaled image, so that the occluded image is used as the damaged query.
[0076] In this embodiment S21, the width and height of the white rectangular occlusion area are randomly selected within the range of 96 to 128 pixels, and the position of the upper left corner is randomly determined within a range that ensures the occlusion area does not exceed the image boundary. Complete queries and damaged queries are respectively cropped, enhanced, and normalized according to the above preprocessing steps.
[0077] S22. The multi-view images of candidate CAD models related to a query image are taken as relevant multi-view images. A quadruple consisting of the complete query, the damaged query, a view from the relevant multi-view images, and the category label of the query image corresponding to the same query image is used as a sample in the training batch. A shared visual network with modal identity encodes the complete query, the damaged query, and the single view in the sample, respectively, to obtain the corresponding complete query visual vector. Damaged query visual vector Visual vectors of a single view The visual vector of a single view is used as the CAD visual vector of the candidate CAD model. ;in This refers to the sample index in the training batch. This represents the number of samples in the training batch. For view indexes.
[0078] It should be noted that in step S2 of the present invention, as Figure 2 As shown, the processing flow of the shared visual network is as follows: The complete query, damaged query, or single view is taken as the input of the shared visual network. First, it is processed by the shared visual backbone to obtain shape features, and then processed by the shared projection network to obtain projected visual features. These features are then input into the shared residual adapter to obtain adapted features and perform L2 normalization, thereby obtaining the visual vectors of the complete query, damaged query, or single view.
[0079] Furthermore, in this embodiment, ResNet-18 is used as a shared visual backbone to extract shape features common to complete queries, damaged queries, and a single view.
[0080] Furthermore, the shared projection network is composed of a first linear layer, a GELU nonlinear activation layer, and a second linear layer cascaded in sequence, which is used to map the shape feature to a unified low-dimensional retrieval space, thereby obtaining 256-dimensional projected visual features.
[0081] Furthermore, the shared residual adapter includes layer normalization. 64-dimensional reduced linear layer, GELU nonlinear activation layer The calculation of the linear layer of increasing dimension is as follows:
[0082]
[0083] In the formula, The projected visual features output by the shared projection network; For learnable residual scaling parameters; , and , These are the parameters for the dimension-reducing linear layer and the dimension-increasing linear layer, respectively. It is the sigmoid activation function; Modal indexes for the input of a shared visual network; when using a shared visual network to encode complete or damaged queries, To query modal identity vectors When using a shared visual network to encode the view, CAD modal identity vector ; Features after adaptation.
[0084] In this embodiment, It is a 256-dimensional learnable parameter vector, shared by all complete queries or damaged queries. Shared by all views The parameters of the shared visual network are jointly updated by the total loss. Except for the modality identity vectors used, the query encoding path and the view encoding path use the same network parameters. A shared residual adapter is used to compensate for modal differences between the 2D query and view with controlled residual correction while preserving the shared retrieval space. It is important to note that the modality identity vector acts on the layer-normalized projected visual features, but it does not participate in the layer normalization process itself.
[0085] In this embodiment, after L2 normalization of the adapted features, when the shared visual network takes a complete query as input, it can obtain a complete query visual vector; when the shared visual network takes a damaged query as input, it can obtain a damaged query visual vector; and when the shared visual network takes a single view as input, it can obtain a single view visual vector. Here, L2 normalization is used to eliminate the influence of vector magnitude on similarity calculation, making the subsequent vector inner product equivalent to cosine similarity.
[0086] In the training phase of this embodiment, instead of performing multi-view fusion of candidate CAD models, a view is selected from relevant multi-view images of the same query category as the current query, and the visual vector of that view is used as the CAD visual vector for this training. When resampling the same query image, different views are selected so that different viewing directions can participate in query-CAD alignment, while avoiding the repeated encoding of all views of the same candidate CAD model during each parameter update.
[0087] S23. Calculate the representation consistency loss before and after occlusion based on the complete query visual vector and the damaged query visual vector. Calculate the category supervision loss using the category prediction layer. Calculate the joint retrieval loss based on two similarity matrices. The first similarity matrix is used to represent the similarity between the complete query visual vector and the CAD visual vector, and the second similarity matrix is used to represent the similarity between the damaged query visual vector and the CAD visual vector. Calculate the total loss to update the shared visual network and the category prediction layer.
[0088] Furthermore, the specific calculation method for the joint retrieval loss is as follows:
[0089] S231. Calculate the first A complete query visual vector With the CAD visual vectors Similarity between This forms the first similarity matrix. Calculate the first A damaged query visual vector With the CAD visual vectors Similarity between This forms the second similarity matrix. ;in, As a learnable temperature scaling factor, this embodiment initializes it to... And its maximum value is limited to 100 to adjust the concentration of the similarity distribution; This is a transpose.
[0090] S232. Construct a multi-positive sample label matrix based on the class labels of each sample in the training batch. When the first The query image of the first sample and the first When the category labels of individual views of samples are the same, the position in the multi-positive sample label matrix value at ,otherwise .
[0091] S233. Calculate the multi-positive sample loss for a complete query in the CAD direction. Damaged queries reveal multiple positive sample loss in the CAD direction. Multiple positive sample loss from CAD to complete query direction CAD to damaged query direction multi-positive sample loss This allows us to calculate the retrieval loss for the complete query. and retrieval loss of damaged queries The average of the two retrieval losses is used to construct the joint retrieval loss. :
[0092]
[0093]
[0094]
[0095]
[0096]
[0097]
[0098]
[0099] in, Position in the multi-positive sample label matrix The value at; For the first The complete query visual vector and the first Similarity between CAD visual vectors; For the first A damaged query visual vector With the Similarity between CAD visual vectors.
[0100] Furthermore, the occlusion before and after represents a loss of consistency. The specific calculation method is as follows:
[0101]
[0102] Furthermore, the category-based supervision loss The specific calculation method is as follows:
[0103]
[0104] in, Cross-entropy; , , This includes the complete query visual vector, the damaged query visual vector, and the CAD visual vector for all samples in the current training batch. Category labels; This is the category prediction layer.
[0105] In this embodiment, the total number of categories is assumed to be The category prediction layer is derived from a 256-dimensional retrieval vector. A linear mapping of category scores is used. Cross-entropy is calculated for the complete query visual vector, the damaged query visual vector, and the CAD visual vector, and the average of the three is taken to construct the above-mentioned category supervision loss. This loss is used to ensure that the retrieval vector retains category discrimination information and enables the category prediction layer to output the category probabilities of the query and candidate CAD models in step S3.
[0106] Following the above calculation process, this embodiment weights and sums the joint retrieval loss, category supervision loss, and representation consistency loss before and after occlusion to obtain the total loss. :
[0107]
[0108] in, , As the weight, this embodiment takes , Backpropagation is performed based on the total loss to jointly update the shared visual network (including the query modality identity vector, CAD modality identity vector, residual scaling parameter, and temperature scaling factor) and the category prediction layer. It is important to note that the representation consistency loss before and after occlusion is not used separately; the complete query and the impaired query are still aligned with a single view of the candidate CAD model using the joint retrieval loss.
[0109] S3. Based on the trained shared visual network and category prediction layer, calculate the visual-category joint basic score between the query image to be retrieved and each candidate CAD model in the model library, and sort the candidate CAD models in descending order according to the visual-category joint basic score to form the basic ranking of candidate CAD models, which is used to determine whether to enable source semantic graph correction.
[0110] It should be noted that in step S3 of this invention, the image to be retrieved is... With a candidate CAD model Visual-Category Joint Basic Score Specifically:
[0111]
[0112] in, The query visual vector is obtained by encoding the query image to be retrieved from a pre-trained shared visual network; The CAD visual vector of the candidate CAD model is obtained by encoding all views in the multi-view image through a trained shared visual network and fusing the visual vectors of each view. The class probability vector is obtained by processing the visual vector of the query through a trained class prediction layer and the Softmax function; The class probability vector is obtained by processing the CAD visual vector through a trained class prediction layer and the Softmax function. The non-negative category semantic weights are taken in this embodiment. .
[0113] In this embodiment, the first item For visual shape similarity, the second term This represents the semantic compatibility of the category. Because... and Both are L2 normalized, and their inner product is... Equivalent to cosine similarity, it is used to measure the visual shape similarity between the query image to be retrieved and the candidate CAD model. This indicates the degree of compatibility between the category probability distribution of the query image and the candidate CAD model, and is used to supplement the category discrimination information when relying solely on visual shape similarity.
[0114] Furthermore, in this embodiment, to complete the above calculation using a single vector inner product, the... and By attaching these vectors to the query visual vector and the CAD visual vector respectively, we obtain the visual-category joint vector of the query image to be retrieved. Visual-class joint vector of candidate CAD models ,at this time .
[0115] It should also be noted that, unlike the training phase, the retrieval and testing phases require multi-view fusion. This involves encoding all views of the same candidate CAD model separately, averaging the visual vectors obtained from each view, and then performing L2 normalization again. The CAD visual vector of the candidate CAD model is obtained. :
[0116]
[0117] in, In this embodiment, the number of views for the candidate CAD model is... ; This represents L2 normalization. Through the above multi-view fusion, the impact of a single viewing direction on the CAD visual vector can be reduced, allowing the CAD visual vector to simultaneously contain shape information from different directions. After the above processing, each complete query and damaged query corresponds to a query visual vector, and each candidate CAD model corresponds to a CAD visual vector that fuses multi-view information. Both the query visual vector and the CAD visual vector are 256-dimensional unit vectors.
[0118] S4. The difference between the two highest visual-category joint basic scores is used as the ranking interval for the query image to be retrieved: If the ranking interval reaches the preset gating threshold, the visual-category joint basic score is used as the final retrieval score of the candidate CAD model; if the ranking interval does not reach the preset gating threshold, the query structure vector is obtained based on the query visual vector, and the CAD model structure vector is obtained based on the constructed source semantic graph. The similarity between the query structure vector and the CAD model structure vector is calculated and used as the structural similarity. The structural similarity is weighted with bounded structural residual weights to form a correction term, which is used to correct the visual-category joint basic score. The corrected score is used as the final retrieval score of the candidate CAD model.
[0119] It should be noted that in step S4 of this invention, the candidate CAD models are sorted in descending order according to the visual-category joint basic score obtained in step S3, and the first candidate CAD model and the second candidate CAD model are denoted as follows: and The visual-class joint basic scores corresponding to the two candidate CAD models are denoted as follows: and The visual-category joint basic score difference between the two is defined as the difference between the visual and categorical scores of the query image to be retrieved. sorting interval :
[0120]
[0121] The larger the sorting interval, the clearer the visual-class scoring advantage of the first candidate CAD model over the second candidate CAD model; the smaller the sorting interval, the more likely the queried image to generate sorting ambiguity in the current model library.
[0122] After calculating the sorting interval, the present invention further determines whether the sorting interval of the current query image to be retrieved reaches the gating threshold. This embodiment first uses a validation query set to determine the gating threshold. Specifically, in this embodiment, the sorting interval of each query image in the verification query set is calculated and arranged in ascending order, and the median is taken as the gating threshold. Once the gating threshold is determined, it is fixed for both testing and actual retrieval and is not adjusted based on the retrieval results of the test query set.
[0123] After determining the gating threshold, the present invention further determines the retrieval path:
[0124] When the sorting interval is greater than or equal to the gating threshold, it is considered that the sorting interval has reached the preset gating threshold. At this time, the query image to be retrieved is judged as a high-confidence query, the structural similarity calculation between the query structure vector and the CAD model structure vector is skipped, and the final retrieval score of all candidate CAD models is set. The visual-category joint basic score, i.e. Therefore, the high-confidence retrieval path not only keeps the top-ranked candidate CAD model unchanged, but also keeps the visual-class joint basic score and basic ranking of all candidate CAD models unchanged.
[0125] When the sorting interval is less than the threshold, it is considered that the sorting interval has not reached the preset threshold. At this time, the query image to be retrieved is judged as a low-confidence query, and the query structure vector is read. and CAD model structure vector And calculate structural similarity. ,because and Both are L2 normalized, and their inner product is equivalent to their cosine similarity. In actual calculations, only the query structure vector corresponding to the low-confidence query is multiplied by the CAD model structure vector; the query structure vector for the high-confidence query is not constructed. Furthermore, bounded structural residual weights need to be calculated based on the sorting interval of the query images to be retrieved. The calculated structural similarity scores are then weighted to determine the corrected score, which is used as the final retrieval score. .
[0126] The following sections explain the specific construction methods of the query structure vector, CAD model structure vector, and bounded structure residual weights.
[0127] Furthermore, the query structure vector and the CAD model structure vector are obtained in the following way:
[0128] S41. First, construct the source semantic graph of the candidate CAD model. The source semantic graph includes a node feature matrix. Node adjacency matrix Fragment feature matrix and source association matrix , The number of nodes; This refers to the number of surface fragments or proxy fragments.
[0129] Further, in step S41 of this embodiment, for candidate CAD models that can obtain B-Rep geometric surfaces and their source associations with triangular facets, this embodiment uses the B-Rep geometric surface as a node in the source semantic graph, and establishes node adjacency relationships when two B-Rep geometric surfaces share a boundary. The five-dimensional features of each node consist of the normalized area, normalized three-dimensional centroid coordinates, and surface type encoding of the B-Rep geometric surface. The individual triangular facets obtained after discretizing the B-Rep geometric surface are used as surface fragments. Each row of the fragment feature matrix represents the four-dimensional features of a surface fragment, consisting of the three components of its unit normal vector and the plane offset. When the... The surface element is composed of the first When the B-Rep geometric surfaces are discretized, let the position of the source correlation matrix be... value at Otherwise Therefore, a node can be associated with multiple surface fragments, and each surface fragment is associated with only one node.
[0130] Further, in step S41 of this embodiment, for candidate CAD models that can only obtain triangular meshes but not the source association between B-Rep geometric surfaces and triangular facets, the triangular meshes are first translated to a point with the center of the candidate CAD model's bounding box as the origin, and then scaled according to the diagonal length of the bounding box. After removing degenerate triangular facets, the normalized three-dimensional centroid coordinates, unit normal vector, and plane offset of each remaining triangular facet are calculated, and the unit normal vector is made to point away from the center of the bounding box. The three components of the unit normal vector are multiplied by the first quantization factor and rounded, and the plane offset is multiplied by the second quantization factor and rounded. Triangular facets with the same four quantization results are grouped into the same proxy plane group. The areas of the triangular facets in each proxy plane group are summed to obtain the area of the proxy plane group. A predetermined number of proxy plane groups are retained in descending order of size, and each retained proxy plane group is used as a proxy entity to construct a node in the source semantic graph. The normalized 3D centroid coordinates of each triangular facet within each proxy plane group are weighted and averaged to obtain the corresponding proxy entity centroid. The four nearest neighbors of each node are determined based on the Euclidean distance between the centroids of each proxy entity. When any node belongs to the four nearest neighbors of another node, a bidirectional adjacency relationship is established between them. The five-dimensional features of each node consist of the proportion of the area of the corresponding proxy plane group to the total area of all retained proxy plane groups, the three coordinates of the proxy entity centroid, and the surface type code, which is uniformly set to 0. The unit normal vectors of each triangular facet within the proxy plane group are weighted and averaged, and the weighted average result is normalized again to obtain a new unit normal vector. And according to the centroid of the agent entity Calculate the new plane offset Each row of the fragment feature matrix represents a four-dimensional feature of a proxy fragment, consisting of the three components of the new unit normal vector and the new plane offset, approximating the four-dimensional feature of the surface fragment, and the source correlation matrix is the identity matrix.
[0131] Specifically, in the above process, the first quantization factor is 4, the second quantization factor is 32, and a total of 128 proxy plane groups are retained. Each proxy plane group generates one node and one proxy fragment, which have a one-to-one correspondence. Therefore, when only triangular meshes can be obtained and the source association between B-Rep geometry and triangular facets cannot be obtained, the number of nodes is equal to the number of proxy fragments, and the source association matrix is the identity matrix.
[0132] It is important to note that while both data formats employ the same source semantic graph data structure, the granularity of surface fragments differs: for candidate CAD models with native B-Rep geometry-triangular facet associations, the surface fragment is a single triangular facet obtained by discretizing the B-Rep geometry; for candidate CAD models containing only triangular meshes, surrogate fragments obtained by aggregating multiple triangular facets within the same surrogate plane group are used to approximate the surface fragments. Both data formats output the node feature matrix, node adjacency matrix, fragment feature matrix, and source association matrix.
[0133] S42. Input the source semantic graph into the trained source semantic graph encoder. First, the node encoder encodes the node information to obtain the node representation, and the fragment encoder encodes the fragment information to obtain the fragment representation. Then, the source constraint decoder performs constrained information fusion on the node representation and fragment representation based on the source association between fragments and nodes to obtain the initial CAD model structure vector.
[0134] Further, in step S42 of this embodiment, as follows Figure 3 As shown, the node encoder and fragment encoder are used to encode node information and fragment information respectively as follows:
[0135] For each node, each node feature in the node feature matrix is first converted into a 128-dimensional hidden feature using a linear mapping. For the node adjacency matrix By adding self-joins and normalizing by row, we obtain a normalized adjacency matrix. In the first In the graph message passing layer, the first Each node aggregates its own features and the features of its neighboring nodes to form neighborhood information. The hidden features and neighborhood information are then linearly mapped, and the retention ratio of each is determined dimension-by-dimensionally using a learnable Sigmoid gating vector. The gating fusion result is summed with the input of this layer and normalized. Then, after feedforward transformation, a second summing of residuals, and layer normalization, the output of the graph message passing layer is obtained. In this embodiment, two graph message passing layers are concatenated, and the hidden features of each node are linearly mapped to a 128-dimensional structure space and L2 normalized to obtain the node representation. .
[0136] For each fragment, the four-dimensional features of each fragment are first mapped using a linear mapping. The features are converted into 128-dimensional hidden features, then passed through two feedforward coding layers consisting of layer normalization, linear transformation, GELU activation, and linear transformation. Finally, the fragment representation is obtained through linear mapping and L2 normalization. Each fragment uses the same network parameters and is encoded separately; no cross-fragment message passing occurs at this stage.
[0137] Further, in step S42 of this embodiment, as follows Figure 3 As shown, the specific steps for fusing node representations and fragment representations using a source-constrained decoder are: [The steps involve] fusing the node representations... By querying the projection matrix , to represent the fragment Projected through the key matrix respectively Sum projection matrix Calculate the first one according to the following formula The node is the first Attention weight of each fragment , thus calculating the first fragment context of each node and source enhancement And by enhancing the representation of the source of each node, the initial CAD model structure vector is calculated. :
[0138]
[0139]
[0140]
[0141]
[0142] in, Position in the source correlation matrix The value at; The dimension of the structure space; Indicates splicing; , For parameters.
[0143] Furthermore, in step S42 of this embodiment, before Softmax normalization, the source correlation matrix is used. Fragments that have no source association with the current node are excluded. In this embodiment, For each valid node, the source association matrix provides at least one fragment that can participate in the aggregation. Therefore, this embodiment obtains the initial CAD model structure vector by performing mean aggregation and L2 normalization on the source augmentation representations of all valid nodes. In data with native B-Rep geometry-triangle patch associations, a node can aggregate multiple surface fragments obtained by discretizing the B-Rep geometry according to different learned weights; while in data without native B-Rep geometry-triangle patch associations, the source association matrix is an identity matrix, and each node only fuses its one-to-one corresponding surrogate fragment.
[0144] Further, in step S42 of this embodiment, the training process of the source semantic graph encoder is as follows: Before training, a starting node is randomly determined from the effective nodes, and then the process is expanded layer by layer along the node adjacency relationship until a predetermined number of nodes are selected and occluded to form occluded nodes; when the number of nodes in the connected component where the starting node is located is insufficient, it is supplemented from the remaining effective nodes. After the node features have completed linear mapping, the hidden features of the selected nodes are replaced with learnable 128-dimensional occlusion vectors, and the unoccluded nodes continue to provide topological context. During training, for each occluded node, the number of surface fragments obtained by discretizing the B-Rep geometry corresponding to the node is first counted, and then an equal number of surface fragments whose plane is closest to the centroid of the node are selected from all the effective surface fragments of the current CAD model. The source-constrained decoder only gathers information from the selected surface fragments to generate source-enhanced representations of occluded nodes, which are then input into the geometric attribute prediction head, the solid manufacturing feature prediction head, and the surface type prediction head, respectively. The geometric attribute prediction head is used to predict the normalized area and the normalized 3D centroid coordinates. The solid manufacturing feature prediction head outputs the prediction probabilities of 25 solid manufacturing feature categories based on the additional solid manufacturing feature category labels provided in the training set. The surface type prediction head outputs the prediction probabilities of 12 surface types.
[0145] Therefore, the loss used by the source semantic graph encoder during training... for:
[0146]
[0147] in, To mask the mean square error of the geometric properties of the nodes, and The cross-entropy losses are respectively for entity manufacturing feature categories and surface types. As the weight. In this embodiment, the node occlusion ratio is set to 35%, and taken as... , After training, the node encoder, fragment encoder, and source constraint decoder are retained, while the three prediction heads used only for training are removed.
[0148] S43. The trained structural residual adaptation network performs residual mapping on the initial CAD model structure vector. First, the initial CAD model structure vector is layer normalized, and then processed through the third and fourth linear layers to obtain a residual correction vector with the same dimension as the initial CAD model structure vector. After adjusting the initial CAD model structure vector using the residual correction vector, the final CAD model structure vector is obtained. :
[0149]
[0150]
[0151] in, A learnable scalar shared by all candidate CAD models; The parameters for the third linear layer range from 128 to 256 dimensions; The parameters for the fourth linear layer range from 256 to 128 dimensions.
[0152] S44. The trained query structure mapping network first linearly maps the query visual vector into a vector (128-dimensional vector), which is then activated by GELU and passed through a 128-dimensional linear layer, followed by L2 normalization to obtain the 128-dimensional query structure vector. .
[0153] Furthermore, in steps S43-S44 of this embodiment, during the training of the structural residual adaptation network and the query structure mapping network, the structural residual adaptation network and the query structure mapping network are combined into a structural retrieval adapter, thereby jointly optimizing the two. In this embodiment, the learnable scalar is initialized to -2, and through... The residual scaling factor is limited to between 0 and 1. The parameters of the fourth linear layer are initialized to zero, so the residual correction vector is zero at the start of the adaptation; as training progresses, the structural residual adaptation network gradually learns structural representation adjustments oriented towards the retrieval task.
[0154] The structural retrieval adapter is trained using a four-term loss. The first term is the multi-positive-sample retrieval loss for the direction of the CAD model being retrieved. The first term takes the final retrieval score between the query image and all candidate CAD models as input, and considers all candidate CAD models with the same category label as the query image as positive samples. The second term is the structural category loss. The similarity between the structural vectors of the CAD models is divided by a fixed temperature of 0.07, and candidate CAD models with the same category label are used as positive samples to calculate the multi-positive sample loss. The third term is the relation preservation loss, for the... For each candidate CAD model, its corresponding visual-class joint vector is used. ,set up This is a matrix composed of all the structural vectors of the CAD model, arranged by rows. For each row-wise matrix of the visual-category joint vectors of all candidate CAD models, calculate the structural relation matrix. Visual-Category Relationship Matrix Let the number of candidate CAD models be... The relationship remains intact despite losses. for This loss constraint preserves the relative relationships between models represented by the visual-class joint vector in the CAD model structure vector, without requiring element-wise equality between the two vectors. The fourth term is the hard sample ranking loss, which, for the first sample in a batch... For each query image, the lowest final retrieval score among all relevant candidate CAD models is recorded as . The highest final retrieval score among all irrelevant candidate CAD models is recorded as . The loss for sorting difficult samples for:
[0155]
[0156] in, As a preset hyperparameter, this embodiment takes... The weighted sum of the four losses yields the loss for training the structure retrieval adapter. :
[0157]
[0158] Furthermore, during the training of the structure retrieval adapter, the shared visual network and the trained source semantic graph encoder are fixed, while the query structure mapping network and structure residual adaptation network in the structure retrieval adapter are updated. The model round is selected based on the validation set retrieval results; test queries are not involved in parameter updates, model round selection, or path selection threshold determination. The CAD model structure vectors are pre-calculated and saved during the library creation phase, and the source semantic graph encoding and structure residual adaptation processes for candidate CAD models are not repeated during retrieval.
[0159] Furthermore, the bounded structural residual weights The calculation method is as follows:
[0160]
[0161]
[0162] in, As the upper limit of the structural residual amplitude, this embodiment will Initialize to 0.05; As a preset upper limit for structural residual weights, this embodiment takes... =0.5; , and These are learnable parameters; This is the weight variation coefficient, used to control the steepness of the change in the weight of the bounded structure residuals with respect to the sorting interval; To prevent The positive number set by taking zero, in this embodiment is taken as... = ; It is the sigmoid activation function; This is the softplus activation function; The sorting interval for the images to be retrieved.
[0163] As can be seen from the above formula, the smaller the sorting interval, the larger the weight of the bounded structure residual; as the sorting interval increases, the weight of the bounded structure residual decreases accordingly. It should be noted that the gating threshold in step S4 is used to determine whether to calculate structural similarity; the learnable parameters... Only after entering the low-confidence retrieval path is the structural residual weight continuously adjusted according to the sorting interval; the two have different functions.
[0164] S5. Arrange all candidate CAD models in descending order of their final search scores to obtain the final ranking and complete the search.
[0165] It should be noted that in step S5 of this invention, after obtaining the final retrieval score, the candidate CAD models with higher final retrieval scores can be considered as CAD models similar to the image to be retrieved. In mechanical design, tooling design, and non-standard automation design processes, when a user draws a hand-drawn sketch, outline, or engineering line drawing of a bearing, it can be used as the query image. The method of this invention can then be used to retrieve the most similar CAD model from the model library, allowing the user to find similar parts and reuse historical designs. For example, the user can directly use the retrieved CAD model of the bearing as a 3D base, and by modifying key dimensions (such as aspect ratio, bore diameter, and center distance), quickly generate the bearing's engineering drawings and BOM, thus avoiding modeling from scratch.
[0166] To better demonstrate the specific implementation and technical effects of the present invention, the CAD model retrieval method based on robust visual representation and source semantic graph shown in steps S1 to S5 of the above preferred implementation is applied to a specific example.
[0167] Example
[0168] The specific implementation process of the CAD model retrieval method based on robust visual representation and source semantic graph used in this embodiment is as described above and will not be repeated here.
[0169] To verify the effectiveness of modal identity vector adaptation, this embodiment conducts a parameter comparison experiment using 624 training queries and 176 test queries from the publicly available CADSketchNet-B dataset. From the training queries, 555 optimized queries and 69 validation queries were categorized, while the test queries were not included in training or round selection. Both the shared conditional comparison and the modal conditional method contain 11,615,404 trainable parameters. The former uses the same modal identity vector in both the query path and the CAD path, while the latter uses the query modal identity vector and the CAD modal identity vector, respectively. The shared conditional comparison achieved mAP, Top-10 similarity ratio, and Hit@1 of 0.760968, 0.655114, and 0.693182, respectively; after introducing the modal identity vector, these three metrics reached 0.855245, 0.759091, and 0.812500, respectively.
[0170] To further verify the retrieval performance of incomplete sketches, this embodiment merges images with identical content and duplicate grids with identical geometry in the publicly available data, resulting in 350 optimized queries, 76 validation queries, 77 test queries, and 566 candidate CAD geometry groups. No identical image content exists between the three query partitions. It should be noted that the merging operation is only used to prevent identical content from entering different partitions simultaneously; it does not change the definition of the retrieval task, which uses candidates of the same category as relevant results. All models complete rounds of selection on the validation set before calculating test results.
[0171] This experiment uses the base ordering formed in steps S1 to S3 to independently verify the role of impaired queries in joint training. The method of this invention and the control method are initialized using the same modal conditional model parameters, maintaining consistency in network structure, number of trainable parameters, number of training rounds, number of parameter updates, category supervision, and base scoring method. Both methods select the optimal round based on the same set of impaired verification queries; the control method does not generate impaired training queries, while the method of this invention performs joint training in step S2. The test results are shown in Table 1.
[0172] Table 1. Search results of the method of the present invention and the method of comparison with the same structure
[0173]
[0174] On the same 77 damaged test queries, the method of this invention improved the mAP by 0.219838 compared to the control method, with a 95% confidence interval of [0.149299, 0.294279] obtained by query-by-query resampling; the proportion of Top-10 similar queries improved by 0.222078, with a 95% confidence interval of [0.146753, 0.301299]; and the Hit@1 score improved by 0.259740, with a 95% confidence interval of [0.155844, 0.363636]. These results demonstrate that jointly training complete and damaged queries directly in the CAD model retrieval space can significantly improve the retrieval performance for locally missing sketches.
[0175] To verify the effect of fragment-node association on structural vector learning, occluded node recovery experiments were conducted on the MFCAD++ public dataset. Each experiment used 4000 models for parameter optimization, 500 models for validation, and another 500 models for testing; the occlusion ratio of connected nodes, occluded nodes, and various training parameters remained consistent under different conditions. Based on the fragment features received by the source-constrained decoder, the experiment was set to four conditions: the first used only the node feature matrix and node adjacency matrix, without inputting the fragment feature matrix and source association matrix; the second randomly selected an equal number of fragments based on the number of source fragments for each occluded node; the third selected an equal number of geometric neighborhood fragments based on planar distance; and the fourth selected source fragments based on the native fragment-B-Rep surface association. The mean square error of occluded node geometric parameter reconstruction was calculated for each of the 500 test models. The average error of a single training session was calculated with equal weights for the test models, and then the results of three independent training sessions were averaged. The results are shown in Table 2.
[0176] Table 2. Reconstruction error of geometric parameters of occluded nodes under different conditions
[0177]
[0178] Compared to the other three conditions, the native source fragment condition reduced the average reconstruction error by 0.007948, 0.011222, and 0.003871, respectively. Further replacing the correct source fragments with an equal number of fragments from other models increased the average reconstruction error by 0.022129. It is important to note that this experiment examines the structural representation recovery capability of occluded nodes; the results show that the source-constrained decoder can utilize the correspondence between fragments and B-Rep surfaces to provide source information for the CAD model's structural vectors that is distinct from node adjacency and geometric neighborhood.
[0179] In addition, this embodiment also verifies the retrieval path based on the gating threshold in step S4 of the CADSketchNet-B damaged query. Specifically, this embodiment sorts the sorting interval of the top two basic scores of the 76 damaged verification queries in ascending order and takes the median of 0.007993 as the gating threshold. Of the 77 damaged test queries, 43 reached the gating threshold and were directly sorted using the basic ranking without structural correction, while the remaining 34 damaged test queries underwent structural correction, resulting in a structural correction skipping rate of 55.84%.
[0180] It is understood that the CAD model retrieval method based on robust visual representation and source semantic graph described in S1-S5 above can essentially be implemented by a computer program. Therefore, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer program product corresponding to the CAD model retrieval method based on robust visual representation and source semantic graph provided in the above embodiments. This product includes a computer program / instruction that, when executed by a processor, can implement the CAD model retrieval method based on robust visual representation and source semantic graph as described in the above embodiments.
[0181] Similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer electronic device corresponding to the CAD model retrieval method based on robust visual representation and source semantic graph provided in the above embodiments, such as... Figure 4 As shown, it includes a memory and a processor;
[0182] The memory is used to store computer programs;
[0183] The processor is configured to implement the CAD model retrieval method based on robust visual representation and source semantic graph in the above embodiments when executing the computer program.
[0184] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.
[0185] Therefore, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer-readable storage medium corresponding to the CAD model retrieval method based on robust visual representation and source semantic graph provided in the above embodiments. The storage medium stores a computer program, which, when executed by a processor, can realize the CAD model retrieval method based on robust visual representation and source semantic graph in the above embodiments.
[0186] Specifically, in the computer-readable storage medium of the above three embodiments, the stored computer program is executed by a processor, which can perform the aforementioned steps S1 to S5.
[0187] It is understood that the aforementioned storage media may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Furthermore, the storage media may also be various media capable of storing program code, such as USB flash drives, external hard drives, magnetic disks, or optical discs.
[0188] It is understood that the processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0189] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained through equivalent substitution or transformation fall within the protection scope of the present invention.
Claims
1. A CAD model retrieval method based on robust visual representation and source semantic graph, characterized in that, Includes the following steps: S1. Obtain a query set consisting of two-dimensional query images and a model library consisting of candidate CAD models. Each candidate CAD model represents a three-dimensional model of a mechanical part. Each query image is used to describe a mechanical part. Multi-view images and geometric structure data are associated with each candidate CAD model. At the same time, candidate CAD models that are related to or unrelated to the query images are recorded. S2. Form complete query and damaged query from the same query image, and perform multi-positive sample contrast learning on the candidate CAD models related to the query image for complete query and damaged query respectively. The joint retrieval loss, the representation consistency loss before and after occlusion loss and the category supervision loss are weighted and summed to form the total loss. Based on minimizing the total loss, the shared visual network and the category prediction layer are jointly trained. S3. Based on the trained shared visual network and category prediction layer, calculate the visual-category joint basic score between the query image to be retrieved and each candidate CAD model in the model library, and sort the candidate CAD models in descending order according to the visual-category joint basic score, thereby forming the basic ranking of the candidate CAD models, which is used to determine whether to enable source semantic graph correction. S4. The difference between the two highest visual-category joint basic scores is used as the ranking interval for the query image to be retrieved: If the ranking interval reaches the preset gate threshold, the visual-category joint basic score is used as the final retrieval score of the candidate CAD model; if the ranking interval does not reach the preset gate threshold, the query structure vector is obtained based on the query visual vector, and the CAD model structure vector is obtained based on the constructed source semantic graph. The similarity between the query structure vector and the CAD model structure vector is calculated and used as the structural similarity. The structural similarity is weighted with bounded structural residual weights to form a correction term, which is used to correct the visual-category joint basic score. The corrected score is used as the final retrieval score of the candidate CAD model. S5. Arrange all candidate CAD models in descending order of their final search scores to obtain the final ranking and complete the search.
2. The CAD model retrieval method based on robust visual representation and source semantic graph as described in claim 1, characterized in that, In S2, the specific process of jointly training the shared visual network and category prediction layer is as follows: S21. First, obtain a query image and use it as the complete query. Then, scale the query image to a preset size and randomly set a white rectangular occlusion area within the scaled image. The occluded image will then be used as the damaged query. S22. The multi-view images of candidate CAD models related to a query image are taken as relevant multi-view images. A quadruple consisting of the complete query, the damaged query, a view from the relevant multi-view images, and the category label of the query image corresponding to the same query image is used as a sample in the training batch. A shared visual network with modal identity encodes the complete query, the damaged query, and the single view in the sample, respectively, to obtain the corresponding complete query visual vector. Damaged query visual vector Visual vectors of a single view The visual vector of a single view is used as the CAD visual vector of the candidate CAD model. ;in This refers to the sample index in the training batch. This represents the number of samples in the training batch. For view index; S23. Calculate the representation consistency loss before and after occlusion based on the complete query visual vector and the damaged query visual vector. Calculate the category supervision loss using the category prediction layer. Calculate the joint retrieval loss based on two similarity matrices. The first similarity matrix is used to represent the similarity between the complete query visual vector and the CAD visual vector, and the second similarity matrix is used to represent the similarity between the damaged query visual vector and the CAD visual vector. Calculate the total loss to update the shared visual network and the category prediction layer.
3. The CAD model retrieval method based on robust visual representation and source semantic graph as described in claim 2, characterized in that, In S22, the processing flow of the shared visual network is as follows: The complete query, damaged query, or single view is used as the input of the shared visual network. First, it is processed by the shared visual backbone to obtain shape features, and then processed by the shared projection network to obtain projected visual features. These features are then input into the shared residual adapter to obtain adapted features and perform L2 normalization, thereby obtaining the visual vectors of the complete query, damaged query, or single view.
4. The CAD model retrieval method based on robust visual representation and source semantic graph as described in claim 3, characterized in that, The shared projection network is composed of a first linear layer, a GELU nonlinear activation layer, and a second linear layer cascaded in sequence. It is used to map the shape feature to a unified low-dimensional retrieval space to obtain the projected visual feature. The shared residual adapter includes hierarchical normalization. Dimensionality reduction linear layer, GELU nonlinear activation layer The calculation of the linear layer of increasing dimensionality is as follows: ; In the formula, The projected visual features output by the shared projection network; For learnable residual scaling parameters; , and , These are the parameters for the dimension-reducing linear layer and the dimension-increasing linear layer, respectively. It is the sigmoid activation function; Modal indexes for the input of a shared visual network; when using a shared visual network to encode complete or damaged queries, To query modal identity vectors When using a shared visual network to encode the view, CAD modal identity vector ; Features after adaptation.
5. The CAD model retrieval method based on robust visual representation and source semantic graph as described in claim 2, characterized in that, The specific calculation method for the joint retrieval loss is as follows: S231. Calculate the first A complete query visual vector With the CAD visual vectors similarity between This forms the first similarity matrix. Calculate the first A damaged query visual vector With the CAD visual vectors similarity between This forms the second similarity matrix. ;in, A learnable temperature scaling factor; For transpose; S232. Construct a multi-positive sample label matrix based on the class labels of each sample in the training batch. When the first The query image of the first sample and the first When the category labels of individual views of samples are the same, the position in the multi-positive sample label matrix value at ,otherwise ; S233. Calculate the multi-positive sample loss for a complete query in the CAD direction. Damaged queries reveal multiple positive sample loss in the CAD direction. Multiple positive sample loss from CAD to complete query direction CAD to damaged query direction multi-positive sample loss This allows us to calculate the retrieval loss for the complete query. and retrieval loss of damaged queries The average of the two retrieval losses is used to construct the joint retrieval loss. : ; ; ; ; ; ; ; in, Position in the multi-positive sample label matrix The value at; For the first The complete query visual vector and the first Similarity between CAD visual vectors; For the first A damaged query visual vector With the Similarity between CAD visual vectors.
6. The CAD model retrieval method based on robust visual representation and source semantic graph as described in claim 2, characterized in that, In S3, the image to be retrieved With a candidate CAD model Visual-Category Joint Basic Score Specifically: ; in, The query visual vector is obtained by encoding the query image to be retrieved from a pre-trained shared visual network; The CAD visual vector of the candidate CAD model is obtained by encoding all views in the multi-view image through a trained shared visual network and fusing the visual vectors of each view. The class probability vector is obtained by processing the visual vector of the query through a trained class prediction layer and the Softmax function; The class probability vector is obtained by processing the CAD visual vector through a trained class prediction layer and the Softmax function. These are non-negative category semantic weights.
7. The CAD model retrieval method based on robust visual representation and source semantic graph as described in claim 5, characterized in that, In S4, the query structure vector and the CAD model structure vector are obtained in the following ways: S41. First, construct the source semantic graph of the candidate CAD model. The source semantic graph includes a node feature matrix. Node adjacency matrix Fragment feature matrix and source association matrix , The number of nodes; This refers to the number of surface fragments or proxy fragments. S42. Input the source semantic graph into the trained source semantic graph encoder. First, the node encoder encodes the node information to obtain the node representation, and the fragment encoder encodes the fragment information to obtain the fragment representation. Then, the source constraint decoder performs constrained information fusion on the node representation and fragment representation based on the source association between fragments and nodes to obtain the initial CAD model structure vector. S43. The trained structural residual adaptation network performs residual mapping on the initial CAD model structure vector. First, the initial CAD model structure vector is layer normalized, and then processed through the third and fourth linear layers to obtain a residual correction vector with the same dimension as the initial CAD model structure vector. After adjusting the initial CAD model structure vector using the residual correction vector, the final CAD model structure vector is obtained. ;in, A learnable scalar shared by all candidate CAD models; S44. The trained query structure mapping network first linearly maps the query visual vector into a vector, which is then activated by GELU and passed through a linear layer and normalized by L2 to obtain the query structure vector.
8. The CAD model retrieval method based on robust visual representation and source semantic graph as described in claim 7, characterized in that, In S41, for candidate CAD models where only triangular meshes are available but the source association between B-Rep geometry and triangular facets cannot be obtained, the triangular meshes are first translated to a point with the center of the candidate CAD model's bounding box as the origin, and then normalized according to the diagonal length of the bounding box. After removing degenerate triangular facets, the normalized 3D centroid coordinates, unit normal vector, and plane offset of each remaining triangular facet are calculated, and the unit normal vector is made to point away from the center of the bounding box. The three components of the unit normal vector are multiplied by the first quantization factor and rounded, and the plane offset is multiplied by the second quantization factor and rounded. Triangular facets with the same four quantization results are grouped into the same proxy plane group. The areas of the triangular facets in each proxy plane group are summed to obtain the area of the proxy plane group. A preset number of proxy plane groups are retained in descending order of their areas, and each retained proxy plane group is used as a proxy entity to construct a node in the source semantic graph. The normalized 3D centroid coordinates of each triangular facet in each proxy plane group are weighted and averaged to obtain the centroid of the corresponding proxy entity. The four nearest neighbors of each node are determined based on the Euclidean distance between the centroids of each proxy entity; when any node belongs to the four nearest neighbors of another node, a bidirectional adjacency relationship is established between them; the five-dimensional features of each node consist of the proportion of the area of the corresponding proxy plane group to the total area of all retained proxy plane groups, the three coordinates of the centroid of the proxy entity, and the surface type code, wherein the surface type code is uniformly set to 0; the unit normal vectors of each triangular facet within the proxy plane group are weighted and averaged, and the weighted average result is normalized again to obtain a new unit normal vector. And according to the centroid of the agent entity Calculate the new plane offset Each row of the fragment feature matrix represents a four-dimensional feature of a proxy fragment, consisting of the three components of the new unit normal vector and the new plane offset, approximating the four-dimensional feature of the surface fragment, and the source correlation matrix is the identity matrix.
9. The CAD model retrieval method based on robust visual representation and source semantic graph as described in claim 7, characterized in that, In S42, the specific steps for fusing node representations and fragment representations using the source-constrained decoder are: fusing the node representations... By querying the projection matrix , to represent the fragment Projected through the key matrix respectively Sum projection matrix Calculate the first one according to the following formula The node is the first Attention weight of each fragment , thus calculating the first fragment context of each node and source enhancement And by enhancing the representation of the source of each node, the initial CAD model structure vector is calculated. : ; ; ; ; in, Position in the source correlation matrix The value at; The dimension of the structure space; Indicates splicing; , For parameters.
10. The CAD model retrieval method based on robust visual representation and source semantic graph as described in claim 1, characterized in that, In S4, the bounded structural residual weights The calculation method is as follows: ; ; in, This represents the upper limit of the structural residual amplitude. The upper limit of the pre-set structural residual weight; , and These are learnable parameters; This is the weighting change coefficient; To prevent A positive number set to zero; It is the sigmoid activation function; This is the softplus activation function; The sorting interval for the images to be retrieved.
