A method and system for extracting features of a CAD model visual fusion feature of an optimal feature view angle
By constructing prior structural information and selecting the optimal perspective, and combining it with a visual language model to extract visual features from CAD models, the problems of disconnect between visual models and geometric semantics and multi-perspective redundancy are solved. This achieves high visibility and low redundancy semantic feature output, and improves the consistency of visual feature expression in CAD models.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHONGQING NUOYUAN IND SOFTWARE TECHNOLOGY CO LTD
- Filing Date
- 2026-02-28
- Publication Date
- 2026-06-16
AI Technical Summary
Existing technologies for visual feature extraction from CAD models suffer from problems such as a disconnect between the visual model and geometric semantics, a lack of perspective selection strategies, and severe redundancy of features from multiple perspectives. This results in key structures being occluded or weakened from some perspectives, affecting the effectiveness of subsequent applications.
By constructing structural prior information, generating an optimal set of viewpoints, performing rendering and quantitative evaluation, selecting the optimal viewpoint for image generation, and using a visual language model for semantic feature extraction, combined with structural prior information for constraint and fusion, a stable and consistent semantic feature output is achieved.
It achieves high visibility representation and low redundancy input of CAD model structural features, improves the discriminativeness and expression consistency of semantic features, and is applicable to various CAD models and rendering methods.
Smart Images
Figure CN121746876B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the fields of CAD and data processing technology, and in particular to a method and system for visual fusion feature extraction of CAD models with optimal feature perspective. Background Technology
[0002] With the widespread application of 3D CAD models in mechanical design, industrial manufacturing, and other fields, technologies for automatic feature extraction and understanding of CAD models are constantly developing. Existing technologies mainly include the following categories:
[0003] Feature extraction techniques based on geometric rules or topology analysis identify structural features such as holes, slots, and chamfers by parsing the B-Rep representation of CAD models and considering surface type, topological relationships, or geometric parameter rules. While this method offers high geometric accuracy, it heavily relies on manual rule design, making it difficult to adapt to complex and diverse part structures, and it cannot express the overall morphological semantics.
[0004] Based on 2D projection or multi-view rendering, some existing techniques render 3D CAD models into 2D images and extract features using traditional visual models or deep learning models. These methods typically generate rendered images by fixing the three views or uniformly sampling multiple viewpoints.
[0005] With the development of multimodal models, some deep learning-based visual feature learning methods have begun to attempt to use visual models to encode features of CAD rendered images. However, most of them still treat the models as "black box feature extractors" and lack targeted modeling of CAD structural semantics.
[0006] Existing technologies generally suffer from the following shortcomings: First, the visual model is disconnected from the geometric semantics of CAD. Traditional visual models are primarily trained on natural images and cannot effectively understand the structural semantics and engineering features inherent in CAD models. Second, viewpoint selection strategies are lacking or overly simplistic. Existing methods often employ fixed or uniformly sampled viewpoints, failing to consider the differences in the expressive power of different viewpoints on the structure of CAD models, resulting in key structures being occluded or weakened in some viewpoints. Third, multi-viewpoint features suffer from severe redundancy. The lack of effective filtering and weighting of viewpoint information leads to a large amount of redundant or invalid information in the features, affecting the effectiveness of subsequent applications. Summary of the Invention
[0007] This application provides a method and system for visual fusion feature extraction of CAD models with optimal feature perspective, achieving high visibility expression of structural features of CAD models, low redundant input, and stable and consistent semantic feature output.
[0008] This application provides a method for visual fusion feature extraction from CAD models based on optimal feature perspective, including:
[0009] Obtain the CAD model file to be processed, parse its boundary representation B-Rep, and construct structural prior information based on the B-Rep; the structural prior information includes at least: face type and geometric parameters determined based on the geometric type of the face in the B-Rep, a topological adjacency graph reflecting the face-edge-face adjacency relationship, and a key structure candidate set K automatically generated based on preset structural discrimination rules, wherein any structure k in the key structure candidate set K is bound to a set of face or edge IDs Fk;
[0010] Based on the geometric properties of the CAD model file or a preset sampling strategy, a candidate viewpoint set V containing multiple camera pose parameters is generated.
[0011] Based on each candidate viewpoint vi in the candidate viewpoint set V, the CAD model is rendered to generate a rendering output that includes at least an RGB image I and a face ID segmentation map S.
[0012] Based on the rendering output, a quantitative evaluation index is calculated, and the optimal viewpoint is selected from the candidate viewpoint set V to form the best viewpoint set A based on the calculated quantitative evaluation index.
[0013] Render the viewpoints corresponding to the optimal viewpoint set A to generate an input image set;
[0014] Based on the input image set, semantic features are extracted using the Visual Language Model (VLM). During the extraction process, the key structure candidate set K in the structural prior information is constructed into a text prompt P and injected into the VLM to constrain the structured semantic features and corresponding embedding vectors output by the VLM.
[0015] The embedded vectors and structured semantic features output from each perspective are fused to obtain semantic feature representations.
[0016] This application provides a CAD model visual fusion feature extraction system with optimal feature perspective, including a processor and a memory. The memory stores a computer program, which, when executed by the processor, implements the steps of the CAD model visual fusion feature extraction method with optimal feature perspective as described above.
[0017] This application embodiment achieves high visibility representation of CAD model structural features, low redundancy input, and stable and consistent semantic feature output through a closed-loop process of structure prior construction, face ID segmentation and rendering, quantified perspective scoring and redundancy removal selection, structure prior hint injection and structured output constraints, multi-perspective weighted fusion and consistency merging.
[0018] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description
[0019] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:
[0020] Figure 1 This is a schematic diagram of the basic process of the CAD model visual fusion feature extraction method, which is the best feature perspective for this application. Detailed Implementation
[0021] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
[0022] This application provides a method for visual fusion feature extraction from CAD models based on optimal feature perspective, such as... Figure 1 As shown, it includes the following steps:
[0023] In step S101, the CAD model file to be processed is obtained, its boundary representation B-Rep is parsed, and structural prior information is constructed based on the B-Rep. The structural prior information includes at least: face type and geometric parameters determined based on the geometric type of the faces in the B-Rep, a topological adjacency graph reflecting the face-edge-face adjacency relationship, and a key structure candidate set K automatically generated based on preset structural discrimination rules. Each structure k in the key structure candidate set K is bound to a set of face or edge IDs Fk. In a specific example, the face type and geometric parameters are: each face is determined to be a plane, cylindrical surface, conical surface, spherical surface, or freeform surface, and the corresponding geometric parameters are recorded; the topological adjacency graph is used for subsequent generation and statistics of structural candidates.
[0024] In some embodiments, the prior structural information also includes a summary of structural attributes (such as orientation, size range, cylinder axis, etc.) for subsequent visibility quantification and semantic cues construction.
[0025] In step S102, based on the geometric attributes of the CAD model file or a preset sampling strategy, a candidate viewpoint set V={v1…vi…vn} containing multiple camera pose parameters is generated, where each candidate viewpoint vi corresponds to camera pose parameters (observation direction, distance, field of view, etc.). The candidate viewpoints can be generated using uniform sampling, discrete sampling based on the principal axis, or other existing generation methods, and are not limited thereto.
[0026] In step S103, the CAD model is rendered based on each candidate viewpoint vi in the candidate viewpoint set V, generating a rendering output that includes at least an RGB image I and a face ID segmentation map S.
[0027] For each candidate viewpoint vi, render the CAD model and generate at least the following output:
[0028] 1) RGB rendering image Ii;
[0029] 2) Surface ID segmentation map Si: used to record the visible B-Rep surface ID corresponding to each pixel, and occluded or background pixels are recorded as 0.
[0030] In some specific examples, at least one of a depth map Di, a normal map Ni, and a contour edge map Ei is also generated to enhance the representation of occlusion relationships and structural boundaries.
[0031] In step S104, based on the rendering output, a quantitative evaluation index is calculated, and the optimal viewpoint is selected from the candidate viewpoint set V to form the best viewpoint set A. In some examples, the quantitative evaluation index includes: visible area ratio V(i), contour information C(i), key structure visibility S(i), and viewpoint redundancy R(i, A). A scoring function is constructed based on the quantitative evaluation index to perform the selection of the best viewpoint set.
[0032] In step S105, the viewpoints corresponding to the optimal viewpoint set A are rendered to generate an input image set.
[0033] In step S106, semantic features are extracted using the Visual Language Model (VLM) based on the input image set. During the extraction process, the key structure candidate set K in the structural prior information is constructed into a text prompt P and injected into the VLM to constrain the structured semantic features and corresponding embedding vectors output by the VLM.
[0034] In step S107, the embedded vectors and structured semantic features output from each perspective are fused to obtain the final semantic feature representation.
[0035] In some embodiments, the key structure visibility S(i) in the quantitative evaluation index is obtained by calculating the visible pixel ratio Sk(i) of its bound face set Fk for each structure k in the key structure candidate set K based on the face ID segmentation map S, and then weighting and summing the results according to the weight wk calculated based on the structure area and curvature; the viewpoint redundancy R(i, A) is calculated by comparing the similarity between the candidate viewpoint vi and all viewpoints in the best viewpoint set A, and taking the maximum value as R(i, A). In some specific examples, the weight wk can be obtained by combining the area ratio and curvature ratio of structure k. An exemplary combination coefficient of area ratio and curvature ratio can be 0.7 and 0.3, respectively, where the area ratio is the ratio of the sum of the areas of the bound face set Fk of structure k to the sum of the areas of each bound face set in the key structure candidate set K, and the curvature ratio is the ratio of the sum of the curvature indices of the bound face set Fk of structure k to the sum of the curvature indices of each bound face set in the key structure candidate set K.
[0036] Specific quantitative evaluation indicators include: 1) Visible area ratio V(i): Statistically calculate the proportion of non-zero pixels in Si, or perform area-weighted statistics on the visible surface area to obtain the proportion of the visible area of the model under the viewpoint vi.
[0037] 2) Contour information C(i): The contour information is obtained by statistically analyzing the proportion of contour pixels, the total length of the contour, or the edge density of Ei and normalizing them.
[0038] 3) Visibility of key structures S(i): For each structure k (bound face set Fk) in the prior set of structures K, the visibility ratio Sk(i) of the corresponding face ID of the structure is calculated by Si, and S(i) is obtained by weighted summation according to the structure weight wk; where wk can be preset according to the importance of the structure category or set according to the structure area / curvature rules, etc.
[0039] 4) Viewpoint Redundancy R(i,A): For the currently selected viewpoint set A, calculate the maximum similarity between the candidate viewpoint vi and any viewpoint in A. The similarity can be based on the Jaccard similarity of the visible face ID set Faces(i) and at the same time, compare the cosine similarity based on the viewpoint embedding vector.
[0040] Construct a viewpoint rating function and select the Top-k viewpoint set A':
[0041] Score(i)=α·V(i)+β·C(i)+γ·S(i)-λ·R(i,A)
[0042] Where α, β, γ, and λ are non-negative weighting coefficients, which can be set as preset constants. An iterative greedy strategy is used to sequentially select the viewpoints that maximize Score(i) from V and add them to set A, until |A|=k or the information gain of the newly added viewpoints is lower than the threshold τ, thus obtaining the optimal viewpoint set A'.
[0043] In some embodiments, rendering the viewpoints corresponding to the optimal viewpoint set A to generate the input image set I={I1…Ik} further includes: performing at least one preprocessing step on the input image set I={I1…Ik}, including scaling, rotation, background, and lighting, to obtain a standardized optimal viewpoint image set I'. For example, the input image set I can be scaled, rotated and aligned, its field of view unified, its background consistent, and its brightness / contrast normalized. Dimensionality reduction processing can also be performed on the multi-view feature vectors to reduce noise and redundancy, ultimately obtaining the optimal viewpoint image set I'.
[0044] In some embodiments, semantic feature extraction using a Visual Language Model (VLM) based on the input image set further includes, during the extraction process, using at least one of the rendered depth map, normal map, and contour map as auxiliary input and jointly encoding it with the optimal viewpoint image set I' to constrain the structured semantic features output by the VLM and the corresponding embedding vectors. In some embodiments, the structured semantic features output by the VLM are in the format of structure labels, attribute key-value pairs, and confidence scores.
[0045] In a specific example, the optimal viewpoint image set I' is input into the visual language model (VLM) for semantic feature extraction. Unlike directly inputting the image as a black box, this embodiment of the application adopts the following mechanism:
[0046] 1) Structural prior cue injection: The key structural candidate set K and its structural attribute summary obtained in step S101 are constructed into a text cue P, which is input into VLM along with the viewpoint image, so that VLM outputs semantic labels and attribute information related to K;
[0047] 2) Multi-source rendering auxiliary input: Depth map Di, normal map Ni, and contour map Ei are used as auxiliary inputs and jointly encoded with the RGB image to enhance the expression of structural boundaries and occlusion relationships.
[0048] 3) Output structure constraints: The VLM output is constrained to meet the preset structure format (e.g., a semi-structured result of “structure label + attribute key-value pair + confidence”) to obtain the semi-structured semantic features Ti of each perspective; at the same time, the embedding vector ei of each perspective is obtained for retrieval, similarity calculation and fusion.
[0049] In some embodiments, fusing the embedded vectors and structured semantic features output from each viewpoint to obtain a semantic feature representation includes: weighted fusion of the embedded vectors output from each viewpoint to obtain a CAD model-level semantic vector E, wherein the weight of the embedded vector from each viewpoint in the weighted fusion is obtained by normalizing the quantitative evaluation index or key structure visibility S(i) of that viewpoint. Specifically, weighted fusion is performed on the embedded vectors {ei} from k viewpoints to obtain the CAD model-level semantic vector E, and the weights can be obtained by normalizing Score(i) or S(i) in step S3:
[0050] wi = Score(i) / ΣScore(i)
[0051] E=Σ wi·ei
[0052] Further consistency merging is performed on the semi-structured semantic features {Ti}. In some embodiments, the embedding vectors and structured semantic features output from each perspective are fused to obtain the semantic feature representation, which also includes:
[0053] The structured semantic features from each perspective are deduplicated and conflict-resolved, and then merged to generate the final structured semantic result T.
[0054] The output includes the final semantic feature representation F={E,T} containing the model-level semantic vector E and the structured semantic result, where T is the merged structured semantic result.
[0055] Furthermore, the model-level semantic vector E is written into the vector library and a vector index is established. The structured semantic result T is written into the structured index to support field retrieval and filtering, forming a feature base for multimodal retrieval and downstream applications for CAD models.
[0056] This application introduces a visual language model to achieve automatic semantic extraction of visual features from CAD models, reducing reliance on manual rules. Through an optimal viewpoint selection strategy, the rendered image of the input visual language model can more fully reflect the structural features of the CAD model. The method of this application avoids feature redundancy problems caused by fixed viewpoints or blind multi-viewpoint rendering. The method of this application can improve the discriminativeness and consistency of expression of visual features of CAD models. It has good versatility and is applicable to various CAD models and rendering methods.
[0057] This application also proposes a CAD model visual fusion feature extraction system with optimal feature perspective, including a processor and a memory. The memory stores a computer program, and when the computer program is executed by the processor, it implements the steps of the CAD model visual fusion feature extraction method with optimal feature perspective as described above.
[0058] It should be noted that, in the embodiments of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.
[0059] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0060] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0061] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims. All of these forms are within the protection scope of this application.
Claims
1. A method for visual fusion feature extraction from CAD models with optimal feature perspective, characterized in that, include: Obtain the CAD model file to be processed, parse its boundary representation B-Rep, and construct structural prior information based on the B-Rep; The prior information of the structure includes at least: the face type and geometric parameters based on the geometric type determination of the face in B-Rep, the topological adjacency graph reflecting the face-edge-face adjacency relationship, and the key structure candidate set K automatically generated based on the preset structure discrimination rules, wherein any structure k in the key structure candidate set K is bound to a set of face or edge IDs Fk. Based on the geometric properties of the CAD model file or a preset sampling strategy, a candidate viewpoint set V containing multiple camera pose parameters is generated. Based on each candidate viewpoint vi in the candidate viewpoint set V, the CAD model is rendered to generate a rendering output that includes at least an RGB image I and a face ID segmentation map S. Based on the rendering output, a quantitative evaluation index is calculated. The optimal viewpoint is selected from the candidate viewpoint set V using this calculated quantitative evaluation index to form the best viewpoint set A. The quantitative evaluation index includes: Visible area ratio V(i), contour information C(i), key structure visibility S(i), and viewpoint redundancy R(i, A), where, The visibility S(i) of the key structure is obtained by calculating the visible pixel ratio Sk(i) of its bound face set Fk based on the face ID segmentation map S for each structure k in the key structure candidate set K, and then summing the results by weighting wk based on the structure area and curvature. The viewpoint redundancy R(i, A) is calculated by comparing the similarity between the candidate viewpoint vi and all viewpoints in the best viewpoint set A, and taking the maximum value as R(i, A). Render the viewpoints corresponding to the optimal viewpoint set A to generate an input image set; Based on the input image set, semantic features are extracted using the Visual Language Model (VLM). During the extraction process, the key structure candidate set K in the structural prior information is constructed into a text prompt P and injected into the VLM to constrain the structured semantic features and corresponding embedding vectors output by the VLM. The embedded vectors and structured semantic features output from each perspective are fused to obtain semantic feature representations.
2. The CAD model visual fusion feature extraction method based on the optimal feature perspective as described in claim 1, characterized in that, The prior structural information also includes a structural attribute summary, which includes at least one of orientation, size range, or axis information.
3. The CAD model visual fusion feature extraction method based on the optimal feature perspective as described in claim 1, characterized in that, Rendering the viewpoints corresponding to the optimal viewpoint set A and generating the input image set further includes: performing at least one preprocessing step on the input image set, including scaling, rotation, background and lighting, to obtain a standardized optimal viewpoint image set.
4. The CAD model visual fusion feature extraction method based on the optimal feature perspective as described in claim 3, characterized in that, Semantic feature extraction based on the input image set using the Visual Language Model (VLM) also includes, during the extraction process, using at least one of the rendered depth map, normal map, and contour map as auxiliary inputs and jointly encoding them with the optimal viewpoint image set to constrain the structured semantic features and corresponding embedding vectors output by the VLM.
5. The CAD model visual fusion feature extraction method based on the optimal feature perspective as described in claim 1, characterized in that, The structured semantic features output by the VLM are in the format of structure labels, attribute key-value pairs, and confidence scores.
6. The CAD model visual fusion feature extraction method based on the optimal feature perspective as described in claim 1, characterized in that, The semantic feature representation is obtained by fusing the embedded vectors and structured semantic features output from each perspective. This includes weighted fusing of the embedded vectors output from each perspective to obtain the CAD model-level semantic vector E. The weight of the embedded vector of each perspective in the weighted fusing is obtained by normalizing the quantitative evaluation index or the key structure visibility S(i) of that perspective.
7. The CAD model visual fusion feature extraction method based on the optimal feature perspective as described in claim 6, characterized in that, The semantic feature representation obtained by fusing the embedded vectors and structured semantic features output from various perspectives also includes: The structured semantic features from each perspective are deduplicated and conflict-resolved, and then merged to generate the final structured semantic result T. The output contains the final semantic feature representation, which includes the model-level semantic vector E and the structured semantic result T.
8. A CAD model visual fusion feature extraction system with optimal feature perspective, characterized in that, It includes a processor and a memory, wherein the memory stores a computer program that, when executed by the processor, implements the steps of the CAD model visual fusion feature extraction method based on the optimal feature perspective as described in any one of claims 1 to 7.