CAD (Computer Aided Design) language reasoning method for guiding attention based on layered sketch instance
By using an autoregressive strategy and hierarchical sketch instances to guide attention, combined with the embedding encoding and cross-attention mechanism of point clouds and CAD sequences, the problem of insufficient model flexibility and practicality in point cloud 3D reverse engineering is solved, and high-precision CAD model reconstruction is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIHANG UNIV
- Filing Date
- 2026-01-06
- Publication Date
- 2026-05-01
AI Technical Summary
In existing point cloud 3D reverse engineering, the separate learning of point cloud visual representation and CAD language representation leads to insufficient model flexibility and practicality, and the lack of interactive CAD inference affects the accuracy and reliability of the model.
It employs an autoregressive strategy and hierarchical sketch instances to guide attention, progressively predicts commands and allows users to interactively modify them, and combines point cloud and CAD sequence embedding encoding and cross-attention mechanisms to achieve end-to-end CAD language reasoning.
It improves the flexibility and practicality of the model, enhances the accuracy and reliability of 3D reverse engineering, and achieves high-precision CAD model reconstruction.
Smart Images
Figure CN121960746A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, and more specifically to a CAD language reasoning method based on attention guided by hierarchical sketch instances. Background Technology
[0002] Reverse engineering, also known as reverse engineering, is a technical imitation process that involves reverse analysis and research of a target product to deduce its processing flow, organizational structure, functional performance specifications, and other design elements, in order to create a product with similar but not identical functions. CAD technology has become an indispensable part of engineering design. Digital modeling has significantly improved the speed and accuracy of drawing, providing a more vivid and intuitive form of design representation. Drawing methods have evolved from 2D hand-drawn drafting to 3D CAD drafting. With the popularity of CAD, reverse engineering has become a method for constructing 3D virtual models of existing physical components using CAD, CAM, CAE, or other software.
[0003] How to effectively learn CAD visual language representations from point clouds and CAD sequences for use in 3D reverse engineering is a difficult point in 3D reverse engineering research. Existing methods have two main limitations: (1) Visual representations and CAD language representations are learned separately, and then the mapping between the two representations is learned. This separate learning may produce specific modal features that are unrelated to point cloud CAD language inference. These modal features will affect the model attention, thereby affecting the effect of cross-modal alignment and reducing the accuracy and reliability of inference. The learning of CAD language representations is achieved through a feedforward strategy, that is, the CAD language of the complete design history is inferred at once, which greatly reduces the flexibility and practicality of the model.
[0004] Therefore, improving the flexibility and practicality of the model is an urgent problem to be solved in this field. Summary of the Invention
[0005] In view of this, the present invention provides a CAD language reasoning method based on attention guided by hierarchical sketch instances. It adopts an autoregressive strategy, predicts commands step by step and updates the CAD sequence, and allows users to provide input based on the current prediction results at each design step, such as modifying sketch parameters or stretching commands, thereby realizing interactive custom modifications and improving the flexibility and practicality of the model.
[0006] To achieve the above objectives, the present invention adopts the following technical solution: A CAD language reasoning method based on attention guided by hierarchical sketch instances includes the following steps: Acquire point cloud data and an initial CAD sequence; the CAD sequence consists of executing multiple modeling commands sequentially; the modeling commands include extrusion commands, sketch commands, and split commands, and different modeling commands have corresponding flag bits; After embedding and encoding the point cloud data and the initial CAD sequence, autoregressive decoding is performed. During decoding, cross-attention is used to associate point cloud features and CAD features, and nonlinear transformation is performed. Finally, the next bit modeling command is obtained through a linear layer. Decoding is repeated, and the predicted CAD sequence is generated bit by bit. In the autoregressive decoding process, flag bits in the current CAD sequence are identified. When the recognition result is a separation command, the stretching command is obtained by repeatedly decoding; When the recognition result is a stretching command, the corresponding sketch instance is selected from the point cloud features according to the stretching command; the corresponding local point cloud features are selected to participate in the interaction using the corresponding mask matrix, and finally the corresponding sketch instruction is obtained. The autoregressive decoding is repeated and the decoding result is accumulated bit by bit until the separation instruction is recognized.
[0007] Preferably, a point cloud encoder is used to perform embedding encoding processing on the point cloud data, and the encoding method of the point cloud encoder includes:
[0008]
[0009] in, It is a learned embedding, where X is point cloud data. It is a learnable matrix; This is the initial point cloud embedding vector; The weight matrix is a learnable linear projection.
[0010] Preferably, the initial CAD sequence is embedded, including: For sequences any one To be honest
[0011]
[0012]
[0013] Wherein, C is the initial CAD sequence. Encoding is embedded in the command. It is a sequence type. Encode the embedded parameters. For command parameters, Fill the parameter with a mask. For learning location encoding, and It is a learnable embedding matrix; Embedding of the entire sequence C It can be represented as:
[0014] in, The initial CAD feature vector, It's a sequence mask, the purpose of which is to retain only the valid parts of the sequence. Indicates splicing.
[0015] Preferably, the autoregressive decoding is performed using a Transformer multilayer decoder, including: Obtain the CAD embedding features output from layer b-1 and point cloud embedding ; The point cloud features are obtained by performing local feature aggregation based on the point cloud embedding. A second CAD feature is generated based on the CAD embedding feature using a self-attention mechanism and a normalization layer. ; The point cloud features and the second CAD features are fused using a cross-attention layer to obtain cross-embedding features; The cross-embedding and the second CAD feature vector are normalized again to obtain the third CAD feature; The third CAD feature is added to the first CAD feature vector through a feedforward network to obtain the final CAD embedding.
[0016] Preferably, the attention cross-association step includes: Based on the point cloud feature vector, the key K and value V are extracted, and the query Q is extracted based on the initial CAD feature vector. The interaction embedding feature is obtained by combining the mask matrix with cross-attention weighting.
[0017] Preferably, the process of selecting sketch instances from the point cloud based on the stretch command includes: The stretch command confirms the projected bounding box projected from the initial xy plane onto the sketch plane; The points located within the projected bounding box in the point cloud features are selected to obtain the sketch instance.
[0018] Preferably, the method for confirming the projected bounding box is as follows:
[0019] in, For unit boundaries, Euler angles, It is a translation vector. The scaling factor for the stretch command; It is a rotation matrix.
[0020] Preferably, the steps further include: obtaining multiple sketch commands corresponding to multiple probabilities during the autoregressive decoding process, and obtaining multiple CAD sequence prediction results through mixed sampling; CAD models are reconstructed based on the prediction results of each CAD sequence, and the optimal model is obtained through evaluation.
[0021] Preferably, the steps further include: during the autoregressive decoding process, before each step of updating the CAD sequence, actively modifying the predicted sketch command to customize the sketch parameters or stretch command parameters.
[0022] This invention addresses the problem of separate learning of point cloud visual representation and CAD language representation in current point cloud 3D reverse engineering, and the lack of interactive CAD inference. It employs an end-to-end trainable autoregressive network to infer CAD language from input point clouds; utilizes a multimodal Transformer module with a hierarchical cross-attention mechanism between point cloud and CAD language embeddings; and proposes a Sketch Instance Guided Attention (SGA) module to guide the hierarchical cross-attention mechanism to focus on relevant regions of the point cloud to predict sketch parameters, thus achieving high-precision, interactive point cloud 3D reverse engineering. Attached Figure Description
[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0024] Figure 1 This is a schematic diagram of a CAD language reasoning method based on attention guided by layered sketch instances, provided in an embodiment of the present invention. Detailed Implementation
[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0026] like Figure 1This invention discloses a CAD language reasoning method based on hierarchical sketch example-guided attention, comprising the following steps: A CAD language reasoning method based on attention guided by hierarchical sketch instances, characterized by the following steps: Acquire point cloud data and an initial CAD sequence; the CAD sequence consists of executing multiple modeling commands sequentially; the modeling commands include extrusion commands, sketch commands, and split commands, and different modeling commands have corresponding flag bits.
[0027] After embedding and encoding the point cloud data and the initial CAD sequence, autoregressive decoding is performed. During decoding, cross-attention is used to associate point cloud features and CAD features, and nonlinear transformation is performed. Finally, the next bit modeling command is obtained through a linear layer. Decoding is repeated, and the predicted CAD sequence is generated bit by bit. During autoregressive decoding, flag bits in the current CAD sequence are identified. In this embodiment, the flag bits of the CAD sequence indicate the command type of the last element of the current sequence.
[0028] When the trained model makes predictions, it first decodes and predicts the stretching command. When the recognition result is a stretching command, it selects the corresponding sketch instance from the point cloud features according to the stretching command. It then uses the corresponding mask matrix to select the corresponding local point cloud features to participate in the interaction, and finally obtains the corresponding sketch command. The autoregressive decoding is repeated and the decoding result is accumulated bit by bit until the separation command is recognized. When the recognition result is a separation command, the next stretching command is predicted, and the above process is repeated.
[0029] In this embodiment, the modeling process of a model is represented using a special CAD language, that is, the modeling process of a model is represented by a CAD sequence C. N represents a fixed sequence length. ,t represents a command, including the start of the sequence. <start>, separator <sol>Lines, arcs, circles, stretching, and sequence endings <end>. The parameters within [·] represent the parameters of the modeling operations. Only the commands of line, arc, circle, and extrusion have corresponding parameters, and the parameters of other commands are filled with placeholders. The valid parameters of each command are shown in Table 1. represents the sketch command, and m represents the command type, including line , arc , circle , a total of three types. The specific descriptions of the modeling commands and parameters are shown in Table 1.
[0030] Table 1 Descriptions of Modeling Commands and Parameters
[0031] During the training process, the modeling sequence C is generated一次性生成的,其有效长度为n(n<N),即 , and the remaining sequence lengths are filled with placeholders because the model can only generate sequences of fixed length during training. During the inference process of the model, the sequence is generated step by step rather than一次性生成的, so a fixed length is not required, but the effective length cannot exceed N.
[0032] The invention principle of the present invention is to decompose the CAD model modeling process into several combinations of sketches and extrusions, that is, different sketch commands form a closed sketch, and then through the extrusion command , a solid is formed. This process is repeated continuously, and the final 3D model is formed by different solid combinations.
[0033] At the beginning stage, the CAD sequence only has a start flag. The model predicts the first extrusion command and gradually obtains the corresponding sketch commands for this extrusion command through autoregressive decoding, updates the CAD sequence step by step, and obtains the final complete CAD sequence by repeating this process.
[0034] Specifically, this embodiment is an end-to-end CAD part modeling sequence generation model based on Transformer, which consists of a point cloud subset selector, a point cloud encoder, a CAD sequence embedding layer, and multiple decoders.
[0035] In the point cloud encoder, given the point cloud , where is the number of additional features estimated for each point. First, a linear layer is applied, and then ReLU is applied, as shown below:
[0036] Among them, is the learned embedding, is the learnable matrix, . The embedding of the point cloud It should be noted that there seems to be an incorrect expression "一次性生成的" in the original text. I have translated it as "一次性生成的" as it is, but it might need to be corrected to a proper phrase in the source language for a more accurate translation.Subsequently, the Local Feature Aggregation (LFA) module was used to further enrich the feature set. The features obtained for each point are obtained. LFA uses k-NearestNeighbor (k-NN) to aggregate the features of neighboring points through a linear combination of learned attention weights.
[0037] The first LFA module processes the feature point cloud embedding formula as follows:
[0038] in, The weight matrix represents the learnable linear projection.
[0039] Similarly, the b-th LFA module is applied to The formula is as follows:
[0040] in, The weight matrix represents the learnable linear projection. It is a point cloud feature used to generate modeling sequences.
[0041] Corresponding to the CAD sequence embedding layer, given the input design sequence The sequence consists of n modeling operations, each modeling operation using express, It consists of a command and parameters. The command is represented by a one-dimensional vector, and the parameters are represented by a 16-dimensional vector. It is a 17-dimensional vector, and the parameters are shown in Table 1.
[0042] By chaining these modeling operations and using one-hot encoding, the initial embedding of the CAD language is achieved. This is obtained using the command notation described above in the linear layer, and its value is:
[0043] in, ; ;
[0044] The initial CAD feature vector, It's a sequence mask, the purpose of which is to retain only the valid parts of the sequence. This indicates splicing; C represents the initial CAD sequence. Encoding is embedded in the command. It is a sequence type. Encode the embedded parameters. For command parameters, Fill the parameter with a mask. For learning location encoding, and It is a learnable embedding matrix.
[0045] For predictive decoding, based on the above embedding, this invention utilizes a multi-layer Transformer decoder based on cross-attention to achieve cross-modal generation from point clouds to CAD modeling sequences. The decoder consists of a B-layer Transformer decoder module.
[0046] Specifically, for the b-th layer of the multi-layer decoder, the features output by the (b-1)-th layer are... and point cloud embedding As input.
[0047] First, the CAD embedding features are processed using a self-attention mechanism (SA) and a normalization layer (AddNorm), as shown below:
[0048]
[0049] In this context, the query Q, key K, and value V are all obtained from a learnable matrix. Extracted from M, M is a standard self-attention mask. and These are the intermediate processing results of the b-layer encoder, namely the first CAD feature and the second CAD feature.
[0050] To enable information transfer between CAD language and point cloud embedding within each block, and A cross-attention layer (CA) was used. This is achieved by linearly projecting the point cloud features... Extracting keys Sum This is achieved by embedding from CAD. Extract query from The cross-attention layer calculates the CAD visual language embedding using formula (5). as follows:
[0051] Where 0 is Zero matrix.
[0052] In addition, cross embedding and self-joining embedding and The network is then passed through a normalization layer to further learn the geometric relationships between CAD marks, resulting in a third CAD feature. .
[0053] Finally, Apply a feedforward network (FFN) and add it to In this process, the final CAD embedding is formed. ,and Pass them together to the next level.
[0054] To further implement the above technical solution, this invention achieves cross-interest between sketch instances and corresponding local point cloud sets through point cloud subset selection. Specifically, based on Equation 7, feature selection of point cloud features is achieved by replacing the 0 matrix with a specific mask matrix, further facilitating interaction through the CA layer.
[0055] Traditional multimodal Transformer modules aim to pass information from all point embeddings to CAD marker embeddings. However, the sketch parameterization of this invention only requires a subset of the point cloud for cross-interest. This invention predicts the stretch command for each design step, followed by the sketch command. This means that sketch instances can be defined on the point cloud using the predicted stretch command for cross-interest with the sketch marker embedding.
[0056] In this embodiment, by selectively replacing the zero matrix, the network structure replacement for predicting sketch commands and predicting stretch commands can be realized. When predicting stretch commands, the zero matrix is used to explore stretch commands in the point cloud data. When a stretch command is identified, the corresponding local point cloud information is obtained from the point cloud data through the corresponding mask matrix for interactive attention.
[0057] Local point cloud cluster , is a subset of the input point cloud X. It is extracted by selecting points within the bounding box derived from the corresponding predictive stretch command on the sketch plane. The stretch command refers to the operation of determining the stretch surface by rotation and translation based on an initial plane, and stretching in a specific direction based on the stretch surface.
[0058] Given a set of extrusion commands E, which records the parameters for projecting a unit bounding box U from the initial plane onto the bounding box on the sketch plane (i.e., the extruded plane), the bounding box of the extruded plane is our target bounding box. For example, given a point... Initial definition Unit boundary on a plane, Euler angles Translation vector and the scaling factor defined by the stretching operation E Then the projected bounding box Given by the following formula:
[0059] in, In special orthogonal groups Euler angles are incorporated into the rotation matrix. Sketch instance I is defined by the X point located within this bounding box, i.e. ,in It is an operator used to check the input point. Is it located within the projection bounding box? Inside.
[0060] It is important to note that during training, the real-world stretch command is used to define the bounding box. During inference, the predicted stretch command is used. To avoid affecting the slight errors in the stretch command and point cloud sampling prediction of the sketch plane, the bounding box is set to... Amplification of minute margins.
[0061] Then, the sketch is extracted by using a suitable mask instead of the zero matrix. The extracted sketch instances can be used for cross attention as defined in equation (7), and are only used for sketch command embedding.
[0062] Specifically, let For this mask, Its value, used for the first The first command and the first Attention between point embeddings.
[0063] Introduction This is to mask the sketch command's focus on points outside its corresponding sketch instance. Therefore, if the first... If the embedded command does not represent a sketch, then... Set to 0. If the first The first marker represents a sketch, then the second... When a point embedding is part of the corresponding sketch instance Set to 0, otherwise set to It is important to note that after identifying the sketch instance, it is necessary to... The corresponding subset uses four linear layers to refine its embedding before extracting the keys and values that cross-reference with the sketch command embedding.
[0064] After the multi-layer decoder outputs the final features, CAD embedding... The sequence is passed to two separate linear layers for predictive modeling. CAD modeling commands are predicted using an autoregressive method based on a two-dimensional probability matrix.
[0065] Training: During training, a teacher-forced strategy is employed, using real-world scenarios as input. Cross-entropy loss. It is used as the objective function.
[0066] Reasoning: In the reasoning process, given the input point cloud X and the... The initial CAD sequence is formed, and the next command will be automatically generated recursively until the final command is predicted.
[0067] Hybrid Sampling: The autoregressive properties of this invention demonstrate that, with the same training and inference time, different command predictions will produce different final CAD sequences. This allows for the generation of multiple reliable predictions given a point cloud. In particular, given the output probability... In this case, we can either take the first bit to obtain the prediction command, or choose a different selection strategy for each command to obtain different final CAD sequences. To demonstrate this, we used a hybrid sampling method during the inference process, selecting the first 5 probabilities for the first command and the first 1 probability for subsequent commands. This results in 5 different final CAD sequences given a point cloud. Finally, the best CAD sequence is selected by choosing the one closest to the input point cloud. The evaluation method involves reconstructing the CAD model based on the prediction sequence, sampling the point cloud on the model, and then selecting the model with the smallest chamfer distance from the input point cloud.
[0068] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.
[0069] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.< / end> < / sol> < / start>
Claims
1. A CAD language reasoning method based on attention guided by hierarchical sketch instances, characterized in that, Includes the following steps: Acquire point cloud data and an initial CAD sequence; the CAD sequence consists of executing multiple modeling commands sequentially; the modeling commands include extrusion commands, sketch commands, and split commands, and different modeling commands have corresponding flag bits; After embedding and encoding the point cloud data and the initial CAD sequence, autoregressive decoding is performed. During decoding, cross-attention is used to associate point cloud features and CAD features, and nonlinear transformation is performed. Finally, the next bit modeling command is obtained through a linear layer. Decoding is repeated, and the predicted CAD sequence is generated bit by bit. In the autoregressive decoding process, flag bits in the current CAD sequence are identified. When the recognition result is a stretching command, the corresponding sketch instance is selected from the point cloud features according to the stretching command; the corresponding local point cloud features are selected to participate in the interaction using the corresponding mask matrix, and finally the corresponding sketch instruction is obtained. The autoregressive decoding is repeated and the decoding result is accumulated bit by bit until the separation instruction is recognized.
2. The CAD language reasoning method based on hierarchical sketch instance-guided attention as described in claim 1, characterized in that, The point cloud data is embedded and encoded using a point cloud encoder. The encoding method of the point cloud encoder includes: in, It is a learned embedding, where X is point cloud data. It is a learnable matrix; This is the initial point cloud embedding vector; The weight matrix is a learnable linear projection.
3. The CAD language reasoning method based on hierarchical sketch instance-guided attention as described in claim 2, characterized in that, Embedding the initial CAD sequence includes: For sequences any one To be honest Wherein, C is the initial CAD sequence. Encoding is embedded in the command. It is a sequence type. Encode the embedded parameters. For command parameters, Fill the parameter with a mask. For learning location encoding, and It is a learnable embedding matrix; Embedding of the entire sequence C It can be represented as: in, The initial CAD feature vector, It's a sequence mask, the purpose of which is to retain only the valid parts of the sequence. Indicates splicing.
4. The CAD language reasoning method based on hierarchical sketch instance-guided attention as described in claim 3, characterized in that, The autoregressive decoding is performed using a Transformer multilayer decoder, including: Obtain the CAD embedding features output from layer b-1 and point cloud embedding ; The point cloud features are obtained by performing local feature aggregation based on the point cloud embedding. A second CAD feature is generated based on the CAD embedding feature using a self-attention mechanism and a normalization layer. ; The point cloud features and the second CAD features are fused using a cross-attention layer to obtain cross-embedding features; The cross-embedding and the second CAD feature vector are normalized again to obtain the third CAD feature; The third CAD feature is added to the first CAD feature vector through a feedforward network to obtain the final CAD embedding.
5. A CAD language reasoning method based on hierarchical sketch instance-guided attention as described in claim 4, characterized in that, The steps of the attention cross-association include: Based on the point cloud feature vector, the key K and value V are extracted, and the query Q is extracted based on the initial CAD feature vector. The interaction embedding feature is obtained by combining the mask matrix with cross-attention weighting.
6. The CAD language reasoning method based on hierarchical sketch instance-guided attention as described in claim 1, characterized in that, The stretch command is used to filter sketch instances from the point cloud special row, including: The stretch command confirms the projected bounding box projected from the initial xy plane onto the sketch plane; The points located within the projected bounding box in the point cloud features are selected to obtain the sketch instance.
7. A CAD language reasoning method based on hierarchical sketch instance-guided attention as described in claim 6, characterized in that, The method for confirming the projected bounding box is as follows: in, For unit boundaries, Euler angles, It is a translation vector. The scaling factor for the stretch command; It is a rotation matrix.
8. The CAD language reasoning method based on hierarchical sketch instance-guided attention as described in claim 1, characterized in that, The steps also include: during the autoregressive decoding process, obtaining multiple sketch commands corresponding to multiple probabilities, and obtaining multiple CAD sequence prediction results through mixed sampling; CAD models are reconstructed based on the prediction results of each CAD sequence, and the optimal model is obtained through evaluation.
9. A CAD language reasoning method based on hierarchical sketch example-guided attention according to claim 1, characterized in that, The steps also include: during the autoregressive decoding process, before each step of updating the CAD sequence, actively modifying the predicted sketch command to customize the sketch parameters or stretch command parameters.