A CAD model generation method and system based on multi-modal semantic fusion and feature regression constraint, and a storage medium

Abstract

This application provides a method, system, and storage medium for generating CAD models based on multimodal semantic fusion and feature regression constraints. The method includes: Step S1, acquiring a multimodal dataset; Step S2, training an autoencoder model using CAD operation sequences, and freezing the encoder and decoder of the autoencoder model after training; Step S3, fusing multimodal information using a pre-constructed multimodal semantic fusion model to obtain a fusion result; Step S4, minimizing the latent semantic representation of the fusion result and the CAD operation sequences, and training the multimodal semantic fusion model; Step S5, sequentially inputting the target data of the CAD model to be generated into the trained multimodal semantic fusion model and decoder to obtain a CAD operation sequence, and generating the CAD model using the CAD operation sequence. This application has higher automatic semantic expression capability, structural editability, and multimodal adaptability for generating CAD models, and has good engineering application value.

Description

Technical Field

[0001] This application relates to the field of CAD drafting technology, specifically a CAD model generation method, system, and storage medium based on multimodal semantic fusion and feature regression constraints. Background Technology

[0002] In the field of Computer-Aided Design (CAD), parametric CAD models are typically expressed as a sequence of operations. This sequence consists of a series of modeling commands and their corresponding parameters, used to describe the model construction process. Compared to geometric representations such as point clouds and meshes, CAD operation sequences can not only depict the geometric shape of the model but also explicitly express the design intent and parameter constraints during the modeling process, making it an important representation method for achieving editable CAD modeling. Based on this, generation methods centered on CAD operation sequences have gradually emerged in recent years.

[0003] Early research primarily focused on generating unconditional CAD operation sequences, employing autoregressive models or sequence modeling methods to learn and generate these sequences. While these methods can reconstruct the construction process of CAD models to some extent, the lack of external constraint information limits their generation results in terms of geometric rationality and design consistency.

[0004] Building upon this foundation, to enhance the controllability and expressiveness of the generated results, existing technologies further incorporate external conditional information, extending CAD operation sequence generation to a conditional generation paradigm. This involves guiding the modeling process through input point clouds, images, or text. These methods, by constructing cross-modal mapping relationships, achieve the conversion from different forms of input data to CAD operation sequences, thereby improving the diversity and adaptability of the generated results to some extent. However, due to the significant differences in geometric and semantic expressive capabilities among different modalities, existing methods often employ simple feature splicing or alignment strategies, lacking an effective modeling mechanism for multimodal semantic consistency. This results in semantic shifts and information inconsistencies in the generated operation sequences under different modal conditions, making it difficult to accurately reflect the true design intent.

[0005] Therefore, how to achieve semantically consistent feature fusion under multimodal input conditions and effectively constrain the command parameters in the operation sequence to improve the accuracy and executability of the generated results has become a pressing technical problem that needs to be solved. Summary of the Invention

[0006] To overcome the problems existing in related technologies, this application provides a CAD model generation method, system and storage medium based on multimodal semantic fusion and feature regression constraints.

[0007] According to a first aspect of the embodiments of this specification, a method for generating CAD models based on multimodal semantic fusion and feature regression constraints is provided, comprising:

[0008] Step S1: Obtain a multimodal dataset, which includes parameterized CAD operation sequences and their corresponding multimodal information; the multimodal information includes: multi-view images, text descriptions, and engineering drawing data;

[0009] Step S2: Train an autoencoder model using CAD operation sequences, and freeze the encoder and decoder of the autoencoder model after training; wherein, the encoder is used to generate the latent semantic representation of the CAD operation sequence, and the decoder is used to autoregressively generate the corresponding CAD operation sequence based on the input latent semantic representation.

[0010] Step S3: Use a pre-built multimodal semantic fusion model to fuse multimodal information and obtain the fusion result of multimodal information;

[0011] Step S4: Train a multimodal semantic fusion model by minimizing the fusion result of multimodal information and the latent semantic representation of the CAD operation sequence;

[0012] Step S5: Input the target data of the CAD model to be generated into the trained multimodal semantic fusion model to obtain the fusion result corresponding to the target data. Input the fusion result corresponding to the target data into the trained decoder to obtain the CAD operation sequence. Use the CAD operation sequence to generate the CAD model.

[0013] In some preferred embodiments, step S3, constructing the multimodal semantic fusion model includes: constructing a multimodal feature encoding alignment module, a cross-modal semantic fusion module, and a feature mapping module;

[0014] The multimodal feature encoding alignment module is used to extract and project features from multimodal information to obtain a multimodal feature representation with uniform dimensions.

[0015] The cross-modal semantic fusion module utilizes the correlation between features of different modalities to adaptively fuse multimodal feature representations and obtain multimodal fused feature representations;

[0016] The feature mapping module is used to perform feature mapping on the multimodal fusion feature representation to obtain the fusion result, which has the same dimension as the latent semantic representation.

[0017] According to a second aspect of the embodiments of this specification, a CAD model generation system based on multimodal semantic fusion and feature regression constraints is provided, comprising:

[0018] The data preprocessing module is used to acquire data and construct a multimodal dataset, which includes parameterized CAD operation sequences and their corresponding multimodal information; the multimodal information includes: multi-view images, text descriptions, and engineering drawing data;

[0019] The autoencoder module is used to train an autoencoder model using CAD operation sequences. After training, the encoder and decoder of the autoencoder model are frozen. The encoder is used to generate the latent semantic representation of the CAD operation sequence, and the decoder is used to autoregressively generate the corresponding CAD operation sequence based on the input latent semantic representation.

[0020] The multimodal semantic fusion module is used to fuse multimodal information and obtain the fusion result of multimodal information;

[0021] The fusion training module is used to train a multimodal semantic fusion model by minimizing the fusion result of multimodal information and the latent semantic representation of CAD operation sequences;

[0022] The CAD kernel execution module is used to obtain the CAD operation sequence obtained by the target data of the CAD model to be generated after training and processing by the multimodal semantic fusion module and decoder, so as to generate the CAD model using the CAD operation sequence.

[0023] According to a third aspect of the embodiments of this specification, a computer-readable storage medium is provided, on which a computer program is stored, which, when run on a processor, performs the steps of the method described above.

[0024] The technical solutions provided by the embodiments of this application may include the following beneficial effects:

[0025] This application employs a parametric modeling approach based on operation sequences, representing the CAD model generation process as a structured sequence of operations. This results in highly interpretable and editable generated models, which is more beneficial for subsequent modifications and engineering applications compared to methods that directly generate geometric models. By constructing a unified semantic representation space for CAD operation sequences, it achieves an effective mapping of multimodal information to the parametric modeling process, overcoming the reliance on single-modal input in existing methods and improving the ability to express complex design intentions. Through multimodal information fusion and semantic regression constraints, it achieves precise alignment between multimodal information and the latent semantic space of CAD, improving the consistency and stability of the generated results while avoiding the dependence on sample matching relationships found in contrastive learning methods. Furthermore, this application is based on multimodal training, is backward compatible, supports flexible combinations of modal inputs, and maintains good generation performance even when some modalities are missing, enhancing the robustness and practical application adaptability of the method. In summary, this application has significant advantages in the semantic expressiveness, structural editability, and multimodal adaptability of automatic CAD model generation, and has good engineering application value.

[0026] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this specification. Attached Figure Description

[0027] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this specification and, together with the description, serve to explain the principles of this specification.

[0028] Figure 1 This specification illustrates a flowchart of the steps involved in generating a CAD model based on multimodal semantic fusion and feature regression constraints, according to an exemplary embodiment.

[0029] Figure 2 This specification is a schematic diagram of the information flow processing of a CAD model generation method based on multimodal semantic fusion and feature regression constraints, according to an exemplary embodiment.

[0030] Figure 3 This is a schematic diagram illustrating the parametric CAD model generation principle of a CAD model generation method based on multimodal semantic fusion and feature regression constraints, according to an exemplary embodiment of this specification.

[0031] Figure 4 This is a schematic diagram of a multimodal fusion module and a feature mapping module for a CAD model generation method based on multimodal semantic fusion and feature regression constraints, according to an exemplary embodiment of this specification.

[0032] Figure 5 This specification illustrates a multimodal information example of a CAD model generation method based on multimodal semantic fusion and feature regression constraints, according to an exemplary embodiment.

[0033] Figure 6 This specification presents a model generation visualization example and comparison of a CAD model generation method based on multimodal semantic fusion and feature regression constraints, according to an exemplary embodiment.

[0034] Figure 7 This is a block diagram illustrating an exemplary embodiment of a CAD model generation system based on multimodal semantic fusion and feature regression constraints. Detailed Implementation

[0035] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this specification as detailed in the appended claims.

[0036] The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of this specification. The singular forms “a,” “the,” and “the” as used in this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0037] It should be understood that although the terms first, second, third, etc., may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this specification, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0038] The embodiments described in this specification will now be described in detail.

[0039] like Figure 1 As shown, this application discloses a CAD model generation method based on multimodal semantic fusion and feature regression constraints, including:

[0040] Step S1: Obtain a multimodal dataset, which includes parameterized CAD operation sequences and their corresponding multimodal information. The multimodal information includes: multi-view images, text descriptions, and engineering drawing data.

[0041] Step S2: Train an autoencoder model using CAD operation sequences, and freeze the encoder and decoder of the autoencoder model after training is complete; wherein, the encoder is used to generate the latent semantic representation of the CAD operation sequence, and the decoder is used to autoregressively generate the corresponding CAD operation sequence based on the input latent semantic representation.

[0042] Step S3: Use a pre-built multimodal semantic fusion model to fuse multimodal information and obtain the fusion result of multimodal information.

[0043] Step S4: Train a multimodal semantic fusion model by minimizing the fusion result of multimodal information and the latent semantic representation of the CAD operation sequence.

[0044] Step S5: Input the target data of the CAD model to be generated into the trained multimodal semantic fusion model to obtain the fusion result corresponding to the target data. Input the fusion result corresponding to the target data into the trained decoder to obtain the CAD operation sequence. Use the CAD operation sequence to generate the CAD model.

[0045] The target data is one or more of the following: multi-view images, text descriptions, and engineering drawing data corresponding to the CAD model to be generated.

[0046] Through the above technical solution, the embodiments of this application introduce a parameter-level feature regression constraint mechanism in the operation sequence generation process to achieve collaborative modeling of CAD operation sequence command types and parameters, thereby improving the semantic consistency, geometric accuracy and modeling executability of the generated results, and improving the accuracy of CAD models generated under semantic fusion.

[0047] In some embodiments of this application, the multimodal dataset in step S1 includes multiple CAD instances, each of which contains a one-to-one parameterized CAD operation sequence and multimodal information; the CAD operation sequence includes a predefined set of operation commands, which includes contour initiation operation commands, geometric drawing operation commands, and 3D modeling operation commands. Each operation command includes a command type and command parameters.

[0048] In some embodiments of this application, the multi-view images in step S1 are generated using uniformly preset view parameters; the engineering drawing data includes single-view isometric engineering drawings and multi-view orthogonal three-view engineering drawings.

[0049] In some embodiments of this application, the multimodal semantic fusion model in step S3 includes a multimodal feature encoding alignment module, a cross-modal semantic fusion module, and a feature mapping module, wherein:

[0050] The multimodal feature encoding alignment module is used to extract and project features from the multimodal information to obtain a multimodal feature representation with unified dimensions.

[0051] The cross-modal semantic fusion module utilizes the correlation between features of different modalities to adaptively fuse the multimodal feature representations to obtain a multimodal fused feature representation;

[0052] The feature mapping module is used to perform feature mapping on the multimodal fusion feature representation to obtain the fusion result, which has the same dimension as the latent semantic representation.

[0053] In some possible implementations of this embodiment, step S2, which involves training an autoencoder model using the CAD operation sequence, includes:

[0054] Each operation command in the CAD operation sequence is embedded with features in a combination of command type embedding, parameter embedding, and position embedding to obtain the embedded representation of the CAD operation sequence.

[0055] The embedded representation of the CAD operation sequence is input into the encoder for feature encoding to obtain the latent semantic representation of the CAD operation sequence.

[0056] Next, the CAD model generation process of this application will be described in detail through examples.

[0057] First, construct a multimodal dataset.

[0058] In this embodiment, the multimodal dataset is based on the instance-level one-to-one correspondence between multimodal information and CAD operation sequences, and includes multi-view rendered images, text descriptions, and engineering data. Figure 3 Multimodal data is used and strictly aligned with the target CAD operation sequence to establish a unified multimodal semantic modeling benchmark.

[0059] For example, the multimodal dataset contains 157,591 samples and is divided into training set, validation set and test set according to a preset ratio, with the training set accounting for 90%, the validation set accounting for 5%, and the test set accounting for 5%, in order to ensure the stability of the model training and evaluation process and the comparability between different methods.

[0060] During the construction of the multimodal dataset, the data for each modality were processed as follows:

[0061] (1) Construction of multi-view images:

[0062] CAD modeling software combined with scripting tools is used to automatically render each CAD model offline. Multi-view images are generated using a preset fixed viewpoint configuration to ensure the consistency of viewpoints between different samples. During the rendering process, the geometric structure information and lighting and shadow features of the model are preserved, and the generated images are uniformly scaled to a resolution of 224×224 as image modal input.

[0063] (2) Construction of text description:

[0064] The text modality is derived from the publicly available dataset Text2CAD. In this embodiment, a high-precision text description containing geometric relationships, dimensional constraints, and structural information is selected as input to improve the correspondence between text semantics and parametric modeling semantics.

[0065] (3) Construction of engineering drawing data:

[0066] The engineering drawing modalities are derived from the CAD-VGDrawing dataset. This embodiment uses vectorized engineering drawing data to provide geometric projection relationships and contour structure constraint information of the CAD model, thereby enhancing the model's ability to express structural details.

[0067] Images, text, and projects constructed using the methods described above Figure 3 All data points to the same CAD instance and its corresponding modeling process, forming a strict one-to-one correspondence, which provides a data foundation for the accurate mapping of multimodal semantics to the parametric modeling semantic space.

[0068] Then, the autoencoder model is trained.

[0069] To obtain a stable, compact, and generalizable parametric modeling semantic representation, this embodiment constructs an autoencoder model for CAD operation sequences, which is used to learn the latent semantic space of the CAD modeling process and realize the reconstruction and generation of operation sequences.

[0070] This embodiment represents the CAD model construction process as a sequence of multiple operation commands, each command including an operation type and a corresponding set of parameters. Considering the characteristic of CAD operations containing both discrete type information and continuous parameter information, a decompositional embedding model is performed on the operation commands. Each operation is represented as a combination of command type embedding, parameter embedding, and position embedding, thereby obtaining a unified-dimensional embedding representation to characterize the operation semantics and their temporal relationships.

[0071] Based on this, the embedded representation of the CAD operation sequence is input into the encoder for feature extraction. The encoder adopts a sequence modeling structure based on a self-attention mechanism, which models the dependencies between different operation commands through a multi-layer network, thereby capturing the structural constraints and semantic associations across steps in the CAD modeling process.

[0072] This embodiment introduces an attention-weighted convergence mechanism into the operational-level features output by the encoder, assigning adaptive weights to different operations to obtain a global latent semantic vector representing the entire CAD operation sequence. This latent vector serves as a compact representation of the CAD modeling process in the latent space, used for subsequent decoding, reconstruction, and multimodal semantic alignment.

[0073] During the decoding phase, a decoder corresponding to the encoder structure is constructed, and the latent semantic representation is used as a global conditional input to guide the decoding process in gradually generating a CAD operation sequence. During decoding, the operation type and its corresponding parameters are predicted progressively through a sequence modeling structure, thereby reconstructing the original CAD operation sequence.

[0074] During the training of the autoencoder model, the difference between the original CAD operation sequence and the reconstructed sequence is used as the optimization objective to perform end-to-end training on the autoencoder. Specifically, in the loss calculation for parameter prediction, a parameter constraint method with a certain tolerance for bias is introduced to reduce the model's oversensitivity to small parameter deviations, thereby improving the model's robustness and generalization ability.

[0075] After training, the decoder can be used as a CAD operation sequence generation model to generate the corresponding parametric modeling process based on the latent semantic representation.

[0076] Next, a multimodal semantic fusion model is trained.

[0077] The multimodal semantic fusion model in this embodiment includes a multimodal feature encoding alignment module, a cross-modal semantic fusion module, and a feature mapping module.

[0078] In this embodiment, in order to achieve comparability and fusion of different modal data in a unified semantic space, feature encoding is performed on image, text and engineering drawing modalities respectively, and feature alignment is achieved through projection mapping.

[0079] refer to Figure 3 and Figure 4 As shown, assume that the multimodal data input to the multimodal feature encoding alignment module are multi-view image sets. Text description and engineering drawing data Each modal data is mapped to a unified feature space through the encoder and projection layer of the multimodal feature encoding and alignment module, resulting in a multimodal feature representation with unified dimensions. :

[0080] (1)

[0081] in, This represents the feature encoder corresponding to the modality. Represents the feature projection layer. To unify the embedding dimensions.

[0082] The above mapping unifies the distribution and feature scale of different modalities, providing a consistent input representation for subsequent multimodal fusion.

[0083] Continue to refer to Figure 3 and Figure 4 As shown, after obtaining a unified representation of multimodal features, the cross-modal semantic fusion module combines the feature representations of each modality to construct a multimodal feature sequence. :

[0084] (2)

[0085] in, This indicates the number of currently available modes.

[0086] To achieve semantic interaction and information complementarity between different modalities, the cross-modal semantic fusion module introduces a fusion method based on cross attention mechanism, using key linear layer and value linear layer as connection layers to perform weighted aggregation of multimodal feature representations.

[0087] Specifically, this involves introducing learnable query vectors, i.e. And construct the attention calculation process:

[0088] (3)

[0089] in, , The key feature projection matrix is ​​used to characterize the semantic index information of each modality feature in the current modeling task, so that different modality features can be relevance calculated with the query vector under a unified semantic scale, thereby determining the importance weight of each modality information; The feature projection matrix is ​​used to carry the semantic content information that actually participates in the fusion calculation.

[0090] Cross-modal feature attention weights are calculated using scaled dot product attention. :

[0091] (4)

[0092] After the attention weights are determined, the value vectors are weighted and aggregated according to their corresponding weights to obtain the multimodal fusion feature representation. :

[0093] (5)

[0094] Through the above method, the cross-modal semantic fusion module of this embodiment realizes the adaptive allocation of the importance of different modal features, enabling the model to dynamically select modal information that contributes more to the semantics of parametric modeling based on the input conditions.

[0095] After obtaining the multimodal fusion feature representation, the feature mapping module performs feature mapping on the multimodal fusion feature representation to obtain the fusion result, which has the same dimension as the latent semantic representation.

[0096] In this embodiment, the feature mapping module uses a deep neural network structure to perform nonlinear mapping on the multimodal fusion feature representation. (See reference...) Figure 4As shown, the feature mapping module uses multi-head attention (MHA) to project and stitch different subspaces, and introduces a feedforward (FF) mechanism to introduce nonlinear capabilities, enabling the model to capture the complex dependencies within the features. Its semantic alignment uses a regression loss function to minimize the differences.

[0097] Continue to refer to Figure 3 and Figure 4 As shown, in order to achieve accurate mapping of multimodal fusion feature representation to the latent semantic space of CAD parametric modeling, the feature mapping module introduces an adaptive mapping mechanism and regression constraints to the multimodal fusion feature representation.

[0098] First, the fused vector is expanded into a sequence form and input into the mapping network for feature transformation:

[0099] (6)

[0100] (7)

[0101] in, This represents a mapping network consisting of multiple Transformer encoders.

[0102] After layer normalization (i.e., Add & Norm), the output is compressed into vector form. (Corresponding to different stages, they are respectively divided into training stages) and the reasoning stage ):

[0103] (8)

[0104] Furthermore, to achieve consistency between cross-modal semantics and the latent semantic representation of CAD operation sequences, feature regression constraints are introduced during the training phase.

[0105] Suppose that the latent semantic representation of the CAD operation sequence obtained by passing it through a pre-trained encoder is... :

[0106] (9)

[0107] in, This represents the CAD encoder trained in step S2.

[0108] By minimizing the distance between the multimodal fusion features and the target semantic representation, alignment between multimodal semantics and parametric modeling semantics is achieved.

[0109] (10)

[0110] like Figure 3As shown, this embodiment adopts a phased training approach in terms of training strategy:

[0111] First, the CAD operation sequence autoencoder model is pre-trained. After training, the parameters of the encoder and decoder are frozen to obtain a stable parametric modeling semantic space.

[0112] Based on freezing the parameters of the CAD encoder and decoder, only the parameters in the multimodal feature encoding alignment module, cross-modal semantic fusion module and feature mapping module of the multimodal semantic fusion model are optimized, so that the multimodal fusion features gradually approach the latent semantic representation of the target CAD, thereby reducing the training complexity and improving the stability of semantic alignment.

[0113] During the inference phase, the target data of the CAD model to be generated is input into the multimodal semantic fusion model to obtain the fusion result corresponding to the target data. Input the trained CAD decoder to generate the operation sequence. :

[0114] (11)

[0115] In this way, a CAD model can be constructed using this sequence of operations.

[0116] After training in this step, a multimodal semantic fusion model with fixed parameters can be obtained. This model can realize the mapping of multimodal input data to a latent semantic representation consistent with the semantics of CAD operation sequences, so as to reconstruct CAD operation sequences to generate CAD models.

[0117] Next, the generated results will be evaluated.

[0118] In this embodiment, after model training is completed, the performance of the constructed model is evaluated. Specifically, on a pre-divided test dataset, the generated CAD operation sequences are compared and analyzed with the real sequences, and the model performance is quantitatively evaluated from aspects such as the accuracy of operation type prediction and parameter prediction error.

[0119] During model training, a preset training configuration is used to ensure model convergence stability and training efficiency. This configuration includes batch size, number of training epochs, optimizer type, learning rate scheduling strategy, and gradient pruning parameters. Specifically, the multimodal semantic fusion model and the CAD operation sequence autoencoder model employ different training strategies to adapt to their respective task characteristics. A warm-up mechanism is introduced in the early stages of training to stabilize the model convergence process, and gradient pruning is used to avoid gradient anomalies during training.

[0120] In the process of multimodal feature extraction, a pre-trained model is used to encode data of different modalities, and its parameters are kept fixed during training to improve the consistency and generalization ability of multimodal representation.

[0121] This embodiment provides a quantitative evaluation of the generated CAD operation sequence and its corresponding geometric model, based on command type accuracy. Parameter accuracy Inefficiency and chamfer distance The generated results are quantitatively evaluated using multiple evaluation indicators, the definitions of which are as follows:

[0122] (1) Command type accuracy The accuracy of command type prediction in the generated operation sequence is measured and calculated as follows:

[0123] (12)

[0124] in, Indicates the length of the CAD operation sequence. and They represent the first The true type and predicted type of each command This is an indicator function.

[0125] 2) Parameter accuracy It is used to measure the prediction accuracy of command parameters assuming the command type prediction is correct; it is defined as follows: :

[0126] (13)

[0127] in, and They represent the first The first command The true and predicted values ​​of each parameter The normalization coefficient is... This is the error tolerance threshold.

[0128] 3) Chamfer distance (CD) is used to measure the geometric difference between the 3D model reconstructed from the generative operation sequence and the real model. It is defined as follows: :

[0129] (14)

[0130] in, and These represent the point sets sampled from the surfaces of the generated model and the real model, respectively.

[0131] 4) Inefficiency (IR) measures the proportion of invalid sequences in the generated CAD operation sequence that cannot be successfully executed. It is defined as follows: :

[0132] (15)

[0133] in, Indicates the number of invalid generated results. This indicates the total number of generated instances.

[0134] During the evaluation process, the analysis was conducted from two aspects: the ability to reconstruct CAD operation sequences and the ability to generate multimodal conditions.

[0135] Table 1. Comparison of metrics between the CAD operation sequence autoencoder model and the baseline method on the main dataset.

[0136]

[0137] First, in the performance evaluation of CAD operation sequence modeling, the CAD operation sequence autoencoder model constructed in step S2 is compared with existing methods. Under the same dataset and training conditions, the generated results are quantitatively evaluated in terms of operation command type prediction accuracy, parameter prediction accuracy, and geometric consistency of the generated results. The quantitative evaluation results are shown in Table 1. The evaluation results show that the autoencoder model constructed in this embodiment has high performance in terms of operation sequence modeling accuracy and parameter representation ability, can more accurately reconstruct the parametric modeling process, and effectively reduce invalid operations and geometric errors.

[0138] Furthermore, to verify the model's generalization ability, tests were conducted on datasets from different sources. The test results are shown in Table 2. The results show that, even with differences in data distribution, the method in this embodiment can still maintain high generation accuracy and stability, indicating that the constructed parametric modeling semantic space has good generalization ability.

[0139] Table 2 Comparison of generalization ability of CAD operation sequence autoencoder model and baseline method across datasets

[0140]

[0141] Secondly, in the performance evaluation of CAD generation under multimodal conditions, the method of this embodiment is compared and analyzed with existing condition-based CAD modeling methods. The generated CAD operation sequences and their corresponding 3D models are evaluated under different modal input combinations. In the evaluation results, each modal input is identified as follows: image represents multi-view image input; text represents text description input; Drawing(1x) represents single-view isometric engineering drawing data input; Drawing(3x) represents multi-view orthogonal three-view engineering drawing data input. The evaluation results are shown in Table 3. The results show that the method of this embodiment is superior to the comparative methods in terms of operation sequence prediction accuracy, parameter consistency, and the geometric rationality of the generated model. Especially under multimodal joint input conditions, it can fully utilize the complementary information between different modalities to generate a more accurate and stable parametric CAD model.

[0142] Table 3. Performance comparison of different methods in parametric CAD operation sequence generation under multimodal input combination conditions.

[0143]

[0144] also, Figure 6 A visual comparison diagram of the generation results of the method in this embodiment and the comparative method is shown under multimodal input conditions. As can be seen from the diagram, the CAD model generated by this embodiment is closer to the real model (GT) in both overall structure and local details. This demonstrates that the method in this embodiment can effectively reduce structural omissions and parameter deviation accumulation problems under complex structures and multiple constraints, exhibiting superior performance in both local detail representation and overall structural consistency.

[0145] In summary, the method proposed in this embodiment can achieve accurate modeling and stable generation of parametric CAD modeling semantics under multimodal input conditions, and has good engineering application value. It can be applied to scenarios such as intelligent CAD modeling, human-computer interaction design and automated modeling.

[0146] According to another aspect of this application, a CAD model generation system 70 based on multimodal semantic fusion and feature regression constraints is also provided, such as... Figure 7 As shown, the system includes:

[0147] The data preprocessing module 71 is used to acquire data and construct a multimodal dataset, which includes parameterized CAD operation sequences and their corresponding multimodal information. The multimodal information includes: multi-view images, text descriptions, and engineering drawing data.

[0148] Autoencoder module 72 is used to train an autoencoder model using CAD operation sequences and freeze the encoder and decoder of the autoencoder model after training. The encoder generates a latent semantic representation of the CAD operation sequence, and the decoder generates the corresponding CAD operation sequence via autoregression based on the input latent semantic representation.

[0149] The multimodal semantic fusion module 73 is used to fuse multimodal information and obtain the fusion result of multimodal information.

[0150] The fusion training module 74 is used to train a multimodal semantic fusion model by minimizing the fusion result of multimodal information and the latent semantic representation of CAD operation sequence;

[0151] The CAD kernel execution module 75 is used to acquire the CAD operation sequence obtained by the multimodal semantic fusion module and decoder after training the target data of the CAD model to be generated, so as to generate the CAD model using the CAD operation sequence. Specifically, the process of obtaining the corresponding CAD operation sequence from the target data of the CAD model to be generated is as follows: the target data of the CAD model to be generated is input into the trained multimodal semantic fusion model to obtain the fusion result corresponding to the target data; then, the fusion result corresponding to the target data is input into the trained decoder to obtain the CAD operation sequence.

[0152] The implementation process of the functions and roles of each module in the above system is detailed in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0153] For system embodiments, since they basically correspond to method embodiments, relevant details can be found in the descriptions of the method embodiments. The apparatus embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of the solution described in this specification, depending on actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0154] According to another aspect of this application, a computer-readable storage medium is provided, on which a computer program is stored, the program performing the steps of any of the methods described above when run on a processor.

[0155] Other embodiments of this specification will readily occur to those skilled in the art upon consideration of the specification and practice of the solutions applied herein. This specification is intended to cover any variations, uses, or adaptations that follow the general principles of this specification and include common knowledge or customary techniques in the art not claimed herein. The specification and embodiments are to be considered exemplary only.

[0156] It should be understood that this specification is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this specification is limited only by the appended claims.

[0157] The above description is merely a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of protection of this specification.

Claims

1. A CAD model generation method based on multi-modal semantic fusion and feature regression constraint, characterized in that, include: Step S1: Obtain a multimodal dataset, which includes parameterized CAD operation sequences and their corresponding multimodal information; The multimodal information includes: multi-view images, text descriptions, and engineering drawing data; Step S2: Train an autoencoder model using the CAD operation sequence, and freeze the encoder and decoder of the autoencoder model after training is completed; wherein, the encoder is used to generate a latent semantic representation of the CAD operation sequence, and the decoder is used to autoregressively generate the corresponding CAD operation sequence based on the input latent semantic representation. Step S3: The multimodal information is fused using a pre-built multimodal semantic fusion model to obtain the fusion result of the multimodal information; Step S4: Train the multimodal semantic fusion model by minimizing the fusion result of the multimodal information and the latent semantic representation of the CAD operation sequence; Step S5: Input the target data of the CAD model to be generated into the trained multimodal semantic fusion model to obtain the fusion result corresponding to the target data. Input the fusion result corresponding to the target data into the trained decoder to obtain the CAD operation sequence. Use the CAD operation sequence to generate the CAD model.

2. The CAD model generation method based on multi-modal semantic fusion and feature regression constraint according to claim 1, characterized in that, In step S3, constructing the multimodal semantic fusion model includes: constructing a multimodal feature encoding alignment module, a cross-modal semantic fusion module, and a feature mapping module; The multimodal feature encoding alignment module is used to extract and project features from the multimodal information to obtain a multimodal feature representation with unified dimensions. The cross-modal semantic fusion module utilizes the correlation between features of different modalities to adaptively fuse the multimodal feature representations to obtain a multimodal fused feature representation; The feature mapping module is used to perform feature mapping on the multimodal fusion feature representation to obtain the fusion result, which has the same dimension as the latent semantic representation.

3. The CAD model generation method based on multi-modal semantic fusion and feature regression constraint according to claim 2, characterized in that, The cross-modal semantic fusion module adopts an attention-based weighted fusion method to adaptively aggregate features from different modalities.

4. The CAD model generation method based on multi-modal semantic fusion and feature regression constraint according to claim 2, characterized in that, The feature mapping module uses a deep neural network structure to perform nonlinear mapping on the multimodal fusion feature representation.

5. The CAD model generation method based on multi-modal semantic fusion and feature regression constraint according to claim 1, characterized in that, The multimodal dataset includes multiple CAD instances, each of which contains a one-to-one parameterized CAD operation sequence and multimodal information.

6. The CAD model generation method based on multimodal semantic fusion and feature regression constraints according to claim 5, characterized in that, The CAD operation sequence includes a predefined set of operation commands, which includes contour start operation commands, geometric drawing operation commands, and 3D modeling operation commands.

7. The CAD model generation method based on multimodal semantic fusion and feature regression constraints according to claim 6, characterized in that, Each operation command includes a command type and command parameters. In step S2, the autoencoder model is trained using the CAD operation sequence, including: Each operation command in the CAD operation sequence is embedded with features in a combination of command type embedding, parameter embedding, and position embedding to obtain the embedded representation of the CAD operation sequence. The embedded representation of the CAD operation sequence is input into the encoder for feature encoding to obtain the latent semantic representation of the CAD operation sequence.

8. The CAD model generation method based on multimodal semantic fusion and feature regression constraints according to claim 1, characterized in that, The multi-view images are generated using uniformly preset view parameters; the engineering drawing data includes single-view isometric engineering drawings and multi-view orthogonal three-view engineering drawings.

9. A CAD model generation system based on multimodal semantic fusion and feature regression constraints, characterized in that, include: The data preprocessing module is used to acquire data and construct a multimodal dataset, which includes parameterized CAD operation sequences and their corresponding multimodal information. The multimodal information includes: multi-view images, text descriptions, and engineering drawing data; An autoencoder module is used to train an autoencoder model using the CAD operation sequence, and freeze the encoder and decoder of the autoencoder model after training is completed; wherein, the encoder is used to generate a latent semantic representation of the CAD operation sequence, and the decoder is used to autoregressively generate the corresponding CAD operation sequence based on the input latent semantic representation; A multimodal semantic fusion module is used to fuse the multimodal information to obtain a fusion result of the multimodal information; The fusion training module is used to train the multimodal semantic fusion model by minimizing the fusion result of the multimodal information and the latent semantic representation of the CAD operation sequence; The CAD kernel execution module is used to obtain the CAD operation sequence obtained by the multimodal semantic fusion module and the decoder after training the target data of the CAD model to be generated, so as to generate the CAD model using the CAD operation sequence.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is run on a processor, it performs the steps of the method according to any one of claims 1 to 8.

Images