A Method for Constructing Multimodal Parametric Datasets for Language-Driven 3D CAD Generation
By constructing a multimodal parameterized dataset, rendering multi-view composite images, and generating natural language descriptions, the problem of existing datasets being unable to provide strict alignment is solved, and efficient training and evaluation of 3D CAD generation models are achieved.
Patent Information
- Application Number
- CN202611233813.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-14
- Publication Date
- 2026-11-17
AI Technical Summary
Existing 3D model datasets cannot simultaneously provide strictly corresponding geometric appearance, parametric modeling process, and natural language description, making it impossible to train models that understand the mapping relationship between design intent and geometric operations. Furthermore, the data of each modality is stored in a scattered manner and lacks a unified retrieval and verification system.
A multimodal parametric dataset is constructed by acquiring native parametric 3D CAD part models, rendering multi-view composite images and extracting parametric modeling text, generating natural language descriptions using a large language model, establishing sample units and storing them in cross-modal associations, and achieving strict alignment between geometry, vision, language and the modeling process.
It achieves multimodal alignment at the sample level, and the generated natural language description is strictly aligned with the geometric entity of the model, providing direct supervision signals and improving the model's generalization ability on complex industrial parts.
Smart Images

Figure CN123116369A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer-aided design technology and discloses a method for constructing a multimodal parametric dataset for language-driven 3D CAD generation. Background Technology
[0002] Language-driven 3D CAD generation is a cutting-edge direction in the field of intelligent design and manufacturing. Its core lies in training a model that can directly convert natural language design intentions into a sequence of executable CAD commands. High-quality data is key to driving the training of such models, but existing 3D model datasets generally cannot simultaneously provide strictly corresponding geometric appearances, modeling steps, and natural language descriptions.
[0003] Currently widely used mesh and point cloud datasets, such as ModelNet and ShapeNet, only save the final 3D shape, completely lacking information about the modeling process. Parametric CAD datasets, such as ABC and Fusion360, while including modeling history, lack accompanying natural language descriptions and multi-view appearance images, making it impossible to train models that understand the mapping relationship between "design intent" and "geometric operations." Some Text-to-CAD datasets introduce text descriptions, but these texts are mostly template-generated, often containing semantic noise inconsistent with the actual geometric structure or modeling steps. A more prominent problem is that existing datasets have not established an organizational mechanism for strictly aligning part geometry files, multi-view appearance images, parametric modeling text, and natural language descriptions at the sample level. The data of each modality is stored in a scattered manner, lacking a unified retrieval and verification system, which severely restricts the training and evaluation of large-scale multimodal generative CAD models.
[0004] Therefore, there is an urgent need for a method that can systematically construct datasets that are multimodal aligned, semantically consistent, and have a hierarchical structure of complexity. Summary of the Invention
[0005] To address the problems existing in the prior art, this invention proposes a method for constructing a multimodal parametric dataset for language-driven 3D CAD generation, which involves the following steps performed by a computer device: S1: Obtain a valid native parametric 3D CAD part model; S2: Render the front view, rear view, left view, right view, top view, bottom view and isometric view of the effective native parametric 3D CAD part model, and combine the seven rendered views onto a canvas with a fixed arrangement rule to form a multi-view composite image. S3: Extract structured parametric modeling text from the effective native parametric 3D CAD part model. The parametric modeling text records the feature sequence, feature type, and parameter information of the modeling process. Using the multi-view composite image and the parametric modeling text as input, and using preset prompt word constraints, control the multimodal large language model to generate natural language description text. The natural language description text simultaneously describes the appearance, structure, and modeling process of the effective native parametric 3D CAD part model. S4: Establish sample units for the effective native parametric 3D CAD part model. The sample units are associated with and stored through a unified sample number, including the effective native parametric 3D CAD part model, the multi-view composite image, the parametric modeling text, and the natural language description text. S5: Based on the number of features recorded in the parametric modeling text, label the sample unit with a complexity level label.
[0006] Preferably, the effective native parametric 3D CAD part model is a model with an effective modeling history and containing at least one specific feature operation, which includes 2D sketch, 3D sketch, extrusion, extrusion cut, rotation, rotation cut, hole, reference plane, reference axis, chamfer, fillet, mirror array, and linear array.
[0007] More preferably, when rendering the front view, the rear view, the left view, the right view, the top view, the bottom view, and the isometric view, orthogonal projection is used, and a uniform camera distance and model bounding box scaling ratio are set so that the model geometry is completely within the view range.
[0008] More preferably, the canvas with a fixed arrangement rule is a 3×3 grid, the isometric view is placed in the center grid of the 3×3 grid, the top view and the bottom view are placed in the upper and lower grids of the center grid, respectively, the left view and the right view are placed in the left and right grids of the center grid, respectively, and the front view and the rear view are placed in the remaining corner grids of the 3×3 grid.
[0009] Further preferably, the preset prompt word constraints include: a first constraint, used to control the multimodal large language model to describe the overall appearance, outline, and symmetry of the effective native parametric 3D CAD part model based on the multi-view composite image; a second constraint, used to control the multimodal large language model to summarize the modeling process including stretching, cutting, rotation, and arraying based on the parametric modeling text; a third constraint, used to control the multimodal large language model to infer functional uses and prohibit the fabrication of assembly relationships or material grades that have no basis in the multi-view composite image and the parametric modeling text; and a fourth constraint, used to control the natural language description text output by the multimodal large language model to include appearance information, structural information, modeling process information, and possible functional information, and prohibit the output of codes and tables.
[0010] More preferably, the unified sample number is composed of a dataset version segment, a source code segment, a six-digit serial number segment, a level segment corresponding to the complexity level label, and a dataset partitioning label segment connected by a connector.
[0011] More preferably, the method further includes: An index file is constructed to store the cross-modal association structure. The fields of the index file include the unified sample number, the storage path of each modal file, the number of features, the complexity level label, the dataset partition label, and the verification status field. Through the index file, and using the unified sample number as the primary key, the various types of files within the sample unit are retrieved and associated.
[0012] Further preferably, the step of labeling the sample unit with a complexity level label based on the number of features recorded in the parametric modeling text includes: When the number of features is 1 to 5, the sample unit is labeled as simple level; when the number of features is 6 to 10, the sample unit is labeled as medium level; when the number of features is 11 to 100, the sample unit is labeled as advanced level; when the number of features is 101 or more, the sample unit is labeled as expert level.
[0013] More preferably, the method further includes: The sample units are subjected to multimodal alignment verification, which includes: The storage paths of the files of the valid native parametric 3D CAD part models, the files of the multi-view composite images, the files of the parametric modeling text, and the files of the natural language description text in the sample unit are all present and all associated with the same unified sample number. Verify that the file of the valid native parametric 3D CAD part model can be opened and that the feature tree can be completely traversed. Verify that the number of feature tree nodes recorded in the parametric modeling text is consistent with the number of valid feature tree nodes directly obtained from the valid native parametric 3D CAD part model; The specific geometric features recorded in the natural language description text can all be verified in the multi-view composite image or the parametric modeling text.
[0014] More preferably, the method further includes: obtaining the occurrence frequency of each CAD feature type as pre-statistically calculated; Based on the complexity level label and the occurrence frequency, all sample units that pass the multimodal alignment verification are divided to obtain a training set, a validation set, and a test set; During the partitioning process, if the number of sample units corresponding to a certain complexity level label in the training set is lower than the first threshold, then the sample units corresponding to that complexity level label are retained or oversampled; if the number of sample units corresponding to a certain CAD feature type in the training set is lower than the second threshold, then the sample units corresponding to that CAD feature type are retained or oversampled.
[0015] Technical effects: This invention achieves, for the first time, strict sample-level alignment of four modalities—geometric, visual, linguistic, and modeling process—by constructing associated original models, standard geometric models, multi-view composite images, parametric modeling text, and natural language descriptive text for the same part. It uses seven-view composite images to compress complete appearance information into a single regularly arranged canvas, reducing the length of multimodal input while avoiding inconsistent view scales. The parametric modeling text fully preserves feature order and dimensional parameters, providing direct supervision for generating executable CAD command sequences. A multimodal large language model, under four layers of cue word constraints, generates natural language text synchronously describing the appearance, structure, and modeling process, effectively eliminating semantic ambiguity between synthesized text and geometric entities. Complexity-based hierarchical annotation based on feature quantity provides structured difficulty labels for course learning, significantly improving the model's generalization ability on complex industrial parts. Attached Figure Description
[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0017] Figure 1 This is a flowchart illustrating the core steps of constructing the multimodal parameterized dataset for this invention. Figure 2This is a schematic diagram of the arrangement rules for the seven-view composite image synthesis of the present invention; Figure 3 This is a time-series interaction diagram of multimodal data generation and associated storage in this invention. Detailed Implementation
[0018] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.
[0019] It should be understood that, when used in this application specification and the appended claims, the term includes indicating the presence of the described feature, integral, step, operation, element and / or component, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.
[0020] Please see Figures 1-3 This technical solution provides a method for constructing a multimodal parametric dataset for language-driven 3D CAD generation. This method is executed by computer equipment and aims to systematically construct a structured dataset with strict alignment of four modalities: geometry, vision, language, and modeling process, providing a data foundation for training and evaluating language-driven 3D CAD generation models.
[0021] The technical problem addressed by this method is that existing 3D model datasets generally fail to simultaneously provide strictly corresponding geometric appearances, parametric modeling processes, and natural language descriptions. Mesh and point cloud datasets only save the final shape, lacking information about the modeling process. While parametric CAD datasets include modeling history, they lack accompanying natural language descriptions and multi-view appearance images. Some datasets that incorporate text descriptions often use template-generated text, resulting in semantic inconsistencies with the actual geometric structure or modeling steps. More critically, existing datasets lack a mechanism for strictly aligning part geometry files, multi-view appearance images, parametric modeling text, and natural language descriptions at the sample level. Modal data is stored fragmentedly, lacking a unified retrieval and verification system. These issues prevent the training of generative CAD models capable of accurately inferring the complete modeling intent from natural language.
[0022] To address the aforementioned issues, this method constructs a complete processing pipeline from data filtering, multimodal information extraction, semantic alignment to structured organization.
[0023] This method acquires valid native parametric 3D CAD part models. A valid native parametric 3D CAD part model is defined as a model with a valid modeling history and containing at least one specific feature operation. Specific feature operations include 2D sketches, 3D sketches, extrusions, extrusion cuts, revolving, revolving cuts, holes, reference planes, reference axes, chamfers, fillets, mirror arrays, and linear arrays. During the acquisition process, a computer device scans native parametric 3D CAD part model files in a specified storage path and performs a compliance check on each file. The rejection criteria for compliance checks include: file not existing, file extension or version not supported, error code returned when the CAD software opens the document, file corruption or requiring manual repair, model with no entities or empty entities, missing feature tree or containing only system folders, not containing any of the above specific feature operations, empty feature definition objects, missing critical dimension parameters, values that are NaN or outside a reasonable range, missing dependent sketches or referenced objects for hole features, array features, or mirror features, unresolved external references, model reconstruction failure, and failure to export to a standard exchange format. The rejected samples are written to an isolation log, which records the sample identifier, source path, failure stage, error code, failure feature name, and reason for rejection. Rejected samples are moved to an isolation directory and are not included in the training, validation, and test set divisions. The models retained through this screening process are the valid native parametric 3D CAD part models.
[0024] For the acquired valid native parametric 3D CAD part models, render the front, rear, left, right, top, bottom, and isometric views of the valid native parametric 3D CAD part models. When rendering the front, rear, left, right, top, bottom, and isometric views, orthographic projection is used, and a uniform camera distance and model bounding box scaling ratio are set to ensure that the model geometry is completely within the view area. The rendering resolution of each view is uniformly set. Before rendering, the model bounding box and view projection boundaries are calculated, controlling the longest projected edge of the part to within 90% of the view cell's width and height, with at least 5% white space around the edges. If a view's projection exceeds the cell boundary, it is scaled down proportionally without clipping or changing the viewpoint. Each view uses a uniform background color, uniform engineering material, uniform lighting intensity, and uniform shadow on / off state. The background color is neutral gray, the material is the default engineering material, and the lighting uses a standard three-point lighting arrangement. After rendering the seven views, they are composited onto a canvas with a fixed layout, forming a multi-view composite image. The canvas with the fixed layout is a 3x3 grid. The isometric view is placed in the center cell of the 3x3 grid. The top view is placed in the cell above the center cell. The bottom view is placed in the cell below the center cell. The left view is placed in the cell to the left of the center cell. The right view is placed in the cell to the right of the center cell. The front and back views are placed in the remaining corner cells of the 3x3 grid. The single-view rendering size is 512 pixels × 512 pixels, and the composite canvas size is 1536 pixels × 1536 pixels. During compositing, image data is confined to its respective grid; cross-grid overlap is prohibited. The multi-view composite image is output in PNG format, and view layout parameters, resolution parameters, scaling parameters, camera parameters, and material parameters are recorded. This process compresses the appearance information from seven directions into a single file, preserving the complete geometric appearance while avoiding the problem of the visual data sequence length increasing linearly with the number of views caused by inputting multiple images separately.
[0025] While obtaining a multi-view composite image, structured parametric modeling text is extracted from the effective native parametric 3D CAD part model. This parametric modeling text records the feature sequence, feature type, and parameter information of the modeling process. The extraction process reads the feature tree structure of the effective native parametric 3D CAD part model, traverses each node of the feature tree, and extracts the feature name, feature type, parent-child relationship, sketch plane identifier, geometric entity identifier, dimension value, direction vector, offset parameter, array parameter, chamfer parameter, fillet parameter, and hole parameter corresponding to each node. The extracted information is arranged according to the original operation order of the feature tree to generate structured text. In this structured text, each feature corresponds to a text record unit, and text record units are separated by delimiters. Within each text record unit, various information is recorded in key-value pair format. Parametric modeling text transforms the modeling process from a closed binary data format into a standard text format that can be directly used as a supervision signal for sequence generation tasks.
[0026] This method uses both the multi-view composite image and the parametric modeling text as input. Utilizing preset cue word constraints, it controls a multimodal large language model to generate natural language descriptive text. This natural language descriptive text simultaneously describes the appearance, structure, and modeling process of the effective native parametric 3D CAD part model. The multimodal large language model employs the Qwen2.5-VL-7B visual language model, which has dual-modal input channels for images and text, enabling joint processing of multi-view composite images and parametric modeling text. The preset cue word constraints include four layers of constraints. The first constraint controls the multimodal large language model to describe the overall appearance, contour, and symmetry of the effective native parametric 3D CAD part model based on the multi-view composite image. This constraint requires the model to identify the part's outline, hole and slot boss structure, symmetry features, and visible machining marks from the seven-view composite image. The second constraint controls the multimodal large language model to summarize the modeling process, including extrusion, cutting, rotation, and arraying, based on the parametric modeling text. The first constraint requires the model to describe the modeling steps—sketch creation, extrusion, cut, rotation, hole creation, chamfering, filleting, mirroring, and arraying—in the order of the feature tree in the parametric modeling text. The second constraint controls the inference of the multimodal large language model's intended use and prohibits the fabrication of assembly relationships or material grades that are not supported by the multi-view composite image or the parametric modeling text. This constraint allows the model to reasonably infer the functional use of parts based on geometric structure, but strictly prohibits the model from fabricating assembly mating relationships, material grades, load conditions, or manufacturing processes that are not visible in the image and exceed the given data range. The third constraint controls the natural language description text output by the multimodal large language model to include appearance information, structural information, modeling process information, and possible functional information, and prohibits the output of codes and tables. This constraint specifies the information organization structure of the output text, requiring the output natural language description text to be presented as a continuous natural language text, covering four parts: appearance description, structural description, modeling process description, and possible functional description. It also prohibits the model from outputting any explanatory text unrelated to the current sample, code blocks, or tables. The four-layer constraint, encompassing visual, structural, rule-based, and output format dimensions, jointly regulates the scope and quality of the generated content, ensuring strict alignment between the generated natural language description text, the geometric entities reflected in the multi-view composite image, and the modeling operations recorded in the parametric modeling text.
[0027] This method establishes sample units for the effective native parametric 3D CAD part models. These sample units are linked and stored using a unified sample number, which associates and stores the effective native parametric 3D CAD part model, the multi-view composite image, the parametric modeling text, and the natural language description text. The unified sample number is composed of a dataset version segment, a source code segment, a six-digit sequence number segment, a level segment corresponding to the complexity level label, and a dataset partitioning label segment connected by a connector. The dataset version segment identifies the release version of the dataset, the source code segment identifies the model's acquisition source, the six-digit sequence number is the deduplicated sequential number within the same source, the level segment corresponds to the complexity level label, and the partitioning label segment identifies the sample's affiliation in the training set, validation set, or test set. In one embodiment, a unified sample number is SNP01-GC-000123-L2-TR, where SNP01 represents the dataset version number, GC indicates the source is the GrabCAD platform, 000123 is the six-digit sequence number, L2 indicates a medium complexity level, and TR indicates that it belongs to the training set. Each modal file within a sample unit is named using a unified sample number followed by a modal suffix. For example, a valid native parametric 3D CAD part model is named 000123.sldprt, a multi-view composite image is named 000123.png, parametric modeling text is named 000123_encoder.txt, natural language description text is named 000123_des.txt, and a standard geometry exchange format file is named 000123.step. These five file types are associated through their sample numbers.
[0028] To support the retrieval and management of cross-modal data, this method constructs an index file storing the cross-modal association structure. The index file's fields include the unified sample ID, the storage path of each modality file, the number of features, the complexity level label, the dataset partition label, and a validation status field. The index file is stored in a structured data format, with each record corresponding to a sample unit. Using the index file and the unified sample ID as the primary key, various files within the sample unit are retrieved. When it is necessary to obtain all modality data for a sample, the unified sample ID is used as the primary key to query the index file, obtaining the storage paths of each modality file at once, and then the corresponding files are read according to the storage paths. The index file supports combined filtering and retrieval based on feature type, complexity level label, dataset partition label, and validation status field.
[0029] This method labels the sample units with complexity levels based on the number of features recorded in the parametric modeling text. The operation of labeling the sample units with complexity levels based on the number of features recorded in the parametric modeling text includes: when the number of features is 1 to 5, the sample unit is labeled as simple level; when the number of features is 6 to 10, the sample unit is labeled as medium level; when the number of features is 11 to 100, the sample unit is labeled as high level; and when the number of features is 101 or more, the sample unit is labeled as expert level. The complexity level labels are stored in the corresponding fields of the index file in a specified encoding format. Example encoding: L1 represents simple level, L2 represents medium level, L3 represents high level, and L4 represents expert level. Complexity stratification labels provide a difficulty ranking basis for course learning during the training process. They support progressively inputting training samples in the order of simple, medium, advanced, and expert levels, allowing the model to first learn the modeling logic of simple parts and then gradually learn the modeling logic of complex parts. During the model evaluation phase, complexity stratification labels support grouping and statistically analyzing evaluation metrics by level, allowing for the analysis of performance differences across different complexity levels.
[0030] This method performs multimodal alignment verification on the sample unit. The multimodal alignment verification includes multiple verification items. The first verification item verifies that the storage paths of the files of the valid native parametric 3D CAD part model, the multi-view composite image, the parametric modeling text, and the natural language description text in the sample unit all exist and are all associated with the same unified sample number. This verification item ensures that all modal files claimed to be included in the sample unit actually exist at the physical storage level, and that the association between each file and the unified sample number is correct. The second verification item verifies that the file of the valid native parametric 3D CAD part model can be opened and that the feature tree can be completely traversed. This verification item verifies that the model file has not been damaged after acquisition and that the feature tree structure is complete and readable by calling the CAD software interface to attempt to open the model file and traversing its internal feature tree structure. The third verification item verifies that the number of feature tree nodes recorded in the parametric modeling text is consistent with the number of valid feature tree nodes directly obtained from the valid native parametric 3D CAD part model. This verification item compares the feature node count recorded in the parametric modeling text with the number of valid nodes directly counted from the model feature tree. If the two values are equal, the verification passes; otherwise, the alignment is marked as failed, indicating that there may be feature omissions or duplicate feature records during the extraction process of the parametric modeling text. The fourth verification item verifies that the specific geometric features recorded in the natural language description text can all be verified in the multi-view composite image or the parametric modeling text. This verification item performs semantic parsing on the natural language description text, extracting the geometric feature description terms recorded therein, including hole feature description terms, groove feature description terms, boss feature description terms, chamfer feature description terms, rounded corner feature description terms, array feature description terms, and symmetry feature description terms. The extracted geometric feature description terms are compared with the visual analysis results of the multi-view composite image and the feature types recorded in the parametric modeling text. If a certain geometric feature description term cannot be confirmed by visual detection in the multi-view composite image and cannot be found in the parametric modeling text, then the description term is determined to be unsubstantiated information, and the natural language description text is marked as semantically inconsistent. If any of the four verification items fails, the alignment status of the corresponding sample unit will be marked as failed and written to the verification status field. Only sample units that pass the multimodal alignment verification can be used for subsequent dataset partitioning and model training.
[0031] This method obtains the pre-statistical frequency of occurrence of each CAD feature type. The statistical dimensions include the occurrence frequency of 13 core CAD feature categories: 2D sketch, 3D sketch, extrusion, extrusion cut, rotation, rotation cut, hole, reference plane, reference axis, chamfer, fillet, mirror array, and linear array. The statistical work traverses the parametric modeling text of all sample units, summarizes the feature type histogram of each sample unit, and calculates the total occurrence frequency and occurrence frequency of each feature type in all samples. After obtaining the occurrence frequency, based on the complexity level label and the occurrence frequency, all sample units that have passed the multimodal alignment verification are divided into training set, validation set, and test set. The division adopts a hierarchical sampling strategy. First, the units are divided according to the complexity level label. Within each complexity level, random sampling is performed based on the occurrence frequency distribution to ensure that the training set, validation set, and test set maintain a similar proportion to the overall distribution in both the complexity level dimension and the feature type dimension. During the partitioning process, if the number of sample units corresponding to a certain complexity level label in the training set is lower than a first threshold, then a retention or oversampling operation is performed on the sample units corresponding to that complexity level label. The retention operation means that all sample units corresponding to that complexity level label are included in the training set without any reduction in sampling. The oversampling operation means that sample units corresponding to that complexity level label are sampled with replacement, and the sampled copies are added to the training set until the number of sample units corresponding to that complexity level label in the training set reaches the first threshold. If the number of sample units corresponding to a certain CAD feature type in the training set is lower than a second threshold, then a retention or oversampling operation is performed on the sample units corresponding to that CAD feature type. The operation method is the same as the processing method described above for complexity level labels, and the sampling object is the sample units containing that CAD feature type. This mechanism ensures that the partitioned dataset maintains a reasonable distribution in both the difficulty and feature type dimensions, preventing the model training process from being dominated by high-frequency simple features and neglecting complex or scarce modeling operations.
[0032] The dataset constructed using the above method can support the training and evaluation of language-driven 3D CAD generation models. During the training phase, the natural language description text in the sample units is used as the input sequence of the model, and the parametric modeling text is used as the target sequence of the supervision signal. The model is trained to establish the mapping ability from natural language design intent to executable CAD modeling command sequences. Training can adopt a sequence-to-sequence Transformer model architecture. The input text is segmented and added to the positional encoding before being fed into the encoder. The encoder's self-attention mechanism learns the dependencies between semantic units in the natural language instructions. The decoder uses a cross-attention mechanism to focus on the relevant parts of the input instructions when generating each CAD command. The output generates a sequence of feature commands from the parametric modeling text token by token in an autoregressive manner. The training loss function uses cross-entropy loss.
[0033] During the evaluation phase, multiple metrics can be used to compare the CAD command text generated by the model with the standard parametric modeling text. The strict matching accuracy metric requires that the feature tree order, feature type sequence, and required fields of the generated text be completely consistent with the standard parametric modeling text. If they are completely consistent, it is considered a match; otherwise, it is considered a mismatch. The strict matching accuracy is calculated as the number of matched samples divided by the total number of samples.
[0034] The command-level F1 score metric treats each feature command in the generated text as a matching unit. It determines whether the matching unit is a positive class match based on three conditions: whether the feature type matches, whether the parent-child relationship matches, and whether the key field matches. The number of positive class matching units in the generated text is counted as the number of true positives, the number of non-positive class matching units in the generated text is counted as the number of false positives, and the number of feature commands that exist in the standard parametric modeling text but are not matched in the generated text is counted as the number of false negatives. Precision is calculated as the number of true positives divided by the sum of the number of true positives and false positives. Recall is calculated as the number of true positives divided by the sum of the number of true positives and false negatives. The command-level F1 score is calculated as twice the product of precision and recall divided by the sum of precision and recall.
[0035] The partial match rate metric is calculated as the ratio of the number of matched features to the number of standard features, with penalties applied for feature missing errors, feature duplication errors, feature order errors, and dependency errors. The penalty for feature missing errors is one unit deducted for each missing feature; for feature duplication errors, one unit deducted for each duplicate feature; for feature order errors, one unit deducted for each pair of adjacent features whose order deviates from the standard order; and for dependency errors, one unit deducted for each parent-child dependency that deviates from the standard dependency. After deducting the penalties for each type of error from the total score, the corrected match count is obtained. The partial match rate is the corrected match count divided by the number of standard features.
[0036] The above evaluation indicators, combined with complexity stratification labels, can be statistically analyzed separately for simple, medium, advanced, and expert levels to examine the differences in the model's generation capabilities at different complexity levels; they can also be statistically analyzed separately for feature types to examine the model's generation accuracy on different types of CAD features, providing data support for targeted model improvements.
[0037] In a specific embodiment, a bracket part from a publicly available industrial model library is used as the processing object, with its unified sample number SNP01-GC-000123-L2-TR. The computer device downloads the native parametric 3D CAD part model file 000123.sldprt of this bracket part, opens the model, and confirms that its feature tree contains sketch features, extrusion features, hole guide features, fillet features, and linear array features. The number of valid features recorded in the feature tree is 8, meeting the criteria for a medium-level judgment. The model is then rendered with seven views: front view, rear view, left view, right view, top view, bottom view, and isometric view. Each view uses orthographic projection, with a neutral gray background color and RGB values of 128, 128, and 128. The material is the default engineering material, and the lighting is a standard three-point light source arrangement. The camera distance and bounding box scaling are uniformly set to ensure the part is completely within the view cell. The seven views are arranged in a 3x3 grid to create a composite multi-view image, with the isometric view centered and the six orthogonal views arranged around it. The composite image is saved as PNG file 000123.png. Parametric modeling text is extracted from the model, and the feature tree is traversed to obtain feature order, feature type, parent-child relationship, and size parameters, generating the file 000123_encoder.txt. Both 000123.png and 000123_encoder.txt are input into the Qwen2.5-VL-7B model, generating natural language description text 000123_des.txt under four layers of cue word constraints. A sample unit was created for this support component. The files 000123.sldprt, 000123.step, 000123.png, 000123_encoder.txt, and 000123_des.txt were stored and associated with a unified sample number. The index file recorded the file path, number of features (8), complexity level label (L2), partition label (TR), and verification status. Multimodal alignment verification was performed on the sample unit. This verified that each file path existed and was associated with the same sample number; that the model file could be opened and the feature tree could be fully traversed; that the number of feature tree nodes in the parametric modeling text matched the actual number of effective nodes in the model; and that the hole features, rounded corner features, and array features described in the natural language description text could be verified in the multi-view composite image and the parametric modeling text. The verification status was marked as passed. This sample unit constitutes a complete, searchable, trainable, and evaluable sample.
[0038] In another embodiment, the original CAD format is not limited to sldprt format; it can also be a parametric CAD format with a modeling history, such as f3d, fcstd, prt, or catpart formats. The standard geometry exchange format is not limited to step format; it can also be iges, brep, x_t, or sat formats. The number of views is not limited to seven; sectional views, enlarged views, or depth maps can be added according to the training task requirements. The natural language description text generation model is not limited to Qwen2.5-VL-7B; it can also use GPT series visual language models, BLIP series models, or LLaVA series models. The criterion for determining complexity hierarchy is not limited to the single indicator of the number of features; it can also be determined by combining command sequence length, number of sketch entities, number of parameters, or topological complexity. The model source is not limited to GrabCAD; it can also be collected from the McMaster-Carr industrial parts library, the FreeCAD community library, or an enterprise's internal model library.
[0039] Unless otherwise specified, the equipment components involved in the above embodiments are all conventional equipment components, and the connection methods and control methods involved are all conventional connection methods and control methods unless otherwise specified.
[0040] The present invention has been described in detail above with reference to the embodiments. However, those skilled in the art will understand that, without departing from the spirit of the present invention, various specific parameters in the above embodiments can be changed to form multiple specific embodiments, all of which are common variations of the present invention, and will not be described in detail here.
Claims
1. A method for constructing a multimodal parametric dataset for language-driven 3D CAD generation, characterized in that, The following steps are performed by the computer device: S1: Obtain a valid native parametric 3D CAD part model; S2: Render the front view, rear view, left view, right view, top view, bottom view and isometric view of the effective native parametric 3D CAD part model, and combine the seven rendered views onto a canvas with a fixed arrangement rule to form a multi-view composite image. S3: Extract structured parametric modeling text from the effective native parametric 3D CAD part model. The parametric modeling text records the feature sequence, feature type, and parameter information of the modeling process. Using the multi-view composite image and the parametric modeling text as input, and using preset prompt word constraints, control the multimodal large language model to generate natural language description text. The natural language description text simultaneously describes the appearance, structure, and modeling process of the effective native parametric 3D CAD part model. S4: Establish sample units for the effective native parametric 3D CAD part model. The sample units are associated with and stored through a unified sample number, including the effective native parametric 3D CAD part model, the multi-view composite image, the parametric modeling text, and the natural language description text. S5: Based on the number of features recorded in the parametric modeling text, label the sample unit with a complexity level label.
2. The method for constructing a multimodal parametric dataset for language-driven 3D CAD generation according to claim 1, characterized in that, The effective native parametric 3D CAD part model is a model with an effective modeling history and containing at least one specific feature operation, including 2D sketch, 3D sketch, extrusion, extrusion cut, rotation, rotation cut, hole, reference plane, reference axis, chamfer, fillet, mirror array, and linear array.
3. The method for constructing a multimodal parametric dataset for language-driven 3D CAD generation according to claim 1, characterized in that, When rendering the front view, rear view, left view, right view, top view, bottom view, and isometric view, orthogonal projection is used, and a uniform camera distance and model bounding box scaling ratio are set so that the model geometry is completely within the view range.
4. The method for constructing a multimodal parametric dataset for language-driven 3D CAD generation according to claim 1 or 3, characterized in that, The canvas with a fixed layout rule is a 3×3 grid. The isometric view is placed in the center cell of the 3×3 grid. The top view and the bottom view are placed in the cells above and below the center cell, respectively. The left view and the right view are placed in the cells to the left and right of the center cell, respectively. The front view and the back view are placed in the remaining corner cells of the 3×3 grid.
5. The method for constructing a multimodal parametric dataset for language-driven 3D CAD generation according to claim 1, characterized in that, The preset prompt word constraints include: a first constraint, used to control the multimodal large language model to describe the overall appearance, outline, and symmetry of the effective native parametric 3D CAD part model based on the multi-view composite image; a second constraint, used to control the multimodal large language model to summarize the modeling process including stretching, cutting, rotation, and arraying based on the parametric modeling text; a third constraint, used to control the multimodal large language model to infer functional uses and prohibit the fabrication of assembly relationships or material grades that are not based on either the multi-view composite image or the parametric modeling text; and a fourth constraint, used to control the natural language description text output by the multimodal large language model to include appearance information, structural information, modeling process information, and possible functional information, and prohibit the output of codes and tables.
6. The method for constructing a multimodal parametric dataset for language-driven 3D CAD generation according to claim 1, characterized in that, The unified sample number is composed of a dataset version segment, a source code segment, a six-digit serial number segment, a level segment corresponding to the complexity level label, and a dataset partitioning label segment connected by a connector.
7. The method for constructing a multimodal parametric dataset for language-driven 3D CAD generation according to claim 6, characterized in that, The method further includes: constructing an index file for storing cross-modal association structures, wherein the fields of the index file include the unified sample number, the storage path of each modal file, the number of features, the complexity level label, the dataset partition label, and the verification status field; and using the index file and the unified sample number as the primary key, performing association retrieval on various types of files within the sample unit.
8. The method for constructing a multimodal parametric dataset for language-driven 3D CAD generation according to claim 1, characterized in that, The method of labeling the sample unit with a complexity level based on the number of features recorded in the parametric modeling text includes: labeling the sample unit as simple level when the number of features is 1 to 5; labeling the sample unit as medium level when the number of features is 6 to 10; labeling the sample unit as high level when the number of features is 11 to 100; and labeling the sample unit as expert level when the number of features is 101 or more.
9. The method for constructing a multimodal parametric dataset for language-driven 3D CAD generation according to claim 1, characterized in that, The method further includes: performing multimodal alignment verification on the sample unit, wherein the multimodal alignment verification includes: verifying that the storage paths of the files of the valid native parametric 3D CAD part models, the multi-view composite images, the parametric modeling text, and the natural language description text in the sample unit all exist and are all associated with the same unified sample number; verifying that the file of the valid native parametric 3D CAD part models can be opened and the feature tree can be completely traversed; verifying that the number of feature tree nodes recorded in the parametric modeling text is consistent with the number of valid feature tree nodes directly obtained from the valid native parametric 3D CAD part models; and verifying that the specific geometric features recorded in the natural language description text can all be verified in the multi-view composite images or the parametric modeling text.
10. The method for constructing a multimodal parametric dataset for language-driven 3D CAD generation according to claim 9, characterized in that, The method further includes: obtaining the pre-statistical occurrence frequency of each CAD feature type; Based on the complexity level label and the occurrence frequency, all sample units that pass the multimodal alignment verification are divided to obtain a training set, a validation set, and a test set; During the partitioning process, if the number of sample units corresponding to a certain complexity level label in the training set is lower than the first threshold, then the sample units corresponding to that complexity level label are retained or oversampled; if the number of sample units corresponding to a certain CAD feature type in the training set is lower than the second threshold, then the sample units corresponding to that CAD feature type are retained or oversampled.
