Semantic driving and multi-modal retrieval-based mechanical assembly generation method

Abstract

The invention discloses a mechanical assembly generation method based on semantic driving and multi-modal retrieval, and belongs to the field of computer aided design. The method comprises the following steps: constructing a multi-modal data set containing a mechanical part geometric file, B-Rep model data, a professional and amateur dual-channel description text and ISO standard information; in combination with RAG and CLIP retrieval strategies, a natural language instruction of a user is accurately mapped into candidate three-dimensional parts; an assembly relation and a pose matrix between the parts are calculated through a JoinAble assembly inference engine; automatic visualization generation of the assembly is realized by means of a three-dimensional software API, and key engineering indexes are controlled. The method reduces the professional threshold of mechanical design, simplifies the traditional tedious process, ensures that the assembly meets the engineering standard and physical rationality, and solves the problems that the traditional CAD depends on professional knowledge and the natural language intention is difficult to convert in the prior art.

Description

Technical Field

[0001] This invention relates to the field of computer-aided design technology, and in particular to a method for generating mechanical assemblies based on semantic-driven and multimodal retrieval. Background Technology

[0002] In the field of mechanical design and manufacturing, 3D assembly modeling is a core step in the development of complex products, directly impacting product iteration efficiency and market responsiveness. With the popularization of agile manufacturing concepts, traditional design models that rely on professional experience and involve cumbersome processes are no longer suitable for rapid R&D needs. Lowering the technical threshold of assembly design and automating the design process have become key directions for promoting efficient development in the industry. At the same time, they can also allow non-professional users to participate in the design process, further expanding the application scenarios of design.

[0003] Currently, 3D assembly modeling mainly relies on computer-aided design software. Traditional methods construct part geometric models through boundary representation, requiring designers to manually define joint constraints between parts. While this ensures engineering accuracy, it suffers from high design barriers and long iteration cycles. In recent years, assembly automation research has focused on optimizing geometric constraint solving algorithms. Some solutions attempt to combine data-driven methods to predict part connection relationships. However, existing technologies have failed to effectively and accurately translate the user's natural language design intent into specific geometric models and assembly relationships. Furthermore, when handling complete, industry-standard assembly processes, they lack an understanding of global functional semantics and standard assembly procedures, making it difficult to cope with fuzzy design instructions.

[0004] Based on the limitations of existing technologies, this application aims to solve the core technical problems: first, how to lower the professional technical threshold of 3D assembly design so that non-professional users can initiate design requirements through natural language; second, how to achieve efficient and automatic conversion from natural language intent to precise geometric assembly, simplifying the traditional cumbersome manual operation process. Summary of the Invention

[0005] To address the problems existing in the background art, one aspect of the present invention proposes a method for generating mechanical assemblies based on semantic-driven and multimodal retrieval, comprising:

[0006] S1: Construct a multimodal dataset, which includes: geometric files of mechanical parts, B-Rep model data, professional description text, amateur description text, ISO standard information, and known assembly information of mechanical parts;

[0007] S2: Based on the query text entered by the user, a retrieval strategy that combines RAG and CLIP is adopted to recall matching candidate mechanical parts from the multimodal dataset and return the known assembly information containing the candidate mechanical parts.

[0008] S3: Extract all associated mechanical parts from the known assembly information containing candidate mechanical parts as key mechanical parts, and construct a set of key mechanical parts;

[0009] S4: Use the JoinAble inference engine to infer the assembly relationships between key mechanical parts. Based on the inference results, import the geometric files of the key mechanical parts into the 3D software for assembly, and then display the assembled assembly to the user.

[0010] Another aspect of the present invention provides a mechanical assembly generation system based on semantic-driven and multimodal retrieval, the system comprising a memory and a processor; the memory is used to store an application program; the processor is used to run the application program and execute the mechanical assembly generation method based on semantic-driven and multimodal retrieval as described above.

[0011] Another aspect of the present invention provides a computer storage medium storing a remote monitoring program, which, when executed by a processor, implements the aforementioned method for generating mechanical assemblies based on semantic-driven and multimodal retrieval.

[0012] The present invention has at least the following beneficial effects

[0013] This invention constructs a multimodal parts dataset containing both professional and amateur text descriptions, and combines a retrieval strategy integrating RAG and CLIP. This achieves accurate mapping of user natural language commands to 3D parts models, effectively lowering the professional technical threshold for mechanical design. Non-professional users can express their design intentions using only everyday language without needing to master complex technical terms, thus solving the problem of traditional CAD design's heavy reliance on professional knowledge. Relying on the JoinAble assembly inference engine, it can automatically calculate the assembly relationships and pose matrices between key parts, and then combine it with 3D software APIs to achieve end-to-end intelligent generation of assemblies. This greatly simplifies the tedious process of manually defining part constraints and repeatedly adjusting poses in traditional design, significantly improving assembly design efficiency and adapting to the rapid design iteration needs of agile manufacturing. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0015] Figure 2 This is a diagram illustrating the similarity between the image-coded features of a mechanical part and the user-coded text features. Detailed Implementation

[0016] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0017] Please see Figure 1 This invention provides a method for generating mechanical assemblies based on semantic-driven and multimodal retrieval, comprising:

[0018] S1: Construct a multimodal dataset, which includes: geometric files of mechanical parts, B-Rep model data, professional description text, amateur description text, ISO standard information, and known assembly information of mechanical parts;

[0019] Furthermore, the known assembly information of the mechanical parts includes: assembly name, index information of all mechanical parts that make up the assembly, and assembly order of all mechanical parts that make up the assembly.

[0020] Further, step S1 includes:

[0021] S11: Input the geometric file of the mechanical part into the B-Rep model to extract and generate the B-Rep model data of the mechanical part;

[0022] S12: By standardizing the rendering of the geometric files of mechanical parts, multiple views of the rendered mechanical parts are extracted. These multiple views are then input into the Tongyi Qianwen large model to generate professional and amateur descriptive texts for the mechanical parts.

[0023] S13: Obtain ISO standard information for mechanical parts using OCC software based on the geometric files of the mechanical parts.

[0024] In this embodiment, the geometric files of the mechanical parts are mainly derived from two publicly available engineering datasets: Fusion 360 Gallery Assemblies and Machining Feature Dataset. These two datasets cover a variety of common mechanical parts such as screws, gears, shafts, and brackets, and the models conform to industrial design standards, meeting the assembly needs of multiple scenarios. After obtaining the files, preprocessing is required: all models are uniformly converted to .obj format, broken surfaces are repaired, surface normals are unified, and the origin of the local coordinate system is reset to lay the foundation for subsequent geometric analysis and data construction. At the same time, in the publicly available engineering datasets such as Fusion 360Gallery Assemblies and Machining Feature Dataset, by inputting the key identifier of the target part, the dataset will return all assembly model entries containing that part. Each entry includes information such as the assembly name, part list, and assembly order. The assembly model or document can be downloaded directly to view the details. In this embodiment, it is only based on an example for illustration. There is also a publicly available design website where assembled assembly information can be obtained. In actual operation, the known assembly information of mechanical parts can be defined for updating.

[0025] In this embodiment, the M4×10mm socket head cap screw .obj file is used as an example to explain step S1 in detail;

[0026] In step S11, the M4×10mm socket head cap screw .obj file is first input into the B-Rep model analysis tool (such as OpenCASCADE). The topology and geometry analysis module automatically extracts two types of core data:

[0027] 1. Topological relationship data:

[0028] Surface: Identify 5 types of functional surfaces: "head cylindrical surface", "head top surface", "internal hexagonal groove surface", "threaded cylindrical surface", and "threaded end face", and record the type (cylindrical surface, plane, hexagonal surface) and adjacency relationship of each surface (e.g., "head cylindrical surface and head top surface are perpendicularly connected").

[0029] Edges: Extract edge features at the intersection of each face, such as "the annular edge between the head cylindrical surface and the top surface" and "the hexagonal edge of the inner hexagonal groove", and record the length and curvature (straight edge / curved edge) of the edge.

[0030] Vertex: Locates the coordinates of key vertices, such as the six vertices of an internal hexagonal groove or the center point of the end face of a threaded cylinder.

[0031] 2. Geometric parameter data

[0032] Key dimensions: cylindrical surface radius (4mm, corresponding to "M4" specification), head thickness (3mm), thread length (10mm), internal hexagonal groove depth (2mm).

[0033] Geometric characteristics: thread profile angle (60°), coaxiality error of the head cylindrical surface (≤0.1mm).

[0034] Ultimately, this data is stored in a structured format (such as JSON) as the core basis for "part geometric feature matching" in subsequent assembly reasoning.

[0035] In step S12, generate dual-channel description text.

[0036] 1. View rendering: Use Blender software to perform "standardized multi-view rendering" on the preprocessed .obj model: fix the rendering parameters (resolution 1024×1024, lighting intensity, background color) and generate three view images: front view (showing the head and thread post of the screw orthographic projection), side view (showing the head thickness and thread length), and top view (showing the hexagonal shape of the internal hexagonal groove), to ensure that the key appearance features of the part are covered.

[0037] 2. Text Generation: Input the three rendered views into the Tongyi Qianwen multimodal large model to generate two types of descriptive text:

[0038] Professional description: Focusing on engineering terminology, standards and precise parameters, the content is "M4×10mm socket head cap screw, conforming to ISO4762 standard, thread tolerance 6g, performance grade 8.8, head diameter 8mm, thickness 3mm, socket head cap recess across flats dimension 2.5mm";

[0039] Amateur description: Focusing on everyday cognitive dimensions (appearance, use, tools used), the content is "a small silver metal screw with a cylindrical head, a hexagonal recess at the top, and a spiral pattern on the shaft. It needs to be tightened with a hex wrench and is commonly used to fix mechanical parts."

[0040] In step S13, ISO standard information is obtained: The .obj geometric file of the screw is parsed using the occ software (an open-source toolkit of OpenCASCADE). The software's built-in "Industrial Standard Matching Module" is called to automatically extract the ISO standard information that is compatible with the screw. This information mainly includes: standard code (ISO4762), core standard parameters (performance grade (8.8, representing tensile strength ≥800MPa, yield strength ≥640MPa), dimensional tolerances (thread diameter tolerance ±0.015mm, head thickness tolerance ±0.1mm)), and material compatibility (recommended compatible materials (such as 45# steel, stainless steel 304), which must meet the "mechanical properties and corrosion resistance requirements" in the standard).

[0041] In this embodiment, step S1 selects publicly available engineering datasets conforming to industry standards as the source of geometric files for mechanical parts. Standardized preprocessing ensures data consistency. Then, relying on the B-Rep model, the Tongyi Qianwen large-scale model, and OCC software, the system extracts part geometric topology data, generates professional and amateur dual-channel descriptive text, obtains ISO standard information, and integrates known assembly information to construct a multi-dimensional, structured, multimodal dataset. This step not only provides rich and high-quality data support for subsequent accurate retrieval by integrating RAG and CLIP, but its dual-channel text description also adapts to the needs of users with different knowledge backgrounds. Furthermore, comprehensive part and assembly information provides complete contextual basis for subsequent assembly relationship reasoning, laying the foundation from a data perspective for the entire method to lower the design threshold and improve the accuracy and rationality of assembly generation.

[0042] S2: Based on the query text entered by the user, a retrieval strategy that combines RAG and CLIP is adopted to recall matching candidate mechanical parts from the multimodal dataset and return the known assembly information containing the candidate mechanical parts.

[0043] Further, step S2 includes:

[0044] S21: Input the user's query text into the sentence embedding model and encode it to obtain the user's query text vector; concatenate the professional description, amateur description and ISO standard information of the mechanical parts, and then encode them through the sentence embedding model to obtain the description text vector of the mechanical parts;

[0045] S22: Calculate the cosine similarity between the user's query text vector and the description text vector of the mechanical parts, and select the K mechanical parts with the highest similarity to construct the initial candidate parts set;

[0046] S23: Standardize the geometric files of mechanical parts in the initial candidate part set, extract multiple views of the rendered mechanical parts, stitch the multiple views of the rendered mechanical parts together, and input the stitched image into the image encoder of the CLIP model to obtain the image encoding features of the mechanical parts.

[0047] S24: Input the user's query text into the text encoder of the CLIP model to obtain the user's text encoding features; calculate the similarity between the image encoding features of the mechanical parts and the user's text encoding features, and select the mechanical parts with the highest similarity from the initial candidate parts set as candidate mechanical parts;

[0048] S25: Retrieve known assembly information containing candidate mechanical parts from known assembly information of mechanical parts.

[0049] Further, in step S25, if there is no known assembly information containing the candidate mechanical part, the B-Rep model data of the mechanical part is encoded using the BRepNet encoder to obtain the B-Rep encoding features of the mechanical part. The similarity between the B-Rep encoding features of the candidate mechanical part and the B-Rep encoding features of the other mechanical parts is calculated. The other mechanical parts with a similarity greater than a set threshold are selected as similar mechanical parts of the candidate mechanical part. The known assembly information containing the similar mechanical part is used as the known assembly information containing the candidate mechanical part.

[0050] This embodiment uses a specific example to illustrate the application in detail:

[0051] Suppose the user inputs the query text "Generate a simple bracket assembly for fixing a small motor". This text is input into a sentence embedding model (such as the Sentence-BERT model) to generate a query text vector. The professional description, amateur description, and ISO standard information of the mechanical parts are concatenated and then encoded by the sentence embedding model (such as the Sentence-BERT model) to obtain the description text vector of the mechanical parts. The cosine similarity between the query text vector and the description text vectors of all mechanical parts is calculated, and the K mechanical parts with the highest similarity are selected to construct the initial candidate part set.

[0052] For the geometric files of mechanical parts in the initial candidate part set, Blender renders the part views with fixed parameters. In this embodiment, the main views, side views, and top views are rendered. Of course, more detailed information such as perspective views can also be included, depending on the needs of those skilled in the art. The multi-view stitched image of each part is input into the CLIP image encoder to generate part image encoding features. The user's query text is input into the CLIP text encoder to generate user text encoding features, and the matching degree between the user text and the part image features is calculated. For details, please refer to [reference needed]. Figure 2 As shown, Figure 2This represents the matching degree between the text encoding features of different user query texts and the image encoding features of each part; the mechanical part with the highest similarity is selected as the candidate mechanical part, or multiple candidate mechanical parts with similarity greater than a set threshold can be selected; in this embodiment, taking a mechanical part as an example, candidate mechanical parts (such as the L-shaped bracket body) are filtered out. At this time, the assembly information containing (such as the L-shaped bracket body) is retrieved from the "Known Assembly Information" of the multimodal dataset. The assembly information can contain multiple parts, and multiple assembly information will be displayed to the user. The user selects the assembly they need as the known assembly information of the candidate mechanical part according to the name of each assembly; if multiple candidate mechanical parts are returned, the known assembly information containing all candidate mechanical parts is retrieved from the "Known Assembly Information" of the multimodal dataset and returned. The user selects the assembly they need as the known assembly information of the candidate mechanical part according to the name of each assembly.

[0053] In step S25, if there is no known assembly information containing the candidate mechanical part, the B-Rep model data of the mechanical part is encoded using the BRepNet encoder to obtain the B-Rep encoding features of the mechanical part. The similarity between the B-Rep encoding features of the candidate mechanical part and the B-Rep encoding features of the other mechanical parts is calculated. The other mechanical parts with a similarity greater than a set threshold are selected as similar mechanical parts of the candidate mechanical part. The known assembly information containing the similar mechanical part is used as the known assembly information containing the candidate mechanical part. That is, in this embodiment, the candidate mechanical part and its similar mechanical parts share assembly information. If an assembly contains a similar mechanical part of the candidate mechanical part, it can be considered to contain the candidate mechanical part. Based on this, the assembly information of mechanical parts with less assembly information can be supplemented.

[0054] In this embodiment, step S2 employs a two-stage retrieval strategy that integrates RAG and CLIP. First, it uses a sentence embedding model to semantically match the user's natural language query with the part's text information, initially filtering out candidate parts with high relevance. Then, the CLIP model performs image-text matching between the part's geometric view and the query text for finer filtering, effectively balancing semantic relevance and geometric adaptability. This solves the problems of traditional retrieval methods, such as difficulty in accurately converting natural language design intent and the tendency to overlook geometric shape matching. Furthermore, after retrieving candidate parts, this step returns information about known assemblies containing them, providing a complete context for subsequent construction of key part sets and inference of assembly relationships. Even if there is no directly related assembly information, the BRepNet encoder can match similar parts based on the part's B-Rep features, reusing the assembly information of similar parts, significantly improving the method's adaptability to data-sparse scenarios. Overall, step S2 builds a precise bridge connecting "user needs - candidate parts - assembly context" for the entire assembly generation process, ensuring the accuracy of part retrieval and laying the foundation for efficient progress in subsequent steps, further helping to reduce the reliance on professional knowledge in mechanical design.

[0055] S3: Extract all associated mechanical parts from the known assembly information containing candidate mechanical parts as key mechanical parts, and construct a set of key mechanical parts;

[0056] S4: Use the JoinAble inference engine to infer the assembly relationships between key mechanical parts. Based on the inference results, import the geometric files of the key mechanical parts into the 3D software for assembly, and then display the assembled assembly to the user.

[0057] Further, step S4 includes:

[0058] S41: Initialize i=1, take the first mechanical part in the key mechanical parts set as the first sub-assembly, and define the number of key mechanical parts in the key mechanical parts set as N;

[0059] S42: Input the B-Rep model data of the i-th sub-assembly and the (i+1)-th mechanical part in the key mechanical parts set into the JoinAble inference engine to predict the pose transformation matrix between the i-th sub-assembly and the (i+1)-th mechanical part.

[0060] S43: Input the geometry file of the i-th sub-assembly and the geometry file of the (i+1)-th mechanical part into the 3D software and assemble them according to the pose transformation matrix between the i-th sub-assembly and the (i+1)-th mechanical part to generate the (i+1)-th sub-assembly.

[0061] S44: Export the geometry file of the (i+1)th subassembly using 3D software, and input the geometry file of the (i+1)th subassembly into the B-Rep model to extract and generate the B-Rep model data of the (i+1)th subassembly;

[0062] S45: Determine if i is equal to N. If i is less than N, set i = i + 1 and return to step S42. If i = N, then display the Nth sub-assembly as the assembled assembly to the user.

[0063] In this embodiment, a practical example is used to explain step S4 in detail. It is assumed that through the above steps S1~S3, the key parts to be assembled and the preliminary sequence have been extracted from the known information of the "small motor simple bracket assembly": first, combine the gasket and screw, then assemble the screw with the gasket and the motor, and finally fix the motor (including screws and gaskets) to the bracket body to form a complete assembly; based on the known information, the assembly relationship reasoning and pose prediction are performed by the JoinAble inference engine;

[0064] 1. First round of reasoning: Calculation of the assembly position of the washer and screw.

[0065] Input data: Input the B-Rep model data (including screw diameter, washer inner hole size, and geometric topology of both) of “M5×15mm screw” and “motor damping pad” into the JoinAble inference engine.

[0066] Reasoning process: The engine analyzes the geometric features of the parts (the diameter of the screw rod and the inner diameter of the washer are compatible, both are cylindrical structures) to identify the assembly relationship of "wafer fitting into the screw rod"; at the same time, it predicts the pose transformation matrix - this matrix contains parameters such as "wafer rotates 0° around the screw axis (to ensure that the inner hole is coaxial with the screw rod) and translates 4mm along the screw axis towards the head (to make the washer close to the screw head and avoid loosening after assembly)" to ensure that the washer and screw are precisely fitted.

[0067] 2. Second round of reasoning: Calculation of the assembly posture of the screw with washer and the motor.

[0068] Input data: First, take "screws for fitting washers" as "subassembly 1" and extract its B-Rep model data; then input the B-Rep data of "small motor" (including motor mounting hole diameter and mounting surface position).

[0069] Reasoning process: The engine recognizes the assembly relationship that "the screw needs to pass through the motor mounting hole" and predicts the pose transformation matrix: Adjust the position of sub-assembly 1 (screw with washer) relative to the motor so that the screw rod is coaxial with the motor mounting hole and the washer is fully in contact with the motor mounting surface (to avoid direct contact between the motor and the screw, which would cause wear), while ensuring that the length of the screw rod extending out of the motor mounting hole meets the requirements for subsequent fixation with the bracket.

[0070] 3. Three-round reasoning: Calculation of the assembly posture of the motor (including screws and washers) and the main body of the bracket.

[0071] Input data: Take "motor + screw with washer" as "sub-assembly 2" and extract its B-Rep data; input the B-Rep data of "L-shaped bracket body" (including the position of the threaded holes on the bracket horizontal plate and the plate thickness).

[0072] Reasoning process: Based on the requirement that "the motor needs to be fixed on the horizontal plate of the bracket", the engine identifies the assembly relationship that "the screw needs to be screwed into the threaded hole of the bracket" and predicts the pose transformation matrix: adjust the position of sub-assembly 2 relative to the bracket so that the bottom of the motor is in contact with the horizontal plate of the bracket and the screw rod is precisely aligned with the threaded hole of the bracket (to ensure no offset when screwing in later), while ensuring that the position of the motor on the bracket meets the functional requirement of "stable support".

[0073] III. 3D Software Assembly and Final Display

[0074] 1. Importing geometry files and applying poses

[0075] Import the geometry files (.obj format) of all key parts into 3D software (such as Blender), and perform automated assembly according to the pose transformation matrix output by the JoinAble engine:

[0076] First, based on the first round matrix, fit the shim onto the screw to generate subassembly 1;

[0077] Then, based on the next round matrix, sub-assembly 1 is assembled with the motor to generate sub-assembly 2;

[0078] Finally, based on the three-wheel matrix, sub-assembly 2 is fixed to the main body of the support, completing the assembly of all parts.

[0079] 2. Visualization of Assembly

[0080] The 3D software automatically renders the complete assembly, showing users the final effect of "the L-shaped bracket supporting a small motor, which is fixed to the bracket by two M5 screws with shock-absorbing pads." Users can intuitively view the relative positions between parts and the completeness of the assembly. They can also use the software's interactive functions (such as rotating and zooming the view) to confirm whether the assembly meets the design intent from different angles, realizing "from reasoning results to the practical presentation of the visualized assembly."

[0081] In this embodiment, step S4 automatically analyzes the assembly relationships between key components and accurately predicts the pose transformation matrix using the JoinAble inference engine. This eliminates the need for manual definition of component constraints, effectively avoiding errors that can easily occur when manually adjusting poses in traditional design and significantly reducing reliance on specialized assembly knowledge. Its iterative assembly process, using sub-assemblies as units and synchronously updating B-Rep model data, ensures that each assembly step is based on accurate geometric information, preventing error accumulation and guaranteeing the physical rationality and structural stability of the final assembly. Simultaneously, relying on 3D software, it achieves automatic assembly generation and visualization, allowing users to intuitively verify the relative positions of parts and the integrity of the assembly. It also eliminates the tedious manual modeling and repeated debugging steps of traditional CAD, significantly improving assembly efficiency. This step effectively connects to the previously constructed set of key components, ultimately realizing the generation of a complete assembly from scattered parts. It provides core execution support for the entire method to achieve the goal of "end-to-end intelligent generation," ensuring that the generated assembly meets the accuracy and practicality requirements of engineering applications.

[0082] Another aspect of the present invention provides a mechanical assembly generation system based on semantic-driven and multimodal retrieval, the system comprising a memory and a processor; the memory is used to store an application program; the processor is used to run the application program and execute the mechanical assembly generation method based on semantic-driven and multimodal retrieval as described above.

[0083] Another aspect of the present invention provides a computer storage medium storing a remote monitoring program, which, when executed by a processor, implements the aforementioned method for generating mechanical assemblies based on semantic-driven and multimodal retrieval.

[0084] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0085] In summary, this invention constructs a multimodal parts dataset containing both professional and amateur text descriptions, and combines a retrieval strategy integrating RAG and CLIP. This achieves accurate mapping of user natural language commands to 3D parts models, effectively lowering the professional technical threshold for mechanical design. Non-professional users can express their design intent using only everyday language without needing to master complex technical terminology, thus solving the problem of traditional CAD design's heavy reliance on professional knowledge. Furthermore, relying on the JoinAble assembly inference engine, it can automatically calculate the assembly relationships and pose matrices between key parts, and combined with 3D software APIs, achieve end-to-end intelligent generation of assemblies. This significantly simplifies the tedious process of manually defining part constraints and repeatedly adjusting poses in traditional design, significantly improving assembly design efficiency and adapting to the rapid iteration needs of agile manufacturing.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for generating mechanical assemblies based on semantic-driven and multimodal retrieval, characterized in that, include: S1: Construct a multimodal dataset, which includes: geometric files of mechanical parts, B-Rep model data, professional description text, amateur description text, ISO standard information, and known assembly information of mechanical parts; S2: Based on the query text entered by the user, a retrieval strategy that combines RAG and CLIP is adopted to recall matching candidate mechanical parts from the multimodal dataset and return the known assembly information containing the candidate mechanical parts. S3: Extract all associated mechanical parts from the known assembly information containing candidate mechanical parts as key mechanical parts, and construct a set of key mechanical parts; S4: Use the JoinAble inference engine to infer the assembly relationships between key mechanical parts. Based on the inference results, import the geometric files of the key mechanical parts into the 3D software for assembly, and then display the assembled assembly to the user.

2. The method for generating mechanical assemblies based on semantic-driven and multimodal retrieval according to claim 1, characterized in that, The known assembly information of the mechanical parts includes: assembly name, index information of all mechanical parts that make up the assembly, and assembly order of all mechanical parts that make up the assembly.

3. The method for generating mechanical assemblies based on semantic-driven and multimodal retrieval according to claim 1, characterized in that, Step S1 includes: S11: Input the geometric file of the mechanical part into the B-Rep model to extract and generate the B-Rep model data of the mechanical part; S12: By standardizing the rendering of the geometric files of mechanical parts, multiple views of the rendered mechanical parts are extracted. These multiple views are then input into the Tongyi Qianwen large model to generate professional and amateur descriptive texts for the mechanical parts. S13: Obtain ISO standard information for mechanical parts using OCC software based on the geometric files of the mechanical parts.

4. The method for generating mechanical assemblies based on semantic-driven and multimodal retrieval according to claim 1, characterized in that, Step S2 includes: S21: Input the user's query text into the sentence embedding model and encode it to obtain the user's query text vector; concatenate the professional description, amateur description and ISO standard information of the mechanical parts, and then encode them through the sentence embedding model to obtain the description text vector of the mechanical parts; S22: Calculate the cosine similarity between the user's query text vector and the description text vector of the mechanical parts, and select the K mechanical parts with the highest similarity to construct the initial candidate parts set; S23: Standardize the geometric files of mechanical parts in the initial candidate part set, extract multiple views of the rendered mechanical parts, stitch the multiple views of the rendered mechanical parts together, and input the stitched image into the image encoder of the CLIP model to obtain the image encoding features of the mechanical parts. S24: Input the user's query text into the text encoder of the CLIP model to obtain the user's text encoding features; calculate the similarity between the image encoding features of the mechanical parts and the user's text encoding features, and select the mechanical parts with the highest similarity from the initial candidate parts set as candidate mechanical parts; S25: Retrieve known assembly information containing candidate mechanical parts from known assembly information of mechanical parts.

5. The method for generating mechanical assemblies based on semantic-driven and multimodal retrieval according to claim 4, characterized in that, In step S25, if there is no known assembly information containing the candidate mechanical part, the B-Rep model data of the mechanical part is encoded using the BRepNet encoder to obtain the B-Rep encoding features of the mechanical part. The similarity between the B-Rep encoding features of the candidate mechanical part and the B-Rep encoding features of the other mechanical parts is calculated. The other mechanical parts with a similarity greater than a set threshold are selected as similar mechanical parts of the candidate mechanical part. The known assembly information containing the similar mechanical part is used as the known assembly information containing the candidate mechanical part.

6. The method for generating mechanical assemblies based on semantic-driven and multimodal retrieval according to claim 1, characterized in that, Step S4 includes: S41: Initialize i=1, take the first mechanical part in the key mechanical parts set as the first sub-assembly, and define the number of key mechanical parts in the key mechanical parts set as N; S42: Input the B-Rep model data of the i-th sub-assembly and the (i+1)-th mechanical part in the key mechanical parts set into the JoinAble inference engine to predict the pose transformation matrix between the i-th sub-assembly and the (i+1)-th mechanical part. S43: Input the geometry file of the i-th sub-assembly and the geometry file of the (i+1)-th mechanical part into the 3D software and assemble them according to the pose transformation matrix between the i-th sub-assembly and the (i+1)-th mechanical part to generate the (i+1)-th sub-assembly. S44: Export the geometry file of the (i+1)th subassembly using 3D software, and input the geometry file of the (i+1)th subassembly into the B-Rep model to extract and generate the B-Rep model data of the (i+1)th subassembly; S45: Determine if i is equal to N. If i is less than N, set i = i + 1 and return to step S42. If i = N, then display the Nth sub-assembly as the assembled assembly to the user.

7. A mechanical assembly generation system based on semantic-driven and multimodal retrieval, characterized in that, The system includes a memory and a processor; the memory is used to store an application program; the processor is used to run the application program and execute a method for generating mechanical assemblies based on semantic-driven and multimodal retrieval as described in any one of claims 1 to 6.

8. A computer storage medium, characterized in that, The computer storage medium stores a remote monitoring program, which, when executed by a processor, implements a method for generating mechanical assemblies based on semantic-driven and multimodal retrieval as described in any one of claims 1 to 6.

Images