Engineering design drawing auxiliary method based on multi-modal large model

Abstract

The invention relates to the technical field of intelligent engineering design assistance, in particular to an engineering design drawing auxiliary method based on a multi-modal large model, which comprises the following steps of: constructing a multi-source knowledge base containing document text class and drawing class knowledge sub-libraries based on a retrieval enhancement generation technology; the structured storage is carried out through semantic analysis, entity modeling and information extraction; the system is deployed in AutoCAD and Civil3D in a plug-in form, and supports button or text interaction. Receiving a to-be-processed drawing and a professional specification range, cooperatively identifying an entity through a CAD interface and a multi-mode large model, comparing retrieval specifications, outputting a drawing checking result containing error positioning and deviation early warning, and supporting case recommendation of cosine distance matching; the problems that in the prior art, the image feature extraction and image-text alignment precision is insufficient, the multi-modal information fusion performance is poor, and the design information processing efficiency is low are solved, and the engineering design drawing examination accuracy and the design scheme generation efficiency are improved.

Description

Technical Field

[0001] This invention belongs to the field of intelligent engineering design assistance technology, specifically relating to an engineering design drawing assistance method based on a multimodal large model. Background Technology

[0002] In the field of engineering construction, engineering design drawings (covering scenarios such as building construction, hydropower stations, and highways) are the core carriers for conveying design intent and guiding construction. Their accuracy and efficiency directly determine project quality and timeline. With the increasing specialization of the industry, design specifications have become increasingly complex, requiring adherence to national / industry-wide standards as well as adaptation to project-specific requirements (such as seismic resistance, fire resistance ratings, and regional climate). Furthermore, different disciplines interpret and implement these specifications significantly differently. The existing design process suffers from two major pain points: first, manual review of drawings relies on experience, is time-consuming and labor-intensive, and prone to errors due to omissions in specifications or oversights in details, resulting in high subsequent modification costs; second, designers must manually sift through a massive number of drawings to find reference cases, making it difficult to quickly match successful cases similar to the current project (geographical environment, design type, professional category), thus hindering efficiency and innovation. Specifically, this includes: Existing patent document 1 (publication number: CN119323300A) discloses a multi-model management system and method for construction schemes. This system, based on a natural language processing model of construction schemes, consists of four subsystems: a multi-modal professional knowledge question-and-answer system for the construction industry, a professional document compilation system for the construction industry, a multi-modal retrieval system for the construction industry, and a construction scheme content review system. Among these, the professional document compilation system for the construction industry employs multi-modal diffusion model technology, and the established construction image generation system can intelligently respond to user-inputted text requests and automatically generate matching construction images. However, this system still suffers from insufficient accuracy in the image feature extractor and image-text feature aligner within the image analysis module. Existing patent document 2 (publication number: CN118779469A) discloses a method for constructing a domain-wide large-scale model multimodal knowledge base based on feature representation. This method includes collecting and processing information for constructing the knowledge base, extracting features from multimodal information to construct the multimodal knowledge base, and outputting the query results requested by the user based on the domain-wide large-scale model generation capability. However, this system still suffers from performance deficiencies in the fusion and interaction methods of multimodal information.

[0003] In summary, existing patents and conventional technologies in the industry, due to the aforementioned deficiencies in accuracy and fusion performance, are unable to meet the engineering design field's needs for accurate intelligent drawing review and practical design scheme recommendations.

[0004] In view of this, the present invention is hereby proposed. Summary of the Invention

[0005] To address the aforementioned technical problems in the existing technology, this invention provides an engineering design drawing assistance method based on a multimodal large model, which solves the problems of insufficient image feature extraction and image-text alignment accuracy, poor multimodal information fusion performance, and low design information processing efficiency in the existing technology.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows: A method for assisting in the drawing of engineering design diagrams based on a multimodal large model, comprising: S1. Construct a multi-source knowledge base based on retrieval enhancement generation technology. The knowledge sources of the multi-source knowledge base include: document, text and drawing knowledge. Use corresponding knowledge extraction and processing methods for different knowledge to complete knowledge storage. S2. Receive the user-inputted engineering design drawing to be processed; S3. Perform entity recognition on the engineering design drawing to be processed, and obtain the attribute information of the entities in the drawing; S4. Based on the attribute information of the entity, retrieve matching design specification-related knowledge from the multi-source knowledge base; S5. Call the multimodal large model, compare the attribute information of the entity with the retrieved design specification-related knowledge, determine whether the engineering design drawing to be processed conforms to the design specification, and output the review result including errors and error types.

[0007] Furthermore, the knowledge extraction and processing methods for the aforementioned documents and text-based knowledge include: Preprocess document and text-based knowledge: convert all types of knowledge into plain text in a uniform format and filter out irrelevant information such as headers, footers, and advertisements; The preprocessed long text is divided into blocks: fixed window block division, sliding window block division, or semantic-based block division is used to divide it into text blocks of a preset byte length. Text embedding processing is performed on the segmented text blocks: the text blocks are converted into high-dimensional vectors that capture semantic information, and the high-dimensional vectors are stored in the retrieval database to form document and text-based knowledge sub-bases.

[0008] Furthermore, the text embedding process employs a pre-trained model, which is OpenAItext-embedding-ada-002, BERT, or Sentence-BERT; The retrieval database is pgVector or Milvus; and the multi-source knowledge base is configured with a dynamic update interface, supporting the addition or deletion of knowledge in the knowledge base.

[0009] Furthermore, the fixed window segmentation involves sequentially extracting text blocks of corresponding lengths from the original text at fixed lengths. The sliding window is divided into blocks: text blocks of corresponding lengths are extracted from the original text in sequence with a fixed length, and there is overlap between the text blocks; The semantic-based segmentation: segmenting by semantic units to ensure semantic coherence and independence of text blocks.

[0010] Furthermore, the methods for knowledge extraction and processing of the aforementioned drawing-related knowledge include: The semantics of the drawings are parsed from the structured data of the drawings to identify detailed information of entity objects in the drawings: the detailed information of entity objects includes entity category and entity parameter information; Analyze the relationships between entity objects and establish a model of the compositional relationships between entities; Extract the project information corresponding to the drawings: The project information includes design specifications and geographical and climatic information; The detailed information of the entity objects, the model of the combination relationship between entities, and the project information are stored in a multi-source knowledge base to form a drawing-type knowledge sub-base.

[0011] Furthermore, the design specifications include: seismic resistance rating, fire resistance rating, and compressive strength rating; The geographical and climatic information includes: geological information and climate information of the project location.

[0012] Furthermore, when outputting the review results, which include errors and error types, the following is also included: Mark the location of errors in the drawing canvas of the engineering drawing tool and clearly inform the user of the error type: the error types include basic drawing errors and errors that do not conform to design specifications; The system analyzes and compares the design drawings of the project to be processed with the drawings of successful cases in a multi-source knowledge base, and outputs a design deviation warning prompt between the current drawing and the drawings of successful cases.

[0013] Furthermore, it also includes self-updating of the knowledge base: Receive user confirmation and correction instructions for the drawing review results and design deviation warning prompts: if the user confirms that the error is real, prompt the user to modify the drawing; if the user determines that it is a misjudgment, receive the misjudgment feedback information submitted by the user. User correction commands and misjudgment feedback are fed into a multi-source knowledge base to update the comparison parameters and knowledge base content of the multimodal large model, thereby optimizing the accuracy of subsequent image review.

[0014] Furthermore, it also includes design case recommendations: Receive text instructions input by the user: The text instructions include project design specifications, project geographical and climatic information, and reference case category requirements; The text instruction is converted into a feature vector, and the cosine distance between the feature vector and the feature vector corresponding to the drawing in the drawing knowledge sub-base of the multi-source knowledge base is calculated: the feature vector of the drawing in the drawing knowledge sub-base is generated by converting the detailed information of entity objects extracted from the drawing knowledge, the combination relationship model between entities, and the project information. The top N drawing examples with the smallest cosine distance are selected and output as reference design examples to the user.

[0015] Furthermore, the engineering design drawing assistance method is deployed in the form of a plug-in within an engineering drawing tool, such as AutoCAD or Civil 3D.

[0016] Compared with existing technologies, the above-mentioned engineering design drawing assistance method based on multimodal large model provided by the present invention includes constructing a multi-source knowledge base based on retrieval enhancement generation technology. The knowledge base includes a document text knowledge sub-base and a drawing knowledge sub-base. The document text knowledge is stored in the retrieval database after preprocessing, segmentation, and text embedding. The drawing knowledge is structured and stored through semantic parsing, entity relationship modeling, and project information extraction. The method is deployed as a plug-in in engineering drawing tools such as AutoCAD and Civil 3D, and supports button click and text interaction triggering functions. The core process includes: receiving user-inputted engineering design drawings and specifying the professional and specification scope; collaboratively identifying the entity attributes of the drawings through a CAD interface and a multimodal large model; retrieving design specification knowledge from the knowledge base and conducting compliance comparisons; and outputting review results including error location and type, as well as design deviation warnings. It also supports design case recommendations based on cosine distance and dynamically optimizes the knowledge base and multimodal large model based on user feedback. This addresses existing technologies' shortcomings in image feature extraction and text-image alignment accuracy, poor multimodal information fusion performance, and low design information processing efficiency, thereby improving the accuracy of engineering design drawing review and the efficiency of design scheme generation. Attached Figure Description

[0017] Figure 1 A flowchart of an engineering design drawing assistance method provided in an embodiment of the present invention. Detailed Implementation

[0018] The technical solution of the present invention will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are not all embodiments of the present invention. All other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0019] It should be noted that, unless otherwise specifically stated, the relative arrangement and numerical expressions of the components and steps described in these embodiments should not be construed as limiting the scope of the invention.

[0020] The following description of exemplary embodiments is merely illustrative and is not intended to limit the invention or its application or use in any way. Techniques, methods, and apparatus known to those skilled in the art may not be discussed in detail herein, but where applicable, such techniques, methods, and apparatus should be considered part of this specification.

[0021] Example 1 See Figure 1 , Figure 1 This is a flowchart of an engineering design drawing assistance method based on a multimodal large model proposed in this invention. The method combines Retrieval-Augmented Generation (RAG) technology with a multimodal large model to solve problems such as insufficient image feature extraction and image-text alignment accuracy, poor multimodal information fusion performance, and low design information processing efficiency in existing technologies, thereby realizing intelligent review of engineering design drawings and recommendation of design cases.

[0022] In this embodiment, the engineering design drawing assistance method is integrated into commonly used engineering drawing tools by designers as a plug-in, supporting mainstream versions such as AutoCAD and Civil 3D. The plug-in includes three core function buttons: "Drawing Review," "Case Recommendation," and "Knowledge Base Update," along with corresponding text interaction boxes. Users can trigger subsequent operations by clicking the function buttons or entering text commands (such as design requirements or drawing review scope). Specific steps may include: S1. A multi-source knowledge base is constructed based on retrieval enhancement generation technology. The knowledge sources of the multi-source knowledge base include: document, text, and drawing knowledge. Corresponding knowledge extraction and processing methods are used for the knowledge. Among them, HTML, databases, documents, and text together constitute non-drawing knowledge. HTML sources include industry standard update pages, professional content from technical forums, etc., while database sources are the design unit's internal historical project standard library and archived owner requirements data. Both types of knowledge are stored according to the extraction and processing flow of document and text knowledge. S11. The specific steps for knowledge extraction and processing of document and text-based knowledge include: S111. Preprocessing: Preprocess document and text-based knowledge: convert various types of knowledge into plain text in a uniform format, and filter irrelevant information such as headers, footers, and advertisements; The system uses Python's python-docx and pdfplumber libraries to parse documents (Word / PDF format), HTML pages, and text data exported from databases, converting all types of knowledge into UTF-8 encoded plain text format. At the same time, it uses regular expressions to filter irrelevant information such as headers (e.g., "Page X of Y"), footers (e.g., design unit logo description), and residual text from pop-up ads, ensuring text purity.

[0023] S112, Block processing: The preprocessed long text is divided into blocks: fixed window block, sliding window block, or semantic-based block method is used to divide it into text blocks of preset byte length. Fixed window segmentation: Extracts text blocks of corresponding lengths from the original text sequentially with a fixed length; extracts text blocks sequentially from the beginning of the text with a fixed length of 512 bytes, suitable for well-formatted table-type text (such as parameter reference tables in specifications). Sliding window segmentation: Extracts text blocks of corresponding lengths from the original text sequentially with a fixed length, and there is overlap between the text blocks; with a fixed length of 512 bytes, it slides 400 bytes each time to extract text blocks (112 bytes overlap between blocks), which is suitable for technical description text with continuous paragraphs and avoids semantic fragmentation. Semantic-based segmentation: Segmentation by semantic units ensures semantic coherence and independence of text blocks; LangChain's RecursiveCharacterTextSplitter tool is used to segment text by sentence boundaries or topic keywords, ensuring that each text block revolves around a single technical topic, such as integrating content related to "shear wall thickness design requirements" into a semantic block.

[0024] S113. Text embedding and storage: Text embedding processing is performed on the segmented text blocks: the text blocks are converted into high-dimensional vectors that capture semantic information, and the high-dimensional vectors are stored in the retrieval database to form document and text-based knowledge sub-bases.

[0025] Text embedding processing employs pre-trained models, specifically OpenAItext-embedding-ada-002, BERT, or Sentence-BERT; the retrieval database is pgVector or Milvus; and a multi-source knowledge base is configured with a dynamic update interface, supporting the addition or deletion of knowledge. An API interface is configured for the knowledge base, allowing designers to upload new specification documents or delete invalid standards via plugins; after uploading, the system automatically synchronizes the knowledge in real time.

[0026] S12. Methods for knowledge extraction and processing of drawing-related knowledge include: S121. Drawing semantic parsing: Parse the semantics of the drawing from the structured data of the drawing, and identify the detailed information of the entity objects in the drawing: The detailed information of the entity objects includes entity category and entity parameter information; The specific implementation method is as follows: the secondary development interface of the CAD platform (such as the ObjectARX interface of AutoCAD) is used to read the structured data of the drawing (including layer ownership, element attribute definition, block reference association data, etc.), and the entity object recognition is carried out simultaneously in combination with the computer vision object detection model (such as the YOLOv8 model) to form a collaborative processing mechanism of data parsing and visual recognition. Entity Categories: Subdivided into specific types according to engineering fields and professional attributes. In the building construction field, the "wall" category is subdivided into shear walls and masonry walls (further subdivided by material into concrete shear walls and brick masonry walls), and the "door" category is subdivided into sliding doors and fire doors (subdivided by fire resistance rating into Class A fire doors and Class B fire doors). In the hydropower engineering field, the "dam" category is subdivided into gravity dams, arch dams, and pier dams, ensuring that the category classification is consistent with the professional standards of engineering design. Entity parameter information: Extract core parameters for entity design and construction, such as the thickness of shear walls (200mm), concrete strength grade (C30), reinforcement parameters (Φ12@200), fire door specifications (1000mm×2100mm), and fire resistance limit (1.5h). The parameter extraction range covers the key indicators required for entity function and compliance verification.

[0027] S122. Entity Relationship Model Construction: Centered on entity association logic analysis, this section constructs a composite relationship model between entities based on Graph Neural Networks (GNNs), clarifying entity interaction rules and constraint boundaries. The specific process is as follows: Graph Neural Networks (GNNs) are used to mine spatial, functional, and compliance relationships between entities, such as the connection relationship between "shear wall and beam" (with a structural constraint that the beam end is embedded in the shear wall to a depth of ≥300mm) and the matching relationship between "fire door and firewall" (restricting that Class A fire doors must be paired with firewalls with a fire resistance rating of ≥3h). These relationships are modeled using a structured paradigm of "entity 1 - association type - entity 2 - constraint condition" to form a searchable and reusable composite relationship model, such as "fire door M1022 - installation association - firewall FW1 - constraint condition: gap between door and wall ≤3mm".

[0028] S123. Project Information Extraction: Extract comprehensive project information from standardized drawing information carriers. This project information includes two categories: design specification requirements and geographical and climatic characteristics, ensuring that the information covers design compliance and regional adaptability verification needs. Information extraction sources are standardized carriers such as drawing title blocks and design description pages. Specific content includes: Design specifications require: Extract the compliance indicators that the project must follow, such as seismic fortification level (6-degree seismic fortification), building fire resistance rating (Level 1), and structural compressive strength (dam concrete compressive strength ≥ C40). The values ​​of these indicators must strictly correspond to national / industry design specifications (such as GB50011 "Code for Seismic Design of Buildings" and GB50010 "Code for Design of Concrete Structures"). Geographic and climatic information: Extract environmental and geological parameters of the project site, such as the geological properties (silty clay, foundation bearing capacity characteristic value 180kPa) and climate conditions (extreme low temperature of -15℃ in winter, average annual precipitation of 600mm) of the project site (Haidian District, Beijing), to provide data support for regional adaptability design verification.

[0029] S124. Knowledge Storage: Detailed information about the entity objects, the model of relationships between entities, and project information are stored in a multi-source knowledge base to form a drawing-related knowledge sub-base, enabling structured storage and associated retrieval of drawing knowledge. The specific processing method is as follows: Detailed information about entity objects, the model of relationships between entities, and project information are converted into structured data in JSON format. The corresponding drawing thumbnails (PNG format, ensuring that the thumbnails contain key features of the entities) are synchronously associated with the entities and stored together in a multi-source knowledge base. A unique feature vector is generated for each drawing case (generated by text / numerical embedding of key entity parameters and core project information) to achieve similarity retrieval and accurate matching of drawing cases, ultimately forming a logically closed-loop drawing-type knowledge sub-base.

[0030] S2. Receive the user-inputted engineering design drawing to be processed; For the input receiving stage of the engineering design drawings to be processed, the input process needs to be triggered through plug-in interaction after the user completes part / all of the drawing on the AutoCAD or Civil 3D platform. The specific steps are as follows: S21. Drawing review mode selection: After the user clicks the plugin's drawing review button, they first select the drawing review mode: If they select to review while drawing, the system will read the completed drawing data in the current drawing interface in real time; if they select overall review, the subsequent steps S22-S24 will be executed to receive the complete drawing.

[0031] S22. Professional Selection Configuration: When the user clicks the drawing review function button on the plugin interface, the system will automatically pop up a professional configuration interaction window, which will prioritize accepting the design professional category specified by the user (such as "Building Construction - Structural Engineering" or "Hydropower - Dam Engineering"). This selection will serve as the professional dimension basis for subsequent code retrieval and drawing review judgment. The drawing review function supports two modes: one is to review while drawing, in which the system reads the drawing data of the completed part in real time during the drawing process, automatically triggers entity recognition and specification comparison, and checks for omissions and deficiencies in the currently completed section; the other is to review the whole drawing after the drawing is completed, read the complete engineering design drawing to be processed for full drawing verification, and users can choose the mode according to the drawing progress.

[0032] S23. Definition of the scope of the code library: The system further receives the user's input of the scope of the design code library search. If the user explicitly enters the code number (e.g., "GB50011-2010+GB50010-2010"), the search scope will be limited to the specified code; if the user does not enter it, the system will default to triggering a full code search of the code library corresponding to the currently selected discipline (e.g., the structural discipline will default to covering seismic codes, concrete structure codes, building structure load codes, etc.). S24. Automatic Drawing Retrieval: After the user confirms the scope of the profession and specifications, the system automatically retrieves the engineering design drawings (supporting mainstream DWG formats in the engineering design field) that are in the editing state in the current drawing interface through the interface with the CAD platform, thus completing the drawing reception in step S2.

[0033] S3. Perform entity recognition on the engineering design drawing to be processed, and obtain the attribute information of the entities in the drawing; The system calls the entity recognition module, which combines a multimodal large model (such as GPT-4V) with a CAD interface, to parse the drawing to be processed received by S2. The specific steps include: S31. Preprocessing of drawing data: The system first extracts the structured drawing data (covering basic elements such as lines, block references, and text annotations) of the drawing through the CAD secondary development interface (such as AutoCADObjectARX), and filters non-physical elements such as auxiliary lines and dimension annotations through preset rules (these elements do not carry physical function information, and excluding them can improve the recognition accuracy). S32. Multimodal Collaborative Recognition: Based on preprocessed image data, the system calls a multimodal large model (e.g., GPT-4V) to perform image recognition on drawings. Through dual verification of "image data verification + image semantic understanding", the entity category and core parameters are confirmed. Examples include: identifying the door numbered "QM1" in the drawing as a Class B fire door, clarifying its specifications (1200mm×2100mm) and installation location (at the entrance of the stairwell); identifying the walls in the "A-3~A-5" section of the 3rd floor grid as shear walls, and determining their structural parameters (thickness 180mm, concrete strength grade C25). S33. Structured Output of Recognition Results: The system integrates the recognition information of all entities into a structured entity recognition result table. The table contains key fields such as "Entity ID (unique identifier), Entity Category (sub-professional category), Detailed Parameters (functional / structural parameters), and Location Coordinates (drawing space coordinates)". This result table will serve as the standardized basic data for subsequent specification comparison and error judgment.

[0034] S4. Based on the attribute information of the entity, retrieve matching design specification-related knowledge from the multi-source knowledge base; the system performs precise retrieval in document and text-based knowledge sub-bases based on the entity recognition results and the scope of the professional and specification databases selected by the user. Specific steps include: S41. Retrieval Feature Vector Matching: Using the combination of key attributes of entities as retrieval features, the features are converted into high-dimensional semantic vectors through a pre-trained embedding model. Cosine similarity is calculated with the standard text vectors stored in the "Document and Text Knowledge Sub-base" (the similarity threshold is set at 0.85, and a match higher than this threshold is considered valid). S42. Standard Knowledge Extraction: Based on the vector matching results, extract design standard clauses that are highly related to the current entity attributes, such as the mandatory clause in the "Code for Seismic Design of Buildings" GB50011-2010 that "in seismic fortification zones of intensity 6, the thickness of shear walls shall not be less than 180mm (except for short-limb shear walls)", and the restrictive provision in the "Code for Design of Concrete Structures" GB50010-2010 that "the applicable height of C25 concrete shear walls shall be ≤80m", forming a set of standard comparison bases.

[0035] S5. Call the multimodal large model, compare the attribute information of the entity with the retrieved design specification-related knowledge, determine whether the engineering design drawing to be processed conforms to the design specification, and output the review result including errors and error types. The specific process includes: S51. Entity parameter compliance judgment: The model prioritizes verifying the consistency between entity technical parameters and specification requirements. For example, a 180mm thick non-short-limb shear wall in the drawing to be processed meets the specification requirement of "thickness ≥ 180mm" for a seismic intensity 6 zone. If a beam is reinforced with Φ10@250, and its reinforcement ratio is calculated to be 0.6%, which is lower than the retrieved specification requirement of "reinforcement ratio of tension zone at beam end ≥ 0.8%", then it is judged to be non-compliant with the design specification. S52. Identification of drawing standardization errors: The model synchronously checks for consistency issues at the drawing level. For example, if there is a discrepancy between the dimension marked as “QM1 fire door” (1000mm×2100mm) in the drawing and the actual recognized dimension (1200mm×2100mm), it is judged as a low-level drawing error (inconsistent dimension marking). S53. Output the drawing review results: S531. Error Location and Labeling: In the AutoCAD / Civil3D drawing interface, use a red box to accurately label the location of the erroneous entity (such as a beam with insufficient reinforcement or a fire door with incorrect dimensions), and display the structured error information through a pop-up window (e.g., "

Non-compliant with specifications

Basic error

[0036] S6. Dynamic optimization based on user feedback: By capturing user interaction feedback on the review results, the knowledge base and model parameters are iteratively upgraded. Specific steps include: S61. Error Confirmation and Closed-Loop Processing: If the user confirms that the error is real (e.g., by clicking the "Confirm Insufficient Reinforcement" button), the system will automatically guide the user to modify the drawings. After the modification is completed, the drawing review can be triggered again. S62. Feedback and Explanation Submission for Misjudgment: If the user's judgment result is a misjudgment (e.g., clicking the "Misjudgment - Incorrect Application of Standards" option), supplementary feedback and explanation must be submitted (e.g., "This beam is a secondary beam, applicable to GB50010-2010, Clause 9.2.2, reinforcement ratio requirement ≥0.6%), clearly stating the reason and basis for the misjudgment; S63. Application and updating of feedback information: The system stores user feedback data in the feedback database and automatically performs two-way optimization. Fine-tuning of parameters for multimodal large models: For example, adding "distinguishing weights for the reinforcement ratio of secondary beams and main beams in the code clauses" to improve the accuracy of judgment in similar scenarios and reduce repeated misjudgments; Knowledge base supplementary annotation: The "GB50010-2010 Article 9.2.2" mentioned by users will be associated with the "secondary beam" tag for storage, which will strengthen the mapping relationship between the standard clauses and application scenarios and improve the accuracy of subsequent searches.

[0037] S7. The intelligent case recommendation process based on user needs is triggered by the case recommendation plugin to provide users with the most relevant design reference cases. Specific steps include: S71. Requirement Instruction Parsing: Receives structured text instructions input by users (e.g., "Beijing area, seismic intensity 6, 18-story residential building, refer to shear wall structure beam reinforcement case"). The instructions must include the core attributes of the project: design code requirements (seismic intensity 6), geographical and climatic characteristics (Beijing area), and reference content category (beam reinforcement design). S72. Demand Feature Vector Conversion: Convert text instructions into high-dimensional feature vectors, with the vector dimensions consistent with the case feature vectors in the "Drawing Knowledge Sub-base" (e.g., 128 dimensions, covering key features such as seismic resistance level, number of building floors, regional attributes, and structural type). S73. Case Similarity Calculation: The case matching degree is quantified by calculating the cosine distance between the feature vector of the requirement and the feature vector of similar cases (such as shear wall drawings for residential buildings) in the "Drawing Knowledge Sub-base". (The smaller the distance, the higher the similarity. For example, if the cosine distance between a case and the requirement is 0.12, it is judged as a high match). S74. Case Results Display: Filter the top N cases with the smallest cosine distance (N defaults to 5, can be adjusted via plugin parameters) and display them in the form of "thumbnail + core parameters" (e.g., "Reinforcement drawing of shear wall beam of an 18-story residential building in Beijing in 2023, seismic intensity 6, beam reinforcement ratio 0.8%"). Users can click on the thumbnail to trigger the function of viewing the original drawings and design specifications (the original drawing is linked to the multimodal original data storage address through a link placeholder to achieve one-click jump).

[0038] In summary, the present invention has the following advantages: 1. By constructing a multi-source knowledge base covering documents, HTML, databases, and text, and combining corresponding knowledge extraction and processing methods, we can achieve in-depth mining and intelligent analysis of complex information such as design specification clauses, unit design standards, and owner design requirements, effectively improving the efficiency and accuracy of design information processing. 2. Specialized processing of drawing-related knowledge is carried out. By identifying the attribute information and project information of entity objects in the drawings and establishing a relationship model between entities, intelligent identification and analysis of design drawings can be achieved, effectively improving the accuracy and efficiency of drawing review. 3. Based on user text commands and image input, and leveraging the generation capabilities of multimodal large models, it provides reference design cases that match the requirements, significantly improving the generation efficiency and reference value of design solutions; 4. Through user interaction and feedback mechanisms, continuously optimize the ability to generate design solutions for large models, and further improve the system's intelligence level and the quality of design solutions; 5. By setting professional names, refined configuration can be achieved, and intelligent analysis and processing can be carried out for specific architectural design professions to improve the accuracy and practicality of feedback results; 6. Employ advanced multimodal information fusion and interaction technologies to achieve effective integration and analysis of text and image information, thereby improving the quality and optimization effect of design schemes.

[0039] The above specific embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to examples, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for assisting in the drawing of engineering design diagrams based on a multimodal large model, characterized in that, include: S1. Construct a multi-source knowledge base based on retrieval enhancement generation technology. The knowledge sources of the multi-source knowledge base include: document, text and drawing knowledge. Use corresponding knowledge extraction and processing methods for different knowledge to complete knowledge storage. S2. Receive the user-inputted engineering design drawing to be processed; S3. Perform entity recognition on the engineering design drawing to be processed, and obtain the attribute information of the entities in the drawing; S4. Based on the attribute information of the entity, retrieve matching design specification-related knowledge from the multi-source knowledge base; S5. Call the multimodal large model, compare the attribute information of the entity with the retrieved design specification-related knowledge, determine whether the engineering design drawing to be processed conforms to the design specification, and output the review result including errors and error types.

2. The engineering design drawing assistance method based on a multimodal large model according to claim 1, characterized in that, The knowledge extraction and processing methods for the aforementioned documents and text-based knowledge include: Preprocess document and text-based knowledge: convert all types of knowledge into plain text in a uniform format and filter out irrelevant information such as headers, footers, and advertisements; The preprocessed long text is divided into blocks: fixed window block division, sliding window block division, or semantic-based block division is used to divide it into text blocks of a preset byte length. Text embedding processing is performed on the segmented text blocks: the text blocks are converted into high-dimensional vectors that capture semantic information, and the high-dimensional vectors are stored in the retrieval database to form document and text-based knowledge sub-bases.

3. The engineering design drawing assistance method based on a multimodal large model according to claim 2, characterized in that, The text embedding process uses a pre-trained model, which is OpenAItext-embedding-ada-002, BERT, or Sentence-BERT. The retrieval database is pgVector or Milvus; and the multi-source knowledge base is configured with a dynamic update interface, supporting the addition or deletion of knowledge in the knowledge base.

4. The engineering design drawing assistance method based on a multimodal large model according to claim 2, characterized in that, The fixed window segmentation: extracts text blocks of corresponding lengths from the original text sequentially with a fixed length; The sliding window is divided into blocks: text blocks of corresponding lengths are extracted from the original text in sequence with a fixed length, and there is overlap between the text blocks; The semantic-based segmentation: segmenting by semantic units to ensure semantic coherence and independence of text blocks.

5. The engineering design drawing assistance method based on a multimodal large model according to claim 1, characterized in that, The methods for knowledge extraction and processing of the aforementioned drawing-related knowledge include: The semantics of the drawings are parsed from the structured data of the drawings to identify detailed information of entity objects in the drawings: the detailed information of entity objects includes entity category and entity parameter information; Analyze the relationships between entity objects and establish a model of the compositional relationships between entities; Extract the project information corresponding to the drawings: The project information includes design specifications and geographical and climatic information; The detailed information of the entity objects, the model of the combination relationship between entities, and the project information are stored in a multi-source knowledge base to form a drawing-type knowledge sub-base.

6. The engineering design drawing assistance method based on a multimodal large model according to claim 5, characterized in that, The design specifications include: seismic resistance rating, fire resistance rating, and compressive strength rating; The geographical and climatic information includes: geological information and climate information of the project location.

7. The engineering design drawing assistance method based on a multimodal large model according to claim 1, characterized in that, When outputting the review results, including errors and error types, the following should also be included: Mark the location of errors in the drawing canvas of the engineering drawing tool and clearly inform the user of the error type: the error types include basic drawing errors and errors that do not conform to design specifications; The system analyzes and compares the design drawings of the project to be processed with the drawings of successful cases in a multi-source knowledge base, and outputs a design deviation warning prompt between the current drawing and the drawings of successful cases.

8. The engineering design drawing assistance method based on a multimodal large model according to claim 1, characterized in that, It also includes knowledge base self-updating: Receive user confirmation and correction instructions for the drawing review results and design deviation warning prompts: if the user confirms that the error is real, prompt the user to modify the drawing; if the user determines that it is a misjudgment, receive the misjudgment feedback information submitted by the user. User correction commands and misjudgment feedback are fed into a multi-source knowledge base to update the comparison parameters and knowledge base content of the multimodal large model, thereby optimizing the accuracy of subsequent image review.

9. The engineering design drawing assistance method based on a multimodal large model according to claim 1, characterized in that, It also includes design case recommendations: Receive text instructions input by the user: The text instructions include project design specifications, project geographical and climatic information, and reference case category requirements; The text instruction is converted into a feature vector, and the cosine distance between the feature vector and the feature vector corresponding to the drawing in the drawing knowledge sub-base of the multi-source knowledge base is calculated: the feature vector of the drawing in the drawing knowledge sub-base is generated by converting the detailed information of entity objects extracted from the drawing knowledge, the combination relationship model between entities, and the project information. The top N drawing examples with the smallest cosine distance are selected and output as reference design examples to the user.

10. The engineering design drawing assistance method based on a multimodal large model according to claim 1, characterized in that, The engineering design drawing assistance method is deployed in the form of a plug-in within an engineering drawing tool, such as AutoCAD or Civil 3D.

Images