Engineering drawing auditing method and system based on graphics and text information and large language model

Abstract

The invention relates to the field of drawing auditing, and provides an engineering drawing auditing method and system based on graphic and text information and a large language model in order to improve the engineering drawing auditing efficiency and accuracy, the core of the method is that graphic and text multi-modal data of an engineering drawing to be audited is split into drawing data and text data; for each independent design in the to-be-audited engineering drawing, combining the sorted multi-dimensional views in sequence by using a preset sliding window to obtain a plurality of combined views, and determining key features of each combined view; obtaining a technical description corresponding to the independent design from the text data, filling the technical description into a preset cue word template to obtain a first text corresponding to the independent design, and inputting the first text into a preset large language model for expansion to obtain a second text; according to the second text, judging whether the key features in the combined view are designed to reach the standard or not, and determining whether the corresponding independent design and the engineering drawing are designed to reach the standard or not.

Description

Technical Field

[0001] This invention relates to the field of drawing review, and in particular to a method and system for reviewing engineering drawings based on graphic and textual information and a large language model. Background Technology

[0002] Engineering drawing review is a crucial process in engineering projects, aiming to ensure that design documents (including but not limited to architectural, structural, electrical, plumbing, HVAC, and other professional drawings) comply with relevant regulations, standards, technical specifications, and project-specific requirements. This process is typically performed by experienced engineers or professional third-party organizations. Manual review suffers from low efficiency and a high error rate.

[0003] In addition, some solutions use image recognition technology to identify and review engineering drawings. However, these solutions usually only analyze a single view of the structure and cannot automatically associate multiple views of the same design. This results in insufficient spatial consistency checks of various engineering structures in the drawings and makes it impossible to determine whether the subtle features of the engineering structure meet the review requirements. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method and system for reviewing engineering drawings based on graphic information and a large language model, in order to solve the problems of low efficiency and low accuracy in reviewing engineering drawings.

[0005] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:

[0006] On the one hand, this invention provides a method for reviewing engineering drawings based on graphic and textual information and a large language model, including:

[0007] Obtain the multimodal data of the drawings to be reviewed, and split the multimodal data into drawing data and text data;

[0008] For each independent design in the engineering drawings to be reviewed, obtain the multi-dimensional view of the corresponding independent design from the drawing data, sort the obtained multi-dimensional views, and combine the sorted multi-dimensional views in turn using a preset sliding window to obtain multiple combined views, and determine the key features of each combined view.

[0009] The technical description corresponding to the independent design is obtained from the text data, and the technical description corresponding to the independent design is filled into the preset prompt word template to obtain the first text corresponding to the independent design. The first text is then input into the preset large language model for expansion to obtain the second text.

[0010] Based on the second text, determine whether the key features in the combined view meet the design standards. Based on the design compliance rate of the key features, determine whether the corresponding independent design meets the design standards. Based on the compliance rate of the independent design, determine whether the engineering drawings to be reviewed meet the design standards.

[0011] Furthermore, the acquired multi-dimensional views are sorted, and a preset sliding window is used to sequentially combine the sorted multi-dimensional views to obtain multiple combined views, including:

[0012] Obtain different types of views for each individual design, including plan views, elevations, and sections from different perspectives;

[0013] Sort different types of views according to a preset order;

[0014] Based on the sorting results of the views, different types of views are combined sequentially using a preset sliding window to obtain multiple combined views, wherein the size of the preset sliding window is at least 2.

[0015] Furthermore, the technical specifications corresponding to the independent design are filled into a preset prompt template to obtain the first text corresponding to the independent design, including:

[0016] Construct a preset prompt word template, which includes a data background section, an analysis target section, and an output requirement section;

[0017] Fill in the corresponding technical specifications for the independent design into the data background section;

[0018] The analysis objectives and output requirements of the independent design are determined based on the technical specifications of the independent design. The analysis objectives are used to describe the review purpose of each key feature in the corresponding independent design, and the output requirements are used to limit the rationality and compliance of the analysis objectives.

[0019] Fill the analysis target into the analysis target section and the output requirements into the output requirements section to obtain the first text.

[0020] Furthermore, the text data also includes references to relevant drawings, regulatory compliance statements, and supplementary information on special conditions or restrictions;

[0021] When constructing the prompt word template, the supplementary information and the corresponding technical descriptions of the independent design are filled into the data background section.

[0022] Furthermore, the first text is input into a pre-defined large language model for expansion to obtain the second text, which includes:

[0023] The first text is input into the preset large language model to expand the content in the background section;

[0024] Based on the expanded background content, output the response to each analysis objective in the first text according to the output requirements;

[0025] The second text is obtained by filling the output requirement section of the first text with the response to the analysis target.

[0026] Furthermore, the large language model expands the content in the background section based on the collected design requirements, regulations, and rules.

[0027] Furthermore, based on the second text, it is determined whether the key features in the combined view meet the design standards, including:

[0028] For each key feature in the combined view, a response to the analysis objective associated with that key feature is identified in the second text;

[0029] Analyze the consistency between each key feature in the combined view and the corresponding analysis target response to determine whether the design of the corresponding key features meets the target.

[0030] The design compliance rate of the combined view is determined based on whether each key feature in the combined view meets the design standards.

[0031] Furthermore, the compliance rate of independently designed projects is used to determine whether the drawings for the project under review meet the design standards, including:

[0032] The average design compliance rate of multiple independently designed combined views is taken as the design compliance rate of the independent design.

[0033] The average design compliance rate of the independent designs in the series of drawings to be reviewed is taken as the total design compliance rate of the series of drawings to be reviewed.

[0034] Furthermore, key features for each combined view are determined based on GNN, object detection models, instance segmentation models, OCR label recognition, image semantic segmentation, or graph structure modeling.

[0035] On the other hand, the present invention provides an engineering drawing review system based on graphic and textual information and a large language model, the system comprising:

[0036] The acquisition module acquires the multimodal data of the drawings to be reviewed, and splits the multimodal data into drawing data and text data.

[0037] The extraction module retrieves a multi-dimensional view of each independent design from the drawing data for each design in the engineering drawings to be reviewed.

[0038] The combination module is used to combine sorted multi-dimensional views in a pre-defined sliding window to obtain multiple combined views;

[0039] The key feature module is used to determine the key features of each combined view;

[0040] The first text module is used to generate the first text corresponding to each independent design based on the technical specifications in the text data;

[0041] The second text module is used to input the first text into a preset large language model for expansion, and obtain the second text.

[0042] The first compliance rate module determines whether the key features in the combined view meet the design standards based on the second text, and determines whether the corresponding independent design meets the design standards based on the compliance rate of the key features.

[0043] The second compliance rate module determines whether the design of the engineering drawings to be reviewed meets the standards based on the compliance rate of independent designs.

[0044] The beneficial effects of this invention are:

[0045] This invention determines the design compliance rate by combining key features of the combined view of multiple parts after the large language model is expanded. This allows for repeated review of the design scheme, avoiding single-review errors in the model. Furthermore, this invention uses a preset large language model to expand the first text to obtain the second text, which can introduce more relevant background knowledge and contextual information of the engineering drawings. This helps to understand the design intent of the engineering drawings and the complex details in the technical specifications, and improves the accuracy of the large language model in responding to the design objectives.

[0046] This invention combines multi-dimensional views of independently designed drawings using a preset sliding window. By magnifying local areas of independently designed drawings through different view combination methods, it helps to discover contradictions or errors in detailed structures, enhances the ability to identify details, makes it easier to identify inconsistencies between drawings, and improves the efficiency and accuracy of engineering drawing review. Attached Figure Description

[0047] Figure 1 This is a flowchart of the engineering drawing review method based on graphic information and a large language model as described in this invention. Detailed Implementation

[0048] The technical solution of this invention is to solve the above-mentioned technical problems, and the overall idea is as follows:

[0049] Text data and drawing data are extracted from the engineering drawings. Technical descriptions for each independent design are extracted from the text data and filled into a preset prompt template based on the drawing review rules and objectives to obtain the first text. To enrich the design intent and details in the first text, it is input into a large language model for expansion, resulting in a second text that includes responses to each analysis objective in the first text, as required by the output requirements. While processing the text data, views of each independent design from different dimensions are extracted from the drawing data. These multi-dimensional views are combined using a preset sliding window to obtain multiple combined views that reflect the characteristics and details of key features from different perspectives. The consistency between the characteristics and details of each key feature in the combined views and the corresponding analysis objective responses in the second text is analyzed to determine whether the design of the corresponding key features meets the standards.

[0050] like Figure 1 As shown, the engineering drawing review method based on graphic and textual information and a large language model according to the present invention includes steps S1-S8:

[0051] Step 1: Obtain the multimodal data of the drawings of the project to be reviewed, and split the multimodal data of the drawings according to the text boxes of the drawings to be reviewed to obtain the drawing data and text data.

[0052] The series of engineering drawings to be reviewed are a series of engineering drawings for the same engineering objective, and the series of engineering drawings may include multiple engineering drawings.

[0053] Multimodal data with images and text refers to composite data that simultaneously contains image information of engineering structures and text information that describes, explains, limits, and / or supplements the images of engineering structures.

[0054] The text data should include at least technical specifications for each independent design, and may also include supplementary information for the respective independent design, as well as title blocks and designer information. Drawing data includes CAD drawings of the project, architectural floor plans, and related symbols, scales, and auxiliary information.

[0055] Specifically, the technical specifications may include material specifications, construction requirements, dimensioning, symbols, and legends for the corresponding independent design; the title block may include: project name, drawing name, drawing number, scale, version number, date, and designer information; other supplementary information may include: relevant drawing references, regulatory compliance statements, and special conditions or restrictions.

[0056] Step S2: For each independent design in the series of drawings of the project to be reviewed, obtain the multi-dimensional view corresponding to the independent design from the drawing data.

[0057] An independent design refers to a design unit in an engineering project that possesses complete functional and structural significance. An independent design typically corresponds to a functional module, subsystem, or component within the project; it can exist independently, be identified and analyzed, and is relatively independent of other design units. For example, an independent design can be a complete building structure with corresponding floor plans, elevations, or sections; it can also be a room within that building, which also has floor plans, elevations, or sections; it could be the layout of the building's HVAC piping; or a specific installation within the building.

[0058] The multi-dimensional views include plan views (front view, top view, side view, left view, and right view, etc.), elevation views, and sectional views, etc.

[0059] A series of drawings for an engineering project awaiting review may include one or more views of an independent design. A series of drawings for an engineering project awaiting review may also include one or more views of multiple independent designs. For an independent design, the views of that independent design can be extracted separately, that is, the plan view, elevation view and section view of the corresponding independent design from different perspectives can be obtained.

[0060] Step S3: Recombine the multi-dimensional views using a preset sliding window to obtain several partially combined views.

[0061] Specifically, this involves: sorting independently designed multi-dimensional views according to a preset order; and based on the sorting result, sequentially combining the views using preset sliding windows to obtain several combined views. The size of the preset sliding window is at least 2, and the size of the preset sliding window can be smaller than the number of views.

[0062] In this embodiment, the preset sliding window size is 3. The arrangement order of the multi-dimensional views is: front view, top view, side view, left view, right view, elevation view, and section view. Multiple combined views are obtained according to the sliding window size of 3, that is, the front view, top view, and side view are a combined view, the top view, side view, and left view are a combined view, the side view, left view, and right view are a combined view, the left view, right view, and elevation view are a combined view, and the right view, elevation view, and section view are a combined view.

[0063] Step S4: Determine the key features of each combined view.

[0064] The key features include at least structural connection nodes, pipeline intersection nodes, or zoning boundaries. Structural connection nodes refer to the connection points between two or more components via welding, bolting, hinges, etc.; pipeline intersection nodes refer to the locations where pipes, cable trays, etc., in water, electricity, HVAC, fire protection, and other systems intersect, converge, or branch; zoning boundaries include the boundaries between different functional areas defined in the drawings, such as fire compartments, air conditioning compartments, electrical distribution areas, and construction responsibility areas. In addition, depending on the type of drawing, the key features may also include component positioning points, construction process nodes, load application points, etc., without specific limitations.

[0065] Key nodes can be identified not only through work experience, but also through pre-trained graph neural networks (GNNs), object detection models (YOLO, Faster R-CNN), instance segmentation models (such as Mask R-CNN), OCR label recognition, image semantic segmentation, and graph structure modeling.

[0066] Step S5: Obtain the technical description and supplementary information corresponding to the independent design from the text data, and fill the technical description and supplementary information into the preset prompt word template to obtain the first text corresponding to the independent design.

[0067] The prompt template is set according to the purpose and rules of drawing review.

[0068] In this embodiment, the preset prompt word template includes a data background section, an analysis target section, and an output requirement section;

[0069] Fill the data background section with the technical specifications and supplementary information corresponding to the independent design; based on the technical specifications and supplementary information, determine the analysis objectives and output requirements, fill the analysis objectives into the analysis objective section, and fill the output requirements into the output requirements section to obtain the first text. The analysis objectives are used to describe the review purpose of each key feature in the corresponding independent design, and the output requirements are used to limit the rationality and compliance of the analysis objectives.

[0070] In addition to the technical descriptions and supplementary information mentioned above, the background section may also include other text describing the independent design.

[0071] Analyze issues within the target section that can be filled with fixed templates. For example, analyze whether a "certain key node" is designed reasonably; identify the design basis for the "certain key node." When analyzing different independent designs, simply replace "certain key node" with the specific key node.

[0072] The output requirements section sets limits on the questions in the target analysis section. For example, for questions like "Analyze whether a certain key node is designed reasonably", the output requirements section can set the number of characters in the output of this question to not exceed a threshold; for questions like "Find the design basis of a certain key node", the output requirements section can set the design basis in the output of this question to the latest laws or regulations, etc.

[0073] Step S6: Input the first text into the preset large language model for expansion to obtain the second text.

[0074] The first text is input into a preset large language model to expand the content in the background section. Based on the expanded background section content, the response to each analysis target in the first text is output according to the output requirements. The response to the analysis target is filled into the output requirement section of the first text to obtain the second text.

[0075] The preset large language model can be a pre-trained semantic model or a general model such as GPT, Doubao, or Deepseek. After the first text is input into the preset large language model, the first text will answer the analysis target according to the output requirements based on the content in the data background section and the data already collected in the large language model (laws, regulations, design requirements, etc.).

[0076] Step S7: Determine whether the key features in the combined view meet the design standards based on the second text.

[0077] For each key feature in the combined view, determine the response associated with that key feature in the second text;

[0078] The consistency between the key features in the combined view and the corresponding analysis target response is analyzed to determine whether the design of the corresponding key features meets the standards. In order to improve the accuracy, relevant personnel can also conduct a review.

[0079] Based on the design compliance results of each key feature in the composite view, determine the design compliance rate of that part of the composite view, including: ,in, For the first The design compliance rate of each combined view For the first The number of key features required for design compliance in each composite view. For the first The total number of key features in a combined view.

[0080] Step S8: Determine the overall design compliance rate of the series of drawings for the project to be reviewed based on the design compliance rate of the independently designed combined views.

[0081] The average design compliance rate of the combined views in the independent designs is taken as the design compliance rate of that independent design; the average design compliance rate of several independent designs in the series of drawings to be reviewed is taken as the overall design compliance rate of the series of drawings to be reviewed. If the overall design compliance rate is greater than a preset threshold, it means that the design is qualified.

[0082] Based on the same inventive concept, this invention provides an engineering drawing review system based on graphic and textual information and a large language model, comprising:

[0083] The acquisition module retrieves the multimodal data of the engineering drawings to be reviewed, and splits the multimodal data into drawing data and text data.

[0084] The extraction module retrieves a multi-dimensional view of each independent design from the drawing data for each design in the engineering drawings to be reviewed.

[0085] The combination module is used to combine sorted multi-dimensional views in a pre-defined sliding window to obtain multiple combined views;

[0086] The key feature module is used to determine the key features of each part of the combined view;

[0087] The first text module is used to generate the first text corresponding to each independent design based on the technical specifications in the text data;

[0088] The second text module is used to input the first text into a preset large language model for expansion, and obtain the second text.

[0089] The first compliance rate module determines whether the key features in the combined view meet the design standards based on the second text, and determines whether the design of the corresponding independent design meets the design standards based on the compliance rate of the key features.

[0090] The second compliance rate module determines whether the design of the engineering drawings to be reviewed meets the standards based on the compliance rate of independent designs.

[0091] Based on the same inventive concept, the present invention also provides an electronic device, comprising:

[0092] A processor, and a memory for storing processor-executable instructions; wherein the processor is configured to execute an engineering drawing review method based on graphic information and a large language model as described above.

[0093] Based on the same inventive concept, the present invention also provides a non-transitory computer-readable storage medium, which, when the instructions in the storage medium are executed by the processor of an electronic device, enables the electronic device to execute the engineering drawing review method based on graphic information and a large language model as described above.

Claims

1. A method for reviewing engineering drawings based on graphic and textual information and a large language model, characterized in that: include: Obtain the multimodal data of the drawings to be reviewed, and split the multimodal data into drawing data and text data; For each independent design in the engineering drawings to be reviewed, obtain the multi-dimensional view of the corresponding independent design from the drawing data, sort the obtained multi-dimensional views, and combine the sorted multi-dimensional views in turn using a preset sliding window to obtain multiple combined views, and determine the key features of each combined view. The technical description corresponding to the independent design is obtained from the text data, and the technical description corresponding to the independent design is filled into the preset prompt word template to obtain the first text corresponding to the independent design. The first text is then input into the preset large language model for expansion to obtain the second text. Based on the second text, determine whether the key features in the combined view meet the design standards. Based on the design compliance rate of the key features, determine whether the corresponding independent design meets the design standards. Based on the compliance rate of the independent design, determine whether the engineering drawings to be reviewed meet the design standards.

2. The method for reviewing engineering drawings based on graphic and textual information and a large language model according to claim 1, characterized in that, The acquired multi-dimensional views are sorted, and then combined sequentially using a preset sliding window to obtain multiple combined views, including: Obtain different types of views for each individual design, including plan views, elevations, and sections from different perspectives; Sort different types of views according to a preset order; Based on the sorting results of the views, different types of views are combined sequentially using a preset sliding window to obtain multiple combined views, wherein the size of the preset sliding window is at least 2.

3. The engineering drawing review method based on graphic and textual information and a large language model according to claim 1, characterized in that, The technical specifications corresponding to the independent design are filled into a preset prompt template to obtain the first text corresponding to the independent design, including: Construct a preset prompt word template, which includes a data background section, an analysis target section, and an output requirement section; Fill in the corresponding technical specifications for the independent design into the data background section; The analysis objectives and output requirements of the independent design are determined based on the technical specifications of the independent design. The analysis objectives are used to describe the review purpose of each key feature in the corresponding independent design, and the output requirements are used to limit the rationality and compliance of the analysis objectives. Fill the analysis target into the analysis target section and the output requirements into the output requirements section to obtain the first text.

4. The engineering drawing review method based on graphic and textual information and a large language model according to claim 3, characterized in that, The text data also includes references to relevant drawings, regulatory compliance statements, and supplementary information on special conditions or restrictions; When constructing the prompt word template, the supplementary information and the corresponding technical descriptions of the independent design are filled into the data background section.

5. The method for reviewing engineering drawings based on graphic and textual information and a large language model according to claim 3 or 4, characterized in that, The first text is input into a pre-defined large language model for expansion, resulting in the second text, which includes: The first text is input into the preset large language model to expand the content in the background section; Based on the expanded background content, output the response to each analysis objective in the first text according to the output requirements; The second text is obtained by filling the output requirement section of the first text with the response to the analysis target.

6. The method for reviewing engineering drawings based on graphic and textual information and a large language model according to claim 5, characterized in that, The large language model expands the content in the background section based on collected design requirements, rules and regulations.

7. The method for reviewing engineering drawings based on graphic and textual information and a large language model according to claim 5, characterized in that, Based on the second text, determine whether the key features in the combined view meet the design standards, including: For each key feature in the combined view, a response to the analysis objective associated with that key feature is identified in the second text; Analyze the consistency between each key feature in the combined view and the corresponding analysis target response to determine whether the design of the corresponding key features meets the target. The design compliance rate of the combined view is determined based on whether each key feature in the combined view meets the design standards.

8. The method for reviewing engineering drawings based on graphic and textual information and a large language model according to claim 7, characterized in that, The compliance rate of independently designed projects is used to determine whether the drawings for the project under review meet the design standards, including: The average design compliance rate of multiple independently designed combined views is taken as the design compliance rate of the independent design. The average design compliance rate of the independent designs in the series of drawings to be reviewed is taken as the total design compliance rate of the series of drawings to be reviewed.

9. The method for reviewing engineering drawings based on graphic and textual information and a large language model according to claim 1, characterized in that, Key features of each combined view are determined based on GNN, object detection models, instance segmentation models, OCR label recognition, image semantic segmentation, or graph structure modeling.

10. An engineering drawing review system based on graphic and textual information and a large language model, used to implement the engineering drawing review method based on graphic and textual information and a large language model according to any one of claims 1-9, characterized in that, The system includes: The acquisition module acquires the multimodal data of the drawings to be reviewed, and splits the multimodal data into drawing data and text data. The extraction module retrieves a multi-dimensional view of each independent design from the drawing data for each design in the engineering drawings to be reviewed. The combination module is used to combine sorted multi-dimensional views in a pre-defined sliding window to obtain multiple combined views; The key feature module is used to determine the key features of each combined view; The first text module is used to generate the first text corresponding to each independent design based on the technical specifications in the text data; The second text module is used to input the first text into a preset large language model for expansion, and obtain the second text. The first compliance rate module determines whether the key features in the combined view meet the design standards based on the second text, and determines whether the corresponding independent design meets the design standards based on the compliance rate of the key features. The second compliance rate module determines whether the design of the engineering drawings to be reviewed meets the standards based on the compliance rate of independent designs.

Images