Method, device and equipment for identifying general description specifications of multi-modal CAD (Computer Aided Design) drawings

Abstract

The invention provides a method, a device and equipment for identifying total description specifications of multi-modal CAD drawings. The method comprises the steps that a to-be-recognized CAD drawing is received, and the CAD drawing comprises a to-be-recognized description specification belonging to a target specification; identifying a text describing the target specification in the CAD drawing to obtain first text information; obtaining a table name and a vector diagram name cited in the text for describing the target specification; extracting a table and a vector diagram from the CAD drawing according to the table name and the vector diagram name; recognizing texts in the table and the vector diagram to obtain second text information; and obtaining structured specification data corresponding to the description specification to be identified according to the first text information and the second text information. By means of the method and device, the multi-mode CAD drawing general description specification can be better recognized so as to adapt to diversified specification expression modes.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method, apparatus, and device for identifying general specification standards for multimodal CAD drawings. Background Technology

[0002] In the architectural design process, designers typically need to adhere to various building codes to ensure that the architectural design complies with national standards and industry requirements. To facilitate design and review, these codes are often centrally recorded in the general instructions section of CAD drawings. Accurately identifying these codes and transforming them into structured data is crucial for tasks such as code review, design compliance checks, and automated design assistance.

[0003] However, the specification information in CAD drawings has a multimodal representation. Some specifications are presented in plain text, some in tables, and others in vector graphics. This multimodal representation makes the identification of specification content quite complex.

[0004] In existing technologies, rule-based algorithm-based recognition schemes have been proposed, such as identifying standard information by setting fixed layout features or extracting graphic features. While these methods can extract some standard content in certain scenarios, their generalization ability is limited, making it difficult to adapt to the diverse standard expressions in different design institutes, different disciplines, and different versions of CAD drawings. Furthermore, due to the narrow applicability of rule-based algorithms, the overall recognition accuracy has an upper limit, making it difficult to meet the high robustness requirements of engineering practice.

[0005] Therefore, how to better identify the general specification of multimodal CAD drawings in order to adapt to diverse specification expression methods has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0006] The purpose of this invention is to provide a method, apparatus, and device for identifying the general specification of multimodal CAD drawings, in order to solve the above-mentioned technical problems in the prior art.

[0007] On the one hand, in order to achieve the above objectives, the present invention provides a method for identifying the general specification of multimodal CAD drawings.

[0008] The method for identifying the general specification of the multimodal CAD drawing includes: receiving a CAD drawing to be identified, wherein the CAD drawing includes a specification to be identified that belongs to the target specification; identifying the text describing the target specification in the CAD drawing to obtain first text information; obtaining the table name and vector graphic name referenced in the text describing the target specification; extracting the table and vector graphic from the CAD drawing based on the table name and vector graphic name; identifying the text in the table and vector graphic to obtain second text information; and obtaining the structured specification data corresponding to the specification to be identified based on the first text information and the second text information.

[0009] Further, the steps of identifying the text describing the target specification in the CAD drawing, obtaining the first text information, and acquiring the table name and vector graphic name referenced in the text describing the target specification include: acquiring the preset keywords corresponding to the target specification; traversing the text elements in the CAD drawing, finding the text containing the keywords, and obtaining several candidate texts; acquiring the first prompt word template corresponding to the target specification, wherein the first prompt word template is used to determine whether the candidate text is the text describing the target specification, and when the candidate text is the text describing the target specification, outputting the first text information, table name, and vector graphic name; and inputting the candidate text and the first prompt word template into the large language model to obtain the first text information, table name, and vector graphic name.

[0010] Furthermore, the steps of traversing the text elements in the CAD drawing to find text containing keywords and obtain several candidate texts include: traversing the text elements in the CAD drawing to find text blocks containing keywords; merging text blocks belonging to the same row into text lines; obtaining each text line within a predetermined distance range to obtain candidate text lines; calculating the text vector of the candidate text lines; and merging candidate text lines whose text vector similarity meets the preset merging conditions to obtain candidate texts.

[0011] Furthermore, the steps for extracting tables and vector graphics from CAD drawings based on table names and vector graphic names include: determining the position parameters of the corresponding tables and vector graphics in the CAD drawing based on the table names and vector graphic names; cropping the area defined by the position parameters in the CAD drawing to obtain the chart area; and extracting the tables and vector graphics from the chart area.

[0012] Furthermore, the first text information includes text describing the target specification and the coordinates of the text describing the target specification. The steps of identifying text in tables and vector graphics to obtain the second text information include: reading the text and the coordinates of the text in tables and vector graphics respectively to obtain the second text information.

[0013] Further, the step of obtaining the structured specification data corresponding to the specification to be identified based on the first text information and the second text information includes: obtaining a preset second prompt word template, wherein the second prompt word template is used to extract structured specification data from the text information; and inputting the first text information, the second text information, and the second prompt word template into the large language model to obtain the structured specification data, wherein the structured specification data includes multiple specification rules, and each specification rule includes several data units and the coordinates of the data units.

[0014] Furthermore, the method also includes: displaying a jump control corresponding to the data unit; and, in response to a preset operation on the jump control, jumping to display the position indicated by the coordinates of the data unit in the CAD drawing.

[0015] On the other hand, in order to achieve the above objectives, the present invention provides a device for identifying the general specification of multimodal CAD drawings.

[0016] The recognition device for the multimodal CAD drawing general specification includes: a receiving module for receiving a CAD drawing to be recognized, wherein the CAD drawing includes a specification to be recognized belonging to the target specification; a first recognition module for recognizing the text describing the target specification in the CAD drawing to obtain first text information; an acquisition module for acquiring the table names and vector graphic names referenced in the text describing the target specification; an extraction module for extracting tables and vector graphics from the CAD drawing based on the table names and vector graphic names; a second recognition module for recognizing the text in the tables and vector graphics to obtain second text information; and a processing module for obtaining structured specification data corresponding to the specification to be recognized based on the first text information and the second text information.

[0017] On the other hand, to achieve the above objectives, the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above method.

[0018] On the other hand, to achieve the above objectives, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above method.

[0019] The present invention provides a method, apparatus, and device for identifying multimodal CAD drawing general specification standards. First, it receives a CAD drawing containing specification standards. Next, it identifies the text information in the specification section to obtain first text information. Then, it parses the referenced table names and vector graphic names from the specification text and extracts the corresponding tables and vector graphics from the CAD drawing. Further, it identifies the text information in these tables and vector graphics to obtain second text information. Finally, it generates corresponding structured specification data based on the first and second text information. Through this invention, specification information scattered in multimodal forms such as text, tables, and vector graphics can be uniformly identified and integrated to form a structured data representation. This facilitates automated review and retrieval of specifications, improves the accuracy and comprehensiveness of specification identification, and overcomes the problems of poor generalization ability and limited recognition effect of existing rule-based algorithm-based identification schemes. Attached Figure Description

[0020] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A flowchart of the method for identifying the general specification of multimodal CAD drawings provided in Embodiment 1 of the present invention; Figure 2 This is a flowchart of the method for identifying the general specification of multimodal CAD drawings provided in Embodiment 2 of the present invention; Figure 3 This is a schematic diagram illustrating the mapping of coordinate information back to the original CAD drawing in the identification method of the multimodal CAD drawing general specification provided in this embodiment of the invention. Figure 4 A block diagram of the recognition device for the multimodal CAD drawing general specification provided in Embodiment 3 of the present invention; and Figure 5 This is a hardware structure diagram of a computer device provided in Embodiment 4 of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0022] Example 1 This invention provides a method for identifying general specifications for multimodal CAD drawings. This method, without relying on the format or graphic features of the specifications, can adapt to diverse specification expression methods when identifying general specifications for multimodal CAD drawings. Specifically, Figure 1 The flowchart of the method for identifying the general specification of multimodal CAD drawings provided in Embodiment 1 of the present invention is as follows: Figure 1 As shown, the method for identifying the general specification of multimodal CAD drawings provided in this embodiment includes the following steps S101 to S106.

[0023] Step S101: Receive the CAD drawing to be recognized.

[0024] Among them, CAD drawings include unidentified specification specifications that belong to the target specification.

[0025] In this step, the CAD drawing can be a file generated by AutoCAD, Tianzheng CAD, or other CAD design software, with common formats including .dwg and .dxf. The specifications to be identified are generally located in the general description section of the CAD drawing, such as the drawing description column or a dedicated description page. Optionally, upon receiving the drawing, a drawing file format parsing operation is first performed to convert vector objects, text objects, table objects, etc., in the CAD file into a unified internal representation for subsequent processing.

[0026] Step S102: Identify the text describing the target specification in the CAD drawing to obtain the first text information.

[0027] In this step, the text content of the drawing instructions can be identified using the CAD's built-in text extraction interface, an OCR-based text recognition model, or a combination of natural language processing (NLP) methods. Further, the identified text content can be preliminarily segmented, filtered, and categorized using regular expressions or a keyword dictionary to extract text related to the target specifications, forming the first text information. This first text information can contain multiple sentences, such as "Concrete strength grade C30 for this project" or "Masonry mortar strength grade M5".

[0028] Step S103: Obtain the table names and vector graphic names referenced in the text describing the target specification.

[0029] Based on CAD drawing descriptions such as "see Table 1" and "see..." Figure 3-1 "The characteristics of this type of cross-referencing can be addressed in this step by using regular expressions to match identifiers such as 'Table ×' and 'Figure ×' to extract table names and vector graphic names, and establishing an index relationship with CAD objects. Further, alternatively, if there is ambiguity in the table name or vector graphic name, it can be corrected and its uniqueness determined by using object attributes (such as layer name and object ID) in the CAD file."

[0030] Step S104: Extract the table and vector graphics from the CAD drawing based on the table name and vector graphics name.

[0031] This step uses the CAD file parsing module to locate and extract the table and vector object corresponding to the referenced names. For table objects, their rows, columns, cells, and content can be directly extracted; for vector object objects, their lines, annotations, text descriptions, etc., can be extracted. Optionally, the vector object can also be rendered as an image for subsequent OCR recognition.

[0032] Step S105: Identify the text in the table and vector graphics to obtain the second text information.

[0033] In the process of table recognition, the table content can be extracted row by row and column by column to form two-dimensional structured data. In vector graphic recognition, in addition to OCR recognition of text in graphics, the relationship between symbols and explanatory text can also be identified by combining the topological relationships of graphics. For example, the annotation text "HRB400 φ12@200" next to a rebar symbol can be used as a complete specification data entry, ultimately forming the second text information.

[0034] Step S106: Obtain the structured specification data corresponding to the specification to be identified based on the first text information and the second text information.

[0035] In this step, the first and second text information are merged, deduplicated, and formatted to generate structured specification data. This data can be stored in JSON, XML, or database table format, with each piece of specification data including attribute fields (such as specification type, parameter value, and referenced object) and corresponding descriptions. The resulting structured data can be directly used in specification review systems or automated design systems.

[0036] In the multimodal CAD drawing specification recognition method provided in this embodiment, a CAD drawing containing specification information is first received. Then, the text information of the specification section is recognized to obtain first text information. Next, the referenced table names and vector graphic names are parsed from the specification text, and the corresponding tables and vector graphics are extracted from the CAD drawing. Further, the text information in these tables and vector graphics is recognized to obtain second text information. Finally, corresponding structured specification data is generated based on the first and second text information. Using the multimodal CAD drawing specification recognition method provided in this embodiment, specification information scattered in multimodal forms such as text, tables, and vector graphics can be uniformly recognized and integrated to form a structured data representation. This facilitates automated review and retrieval of specifications, improves the accuracy and comprehensiveness of specification recognition, and overcomes the problems of poor generalization ability and limited recognition effect of existing rule-based algorithm-based recognition schemes.

[0037] Optionally, in one embodiment, the steps of identifying text describing the target specification in the CAD drawing to obtain first text information and obtaining the table name and vector graphic name referenced in the text describing the target specification include: obtaining preset keywords corresponding to the target specification; traversing the text elements in the CAD drawing to find text containing the keywords and obtaining several candidate texts; obtaining a first prompt word template corresponding to the target specification, wherein the first prompt word template is used to determine whether the candidate text is text describing the target specification, and when the candidate text is text describing the target specification, outputting the first text information, table name and vector graphic name; inputting the candidate text and the first prompt word template into the large language model to obtain the first text information, table name and vector graphic name.

[0038] Specifically, for different types of explanatory specifications, a set of keywords and a first prompt word template are preset. Keywords can be obtained through manual compilation of a specification terminology dictionary or automatically generated using statistical methods based on historical drawing data. The first prompt word template's function is to use a large language model to determine whether candidate text describes the target specification, and if so, output the first text information, table name, and vector graphic name. This first prompt word template can adopt a parameterized prompt structure, such as: "Please determine whether the following text describes a building specification. If so, please extract the specification content, the relevant table names, and the relevant drawing names, and output them in JSON format." For example, including: { Rule Name: Slab Distribution Reinforcement Definition: "Refined structural steel bars are placed perpendicular to the main reinforcing bars in cast-in-place slabs, composite slabs, and one-way (two-way) slabs. Excludes: window sills, stair slabs, and platform slabs." Core semantics: "Setting distribution reinforcement within slab-type members and explicitly mentioning the reinforcement parameters:\n Specific parameters: grade, diameter, spacing (e.g., A6@200, A6);\n Implicit parameters: parameters are indirectly reflected through referenced tables / figures (must conform to table extraction rules)," "Judgment Principles": "1. Directly judged as compliant (match=1)\nWhen the text simultaneously meets the following conditions:\n1 Component type matching:\nKeywords for slab components appear (e.g., cast-in-place slab, composite slab, combined slab, one-way / two-way slab, etc.);\nThe component type is not explicitly mentioned, but unless explicitly stated as another component, it can be considered a slab component. For example: *not specified\nEven if the text mentions excluded component types (e.g., window sill slab, stair slab, platform slab, etc.), as long as the semantics contain slab components that meet the rules other than these excluded components, the component type can still be considered to match.\n2 Reinforcement description matching: Contains words such as "distribution reinforcement, distribution steel reinforcement, structural reinforcement";\n3 Parameter information matching: Directly provides reinforcement parameters (e.g., A6@200, C8@150) or references tables / figures directly related to slab distribution reinforcement parameters (e.g., "set according to the table", "see details"). Figure 3 2. Directly judged as non-compliant (match=0) when any of the following situations occur: 2.1 Component type exclusion: The component type is beam, column, wall, foundation, window sill, stair slab, platform slab, or opening, and there is no slab component part that conforms to the rules. 2.2 Missing parameter information: The reinforcement parameters are not mentioned (e.g., A6@200); and no table / figure directly related to the slab distribution reinforcement parameters is referenced (e.g., "the following table"). Figure 1 ”); } This first prompt word template, through standardized description of the task, ensures that subsequent large language models can accurately judge and extract information based on candidate texts.

[0039] Based on the aforementioned preset keywords and first prompt word templates, when identifying text describing the target specification and obtaining the table and vector map names referenced in the text, the CAD drawing is first parsed. During the CAD drawing parsing process, all text elements are extracted and matched one by one with the keywords. Candidate text is obtained by finding text elements containing the keywords. For example, if the phrase "The concrete strength grade of this project is C30, see Table 1 for details" is detected in the CAD drawing description, then this sentence is taken as candidate text. Then, the candidate text and prompt word template are used as input to call a large language model (such as GPT, deepseek, etc.) for inference. The large language model will output the judgment result. If the candidate text is text describing the target specification, then the specification content in the text is further extracted as the first text information, and the table and vector map names referenced are parsed out. For example, for the candidate text "The concrete strength grade of this project is C30, see Table 1, see...", the prompt word template is used as input. Figure 3-1 After input, the output results are as follows: The first text information includes "Concrete strength grade is C30"; the table name includes "Table 1"; the vector image name includes " Figure 3-1 ".

[0040] In the multimodal CAD drawing specification identification method provided in this embodiment, keywords and prompt word templates are pre-set for the target specification. After traversing all text elements in the CAD drawing and filtering out candidate texts containing the keywords, the prompt word templates corresponding to the target specification are used to judge the candidate texts. Inference is then performed using a large language model to extract the first text information describing the target specification, as well as the table names and vector graphic names referenced therein. Using the multimodal CAD drawing specification identification method provided in this embodiment, the relevant content of the target specification can be efficiently filtered from a large amount of CAD drawing specification text. The large language model enables accurate parsing of the specification text and its referenced objects. Compared with traditional rule-based matching methods, this method has better generalization ability and recognition accuracy in cross-domain and cross-expression specification identification scenarios.

[0041] Optionally, in one embodiment, the step of traversing the text elements in the CAD drawing, finding text containing keywords, and obtaining several candidate texts includes: traversing the text elements in the CAD drawing, finding text blocks containing keywords; merging text blocks belonging to the same row into text lines; obtaining each text line within a predetermined distance range to obtain candidate text lines; calculating the text vector of the candidate text lines; and merging candidate text lines whose text vectors satisfy preset merging conditions to obtain candidate texts.

[0042] Specifically, the process begins by reading the text objects from the CAD file and matching them against preset keywords. Any text blocks containing the keywords are marked as candidate objects. Then, based on the coordinates of the text blocks, it's determined whether they are at the same baseline height. If they belong to the same horizontal line, these text blocks are concatenated sequentially to form a complete text line. By calculating the vertical or Euclidean distance between text lines, lines within a set threshold range (e.g., vertical spacing not exceeding 20 pixels) are selected as candidate text lines, ensuring that multi-line standardized content can be extracted completely. For each text line, it is vectorized. Optionally, TF-IDF, Word2Vec embedding models, or embedding interfaces of large language models such as BERT are used to convert each text line into a multi-dimensional semantic vector for calculating language similarity and / or relevance. The process then determines whether preset merging conditions are met, merging candidate text lines that meet these conditions into a single entity to obtain the final candidate text. For example, if "The concrete strength grade of this project is C30" and "The reinforcement details are shown in Table 1" are in adjacent rows and have a high degree of correlation, they are merged into a single candidate text for further identification and analysis in subsequent steps.

[0043] In the multimodal CAD drawing specification recognition method provided in this embodiment, text blocks containing keywords are extracted from the CAD drawing, and text blocks in the same line are merged into a single text line. Then, relevant text lines within a predetermined distance range are selected as candidate text lines. The text vectors of the candidate text lines are calculated, and a merging condition is determined based on the text vectors. Candidate text lines that meet the merging condition are merged to obtain the final candidate text. This multimodal CAD drawing specification recognition method effectively solves the problem of incomplete recognition results caused by text being split, split into lines, or described across lines in CAD drawings. Through semantic merging based on text vectors, not only is the integrity of the candidate text guaranteed, but the accuracy of extracting the target specification description is also improved, avoiding omissions and misjudgments caused by methods based solely on coordinates or rules.

[0044] Optionally, in one embodiment, the step of extracting tables and vector graphics from a CAD drawing based on table names and vector graphic names includes: determining the position parameters of the corresponding tables and vector graphics in the CAD drawing based on the table names and vector graphic names; cropping the area defined by the position parameters in the CAD drawing to obtain a chart area; and extracting the tables and vector graphics from the chart area.

[0045] Specifically, the process begins by searching for the corresponding target object in the CAD drawing object index based on the table and vector graphic names parsed in the previous steps. For example, table objects can be mapped to object IDs via table names, and vector graphic objects can be mapped to layer identifiers via drawing names. Position parameters can include the object's start and end coordinates, bounding box range, and object number. Optionally, page numbers can also be added to the position parameters to distinguish objects with the same name on different drawing pages. The above position parameters are then used to perform a region cropping operation on the CAD drawing. If the CAD file is a vector file, all objects within the region are extracted based on the boundary coordinates; if the CAD file needs to be rendered as an image for OCR recognition, the corresponding region is cropped from the rendered image to obtain the chart area image, ensuring that tables or vector graphics are completely preserved. The cropped chart area is then further analyzed. If it is a table area, its row and column structure is identified, cell content is extracted, and two-dimensional table data is formed; if it is a vector graphic area, the lines, symbols, labels, and attached text are parsed, preserving its topological structure. Optionally, the vector graphic area can also be rendered into an image and then subjected to OCR recognition to extract the text descriptions attached to the graphic. The final result is complete tabular and vector data, providing input for the subsequent identification and integration of standardized information.

[0046] In the multimodal CAD drawing specification identification method provided in this embodiment, the position parameters of the table and vector graphics obtained from parsing are located in the CAD drawing. Then, the corresponding area of ​​the CAD drawing is extracted based on the position parameters to obtain the chart area. Finally, the chart area is parsed to extract the tables or vector graphics. Using the multimodal CAD drawing specification identification method provided in this embodiment, the table and vector graphics objects referenced in the specification text can be accurately located and extracted, achieving unified acquisition of tables and vector graphics in CAD drawings and ensuring the completeness and universality of specification identification.

[0047] Optionally, in one embodiment, the first text information includes text describing the target specification and the coordinates of the text describing the target specification. The step of identifying text in the table and vector graphics to obtain the second text information includes: reading the text and the coordinates of the text in the table and vector graphics respectively to obtain the second text information.

[0048] Specifically, in the CAD drawing annotation recognition process, the extracted first text information includes not only the text content itself but also its position coordinates within the CAD drawing. These coordinates can be the coordinates of the text's base point, the coordinates of the top-left and bottom-right corners of the bounding box, or absolute coordinate values ​​from the vector file. Correspondingly, for table areas, the text and its position in the table's coordinate system are read cell by cell, such as row and column indices and cell boundary coordinates, thus obtaining text-coordinate pairs. For vector graphics areas, the annotation text in the vector graphics is recognized, the text content is extracted, and the text's insertion point or bounding box coordinates are recorded. For example, recognizing the text "HRB400 φ12@200" corresponds to coordinates (125, 340). Finally, the text content in the table and vector graphics, along with their corresponding coordinates, together constitute the second text information.

[0049] In the multimodal CAD drawing specification identification method provided in this embodiment, the identified text information includes not only the textual description of the target specification but also the coordinate position of the text in the CAD drawing. This allows for the further binding of spatial location information to the textual content, achieving a one-to-one correspondence between the specification content and the drawing location. This ensures the integrity and traceability of the specification information identification results.

[0050] Optionally, in one embodiment, the step of obtaining the structured specification data corresponding to the specification to be identified based on the first text information and the second text information includes: obtaining a preset second prompt word template, wherein the second prompt word template is used to extract structured specification data from the text information; and inputting the first text information, the second text information, and the second prompt word template into a large language model to obtain structured specification data, wherein the structured specification data includes multiple specification rules, and each specification rule includes several data units and the coordinates of the data units.

[0051] Specifically, a prompt template for extracting standard information is pre-built. For example, a rule-based prompt could be set as follows: "Please extract the standard entries from the input standard text. Each standard entry should contain data units (such as strength grade, reinforcement spacing, seismic grade) and their corresponding coordinate information, and output in JSON format." This can be customized according to different standard types (concrete, masonry, structural construction, etc.) to improve extraction accuracy. Based on the second prompt template, the first text information (including text and coordinates) identified from the explanatory section, and the second text information (including text and coordinates) obtained from the table and vector diagram parsing, along with the second prompt template, are used as input and processed by the large language model. The large language model identifies standard entries from the input text according to the requirements of the second prompt template and outputs structured results.

[0052] The structured specification data includes multiple explanatory specifications, each consisting of several data units. Each data unit contains specific text content (such as "concrete strength grade = C30", "rebar spacing = 200mm") and corresponding coordinate information. The division of data units is based on the coordinate information corresponding to all content in the explanatory specifications. The explanatory specifications are divided into several data units according to several coordinate information, so that there is a one-to-one correspondence between data units and coordinate information.

[0053] In the multimodal CAD drawing specification identification method provided in this embodiment, a pre-set second prompt word template guides a large language model to extract multiple specification rules from the input text information, forming structured data. Each specification rule consists of multiple data units and their coordinate information. Using the multimodal CAD drawing specification identification method provided in this embodiment, the scattered text, tables, and vector graphics content in CAD drawings can be automatically extracted into structured data. This not only ensures the integrity and consistency of the specification information but also achieves traceability between the data and the original drawing through coordinate information.

[0054] Optionally, in one embodiment, the method for identifying the multimodal CAD drawing specification further includes: displaying a jump control corresponding to the data unit; and, in response to a preset operation on the jump control, jumping to the position indicated by the coordinates of the data unit in the CAD drawing.

[0055] Specifically, for each data unit in the structured specification data, a jump control is generated and displayed. The jump control can be a button, hyperlink, icon, list item, or display unit that synchronously displays the data unit in the interface. For example, in the specification data table of the system interface, a "Locate" button is displayed next to the data unit "Concrete Strength Grade = C30," or the data unit itself is a clickable button. When the user performs a preset operation (such as clicking or double-clicking the control), the system locates the corresponding position in the CAD drawing based on the coordinate information of that data unit. Optionally, the positioning method may include panning the view to the coordinate point and centering it, zooming in on the drawing area near the coordinate point to a preset scaling ratio, and displaying a highlighted mark or border at the jump location to indicate the source location of the corresponding specification information to the user.

[0056] In the multimodal CAD drawing specification identification method provided in this embodiment, a corresponding jump control is displayed for each data unit. When the user performs a preset operation, the corresponding position in the CAD drawing is displayed based on the coordinate information of the data unit. This multimodal CAD drawing specification identification method achieves interactive linkage between structured specification data and original CAD drawings. Users can quickly locate the specific position of the data unit in the original drawing using the control. This approach improves the traceability and verifiability of specification data and facilitates bidirectional review and comparison between structured data and original drawings for designers and reviewers, thereby enhancing the efficiency of specification review and design assistance.

[0057] Example 2 Figure 2 The flowchart is a method for identifying the general specification of multimodal CAD drawings provided in Embodiment 2 of the present invention, as follows: Figure 2 As shown, the method for identifying the general specification of multimodal CAD drawings provided in this embodiment specifically includes the following steps: 1. Getting Started. The user imports the CAD drawings to be recognized and specifies the standards (or a list of standards) to be recognized. 2. Search for candidate text using keywords. For the specification to be identified, first use predefined keywords (for example, in the scenario of "additional reinforcement at the bottom of a beamless slab under a wall," keywords could include: masonry wall, cast-in-place slab, additional reinforcement, etc.) to traverse all text elements in the CAD file and find all text containing the above keywords. Merge text within the same line, and then merge text within a certain vertical distance based on text vector similarity to obtain a series of candidate texts.

[0058] 3. Identify precise text using a large-scale model. Using the specified specification name, extract the corresponding prompt word template from a pre-configured prompt word library. Then, using a large-scale language model, determine which of the candidate texts from the previous step are the precise texts describing this specification. The specification name and prompt word template are in dictionary format, with the specification name as the key and the prompt word template as the value.

[0059] 4. Identify table and figure names referenced in the text. Similarly, using the prompt word template shown above, extract the table and vector graphic names referenced in the specification from the precise text describing this specification.

[0060] 5. Locate candidate tables and figures. Locate the approximate position of tables and vector graphics on the drawing by matching the text of the table names and vector graphic names referenced in the specification.

[0061] 6. Classify precise tables and images. Select an area near the approximate location, such as 300 pixels wide on each side (top, bottom, left, and right), and use object detection methods to locate the images and tables, finding candidate images and candidate tables.

[0062] 7. Extract text and coordinates from tables and graphs. Classify the candidate graphs using a convolutional network, and classify the candidate table contents using a text model to obtain the precise graphs and tables corresponding to the specification. Then, convert the precise graphs and tables corresponding to the specification into plain text. The specific conversion method is as follows: For tables, read the text and its coordinates, and finally convert it into a one-dimensional list of text and coordinates. The origin of the coordinates can be the top left corner of the table. A format similar to [slab thickness (0,0,15,15), rebar type (20,20,50,50)...] can be used.

[0063] For graphs, read the text and text coordinates from the graph, and finally convert them into a one-dimensional list of text and coordinates. A format similar to [gluten (0,0,15,15), base gluten (20,20,50,50)...].

[0064] 8. Integrate precise text and extract structured specification results using a large model. By combining the precise text describing the specification with its coordinates, as well as the text converted from tables and coordinate lists, and the text converted from graphs and coordinate lists, a large language model is used to obtain the final structured specification data through prompt word templates, thus completing the automatic extraction of the specification. Similar formats are shown below: { "SideReinfBeam": [{"ExtractedParams": [ "['Beam type', 'Web height', 'Beam width', 'Stretcher parameters', 'Conversion factor for number of stirrups on both sides']", "['beam (0, 0, 20, 20)', 'greater than 450 (50, 70, 90, 100)', 'less than or equal to 300 (70, 80, 100, 100)', 'GC10@200 (60, 60, 100, 120)', '2' (50, 80, 90, 100)]", "['beam (10, 10, 30, 30)', 'greater than 450 (90, 100, 130, 140)', 'greater than 300 (170, 180, 200, 200)', 'GC12@200 (160, 160, 200, 220)', '2 (150, 180, 190, 200)']"]}] } Figure 3 This is a schematic diagram illustrating the mapping of coordinate information back to the original CAD drawing in the identification method of the multimodal CAD drawing general specification provided in this embodiment of the invention. Figure 3 As shown, after the standard extraction is completed, the recognition results are displayed, and the coordinate information in the results is mapped back to the position of the original CAD drawing, which is convenient for users to check and correct errors.

[0065] In the method for recognizing the general specification of multimodal CAD drawings provided in this embodiment, the precise text describing a certain specification and the referenced tables and vector graphics are first obtained. Then, by converting the tables and vector graphics into text + coordinates and adding the precise text describing the specification, and using prompt word engineering and a large language model, high-accuracy automated specification recognition is achieved.

[0066] The process involves: extracting candidate specification text; identifying precise specification text; identifying referenced vector and table names; locating and classifying vector and table data to obtain text, table, and vector information describing a specification. By extracting text and coordinates from tables and vector graphics, they are converted into plain text. This, combined with the text describing the specification, is then parsed using a large language model through prompt word engineering to extract a structured specification for downstream tasks. Fields in the recognition results are mapped back to the original CAD drawings using retained coordinate information and visualized for easy inspection and error correction. These steps address the current problem of poor recognition performance for general specifications in multimodal CAD drawings, achieving joint recognition of text, figures, and tables with high accuracy. On test data, an average accuracy of 92% was achieved.

[0067] Example 3 Corresponding to Embodiment 1 above, Embodiment 2 of the present invention provides a device for identifying the general specification of multimodal CAD drawings. The technical features and corresponding technical effects can be referred to Embodiment 1 above, and will not be repeated in this embodiment. Figure 4 This is a block diagram of the recognition device for the multimodal CAD drawing general specification provided in Embodiment 2 of the present invention, as shown below. Figure 4 As shown, the device includes: a receiving module 201, a first identification module 202, an acquisition module 203, an extraction module 204, a second identification module 205, and a processing module 206.

[0068] The receiving module 201 is used to receive a CAD drawing to be identified, wherein the CAD drawing includes a specification to be identified that belongs to the target specification; the first identification module 202 identifies the text describing the target specification in the CAD drawing to obtain first text information; the acquisition module 203 is used to acquire the table name and vector graphic name referenced in the text describing the target specification; the extraction module 204 is used to extract the table and vector graphic from the CAD drawing according to the table name and vector graphic name; the second identification module 205 is used to identify the text in the table and vector graphic to obtain second text information; and the processing module 206 is used to obtain the structured specification data corresponding to the specification to be identified according to the first text information and the second text information.

[0069] Optionally, in one embodiment, the first identification module and the acquisition module include: a first acquisition unit, configured to acquire preset keywords corresponding to the target specification; a traversal unit, configured to traverse the text elements in the CAD drawing, find the text containing the keywords, and obtain a number of candidate texts; a second acquisition unit, configured to acquire a first prompt word template corresponding to the target specification, wherein the first prompt word template is used to determine whether the candidate text is the text describing the target specification, and when the candidate text is the text describing the target specification, outputting the first text information, the table name, and the vector image name; and a first input unit, configured to input the candidate text and the first prompt word template into a large language model to obtain the first text information, the table name, and the vector image name.

[0070] Optionally, in one embodiment, the traversal unit uses the following steps to obtain several candidate texts: traversing the text elements in the CAD drawing to find text blocks containing the keyword; merging the text blocks belonging to the same row into a text line; obtaining each text line within a predetermined distance range to obtain candidate text lines; calculating the text vector of the candidate text lines; and merging the candidate text lines whose text vector similarity satisfies a preset merging condition to obtain the candidate text.

[0071] Optionally, in one embodiment, the extraction module includes: a determining unit, configured to determine the position parameters of the corresponding table and vector graphic in the CAD drawing based on the table name and vector graphic name; a cropping unit, configured to crop the area defined by the position parameters in the CAD drawing to obtain a chart area; and an extraction unit, configured to extract the table and vector graphic in the chart area.

[0072] Optionally, in one embodiment, the first text information includes the text describing the target specification and the coordinates of the text describing the target specification, and the second recognition module is further configured to read the text and the coordinates of the text in the table and the vector graphic respectively to obtain the second text information.

[0073] Optionally, in one embodiment, the processing module includes: a third acquisition unit, configured to acquire a preset second prompt word template, wherein the second prompt word template is used to extract structured normative data from text information; and a second input unit, configured to input the first text information, the second text information, and the second prompt word template into a large language model to obtain the structured normative data, wherein the structured normative data includes multiple explanatory norms, and each explanatory norm includes several data units and the coordinates of the data units.

[0074] Optionally, in one embodiment, the device further includes: a display module for displaying a jump control corresponding to the data unit; and a response module for responding to a preset operation on the jump control to jump to and display the position indicated by the coordinates of the data unit in the CAD drawing.

[0075] Example 4 This embodiment also provides a computer device, such as a smartphone, tablet computer, laptop computer, desktop computer, rack server, blade server, tower server, or cabinet server (including a standalone server or a server cluster composed of multiple servers), etc., capable of executing programs. Figure 5 As shown, the computer device 01 in this embodiment includes, but is not limited to, a memory 012 and a processor 011 that can be interconnected via a system bus, such as... Figure 5 As shown. It should be noted that, Figure 5 Only a computer device 01 with component memory 012 and processor 011 is shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0076] In this embodiment, the memory 012 (i.e., the readable storage medium) includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 012 may be an internal storage unit of the computer device 01, such as the hard disk or memory of the computer device 01. In other embodiments, the memory 012 may also be an external storage device of the computer device 01, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 01. Of course, the memory 012 may include both the internal storage unit and its external storage device of the computer device 01. In this embodiment, the memory 012 is typically used to store the operating system and various reference software installed on the computer device 01, such as the program code of the identification device for the multimodal CAD drawing general specification in Embodiment 3. In addition, memory 012 can also be used to temporarily store various types of data that have been output or will be output.

[0077] In some embodiments, processor 011 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. Processor 011 is typically used to control the overall operation of computer device 01. In this embodiment, processor 011 is used to run program code stored in memory 012 or process data, such as a method for identifying the general specification of multimodal CAD drawings.

[0078] Example 5 This embodiment also provides a computer-readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, server, App reference store, etc., which stores computer programs. When the program is executed by a processor, it implements corresponding functions. The computer-readable storage medium of this embodiment is used to store a device for identifying the general specification of multimodal CAD drawings. When executed by a processor, it implements the identification method for the general specification of multimodal CAD drawings of each embodiment.

[0079] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0080] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0081] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0082] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method of identifying a multi-modal CAD drawing specification specification, characterized in that, The method comprises: receiving a CAD drawing to be identified, wherein the CAD drawing comprises to-be-identified specification specifications belonging to a target specification; identifying text in the CAD drawing describing the target specification to obtain first text information; obtaining table names and vector diagram names referenced in the text describing the target specification; extracting tables and vector diagrams in the CAD drawing according to the table names and the vector diagram names; identifying text in the tables and the vector diagrams to obtain second text information; and obtaining structured specification data corresponding to the to-be-identified specification specification according to the first text information and the second text information.

2. The method of claim 1, wherein, The steps of identifying text in the CAD drawing describing the target specification to obtain first text information and obtaining table names and vector diagram names referenced in the text describing the target specification comprise: obtaining a preset keyword corresponding to the target specification; traversing text elements in the CAD drawing to find text containing the keyword to obtain a plurality of candidate texts; obtaining a first prompt word template corresponding to the target specification, wherein the first prompt word template is used to determine whether the candidate text is the text describing the target specification, and when the candidate text is the text describing the target specification, the first text information and the table names and the vector diagram names are outputted; inputting the candidate text and the first prompt word template into a large language model to obtain the first text information, the table names and the vector diagram names.

3. The method of claim 2, wherein: The steps of traversing text elements in the CAD drawing to find text containing the keyword to obtain a plurality of candidate texts comprise: traversing text elements in the CAD drawing to find text blocks containing the keyword; merging the text blocks belonging to the same line into text lines; obtaining each text line within a predetermined distance range to obtain a candidate text line; calculating a text vector of the candidate text line; merging candidate text lines with a text vector similarity satisfying a preset merging condition to obtain the candidate text.

4. The method of claim 1, wherein: The steps of extracting tables and vector diagrams in the CAD drawing according to the table names and the vector diagram names comprise: determining position parameters of corresponding tables and vector diagrams in the CAD drawing according to the table names and the vector diagram names; cutting out a chart area defined by the position parameters in the CAD drawing; and extracting tables and vector diagrams in the chart area.

5. The method of claim 1, wherein: The first text information comprises the text describing the target specification and coordinates of the text describing the target specification, and the steps of identifying text in the tables and the vector diagrams to obtain second text information comprise: respectively reading text and coordinates of the text in the tables and the vector diagrams to obtain the second text information.

6. The method of identifying a multi-modal CAD drawing specification according to claim 5, wherein, The steps of obtaining structured specification data corresponding to the to-be-identified specification specification according to the first text information and the second text information comprise: obtaining a preset second prompt word template, wherein the second prompt word template is used to extract structured specification data from text information; and inputting the first text information, the second text information and the second prompt word template into a large language model to obtain the structured specification data, wherein the structured specification data comprises a plurality of specification descriptions, and each specification description comprises a plurality of data units and coordinates of the data units.

7. The method of claim 6, wherein: The method further comprises: displaying a jump control corresponding to the data unit; in response to a preset operation on the jump control, jumping to display a position in the CAD drawing indicated by the coordinates of the data unit.

8. A device for identifying a multi-modal CAD drawing master specification, characterized in that, comprises: a receiving module configured to receive a CAD drawing to be identified, wherein the CAD drawing comprises an identified specification description belonging to a target specification; a first identifying module configured to identify text in the CAD drawing describing the target specification to obtain first text information; an obtaining module configured to obtain table names and vector diagram names referred to in the text describing the target specification; an extracting module configured to extract tables and vector diagrams from the CAD drawing according to the table names and the vector diagram names; a second identifying module configured to identify text in the tables and the vector diagrams to obtain second text information; and a processing module configured to obtain structured specification data corresponding to the identified specification description according to the first text information and the second text information.

9. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the method of any one of claims 1 to 7.

10. A computer readable storage medium having stored thereon a computer program, characterized in that: The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 7.

Images