An engineering drawing structured information extraction method, device and medium

Abstract

The application discloses an engineering drawing structured information extraction method and device and a medium, and relates to the technical field of digital analysis and information structuring of engineering drawings. The method comprises the following steps: acquiring a DXF file corresponding to a CAD drawing, and acquiring a global coordinate range of a frame in a DXF global coordinate; rendering the content in the frame range into a raster image, inputting the raster image into a multi-modal analysis model for region identification to obtain an image channel identification result, and establishing a bidirectional linear mapping relationship between the DXF global coordinate and a pixel coordinate; based on the pixel coordinate range of each feature region, the bidirectional linear mapping relationship is used to perform targeted extraction from the DXF file to obtain a vector channel extraction result corresponding to the DXF file; the image channel identification result and the vector channel extraction result are cross-verified to obtain correction information corresponding to the DXF file; and the correction information and the global coordinate range of the frame unit are integrated to obtain structured engineering drawing information data.

Description

Technical Field

[0001] This application relates to the field of digital analysis and information structuring of engineering drawings, and in particular to a method, device and medium for extracting structured information from engineering drawings. Background Technology

[0002] Computer-aided design (CAD) construction drawings are generally core documents in the industry. For example, architectural construction drawings are core technical documents in the construction industry, containing all the information of the architectural design scheme. With the accelerated advancement of digital transformation in the construction industry, converting the massive amounts of historically accumulated CAD construction drawings into digital structured data that can be processed by computers has become a fundamental requirement for downstream applications such as Building Information Modeling (BIM) construction, project cost accounting, and intelligent review of building codes.

[0003] However, CAD construction drawings are essentially graphical representations designed for human reading, not structured data intended for machine processing. Information in these drawings is scattered across various media such as geometric primitives, text annotations, dimension markers, tables, and blocks. How to efficiently and accurately extract structured information from these unstructured engineering drawings is a core technical challenge in this field. Summary of the Invention

[0004] This application provides a method, device, and medium for extracting structured information from engineering drawings, in order to solve the following technical problem: how to efficiently and accurately extract structured information from unstructured engineering drawings.

[0005] In a first aspect, embodiments of this application provide a method for extracting structured information from engineering drawings. The method includes: acquiring a DXF file corresponding to a computer-aided design (CAD) drawing, identifying a frame entity in the DXF file, and acquiring the global coordinate range of the frame in the DXF global coordinates, wherein the frame is the drawing area defined by the frame entity. The content within the frame area is rendered as a raster image, and the raster image is input into a multimodal analysis model for region recognition. The pixel coordinate range and region content of the feature region in the raster image are output to obtain the image channel recognition result corresponding to the DXF file. The feature region is the region in the raster image that contains the main drawing content of the drawing. Based on the linear mapping relationship between the global coordinate range of the frame and the pixel size of the raster image, a bidirectional linear mapping relationship between DXF global coordinates and pixel coordinates is established. Based on the pixel coordinate range of each feature region, the global coordinate range of the frame unit is calculated through the bidirectional linear mapping relationship, and targeted extraction is performed from the DXF file based on the global coordinate range of the frame unit to obtain the vector channel extraction result corresponding to the DXF file. The frame unit is the region within the frame corresponding to the feature region. The image channel recognition results are cross-validated with the vector channel extraction results to obtain the correction information corresponding to the DXF file; The correction information and the global coordinate range of the drawing frame unit are integrated to obtain structured engineering drawing information data.

[0006] In one possible implementation, the feature region includes a text region, a table region, and a graphic region; For the identified text region, the multimodal parsing model outputs the pixel coordinate range of the text region and the text string identified from the text region; For the identified table region, the multimodal parsing model outputs the pixel coordinate range of the table region and the structured table text; For the identified graphic region, the multimodal parsing model outputs the pixel coordinate range of the graphic region and the initial graphic type.

[0007] In one possible implementation, after inputting the raster image into a multimodal analytical model for region recognition and outputting the pixel coordinate range and region content of the feature regions in the raster image, the method further includes: For the graphic region, a multimodal large language model is used to perform semantic understanding on the local image of the graphic region to obtain the precise graphic type corresponding to the graphic region, the semantic content description of the graphic region, and the key text annotations identified in the graphic region.

[0008] In one possible implementation, the bidirectional linear mapping relationship is specifically as follows: For any pixel (px, py) in the feature region R_i, the global coordinates (gx, gy) corresponding to the pixel are calculated by the following formula: gx=Xmin_i+px (Xmax_i-Xmin_i) / W_i; gy=Ymax_i-py (Ymax_i-Ymin_i) / H_i; For the global coordinates (gx, gy) corresponding to the pixel, the pixel coordinates (px, py) corresponding to the pixel in the feature region are calculated by the following formula: px=W_i (gx-Xmin_i) / (Xmax_i-Xmin_i); py=H_i (Ymax_i-gy) / (Ymax_i-Ymin_i); Wherein, the global coordinate range of the frame unit is (Xmin_i, Ymin_i) ~ (Xmax_i, Ymax_i), where Xmin_i represents the minimum X-axis coordinate corresponding to the global coordinate range of the frame unit, Ymin_i represents the minimum Y-axis coordinate corresponding to the global coordinate range of the frame unit, Xmax_i represents the maximum X-axis coordinate corresponding to the global coordinate range of the frame unit, and Ymax_i represents the maximum Y-axis coordinate corresponding to the global coordinate range of the frame unit. W_i and H_i are the pixel width and height of the feature region.

[0009] In one possible implementation, after establishing the bidirectional linear mapping relationship between DXF global coordinates and pixel coordinates, the method further includes: For each of the graphic regions, the pixel size of the graphic region in the raster image is checked, wherein the graphic region is rendered at a first resolution; When the pixel size is lower than the minimum pixel threshold corresponding to the graphic type of the graphic region, the global coordinate range of the graphic region is calculated through the bidirectional linear mapping relationship, the global coordinate range of the graphic region is extracted from the DXF file and re-rendered at the second resolution to generate a new high-resolution image, and the semantic understanding step is re-executed for the high-resolution image, wherein the second resolution is higher than the first resolution.

[0010] In one possible implementation, the targeted extraction of the global coordinate range based on the frame element from the exchange format file includes: Call the DXF parsing component to search for all text entities within the global coordinate range of the frame unit, and extract the text string, insertion point coordinates, and layer name of each text entity; The extracted text entities are grouped according to layer names, and the text in the same layer is sorted according to spatial position to form the text sequence of the layer; Within the global coordinate range of the frame unit, search all block reference entities, and for each block reference entity, extract the block name, internal text of the block, and attribute fields.

[0011] In one possible implementation, the step of cross-validating the image channel recognition result with the vector channel extraction result to obtain the correction information corresponding to the DXF file includes: For the text region, the first string in the image channel recognition result and the second string in the vector channel extraction result are used as two independent raw data streams and input into the large language model for cross-validation to obtain the text correction result output by the large language model. The first string and the second string are within the same coordinate range of the same text region. The prompts from the large language model include similar error patterns obtained by vector retrieval of inconsistent character fragments in a character error correction knowledge base, along with the correct corrections, provided as supplementary context to the large language model. For the table region, the structured table text in the image channel recognition result is cross-validated with the table-related text in the vector channel extraction result to obtain the table correction result, wherein the structured table text and the table-related text are within the same coordinate range of the same table region; For the graphic region, the semantic content description of each graphic region in the image channel recognition result is cross-validated with the text and block information extracted from the global coordinate range of the graphic region in the vector channel extraction result to obtain the graphic correction result; Based on the text correction results, the table correction results, and the graphic correction results, the correction information corresponding to the DXF file is obtained.

[0012] In one possible implementation, integrating the correction information and the global coordinate range of the drawing frame unit to obtain structured engineering drawing information data includes: The correction information and the global coordinate range of the drawing frame unit are input into the large language model according to the requirements of the preset structured data template to obtain structured engineering drawing information data. The engineering drawing information data is input into the large language model again. The large language model infers and completes the semantic index relationship between each graphic area based on the drawing specifications, and adds the semantic index relationship to the engineering drawing information data.

[0013] Secondly, embodiments of this application also provide an engineering drawing structured information extraction device, the device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform an engineering drawing structured information extraction method as described above.

[0014] Thirdly, embodiments of this application also provide a computer storage medium storing computer-executable instructions, which, when executed, implement a method for extracting structured information from engineering drawings as described above.

[0015] The method, device, and medium for extracting structured information from engineering drawings provided in this application have the following beneficial effects: In this embodiment, an architecture for parallel processing of image and vector channels is constructed to achieve independent extraction of dual-channel data. The image channel utilizes a multimodal analytical model to perform region analysis on the raster image, segmenting and identifying feature regions to obtain the image channel recognition result. By establishing a bidirectional linear mapping relationship between DXF global coordinates and pixel coordinates, the data from both channels are aligned and fused in the same spatial coordinate system. Then, through coordinate transformation, the vector data corresponding to the feature regions is analyzed to obtain the vector channel extraction result. Next, the image channel recognition result and the vector channel extraction result are cross-validated to obtain correction information. Finally, the correction information and the global coordinate range of the drawing frame unit are integrated to obtain structured engineering drawing information data. This achieves end-to-end automated conversion from unstructured engineering drawings to structured data, significantly improving processing efficiency while ensuring extraction accuracy, and providing a reliable data foundation for downstream applications such as BIM modeling, drawing archiving, and intelligent drawing review. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A flowchart illustrating a method for extracting structured information from engineering drawings, provided as an embodiment of this application; Figure 2 A schematic diagram illustrating the mapping relationship between a frame unit and a feature region, provided for an embodiment of this application; Figure 3A This application provides a schematic diagram illustrating the coordinate mapping relationship between any pixel in an overall raster image and the corresponding point in a DXF file, as exemplified by this application. Figure 3B This is a schematic diagram illustrating a multimodal analytical model for extracting feature regions from a whole raster image, as provided in an embodiment of this application. Figure 3C This is a schematic diagram illustrating the reverse calculation of the frame unit G_i of a feature region R_i, provided in an embodiment of this application. Figure 3D A schematic diagram illustrating the direct coordinate relationship mapping between an independent raster image feature region R_i and a frame unit G_i, provided in an embodiment of this application; Figure 4 A schematic diagram illustrating the process of extracting image channel recognition results provided in an embodiment of this application; Figure 5 A schematic diagram of the extraction process for vector channel extraction results provided in an embodiment of this application; Figure 6 This is a schematic diagram of a process for obtaining correction information provided in an embodiment of this application; Figure 7 A schematic diagram of a structured data template provided in an embodiment of this application; Figure 8 A flowchart illustrating a method for extracting structured information from engineering drawings in an application scenario, as provided in this application embodiment; Figure 9 This is a schematic diagram of the internal structure of an engineering drawing structured information extraction device provided in an embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] One existing technology is an image space-based information extraction method. This method involves rendering CAD engineering drawings into raster images, then using computer vision techniques such as Optical Character Recognition (OCR), object detection, and semantic segmentation to analyze the images and extract text, tables, and graphic regions. Finally, a large language model is used to organize and output the extracted results. A common characteristic of this type of method is that all information extraction operations are based on the rendered raster image, without relying on or directly utilizing the structured information in the CAD vector source file.

[0019] The disadvantages of the above-mentioned image space-based information extraction methods are as follows: 1. Limited by rendering resolution: The resolution of raster images directly affects the recognition accuracy of OCR and visual models. Engineering drawings contain a large number of small-sized text, high-density annotations, and thin-line graphics. Under uniform resolution rendering, character recognition errors, broken lines, or missed areas are prone to occur. If the global rendering resolution is increased, the computational cost and storage overhead will increase dramatically; if a fixed resolution is used, the loss of information in detailed areas is inevitable.

[0020] 2. Inability to utilize CAD vector metadata: Relying solely on image channels cannot obtain structured metadata such as layer names, line types, and block attribute fields in CAD files. This metadata carries important semantic information in engineering drawings. For example, the text in the "text-annotation" layer is usually dimension annotation, while the text in the "text-description" layer is usually construction instructions. Losing this context will prevent subsequent large language model processing from making fine semantic distinctions.

[0021] 3. High error rate in OCR of engineering terms: Professional symbols (such as diameter symbol "φ" and angle symbol "°") and terms (such as concrete strength grade "C30") in engineering drawings have a high error rate in general OCR models, and such errors lack an effective automatic correction mechanism.

[0022] Existing technology two is a vector space-based information extraction method. This method involves directly reading the CAD source file (e.g., DWG or DXF format), parsing the vector metadata (points, lines, arcs, text, blocks, etc.) within the file, performing entity recognition and attribute extraction based on geometric rules or deep learning models, and outputting structured data records. A common feature of these methods is that all information extraction operations are based on the vector data of the CAD file and do not rely on image rendering.

[0023] The disadvantages of the existing technology 2 are as follows: 1. Reliance on CAD format compatibility: Vector parsing methods heavily rely on parsing compatibility with CAD file formats (especially DWG format). CAD files used in actual engineering projects may come from different versions of AutoCAD or different vendors' CAD software. Differences in file formats and non-standard drawing practices can lead to parsing failures or incomplete parsing results.

[0024] 2. Poor robustness to non-standard drawing practices: Vector analysis methods typically assume that drawings follow certain layer naming conventions, line type conventions, and block structures. However, actual engineering drawings vary widely in their drawing practices; the same type of information may be distributed across different layers, and the same block may have different internal structures in different drawings. This makes rule-based or specific training data-based methods lack generalization ability.

[0025] In existing technologies, image-channel-based methods for extracting engineering drawing information are limited by rendering resolution, cannot utilize CAD vector metadata, and suffer from high error rates in optical character recognition of engineering terminology. Vector-channel-based extraction methods rely on CAD format compatibility and standard drawing practices, lack generalization ability, and cannot understand the visual semantics and semantic indexing relationships of drawing areas. Some attempts to fuse two channels employ feature-level numerical alignment strategies rather than cross-validation mechanisms after independent extraction of the two channels, and their output is question-and-answer style rather than structured data, failing to achieve proactive and comprehensive extraction of drawing information.

[0026] To address the aforementioned problems, this application provides a technical solution for extracting structured information from engineering drawings using a dual-channel approach. The technical solution proposed in this application will be described in detail below with reference to the accompanying drawings.

[0027] Figure 1 This is a flowchart illustrating a method for extracting structured information from engineering drawings, provided as an embodiment of this application. Figure 1 As shown in the figure, the method for extracting structured information from engineering drawings provided in this application embodiment specifically includes the following steps: Step 101: Obtain the graphic exchange format DXF file corresponding to the computer-aided design CAD drawing, identify the drawing frame entity in the DXF file, and obtain the global coordinate range of the drawing frame in the DXF global coordinate system.

[0028] The drawing frame is the drawing area defined by the drawing frame entity.

[0029] After engineers complete the construction drawings (such as architectural construction drawings) in CAD software, they export the DWG file as an exchange file in DXF format. DXF is an open, standardized CAD data exchange format that stores all vector metadata in the drawing in text or binary form, including line segments, arcs, polylines, text, dimensions, layer definitions, and block definitions.

[0030] In one possible implementation, the exchange format file is a DXF file; The identification of the frame entity in the exchange format file includes one of the following: Search for block references in the DXF file whose block names contain preset keywords to identify the title block entity; Search for closed rectangular polylines on a specific layer of the DXF file to determine the frame entity.

[0031] In practical applications, a complete set of architectural construction drawings typically contains multiple drawings (e.g., Architectural Drawing-01 to Architectural Drawing-N), each bounded by a title block entity (a block reference or a rectangular primitive on a specific layer). The DXF file is parsed to identify all title block entities. The rectangular area defined by each title block entity constitutes a title block, which contains the complete drawing area of ​​that drawing (including the title block lines, title block, and drawing content). To facilitate identification, identification rules can be set, such as selecting block references in the DXF file whose block names contain keywords like "title block," "_A1," or "_A2," or selecting rectangular polylines on a specific layer (e.g., the "title block" layer).

[0032] In DXF files, title blocks are typically defined as blocks and then inserted multiple times into the drawing using block references. To accurately locate these title blocks, all block reference entities can be traversed, and their corresponding block names checked. If the block name contains specific preset keywords, it can be identified as a title block entity. In practical applications, many drawings directly depict title blocks as geometric shapes. In this case, the title block often appears as a closed polyline or rectangle. In actual engineering, designers typically place title blocks on specific layers. Therefore, by limiting the layers and filtering out closed, approximately rectangular polyline objects within these layers, title block entities can also be identified.

[0033] In practical applications, you can first locate the outermost drawing frame entity, and then identify the title bar next to the drawing frame unit (usually the lower right corner). The title bar is usually a rectangular block containing dense table lines and specific text (such as "drawing number", "name", "scale", "design", "approval").

[0034] In practical applications, for each drawing frame, its range information in the DXF global coordinate system can be obtained.

[0035] Step 102: Render the content within the frame area into a raster image, input the raster image into a multimodal analysis model for region recognition, and output the pixel coordinate range and region content of the feature region in the raster image to obtain the image channel recognition result corresponding to the DXF file.

[0036] In practical applications, the entire frame area can first be output as a raster image (hiding the frame itself and processing the frame entity separately). Then, a multimodal analytical model is used to analyze and recognize the extracted feature regions. Specifically, all vector graphics content (including lines, circles, arcs, text, fills, blocks, etc.) within each frame area can be rendered into a single raster image at a specified high resolution (first resolution). The pixel width of this raster image is denoted as W_r, and the pixel height as H_r. The rendering resolution can be set according to actual needs; for example, a resolution of 0.1 mm per pixel (equivalent to 254 DPI) can be used.

[0037] In one possible implementation, the feature region includes a text region, a table region, and a graphic region; For the identified text region, the multimodal parsing model outputs the pixel coordinate range of the text region and the text string identified from the text region.

[0038] For the identified table region, the multimodal parsing model outputs the pixel coordinate range of the table region and the structured table text; For the identified graphic region, the multimodal parsing model outputs the pixel coordinate range of the graphic region and the initial graphic type.

[0039] In practical applications, the multimodal parsing model can be based on the Transformer architecture. This model, pre-trained on large-scale document image data, possesses the ability to perform layout analysis, text recognition, and content understanding on document images.

[0040] In the above embodiments, the multimodal parsing model can encode and decode the entire raster image, identify and segment three types of feature regions within the frame unit, namely text region, table region and graphic region.

[0041] The text region is a rectangular area containing text content, such as design specifications, construction notes, and drawing titles. For the text region, the aforementioned multimodal analytical model can output its pixel coordinate range and the recognized text string.

[0042] A table area refers to a rectangular area containing table content, such as a door and window schedule, a materials list, or an equipment list. For a table area, the multimodal parsing model can output its pixel coordinate range. After the text within the table area is recognized by OCR, it can be converted into structured table text in Markdown format using a text-based large language model, and the table name (such as "Door and Window Schedule") and the main content of the table can be recognized and distinguished.

[0043] A graphic region refers to a rectangular area containing the architectural content of a drawing. For a graphic region, the multimodal analytical model can output its pixel coordinate range and initial graphic type. For graphic type identification of an image region, it can first be classified into an initial graphic category, or the corresponding graphic major category, which can be further differentiated later based on the actual situation. The initial graphic type identification includes, but is not limited to, the following categories: floor plans (which can be further divided by floor level into first floor plan, second floor plan, roof plan, etc.), elevations (which can be further divided by orientation into south elevation, east elevation, etc.), sections (which can be further divided by section location number into 1-1 section, AA section, etc.), enlarged floor plans (which can be further divided by functional area into enlarged bathroom floor plans, enlarged stairwell floor plans, etc.), and detailed drawings (including wall detailed drawings and general node detailed drawings).

[0044] In one possible implementation, after inputting the raster image into a multimodal analytical model for region recognition and outputting the pixel coordinate range and region content of the feature regions in the raster image, the method further includes: For the graphic region, a multimodal large language model is used to perform semantic understanding on the local image of the graphic region to obtain the precise graphic type corresponding to the graphic region, the semantic content description of the graphic region, and the key text annotations identified in the graphic region.

[0045] For each graphic region, a multimodal large language model can be used to perform semantic understanding on the local image of that region. This allows us to obtain the precise graphic type of the region (e.g., "Second Floor Plan", "South Elevation", "Section 1-1", "Enlarged Bathroom Plan", "Wall Detail Drawing WD-01", "General Node Detail Drawing", etc.), the semantic content description of the region (e.g., the image region includes a lobby, reception room, three offices, and a corridor), and the key text labels identified within the region (e.g., door and window numbers, room names, elevation numbers, etc.). The precise graphic types described above are more accurate and detailed than the initial graphic types identified earlier.

[0046] The index relationship field for the graphic area can be left blank for now, to be filled in later through unified reasoning. In practical applications, index relationship reasoning for the graphic area can also be performed at this stage without specific limitations.

[0047] Step 103: Based on the linear mapping relationship between the global coordinate range of the frame and the pixel size of the raster image, establish a bidirectional linear mapping relationship between the DXF global coordinates and the pixel coordinates.

[0048] Step 104: Based on the pixel coordinate range of each feature region, calculate the global coordinate range of the frame unit through the bidirectional linear mapping relationship, and perform targeted extraction from the DXF file based on the global coordinate range of the frame unit to obtain the vector channel extraction result corresponding to the DXF file.

[0049] The frame unit is the region within the frame that corresponds to the feature region.

[0050] Print the bounding box of the DXF file to output the overall raster image. At this point, there is a global coordinate mapping relationship, that is, there is a one-to-one correspondence between any pixel in the overall raster image and the corresponding point in the DXF file. The overall raster image is then processed by a multimodal analytical model to identify and separate multiple different feature regions. The multimodal analytical model can output the rectangular coordinate range of each feature region in the overall raster image. By using the rectangular coordinate range (pixel coordinate range) of the i-th feature region R_i in the raster image, the rectangular coordinate range (geometric space, global coordinate range) of the corresponding frame unit G_i in the DXF coordinates can be calculated in reverse. A one-to-one coordinate mapping relationship is established between the feature region R_i and the frame unit G_i, which can be easily used directly later.

[0051] In one possible implementation, the bidirectional linear mapping relationship is specifically as follows: For any pixel (px, py) in the feature region R_i, the global coordinates (gx, gy) corresponding to the pixel are calculated by the following formula: gx=Xmin_i+px (Xmax_i-Xmin_i) / W_i; gy=Ymax_i-py (Ymax_i-Ymin_i) / H_i; For the global coordinates (gx, gy) corresponding to the pixel, the pixel coordinates (px, py) corresponding to the pixel in the feature region are calculated by the following formula: px=W_i (gx-Xmin_i) / (Xmax_i-Xmin_i); py=H_i (Ymax_i-gy) / (Ymax_i-Ymin_i); Wherein, the global coordinate range of the frame unit is (Xmin_i, Ymin_i) ~ (Xmax_i, Ymax_i), where Xmin_i represents the minimum X-axis coordinate corresponding to the global coordinate range of the frame unit, Ymin_i represents the minimum Y-axis coordinate corresponding to the global coordinate range of the frame unit, Xmax_i represents the maximum X-axis coordinate corresponding to the global coordinate range of the frame unit, and Ymax_i represents the maximum Y-axis coordinate corresponding to the global coordinate range of the frame unit. W_i and H_i are the pixel width and height of the feature region.

[0052] It should be noted that the pixel coordinates (px, py) of any pixel in the above formula are local coordinates relative to the origin at the top left corner of the feature region R_i, rather than absolute pixel coordinates in the entire raster image.

[0053] The pixel size of the entire raster image is W_r×H_r, the pixel size of the feature region R_i is W_i×H_i, and the offset of R_i in the entire raster image is (rx,ry), where (rx,ry) represents the position of the feature region R_i at the top left corner of the entire raster image, that is, from which pixel in the entire image R_i is placed.

[0054] like Figure 2 As shown, for frame unit i, its global coordinate range is (Xmin_i, Ymin_i) ~ (Xmax_i, Ymax_i). Here, the four values ​​represent the minimum, minimum, maximum, and maximum X-axis coordinates, respectively, of the rectangular coordinate range of the frame unit. This corresponds to the feature region R_i in the rendered raster image, where the pixel width is W_i and the pixel height is H_i.

[0055] Because the rectangular boundaries of the frame unit and the feature region remain completely one-to-one during rendering (without rotation or clipping), there is a precise linear mapping relationship between the two coordinate systems.

[0056] It should be noted that the DXF coordinate system has the X-axis to the right and the Y-axis upward, and the origin of the coordinate system is generally unrelated to the image frame; the geometric space coordinate system is standardized; the image coordinate system has the X-axis to the right and the Y-axis downward, and the origin of the coordinate system is at the upper left corner of the image; the pixel coordinate system of the image is standardized; the overall size of the raster image is width W_F and height H_F, where F represents Full.

[0057] The formula derivation is as follows, such as Figure 3A As shown, the similarity relationship along the X-axis is as follows: Δx' / Δx=W_F / (Xmax_F-Xmin_F); Δx'=px; Δx = gx - Xmin_F; We get: px / (gx-Xmin_F)=W_F / (Xmax_F-Xmin_F); px=W_F (gx-Xmin_F) / (Xmax_F-Xmin_F).

[0058] The similarity relationship along the Y-axis is: Δy' / Δy = H_F / (Ymax_F - Ymin_F); Δy'=H_F-py; Δy = gy - Ymin_F; We get (H_F-py) / (gy-Ymin_F)=H_F / (Ymax_F-Ymin_F); H_F-py=(gy-Ymin_F) H_F / (Ymax_F-Ymin_F); py=(Ymax_F-gy) H_F / (Ymax_F-Ymin_F).

[0059] Therefore, given the coordinates of the four corner points of the frame in the DXF coordinate system, and outputting the frame range as a raster image with a specified width W_F and height H_F, the pixel coordinates (px, py) of any point on the raster image can be expressed as: px=W_F (gx-Xmin_F) / (Xmax_F-Xmin_F); py=(Ymax_F-gy) H_F / (Ymax_F-Ymin_F); The entire formula is reversible; therefore, given any point (px, py) in the raster image, the corresponding point (gx, gy) in the DXF can be calculated in reverse. gx=Xmin_F+px (Xmax_F-Xmin_F) / W_F; gy=Ymax_F-py (Ymax_F-Ymin_F) / H_F; like Figure 3B As shown, the multimodal analytical model can output the offset (rx, ry) of the feature region R_i relative to the origin of the overall raster image, as well as the width W_i and height H_i of R_i; any point (px, py) of the feature region R_i extracted as an independent image has coordinates (rx+px, ry+py) in the overall raster image.

[0060] like Figure 3C As shown, the coordinates of the four corner points of the known feature region R_i in the entire raster image are (rx,ry), (rx,ry+H_i), (rx+W_i,ry+H_i), and (rx+W_i,ry). Using the above formula, the coordinates of the four corner points of the frame element corresponding to DXF can be calculated in reverse, and are denoted as: (Xmin_i,Ymax_i), (Xmin_i,Ymin_i), (Xmax_i,Ymin_i), (Xmax_i,Ymax_i).

[0061] In actual downstream tasks, it is necessary to directly obtain the coordinate mapping of the feature region R_i and the frame unit G_i as independent images; the coordinates of the four corner points of the frame unit G_i in DXF are known, and the coordinates of the four corner points of the feature region R_i as independent raster images are also known.

[0062] like Figure 3D As shown, the similarity relationship along the X-axis is: Δx' / Δx = W_i / (Xmax_i - Xmin_i); Δx'=px; Δx = gx - Xmin_i; Get: px / (gx-Xmin_i)=W_i / (Xmax_i-Xmin_i); px=W_i (gx-Xmin_i) / (Xmax_i-Xmin_i).

[0063] The similarity relationship along the Y-axis is: Δy' / Δy = H_i / (Ymax_i - Ymin_i); Δy'=py; Δy = -Ymax_i - gy; Get: py / (Ymax_i-gy)=H_i / (Ymax_i-Ymin_i)py=H_i (Ymax_i-gy) / (Ymax_i-Ymin_i).

[0064] Therefore, given the frame element G_i in DXF and any point (gx, gy) within it, the pixel coordinates (px, py) of the corresponding point in the feature region R_i of an independent raster image can be expressed as: px=W_i (gx-Xmin_i) / (Xmax_i-Xmin_i); py=H_i (Ymax_i-gy) / (Ymax_i-Ymin_i); The entire formula is reversible. Given any point (px, py) in the feature region R_i of an independent raster image, the corresponding point (gx, gy) in the frame unit G_i in DXF can be expressed as: gx=Xmin_i+px (Xmax_i-Xmin_i) / W_i; gy=Ymax_i-py (Ymax_i-Ymin_i) / H_i; For any pixel (px, py) in image R_i, its corresponding DXF global coordinates (gx, gy) are calculated by the following formula: gx=Xmin_i+(px / W_i)×(Xmax_i-Xmin_i); gy=Ymax_i-(py / H_i)×(Ymax_i-Ymin_i); Conversely, for any point (gx, gy) within the global coordinate range of DXF, its corresponding pixel coordinates (px, py) in the image R_i are calculated by the following formula: px=W_i×(gx-Xmin_i) / (Xmax_i-Xmin_i); py=H_i×(Ymax_i-gy) / (Ymax_i-Ymin_i); Specifically, the above mapping can be divided into two levels: The first level converts the local pixel coordinates (px, py) within R_i into pixel coordinates (px_full, py_full) in the entire raster image using an offset (rx, ry), i.e., px_full = rx + px, py_full = ry + py; the second level maps the pixel coordinates (px_full, py_full) in the entire raster image to DXF global coordinates (gx, gy) through linear interpolation. The key feature of this mapping relationship is that it uses the frame entity in the DXF file as the anchor point. The global coordinate range of the frame entity forms a one-to-one binding relationship with the pixel size of the entire raster image. Therefore, the pixel coordinate ranges of the text region, table region, and graphic region on R_i obtained above can all be converted into the corresponding DXF global coordinate ranges through the above two-level mapping.

[0065] In one possible implementation, after establishing the bidirectional linear mapping relationship between DXF global coordinates and pixel coordinates, the method further includes: For each of the graphic regions, the pixel size of the graphic region in the raster image is checked, wherein the graphic region is rendered at a first resolution; When the pixel size is lower than the minimum pixel threshold corresponding to the graphic type of the graphic region, the global coordinate range of the graphic region is calculated through the bidirectional linear mapping relationship, the global coordinate range of the graphic region is extracted from the DXF file and re-rendered at the second resolution to generate a new high-resolution image, and the semantic understanding step is re-executed for the high-resolution image, wherein the second resolution is higher than the first resolution.

[0066] In practical applications, the physical size of an image (inches) = pixel size (total number of pixels) ÷ resolution (PPI / DPI). For each graphic region, its pixel size (width and height in pixels) in the original raster image R_i is checked. Different graphic types have different preset minimum pixel thresholds; for example, the minimum size of a plan view region is no less than 1600×1200 pixels; the minimum size of an elevation view region is no less than 1000×1200 pixels; and the minimum size of a wall detail drawing and a node detail drawing region is no less than 1000×800 pixels.

[0067] If the pixel size of a certain graphic region in R_i is lower than the threshold of its corresponding initial graphic type, the following resolution adaptation operation can be performed: First, based on the pixel coordinate range of the graphic region in R_i, its global coordinate range in the DXF file can be calculated using a coordinate mapping formula; then, the vector content of this global coordinate range is extracted from the DXF file and re-rendered at a higher resolution (e.g., 0.05 mm per pixel, equivalent to 508 DPI) to generate a new, higher-resolution graphic region image; finally, the original graphic region image is replaced with the newly generated high-resolution image, and the above semantic understanding steps are re-executed. In the rendering stage, when increasing the resolution for rendering, the default is to keep the physical size of the drawing unchanged. In this case, the pixel size will increase synchronously, which can achieve a clearer vector-to-bitmap effect.

[0068] Figure 4This illustration shows a schematic diagram of an image recognition result extraction process provided in an embodiment of this application. For raster images, a Transformer-based multimodal parsing model can be used to process the image, segmenting it into three categories: text regions, table regions, and graphic regions. The pixel coordinate range and basic information of these feature regions are identified (text regions are initially identified as text, table regions as tables, and graphic regions as graphic types). Further, for text regions, a text-based large language model can directly process the text object; for table regions, OCR recognition is performed first, followed by conversion to Markdown format tables by the large language model, distinguishing between table names and table bodies; for graphic regions, a multimodal large language model can be used to identify the graphic type and generate a semantic description, while simultaneously determining whether the pixel size of the region is less than the minimum pixel threshold for the corresponding graphic type. If it is less than the threshold, the region is re-rendered from the original DXF file at a higher resolution and returned for processing; if it is not less than the threshold, the result is directly output. Finally, the recognition results of all regions are summarized into an image channel recognition result.

[0069] In one possible implementation, the targeted extraction of the global coordinate range based on the frame element from the exchange format file includes: Call the DXF parsing component to search for all text entities within the global coordinate range of the frame unit, and extract the text string, insertion point coordinates, and layer name of each text entity; The extracted text entities are grouped according to layer names, and the text in the same layer is sorted according to spatial position to form the text sequence of the layer; Within the global coordinate range of the frame unit, search all block reference entities, and for each block reference, extract the block name, internal text of the block, and attribute fields.

[0070] Figure 5 The extraction process of vector channel extraction results is demonstrated. It should be noted that, unlike the existing technology that performs a comprehensive analysis of CAD files and then filters out general correction information, the above vector channel extraction results are extracted in a targeted manner, that is, only the text and block information within the global coordinate range of each region identified by the image channel is extracted.

[0071] In practical applications, for each text area, table area, and graphics area, its corresponding DXF global coordinate range can be calculated. Then, the DXF parsing component is called to search for all text entities (TEXT and MTEXT entities) within this global coordinate range. The text string, insertion point coordinates, and layer name of each text entity are extracted. Specifically, within a text area, all text strings within that area are extracted; within a table area, all text strings within that table range are extracted (including table headers and cell content); and within a graphics area, all text strings within that graphics area are extracted (such as annotations and descriptions). It's important to note that string extraction is performed entity-by-entity. Each text entity independently records its own text string, insertion point coordinates, and layer name, rather than merging them into a single string. Therefore, the extraction result can be considered a list of text entities with coordinates and layer attributes.

[0072] The extracted text entities can then be grouped according to their layer names (if the layer naming is standardized, the text type can be initially determined based on the layer name; if the layer naming is not standardized, it will be automatically categorized according to the entity type). Typical layer examples include: the layer "Text-Annotation" contains dimension annotation text (such as "3600", "2100"); the layer "Text-Name" contains room / component names (such as "Office", "Lobby"); and the layer "Text-Description" contains construction description text.

[0073] In practical applications, after extracting text entities, the first step is to determine the layer name of the layer containing the text entity. If the layer name conforms to the specifications, the text type can be preliminarily determined to ascertain the nature of the text (e.g., whether it is a "dimension annotation"). If the name is not standardized, the text can be uniformly categorized as "single-line text" or "multi-line text." Although this cannot directly determine whether the text is a "dimension" or "description" as with standardized naming, it ensures that entities of the same format are grouped together, preventing a chaotic appearance.

[0074] In practical applications, text within the same layer can be sorted according to its spatial position (Y-coordinate from top to bottom, X-coordinate from left to right) to form a text sequence for that layer. Layer names and grouping results can serve as auxiliary contextual information for subsequent cross-validation of large language models. Layer information helps improve validation efficiency. It should be noted that the subsequent cross-validation mechanism does not entirely rely on layer standardization; the comparison and validation between image channels and vector channels can still be completed independently even when layer information is missing or non-standardized.

[0075] In CAD drawings, recurring engineering objects such as doors, windows, equipment, elevation symbols, and axis numbers are typically defined as "blocks" and then referenced multiple times using INSERT entities. The INSERT entity records information such as the block name. Within the aforementioned global coordinate range, all block-referenced entities (INSERT entities) can be searched. For each block-referenced entity, the following information is extracted: block name, internal block text, and attribute fields. The block name identifies the block type; for example, the block name "Door-M1" indicates door number M1. For the internal block text, the sub-entities in the block definition can be recursively traversed to extract the text entities they contain. For attribute fields, the attribute definition in the block definition (ATTDEF entity) and the corresponding attribute value in the block reference (ATTRIB entity) can form key-value pairs of "attribute field name - attribute field value," such as "number" = "M1," and "opening size" = "1000×2100."

[0076] It's important to note that simple TEXT / MTEXT extraction only yields fragmented strings, failing to determine whether "3600" represents the door width or the room area. INSERT, however, binds these values ​​to engineering objects using block names and attribute tags. For example, extracting only text might result in a string like "M1, 1000×2100, fire door," but extracting attributes via INSERT provides clear key-value pairs: Number = M1, Opening Size = 1000×2100, Type = fire door. Thus, extracting text entities identifies what's written on the drawing, while extracting block reference entities reveals which engineering objects are on the drawing, what they are, and what their attributes are. The layer names and grouping results mentioned above can then serve as supplementary contextual information for subsequent cross-validation.

[0077] Step 105: Cross-validate the image channel recognition result with the vector channel extraction result to obtain the correction information corresponding to the DXF file.

[0078] In one possible implementation, the step of cross-validating the image channel recognition result with the vector channel extraction result to obtain the correction information corresponding to the DXF file includes: For the text region, the first string in the image channel recognition result and the second string in the vector channel extraction result are used as two independent raw data streams and input into the large language model for cross-validation to obtain the text correction result output by the large language model. The first string and the second string are within the same coordinate range of the same text region. The prompts from the large language model include similar error patterns obtained by vector retrieval of inconsistent character fragments in a character error correction knowledge base, along with the correct corrections, provided as supplementary context to the large language model. For the table region, the structured table text in the image channel recognition result is cross-validated with the table-related text in the vector channel extraction result to obtain the table correction result, wherein the structured table text and the table-related text are within the same coordinate range of the same table region; For the graphic region, the semantic content description of each graphic region in the image channel recognition result is cross-validated with the text and block information extracted from the global coordinate range of the graphic region in the vector channel extraction result to obtain the graphic correction result; Based on the text correction results, the table correction results, and the graphic correction results, the correction information corresponding to the DXF file is obtained.

[0079] In one possible implementation, the character error correction knowledge base is constructed and used in the following manner: Start with a seed knowledge base containing common optical character recognition error patterns in the field of engineering drawings; During each cross-validation process, after the large language model identifies and corrects a new character error, it collects record pairs including erroneous text fragments and correct text fragments. If the record pair does not exist in the character error correction knowledge base, it adds the record pair to the character error correction knowledge base.

[0080] Figure 6 The flowchart illustrates the process of obtaining correction information through cross-validation. For each text region, the first string T_img identified by its image channel (which may have recognition errors, such as misjudgment due to similar character shapes) and the second string T_vec extracted by the vector channel within the same coordinate range (which is theoretically accurate, but may be incomplete due to layer filtering or omissions) can be used as two independent raw data and simultaneously input into a large text-based language model for cross-validation.

[0081] The prompts of the aforementioned large language model include a cross-validation instruction, which is used to instruct the large language model to perform a semantic-level comparison between T_img and T_vec, and judge the content inconsistencies (for example, "φ" is recognized as "Zhong" in image recognition) in combination with the engineering context and preset inspection and constraint rules; wherein the preset inspection and constraint rules include an engineering term dictionary (for example, "φ" is the diameter symbol, and "C30" is a valid value of the concrete strength grade), punctuation rules, numerical format rules for dimension marking, and the like; at the same time, vector retrieval can be performed on the inconsistent character segments in the character error correction knowledge base, and the retrieved similar error patterns and correct correction results are provided to the large language model as additional context, helping the large language model to more accurately determine how to correct the current inconsistency.

[0082] During the construction and use of the character error correction knowledge base, at the initial construction stage, it can be started with an initial seed knowledge base, which contains common optical character recognition error patterns in the engineering drawing field. For example: "φ" (diameter symbol) is often recognized as "Zhong", "Yi", or "Chuan"; "Φ" (reinforcement diameter symbol) is often recognized as "Zhong", "Feng", or "Zheng"; "°" (angle symbol) is often recognized as "o", "0", or "。"; "C30" is often recognized as "0", "C0", or "030".

[0083] In each cross-validation process, after the large language model identifies a new character error and corrects it, the "error text segment - correct text segment" record pair is automatically collected. A vector similarity threshold can be used to determine whether the record pair already exists in the knowledge base, and if it does not exist, it is stored in the vector database.

[0084] In the subsequent validation process, when the large language model detects a text segment that is inconsistent between the image channel and the vector channel, it first performs vector retrieval on the segment in the knowledge base, and the retrieval result can be sent to the large language model as context information together with the original prompt to assist its judgment.

[0085] The aforementioned character error correction knowledge base is continuously updated incrementally during use, and the newly included error patterns continuously enrich the coverage of the knowledge base, so that the character correction capability is gradually improved over time without any parameter fine-tuning of the large language model.

[0086] The text correction result output by the large language model simultaneously includes the following information: the corrected text string, the position of the corrected character, and the correction basis (from rules or the knowledge base or model inference). In this way, the LLM can use the accuracy of the vector to correct OCR misrecognition, and at the same time use the comprehensiveness of OCR to make up for the text that may be missed by the vector, and finally output the most reliable text content of each text area for subsequent structured output.

[0087] In practical applications, when the large language model cross-validates the structured table content converted to Markdown format with the table-related text extracted from the vector channel within the same coordinate range, the key validation points include: the accuracy and consistency of table names; the spelling and terminology accuracy of table column headings; and the correctness of table cell values. For key fields in the table, such as building component numbers (e.g., door and window numbers "M-1", "C-2"), material specifications (e.g., "1000×2100"), and quantity values, stricter validation rules are applied than for ordinary text.

[0088] The semantic content description of each generated graphic region is cross-validated with the list of text and tile attributes extracted within the global coordinate range of that region. The Large Language Model (LLM) determines whether the description generated on the image side matches the actual text annotation on the vector side, correcting any discrepancies. For example, if the image side describes a graphic region as "Level 1 Plane View," while the vector side extracts specific text entities (such as "Level 2") for that region, the LLM compares the two for consistency. If a discrepancy is found, correction is triggered, and the LLM re-determines the level of the graphic region. If the image side describes a region as "Gate M1," but the tile attributes extracted by the vector side show "Number = M2," then the LLM determines that the precise attributes from the vector side should prevail, correcting the image side's description.

[0089] For graphic regions, the graphic type can also be validated. The graphic type on the image side is a preliminary judgment based on visual features and may contain misjudgments (e.g., misclassifying an "elevation view" as a "plan view"). Cross-validation can be performed by comparing with information from the vector channels. For example, it can be compared with the block names in the vector channels. The block names extracted (INSERT) from the global coordinate range of the graphic region by the vector channels often directly reflect the true type of the graphic object (e.g., a block name of "Door-M1" indicates that the region is a door legend, not the main body of a plan view). Simultaneously, it can be compared with the layer affiliation in the vector channels. The layer to which the entity in the region belongs in the vector channels (e.g., "door and window layer," "dimension annotation layer") can also help determine the graphic type. Pre-defined rules can also be used (e.g., plan views usually include elements such as axes and dimensions) to determine the reasonableness of the type given on the image side. If inconsistencies are found, LLM can make a decision based on the context and correct the graphic type if necessary.

[0090] It should be noted that although this application primarily focuses on architectural construction drawings as the main application scenario to illustrate the method's steps, its core technical architecture is also applicable to the digital analysis and information extraction of CAD as-built drawings in fields such as municipal engineering, water conservancy engineering, power engineering, and rail transit engineering. In practical applications, the multimodal analysis model's ability to recognize graphic types and specific table formats within its domain, as well as the content of professional domain-specific check constraint rules and character error correction knowledge bases, can be adjusted according to the actual conditions of each field, thus adapting to the needs of different domains.

[0091] Step 106: Integrate the correction information and the global coordinate range of the drawing frame unit to obtain structured engineering drawing information data.

[0092] In one possible implementation, integrating the correction information and the global coordinate range of the drawing frame unit to obtain structured engineering drawing information data includes: The correction information and the global coordinate range of the drawing frame unit are input into the large language model according to the requirements of the preset structured data template to obtain structured engineering drawing information data. The engineering drawing information data is input into the large language model again. The large language model infers and completes the semantic index relationship between each graphic area based on the drawing specifications, and adds the semantic index relationship to the engineering drawing information data.

[0093] In the initial structured engineering drawing data, each graphic area originally only has its own attributes (coordinates, type, text, etc.) and does not include indexing relationships with other areas. Through reasoning using a large language model, semantic indexing relationships can be obtained and added to the engineering drawing data. This engineering drawing data can then serve as an input data source for BIM modeling software (such as Autodesk Revit) to automatically create component objects and their attribute information in the BIM model.

[0094] In one possible implementation, the semantic index relationship includes the following types: Detailed drawing reference relationships, wherein the detailed drawing reference relationships are established based on the matching of reference words in the text of the graphic area with the name of the referenced graphic area; Section index relationship, wherein the section index relationship is established based on the matching of section symbol numbers and section names in the plan view; Facade index relationship, wherein the facade index relationship is established based on the logic of associating the floor plan and the elevation plan with the same floor; The table association relationship is established based on the matching of component numbers in the table area with component tags in the graphic area.

[0095] In practical applications, a set of structured data templates can be predefined to describe the parsed engineering drawing information. For example... Figure 7 As shown, exemplarily, the above structured data template includes frame-level information, region-level information, and attribute-level information. Frame-level information includes: drawing name, drawing number, drawing type, and the global coordinate range of the frame. Region-level information includes region identifier, region type (text region / table region / graphic region), initial or precise graphic type (plan view / elevation view / section view / enlarged plan view / wall detail / node detail), pixel coordinate range, global coordinate range, and region content. Attribute-level information includes data source annotations (annotating that each field's data comes from "image channel," "vector channel," or "LLM correction"), layer name, block name, text within the block, and block attribute fields. The drawing name, drawing number, drawing type, etc., can be obtained through OCR recognition of the raster image; alternatively, during vector channel extraction, the INSERT entity can be searched near the lower right corner of the frame unit, and the block name typically contains keywords such as "title block," "TITLE_BLOCK," and "TB." The latter directly obtains structured values ​​without requiring OCR recognition of the raster image.

[0096] The corrected information obtained after cross-validation, along with the global coordinate range, layer information, and data source annotations of each drawing unit, are input into the large language model according to the requirements of a preset structured data template. The large language model then performs integration, formatting, and integrity checks, ultimately outputting structured engineering drawing information data. This data can be stored in the form of JSON, XML, or relational database records.

[0097] For example, the engineering drawing information data corresponding to a certain DXF file is as follows: ["Drawing Name":"First Floor Plan","Drawing Number":"Construction-01","Drawing Type":"Architectural Drawing","Global Coordinate Range":[0,0,594,841],"Region List":[{"Region Identifier":"R1","Region Type":"Graphic Region","Initial Graphic Type":"Floor Plan","Precise Graphic Type":"First Floor Plan","Pixel Coordinate Range":[100,200,500,600],"Global Coordinate Range":[50,60,450,550],"Region Content":"First Floor Plan Main Region...","Data Source":{"Region Type":"Image Channel","Precise Graphic Type":"LLM Correction","Region Content":"Vector Channel"}.

[0098] At this point, the structured data contains complete information about all graphic regions within the current frame cell: graphic type, semantic content description, coordinate range, and identified key text annotations. However, the index relationship field for each graphic region remains empty.

[0099] In practical applications, the generated structured data can be input again into the large language model. The large language model infers the semantic index relationship between graphic regions based on the provisions on drawing index in architectural construction drawing specifications (such as GB / T 50001 "Unified Standard for Building Drawing").

[0100] Large language models can recognize semantic indexing relationships including but not limited to the following types: (1) Detail reference relationship: When the text in a drawing area contains the phrase "see XXX" or "see XXX", and the referenced XXX matches the name or number of another drawing area, a detail reference index is established. For example, if the text in a first-floor plan contains "see wall detail WD-01", and there is a drawing area in the same drawing frame unit whose name contains "wall detail WD-01", then a detail reference index of "plan → detail" is established between the two.

[0101] (2) Section index relationship: When a graphic area is a plan view, its text contains section symbol numbers (such as "1-1", "AA", "II"), and there are one or more graphic areas of type "section view" with names matching the number in the same or related frame unit, a section index relationship is established.

[0102] (3) Elevation index relationship: When one graphic area is a floor plan and another graphic area is an elevation plan (such as "South Elevation" or "East Elevation"), and the two are associated with the same floor, an elevation index relationship is established.

[0103] (4) Table association: When the entry number in the table area matches the component mark in the graphic area (such as “M-1” in the door and window table and “M-1” marked in the floor plan), establish an association index between the table and the graphic area.

[0104] It should be noted that the large language model follows the following constraints when inferring the above semantic index relationships: it can only reference existing graphic region identifiers (region IDs) in the input structured data, and it is prohibited to fabricate non-existent drawing numbers or graphic regions; the inference of index relationships must be based on explicit textual evidence (such as direct text quotations, number matching), and it is not allowed to make unsubstantiated inferences based solely on graphic type; the inference results are graded according to confidence level: high confidence (existing precise text matching evidence) is directly completed; medium confidence (existing dual matching of number and type but lacking direct text quotations) is labeled as "inference" and the basis for the inference is recorded; low confidence or no evidence is left blank.

[0105] Write the index relationships of the large language model inference completion into the index relationship field of the corresponding graphic region and update the structured data. For example, update the index relationship field of graphic region R1 (first floor plan) as follows: [{"Target Region":"R3","Relation Type":"Detail Reference","Confidence":"High","Basis":"R1 text contains 'See wall detail WD-01', R3 name is 'Wall detail WD-01, General node detail'"}][{"Target Region":"R2","Relation Type":"Section Index","Confidence":"Medium","Basis":"R1 text contains '1-1', R2 type is '1-1 section diagram'"}).

[0106] In practical applications, multiple drawing frame units can simultaneously execute the above steps on different computing nodes, and finally the structured data of each drawing frame unit is summarized to form complete engineering drawing information data, which can improve processing speed.

[0107] When engineering drawings undergo partial revisions, information extraction can be re-executed only for the affected drawing frame units. Change records are generated by comparing the structured data outputs of the old and new versions. The bidirectional linear mapping relationship established in the above steps allows targeted extraction to process only the changed areas, without needing to reprocess the entire drawing.

[0108] In this embodiment, an architecture for parallel processing of image and vector channels is constructed to achieve independent extraction of dual-channel data. The image channel utilizes a multimodal analytical model to perform region analysis on the raster image, segmenting and identifying feature regions to obtain the image channel recognition result. By establishing a bidirectional linear mapping relationship between DXF global coordinates and pixel coordinates, the data from both channels are aligned and fused in the same spatial coordinate system. Then, through coordinate transformation, the vector data corresponding to the feature regions is analyzed to obtain the vector channel extraction result. Next, the image channel recognition result and the vector channel extraction result are cross-validated to obtain correction information. Finally, the correction information and the global coordinate range of the drawing frame unit are integrated to obtain structured engineering drawing information data. This achieves end-to-end automated conversion from unstructured engineering drawings to structured data, significantly improving processing efficiency while ensuring extraction accuracy, and providing a reliable data foundation for downstream applications such as BIM modeling, drawing archiving, and intelligent drawing review.

[0109] Figure 8 This illustration shows a schematic diagram of a method for extracting structured information from engineering drawings in an application scenario according to an embodiment of this application. Specifically, using a DXF file as input, the method employs parallel processing through both image and vector channels: the image channel performs multimodal analysis and coordinate mapping on the raster image of the drawing, while the vector channel extracts targeted information. After cross-validation of the two data streams, preliminary structured engineering drawing information data is output. This data is then further processed by a large language model to complete the semantic index relationships between graphic regions, ultimately generating updated structured engineering drawing information data containing complete logical relationships.

[0110] The above are embodiments of the method proposed in this application. Based on the same inventive concept, embodiments of this application also provide a device for extracting structured information from engineering drawings, the structure of which is as follows: Figure 9 As shown.

[0111] Figure 9 This is a schematic diagram of the internal structure of an engineering drawing structured information extraction device provided in an embodiment of this application. For example... Figure 9 As shown, the device includes: At least one processor 901; And a memory 902 that is communicatively connected to at least one processor; The memory 902 stores instructions that can be executed by at least one processor. The instructions are executed by at least one processor 901 so that at least one processor 901 can: execute the above-described method for extracting structured information from engineering drawings.

[0112] Some embodiments of this application provide corresponding to Figure 1A non-volatile computer storage medium stores computer-executable instructions, which are configured to execute the above-mentioned method for extracting structured information from engineering drawings.

[0113] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments for IoT devices and media are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0114] The systems, media, and methods provided in this application are one-to-one correspondences. Therefore, the systems and media also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the systems and media will not be repeated here.

[0115] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0116] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0117] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0118] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0119] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0120] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0121] Computer-readable media include both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0122] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0123] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for extracting structured information from engineering drawings, characterized in that, The method includes: Obtain the DXF file corresponding to the computer-aided design (CAD) drawing, identify the drawing frame entity in the DXF file, and obtain the global coordinate range of the drawing frame in the DXF global coordinate system, wherein the drawing frame is the drawing area defined by the drawing frame entity. The content within the frame area is rendered as a raster image, and the raster image is input into a multimodal analysis model for region recognition. The pixel coordinate range and region content of the feature region in the raster image are output to obtain the image channel recognition result corresponding to the DXF file. The feature region is the region in the raster image that contains the main drawing content of the drawing. Based on the linear mapping relationship between the global coordinate range of the frame and the pixel size of the raster image, a bidirectional linear mapping relationship between DXF global coordinates and pixel coordinates is established. Based on the pixel coordinate range of each feature region, the global coordinate range of the frame unit is calculated through the bidirectional linear mapping relationship, and targeted extraction is performed from the DXF file based on the global coordinate range of the frame unit to obtain the vector channel extraction result corresponding to the DXF file. The frame unit is the region within the frame corresponding to the feature region. The image channel recognition results are cross-validated with the vector channel extraction results to obtain the correction information corresponding to the DXF file; The correction information and the global coordinate range of the drawing frame unit are integrated to obtain structured engineering drawing information data.

2. The method according to claim 1, characterized in that, The feature regions include text regions, table regions, and graphic regions; For the identified text region, the multimodal parsing model outputs the pixel coordinate range of the text region and the text string identified from the text region; For the identified table region, the multimodal parsing model outputs the pixel coordinate range of the table region and the structured table text; For the identified graphic region, the multimodal parsing model outputs the pixel coordinate range of the graphic region and the initial graphic type.

3. The method according to claim 2, characterized in that, After inputting the raster image into a multimodal analytical model for region recognition and outputting the pixel coordinate range and region content of the feature regions in the raster image, the method further includes: For the graphic region, a multimodal large language model is used to perform semantic understanding on the local image of the graphic region to obtain the precise graphic type corresponding to the graphic region, the semantic content description of the graphic region, and the key text annotations identified in the graphic region.

4. The method according to claim 1, characterized in that, The bidirectional linear mapping relationship is specifically as follows: For any pixel (px, py) in the feature region R_i, the global coordinates (gx, gy) corresponding to the pixel are calculated by the following formula: gx=Xmin_i+px (Xmax_i-Xmin_i) / W_i; gy=Ymax_i-py (Ymax_i-Ymin_i) / H_i; For the global coordinates (gx, gy) corresponding to the pixel, the pixel coordinates (px, py) corresponding to the pixel in the feature region are calculated by the following formula: px=W_i (gx-Xmin_i) / (Xmax_i-Xmin_i); py=H_i (Ymax_i-gy) / (Ymax_i-Ymin_i); Wherein, the global coordinate range of the frame unit is (Xmin_i, Ymin_i) ~ (Xmax_i, Ymax_i), where Xmin_i represents the minimum X-axis coordinate corresponding to the global coordinate range of the frame unit, Ymin_i represents the minimum Y-axis coordinate corresponding to the global coordinate range of the frame unit, Xmax_i represents the maximum X-axis coordinate corresponding to the global coordinate range of the frame unit, and Ymax_i represents the maximum Y-axis coordinate corresponding to the global coordinate range of the frame unit. W_i and H_i are the pixel width and height of the feature region.

5. The method according to claim 3, characterized in that, After establishing the bidirectional linear mapping relationship between DXF global coordinates and pixel coordinates, the method further includes: For each of the graphic regions, the pixel size of the graphic region in the raster image is checked, wherein the graphic region is rendered at a first resolution; When the pixel size is lower than the minimum pixel threshold corresponding to the graphic type of the graphic region, the global coordinate range of the graphic region is calculated through the bidirectional linear mapping relationship, the global coordinate range of the graphic region is extracted from the DXF file and re-rendered at the second resolution to generate a new high-resolution image, and the semantic understanding step is re-executed for the high-resolution image, wherein the second resolution is higher than the first resolution.

6. The method according to claim 1, characterized in that, The targeted extraction of the global coordinate range based on the frame unit from the DXF file includes: Call the DXF parsing component to search for all text entities within the global coordinate range of the frame unit, and extract the text string, insertion point coordinates, and layer name of each text entity; The extracted text entities are grouped according to layer names, and the text in the same layer is sorted according to spatial position to form the text sequence of the layer; Within the global coordinate range of the frame unit, search all block reference entities, and for each block reference entity, extract the block name, internal text of the block, and attribute fields.

7. The method according to claim 3, characterized in that, The step of cross-validating the image channel recognition result with the vector channel extraction result to obtain the correction information corresponding to the DXF file includes: For the text region, the first string in the image channel recognition result and the second string in the vector channel extraction result are used as two independent raw data streams and input into the large language model for cross-validation to obtain the text correction result output by the large language model. The first string and the second string are within the same coordinate range of the same text region. The prompts from the large language model include similar error patterns obtained by vector retrieval of inconsistent character fragments in a character error correction knowledge base, along with the correct corrections, provided as supplementary context to the large language model. For the table region, the structured table text in the image channel recognition result is cross-validated with the table-related text in the vector channel extraction result to obtain the table correction result, wherein the structured table text and the table-related text are within the same coordinate range of the same table region; For the graphic region, the semantic content description of each graphic region in the image channel recognition result is cross-validated with the text and block information extracted from the global coordinate range of the graphic region in the vector channel extraction result to obtain the graphic correction result; Based on the text correction results, the table correction results, and the graphic correction results, the correction information corresponding to the DXF file is obtained.

8. The method according to claim 2, characterized in that, The process of integrating the correction information and the global coordinate range of the drawing frame unit to obtain structured engineering drawing information data includes: The correction information and the global coordinate range of the drawing frame unit are input into the large language model according to the requirements of the preset structured data template to obtain structured engineering drawing information data. The engineering drawing information data is input into the large language model again. The large language model infers and completes the semantic index relationship between each graphic area based on the drawing specifications, and adds the semantic index relationship to the engineering drawing information data.

9. A device for extracting structured information from engineering drawings, characterized in that, The device includes: At least one processor; And, a memory communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform a method for extracting structured information from engineering drawings as described in any one of claims 1-8.

10. A computer storage medium storing computer-executable instructions, characterized in that, When the computer-executable instructions are executed, they implement a method for extracting structured information from engineering drawings as described in any one of claims 1-8.

Images