An intelligent identification and semantic analysis and data reconstruction system for engineering drawings

Abstract

The application discloses an intelligent recognition, semantic analysis and data reconstruction system for engineering drawings, and relates to the technical field of drawing processing. The system comprises the following steps: obtaining an engineering drawing sample image to form a training image set and training a target detection model; performing basic element recognition on an engineering drawing image; reading label information table to establish a corresponding relationship between category identification and Chinese category name, explanation information, element use information and associated prompt information; generating an engineering object semantic result according to spatial adjacency relationship, lead direction relationship, size coverage relationship, number adjacency relationship and table corresponding relationship; and reconstructing a structured engineering data unit.

Description

Technical Field

[0001] This invention relates to the field of drawing processing technology, and more specifically, to an intelligent recognition, semantic parsing and data reconstruction system for engineering drawings. Background Technology

[0002] During the processes of engineering design, construction delivery, drawing review and verification, operation and maintenance archiving, and digitization of historical drawings, a large amount of engineering information is still stored in the form of drawings such as floor plans, detailed node drawings, table descriptions, index marks, and dimension annotations. In these drawings, graphic outlines, text descriptions, leader lines, numbered indexes, and table fields are usually scattered in different areas, and are often accompanied by issues such as page scaling, partial screenshots, dense line segments, overlapping annotations, and inconsistent layouts. In existing technologies, common practices often involve using object detection, text recognition, or layout analysis methods to identify local content in the drawings, and then categorizing the recognition results by type, coordinate frame, or text. This field is output in this format. Although this processing method can obtain single-item recognition results of graphics, text, or tables, the attribution relationship between different basic elements usually still needs to be judged manually. For example, which graphic object does a certain piece of text belong to, which label object does a certain leader line correspond to, and which table item does a certain index number correspond to? Existing systems often lack stable association mechanisms. Furthermore, even if some recognition results have been obtained, due to the lack of a semantic organization method oriented towards engineering objects, the scattered recognition results can usually only form scattered records, which are difficult to further reconstruct into structured engineering data that can be directly used for retrieval, comparison, archiving, or subsequent business calls. Therefore, how to not only complete the identification of basic elements in the process of engineering drawing recognition, but also perform effective semantic associations on graphic elements, text elements, annotation elements, index elements and table elements, and form structured engineering data results on this basis, has become a problem in the current field of intelligent processing of engineering drawings. Summary of the Invention

[0003] To overcome the aforementioned deficiencies of the prior art, this invention provides an intelligent recognition, semantic parsing, and data reconstruction system for engineering drawings. By performing basic element recognition and semantic association on engineering drawing images, and reconstructing structured engineering data units based on the association results, the scattered elements in the engineering drawings can form semantic and data results oriented towards engineering objects, thereby improving the associativity and reusability of the engineering drawing processing results.

[0004] To achieve the above objectives, the present invention provides the following technical solution: A system for intelligent recognition, semantic parsing, and data reconstruction of engineering drawings, comprising: The drawing acquisition and model building module acquires engineering drawing sample images to form a training image set and acquires engineering drawing images to be processed; performs image enhancement processing on the training image set, and trains the pre-trained target detection model based on the enhanced training image set to obtain the trained target detection model; The element recognition module reads the dimension information of the engineering drawing image, determines the corresponding inference input dimension based on the dimension information, and performs recognition on the engineering drawing image based on the trained object detection model to obtain the basic element results and the corresponding category results, confidence results, and bounding box results for each basic element result. The basic element results include one or more of the following: graphic element results, text element results, annotation element results, index element results, and table element results. The tag parsing and semantic mapping module obtains the tag information table, reads the category identifiers in the tag information table, performs character format uniform processing on the category identifiers, and establishes the correspondence between the category identifiers and the Chinese category names, description information, element usage information, and associated prompt information; The semantic association and data reconstruction module receives the basic element results and their corresponding relationships. Based on the spatial adjacency, leader pointing, size coverage, numbering adjacency, and table correspondence relationships between the basic elements, it determines the attribution relationship between multiple basic elements, generates the semantic results of the engineering object, and generates structured engineering data units based on the semantic results of the engineering object. The results output module generates target annotation result map based on the bounding box results and category results, and outputs the semantic results of engineering objects and structured engineering data results.

[0005] The technical effects and advantages of this invention are as follows: 1. By identifying graphic elements, text elements, annotation elements, index elements and table elements, and generating semantic results of engineering objects based on spatial adjacency, leader pointing relationship, size coverage relationship, number adjacency relationship and table correspondence, the structured engineering data unit is reconstructed, so that the basic element identification results can be further formed into structured output oriented towards engineering objects. 2. First, determine the inference input size based on the image size. Then, divide the graphic area, text annotation area, and table area according to the distribution of line segments, text, and table boundaries, and perform corresponding type recognition for each. Since the recognition path matches the drawing layout features, it helps to improve the relevance of the basic element recognition results and the effectiveness of subsequent associated inputs. 3. Based on the bounding box results corresponding to the graphic element results, candidate association groups are constructed, and the affiliation relationship is determined through adjacency region judgment, leader endpoint location, number field consistency judgment and conflict resolution. Since multiple basic elements are converged before affiliation is determined, the probability of erroneous association in complex drawing pages can be reduced. 4. Merge multiple basic element results belonging to the semantic results of the same engineering object into the same structured engineering data unit, and form a cross-page unified data unit and parameterized object record when the conditions are met; the identification results are further reorganized into object-level data, which is convenient for subsequent archiving, retrieval, comparison and business calls. Attached Figure Description

[0006] Figure 1 This is a schematic diagram of the overall system of the present invention; Figure 2 This is a schematic diagram of the element identification process of the present invention; Figure 3 This is a schematic diagram of the semantic association and data reconstruction process of the present invention. Detailed Implementation

[0007] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0008] Refer to the instruction manual appendix Figure 1-3 A system for intelligent recognition, semantic parsing, and data reconstruction of engineering drawings, comprising: Drawing Acquisition and Model Building Module The process involves acquiring sample images of engineering drawings to form a training image set, and then acquiring the engineering drawing images to be processed. Specifically, in the scenario of digitizing engineering drawings for residential building projects, manually annotated page images are first read from existing building floor plans, door and window index diagrams, detailed node diagrams, and equipment lists of the project as sample images of the engineering drawings. The manually annotated content and the basic element results output by the subsequent element recognition module adopt the same category system, including at least the category identifiers and bounding box coordinates corresponding to graphic elements, text elements, annotation elements, index elements, and table elements. Subsequently, the sample images of the engineering drawings and their annotation records are organized into a training image set on a page-by-page basis. Each training record in the training image set includes one page image. The image set includes the category identifiers and bounding box coordinates of all basic elements on the page. The bounding box coordinates are represented by the x-coordinate of the top left corner, the y-coordinate of the top left corner, the x-coordinate of the bottom right corner, and the y-coordinate of the bottom right corner. After the training image set is formed, the engineering drawing images to be processed are read from the newly received drawing pages of the residential building project. The engineering drawing images to be processed can be full-page floor plans or images of concentrated areas of doors and windows, node areas, or partial areas of equipment tables extracted from full-page drawings. To ensure that the category results output by the subsequent element recognition module can correctly correspond to the category identifiers in the label parsing and semantic mapping module, the basic element category system in the engineering drawing images to be processed is consistent with the basic element category system in the training image set.

[0009] Image enhancement processing performed on the training image set includes scale perturbation enhancement, image stitching enhancement, and orientation flip enhancement. Specifically, before inputting the training image set into the pre-trained object detection model, image enhancement processing is performed on each sample image in the training image set, and the bounding box coordinates in the corresponding annotation records are updated synchronously. When performing scale perturbation enhancement, the width and height of the sample image are scaled by the same proportion, and then the x-coordinate of the upper left corner, y-coordinate of the upper left corner, x-coordinate of the lower right corner, and y-coordinate of the lower right corner of each bounding box in the sample image are synchronously converted by the same proportion. In the scenario of this residential building project, this step is used to cover the situation where the same door / window number, axis number, or equipment table field is displayed at different sizes in full-page browsing and partial screenshot modes. When performing image stitching enhancement, multiple sample images are selected from the training image set and stitched into a combined image according to a predetermined arrangement. Then, based on the placement area of ​​each sample image in the combined image, the original bounding box coordinates are recalculated. The calculation yields the annotation results corresponding to the combined image. In this residential building project scenario, the building floor plan page, the door and window index page, and the equipment table page can be stitched together into the same training image, allowing graphic elements, text elements, index elements, and table elements that were originally distributed on different pages to coexist in the same image. When performing orientation flip enhancement, the sample image is flipped along the horizontal or vertical direction, and the horizontal or vertical coordinates of each bounding box are recalculated according to the flip direction. In the residential building project scenario, this execution process is used to cover the situation where local node details are mirrored or have changed orientation during screenshotting, archiving, or transmission. When the three enhancement processes are combined, the scale perturbation enhancement is first performed on the original sample image, then the scale perturbation sample image is used for image stitching enhancement, and finally, orientation flip enhancement is performed on the stitching result. After each enhancement process is completed, the enhanced image and the synchronously updated annotation results are written to the training cache as input for subsequent model training.

[0010] When training the pre-trained object detection model based on the enhanced training image set, training images of multiple different input sizes are input into the pre-trained object detection model. The parameters are updated based on the training results at each input size, resulting in a trained object detection model suitable for different drawing scaling ratios, screenshot cropping ratios, and local drawing deformation conditions. Specifically, after the enhanced training image set is formed, a pre-trained object detection model that has completed general object learning is selected as the initial model. Then, multiple different input sizes are selected sequentially from the candidate input size set. Training images from the same batch are scaled to the image size corresponding to each input size before being input into the pre-trained object detection model. The candidate input size set is used in the element recognition module. A consistent size configuration table is used for the inference input size set. This table records multiple available input size values ​​in ascending order. Both the training and recognition phases read candidate sizes from this same size configuration table, ensuring consistency between the input scale distribution during training and recognition. After each forward computation using an input size, the pre-trained object detection model outputs the predicted class and bounding box results for each basic element at that input size. Subsequently, the predicted class results are compared item by item with the class labels in the synchronously scaled annotation records, and the predicted bounding box results are compared item by item with the synchronously scaled bounding box coordinates. Class error and position error are calculated respectively, and a result is formed based on the class error and position error for the current input size. The training results are used to update the parameters in the pre-trained target detection model after the training results are generated for the current input size. Then, the process is repeated for the next input size. In this way, the same batch of training images will participate in training at multiple different input sizes, and the model parameters will be continuously constrained and corrected under multiple display scales. In the scenario of residential building projects, this training method corresponds to three types of actual page states: the first type is the state of displaying the entire floor plan in a thumbnail, at which time the display size of the axis numbers, dimension numbers, and equipment table fields on the page is reduced; the second type is the state of displaying a screenshot after cropping the concentrated area of ​​doors and windows, node area, or partial area of ​​equipment table from the entire drawing, at which time the display size of the door and window numbers, dimension numbers, and equipment table fields is reduced. Line annotations and table fields are magnified relative to the full-page state; the third type is the local drawing deformation state formed after the page is compressed, stretched, or converted in format, at which time the boundary ratio of some graphic elements changes; after completing the training image input and parameter update under multiple different input sizes, the target detection model under the current parameter state is used as the output of the trained target detection model; since the pre-trained target detection model has repeatedly executed category error and position error constraints and completed parameter updates on the training images corresponding to the above states, the output trained target detection model can maintain a stable basic element recognition ability for full-page engineering drawing images, local area engineering drawing images, and engineering drawing images with local deformation when called by the subsequent element recognition module.

[0011] Element recognition module Determining the corresponding inference input size based on the size information includes: reading the width and height of the engineering drawing image; when the width is greater than the height, using the width as the size determination value; when the height is greater than or equal to the width, using the height as the size determination value; calculating the scaling change between the size determination value and each candidate inference input size, selecting the candidate inference input size that minimizes the scaling change as the corresponding inference input size, and using this inference input size to perform recognition. Specifically, after the drawing acquisition and model building module outputs the trained target detection model and inputs the engineering drawing image to be processed, the element recognition module first reads the pixel width and pixel height of the current engineering drawing image. In this implementation, the objects to be processed are full-page floor plans, partial screenshots of door and window indexes, partial screenshots of stairwell detail drawings, and partial area images of equipment tables in residential building projects. When the width is greater than the height in the reading result, the width is used as the size determination value; when the height is greater than or equal to the width, the height is used as the size determination value. Subsequently, the scaling change between the size determination value and each candidate inference input size is calculated. The scaling change is the absolute value of the difference between the size determination value and the candidate inference input size, which is used to represent the adjustment required for the current image after inputting the candidate inference input size. Scale range; after all the scaling changes corresponding to multiple candidate inference input sizes have been calculated, the candidate inference input size with the smallest scaling change is selected as the inference input size of the current engineering drawing image. The engineering drawing image is then scaled according to this inference input size and fed into the trained target detection model. For example, when performing recognition on a full-page floor plan, if the current page width is significantly larger than the screenshot of the concentrated area of ​​doors and windows extracted from the floor plan, the size judgment value corresponding to the full-page floor plan is larger, and the selected inference input size is more suitable for the scaling requirements of the full-page page. However, since the size judgment value of the screenshot of the concentrated area of ​​doors and windows is smaller, the selected inference input size is closer to the original display scale of the local image, thereby reducing the distortion of the door and window number edges and the compression of the size numbers caused by excessive magnification or compression during input.

[0012] The recognition process for engineering drawing images includes: first, dividing the engineering drawing image into graphic regions, text annotation regions, and table regions based on the distribution of line segments, text, and table boundaries in the image; then, performing graphic element recognition on the graphic regions, text element recognition and annotation element recognition on the text annotation regions, and index element recognition and table element recognition on the table regions; and finally, merging the recognition results of each region to form the basic element result. Specifically, after the current engineering drawing image completes the inference input size adjustment, the element recognition module first performs region division on the image content distribution. Line segment distribution represents the clustering state of wall boundaries, door and window outlines, component outlines, node frames, and table separator lines on the page; text distribution represents the clustering state of room numbers, axis numbers, dimension numbers, explanatory text, and node numbers; table boundary distribution represents the grid boundary state formed by horizontal and vertical separator lines in equipment lists, door and window lists, or material lists. In this residential building project scenario, wall outlines, door and window boundaries, and stair outlines in floor plans typically form dense areas of continuous line segments; door and window numbers, room names, and dimension numbers typically form dense areas of text annotations; and table grids in equipment lists typically form regular boundary areas. Accordingly, the element recognition module divides locations with dense line segments and clear closed boundaries into graphic regions, locations with concentrated character blocks and leader text into text annotation regions, and locations where horizontal and vertical boundary lines intersect to form regular grids into table regions. When both graphic and table boundaries exist at the same location, the table boundary is prioritized. The boundary distribution is divided into table areas, and then the remaining part is divided into graphic areas and text annotation areas to avoid the number field in the equipment table being misidentified as ordinary text elements. After the area division is completed, graphic element recognition is performed on the graphic area, and the graphic element results corresponding to the door and window boundaries, wall boundaries, stair outlines, and node outlines are output. Text element recognition and annotation element recognition are performed simultaneously on the text annotation area, and the text element results corresponding to the room name, axis number, dimension number, and explanatory text, as well as the annotation element results corresponding to dimension leaders, node leaders, and other annotation structures are output. Index element recognition and table element recognition are performed on the table area, and the index element results corresponding to the door and window index number, node index number, and equipment number, as well as the table element results corresponding to the equipment table area, field area, and table grid area are output. After the recognition of each area is completed, the element recognition module merges the graphic element results, text element results, annotation element results, index element results, and table element results into the same basic element result set according to a unified page coordinate system, so that subsequent modules can read all the recognition results of the current page in the same result cache.

[0013] The trained object detection model is used to recognize engineering drawing images, obtaining basic element results and corresponding category, confidence, and bounding box results for each basic element. Specifically, in the above-mentioned region recognition process, the trained object detection model simultaneously outputs the predicted category, prediction confidence, and predicted bounding box results for each candidate object; the element recognition module determines the basic element type of the candidate object based on the predicted category, indicates the credibility of the type determination based on the prediction confidence, and indicates the location range of the candidate object in the current engineering drawing image based on the predicted bounding box; then, the candidate objects that pass the screening are retained and written into the basic element result set, and the corresponding category, confidence, and bounding box results are written into the recognition result cache; in the scenario of residential building projects, when performing recognition on a full-page floor plan, the boundaries of doors and windows, wall edges, etc., can be obtained. The candidate recognition results corresponding to the boundary and staircase outline can also be used to obtain candidate recognition results corresponding to room name, axis number and size number; when performing recognition on a partial screenshot of the door and window index, candidate recognition results corresponding to the index number and adjacent table area can be obtained; when performing recognition on a partial area of ​​the equipment table, candidate recognition results corresponding to the table grid area, field text and equipment number can be obtained; since the category results, confidence results and bounding box results are in item-by-item correspondence with the results of each basic element, the subsequent label parsing and semantic mapping module can directly read the corresponding category identifier according to the category result, and the semantic association and data reconstruction module can also directly perform spatial adjacency relationship, leader line pointing relationship, size coverage relationship, number adjacency relationship and table correspondence relationship determination according to the bounding box result.

[0014] The basic element results include one or more of the following: graphic element results, text element results, annotation element results, index element results, and table element results. Specifically, after the identification of each area is completed and the results are merged, the element recognition module does not require each engineering drawing image to output all five categories of basic element results simultaneously. Instead, it forms a corresponding set of basic element results based on the types of basic elements actually contained in the current page. For example, in a full-page floor plan, it usually contains graphic element results, text element results, and annotation element results simultaneously; in the door and window index area, it usually contains index element results and text element results; in the equipment table area, it usually contains table element results, index element results, and some text element results. Therefore, when organizing the recognition output, the element recognition module writes one or more of the graphic element results, text element results, annotation element results, index element results, and table element results actually identified on the current page into the same set of basic element results, and keeps the corresponding category results, confidence results, and bounding box results of these basic element results stored synchronously. After this processing, the subsequent label parsing and semantic mapping module can directly read the corresponding category identifier and description information according to the category results, and the semantic association and data reconstruction module can also directly perform spatial determination and object attribution determination according to the bounding box results.

[0015] Tag parsing and semantic mapping module The process involves obtaining a label information table, reading category identifiers from the table, performing character format standardization on the category identifiers, and establishing a correspondence between category identifiers and Chinese category names, descriptive information, element usage information, and associated prompts. Specifically, after the element recognition module outputs the basic element results and the corresponding category results, confidence results, and bounding box results for each basic element result, the label parsing and semantic mapping module begins to semantically interpret the category results in the recognition results. In this implementation, the processing objects are the floor plans, door and window index diagrams, detailed drawings of local nodes, and equipment table pages of residential building projects. The label parsing and semantic mapping module first obtains the label information table corresponding to the current engineering drawing category system, and then reads the category identifiers line by line from the label information table. After reading, the category identifiers are processed to ensure that the same category can be mapped to the same semantic description item under different page sources, different input methods, and different file formats. After the character format standardization is completed, the processed category identifiers are used as the basis for the semantic interpretation. As a matching field, a correspondence is established between the category identifier and the Chinese category name, description information, element usage information, and association prompt information. The Chinese category name provides the Chinese name of the category in the engineering drawing; the description information provides a description of the category's content; the element usage information provides a description of the usage of the basic elements of the category in the engineering drawing; and the association prompt information provides a prompt indicating the category of the associated object that this category will preferentially correspond to in subsequent semantic association. Thus, when the semantic association and data reconstruction module receives the basic element results, it can not only read the category results and bounding box results corresponding to the basic elements, but also simultaneously read the corresponding Chinese category name, description information, element usage information, and association prompt information to support object attribution determination and structured reconstruction.

[0016] The obtained label information table is created by writing the category identifier, Chinese category name, description information, element usage information, and associated prompt information corresponding to each category into a table file, with each row corresponding to one category. The category identifier in each row is read, converted into a character type, and the category identifier is processed to remove leading and trailing whitespace characters and abnormal whitespace difference characters. Then, the correspondence between the category identifier and the Chinese category name, description information, element usage information, and associated prompt information is established based on the processed category identifier. Specifically, in this implementation process, a table file is first established based on the basic element category system adopted by the residential building project. Each row in the table file corresponds to a category; for example, door and window boundaries correspond to a category, room names correspond to a category, dimension leaders correspond to a category, door and window index numbers correspond to a category, and equipment table areas correspond to a category. In each row, the category identifier, Chinese category name, description information, element usage information, and associated prompt information are written sequentially to form a complete tag information table. When the tag parsing and semantic mapping module calls this table file, it first reads the category identifiers row by row, and then converts the category identifiers into character types to eliminate inconsistencies in fields caused by numerical, character, or mixed input methods. Subsequently, the category identifiers are processed to remove leading and trailing whitespace characters to remove whitespace caused by manual input, copying and pasting, or table editing. Spaces; then perform abnormal whitespace character removal processing on the category identifier to remove invisible whitespace characters or special whitespace characters, so that the same category identifier remains consistent across different source files; after the above processing is completed, the processed category identifier is used as a matching field to establish a one-to-one correspondence with the Chinese category name, description information, element usage information, and associated prompt information; for example, in this residential building project, a certain door and window index number category may be entered in different forms on different pages, such as with a leading space, with a trailing space, or with abnormal whitespace characters. After character type conversion, leading and trailing whitespace character removal processing, and abnormal whitespace character removal processing, these different entry forms are unified into the same category identifier, thereby ensuring that the door and window index number can be mapped to the same Chinese category name, description information, element usage information, and associated prompt information in the floor plan page, index map page, and table page.

[0017] Each row in the table file contains a name description field, a content description field, a purpose description field, and an association description field. Specifically, these fields, along with the category identifier field, constitute the same row of tag records. The category identifier field is the first field of this row of tag records, followed by the name description field, content description field, purpose description field, and association description field. When the tag parsing and semantic mapping module reads the tag information table, it reads the contents of these fields simultaneously on a row-by-row basis. This ensures that after the character format of the same category identifier is standardized, it can be directly mapped to the Chinese category name, description information, element purpose information, and association prompt information in the same row of tag records.

[0018] After establishing the correspondence between category identifiers and Chinese category names, description information, element usage information, and associated prompt information, the Chinese category name is written into the corresponding category's name description field, the description information is written into the corresponding category's content description field, the element usage information is written into the corresponding category's usage description field, and the associated prompt information is written into the corresponding category's association description field. Among these, the element usage information is formed by writing the usage description of the basic element of this category in the engineering drawings, and the associated prompt information is formed by writing the category name of the associated object corresponding to the basic element of this category when performing semantic association. Specifically, in this implementation, in addition to the category identifier field, the table file also configures a name description field, content description field, purpose description field, and association description field for each category. After the correspondence between the category identifier and the Chinese category name, description information, element purpose information, and association prompt information is established, the tag parsing and semantic mapping module writes the Chinese category name into the corresponding category's name description field to indicate the Chinese name of the basic element of that category; writes the description information into the corresponding category's content description field to indicate the content explanation of the basic element of that category; writes the element purpose information into the corresponding category's purpose description field to indicate the position or purpose type of the basic element of that category in the engineering drawings; and writes the association prompt information into the corresponding category's association description field to indicate the object category that the basic element of that category will be preferentially associated with when performing semantic association. In this residential building project scenario, if a category identifier corresponds to a door / window index number, its Chinese category name can be written as "Door / Window Index Number". The tag number and description information can be written as a number used to identify the type or serial number of a door or window. The element usage information can be written as a correspondence between the element and the door or window boundary or door or window table field. The associated prompt information can be written as the door or window boundary category or door or window table field category. If a category identifies a corresponding dimension leader, its Chinese category name can be written as the dimension leader. The description information can be written as a leader structure used to indicate the dimension annotation direction and coverage. The element usage information can be written as a dimension coverage relationship with the graphic element boundary. The associated prompt information can be written as the graphic element category. Through this writing method, the tag parsing and semantic mapping module outputs not only a simple category name explanation, but also the category's name description, content description, usage description, and association description in the engineering drawing. This allows the subsequent semantic association and data reconstruction module to directly use the corresponding field content to complete object attribution analysis and data reconstruction when determining the attribution relationship of text elements, annotation elements, index elements, and table elements.

[0019] Semantic association and data reconstruction module Receives the results of basic elements and their corresponding relationships. Specifically, after the element recognition module outputs the basic element results and the corresponding category results, confidence results, and bounding box results for each basic element result, the semantic association and data reconstruction module reads the graphic element results, text element results, annotation element results, index element results, and table element results already identified in the current engineering drawing page as associated input. At the same time, it reads the Chinese category name, description information, element usage information, and associated prompt information corresponding to each category result from the label parsing and semantic mapping module. In this implementation, the processing objects are the floor plans, door and window index diagrams, partial node details, and equipment table pages in residential building projects. For example, after the recognition of a certain floor plan page is completed, the input received by the semantic association and data reconstruction module can simultaneously include the graphic element results corresponding to the door and window boundaries, the text element results corresponding to the door and window numbers, the annotation element results corresponding to the dimension leaders, the index element results corresponding to the door and window index numbers, and the table element results corresponding to the adjacent table areas. In this step, these input results keep the original category results, bounding box results, and description fields unchanged, serving as the basic data for subsequent attribution determination, engineering object semantic result generation, and structured engineering data unit generation.

[0020] Before generating semantic results for engineering objects, candidate association groups are constructed based on the bounding box results corresponding to each graphic element result. When constructing candidate association groups, the bounding box result of the corresponding graphic element result is used as the center, and the same pixel width is extended to the four sides of the bounding box to form an outer adjacency range. Text element results located within this outer adjacency range are collected, and annotation element results whose leader endpoints fall within the bounding box result range are collected. A square area with a predetermined side length is taken as the center of the bounding box of the graphic element result as the local area. The number field in the table element results within the local area is read. The index element results, graphic element results, text element results, annotation element results, and table element results with the same number content as the number field are used as the basic element results in the same candidate association group. Then, the attribution relationship determination is performed in each candidate association group. Specifically, in this implementation, each graphic element result is first used as the central element of a potential engineering object. For example, after identifying a door or window boundary in the floor plan, the boundary frame result corresponding to that door or window boundary is used as the center, and an outer adjacent range is formed by expanding outwards to the top, bottom, left, and right sides of the boundary frame. This outer adjacent range is used to collect door and window number text, size text, or explanatory text near the door or window boundary. The outward expansion width is taken as a fixed proportion of the length of the long side of the current graphic element result's boundary frame, and is not less than the pixel value corresponding to the average text height of the current page. The average text height of the current page is obtained by averaging the boundary frame heights of the identified text element results on the page. In this way, when the door or window boundary is large, the outward expansion width increases synchronously with the boundary size; when the door or window boundary is small, the outward expansion width is still not less than the basic height of the recognizable text on the current page, so that text elements near the graphic element can be stably collected. At the same time, for those adjacent to the door or window boundary... The annotation element results are positioned by the endpoint of the leader line. All annotation element results whose endpoints fall within the boundary frame of the door / window are included in the current candidate association group. Then, a local area is constructed with the center point of the door / window boundary frame as the center. The local area is in the form of a square area, and its side length is a fixed multiple of the length of the long side of the current graphic element result boundary frame, and is not less than twice the pixel value corresponding to the average text height of the current page. Then, the number field of the table element results in the local area is read, such as the door / window number field in the door / window table. The index element results with the same number content as the number field are added to the current candidate association group. After this process, the current candidate association group contains graphic element results, text element results, annotation element results, index element results and table element results at the same time. This brings the basic elements that were originally scattered in different positions on the page to the same candidate project object, providing a unified input basis for subsequent attribution determination.

[0021] Determining the hierarchical relationships between multiple basic elements includes: based on the bounding box results corresponding to the graphic element results, expanding outwards from the bounding box by a predetermined pixel width to form an adjacent region; when the center point of the bounding box of the text element results is located within the adjacent region, it is determined that there is an object association relationship between the text element results and the graphic element results; performing line segment tracing on the leaders in the annotation element results, taking the endpoint of the line segment opposite to the text element results as the leader endpoint; when the leader endpoint falls within the bounding box range of the graphic element results, it is determined that there is an annotation association relationship between the annotation element results and the graphic element results; when the number content in the index element results is consistent with the number field in the table element results, it is determined that there is a correspondence relationship between the index element results and the table element results; the semantic association and data reconstruction module generates the semantic results of the engineering objects based on the object association relationship, annotation association relationship, and correspondence relationship.Specifically, after the candidate association group is formed, the object association relationship is first determined around the graphic element result. The adjacent region is formed by expanding the top, bottom, left, and right sides of the graphic element result's bounding box by a predetermined pixel width. The annular area between the original bounding box and the expanded bounding box is considered the adjacent region. The predetermined pixel width is a fixed proportion of the length of the long side of the current graphic element result's bounding box and is not less than the pixel value corresponding to the average text height of the current page. If the center point of the bounding box of a text element result falls into this adjacent region, then the text element result is considered to have an object association relationship with the graphic element result. For example, the bounding box of a door / window number text. If the center point is located in the adjacent area outside the door / window boundary, then the door / window number text is associated with the door / window boundary as the same object. Next, line segment tracing is performed on the leader lines in the annotation element results. When the leader lines in the annotation element results are formed by connecting multiple line segments end-to-end, first, continuous thin line segments are extracted within the bounding box corresponding to the annotation element results, and then multiple line segments are sequentially connected according to the connection relationship of the line segment endpoints to form the leader line trajectory. Taking the end that is in contact with or closest to the bounding box of the text element results as the leader line starting point, each connecting line segment is traversed sequentially along the leader line trajectory, and the endpoint of the last line segment opposite to the text element results is taken as the leader line ending point. If there is an arrow, dot, or short diagonal mark at the endpoint of the last line segment... If a marker is found to be connected to a graphic element result, the end of the marker that contacts the graphic element result is taken as the final leader line endpoint. If multiple connectable leader line trajectories exist within the same marker element result, the trajectory closest to the bounding box of the text element result is retained as the valid leader line trajectory. If the leader line endpoint falls within the bounding box of the graphic element result, the marker element result and the graphic element result are determined to have a marker association relationship. For example, if the endpoint of a dimension leader line falls within the bounding box of a door or window, the dimension marker corresponding to the dimension leader line is associated with the door or window boundary. Then, a numbering consistency check is performed on the index element result and the table element result. If the numbering content in the index element result matches that in the table element result... If the number fields in the two are consistent, then the two are identified as corresponding. In the scenario of residential building projects, this process can match the index number in the door and window index diagram with the number field in the door and window table. After the object association, annotation association and correspondence are all formed, the basic elements that have completed the attribution determination within the same candidate association group are combined to generate the semantic result of the engineering object. The semantic result of the engineering object includes at least the graphic element corresponding to the object, the text element corresponding to the object, the annotation element corresponding to the object, and the index and table information related to the object, so that the subsequent data reconstruction no longer faces scattered recognition results, but faces a result set that has completed object-level semantic merging.

[0022] When determining the attribution relationship within a candidate association group, if the same text element result is located within the adjacent regions of multiple graphic element results, the shortest boundary distance between the bounding box of the text element result and the bounding box of each graphic element result is calculated, and the graphic element result corresponding to the shortest boundary distance is determined as the associated object of the text element result; if the endpoint of the leader line of the same label element result falls within the bounding box of multiple graphic element results, the distance between the endpoint of the leader line and the center position of the bounding box of each graphic element result is calculated, and the graphic element result with the smallest distance is determined as the associated object of the label element result; if the same index element result is consistent with the number field in multiple table element results, the table element result located within the same local region as the index element result and whose bounding box center position is closest to the index element result is determined as the associated object of the index element result. Specifically, in the actual pages of residential building projects, door and window number text, dimension leaders, or index numbers may be close to multiple candidate project objects simultaneously. Therefore, after establishing the initial attribution relationship, further conflict resolution is required. If the same text element result falls within the adjacent area of ​​two adjacent door and window boundaries, the shortest boundary distance between the text element's bounding box and the two door and window bounding boxes is calculated, and the door and window boundary with the smaller distance is determined as the final associated object of the text element. If the leader endpoint of the same annotation element result falls within the bounding box of two adjacent graphic elements, the distance from the leader endpoint to the center of the two graphic element bounding boxes is calculated, and the one with the smaller distance is determined as the final associated object. The final associated object for this annotation element is determined by the following: If the same index element result is consistent with the number field in multiple table element results, the selection is first limited to table element results located in the same local area as the index element, and then the table element result with the closest bounding box center position is selected as the final associated object. The bounding box center position is calculated based on the average of the x-coordinates of the top left corner and the bottom right corner, and the average of the y-coordinates of the top left corner and the bottom right corner. After conflict resolution, each text element result, annotation element result, and index element result retains only one final associated object, thus ensuring that the attribution relationship in the semantic result of the engineering object is unique and executable.

[0023] When generating structured engineering data units, multiple basic element results belonging to the semantic results of the same engineering object are merged and written into the same structured engineering data unit. The minimum x-coordinate, minimum y-coordinate, maximum x-coordinate, and maximum y-coordinate of each bounding box result involved in the merging form the position range; the category results involved in the merging form the object category; the Chinese category name involved in the merging forms the object name; and the descriptive information involved in the merging forms the associated descriptive information. When the same object category and object name appear repeatedly on different engineering drawing pages, and the relative positional relationship of the corresponding position range is consistent with the numbering content, the structured engineering data units on multiple pages are merged into a cross-page unified data unit. When the cross-page unified data unit contains multiple dimension values, the dimension value that appears most frequently is used as the dimension parameter. When multiple dimension values ​​appear the same number of times, the dimension value derived from the table field content is used as the dimension parameter. The object identifier parameter is extracted from the numbering content, and the attribute parameter is extracted from the table field content. The dimension parameter, object identifier parameter, and attribute parameter are written into the same parameterized object record. Specifically, after the semantic results of an engineering object are formed, the semantic association and data reconstruction module first merges the results of multiple basic elements belonging to the semantic results of the same engineering object into the same structured engineering data unit. For example, for the same door and window object, its graphic element results, the text element results corresponding to the door and window number text, the annotation element results corresponding to the dimension leader, and the table element results of the corresponding fields in the door and window table can be written into the same structured engineering data unit. When writing, the horizontal and vertical coordinates are first read from the bounding box results corresponding to these basic element results, and then the position range of the engineering object is formed by the minimum horizontal coordinate, minimum vertical coordinate, maximum horizontal coordinate, and maximum vertical coordinate. Then, the object category is generated according to the category results involved in the merging, the object name is generated according to the Chinese category name involved in the merging, and the association description information is generated according to the description information involved in the merging. When the same object When category and object names appear repeatedly on different engineering drawing pages, and the relative positional relationship of their locations on these pages is consistent with the numbering content (e.g., the same door / window object appears simultaneously on both the floor plan page and the door / window index page), the structured engineering data units on multiple pages are further merged into a unified cross-page data unit. The consistency of the relative positional relationship is determined by the page orientation relationship between the object to be compared and adjacent reference objects on the same page. Reference objects are preferentially selected from axis number text, adjacent graphic element boundaries, or adjacent table areas located on the same page as the current object. When objects on two pages are located on the same side of their corresponding reference objects, and the changes in the lateral and longitudinal distances to the center point of the corresponding reference object's bounding box do not exceed a predetermined scale range, the relative positional relationship of the objects on the two pages is determined to be consistent.Both horizontal and vertical distances are represented by the coordinate difference between the center point of the current object's bounding box and the center point of the reference object's bounding box. The predetermined ratio range is determined by a fixed proportion of the length of the long side of the reference object's bounding box. If multiple candidate matching objects exist for the same object on different pages, the candidate matching objects with the same number and the same relative positional relationship are prioritized as the unified cross-page object. In the unified cross-page data unit, if multiple dimension values ​​from dimension annotations exist simultaneously, these dimension values ​​are counted according to their numerical content, and the dimension value with the most occurrences is selected as the dimension parameter. If multiple dimension values ​​have the same number of occurrences, the dimension value from the table field content is prioritized as the dimension parameter. At the same time, object identification parameters are extracted from the number content, such as extracting doors and windows. The system retrieves the object number or equipment number; extracts attribute parameters from the table fields, such as door / window material, equipment model, or node type; finally, it writes the size parameters, object identifier parameters, and attribute parameters into the same parameterized object record; the parameterized object record uses a data record structure with a fixed field order, and the data record structure includes at least an object identifier parameter field, a size parameter field, and an attribute parameter field; when the parameterized object record also contains object category, object name, location range, and associated description information, these are written in the following order: object category, object name, location range, associated description information, object identifier parameter field, size parameter field, and attribute parameter field; thus, the resulting parameterized object record contains not only the object name and location but also the parameter fields required for subsequent comparison, archiving, and business calls.

[0024] Result Output Module Based on the bounding box and category results, a target annotation result map is generated, and the semantic results of engineering objects and structured engineering data results are output. Specifically, after the semantic results of engineering objects and structured engineering data units are generated, the result output module first overlays annotation boxes and category labels on the original engineering drawing image according to the bounding box and category results corresponding to the basic element results, generating a target annotation result map. In the scenario of this residential building project, the target annotation result map can intuitively show the location and category of door and window boundaries, room names, dimension leaders, index numbers, and equipment table areas in the original page. At the same time, the result output module outputs the semantic results of engineering objects and structured engineering data results corresponding to the current page, allowing users to view the distribution of basic elements on the page through the target annotation result map, view the belonging relationships between various basic elements through the semantic results of engineering objects, and view object categories, object names, location ranges, association descriptions, and parameterized object records through the structured engineering data results.

[0025] When outputting structured engineering data results, the object category, object name, location range, associated description information, and parameterized object record are written into the same data record, and the target annotation result image is saved. Specifically, in the structured engineering data result output stage, the result output module writes the object category, object name, location range, associated description information, and parameterized object record corresponding to each engineering object into the same data record in a unified field order. In this implementation, each door / window object, node object, or equipment object in the residential building project corresponds to an independent data record. For example, the data record of a certain door / window object simultaneously includes the door / window category, door / window number, the location range of the door / window on the page, the associated size description information, and the size parameters and attribute parameters extracted from the unified data unit across pages. After the data record is written, the result output module synchronously saves the target annotation result image, so that the same engineering object has both structured data records for subsequent archiving, retrieval, and comparison, and target annotation result images for manual verification of page position and recognition results.

[0026] Working Principle: The system trains a target detection model using labeled engineering drawing sample images, enabling it to adapt to various display conditions, including full-page viewing, partial screenshots, and slight deformation of the drawings. The drawing image to be processed is then fed into the element recognition module. First, an appropriate inference input size is selected based on the image dimensions. Then, graphic elements, text elements, annotation elements, index elements, and table elements are identified according to graphic areas, text annotation areas, and table areas, respectively. Next, the system reads the label information table, mapping the identified category results to Chinese category names, descriptions, element usage information, and associated prompts. Based on this, the system further merges the previously scattered recognition results into semantic results for the same engineering object based on the proximity of basic elements, leader line placement, numbering consistency, and table correspondence. Finally, these semantic results are further organized into structured engineering data units, and the target annotation result image is output. And data recording; for example, in the digitization and organization of drawings for residential building projects, the system first trains the model using pre-annotated architectural floor plans, door and window index diagrams, detail drawings, and equipment table pages; when a newly uploaded floor plan or a partial screenshot of a door or window enters the system, the system first identifies the door and window boundaries, door and window numbers, dimension leaders, index numbers, and table areas, and then uses the label table to know what these identification results represent; if a door or window number is close to a door or window boundary, and a dimension leader falls on this door or window boundary, and the door and window index diagram and door and window table contain the same number, the system will treat these scattered pieces of information as components of the same door or window object, and further generate the category, name, location range, description information, and size and attribute parameters of this door or window object; in this way, the content that was originally scattered in different locations on the drawings will be organized into a searchable, comparable, and verifiable data record.

[0027] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A system for intelligent recognition and semantic analysis and data reconstruction of engineering drawings, characterized in that, include: The drawing acquisition and model building module acquires engineering drawing sample images to form a training image set, and acquires engineering drawing images to be processed; Image augmentation is performed on the training image set, and the pre-trained target detection model is trained based on the augmented training image set to obtain the trained target detection model. The element recognition module reads the dimension information of the engineering drawing image, determines the corresponding inference input dimension based on the dimension information, and performs recognition on the engineering drawing image based on the trained object detection model to obtain the basic element results and the corresponding category results, confidence results, and bounding box results for each basic element result. The basic element results include one or more of the following: graphic element results, text element results, annotation element results, index element results, and table element results. The tag parsing and semantic mapping module obtains the tag information table, reads the category identifiers in the tag information table, performs character format uniform processing on the category identifiers, and establishes the correspondence between the category identifiers and the Chinese category names, description information, element usage information, and associated prompt information; The semantic association and data reconstruction module receives the basic element results and their corresponding relationships. Based on the spatial adjacency, leader pointing, size coverage, numbering adjacency, and table correspondence relationships between the basic elements, it determines the attribution relationship between multiple basic elements, generates the semantic results of the engineering object, and generates structured engineering data units based on the semantic results of the engineering object. The results output module generates target annotation result map based on the bounding box results and category results, and outputs the semantic results of engineering objects and structured engineering data results.

2. The system of claim 1, wherein, In the drawing acquisition and model construction module Image enhancement processing performed on the training image set includes scale perturbation enhancement, image stitching enhancement, and orientation flipping enhancement. When training the pre-trained target detection model based on the enhanced training image set, the training images are input into the pre-trained target detection model with multiple different input sizes. The parameters are updated for the training results under each input size to obtain a trained target detection model suitable for different drawing scaling ratios, screenshot cropping ratios, and local drawing deformation conditions.

3. The system of claim 1, wherein, In the element recognition module Determining the corresponding inference input size based on the size information includes: reading the width and height of the engineering drawing image; when the width is greater than the height, using the width as the size determination value; when the height is greater than or equal to the width, using the height as the size determination value; calculating the scaling change between the size determination value and each candidate inference input size, selecting the candidate inference input size that minimizes the scaling change as the corresponding inference input size, and using this inference input size to perform recognition.

4. The system of claim 3, wherein, In the element recognition module The recognition process for engineering drawing images includes: first, dividing the engineering drawing image into graphic regions, text annotation regions, and table regions based on the distribution of line segments, text, and table boundaries in the image; then, performing graphic element recognition on the graphic regions, text element recognition and annotation element recognition on the text annotation regions, and index element recognition and table element recognition on the table regions; and finally, merging the recognition results of each region to form the basic element result.

5. The system of claim 1, wherein, In the tag parsing and semantic mapping module The obtained label information table is created by writing the category identifier, Chinese category name, description information, element usage information, and associated prompt information corresponding to each category into a table file, with each row corresponding to one category. The category identifier in each row is read, converted into a character type, and the category identifier is processed to remove leading and trailing whitespace characters and abnormal whitespace difference characters. Then, the correspondence between the category identifier and the Chinese category name, description information, element usage information, and associated prompt information is established based on the processed category identifier.

6. The system of claim 5, wherein, In the tag parsing and semantic mapping module Each row in the table file contains a name description field, a content description field, a purpose description field, and an association description field. After establishing the correspondence between the category identifier and the Chinese category name, description information, element purpose information, and association prompt information, the Chinese category name is written into the corresponding category's name description field, the description information is written into the corresponding category's content description field, the element purpose information is written into the corresponding category's purpose description field, and the association prompt information is written into the corresponding category's association prompt field. Among them, the element purpose information is formed by writing the usage description of the basic element of this category in the engineering drawings, and the association prompt information is formed by writing the category name of the associated object corresponding to the basic element of this category when performing semantic association.

7. The system of claim 1, wherein, In the semantic association and data reconstruction module Determining the hierarchical relationships between multiple basic elements includes: based on the bounding box results corresponding to the graphic element results, expanding outwards from the bounding box by a predetermined pixel width to form an adjacent region; when the center point of the bounding box of the text element results is located within the adjacent region, it is determined that there is an object association relationship between the text element results and the graphic element results; performing line segment tracing on the leaders in the annotation element results, taking the endpoint of the line segment opposite to the text element results as the leader endpoint; when the leader endpoint falls within the bounding box range of the graphic element results, it is determined that there is an annotation association relationship between the annotation element results and the graphic element results; when the number content in the index element results is consistent with the number field in the table element results, it is determined that there is a correspondence relationship between the index element results and the table element results; the semantic association and data reconstruction module generates the semantic results of the engineering objects based on the object association relationship, annotation association relationship, and correspondence relationship.

8. The system of claim 7, wherein, In the semantic association and data reconstruction module Before generating semantic results for engineering objects, candidate association groups are constructed based on the bounding box results corresponding to each graphic element result. When constructing candidate association groups, the bounding box result of the corresponding graphic element result is used as the center, and the same pixel width is extended to the four sides of the bounding box to form an outer adjacency range. Text element results located within this outer adjacency range are collected, and annotation element results whose leader endpoints fall within the bounding box result range are collected. A square area with a predetermined side length is taken as the center of the bounding box of the graphic element result as the local area. The number field in the table element results within the local area is read. The index element results, graphic element results, text element results, annotation element results, and table element results with the same number content as the number field are used as the basic element results in the same candidate association group. Then, the attribution relationship determination is performed in each candidate association group.

9. The system of claim 8, wherein, In the semantic association and data reconstruction module When determining the attribution relationship within a candidate association group, if the same text element result is located within the adjacent regions of multiple graphic element results, the shortest boundary distance between the bounding box of the text element result and the bounding box of each graphic element result is calculated, and the graphic element result corresponding to the shortest boundary distance is determined as the associated object of the text element result; if the endpoint of the leader line of the same label element result falls within the bounding box of multiple graphic element results, the distance between the endpoint of the leader line and the center position of the bounding box of each graphic element result is calculated, and the graphic element result with the smallest distance is determined as the associated object of the label element result; if the same index element result is consistent with the number field in multiple table element results, the table element result located within the same local region as the index element result and whose bounding box center position is closest to the index element result is determined as the associated object of the index element result.

10. The system of claim 9, wherein, In the semantic association and data reconstruction module, when generating structured engineering data units, multiple basic element results belonging to the semantic results of the same engineering object are merged and written into the same structured engineering data unit. The minimum horizontal coordinate, minimum vertical coordinate, maximum horizontal coordinate, and maximum vertical coordinate in each bounding box result involved in the merging form the position range. The category results involved in the merging form the object category. The Chinese category name involved in the merging forms the object name. The descriptive information involved in the merging forms the association description information. When the same object category and object name appear repeatedly on different engineering drawing pages, and the relative positional relationship of the corresponding location range is consistent with the number content, the structured engineering data units in multiple pages are merged into a cross-page unified data unit; when the cross-page unified data unit contains multiple dimension values, the dimension value that appears most frequently is used as the dimension parameter. When multiple dimension values ​​appear the same number of times, the dimension value derived from the table field content is used as the dimension parameter; the object identifier parameter is extracted from the number content, the attribute parameter is extracted from the table field content, and the dimension parameter, object identifier parameter, and attribute parameter are written into the same parameterized object record. In the results output module, when outputting structured engineering data results, the object category, object name, location range, associated description information, and parameterized object record are written into the same data record, and the target annotation result diagram is saved.

Images