2D structure drawing information matching system and method based on deep learning

Abstract

The invention discloses a 2D structure drawing information matching system and method based on deep learning, and belongs to the technical field of drawing recognition. The target detection module is used for receiving the 2D structural engineering drawing, identifying components and annotation information in the drawing and outputting a data structure; the text recognition module is used for receiving the labeling information position data, recognizing and analyzing the labeling information position data and outputting recognized text content and position information; the information matching module is used for receiving the position and category information of the component and the text content and position information output by the text recognition module, establishing a drawing coordinate system, matching the component with the annotation information and outputting the matched component and annotation information; the data storage module and the three-dimensional model generation module are used for receiving the XML data, generating a three-dimensional model and outputting a three-dimensional model file; the technical problems of low efficiency, limited applicability, poor robustness and generalization ability and the like in the prior art are solved, and the drawing information acquisition efficiency and accuracy are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of drawing recognition, and in particular, relates to a 2D structure drawing information matching system and method based on deep learning. Background Art

[0002] 2D structural engineering drawings are an indispensable information carrier in fields such as architecture, civil engineering and manufacturing. They are used to record and communicate key information such as the design, specifications and construction of engineering projects. These drawings contain a large number of graphic elements (such as lines, symbols and annotations) and text information (such as titles, labels, dimensions and annotations), and are an important reference in the engineering design and construction process.

[0003] Currently, processing and interpreting 2D structural engineering drawings mainly relies on manual operations. Engineers and designers need to manually analyze various elements in the drawings, extract key information, and convert this information into data that can be used for subsequent work. Although this method can ensure accuracy, it is time-consuming and prone to errors due to human factors.

[0004] In order to improve efficiency, some automated drawing recognition technologies have been developed. These technologies are mainly aimed at structural engineering CAD drawings in DWG format, and require tedious pre-processing operations, including labeling and layer creation. Then, these methods usually rely on a lot of expertise to manually design feature extractors. However, this method has the following technical problems:

[0005] 1. Low efficiency: The pre-processing operation takes a long time, affecting the overall processing efficiency;

[0006] 2. Limited applicability: It is mainly for CAD drawings in DWG format, and has limited processing capabilities for other formats (such as PDF or scanned drawings);

[0007] 3. Poor robustness and generalization ability: Manually designed feature extractors are often difficult to adapt to drawings of different styles and standards, resulting in poor robustness and generalization ability of the system;

[0008] 4. Human factors have a great influence: manually designing feature extractors to extract information based on professional knowledge is easily affected by human factors, increasing the possibility of errors;

[0009] 5. Lack of real-time performance: Due to the complex processing process, it is difficult to meet application scenarios that require real-time response.

[0010] To this end, the present invention provides a more efficient and more universal solution. Summary of the invention

[0011] The purpose of the present invention is to provide a 2D structural drawing information matching system and method based on deep learning, so as to solve the technical problem raised in the above background technology of how to improve the recognition and matching efficiency of 2D structural engineering drawing annotation information, while enhancing the versatility and robustness of the system.

[0012] In order to achieve the above object, the present invention provides the following technical solutions:

[0013] A 2D structural drawing information matching system based on deep learning, comprising:

[0014] The user interface module provides a drawing input interface, receives 2D structural engineering drawings uploaded by users, displays processing progress and intermediate results, displays the final generated 3D model, and provides result export function;

[0015] The target detection module is used to receive the 2D structural engineering drawings uploaded by the user interface module, identify the components and annotation information in the 2D structural engineering drawings using the YOLOv5 algorithm, and output data including component location and category information;

[0016] A text recognition module is used to receive the data of component position and category information output by the target detection module, recognize and analyze the annotation information using OCR technology, and output the recognized text content and corresponding position information;

[0017] An information matching module is used to receive the data of component position and category information output by the target detection module and the text content and position information output by the text recognition module, establish a drawing coordinate system, match the component with the annotation information, and output the matched component information;

[0018] A data storage module, used for receiving the matched component information output by the information matching module, converting the matching result into XML format and performing structured storage, and outputting a structured XML data file;

[0019] The three-dimensional model generation module is used to receive the structured XML data file output by the data storage module, generate a three-dimensional model according to the component information in the XML data file, and output the three-dimensional model file.

[0020] A 2D structural drawing information matching method based on deep learning, comprising:

[0021] S1, uploading a 2D structural engineering drawing through a user interface module, identifying component information and annotation information in the 2D structural engineering drawing through a target detection module, and obtaining a target set containing component information and annotation information calibrated by a bounding box;

[0022] S2, performing text recognition on the annotation information identified by the target detection module through the text recognition module, and parsing the annotation information according to the text category;

[0023] S3, establishing a drawing coordinate system based on the component information identified in S1 through an information matching module, matching the component with the annotation information, and obtaining the component coordinates and corresponding attribute information;

[0024] S4, sorting, organizing and storing all the information obtained in S1 to S3 in a data storage module; calling the information stored in the data storage module through a three-dimensional model generation module to generate a three-dimensional model, and outputting a three-dimensional model file.

[0025] Preferably, in S1, the target detection algorithm is a YOLOv5 target detection algorithm based on deep learning.

[0026] Preferably, in S1, the components include beams, columns and a grid.

[0027] Preferably, in S1, the annotation information includes drawing title, elevation, dimension, location and concentration.

[0028] Preferably, in S2, the text recognition algorithm adopts OCR text recognition, including PaddleOCR tool.

[0029] Preferably, S2 is specifically:

[0030] S21, cropping the annotation information using the bounding box obtained by the target detection algorithm;

[0031] S22, performing text recognition on the image content of the cropped area by using a text recognition algorithm to obtain corresponding text content;

[0032] S23, parsing according to the category of the annotation information.

[0033] Preferably, in S23, different annotation information categories have different parsing rules, and the specific parsing rules are: if the dimension annotation information in the drawing is digital, traditional OCR is directly used for text recognition; if the attribute annotation information in the drawing contains special characters, the position of the special characters is first identified, and the corresponding content behind them is parsed.

[0034] Preferably, S3 is specifically:

[0035] S31, establishing a drawing coordinate system according to the axis grid and calculating the unit pixel length;

[0036] S32, determining the position coordinates of the column in the drawing coordinate system;

[0037] S33, without considering the inclined beam, determining the position coordinates of the beam in the drawing coordinate system;

[0038] S34, determining the matching relationship between the position marking, the concentrated marking and the beam.

[0039] Preferably, S31 is specifically:

[0040] S311, dividing the axis grid into a horizontal axis grid and a vertical axis grid according to the position of the axis grid boundary box and sorting them. The side of the drawing includes an axis grid boundary box, and the axis grid boundary box is provided with a symbol as a position mark. The position mark parallel to the horizontal direction of the drawing is defined as the horizontal axis grid, and the position mark parallel to the vertical direction of the drawing is defined as the vertical axis grid; the horizontal axis grid is sorted from small to large according to the X value of the coordinate of the center point of the boundary box, and the vertical axis grid is sorted from large to small according to the Y value of the coordinate of the center point of the boundary box;

[0041] S312, using dimension matching rules to obtain corresponding dimension between two axis grids.

[0042] S313, taking the intersection of the lower leftmost horizontal axis grid and the vertical axis grid as the coordinate origin of the drawing coordinate system, and determining the coordinates of each axis grid and the coordinates of each axis grid intersection in combination with the dimensioning between the axis grids;

[0043] S314, the calculation rule of the unit pixel length is the average of the ratios of the pixel distances between all axis grids and the corresponding dimension annotation values.

[0044] Preferably, the dimensioning matching rules are as follows:

[0045] a. Position determination: If there is a dimension between two components, the boundary box of the dimension should be located on the center line of the horizontal or vertical distance between the two components;

[0046] b. Parameter setting: set the dimensioning deviation threshold for subsequent judgment;

[0047] c. Component direction determination: determine whether the two components are horizontal or vertical;

[0048] d. Determination of the bounding box range: For horizontal or vertical components, determine the range of the dimensioning bounding box: the maximum value of the vertical or horizontal position is defined as the upper limit, and the minimum value of the vertical or horizontal position is defined as the lower limit;

[0049] e. Candidate dimensioning screening: Calculate the coordinates of the midpoint of the vertical or horizontal distance between two components. If the vertical or horizontal coordinates of the midpoint fall within the following range, the corresponding dimensioning is considered as a candidate dimensioning: upper limit of the range = upper limit of dimensioning + dimensioning deviation threshold, lower limit of the range = lower limit of dimensioning - dimensioning deviation threshold;

[0050] f, Estimated length calculation: Calculate the vertical or horizontal pixel distance between two components, and use the known unit pixel length to convert the pixel distance into the estimated actual length;

[0051] g. Final dimensioning: For all candidate dimensioning, calculate the difference between the dimensioning length and the estimated length. Select the candidate with the smallest difference as the final dimensioning.

[0052] Preferably, S32 is specifically as follows: without considering the XY direction offset of the column, determine whether the column is located at the intersection of the axis grids; if the column is at the intersection of the two axis grids, that is, the intersection of the axis grids is within the boundary box of the column, then the coordinates of the axis grid intersection are the position coordinates of the column; if the column is not at the intersection of the axis grids, that is, all axis grid intersections are not within the boundary box of the column, then obtain the horizontal axis grid and vertical axis grid closest to the center point of the column boundary box, and then use the dimensioning matching rules to respectively obtain the dimensioning of the column and the horizontal axis grid and the vertical axis grid, and combine the axis grid coordinates and the dimensioning values ​​to obtain the position coordinates of the column.

[0053] Preferably, S33 is specifically:

[0054] S331, determining the horizontal or vertical position coordinates of the beam, dividing the beam into a horizontal beam and a vertical beam according to the ratio of the width to the height of the beam boundary box, wherein the start point and the end point of the horizontal beam have the same vertical position coordinate, and the start point and the end point of the vertical beam have the same horizontal position coordinate;

[0055] S332, according to the position of the beam on the drawing, the beam can be divided into an axis grid beam and a non-axis grid beam. The axis grid beam refers to a beam having an axis grid parallel to the beam and within the range of the beam boundary box, and the axis grid is the position axis grid corresponding to the beam. The non-axis grid beam refers to a beam having no axis grid parallel to the beam and within the range of the beam boundary box.

[0056] S333, for the horizontal or vertical coordinate of the axis grid beam, when the axis grid beam is a horizontal beam, the vertical coordinate of the axis grid beam is the position coordinate of the axis grid at the corresponding position; when the axis grid beam is a vertical beam, the horizontal position coordinate of the axis grid beam is the position coordinate of the axis grid at the corresponding position;

[0057] S334, for the horizontal or vertical coordinates of the non-axis grid beam, set a pixel length error threshold, traverse all the axis grids and axis grid beams parallel to the non-axis grid beam, use the dimension matching rule to obtain the dimension between the two, and obtain the horizontal or vertical coordinates of the non-axis grid beam in combination with the dimension, and finally obtain a series of candidate coordinates; calculate the horizontal or vertical pixel distance between the non-axis grid beam and the center point of the corresponding axis grid or axis grid beam boundary box, obtain the estimated horizontal or vertical coordinate through the unit pixel length, retain the candidate coordinate with the smallest difference between the estimated coordinate and the candidate coordinate, and the horizontal or vertical coordinate of the non-axis grid beam is the candidate coordinate.

[0058] S335, determine the position coordinates of the starting point and the end point of the beam. The two end points of the beam can only be on the beam or column. If the beam is a horizontal beam, the horizontal coordinates of the end points of the beam are the horizontal coordinate values ​​of the beam or column where the end points of the beam are located. If the beam is a vertical beam, the vertical coordinates of the end points of the beam are the vertical coordinate values ​​of the beam or column where the end points of the beam are located. Finally, the coordinates of the two end points of the beam can be determined by combining the horizontal coordinates and vertical coordinate values ​​of the beam.

[0059] Preferably, S34 is specifically:

[0060] S341, determining a horizontal beam group and a vertical beam group, setting a beam offset error threshold, and dividing the horizontal beams and the vertical beams into a plurality of horizontal beam groups and vertical beam groups, wherein each horizontal beam group satisfies that the difference in vertical coordinate values ​​of the center points of the bounding boxes of any two beams is less than or equal to the offset error threshold, and each vertical beam group satisfies that the difference in horizontal coordinate values ​​of the center points of the bounding boxes of any two beams is less than or equal to the offset error threshold;

[0061] S342, determining the concentrated annotation corresponding to the beam group, and setting the concentrated annotation distance error threshold, so that the concentrated annotation corresponding to the beam group satisfies that the concentrated annotation text direction is consistent with the direction of the beams in the beam group, and the difference between the horizontal or vertical distance between the two and the annotation distance error threshold is minimal;

[0062] S343, determine the position marking of the beam, the position marking of the beam will appear in four positions of the beam, specifically the upper left, upper middle, lower middle and upper right of the beam, and the upper left and upper right position markings are at the left and right ends of the beam, for this purpose, the beam is divided into 4 sections; set the position marking distance error threshold, for each beam, respectively find the position markings at the upper 1 / 4 of the left side of the beam, the upper and lower 1 / 2 of the middle, and the upper 1 / 4 of the right side, so that the text direction is consistent with the direction of the beam, and the difference between the horizontal or vertical distance between the two and the marking distance error threshold is the smallest.

[0063] Preferably, in S4, the component information is stored in a standardized manner in the data storage module in XML format.

[0064] Preferably, the component information is stored in XML-based structured information, and the component information includes attribute information and collection information, the attribute information includes a unique identifier, name, direction, secondary beam and associated attributes, and the associated attributes include type, name and ID; the collection information includes height, width, length and coordinates, the coordinates include coordinate ID, starting axis and ending axis, and the starting axis and ending axis include offset value and start value respectively.

[0065] Preferably, S4 is specifically:

[0066] S41, summarizing the axis grid information, including the unique identifier of the axis grid, the axis grid name, the direction, the corresponding dimension annotations between the axis grids, and the dimension annotation length;

[0067] S42, summarizing the column information, including the unique identifier of the column, the intersection coordinates of the column, the most adjacent axis grid of the column, and the dimensioning;

[0068] S43, summarize the beam information. When there is a conflict between the attributes in the concentrated marking and the position marking of the beam, the attributes in the position marking are given priority. The summarized information of the beam includes the unique identifier of the beam, the starting coordinates and end coordinates of the beam, the most adjacent axis grid and dimension marking information of the beam, the size and reinforcement information of the beam, etc.

[0069] A computer-readable storage medium includes instructions and data, which, when executed, enables a computer to execute the above-mentioned 2D structure drawing information matching method based on deep learning.

[0070] Compared with the prior art, the present invention has the following beneficial effects:

[0071] The present invention proposes a 2D structure drawing information matching system and method based on deep learning, which effectively solves the problems of low efficiency, limited applicability, poor robustness and generalization ability in the prior art, and improves the efficiency and accuracy of drawing information acquisition. The present invention uses the YOLOv5 target detection algorithm and OCR text recognition technology based on deep learning, without the need for cumbersome preliminary preprocessing operations, greatly improving the efficiency of drawing information acquisition; adopts a general image processing and recognition method, which can not only process DWG format drawings, but also PDF drawings and scanned drawings, expanding the scope of application; uses a deep learning model to automatically learn and adapt to drawings of different styles and standards, without a lot of manual processing, improving the robustness and generalization ability of the system; through automated target detection and text recognition algorithms, manual operations are reduced, reducing the possibility of human errors; using efficient target detection and text recognition algorithms, drawings can be quickly identified in a short time, suitable for application scenarios that require real-time response; standardized storage in XML format is convenient for subsequent processing and analysis, improving data availability and interoperability; BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 It is a flow chart of a preferred embodiment of the present invention;

[0073] Figure 2 This is a result diagram of target detection and recognition in a preferred embodiment of the present invention;

[0074] Figure 3 It is a three-dimensional model generated in Revit in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0075] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. All other embodiments obtained by ordinary technicians in the field without making creative work based on the embodiments of the present invention shall fall within the scope of protection of the present invention.

[0076] like Figure 1-Figure 3 As shown:

[0077] First preferred embodiment:

[0078] A 2D structural drawing information matching system based on deep learning, comprising:

[0079] The user interface module provides a drawing input interface, receives 2D structural engineering drawings uploaded by users, displays processing progress and intermediate results, displays the final generated 3D model, and provides result export function;

[0080] The target detection module is used to receive the 2D structural engineering drawings uploaded by the user interface module, identify the components and annotation information in the 2D structural engineering drawings using the YOLOv5 algorithm, and output data including component location and category information;

[0081] A text recognition module is used to receive the data of component position and category information output by the target detection module, recognize and analyze the annotation information using OCR technology, and output the recognized text content and corresponding position information;

[0082] An information matching module is used to receive the data of component position and category information output by the target detection module and the text content and position information output by the text recognition module, establish a drawing coordinate system, match the component with the annotation information, and output the matched component information;

[0083] A data storage module, used for receiving the matched component information output by the information matching module, converting the matching result into XML format and performing structured storage, and outputting a structured XML data file;

[0084] The three-dimensional model generation module is used to receive the structured XML data file output by the data storage module, generate a three-dimensional model according to the component information in the XML data file, and output the three-dimensional model file.

[0085] Second preferred embodiment:

[0086] A 2D structural drawing information matching method based on deep learning, comprising:

[0087] S1, uploading a 2D structural engineering drawing through a user interface module, identifying component information and annotation information in the 2D structural engineering drawing through a target detection module, and obtaining a target set containing component information and annotation information calibrated by a bounding box;

[0088] The target detection algorithm is a YOLOv5 target detection algorithm based on deep learning;

[0089] The components include beams, columns, and shaft grids, etc.;

[0090] The annotation information includes drawing title, elevation, dimension annotation, position annotation and centralized annotation, etc., so the output categories of the target detection algorithm are at least the above 8 categories.

[0091] S2, performing text recognition on the annotation information identified by the target detection module through the text recognition module, and parsing the annotation information according to the text category;

[0092] The text recognition algorithm uses OCR text recognition, including the PaddleOCR tool;

[0093] S21, using the bounding box obtained by the target detection algorithm to crop the annotation information,

[0094] S22, performing text recognition on the image content of the cropped area by using a text recognition algorithm to obtain corresponding text content;

[0095] S23, parsing is performed according to the category of the annotation information, and different categories of annotation information have different parsing rules.

[0096] The specific parsing rules are as follows: the dimension annotation information in the drawings is digital, so the traditional OCR can be used for text recognition directly; while the attribute annotation information in the drawings contains special characters, which represent different attributes, such as whether it is a span beam, steel bar model, etc. For these attribute annotation information, it is necessary to first identify the location of the special characters and parse the corresponding content behind them.

[0097] S3, establishing a drawing coordinate system based on the component information identified in S1 through an information matching module, matching the component with the annotation information, and obtaining the component coordinates and corresponding attribute information;

[0098] S31, establishing a drawing coordinate system according to the axis grid and calculating the unit pixel length;

[0099] S311, divide the grid into horizontal and vertical grids and sort them according to the position of the grid boundary box; the position of the grid boundary box is generally on the side of the drawing, represented by a circle with numbers or letters, such as: "①", "②", "③", and The horizontal grid is parallel to the horizontal direction of the drawing, and the vertical grid is parallel to the vertical direction of the drawing. The horizontal grid is sorted from small to large according to the X value of the center point coordinate of the bounding box, and the vertical grid is sorted from large to small according to the Y value of the center point coordinate of the bounding box;

[0100] S312, using dimension matching rules to obtain corresponding dimension between two axis grids;

[0101] The dimension matching rules are as follows:

[0102] a. Position determination: If there is a dimension between two components, the boundary box of the dimension should be located on the center line of the horizontal or vertical distance between the two components;

[0103] b. Parameter setting: set the dimensioning deviation threshold for subsequent judgment;

[0104] c. Component direction determination: determine whether the two components are horizontal or vertical;

[0105] d. Determination of the bounding box range: For horizontal or vertical components, determine the range of the dimensioning bounding box: the maximum value of the vertical or horizontal position is defined as the upper limit, and the minimum value of the vertical or horizontal position is defined as the lower limit;

[0106] e. Candidate dimensioning screening: Calculate the coordinates of the midpoint of the vertical or horizontal distance between two components. If the vertical or horizontal coordinates of the midpoint fall within the following range, the corresponding dimensioning is considered as a candidate dimensioning: upper limit of the range = upper limit of dimensioning + dimensioning deviation threshold, lower limit of the range = lower limit of dimensioning - dimensioning deviation threshold;

[0107] f, Estimated length calculation: Calculate the vertical or horizontal pixel distance between two components, and use the known unit pixel length to convert the pixel distance into the estimated actual length;

[0108] g. Final dimensioning: For all candidate dimensioning, calculate the difference between the dimensioning length and the estimated length. Select the candidate with the smallest difference as the final dimensioning;

[0109] S313, taking the intersection of the lower leftmost horizontal axis grid and the vertical axis grid as the coordinate origin of the drawing coordinate system, and determining the coordinates of each axis grid and the coordinates of each axis grid intersection in combination with the dimensioning between the axis grids;

[0110] S314, the calculation rule of the unit pixel length is the average of the ratios of the pixel distances between all axis grids and the corresponding dimension annotation values.

[0111] S32, determining the position coordinates of the column in the drawing coordinate system, and determining whether the column is located at the intersection of the axis grids without considering the offset of the column in the XY direction; if the column is at the intersection of the two axis grids, that is, the intersection of the axis grids is within the boundary box of the column, then the coordinates of the intersection of the axis grids are the position coordinates of the column; if the column is not at the intersection of the axis grids, that is, all the intersections of the axis grids are not within the boundary box of the column, then the horizontal axis grid and the vertical axis grid closest to the center point of the column boundary box are obtained, and then the dimensioning matching rules are used to obtain the dimensioning of the column and the horizontal axis grid and the vertical axis grid respectively, and the position coordinates of the column are obtained by combining the axis grid coordinates and the dimensioning values;

[0112] S33, without considering the inclined beam, determining the position coordinates of the beam in the drawing coordinate system, including the following steps:

[0113] S331, determining the horizontal or vertical position coordinates of the beam, dividing the beam into a horizontal beam and a vertical beam according to the ratio of the width to the height of the beam boundary box, wherein the start point and the end point of the horizontal beam have the same vertical position coordinate, and the start point and the end point of the vertical beam have the same horizontal position coordinate;

[0114] S332, according to the position of the beam on the drawing, the beam can be divided into an axis grid beam and a non-axis grid beam. The axis grid beam refers to a beam having an axis grid parallel to the beam and within the range of the beam boundary box, and the axis grid is the position axis grid corresponding to the beam. The non-axis grid beam refers to a beam having no axis grid parallel to the beam and within the range of the beam boundary box.

[0115] S333, for the horizontal or vertical coordinate of the axis grid beam, when the axis grid beam is a horizontal beam, the vertical coordinate of the axis grid beam is the position coordinate of the axis grid at the corresponding position; when the axis grid beam is a vertical beam, the horizontal position coordinate of the axis grid beam is the position coordinate of the axis grid at the corresponding position;

[0116] S334, for the horizontal or vertical coordinates of the non-axis grid beam, set a pixel length error threshold, traverse all the axis grids and axis grid beams parallel to the non-axis grid beam, use the dimension matching rule to obtain the dimension between the two, and obtain the horizontal or vertical coordinates of the non-axis grid beam in combination with the dimension, and finally obtain a series of candidate coordinates; calculate the horizontal or vertical pixel distance between the non-axis grid beam and the center point of the corresponding axis grid or axis grid beam boundary box, obtain the estimated horizontal or vertical coordinate through the unit pixel length, retain the candidate coordinate with the smallest difference between the estimated coordinate and the candidate coordinate, and the horizontal or vertical coordinate of the non-axis grid beam is the candidate coordinate.

[0117] S335, determine the position coordinates of the starting point and the end point of the beam. The two end points of the beam can only be on the beam or column. If the beam is a horizontal beam, the horizontal coordinates of the end points of the beam are the horizontal coordinate values ​​of the beam or column where the end points of the beam are located. If the beam is a vertical beam, the vertical coordinates of the end points of the beam are the vertical coordinate values ​​of the beam or column where the end points of the beam are located. Finally, the coordinates of the two end points of the beam can be determined by combining the horizontal coordinates and vertical coordinate values ​​of the beam.

[0118] S34, determining the matching relationship between the position mark, the concentrated mark and the beam, specifically:

[0119] S341, determining a horizontal beam group and a vertical beam group, setting a beam offset error threshold, and dividing the horizontal beams and the vertical beams into a plurality of horizontal beam groups and vertical beam groups, wherein each horizontal beam group satisfies that the difference in vertical coordinate values ​​of the center points of the bounding boxes of any two beams is less than or equal to the offset error threshold, and each vertical beam group satisfies that the difference in horizontal coordinate values ​​of the center points of the bounding boxes of any two beams is less than or equal to the offset error threshold;

[0120] S342, determining the concentrated annotation corresponding to the beam group, and setting the concentrated annotation distance error threshold, so that the concentrated annotation corresponding to the beam group satisfies that the concentrated annotation text direction is consistent with the direction of the beams in the beam group, and the difference between the horizontal or vertical distance between the two and the annotation distance error threshold is minimal;

[0121] S343, determine the position marking of the beam, the position marking of the beam will appear in four positions of the beam, specifically the upper left, upper middle, lower middle and upper right of the beam, and the upper left and upper right position markings are at the left and right ends of the beam, for this purpose, the beam is divided into 4 sections; set the position marking distance error threshold, for each beam, respectively find the position markings at the upper 1 / 4 of the left side of the beam, the upper and lower 1 / 2 of the middle, and the upper 1 / 4 of the right side, so that the text direction is consistent with the direction of the beam, and the difference between the horizontal or vertical distance between the two and the marking distance error threshold is the smallest.

[0122] S4, sorting, organizing and storing all the information obtained in S1 to S3 in a data storage module; calling the information stored in the data storage module through a three-dimensional model generation module to generate a three-dimensional model, and outputting a three-dimensional model file.

[0123] In S4, all information is stored in a standardized format in XML.

[0124] Specifically, the component information is stored in XML-based structured information, and the component information includes attribute information and collection information. The attribute information includes a unique identifier, name, direction, secondary beam, and associated attributes, and the associated attributes include type, name, and ID; the collection information includes height, width, length, and coordinates, and the coordinates include a coordinate ID, a starting axis, and an ending axis, and the starting axis and the ending axis include an offset value and a start value, respectively.

[0125] S41, summarizing the axis grid information, including the unique identifier of the axis grid, the axis grid name, the direction, the corresponding dimension annotations between the axis grids, and the dimension annotation length;

[0126] S42, summarizing the column information, including the unique identification of the column, the intersection coordinates of the column, the most adjacent axis grid of the column, and the dimensioning;

[0127] S43, summarize the beam information. When there is a conflict between the attributes in the concentrated marking and the position marking of the beam, the attributes in the position marking are given priority. The summarized information of the beam includes the unique identifier of the beam, the starting coordinates and end coordinates of the beam, the most adjacent axis grid and dimension marking information of the beam, the size and reinforcement information of the beam, etc.

[0128] Third preferred embodiment:

[0129] In this embodiment, a 2D structure drawing information matching method based on deep learning is proposed, including:

[0130] S1, using the target detection algorithm to identify the component information and annotation information in the structural engineering drawings, and obtain the target set containing the component information and annotation information marked by the bounding box

[0131] S2, performs text recognition on the annotation information identified by S1, and parses the annotation information according to the text category;

[0132] S3, establishing a drawing coordinate system based on the component information identified in S1, matching the component with the annotation information, and obtaining the component coordinates and corresponding attribute information;

[0133] S4, sorting, organizing and storing all the information obtained in S1 to S3.

[0134] In this embodiment, the target detection algorithm described in S1 is the YOLOv5 target detection algorithm based on deep learning. The recognition results include two types: components and annotation information. The components include beams, columns, and axis grids, and the annotation information includes drawing titles, dimension annotations, attribute annotations, etc. The specific recognition results are shown in Figure 2 .

[0135] In this embodiment, the text recognition described in S2 adopts the PaddleOCR text recognition algorithm. First, the annotation information is cropped according to the bounding box, and the image content of the cropped area is subjected to text recognition in combination with the text recognition algorithm to obtain the corresponding text content; then, the annotation information is parsed according to the category, and different annotation information categories have different parsing rules.

[0136] In this embodiment, the specific process of the matching rule between the component and the annotation information described in S3 includes:

[0137] S31, establish the drawing coordinate system according to the axis grid and calculate the unit pixel length, divide the axis grid into the horizontal axis grid and the vertical axis grid according to the position of the axis grid boundary box, and use the dimension matching rule to obtain the corresponding dimension between the two axis grids; calculate the unit pixel length, and the calculation rule is the average of the ratio of the pixel distance between all axis grids to the corresponding dimension value; the calculation logic of the unit pixel length is shown in Table 1:

[0138] Table 1 shows the calculation logic of unit pixel length

[0139]

[0140] The intersection of the lower left horizontal grid and the vertical grid is used as the origin of the coordinate system of the drawing. Combined with the dimensioning between the grids, the coordinates of each grid and the coordinates of each grid intersection are determined. The grid coordinates are shown in Table 2:

[0141] Table 2 is the axis grid coordinates

[0142]

[0143] S32, determining the position coordinates of the column in the drawing coordinate system. Without considering the XY direction offset of the column, the coordinate position of the column has the following two situations:

[0144] If the column is at the intersection of two axis grids, that is, the intersection of the axis grids is within the boundary box of the column, then the coordinates of the intersection of the axis grids are the position coordinates of the column;

[0145] If the column is not at the intersection of the axis grids, that is, all the intersection points of the axis grids are not within the boundary box of the column, then obtain the horizontal axis grid and the vertical axis grid closest to the center point of the column boundary box, and then use the dimensioning matching rules to obtain the dimensioning of the column and the horizontal axis grid and the vertical axis grid respectively, and combine the axis grid coordinates and the dimensioning values ​​to obtain the position coordinates of the column; the position coordinates of the column are shown in Table 3:

[0146] Table 3 shows the position coordinates of the columns

[0147]

[0148] S33, determining the position coordinates of the beam in the drawing coordinate system. Without considering the inclined beam, the calculation process of the position coordinates of the beam start point and the beam end point is:

[0149] First, the beams are divided into horizontal beams and vertical beams. The starting point and the end point of the horizontal beam have the same vertical position coordinates, and the starting point and the end point of the vertical beam have the same horizontal position coordinates.

[0150] Then, according to the position of the beam on the drawing, the beam can be divided into a grid beam and a non-grid beam. A grid beam refers to a beam with a grid parallel to the beam and within the range of the beam boundary box. The grid is the position grid corresponding to the beam. A non-grid beam refers to a beam with no grid parallel to the beam and within the range of the beam boundary box.

[0151] For the horizontal or vertical coordinates of the axis grid beam, when the axis grid beam is a horizontal beam, the vertical coordinate of the axis grid beam is the position coordinate of the axis grid at the corresponding position; when the axis grid beam is a vertical beam, the horizontal position coordinate of the axis grid beam is the position coordinate of the axis grid at the corresponding position;

[0152] For the horizontal or vertical coordinates of the non-axis grid beam, a pixel length error threshold is set, and all axis grids and axis grid beams parallel to the non-axis grid beam are traversed. The dimension annotation matching rule is used to obtain the dimension annotation between the two, and the horizontal or vertical coordinates of the non-axis grid beam are obtained in combination with the dimension annotation, and finally a series of candidate coordinates are obtained; the horizontal or vertical pixel distance between the non-axis grid beam and the center point of the corresponding axis grid or axis grid beam boundary box is calculated, and the estimated horizontal or vertical coordinate is obtained by the unit pixel length. The candidate coordinate with the smallest difference between the estimated coordinate and the candidate coordinate is retained, and the horizontal or vertical coordinate of the non-axis grid beam is the candidate coordinate.

[0153] Finally, determine the position coordinates of the starting point and the end point of the beam. The two end points of the beam can only be on the beam or column. If the beam is a horizontal beam, the horizontal coordinates of the end points of the beam are the horizontal coordinate values ​​of the beam or column where the end points of the beam are located. If the beam is a vertical beam, the vertical coordinates of the end points of the beam are the vertical coordinate values ​​of the beam or column where the end points of the beam are located. Finally, the coordinates of the two end points of the beam can be determined by combining the horizontal coordinates and vertical coordinates of the beam. The coordinates of the two end points of the beam are shown in Table 4:

[0154] Table 4 shows the coordinates of the two ends of the beam

[0155]

[0156]

[0157] S34, determining the matching relationship between the position mark, the concentrated mark and the beam, specifically including:

[0158] Determine a horizontal beam group and a vertical beam group, set a beam offset error threshold, and divide the horizontal beams and the vertical beams into a number of horizontal beam groups and vertical beam groups, wherein each horizontal beam group satisfies that the difference in vertical coordinate values ​​of the center points of the bounding boxes of any two beams is less than or equal to the offset error threshold, and each vertical beam group satisfies that the difference in horizontal coordinate values ​​of the center points of the bounding boxes of any two beams is less than or equal to the offset error threshold;

[0159] Determine the concentrated marking corresponding to the beam group, set the concentrated marking distance error threshold, and the concentrated marking corresponding to the beam group satisfies that the concentrated marking text direction is consistent with the direction of the beams in the beam group, and the difference between the horizontal or vertical distance between the two and the marking distance error threshold is the smallest; the concentrated marking corresponding to the beam group is shown in Table 5:

[0160] Table 5 shows the concentrated markings corresponding to the beam groups.

[0161] Beam Group Name member Centralized marking G1 L3 A1 G2 L4,L10 A2 G3 L5 A3 G4 L6,L2 A4 G5 L7,L11,L1 A5 G6 L16,L17 A6 G7 L8 A7 G8 L9 A8 G9 L12 A10 G10 L3,L14,L15 A9

[0162] Determine the position mark of the beam. The position mark of the beam will appear in four positions of the beam, specifically the upper left, upper middle, lower middle and upper right of the beam, and the position marks of the upper left and upper right are at the left and right ends of the beam. For this purpose, the beam is divided into 4 sections; set the position mark distance error threshold. For each beam, find the position mark at the upper 1 / 4 of the left side of the beam, the upper and lower 1 / 2 of the middle, and the upper 1 / 4 of the right side, respectively, so that the text direction is consistent with the direction of the beam, and the difference between the horizontal or vertical distance between the two and the mark distance error threshold is the smallest. The position mark of the beam is shown in Table 6:

[0163] Table 6 shows the beam position markings

[0164]

[0165]

[0166] In this embodiment, the structured information storage object described in S4 is stored in a standardized manner in XML format, specifically including:

[0167] Summarize the grid information, including the unique identification of the grid, the grid name, direction, the corresponding dimensioning between the grids, and the dimensioning length;

[0168] Summarize the column information, including the unique identification of the column, the intersection coordinates of the column, the most adjacent axis grid of the column and the dimensioning;

[0169] The beam information is summarized. When there is a conflict between the attributes in the concentrated marking and the position marking of the beam, the attributes in the position marking take precedence. The summarized information of the beam includes the unique identifier of the beam, the starting and ending coordinates of the beam, the beam's most adjacent axis grid and dimension marking information, the beam's size and reinforcement information, etc.

[0170] Fourth preferred embodiment:

[0171] A computer-readable storage medium includes instructions and data, which, when executed, enables a computer to execute the above-mentioned 2D structure drawing information matching method based on deep learning.

[0172] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When the use is implemented in whole or in part in the form of a computer program product, the computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL) or wireless (e.g., infrared, wireless, microwave, etc.) mode) to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk SolidState Disk (SSD)), etc.

[0173] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A 2D structure drawing information matching system based on deep learning, characterized in that: include: The user interface module provides a drawing input interface, receives 2D structural engineering drawings uploaded by users, displays processing progress and intermediate results, displays the final generated 3D model, and provides result export function; The target detection module is used to receive the 2D structural engineering drawings uploaded by the user interface module, identify the components and annotation information in the 2D structural engineering drawings using the YOLOv5 algorithm, and output data including component location and category information; A text recognition module is used to receive the data of component position and category information output by the target detection module, recognize and analyze the annotation information using OCR technology, and output the recognized text content and corresponding position information; An information matching module is used to receive the data of component position and category information output by the target detection module and the text content and position information output by the text recognition module, establish a drawing coordinate system, match the component with the annotation information, and output the matched component information; A data storage module, used for receiving the matched component information output by the information matching module, converting the matching result into XML format and performing structured storage, and outputting a structured XML data file; The three-dimensional model generation module is used to receive the structured XML data file output by the data storage module, generate a three-dimensional model according to the component information in the XML data file, and output the three-dimensional model file.

2. A 2D structure drawing information matching method based on deep learning, characterized in that: The following steps are involved: S1, uploading a 2D structural engineering drawing through a user interface module, identifying component information and annotation information in the 2D structural engineering drawing through a target detection module, and obtaining a target set containing component information and annotation information calibrated by a bounding box; S2, performing text recognition on the annotation information identified by the target detection module through the text recognition module, and parsing the annotation information according to the text category; S3, establishing a drawing coordinate system based on the component information identified in S1 through an information matching module, matching the component with the annotation information, and obtaining the component coordinates and corresponding attribute information; S4, collating, organizing and storing all the information obtained in S1 to S3 in a data storage module; The three-dimensional model generation module calls the information stored in the data storage module to generate a three-dimensional model and output a three-dimensional model file.

3. The 2D structure drawing information matching method based on deep learning according to claim 2 is characterized in that: In S1, the target detection algorithm is a YOLOv5 target detection algorithm based on deep learning.

4. The 2D structure drawing information matching method based on deep learning according to claim 2, characterized in that: In S1, the components include beams, columns and grids.

5. The 2D structure drawing information matching method based on deep learning according to claim 2, characterized in that: In S1, the annotation information includes drawing title, elevation, dimension annotation, position annotation and concentrated annotation.

6. The 2D structure drawing information matching method based on deep learning according to claim 2, characterized in that: In S2, the text recognition algorithm uses OCR text recognition, including the PaddleOCR tool.

7. The 2D structure drawing information matching method based on deep learning according to claim 2, characterized in that: S2 is specifically: S21, cropping the annotation information using the bounding box obtained by the target detection algorithm; S22, performing text recognition on the image content of the cropped area by using a text recognition algorithm to obtain corresponding text content; S23, parsing according to the category of the annotation information.

8. The 2D structure drawing information matching method based on deep learning according to claim 7, characterized in that: In S23, different annotation information categories have different parsing rules, and the specific parsing rules are as follows: if the dimension annotation information in the drawing is digital, traditional OCR is directly used for text recognition; if the attribute annotation information in the drawing contains special characters, the position of the special characters is first identified, and the corresponding content behind them is parsed.

9. The 2D structure drawing information matching method based on deep learning according to claim 2, characterized in that: S3 is specifically: S31, establishing a drawing coordinate system according to the axis grid and calculating the unit pixel length; S32, determining the position coordinates of the column in the drawing coordinate system; S33, without considering the inclined beam, determining the position coordinates of the beam in the drawing coordinate system; S34, determining the matching relationship between the position marking, the concentrated marking and the beam.

10. The 2D structure drawing information matching method based on deep learning according to claim 9, characterized in that: S31 is specifically: S311, dividing the axis grid into a horizontal axis grid and a vertical axis grid according to the position of the axis grid boundary box and sorting them; the side of the drawing includes an axis grid boundary box, and the axis grid boundary box is provided with a symbol as a position identifier, and the position identifier parallel to the horizontal direction of the drawing is defined as the horizontal axis grid, and the position identifier parallel to the vertical direction of the drawing is defined as the vertical axis grid; the horizontal axis grid is sorted from small to large according to the X value of the coordinate of the center point of the boundary box, and the vertical axis grid is sorted from large to small according to the Y value of the coordinate of the center point of the boundary box; S312, using dimension matching rules to obtain corresponding dimension between two axis grids; S313, taking the intersection of the lower leftmost horizontal axis grid and the vertical axis grid as the coordinate origin of the drawing coordinate system, and determining the coordinates of each axis grid and the coordinates of each axis grid intersection in combination with the dimensioning between the axis grids; S314, the calculation rule of the unit pixel length is the average of the ratios of the pixel distances between all axis grids and the corresponding dimension annotation values.

11. The 2D structure drawing information matching method based on deep learning according to claim 10, characterized in that: The dimensioning matching rules are as follows: a. Position determination: If there is a dimension between two components, the boundary box of the dimension should be located on the center line of the horizontal or vertical distance between the two components; b. Parameter setting: set the dimensioning deviation threshold for subsequent judgment; c. Component direction determination: determine whether the two components are horizontal or vertical; d. Determination of the bounding box range: For horizontal or vertical components, determine the range of the dimensioning bounding box: the maximum value of the vertical or horizontal position is defined as the upper limit, and the minimum value of the vertical or horizontal position is defined as the lower limit; e. Candidate dimensioning screening: Calculate the coordinates of the midpoint of the vertical or horizontal distance between two components; if the vertical or horizontal coordinates of the midpoint fall within the following range, the corresponding dimensioning is considered as a candidate dimensioning: upper limit of the range = upper limit of dimensioning + dimensioning deviation threshold, lower limit of the range = lower limit of dimensioning - dimensioning deviation threshold; f, Estimated length calculation: Calculate the vertical or horizontal pixel distance between two components, and use the known unit pixel length to convert the pixel distance into the estimated actual length; g. Final dimensioning determination: For all candidate dimensionings, calculate the difference between their dimensioning length and the estimated length; select the candidate with the smallest difference as the final dimensioning.

12. The 2D structure drawing information matching method based on deep learning according to claim 9, characterized in that: S32 is specifically as follows: without considering the XY direction offset of the column, determine whether the column is located at the intersection of the axis grids; if the column is at the intersection of the two axis grids, that is, the intersection of the axis grids is within the boundary box of the column, then the coordinates of the intersection of the axis grids are the position coordinates of the column; if the column is not at the intersection of the axis grids, that is, all the intersections of the axis grids are not within the boundary box of the column, then obtain the horizontal axis grid and vertical axis grid closest to the center point of the column boundary box, and then use the dimensioning matching rules to respectively obtain the dimensioning of the column and the horizontal axis grid and the vertical axis grid, and combine the axis grid coordinates with the dimensioning values ​​to obtain the position coordinates of the column.

13. The 2D structure drawing information matching method based on deep learning according to claim 9, characterized in that: S33 is specifically: S331, determining the horizontal or vertical position coordinates of the beam, dividing the beam into a horizontal beam and a vertical beam according to the ratio of the width to the height of the beam boundary box, wherein the start point and the end point of the horizontal beam have the same vertical position coordinate, and the start point and the end point of the vertical beam have the same horizontal position coordinate; S332, according to the position of the beam on the drawing, the beam is divided into an axis grid beam and a non-axis grid beam, the axis grid beam refers to a beam having an axis grid parallel to the beam and within the range of the beam boundary box, the axis grid is the position axis grid corresponding to the beam, and the non-axis grid beam refers to a beam having no axis grid parallel to the beam and within the range of the beam boundary box; S333, for the horizontal or vertical coordinate of the axis grid beam, when the axis grid beam is a horizontal beam, the vertical coordinate of the axis grid beam is the position coordinate of the axis grid at the corresponding position; when the axis grid beam is a vertical beam, the horizontal position coordinate of the axis grid beam is the position coordinate of the axis grid at the corresponding position; S334, for the horizontal or vertical coordinate of the non-axis grid beam, set a pixel length error threshold, traverse all axis grids and axis grid beams parallel to the non-axis grid beam, use the dimensioning matching rule to obtain the dimensioning between the two, and obtain the horizontal or vertical coordinate of the non-axis grid beam in combination with the dimensioning, and finally obtain a series of candidate coordinates; calculate the horizontal or vertical pixel distance between the non-axis grid beam and the center point of the corresponding axis grid or axis grid beam boundary box, obtain the estimated horizontal or vertical coordinate by the unit pixel length, retain the candidate coordinate with the smallest difference between the estimated coordinate and the candidate coordinate, and the horizontal or vertical coordinate of the non-axis grid beam is the candidate coordinate; S335, determine the position coordinates of the starting point and the end point of the beam. The two end points of the beam are on the beam or column. If the beam is a horizontal beam, the horizontal coordinates of the end points of the beam are the horizontal coordinate values ​​of the beam or column where the end points of the beam are located. If the beam is a vertical beam, the vertical coordinates of the end points of the beam are the vertical coordinate values ​​of the beam or column where the end points of the beam are located. Finally, the coordinates of the two end points of the beam are determined by combining the horizontal coordinates and vertical coordinate values ​​of the beam.

14. The 2D structure drawing information matching method based on deep learning according to claim 9, characterized in that: S34 is specifically: S341, determining a horizontal beam group and a vertical beam group, setting a beam offset error threshold, and dividing the horizontal beams and the vertical beams into a plurality of horizontal beam groups and vertical beam groups, wherein each horizontal beam group satisfies that the difference in vertical coordinate values ​​of the center points of the bounding boxes of any two beams is less than or equal to the offset error threshold, and each vertical beam group satisfies that the difference in horizontal coordinate values ​​of the center points of the bounding boxes of any two beams is less than or equal to the offset error threshold; S342, determining the concentrated annotation corresponding to the beam group, and setting the concentrated annotation distance error threshold, so that the concentrated annotation corresponding to the beam group satisfies that the concentrated annotation text direction is consistent with the direction of the beams in the beam group, and the difference between the horizontal or vertical distance between the two and the annotation distance error threshold is minimal; S343, determine the position marking of the beam, the position marking of the beam will appear in four positions of the beam, specifically the upper left, upper middle, lower middle and upper right of the beam, and the upper left and upper right position markings are at the left and right ends of the beam, for this purpose, the beam is divided into 4 sections; set the position marking distance error threshold, for each beam, respectively find the position markings at the upper 1 / 4 of the left side of the beam, the upper and lower 1 / 2 of the middle, and the upper 1 / 4 of the right side, so that the text direction is consistent with the direction of the beam, and the difference between the horizontal or vertical distance between the two and the marking distance error threshold is the smallest.

15. The 2D structure drawing information matching method based on deep learning according to claim 2, characterized in that: In S4, component information is stored in a standardized manner in the data storage module in XML format.

16. The 2D structure drawing information matching method based on deep learning according to claim 15, characterized in that: The component information is stored in XML-based structured information, and includes attribute information and collection information. The attribute information includes a unique identifier, name, direction, secondary beam, and associated attributes, and the associated attributes include type, name, and ID; the collection information includes height, width, length, and coordinates, and the coordinates include a coordinate ID, a starting axis, and an ending axis, and the starting axis and the ending axis include an offset value and a start value, respectively.

17. The 2D structure drawing information matching method based on deep learning according to claim 2, characterized in that: S4 is specifically: S41, summarizing the axis grid information, including the unique identifier of the axis grid, the axis grid name, the direction, the corresponding dimension annotations between the axis grids, and the dimension annotation length; S42, summarizing the column information, including the unique identifier of the column, the intersection coordinates of the column, the most adjacent axis grid of the column, and the dimensioning; S43, summarize the beam information. When there is a conflict between the attributes in the concentrated marking and the position marking of the beam, the attributes in the position marking are given priority. The summarized information of the beam includes the unique identifier of the beam, the starting coordinates and end coordinates of the beam, the most adjacent axis grid and dimension marking information of the beam, and the size and reinforcement information of the beam.

18. A computer-readable storage medium, characterized in that: It includes instructions and data, which, when executed, enable a computer to execute the 2D structure drawing information matching method based on deep learning as described in claims 2 to 17.

Images