Engineering drawing intelligent identification method and system based on deep learning

Abstract

The invention relates to the technical field of drawing recognition, and discloses an engineering drawing intelligent recognition method and system based on deep learning. The method comprises the steps of performing multi-target cooperative detection on a first processing image based on a detection model, identifying primitive information of the first processing image, and generating a second identification image; positioning a text area of the second recognition image, recognizing a character detection range, determining word tags represented by the character detection range, summarizing character information of the character detection range based on the word tags, and generating third image data; obtaining a correlation degree among the symbols, the attributes and the connecting line information, detecting whether the primitive information accords with a preset rule or not, constructing a symbol topological relation graph, and generating correction information containing a conflict position; and fusing the primitive information, the character information and the correction information to generate structured data comprising a symbol hierarchy tree, an attribute incidence matrix and a conflict label. According to the invention, the efficiency and accuracy of engineering drawing intelligent identification are improved.

Description

Technical Field

[0001] The present application relates to the technical field of drawing recognition, and in particular to a method and system for intelligent recognition of engineering drawings based on deep learning. Background Art

[0002] In fields such as engineering construction, architectural design, and mechanical manufacturing, engineering drawings are crucial documents for communicating design intent and guiding construction and production. However, traditional manual drawing recognition methods suffer from inefficiency, error-proneness, and difficulty processing large quantities of drawings. With the advancement of digital transformation, the digital storage and management of engineering drawings is becoming increasingly common, but how to efficiently and accurately extract key information from these massive volumes of drawings remains a pressing challenge.

[0003] Similar prior art includes a Chinese patent application with publication number CN118629066A, which discloses a deep learning-based engineering drawing recognition and automatic inspection system, the system including a complete condition limitation module, which is used to determine the integrity condition corresponding to the drawing based on the drawing information; a drawing integrity inspection module, which determines whether the drawing is complete based on the integrity condition corresponding to the drawing; a verification condition limitation module, which is used to determine the corresponding verification condition based on the drawing information; a drawing association module, which is used to determine the associated drawings of the engineering drawing based on the drawing information, and determine the association level of the associated drawings, and also to determine the consistency content of the engineering drawing based on the association level; a drawing verification module, which is used to determine whether the consistency content is used to supplement the drawing content based on the integrity condition determination result of the drawing to be verified; and the accuracy of engineering drawing recognition and inspection is improved by improving the accuracy of drawing information analysis. There is also a Chinese patent application with publication number CN111079528A, which discloses a method and system for verifying graphic elements and drawings based on deep learning, including: obtaining a graphic element drawing image file to be verified; cutting the image file to be verified into a schematic diagram and text information part according to proportion and recording the corresponding information; inputting the text information image part into a pre-built text recognition model to obtain the detected text information; using image processing methods to verify whether the corresponding schematic diagram is reasonable based on the text information, and outputting the result information.

[0004] The shortcomings of existing technologies are primarily reflected in the low recognition rates of deep learning models for complex scenarios such as chemical and architectural engineering drawings, including non-standard symbols, dense layouts, and degraded drawings. These models also lack the ability to extract fine-grained features of multi-level structural details. Furthermore, traditional image processing methods, when verifying schematics, may fail to identify fuzzy, degraded, or unconventional symbols. In practical situations, it is necessary to automatically and efficiently identify various elements in engineering drawings and extract their semantic information to accurately and rapidly retrieve, classify, and analyze drawings. Summary of the Invention

[0005] This application provides a method and system for intelligent recognition of engineering drawings based on deep learning, which is used to improve the efficiency and accuracy of intelligent recognition of engineering drawings.

[0006] In a first aspect, the present application provides a method for intelligently identifying engineering drawings based on deep learning, the method comprising:

[0007] Acquire an engineering drawing image, perform adaptive illumination compensation preprocessing on the engineering drawing image to generate a first processed image, construct a detection model, perform multi-target collaborative detection on the first processed image based on the detection model, identify primitive information of the first processed image, and generate a second recognized image;

[0008] Locating a text region of the second recognized image based on the primitive information, identifying a text detection range from the text region, determining a word label represented by the text detection range, and summarizing character information of the text detection range based on the word label to generate third image data;

[0009] Summarizing the symbols, attributes, and connection line information contained in the second recognized image and the third image data based on pixel coordinates, obtaining the degree of association between the symbols, the attributes, and the connection line information, detecting whether the graphic element information complies with a preset rule based on the degree of association, and if not, constructing a symbol topology relationship map based on a graph neural network, and generating correction information including conflicting positions based on the symbol topology relationship map;

[0010] The graphic element information, the character information, and the correction information are integrated to generate structured data including a symbol hierarchy tree, an attribute association matrix, and conflict annotations.

[0011] In combination with the first aspect, the construction of the detection model includes:

[0012] Collect multiple engineering drawings based on drawing types and storage categories, and mark the categories of graphic elements contained in the engineering drawings;

[0013] performing an embedding operation on the preprocessed engineering drawings to generate scene vectors, performing cluster analysis on all the scene vectors to generate a plurality of scene clusters, and generating at least one individual type for any engineering drawing based on the scene clusters, wherein the individual type represents a primitive distribution feature;

[0014] Performing object detection on any of the engineering drawings to extract a plurality of graphic elements, cropping pixel regions contained in the graphic elements, and generating a first cropped image;

[0015] performing an embedding operation on the first cropped image to generate an object vector, clustering all the object vectors to generate a plurality of object clusters, and generating an individual type corresponding to the graphic element for any of the engineering drawings based on the object clusters;

[0016] Matching any of the engineering drawings with the generated individual types to generate a bipartite graph;

[0017] Checking the number of individual types that match any of the engineering drawings based on the bipartite graph, setting the engineering drawing with the largest number of matches as a first sampled image, checking the number of remaining individual types that match any of the engineering drawings, setting the engineering drawing with the largest remaining number of matches as a second sampled image, and obtaining sampled images that match all of the individual types based on the same sampling steps;

[0018] A training image set is selected from all the selected images based on the optimization algorithm, a deep learning model is constructed using the YOLO algorithm, the labeled training image set is input into the deep learning model for training, the parameters of the deep learning model are optimized, and the optimized deep learning model is set as the detection model.

[0019] In combination with the first aspect, generating the second identification image includes:

[0020] Inputting the first processed image into the detection model, identifying a plurality of target results in the first processed image, the target results including category information, location information, and a first confidence level, screening the target results based on a confidence level threshold, and setting the screened target results as the primitive information;

[0021] Determining whether the bounding boxes corresponding to the primitive information overlap based on the position information, and if so, extracting an optimal bounding box using a non-maximum suppression algorithm, and updating the position information based on the optimal bounding box;

[0022] The primitive information is marked in the first processed image to generate the second recognized image.

[0023] In combination with the first aspect, identifying a text detection range from the text area includes:

[0024] Acquire a boundary line of the text area, detect intersections of the boundary line and character images in the text area, and store pixel coordinates of all the intersections in a coordinate list;

[0025] Initialize the drawing range, set the upper and lower ends of the drawing range to the Y coordinate value of the first element in the coordinate list, traverse each element in the coordinate list, start from the pixel coordinate of the current intersection, obtain the area where the character drawing points are continuous, and detect the minimum and maximum Y coordinate values;

[0026] If the minimum Y coordinate value is less than the Y coordinate value corresponding to the upper end of the drawing range, the upper end of the drawing range is updated to the minimum Y coordinate value; if the maximum Y coordinate value is greater than the Y coordinate value corresponding to the lower end of the drawing range, the lower end of the drawing range is updated to the maximum Y coordinate value;

[0027] Detecting the minimum X coordinate value and the maximum X coordinate value of the boundary line, and setting them as the left end and the right end of the drawing range respectively;

[0028] The text detection range is generated based on the upper end, lower end, left end, and right end of the drawing range.

[0029] In conjunction with the first aspect, determining the word label represented by the text detection range includes:

[0030] Summarizing all image data within the text detection range and setting them as an image dataset, building a word recognition model based on a convolutional neural network, and outputting the word label and second confidence level contained in any image data after training the word recognition model on the image dataset;

[0031] Based on the second confidence level, determine whether the word label is correct. If not, decompose any of the image data into character image data, use the word recognition model to perform single-character recognition on the character image data, output a single character, and modify the word label based on the single character.

[0032] In conjunction with the first aspect, obtaining the degree of association between the symbol, the attribute, and the connection line information includes:

[0033] calculating a first distance between the symbol and the attribute based on the pixel coordinates, and determining that the symbol is associated with the attribute if the first distance is less than a first preset value;

[0034] calculating a second distance between the symbol and the connection line information, and if the second distance is less than a second preset value, determining that the symbol is associated with the connection line information;

[0035] If two symbols are connected by the connection line information, the two symbols are determined to be associated, and based on the recognition result of the connection line information, the symbols corresponding to the starting point and the end point of the connection line information are marked respectively;

[0036] The association number between any symbol and the attribute, the connection line information, and other symbols is summarized to generate the association degree.

[0037] In combination with the first aspect, generating correction information including conflicting positions based on the symbol topology relationship map includes:

[0038] connecting all the symbols based on the association degree vector to generate a vector drawing, and if the vector drawing does not conform to the preset rule, determining that the graphic element information does not conform to the preset rule;

[0039] The graph neural network sets the symbols as nodes, sets the connection line information as edges, summarizes the nodes and edges in the vector drawing, and generates the symbol topology relationship map;

[0040] Converting the preset rules into graph constraints that can be learned by the graph neural network, inputting the symbolic topological relationship graph into the graph neural network, clustering the output node embeddings, setting the nodes corresponding to the discrete points as conflict points, setting the edges connected to the conflict points as conflict edges, summarizing the conflict points and the conflict edges, and setting them as the conflict positions;

[0041] The conflict position is corrected based on the detection model to generate the correction information.

[0042] In conjunction with the first aspect, generating structured data including a symbol hierarchy tree, an attribute association matrix, and conflict annotations includes:

[0043] generating the symbol hierarchy tree based on the traversal connection relationship of the symbols in the vector drawing;

[0044] The attribute association matrix is ​​generated by setting the serial numbers corresponding to the symbols as rows, the attributes as columns, and the associated quantities as values;

[0045] Setting the correction information corresponding to the conflict position as the conflict mark;

[0046] The symbol hierarchy tree, the attribute association matrix, and the conflict annotations are aggregated based on a spatial dimension to generate the structured data.

[0047] In a second aspect, the present application provides an engineering drawing intelligent recognition system based on deep learning, the engineering drawing intelligent recognition system based on deep learning comprising:

[0048] a detection module configured to acquire an engineering drawing image, perform adaptive illumination compensation preprocessing on the engineering drawing image to generate a first processed image, construct a detection model, perform multi-target collaborative detection on the first processed image based on the detection model, identify primitive information of the first processed image, and generate a second recognized image;

[0049] an extraction module configured to locate a text region of the second recognition image based on the primitive information, identify a text detection range from the text region, determine a word label represented by the text detection range, summarize character information of the text detection range based on the word label, and generate third image data;

[0050] a judgment module, configured to summarize the symbols, attributes, and connection line information contained in the second recognized image and the third image data based on pixel coordinates, obtain a degree of association between the symbols, the attributes, and the connection line information, detect whether the graphic element information complies with a preset rule based on the degree of association, and if not, construct a symbol topology relationship map based on a graph neural network, and generate correction information including conflicting locations based on the symbol topology relationship map;

[0051] A fusion module is used to fuse the graphic element information, the character information, and the correction information to generate structured data including a symbol hierarchy tree, an attribute association matrix, and conflict annotations.

[0052] The technical solution provided in this application first constructs a detection model based on deep learning and graph neural networks to identify primitive information such as symbols, attributes, and connecting line information in engineering drawings. An optimization algorithm is used to select a training image set, and a non-maximum suppression algorithm is used to address the problem of bounding box overlap to improve detection accuracy. The degree of correlation between symbols and attributes and connecting line information is calculated based on pixel coordinates. Graph constraints are also used to detect whether primitive information conforms to preset rules. This allows efficient and accurate identification of primitive information in engineering drawings, significantly improving recognition accuracy and efficiency. Multi-target collaborative detection and association analysis can simultaneously process multiple primitives, avoiding the inefficiency of one-by-one detection in traditional methods. Then, by detecting the intersection of the boundary line of the text area and the character image, a text detection range is dynamically generated. A convolutional neural network is used to construct a word recognition model. Image data within the text detection range is trained, word labels and confidence scores are output, and erroneous word labels are corrected through single-character recognition. This effectively handles text recognition problems in complex backgrounds, improves the accuracy and robustness of text recognition, and the single-character recognition correction mechanism further improves the accuracy of word recognition and reduces errors caused by ambiguity or noise. Finally, a symbol topology relationship map is generated based on the degree of association, the preset rules are converted into graph constraints, and the conflict points and conflict edges are detected through the graph neural network. Correction information containing the conflict positions is generated, a symbol hierarchy tree is generated, the hierarchical relationship between symbols is displayed, an attribute association matrix is ​​constructed, the number of associations between symbols and attributes is recorded, conflict annotations are generated, conflict positions and correction suggestions are marked, and the data are summarized into structured data. It can intelligently detect whether the graphic element information complies with the preset rules, quickly locate conflict points and conflict edges, and provide a basis for the digital management and intelligent application of engineering drawings, realizing the intelligent recognition, conflict detection and structured data generation of engineering drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0054] Figure 1 This is a schematic diagram of an embodiment of a method for intelligent recognition of engineering drawings based on deep learning in an embodiment of the present application;

[0055] Figure 2 This is a schematic diagram of an embodiment of the detection model construction and training process in the embodiment of the present application;

[0056] Figure 3 A schematic diagram of an embodiment of text region recognition and word label generation in an embodiment of the present application;

[0057] Figure 4 This is a schematic diagram of an embodiment of an intelligent recognition system for engineering drawings based on deep learning in an embodiment of the present application. DETAILED DESCRIPTION

[0058] The embodiments of the present application provide a method and system for intelligent recognition of engineering drawings based on deep learning. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0059] For ease of understanding, the specific process of the embodiment of the present application is described below. Figure 1 In the embodiments of the present application, an embodiment of the method for intelligently identifying engineering drawings based on deep learning includes:

[0060] Step S101: Acquire an engineering drawing image, perform adaptive illumination compensation preprocessing on the engineering drawing image to generate a first processed image, construct a detection model, perform multi-target collaborative detection on the first processed image based on the detection model, identify the primitive information of the first processed image, and generate a second recognized image.

[0061] It is understandable that the execution subject of this application can be a deep learning-based engineering drawing intelligent recognition device, or a terminal or server, which is not limited here. The embodiment of this application is described by taking the server as the execution subject as an example.

[0062] Specifically, a digital image of an engineering drawing is acquired using a scanner, camera, or other image acquisition device. The image may contain elements such as lines, symbols, and text. An adaptive illumination compensation algorithm is used to process the image. This algorithm analyzes the illumination distribution in the image and automatically adjusts the brightness and contrast to eliminate the effects of uneven illumination. For example, techniques such as histogram equalization can be used. The preprocessed engineering drawing image is then set as the first processed image. A detection model is constructed using a deep learning framework. This model can be a pretrained general object detection algorithm, such as YOLO+CNN (multi-object detection + text region localization), fine-tuned to the characteristics of engineering drawing images. The preprocessed first processed image is input into the detection model, which simultaneously detects multiple objects in the first processed image, including primitives such as lines, symbols, and text. Through multi-object collaborative detection, the location and category of each primitive in the image can be accurately identified, generating a second recognized image. The primitive information includes the category, location, and recognition confidence of each primitive, such as the length, angle, and pixel coordinates of a line, and the shape, size, and pixel coordinates of a symbol.

[0063] Step S102: Locate the text area of ​​the second recognition image based on the primitive information, identify the text detection range from the text area, determine the word label represented by the text detection range, summarize the character information of the text detection range based on the word label, and generate third image data.

[0064] Specifically, based on primitive information (e.g., the bounding box of a text box), the text region is located in the second recognition image. The text detection range refers to the pixel region containing text characters. Word labels refer to the textual content and semantic content of words within the text detection range. Character information refers to the attribute information corresponding to the word label, including character attributes such as font, size, and color. This character information is integrated into the third image data.

[0065] Step S103: Summarize the symbols, attributes and connection line information contained in the second recognition image and the third image data based on the pixel coordinates, obtain the degree of association between the symbols, attributes and connection line information, and detect whether the graphic element information complies with the preset rules based on the degree of association. If not, construct a symbol topology relationship map based on the graph neural network, and generate correction information containing conflicting positions based on the symbol topology relationship map.

[0066] Specifically, the pixel coordinate information of symbols, attributes and connecting lines in the second recognition image and the third image data is extracted, and the coordinate information corresponding to these pixel coordinates is used to determine the spatial relationship between symbols, attributes and connecting lines. By calculating the geometric features such as the distance and angle between symbols, attributes and connecting lines, the degree of association between them is evaluated. For example, if the two ends of a connecting line are connected to two symbols respectively, it is considered that there is an association between them. Based on preset rules (for example, connection rules between symbols, attribute labeling rules, etc.), it is detected whether the graphic element information meets the requirements. If it is found that the rules are not met (for example, the connecting line incorrectly connects the symbol, the attribute labeling is incorrect, etc.), a graph neural network (GNN) is used to construct a symbol topological relationship map. GNN can learn the topological structure between symbols and generate correction information containing conflicting positions.

[0067] Step S104 : Fusing the primitive information, character information, and correction information to generate structured data including a symbol hierarchy tree, an attribute association matrix, and conflict annotations.

[0068] Specifically, primitive information includes the geometric features and classification information of symbols, lines, and connectors; character information includes text content and formatting; and correction information includes conflict locations and repair suggestions. This fused information is organized into structured data, including a symbol hierarchy tree (representing the hierarchical relationships between symbols), an attribute association matrix (representing the relationships between symbols and attributes), and conflict annotations (marking locations that violate the rules and providing repair suggestions). This structured data enables rapid retrieval, classification, and analysis of engineering drawings, improving the efficiency and accuracy of engineering drawing processing, reducing labor costs, and enhancing the intelligence of engineering management.

[0069] In a specific embodiment, constructing a detection model includes:

[0070] (1) Collect multiple engineering drawings based on drawing type and storage category, and mark the categories of elements contained in the engineering drawings.

[0071] (2) Perform embedding operations on the preprocessed engineering drawings to generate scene vectors, perform cluster analysis on all scene vectors to generate multiple scene clusters, and generate at least one individual type for any engineering drawing based on the scene clusters, where the individual type represents the distribution characteristics of the graphics elements.

[0072] (3) Performing object detection on any engineering drawing to extract multiple primitives, cropping the pixel areas contained in the primitives, and generating a first cropped image.

[0073] (4) Performing an embedding operation on the first cropped image to generate object vectors, clustering all the object vectors to generate multiple object clusters, and generating individual types corresponding to the primitives for any engineering drawing based on the object clusters.

[0074] (5) Match any engineering drawing with the generated individual type to generate a bipartite graph.

[0075] (6) Based on the bipartite graph, check the number of individual types that match any engineering drawing, set the engineering drawing with the largest number of matches as the first sampled image, check the number of remaining individual types that match any engineering drawing, set the engineering drawing with the largest remaining number of matches as the second sampled image, and obtain the sampled images that match all individual types based on the same sampling steps.

[0076] (7) Based on the optimization algorithm, a training image set is selected from all the selected images, a deep learning model is constructed using the YOLO algorithm, the labeled training image set is input into the deep learning model for training, the parameters of the deep learning model are optimized, and the optimized deep learning model is set as the detection model.

[0077] Specifically, Figure 2 Create a flow chart for building and training the detection model. Collect a large number of engineering drawings as image samples based on the drawing type (e.g., architectural drawings, electrical drawings, mechanical drawings, etc.) and storage category (e.g., paper scanned drawings, electronic drawings, etc.). These engineering drawings should cover a variety of common graphic element categories, such as lines, symbols, text, graphics, etc. Label the collected engineering drawings to clarify the category of each graphic element in the drawing. For example, label lines as "straight line", "arc", etc., label symbols as "resistor", "valve", etc., and label text as "title", "note", etc. Labeling can be done using manual labeling tools, or with the help of semi-automatic labeling tools to improve efficiency.

[0078] An embedding operation is performed on the preprocessed engineering drawings to convert each drawing into a high-dimensional scene vector. The embedding operation can be implemented using a deep learning model (such as a convolutional neural network), which extracts the overall characteristics of the drawing, including the distribution and layout of the graphics elements. The scene vector is a high-dimensional feature vector used to represent image content and can capture the key features and semantic information of engineering drawings. Clustering algorithms such as K-Means can be used to ensure that the engineering drawings within each scene cluster have similar graphics element distribution characteristics. Individual types represent the graphics element distribution characteristics of the drawings. For example, the engineering drawings within a certain scene cluster may be dominated by electrical symbols, while the engineering drawings within another scene cluster may be dominated by architectural lines.

[0079] Perform object detection on each engineering drawing, extract multiple primitives from the drawing, use a target detection algorithm (such as YOLO) to locate the position of the primitives, and crop the pixel area contained in the primitives to generate a first cropped image.

[0080] An embedding operation is performed on the first cropped image to generate an object vector. The object vector represents the characteristics of each primitive in the first cropped image, including information such as shape, texture, and color. After clustering, primitives within each object cluster have similar characteristics. Based on the object clusters, an individual type is generated for each primitive, representing the characteristic category of the primitive.

[0081] Match each engineering drawing with the generated individual type and construct a bipartite graph, where the left node of the bipartite graph represents the engineering drawing, the right node represents the individual type, and the edge weight represents the degree of matching.

[0082] Based on the bipartite graph, the number of individual types that match each engineering drawing is checked, and the engineering drawing with the largest number of matches is set as the first selected image. This ensures that each individual type has a corresponding matching selected image.

[0083] The goal of the optimization algorithm is to minimize the total number of selected images while ensuring that each individual type is covered at least once. This optimization algorithm, such as linear programming or integer programming, is used. YOLO is an efficient, real-time object detection algorithm suitable for detecting primitives in engineering drawings. The performance of the deep learning model is optimized by adjusting parameters such as the learning rate, optimizer, and loss function. After training is complete, the optimized deep learning model is set as the detection model.

[0084] In a specific embodiment, generating the second identification image includes:

[0085] (1) Inputting the first processed image into the detection model, identifying multiple target results in the first processed image, the target results including category information, location information and a first confidence level, filtering the target results based on a confidence level threshold, and setting the filtered target results as primitive information.

[0086] (2) Based on the position information, determine whether the bounding boxes corresponding to the primitive information overlap. If they overlap, use the non-maximum suppression algorithm to extract the best bounding box, and update the position information based on the best bounding box.

[0087] (3) Marking primitive information in the first processed image to generate a second recognized image.

[0088] Specifically, each target result includes category information (for example, "line", "circle", "text", etc.), location information (usually expressed in the form of a bounding box, including the coordinates of the upper left corner, width, and height of the bounding box), and a first confidence level (indicating the detection model's confidence in the detection result, usually a value between 0 and 1). A confidence threshold is set (for example, 0.5 or 0.6), and only when the first confidence level of the target result is higher than the threshold will it be retained. Target results below the threshold are considered unreliable and will be discarded. The filtered target results are set as primitive information.

[0089] Overlapping bounding boxes may indicate that the detection model has detected the same primitive multiple times, or detected multiple parts that are adjacent but actually the same primitive. If overlapping bounding boxes are found, the Non-Maximum Suppression (NMS) algorithm is used to process them. The basic idea of ​​the NMS algorithm is to retain the bounding box with the highest confidence, while suppressing (i.e., discarding) other overlapping bounding boxes with lower confidence, including: sorting all overlapping bounding boxes from high to low according to confidence; selecting the bounding box with the highest confidence as the best bounding box and retaining it; suppressing (discarding) other bounding boxes whose overlap with the best bounding box (i.e., intersection-over-union ratio) exceeds a certain threshold (such as 0.5); repeating the above steps until all bounding boxes have been processed. Updating the position information of the primitive information can ensure that each primitive is represented by only one accurate bounding box.

[0090] On the first processed image, the updated primitive information (including category and location) is labeled. For example, a rectangular box can be drawn around the bounding box, and the corresponding category information (e.g., "line," "circle," etc.) can be annotated inside or next to the box. The resulting image is the second recognized image, which visually displays the primitive information identified by the detection model and facilitates subsequent analysis and processing.

[0091] In a specific embodiment, identifying a text detection range from a text region includes:

[0092] (1) Obtain the boundary line of the text area, detect the intersection of the boundary line and the character image in the text area, and store the pixel coordinates of all intersections in a coordinate list.

[0093] (2) Initialize the drawing range, set the upper and lower ends of the drawing range to the Y coordinate value of the first element in the coordinate list, traverse each element in the coordinate list, start from the pixel coordinate of the current intersection, obtain the continuous area of ​​the character drawing points, and detect the minimum Y coordinate value and the maximum Y coordinate value.

[0094] (3) If the minimum Y coordinate value is less than the Y coordinate value corresponding to the upper end of the drawing range, the upper end of the drawing range is updated to the minimum Y coordinate value. If the maximum Y coordinate value is greater than the Y coordinate value corresponding to the lower end of the drawing range, the lower end of the drawing range is updated to the maximum Y coordinate value.

[0095] (4) The minimum X coordinate value and the maximum X coordinate value of the detection boundary line are set as the left end and the right end of the drawing range respectively.

[0096] (5) Generate a text detection range based on the upper end, lower end, left end, and right end of the drawing range.

[0097] Specifically, Figure 3 Generate a schematic diagram for text region recognition and word labeling. Boundaries typically represent the outline of a text region and can be rectangular or other shapes. Character images can be binarized (converting the image to black and white) to highlight the characters. Intersections are locations where boundary lines intersect with character edges. The pixel coordinates (including horizontal and vertical coordinates, X and Y) of all intersections are stored in a coordinate list. For example, the coordinate list may contain multiple coordinate points, such as [(x1, y1), (x2, y2), ..., (xn, yn)]. The upper and lower ends of the drawing range are set to the Y coordinate value of the first element in the coordinate list. For example, if the first element in the coordinate list is (x1, y1), the upper and lower ends of the drawing range are initially set to y1. Each element in the coordinate list is iterated over, starting from the pixel coordinates of the current intersection point, to obtain a continuous region of character drawing points. This region of continuous character drawing points can be determined using connected component analysis, which involves finding all pixels connected to the current intersection point. Within each continuous region of character drawing points, the minimum and maximum Y coordinate values ​​are detected. These two values ​​represent the upper and lower boundaries of the current character drawing region, respectively.

[0098] Dynamically adjust the drawing range to ensure that it contains the vertical boundaries of all characters. Set it to the left and right ends of the drawing range. This can be achieved by analyzing the coordinates of the boundary lines to ensure that the drawing range can also cover the entire text area in the horizontal direction.

[0099] The text detection range can be represented by a rectangular box, whose coordinates are (left end X coordinate value, top end Y coordinate value, right end X coordinate value, bottom end Y coordinate value).

[0100] In a specific embodiment, determining a word label represented by a text detection range includes:

[0101] (1) All image data within the text detection range are aggregated and set as an image dataset. A word recognition model is constructed based on a convolutional neural network. After the word recognition model is trained on the image dataset, it outputs the word label and the second confidence level contained in any image data.

[0102] (2) Determine whether the word label is correct based on the second confidence level. If not, decompose any image data into character image data, use the word recognition model to perform single character recognition on the character image data, output the single character, and modify the word label based on the single character.

[0103] Specifically, all image data within the text detection range are aggregated to form an image dataset, which contains image representations of various words. A word recognition model is built based on a convolutional neural network (CNN). CNN is a deep learning model suitable for processing image data. It can automatically extract features from images. Common CNN architectures (for example, ResNet, VGG, etc.) can be used as a basis and adjusted and optimized according to the word recognition task. The word recognition model is trained using the aggregated image dataset. During the training process, the model learns how to recognize words from images and outputs word labels and a second confidence level. The second confidence level indicates the confidence level of the word recognition model in the recognition results, and is usually a value between 0 and 1. After training, the word recognition model can output word labels and a second confidence level for the input image data.

[0104] A confidence threshold (e.g., 0.8) is set. If the second confidence level is lower than the threshold, the word label is considered to be incorrect. The corresponding image data is then decomposed into individual character image data. This can be achieved using a character segmentation algorithm (e.g., a method based on connected component analysis) to segment the word image into multiple independent character images. A word recognition model is then used to perform single-character recognition on the decomposed character image data, outputting the recognition result and confidence level for each character. The word label is then reassembled based on the single-character recognition results.

[0105] In a specific embodiment, obtaining the degree of association between symbols, attributes, and connection line information includes:

[0106] (1) Calculate a first distance between the symbol and the attribute based on the pixel coordinates. If the first distance is less than a first preset value, determine that the symbol is associated with the attribute.

[0107] (2) Calculating a second distance between the symbol and the connection line information. If the second distance is less than a second preset value, determining that the symbol is associated with the connection line information.

[0108] (3) If two symbols are connected by connecting line information, the two symbols are determined to be associated. Based on the recognition result of the connecting line information, the symbols corresponding to the starting point and the end point of the connecting line information are marked respectively.

[0109] (4) Summarize the number of associations between any symbol and attributes, connection line information, and other symbols to generate the degree of association.

[0110] Specifically, the Euclidean distance between each symbol and the attribute is calculated as the first distance. For example, for symbol S and attribute A, the first distance d(S, A) between their center points is calculated. For example, if there is an attribute label next to a symbol and the first distance between them is less than a first preset value, then the attribute is considered to be a description of the symbol.

[0111] The Euclidean distance between each symbol and the connection line information is calculated as the second distance. For example, for a symbol S and connection line information L, the second distance d(S, L) from the center point of the symbol to the nearest point of the connection line is calculated. For example, if a symbol is close to a connection line information and the distance between them is less than a second preset value, the symbol is considered to be associated with the connection line information.

[0112] If two symbols are connected by connecting line information, the symbols corresponding to the starting and ending points of the connecting line information are marked based on the identification results of the connecting line information. For example, if the starting point of the connecting line information corresponds to symbol S1 and the ending point corresponds to symbol S2, then S1 and S2 are determined to be associated. If the connecting line information connects multiple symbols, the connection relationship can be determined through topological analysis. For example, if the connecting line information sequentially connects symbols S1, S2, and S3, then S1 is associated with S2, and S2 is associated with S3. S1 is the symbol corresponding to the starting point, S2 is the symbol corresponding to the turning point (or intermediate point), and S3 is the symbol corresponding to the ending point.

[0113] For each symbol, summarize its association counts with attributes, connection line information, and other symbols. For example, symbol S is associated with two attributes, one connection line information, and three other symbols via connection lines. The statistical results are used as the symbol's association degree. This association degree can be represented as a vector or matrix, where each element represents the number of associations between the symbol and another element (attribute, connection line, other symbol). For example, the association degree of symbol S can be represented as [2, 1, 3], corresponding to the number of associations with attributes, connection line information, and other symbols, respectively.

[0114] In a specific embodiment, generating correction information including conflicting locations based on a symbol topology relationship graph includes:

[0115] (1) All symbols are connected based on the correlation degree vector to generate a vector drawing. If the vector drawing does not conform to the preset rules, it is determined that the primitive information does not conform to the preset rules.

[0116] (2) The graph neural network sets symbols as nodes and connection line information as edges, summarizes nodes and edges in vector drawings, and generates a symbolic topological relationship map.

[0117] (3) Convert the preset rules into graph constraints that can be learned by the graph neural network, input the symbolic topological relationship map into the graph neural network, cluster the output node embeddings, set the nodes corresponding to the discrete points as conflict points, set the edges connected to the conflict points as conflict edges, summarize the conflict points and conflict edges, and set them as conflict positions.

[0118] (4) Correct the conflict position based on the detection model and generate correction information.

[0119] Specifically, based on the degree of association between symbols (e.g., distance, connectivity), all symbols are connected using vectors to generate a vector drawing. A vector drawing is a graphical representation that shows the connectivity between symbols. The generated vector drawing is then compared against pre-set rules, which can be design specifications for engineering drawings. For example, certain symbols cannot be directly connected, or certain connection lines must follow a specific path. If the vector drawing does not conform to these pre-set rules, it is determined that a symbol in the primitive information does not conform to the pre-set rules.

[0120] Symbols are represented as nodes in a graph neural network (GNN), and connection information is represented as edges. GNNs are deep learning models specifically designed for processing graph-structured data, capable of learning the relationships between nodes and edges. All nodes (symbols) and edges (connection information) are aggregated in a vector drawing to construct a symbol topology graph. This graph is a graph structure in which nodes represent symbols and edges represent the connections between symbols.

[0121] Convert the preset rules into graph constraints that can be learned by the graph neural network. For example, if the rule stipulates that certain symbols cannot be directly connected, this rule can be converted into a graph constraint to restrict certain nodes from having edges. Input the symbol topology relationship graph into the graph neural network, which will learn the characteristics of the nodes and edges in the graph and output the embedding of each node. Perform cluster analysis on the output node embeddings. The clustering algorithm can use K-Means, etc. Through clustering, discrete points in the graph can be identified. The nodes corresponding to these discrete points may be conflict points. The nodes corresponding to the discrete points are set as conflict points, and the edges connected to the conflict points are set as conflict edges. These conflict points and conflict edges are summarized to determine the conflict location.

[0122] Use the detection model to correct conflicting locations. The detection model can provide correction suggestions based on pre-set rules and drawing constraints. For example, if the connection between one symbol and another doesn't conform to the rules, the detection model can suggest deleting or modifying the connection line. The correction results are organized into correction information, including a description of the conflicting location and correction suggestions. This correction information can be used for subsequent engineering drawing revisions and reviews.

[0123] In one embodiment, generating structured data including a symbol hierarchy tree, an attribute association matrix, and conflict annotations includes:

[0124] (1) Generate a symbol hierarchy tree based on the traversal connection relationship of symbols in vector drawing.

[0125] (2) Set the serial numbers corresponding to the symbols as rows, the attributes as columns, and the number of associations as values ​​to generate an attribute association matrix.

[0126] (3) The correction information corresponding to the conflict position is set as the conflict mark.

[0127] (4) Based on the spatial dimension, the symbol hierarchy tree, attribute association matrix and conflict annotation are summarized to generate structured data.

[0128] Specifically, based on the connectivity of symbols in a vector drawing, starting from the root node (usually the main symbol or starting symbol), the connectivity between symbols is traversed layer by layer. For example, a depth-first search or breadth-first search algorithm can be used to traverse the hierarchical relationships between symbols. Based on the traversal results, a symbol hierarchy tree is constructed. The symbol hierarchy tree is a directed acyclic graph, in which each node represents a symbol and the edges represent the connectivity between symbols. The structure of the hierarchy tree reflects the subordinate relationships or logical order between symbols. For example, a main symbol may connect to multiple sub-symbols, which may in turn connect to symbols at deeper levels.

[0129] Set the symbol's serial number as the row of the attribute association matrix, and the attribute as the column. The symbol serial number uniquely identifies the symbol, and the attribute is additional information about the symbol (e.g., label, annotation, etc.). Set the number of associations between the symbol and the attribute as the value of the matrix. For example, if symbol S1 has two associations with attribute A1, enter 2 at position (S1, A1) in the matrix. This step generates a complete attribute association matrix, where each row represents the association between a symbol and all attributes, and each column represents the association between all symbols and a particular attribute.

[0130] Set the correction information corresponding to the conflict location as a conflict annotation. The conflict annotation includes a detailed description of the conflict location (such as the symbol number and connector number) and correction suggestions (such as deleting connectors or changing symbol positions). Display the conflict annotation in a structured manner, such as using JSON or a table format, to clearly display the conflict location and correction suggestions.

[0131] Based on the spatial dimension (i.e., the symbol positional relationships on the drawing), the symbol hierarchy tree, attribute association matrix, and conflict annotations are aggregated. This spatial aggregation ensures spatial consistency among the symbol hierarchy, attribute associations, and conflict annotations. The aggregated symbol hierarchy tree, attribute association matrix, and conflict annotations are integrated into a complete structured data set. For example, this information can be stored in a database or saved in a file format (e.g., XML or JSON) for easy subsequent querying and processing.

[0132] The above describes the method for intelligent recognition of engineering drawings based on deep learning in the embodiment of the present application. The following describes the intelligent recognition system for engineering drawings based on deep learning in the embodiment of the present application. Figure 4 In the embodiments of the present application, an embodiment of the intelligent recognition system for engineering drawings based on deep learning includes:

[0133] The detection module 201 is used to obtain an engineering drawing image, perform adaptive illumination compensation preprocessing on the engineering drawing image, generate a first processed image, construct a detection model, perform multi-target collaborative detection on the first processed image based on the detection model, identify the primitive information of the first processed image, and generate a second recognized image.

[0134] The extraction module 202 is used to locate the text area of ​​the second recognition image based on the primitive information, identify the text detection range from the text area, determine the word label represented by the text detection range, summarize the character information of the text detection range based on the word label, and generate third image data.

[0135] The judgment module 203 is used to summarize the symbols, attributes and connection line information contained in the second recognition image and the third image data according to the pixel coordinates, obtain the degree of association between the symbols, attributes and connection line information, and detect whether the graphic element information complies with the preset rules based on the degree of association. If not, a symbol topology relationship map is constructed based on the graph neural network, and correction information containing conflicting positions is generated based on the symbol topology relationship map.

[0136] The fusion module 204 is used to fuse the primitive information, character information, and correction information to generate structured data including a symbol hierarchy tree, an attribute association matrix, and conflict annotations.

[0137] Through the collaborative efforts of the aforementioned components, a detection model based on deep learning and graph neural networks is first constructed to identify primitive information such as symbols, attributes, and connecting lines in engineering drawings. An optimization algorithm is used to select training image sets, and a non-maximum suppression algorithm is used to address bounding box overlap, improving detection accuracy. The correlation between symbols, attributes, and connecting lines is calculated based on pixel coordinates. Graph constraints are also used to check whether primitive information conforms to preset rules. This allows for efficient and accurate identification of primitive information in engineering drawings, significantly improving recognition accuracy and efficiency. Multi-target collaborative detection and association analysis can simultaneously process multiple primitives, avoiding the inefficiency of individual detection in traditional methods. Then, a text detection range is dynamically generated by detecting the intersection of the text region's boundary lines and character images. A word recognition model is constructed using a convolutional neural network. This model is trained on image data within the text detection range, outputting word labels and confidence scores. Single-character recognition corrects erroneous word labels, effectively handling text recognition in complex backgrounds and improving both accuracy and robustness. The single-character recognition correction mechanism further improves word recognition accuracy and reduces errors caused by blur or noise. Finally, a symbol topology relationship map is generated based on the degree of association, the preset rules are converted into graph constraints, and the conflict points and conflict edges are detected through the graph neural network. Correction information containing the conflict positions is generated, a symbol hierarchy tree is generated, the hierarchical relationship between symbols is displayed, an attribute association matrix is ​​constructed, the number of associations between symbols and attributes is recorded, conflict annotations are generated, conflict positions and correction suggestions are marked, and the data are summarized into structured data. It can intelligently detect whether the graphic element information complies with the preset rules, quickly locate conflict points and conflict edges, and provide a basis for the digital management and intelligent application of engineering drawings, realizing the intelligent recognition, conflict detection and structured data generation of engineering drawings.

[0138] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0139] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program code.

[0140] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for intelligent recognition of engineering drawings based on deep learning, characterized in that: The deep learning-based intelligent recognition method for engineering drawings includes: Acquire an engineering drawing image, perform adaptive illumination compensation preprocessing on the engineering drawing image to generate a first processed image, construct a detection model, perform multi-target collaborative detection on the first processed image based on the detection model, identify primitive information of the first processed image, and generate a second recognized image; Locating a text region of the second recognized image based on the primitive information, identifying a text detection range from the text region, determining a word label represented by the text detection range, and summarizing character information of the text detection range based on the word label to generate third image data; Summarizing the symbols, attributes, and connection line information contained in the second recognized image and the third image data based on pixel coordinates, obtaining the degree of association between the symbols, the attributes, and the connection line information, detecting whether the graphic element information complies with a preset rule based on the degree of association, and if not, constructing a symbol topology relationship map based on a graph neural network, and generating correction information including conflicting positions based on the symbol topology relationship map; The graphic element information, the character information, and the correction information are integrated to generate structured data including a symbol hierarchy tree, an attribute association matrix, and conflict annotations.

2. The method for intelligent recognition of engineering drawings based on deep learning according to claim 1, characterized in that: The construction of the detection model includes: Collect multiple engineering drawings based on drawing types and storage categories, and mark the categories of graphic elements contained in the engineering drawings; performing an embedding operation on the preprocessed engineering drawings to generate scene vectors, performing cluster analysis on all the scene vectors to generate a plurality of scene clusters, and generating at least one individual type for any engineering drawing based on the scene clusters, wherein the individual type represents a primitive distribution feature; Performing object detection on any of the engineering drawings to extract a plurality of graphic elements, cropping pixel regions contained in the graphic elements, and generating a first cropped image; performing an embedding operation on the first cropped image to generate an object vector, clustering all the object vectors to generate a plurality of object clusters, and generating an individual type corresponding to the graphic element for any of the engineering drawings based on the object clusters; Matching any of the engineering drawings with the generated individual types to generate a bipartite graph; Checking the number of individual types that match any of the engineering drawings based on the bipartite graph, setting the engineering drawing with the largest number of matches as a first sampled image, checking the number of remaining individual types that match any of the engineering drawings, setting the engineering drawing with the largest remaining number of matches as a second sampled image, and obtaining sampled images that match all of the individual types based on the same sampling steps; A training image set is selected from all the selected images based on the optimization algorithm, a deep learning model is constructed using the YOLO algorithm, the labeled training image set is input into the deep learning model for training, the parameters of the deep learning model are optimized, and the optimized deep learning model is set as the detection model.

3. The method for intelligent recognition of engineering drawings based on deep learning according to claim 2, characterized in that: Generating the second recognition image includes: Inputting the first processed image into the detection model, identifying a plurality of target results in the first processed image, the target results including category information, location information, and a first confidence level, screening the target results based on a confidence level threshold, and setting the screened target results as the primitive information; Determining whether the bounding boxes corresponding to the primitive information overlap based on the position information, and if so, extracting an optimal bounding box using a non-maximum suppression algorithm, and updating the position information based on the optimal bounding box; The primitive information is marked in the first processed image to generate the second recognized image.

4. The method for intelligent recognition of engineering drawings based on deep learning according to claim 1, characterized in that: The identifying a text detection range from the text area includes: Acquire a boundary line of the text area, detect intersections of the boundary line and character images in the text area, and store pixel coordinates of all the intersections in a coordinate list; Initialize the drawing range, set the upper and lower ends of the drawing range to the Y coordinate value of the first element in the coordinate list, traverse each element in the coordinate list, start from the pixel coordinate of the current intersection, obtain the area where the character drawing points are continuous, and detect the minimum and maximum Y coordinate values; If the minimum Y coordinate value is less than the Y coordinate value corresponding to the upper end of the drawing range, the upper end of the drawing range is updated to the minimum Y coordinate value; if the maximum Y coordinate value is greater than the Y coordinate value corresponding to the lower end of the drawing range, the lower end of the drawing range is updated to the maximum Y coordinate value; Detecting the minimum X coordinate value and the maximum X coordinate value of the boundary line, and setting them as the left end and the right end of the drawing range respectively; The text detection range is generated based on the upper end, lower end, left end, and right end of the drawing range.

5. The method for intelligent recognition of engineering drawings based on deep learning according to claim 4 is characterized in that: Determining the word label represented by the text detection range includes: Summarizing all image data within the text detection range and setting them as an image dataset, building a word recognition model based on a convolutional neural network, and outputting the word label and second confidence level contained in any image data after training the word recognition model on the image dataset; Based on the second confidence level, determine whether the word label is correct. If not, decompose any of the image data into character image data, use the word recognition model to perform single-character recognition on the character image data, output a single character, and modify the word label based on the single character.

6. The method for intelligent recognition of engineering drawings based on deep learning according to claim 1, characterized in that: The obtaining of the degree of association between the symbol, the attribute, and the connection line information includes: calculating a first distance between the symbol and the attribute based on the pixel coordinates, and determining that the symbol is associated with the attribute if the first distance is less than a first preset value; calculating a second distance between the symbol and the connection line information, and if the second distance is less than a second preset value, determining that the symbol is associated with the connection line information; If two symbols are connected by the connection line information, the two symbols are determined to be associated, and based on the recognition result of the connection line information, the symbols corresponding to the starting point and the end point of the connection line information are marked respectively; The association number between any symbol and the attribute, the connection line information, and other symbols is summarized to generate the association degree.

7. The method for intelligent recognition of engineering drawings based on deep learning according to claim 6, characterized in that: Generating correction information including conflicting positions based on the symbol topology relationship map includes: connecting all the symbols based on the association degree vector to generate a vector drawing, and if the vector drawing does not conform to the preset rule, determining that the graphic element information does not conform to the preset rule; The graph neural network sets the symbols as nodes, sets the connection line information as edges, summarizes the nodes and edges in the vector drawing, and generates the symbol topology relationship map; Converting the preset rules into graph constraints that can be learned by the graph neural network, inputting the symbolic topological relationship graph into the graph neural network, clustering the output node embeddings, setting the nodes corresponding to the discrete points as conflict points, setting the edges connected to the conflict points as conflict edges, summarizing the conflict points and the conflict edges, and setting them as the conflict positions; The conflict position is corrected based on the detection model to generate the correction information.

8. The method for intelligent recognition of engineering drawings based on deep learning according to claim 7, characterized in that: The generation of structured data including a symbol hierarchy tree, an attribute association matrix, and conflict annotations includes: generating the symbol hierarchy tree based on the traversal connection relationship of the symbols in the vector drawing; The attribute association matrix is ​​generated by setting the serial numbers corresponding to the symbols as rows, the attributes as columns, and the associated quantities as values; Setting the correction information corresponding to the conflict position as the conflict mark; The symbol hierarchy tree, the attribute association matrix, and the conflict annotations are aggregated based on a spatial dimension to generate the structured data.

9. An intelligent recognition system for engineering drawings based on deep learning, characterized in that: The engineering drawing intelligent recognition system based on deep learning includes: a detection module configured to acquire an engineering drawing image, perform adaptive illumination compensation preprocessing on the engineering drawing image to generate a first processed image, construct a detection model, perform multi-target collaborative detection on the first processed image based on the detection model, identify primitive information of the first processed image, and generate a second recognized image; an extraction module configured to locate a text region of the second recognition image based on the primitive information, identify a text detection range from the text region, determine a word label represented by the text detection range, summarize character information of the text detection range based on the word label, and generate third image data; a judgment module, configured to summarize the symbols, attributes, and connection line information contained in the second recognized image and the third image data based on pixel coordinates, obtain a degree of association between the symbols, the attributes, and the connection line information, detect whether the graphic element information complies with a preset rule based on the degree of association, and if not, construct a symbol topology relationship map based on a graph neural network, and generate correction information including conflicting locations based on the symbol topology relationship map; A fusion module is used to fuse the graphic element information, the character information, and the correction information to generate structured data including a symbol hierarchy tree, an attribute association matrix, and conflict annotations.

Images