A Deep Learning-Based Intelligent Recognition Method and System for Engineering Drawings

Abstract

This application relates to the field of drawing recognition technology, and discloses a method and system for intelligent recognition of engineering drawings based on deep learning. The method includes: performing multi-target collaborative detection on a first processed image based on a detection model, recognizing the primitive information of the first processed image, and generating a second recognized image; locating the text region of the second recognized image, recognizing the text detection range, determining the word labels represented by the text detection range, summarizing the character information of the text detection range based on the word labels, and generating third image data; obtaining the degree of correlation between symbols, attributes, and connecting line information, detecting whether the primitive information conforms to preset rules, constructing a symbol topology graph, and generating correction information including conflict locations; fusing primitive information, character information, and correction information to generate structured data including a symbol hierarchy tree, attribute association matrix, and conflict annotations. This application improves the efficiency and accuracy of intelligent recognition of engineering drawings.

Description

Technical Field

[0001] This application relates to the field of drawing recognition technology, and in particular to an intelligent recognition method and system for engineering drawings based on deep learning. Background Technology

[0002] In fields such as engineering construction, architectural design, and machinery manufacturing, engineering drawings are crucial documents for communicating design intent and guiding construction and production. However, traditional manual drawing recognition methods suffer from inefficiency, error-proneness, and difficulty in handling large-scale drawings. With the advancement of digital transformation, the digital storage and management of engineering drawings is becoming increasingly widespread, but how to efficiently and accurately extract key information from massive amounts of drawings has become an urgent problem to be solved.

[0003] A similar prior art is Chinese patent application CN118629066A, which discloses a deep learning-based engineering drawing recognition and automated inspection system. This system includes a completeness condition constraint module, used to determine the completeness conditions corresponding to the drawing based on the drawing information; a drawing completeness inspection module, used to determine whether the drawing is complete based on the corresponding completeness conditions; a verification condition constraint module, used to determine the corresponding verification conditions based on the drawing information; a drawing association module, used to determine the associated drawings of the engineering drawing based on the drawing information, and to determine the association level of the associated drawings, and also to determine the consistency content of the engineering drawing based on the association level; and a drawing verification module, used to determine whether to use the consistency content to supplement the drawing content based on the completeness condition judgment result of the drawing to be verified. By improving the accuracy of the analysis of drawing information, the accuracy of engineering drawing recognition and inspection is improved. Another Chinese patent application, CN111079528A, discloses a method and system for verifying primitive drawings based on deep learning. The method includes: acquiring image files of primitive drawings to be verified; cutting the image files to be verified into schematic diagrams and text information parts according to a ratio and recording the corresponding information; inputting the text information image part into a pre-built text recognition model to obtain the detected text information; verifying the rationality of the corresponding schematic diagram based on the text information using image processing methods, and outputting the result information.

[0004] The shortcomings of existing technologies are mainly reflected in the fact that when processing complex scenarios such as engineering drawings (e.g., chemical engineering, architectural engineering), the deep learning model still has a low recognition rate for non-standard symbols, dense layouts, and degraded drawings, and lacks the ability to extract fine-grained features of multi-level structural details. Relying on traditional image processing methods to verify schematic diagrams may fail to recognize blurred, degraded, or unconventional symbols. In practical applications, there is a need for automatic and efficient identification of various elements in engineering drawings and extraction of their semantic information to accurately achieve rapid retrieval, classification, and analysis of drawings. Summary of the Invention

[0005] This application provides a method and system for intelligent recognition of engineering drawings based on deep learning, which can improve the efficiency and accuracy of intelligent recognition of engineering drawings.

[0006] Firstly, this application provides a deep learning-based intelligent recognition method for engineering drawings, the deep learning-based intelligent recognition method for engineering drawings comprising:

[0007] Acquire an engineering drawing image, perform adaptive illumination compensation preprocessing on the engineering drawing image to generate a first processed image, construct a detection model, perform multi-target collaborative detection on the first processed image based on the detection model, identify the primitive information of the first processed image, and generate a second recognition image;

[0008] Based on the primitive information, the text region of the second recognition image is located, the text detection range is identified from the text region, the word label represented by the text detection range is determined, and the character information of the text detection range is summarized based on the word label to generate the third image data;

[0009] Based on pixel coordinates, the symbols, attributes, and connection line information contained in the second and third image data are summarized. The degree of correlation between the symbols, attributes, and connection line information is obtained. Based on the degree of correlation, it is detected whether the primitive information conforms to a preset rule. If it does not conform, a symbol topology graph is constructed based on a graph neural network. Correction information containing conflict positions is generated based on the symbol topology graph.

[0010] By integrating the primitive information, the character information, and the correction information, structured data containing a symbol hierarchy tree, an attribute association matrix, and conflict annotations is generated.

[0011] In conjunction with the first aspect, the construction of the detection model includes:

[0012] Collect multiple engineering drawings based on drawing type and storage category, and label the graphic element categories contained in the engineering drawings;

[0013] An embedding operation is performed on the preprocessed engineering drawings to generate scene vectors. All scene vectors are then clustered to generate multiple scene clusters. Based on the scene clusters, at least one individual type is generated for any engineering drawing, where the individual type represents the primitive distribution characteristics.

[0014] Perform object detection on any of the aforementioned engineering drawings to extract multiple graphic elements, crop out the pixel regions contained in the graphic elements, and generate a first cropped image;

[0015] An embedding operation is performed on the first cropped image to generate an object vector. All the object vectors are clustered to generate multiple object clusters. Based on the object clusters, an individual type corresponding to the graphic element is generated for any of the engineering drawings.

[0016] Match any of the aforementioned engineering drawings with the generated individual type to generate a bipartite graph;

[0017] Based on the bipartite graph, check the number of individual types that match any of the engineering drawings, set the engineering drawing with the largest number of matches as the first sampling image, check the number of remaining individual types that match any of the engineering drawings, set the engineering drawing with the largest remaining number of matches as the second sampling image, and obtain all sampling images that match the individual types based on the same sampling steps;

[0018] Based on the optimization algorithm, a training image set is selected from all the sampled images. A deep learning model is constructed using the YOLO algorithm. The labeled training image set is input into the deep learning model for training. The parameters of the deep learning model are optimized. The optimized deep learning model is then set as the detection model.

[0019] In conjunction with the first aspect, generating the second recognition image includes:

[0020] The first processed image is input into the detection model to identify multiple target results in the first processed image. The target results include category information, location information, and a first confidence level. The target results are filtered based on a confidence threshold, and the filtered target results are set as the primitive information.

[0021] Based on the location information, it is determined whether the bounding boxes corresponding to the primitive information overlap. If they overlap, the best bounding box is extracted using the non-maximum suppression algorithm, and the location information is updated based on the best bounding box.

[0022] The primitive information is marked in the first processed image to generate the second recognition image.

[0023] In conjunction with the first aspect, identifying the text detection range from the text region includes:

[0024] Obtain the boundary line of the text region, detect the intersection point of the boundary line and the character image in the text region, and store the pixel coordinates of all the intersection points into a coordinate list;

[0025] Initialize the drawing range by setting the upper and lower ends of the drawing range to the Y coordinate values ​​of the first element in the coordinate list. Iterate through each element in the coordinate list, starting from the pixel coordinates of the current intersection point, to obtain the continuous area of ​​the character drawing point and detect the minimum and maximum Y coordinate values.

[0026] If the minimum Y-coordinate value is less than the Y-coordinate value corresponding to the upper end of the drawing range, then the upper end of the drawing range is updated to the minimum Y-coordinate value; if the maximum Y-coordinate value is greater than the Y-coordinate value corresponding to the lower end of the drawing range, then the lower end of the drawing range is updated to the maximum Y-coordinate value.

[0027] The minimum and maximum X coordinate values ​​of the boundary line are detected and set as the left and right ends of the drawing range, respectively.

[0028] The text detection range is generated based on the upper, lower, left, and right ends of the drawing range.

[0029] In conjunction with the first aspect, determining the word tags represented by the text detection range includes:

[0030] All image data within the text detection range are aggregated and set as an image dataset. A word recognition model is constructed based on a convolutional neural network. After training the image dataset, the word recognition model outputs the word label and second confidence level contained in any image data.

[0031] Based on the second confidence level, determine whether the word label is correct. If it is incorrect, decompose any image data into character image data, use the word recognition model to perform single-character recognition on the character image data, output a single character, and modify the word label based on the single character.

[0032] In conjunction with the first aspect, obtaining the degree of correlation between the symbol, the attribute, and the connection line information includes:

[0033] Calculate a first distance between the symbol and the attribute based on the pixel coordinates. If the first distance is less than a first preset value, then determine that the symbol is associated with the attribute.

[0034] Calculate the second distance between the symbol and the connection line information; if the second distance is less than a second preset value, then determine that the symbol is associated with the connection line information.

[0035] If two symbols are connected by the connecting line information, the two symbols are determined to be related. Based on the recognition result of the connecting line information, the symbols corresponding to the start and end points of the connecting line information are marked respectively.

[0036] The degree of association is generated by summing the number of associations between any symbol and the attribute, the connection information, and other symbols.

[0037] In conjunction with the first aspect, generating correction information containing conflict locations based on the symbol topology graph includes:

[0038] Based on the degree of association, all the symbols are vector-connected to generate a vector drawing. If the vector drawing does not conform to the preset rules, the primitive information is determined to not conform to the preset rules.

[0039] The graph neural network sets the symbols as nodes and the connection line information as edges, and summarizes the nodes and edges in the vector drawing to generate the symbol topology graph;

[0040] The preset rules are transformed into graph constraints that the graph neural network can learn. The symbolic topological relationship graph is input into the graph neural network. The output node embeddings are clustered. The nodes corresponding to the discrete points are set as conflict points. The edges connected to the conflict points are set as conflict edges. The conflict points and the conflict edges are summarized and set as the conflict positions.

[0041] The conflict location is corrected based on the detection model, and the correction information is generated.

[0042] In conjunction with the first aspect, the generation of structured data including a symbol hierarchy tree, an attribute association matrix, and conflict annotations includes:

[0043] Based on the traversal and connection relationships of the symbols in the vector drawing, the symbol hierarchy tree is generated;

[0044] Set the sequence number corresponding to the symbol as the row, the attribute as the column, and the association quantity as the value to generate the attribute association matrix;

[0045] Set the correction information corresponding to the conflict location as the conflict label;

[0046] The structured data is generated by summarizing the symbol hierarchy tree, the attribute association matrix, and the conflict annotations based on the spatial dimension.

[0047] Secondly, this application provides a deep learning-based intelligent recognition system for engineering drawings, the deep learning-based intelligent recognition system for engineering drawings comprising:

[0048] The detection module is used to acquire engineering drawing images, perform adaptive illumination compensation preprocessing on the engineering drawing images to generate a first processed image, construct a detection model, perform multi-target collaborative detection on the first processed image based on the detection model, identify the primitive information of the first processed image, and generate a second recognition image.

[0049] The extraction module is used to locate the text region of the second recognition image based on the primitive information, identify the text detection range from the text region, determine the word label represented by the text detection range, summarize the character information of the text detection range based on the word label, and generate third image data.

[0050] The judgment module is used to summarize the symbol, attribute and connection line information contained in the second recognition image and the third image data according to the pixel coordinates, obtain the degree of correlation between the symbol, the attribute and the connection line information, detect whether the primitive information conforms to the preset rules based on the degree of correlation, and if it does not conform, construct a symbol topology relationship map based on the graph neural network, and generate correction information containing conflict positions based on the symbol topology relationship map.

[0051] The fusion module is used to fuse the primitive information, the character information, and the correction information to generate structured data containing a symbol hierarchy tree, an attribute association matrix, and conflict annotations.

[0052] The technical solution provided in this application firstly constructs a detection model based on deep learning and graph neural networks to identify primitive information such as symbols, attributes, and connecting lines in engineering drawings. An optimization algorithm is used to select a training image set, and a non-maximum suppression algorithm is used to handle bounding box overlap issues, improving detection accuracy. The correlation between symbols, attributes, and connecting lines is calculated based on pixel coordinates. Furthermore, graph constraints are used to detect whether primitive information conforms to preset rules. This approach can efficiently and accurately identify primitive information in engineering drawings, significantly improving recognition accuracy and efficiency. Multi-target collaborative detection and correlation analysis can simultaneously process multiple primitives, avoiding the inefficiency of individual detection in traditional methods. Then, by detecting the intersection of the boundary lines of text regions with character images, a text detection range is dynamically generated. A word recognition model is constructed using a convolutional neural network, trained on image data within the text detection range, and outputs word labels and confidence scores. Single-character recognition is used to correct erroneous word labels, effectively handling text recognition problems in complex backgrounds, improving the accuracy and robustness of text recognition. The single-character recognition correction mechanism further improves the accuracy of word recognition and reduces errors caused by ambiguity or noise. Finally, a symbol topology graph is generated based on the degree of association. The preset rules are transformed into graph constraints. Conflict points and edges are detected through a graph neural network, and correction information containing conflict locations is generated. A symbol hierarchy tree is generated to show the hierarchical relationship between symbols. An attribute association matrix is ​​constructed to record the number of associations between symbols and attributes. Conflict annotations are generated, marking conflict locations and correction suggestions, and summarized into structured data. This system can intelligently detect whether primitive information conforms to preset rules and quickly locate conflict points and edges, providing a foundation for the digital management and intelligent application of engineering drawings. It realizes intelligent recognition, conflict detection, and structured data generation of engineering drawings. Attached Figure Description

[0053] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is a schematic diagram of one embodiment of the deep learning-based intelligent recognition method for engineering drawings in this application.

[0055] Figure 2 This is a schematic diagram of one embodiment of the detection model construction and training process in this application.

[0056] Figure 3 This is a schematic diagram of an embodiment of text region recognition and word tag generation in this application.

[0057] Figure 4 This is a schematic diagram of one embodiment of the deep learning-based intelligent recognition system for engineering drawings in this application. Detailed Implementation

[0058] This application provides a method and system for intelligent recognition of engineering drawings based on deep learning. The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0059] For ease of understanding, the specific process of the embodiments of this application is described below. Please refer to [link / reference]. Figure 1 One embodiment of the deep learning-based intelligent recognition method for engineering drawings in this application includes:

[0060] Step S101: Obtain the engineering drawing image, perform adaptive illumination compensation preprocessing on the engineering drawing image to generate a first processed image, construct a detection model, perform multi-target collaborative detection on the first processed image based on the detection model, identify the primitive information of the first processed image, and generate a second recognition image.

[0061] It is understood that the executing entity of this application can be a deep learning-based intelligent recognition device for engineering drawings, or it can be a terminal or a server; no specific limitation is made here. This application's embodiment uses a server as an example for illustration.

[0062] Specifically, digital images of engineering drawings are acquired through scanners, cameras, or other image acquisition devices. These images may contain elements such as lines, symbols, and text. Adaptive lighting compensation algorithms are used to process the images, analyzing the lighting distribution and automatically adjusting brightness and contrast to eliminate the effects of uneven lighting. For example, histogram equalization can be used. The pre-processed engineering drawing image is then set as the first processed image. A detection model is built using a deep learning framework. This model can be a pre-trained general object detection algorithm, such as YOLO+CNN (multi-object detection + text region localization), and fine-tuned for the characteristics of the engineering drawing image. The pre-processed first image is input into the detection model, which simultaneously detects multiple targets in the first processed image, including primitive information such as lines, symbols, and text. Through multi-object collaborative detection, the location and category of each primitive information in the image can be accurately identified, generating a second recognized image. The primitive information includes the category information, location information, and recognition confidence of each primitive, such as the length, angle, and pixel coordinates of lines, and the shape, size, and pixel coordinates of symbols.

[0063] Step S102: Locate the text region of the second recognition image based on primitive information, identify the text detection range from the text region, determine the word label represented by the text detection range, summarize the character information of the text detection range based on the word label, and generate the third image data.

[0064] Specifically, text regions are located in the second recognition image based on primitive information (e.g., the bounding boxes of text boxes). The text detection range refers to the pixel area containing text characters. The word label refers to the textual and semantic content of the words within the text detection range. Character information refers to the attribute information corresponding to the word labels, including attributes such as font, size, and color of the characters. This character information will be integrated into the third image data.

[0065] Step S103: Summarize the symbol, attribute, and connection line information contained in the second and third image data based on pixel coordinates, obtain the degree of correlation between the symbol, attribute, and connection line information, detect whether the primitive information conforms to the preset rules based on the degree of correlation, if not, construct a symbol topology relationship map based on the graph neural network, and generate correction information containing conflict positions based on the symbol topology relationship map.

[0066] Specifically, pixel coordinate information of symbols, attributes, and connecting lines is extracted from the second and third image data. This pixel coordinate information is used to determine the spatial relationships between symbols, attributes, and connecting lines. The degree of association between symbols, attributes, and connecting lines is evaluated by calculating geometric features such as distances and angles. For example, if a connecting line connects to two symbols at each end, they are considered to be associated. Based on preset rules (e.g., connection rules between symbols, attribute labeling rules, etc.), the primitive information is checked for compliance. If non-compliance is found (e.g., incorrect connection of symbols by connecting lines, incorrect attribute labeling, etc.), a graph neural network (GNN) is used to construct a symbol topological relationship graph. The GNN can learn the topological structure between symbols and generate correction information including conflict locations.

[0067] Step S104: Integrate primitive information, character information, and correction information to generate structured data containing a symbol hierarchy tree, attribute association matrix, and conflict annotations.

[0068] Specifically, the graphic element information includes the geometric features and category information of symbols, lines, and connectors; character information includes text content and its formatting information; and correction information includes conflict locations and repair suggestions. The fused information is organized into structured data, including a symbol hierarchy tree (representing the hierarchical relationships between symbols), an attribute association matrix (representing the association relationships between symbols and attributes), and conflict annotations (marking locations that do not conform to the rules and providing repair suggestions). This structured data enables rapid retrieval, classification, and analysis of engineering drawings, improving the efficiency and accuracy of engineering drawing processing, reducing labor costs, and enhancing the level of intelligent engineering management.

[0069] In one specific embodiment, constructing a detection model includes:

[0070] (1) Collect multiple engineering drawings based on drawing type and storage category, and label the element categories contained in the engineering drawings.

[0071] (2) Perform an embedding operation on the preprocessed engineering drawings to generate scene vectors. Perform cluster analysis on all scene vectors to generate multiple scene clusters. Based on the scene clusters, generate at least one individual type for any engineering drawing, where the individual type represents the distribution characteristics of the primitives.

[0072] (3) Perform object detection on any engineering drawing to extract multiple primitives, crop out the pixel area contained in the primitives, and generate the first cropped image.

[0073] (4) Perform an embedding operation on the first cropped image to generate an object vector, cluster all object vectors to generate multiple object clusters, and generate an individual type corresponding to the primitive for any engineering drawing based on the object clusters.

[0074] (5) Match any engineering drawing with the generated individual type to generate a bipartite graph.

[0075] (6) Based on the bipartite graph, check the number of individual types that match any engineering drawing, set the engineering drawing with the largest number of matches as the first sampling image, check the number of remaining individual types that match any engineering drawing, set the engineering drawing with the largest remaining number of matches as the second sampling image, and obtain the sampling images that match all individual types based on the same sampling steps.

[0076] (7) Based on the optimization algorithm, select the training image set from all the selected images, use the YOLO algorithm to build a deep learning model, input the labeled training image set into the deep learning model for training, optimize the parameters of the deep learning model, and set the optimized deep learning model as the detection model.

[0077] Specifically, Figure 2 This document outlines the process for building and training the detection model. A large number of engineering drawings are collected as image samples based on drawing type (e.g., architectural drawings, electrical drawings, mechanical drawings, etc.) and storage category (e.g., scanned paper drawings, electronic drawings, etc.). These engineering drawings should cover various common graphic element categories, such as lines, symbols, text, and graphics. The collected engineering drawings are then labeled to clearly define the category of each graphic element. For example, lines are labeled as "straight line," "arc," etc.; symbols are labeled as "resistance," "valve," etc.; and text is labeled as "title," "annotation," etc. Labeling can be done manually using labeling tools or with the help of semi-automatic labeling tools to improve efficiency.

[0078] Embedding operations are performed on the preprocessed engineering drawings, converting each drawing into a high-dimensional scene vector. This embedding operation can be implemented using a deep learning model (such as a convolutional neural network), which extracts the overall features of the drawing, including the distribution and layout of primitives. A scene vector is a high-dimensional feature vector used to represent image content, capturing key features and semantic information of the engineering drawing. Clustering algorithms such as K-Means can be used to ensure that the engineering drawings within each scene cluster have similar primitive distribution characteristics. Individual types represent the primitive distribution characteristics of the drawing; for example, engineering drawings in one scene cluster might primarily feature electrical symbols, while those in another scene cluster might primarily feature architectural lines.

[0079] For each engineering drawing, object detection is performed to extract multiple primitives from the drawing. The position of the primitives is located using an object detection algorithm (e.g., YOLO), and the pixel region contained in the primitive is cropped to generate the first cropped image.

[0080] An embedding operation is performed on the first cropped image to generate object vectors, where each object vector represents the features of a single primitive in the first cropped image, including information such as shape, texture, and color. Primitives within each cluster of clustered objects exhibit similar features. Based on these object clusters, an individual type is generated for each primitive, representing the primitive's feature category.

[0081] Each engineering drawing is matched with the generated individual type to construct a bipartite graph, where the left node of the bipartite graph represents the engineering drawing, the right node represents the individual type, and the weight of the edge represents the degree of matching.

[0082] Based on the bipartite graph, the number of matching drawings for each individual type is checked, and the drawing with the largest number of matches is set as the first selected image. This ensures that a corresponding matching image is extracted for each individual type.

[0083] The goal of optimization algorithms is to minimize the total number of selected images while ensuring that each individual type is covered at least once; examples include linear programming or integer programming optimization algorithms. YOLO is an efficient real-time object detection algorithm suitable for detecting primitives in engineering drawings. The performance of the deep learning model is optimized by adjusting parameters such as the learning rate, optimizer, and loss function. After training, the optimized deep learning model is set as the detection model.

[0084] In one specific embodiment, generating a second recognition image includes:

[0085] (1) Input the first processed image into the detection model to identify multiple target results in the first processed image. The target results include category information, location information and first confidence level. The target results are filtered based on the confidence level threshold and the filtered target results are set as primitive information.

[0086] (2) Based on the location information, determine whether the bounding boxes corresponding to the primitive information overlap. If they overlap, use the non-maximum suppression algorithm to extract the best bounding box and update the location information based on the best bounding box.

[0087] (3) Mark the primitive information in the first processed image to generate the second recognition image.

[0088] Specifically, each target result includes category information (e.g., "line", "circle", "text", etc.), location information (usually represented as a bounding box, including the coordinates of the top-left corner, width, and height of the bounding box), and a first confidence score (representing the detection model's confidence in the detection result, typically a value between 0 and 1). A confidence threshold is set (e.g., 0.5 or 0.6), and only target results with a first confidence score higher than this threshold are retained. Target results with a first confidence score lower than the threshold are considered unreliable and are discarded; the filtered target results are then set as primitive information.

[0089] Overlapping bounding boxes may indicate that the detection model detected the same primitive multiple times, or detected multiple parts of adjacent primitives that are actually the same primitive. If overlapping bounding boxes are found, Non-Maximum Suppression (NMS) is used to handle them. The basic idea of ​​NMS is to retain the bounding box with the highest confidence while suppressing (i.e., discarding) other overlapping bounding boxes with lower confidence. This includes: sorting all overlapping bounding boxes in descending order of confidence; selecting the bounding box with the highest confidence as the best bounding box and retaining it; suppressing (discarding) other bounding boxes whose overlap (i.e., intersection-union ratio) with the best bounding box exceeds a certain threshold (e.g., 0.5); repeating the above steps until all bounding boxes have been processed. Updating the location information of primitives ensures that each primitive has only one accurate bounding box representation.

[0090] On the first processed image, labels are created based on the updated primitive information (including category and location information). For example, a rectangle can be drawn around the bounding box, and the corresponding category information (e.g., "line", "circle") can be labeled inside or next to the box. After labeling, the generated image becomes the second recognition image, which can intuitively display the primitive information recognized by the detection model, facilitating subsequent analysis and processing.

[0091] In one specific embodiment, identifying the text detection range from the text region includes:

[0092] (1) Obtain the boundary line of the text region, detect the intersection of the boundary line and the character image in the text region, and store the pixel coordinates of all intersection points into the coordinate list.

[0093] (2) Initialize the drawing range, set the upper and lower ends of the drawing range to the Y coordinate value of the first element in the coordinate list, traverse each element in the coordinate list, start from the pixel coordinate of the current intersection point, obtain the continuous area of ​​the character drawing point, and detect the minimum and maximum Y coordinate values.

[0094] (3) If the minimum Y coordinate value is less than the Y coordinate value corresponding to the upper end of the drawing range, the upper end of the drawing range is updated to the minimum Y coordinate value. If the maximum Y coordinate value is greater than the Y coordinate value corresponding to the lower end of the drawing range, the lower end of the drawing range is updated to the maximum Y coordinate value.

[0095] (4) Detect the minimum and maximum X coordinate values ​​of the boundary line and set them as the left and right ends of the drawing range, respectively.

[0096] (5) Generate the text detection range based on the top, bottom, left and right ends of the drawing range.

[0097] Specifically, Figure 3 This diagram illustrates text region recognition and word tag generation. Boundary lines typically represent the outline of a text region and can be rectangles or other shapes. Character images can be binarized (converted to black and white) to highlight characters. Intersection points are the locations where boundary lines intersect with character edges. The pixel coordinates (including x and y coordinates) of all intersection points are stored in a coordinate list. For example, the coordinate list may contain multiple coordinate points, such as [(x1, y1), (x2, y2), ..., (xn, yn)]. The upper and lower boundaries of the drawing area are set to the y-coordinate value of the first element in the coordinate list. For example, if the first element of the coordinate list is (x1, y1), then initially the upper and lower boundaries of the drawing area are both y1. Each element in the coordinate list is iterated through. Starting from the pixel coordinates of the current intersection point, a continuous region of character drawing points is obtained. This continuous region can be determined through connected component analysis, i.e., finding all pixels connected to the current intersection point. Within each continuous region of character drawing points, the minimum and maximum y-coordinate values ​​are detected. These two values ​​represent the upper and lower boundaries of the current character drawing area, respectively.

[0098] Dynamically adjust the drawing area to ensure that the vertical boundaries of all characters are included. Setting the left and right edges of the drawing area can be achieved by analyzing the coordinates of the boundary lines, ensuring that the drawing area also covers the entire text area horizontally.

[0099] The text detection area can be represented by a rectangle, whose coordinates are (left X coordinate value, top Y coordinate value, right X coordinate value, bottom Y coordinate value).

[0100] In one specific embodiment, determining the word label represented by the text detection range includes:

[0101] (1) Summarize all image data within the text detection range and set it as an image dataset. Construct a word recognition model based on a convolutional neural network. After training the image dataset, the word recognition model outputs the word label and second confidence level contained in any image data.

[0102] (2) Determine whether the word label is correct based on the second confidence level. If it is incorrect, decompose any image data into character image data, use the word recognition model to perform single character recognition on the character image data, output a single character, and modify the word label based on the single character.

[0103] Specifically, all image data within the text detection range are aggregated to form an image dataset containing image representations of various words. A word recognition model is built based on a Convolutional Neural Network (CNN). CNNs are deep learning models suitable for processing image data, capable of automatically extracting features from images. Common CNN architectures (e.g., ResNet, VGG) can be used as a foundation, and adjusted and optimized according to the word recognition task. The aggregated image dataset is then used to train the word recognition model. During training, the model learns how to recognize words from images and outputs word labels and a second confidence score. The second confidence score represents the word recognition model's confidence in the recognition result, typically a value between 0 and 1. After training, the word recognition model can output word labels and a second confidence score for the input image data.

[0104] A confidence threshold (e.g., 0.8) is set. If the second confidence score is lower than this threshold, the word label is considered potentially incorrect. The corresponding image data is then decomposed into individual character image data, which can be achieved using a character segmentation algorithm (e.g., a method based on connected component analysis). The word image is segmented into multiple independent character images. A word recognition model is then used to perform single-character recognition on the decomposed character image data, outputting the recognition result and confidence score for each character. Based on the results of the single-character recognition, the word label is recombined.

[0105] In one specific embodiment, obtaining the degree of correlation between symbols, attributes, and connection line information includes:

[0106] (1) Calculate the first distance between the symbol and the attribute based on the pixel coordinates. If the first distance is less than the first preset value, the symbol and the attribute are determined to be associated.

[0107] (2) Calculate the second distance between the symbol and the connecting line information. If the second distance is less than the second preset value, then determine that the symbol and the connecting line information are associated.

[0108] (3) If two symbols are connected by connecting line information, the two symbols are determined to be related. Based on the recognition result of the connecting line information, the symbols corresponding to the start and end points of the connecting line information are marked respectively.

[0109] (4) Summarize the number of associations between any symbol and attributes, connecting line information, and other symbols to generate the degree of association.

[0110] Specifically, the Euclidean distance between each symbol and attribute is calculated as the first distance. For example, for symbol S and attribute A, the first distance d(S, A) between their center points is calculated. For example, if a symbol is next to an attribute label and the first distance between them is less than a first preset value, then the attribute is considered to be a description of the symbol.

[0111] Calculate the Euclidean distance between each symbol and the connecting line information as a second distance. For example, for a symbol S and connecting line information L, calculate the second distance d(S, L) from the center point of the symbol to the nearest point of the connecting line. For example, if a symbol is close to a connecting line information and the distance between them is less than a second preset value, then the symbol is considered to be associated with the connecting line information.

[0112] If two symbols are connected by a connecting line, the symbols corresponding to the start and end points of the connecting line are marked according to the recognition results. For example, if the start point of the connecting line corresponds to symbol S1 and the end point corresponds to symbol S2, then S1 and S2 are determined to be associated. If the connecting line connects multiple symbols, the connection relationship can be determined through topological analysis. For example, if the connecting line connects symbols S1, S2, and S3 in sequence, then S1 is associated with S2, and S2 is associated with S3. Here, S1 is the symbol corresponding to the start point, S2 is the symbol corresponding to the turning point (or intermediate point), and S3 is the symbol corresponding to the end point.

[0113] For each symbol, summarize the number of associations it has with attributes, connectors, and other symbols. For example, symbol S is associated with 2 attributes, 1 connector, and 3 other symbols via connectors. The statistical results can be used as the degree of association of the symbol, represented as a vector or matrix, where each element represents the number of associations the symbol has with other elements (attributes, connectors, and other symbols). For example, the degree of association of symbol S can be represented as [2, 1, 3], corresponding to the number of associations with attributes, connectors, and other symbols, respectively.

[0114] In one specific embodiment, generating correction information containing conflict locations based on a symbolic topological relationship graph includes:

[0115] (1) Connect all symbols based on the degree of association and generate a vector drawing. If the vector drawing does not conform to the preset rules, the primitive information is determined to not conform to the preset rules.

[0116] (2) The graph neural network sets the symbols as nodes and the connection line information as edges. In the vector drawing, the nodes and edges are summarized to generate a symbol topology graph.

[0117] (3) Transform the preset rules into graph constraints that the graph neural network can learn, input the symbolic topological relationship graph into the graph neural network, cluster the output node embeddings, set the nodes corresponding to the discrete points as conflict points, set the edges connected to the conflict points as conflict edges, summarize the conflict points and conflict edges, and set them as conflict positions.

[0118] (4) Correct the conflict location based on the detection model and generate correction information.

[0119] Specifically, based on the degree of association between symbols (e.g., distance, connection relationships, etc.), all symbols are connected by vectors to generate a vector drawing. Vector drawing is a graphical representation method that shows the connection relationships between symbols. The generated vector drawing is compared with preset rules, which can be design specifications for engineering drawings. For example, some symbols cannot be directly connected, or certain connection lines must follow specific paths. If the vector drawing does not conform to these preset rules, it is determined that there are symbols in the graphic element information that do not conform to the preset rules.

[0120] Symbols are represented as nodes in a Graph Neural Network (GNN), and connection information is represented as edges. A GNN is a deep learning model specifically designed for processing graph-structured data, capable of learning the relationships between nodes and edges. All nodes (symbols) and edges (connection information) are aggregated in the vector drawing to construct a symbol topology graph. This graph is a graph structure where nodes represent symbols and edges represent the connections between symbols.

[0121] Pre-defined rules are transformed into graph constraints that can be learned by the graph neural network. For example, if a rule stipulates that certain symbols cannot be directly connected, this rule can be transformed into a graph constraint restricting certain nodes from having edges. The symbolic topological relationship graph is input into the graph neural network, which learns the features of the nodes and edges in the graph and outputs the embedding of each node. Cluster analysis is performed on the output node embeddings. Clustering algorithms such as K-Means can be used. Through clustering, discrete points in the graph can be identified, and the nodes corresponding to these discrete points may be conflict points. The nodes corresponding to the discrete points are designated as conflict points, and the edges connected to the conflict points are designated as conflict edges. By summarizing these conflict points and conflict edges, the conflict locations are determined.

[0122] A detection model is used to correct conflict locations. Based on preset rules and drawing constraints, the detection model can propose correction suggestions. For example, if the connection between two symbols does not conform to the rules, the detection model can suggest deleting or modifying the connecting line. The corrected results are compiled into correction information, including descriptions of the conflict locations and correction suggestions. This correction information can be used for subsequent engineering drawing modifications and reviews.

[0123] In one specific embodiment, generating structured data including a symbol hierarchy tree, an attribute association matrix, and conflict annotations includes:

[0124] (1) Generate a symbol hierarchy tree based on the traversal connection relationship of symbols in vector drawing.

[0125] (2) Set the sequence number corresponding to the symbol as the row, the attribute as the column, and the association quantity as the value to generate the attribute association matrix.

[0126] (3) Set the correction information corresponding to the conflict location as the conflict label.

[0127] (4) Based on the spatial dimension, summarize the symbol hierarchy tree, attribute association matrix and conflict annotation to generate structured data.

[0128] Specifically, based on the connection relationships between symbols in vector graphics, starting from the root node (usually the master symbol or starting symbol), the connections between symbols are traversed layer by layer. For example, depth-first search or breadth-first search algorithms can be used to traverse the hierarchical relationships between symbols. Based on the traversal results, a symbol hierarchy tree is constructed. The symbol hierarchy tree is a directed acyclic graph, where each node represents a symbol, and edges represent the connections between symbols. The structure of the hierarchy tree reflects the subordinate relationships or logical order between symbols. For example, a master symbol may connect to multiple child symbols, and these child symbols may in turn connect to symbols at deeper levels.

[0129] Set the sequence number corresponding to the symbol as the row of the attribute association matrix, and the attribute as the column. The symbol sequence number is the unique identifier of the symbol, and the attribute is additional information about the symbol (e.g., label, annotation, etc.). Set the number of associations between the symbol and the attribute as the value of the matrix. For example, if symbol S1 is associated with attribute A1 twice, then fill in 2 at the (S1, A1) position in the matrix. Through this step, a complete attribute association matrix is ​​generated. Each row of the matrix represents the association between a symbol and all attributes, and each column represents the association between all symbols and a certain attribute.

[0130] Set the correction information corresponding to the conflict location as a conflict label. The conflict label includes a detailed description of the conflict location (such as symbol number, connector number, etc.) and correction suggestions (such as deleting connectors, modifying symbol positions, etc.). Represent the conflict labels in a structured manner, for example, using JSON format or tables, to clearly show the conflict location and correction suggestions.

[0131] Based on the spatial dimension (i.e., the positional relationship of symbols on the drawing), the symbol hierarchy tree, attribute association matrix, and conflict annotations are summarized. Summarizing the spatial dimension ensures the consistency of the symbol hierarchy, attribute associations, and conflict annotations in space. The summarized symbol hierarchy tree, attribute association matrix, and conflict annotations are then integrated into a complete structured data set. For example, this information can be stored in a database or saved in file format (such as XML or JSON) for easy subsequent querying and processing.

[0132] The above describes the deep learning-based intelligent recognition method for engineering drawings in the embodiments of this application. The following describes the deep learning-based intelligent recognition system for engineering drawings in the embodiments of this application. Please refer to [link / reference]. Figure 4 One embodiment of the deep learning-based intelligent recognition system for engineering drawings in this application includes:

[0133] The detection module 201 is used to acquire engineering drawing images, perform adaptive illumination compensation preprocessing on the engineering drawing images to generate a first processed image, construct a detection model, perform multi-target collaborative detection on the first processed image based on the detection model, identify the primitive information of the first processed image, and generate a second recognition image.

[0134] The extraction module 202 is used to locate the text region of the second recognition image based on the primitive information, identify the text detection range from the text region, determine the word label represented by the text detection range, summarize the character information of the text detection range based on the word label, and generate the third image data.

[0135] The judgment module 203 is used to summarize the symbol, attribute and connection line information contained in the second recognition image and the third image data according to the pixel coordinates, obtain the degree of correlation between the symbol, attribute and connection line information, detect whether the primitive information conforms to the preset rules based on the degree of correlation, and if it does not conform, construct a symbol topology relationship map based on the graph neural network, and generate correction information containing conflict positions based on the symbol topology relationship map.

[0136] The fusion module 204 is used to fuse primitive information, character information, and correction information to generate structured data containing a symbol hierarchy tree, attribute association matrix, and conflict annotations.

[0137] Through the collaborative efforts of the aforementioned components, firstly, a detection model based on deep learning and graph neural networks is constructed to identify primitive information such as symbols, attributes, and connecting lines in engineering drawings. An optimization algorithm is used to select the training image set, and a non-maximum suppression algorithm is employed to address bounding box overlap issues, improving detection accuracy. The correlation between symbols, attributes, and connecting lines is calculated based on pixel coordinates. Furthermore, graph constraints are used to detect whether primitive information conforms to preset rules. This approach enables efficient and accurate identification of primitive information in engineering drawings, significantly improving recognition accuracy and efficiency. Multi-target collaborative detection and correlation analysis can simultaneously process multiple primitives, avoiding the inefficiency of individual detection in traditional methods. Secondly, by detecting the intersection of the boundary lines of text regions with character images, a text detection range is dynamically generated. A word recognition model is constructed using a convolutional neural network, trained on image data within the text detection range, and outputs word labels and confidence scores. Single-character recognition is used to correct erroneous word labels, effectively handling text recognition problems in complex backgrounds, improving the accuracy and robustness of text recognition. The single-character recognition correction mechanism further enhances the accuracy of word recognition and reduces errors caused by ambiguity or noise. Finally, a symbol topology graph is generated based on the degree of association. The preset rules are transformed into graph constraints. Conflict points and edges are detected through a graph neural network, and correction information containing conflict locations is generated. A symbol hierarchy tree is generated to show the hierarchical relationship between symbols. An attribute association matrix is ​​constructed to record the number of associations between symbols and attributes. Conflict annotations are generated, marking conflict locations and correction suggestions, and summarized into structured data. This system can intelligently detect whether primitive information conforms to preset rules and quickly locate conflict points and edges, providing a foundation for the digital management and intelligent application of engineering drawings. It realizes intelligent recognition, conflict detection, and structured data generation of engineering drawings.

[0138] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0139] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0140] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for intelligent recognition of engineering drawings based on deep learning, characterized in that, The deep learning-based intelligent recognition method for engineering drawings includes: Acquire an engineering drawing image, perform adaptive illumination compensation preprocessing on the engineering drawing image to generate a first processed image, construct a detection model, perform multi-target collaborative detection on the first processed image based on the detection model, identify the primitive information of the first processed image, and generate a second recognition image; Based on the primitive information, the text region of the second recognition image is located, the text detection range is identified from the text region, the word label represented by the text detection range is determined, and the character information of the text detection range is summarized based on the word label to generate the third image data; Based on pixel coordinates, the symbols, attributes, and connection line information contained in the second and third image data are summarized. The degree of correlation between the symbols, attributes, and connection line information is obtained. Based on the degree of correlation, it is detected whether the primitive information conforms to a preset rule. If it does not conform, a symbol topology graph is constructed based on a graph neural network. Correction information containing conflict positions is generated based on the symbol topology graph. By integrating the primitive information, the character information, and the correction information, structured data containing a symbol hierarchy tree, an attribute association matrix, and conflict annotations is generated.

2. The intelligent recognition method for engineering drawings based on deep learning according to claim 1, characterized in that, The construction of the detection model includes: Collect multiple engineering drawings based on drawing type and storage category, and label the graphic element categories contained in the engineering drawings; An embedding operation is performed on the preprocessed engineering drawings to generate scene vectors. All scene vectors are then clustered to generate multiple scene clusters. Based on the scene clusters, at least one individual type is generated for any engineering drawing, where the individual type represents the primitive distribution characteristics. Perform object detection on any of the aforementioned engineering drawings to extract multiple graphic elements, crop out the pixel regions contained in the graphic elements, and generate a first cropped image; An embedding operation is performed on the first cropped image to generate an object vector. All the object vectors are clustered to generate multiple object clusters. Based on the object clusters, an individual type corresponding to the graphic element is generated for any of the engineering drawings. Match any of the aforementioned engineering drawings with the generated individual type to generate a bipartite graph; Based on the bipartite graph, check the number of individual types that match any of the engineering drawings, set the engineering drawing with the largest number of matches as the first sampling image, check the number of remaining individual types that match any of the engineering drawings, set the engineering drawing with the largest remaining number of matches as the second sampling image, and obtain all sampling images that match the individual types based on the same sampling steps; Based on the optimization algorithm, a training image set is selected from all the sampled images. A deep learning model is constructed using the YOLO algorithm. The labeled training image set is input into the deep learning model for training. The parameters of the deep learning model are optimized. The optimized deep learning model is then set as the detection model.

3. The intelligent recognition method for engineering drawings based on deep learning according to claim 2, characterized in that, The generation of the second recognition image includes: The first processed image is input into the detection model to identify multiple target results in the first processed image. The target results include category information, location information, and a first confidence level. The target results are filtered based on a confidence threshold, and the filtered target results are set as the primitive information. Based on the location information, it is determined whether the bounding boxes corresponding to the primitive information overlap. If they overlap, the best bounding box is extracted using the non-maximum suppression algorithm, and the location information is updated based on the best bounding box. The primitive information is marked in the first processed image to generate the second recognition image.

4. The intelligent recognition method for engineering drawings based on deep learning according to claim 1, characterized in that, The step of identifying the text detection range from the text region includes: Obtain the boundary line of the text region, detect the intersection point of the boundary line and the character image in the text region, and store the pixel coordinates of all the intersection points into a coordinate list; Initialize the drawing range by setting the upper and lower ends of the drawing range to the Y coordinate values ​​of the first element in the coordinate list. Iterate through each element in the coordinate list, starting from the pixel coordinates of the current intersection point, to obtain the continuous area of ​​the character drawing point and detect the minimum and maximum Y coordinate values. If the minimum Y-coordinate value is less than the Y-coordinate value corresponding to the upper end of the drawing range, then the upper end of the drawing range is updated to the minimum Y-coordinate value; if the maximum Y-coordinate value is greater than the Y-coordinate value corresponding to the lower end of the drawing range, then the lower end of the drawing range is updated to the maximum Y-coordinate value. The minimum and maximum X coordinate values ​​of the boundary line are detected and set as the left and right ends of the drawing range, respectively. The text detection range is generated based on the upper, lower, left, and right ends of the drawing range.

5. The intelligent recognition method for engineering drawings based on deep learning according to claim 4, characterized in that, The step of determining the word tags represented by the text detection range includes: All image data within the text detection range are aggregated and set as an image dataset. A word recognition model is constructed based on a convolutional neural network. After training the image dataset, the word recognition model outputs the word label and second confidence level contained in any image data. Based on the second confidence level, determine whether the word label is correct. If it is incorrect, decompose any image data into character image data, use the word recognition model to perform single-character recognition on the character image data, output a single character, and modify the word label based on the single character.

6. The intelligent recognition method for engineering drawings based on deep learning according to claim 1, characterized in that, The process of obtaining the degree of correlation between the symbol, the attribute, and the connection line information includes: Calculate a first distance between the symbol and the attribute based on the pixel coordinates. If the first distance is less than a first preset value, then determine that the symbol is associated with the attribute. Calculate the second distance between the symbol and the connection line information; if the second distance is less than a second preset value, then determine that the symbol is associated with the connection line information. If two symbols are connected by the connecting line information, the two symbols are determined to be related. Based on the recognition result of the connecting line information, the symbols corresponding to the start and end points of the connecting line information are marked respectively. The degree of association is generated by summing the number of associations between any symbol and the attribute, the connection information, and other symbols.

7. The intelligent recognition method for engineering drawings based on deep learning according to claim 6, characterized in that, The generation of correction information containing conflict locations based on the symbol topology graph includes: Based on the degree of association, all the symbols are vector-connected to generate a vector drawing. If the vector drawing does not conform to the preset rules, the primitive information is determined to not conform to the preset rules. The graph neural network sets the symbols as nodes and the connection line information as edges, and summarizes the nodes and edges in the vector drawing to generate the symbol topology graph; The preset rules are transformed into graph constraints that the graph neural network can learn. The symbolic topological relationship graph is input into the graph neural network. The output node embeddings are clustered. The nodes corresponding to the discrete points are set as conflict points. The edges connected to the conflict points are set as conflict edges. The conflict points and the conflict edges are summarized and set as the conflict positions. The conflict location is corrected based on the detection model, and the correction information is generated.

8. The intelligent recognition method for engineering drawings based on deep learning according to claim 7, characterized in that, The generation of structured data, including a symbol hierarchy tree, attribute association matrix, and conflict annotations, includes: Based on the traversal and connection relationships of the symbols in the vector drawing, the symbol hierarchy tree is generated; Set the sequence number corresponding to the symbol as the row, the attribute as the column, and the association quantity as the value to generate the attribute association matrix; Set the correction information corresponding to the conflict location as the conflict label; The structured data is generated by summarizing the symbol hierarchy tree, the attribute association matrix, and the conflict annotations based on the spatial dimension.

9. A deep learning-based intelligent recognition system for engineering drawings, characterized in that, The deep learning-based intelligent recognition system for engineering drawings includes: The detection module is used to acquire engineering drawing images, perform adaptive illumination compensation preprocessing on the engineering drawing images to generate a first processed image, construct a detection model, perform multi-target collaborative detection on the first processed image based on the detection model, identify the primitive information of the first processed image, and generate a second recognition image. The extraction module is used to locate the text region of the second recognition image based on the primitive information, identify the text detection range from the text region, determine the word label represented by the text detection range, summarize the character information of the text detection range based on the word label, and generate third image data. The judgment module is used to summarize the symbol, attribute and connection line information contained in the second recognition image and the third image data according to the pixel coordinates, obtain the degree of correlation between the symbol, the attribute and the connection line information, detect whether the primitive information conforms to the preset rules based on the degree of correlation, and if it does not conform, construct a symbol topology relationship map based on the graph neural network, and generate correction information containing conflict positions based on the symbol topology relationship map. The fusion module is used to fuse the primitive information, the character information, and the correction information to generate structured data containing a symbol hierarchy tree, an attribute association matrix, and conflict annotations.

Images