Drawing identification method, device, equipment, storage medium and product

Abstract

This application discloses a drawing recognition method, apparatus, device, storage medium, and product. The method includes: acquiring a drawing to be recognized; inputting the drawing to be recognized into a pre-trained functional area recognition model, a symbol recognition model, and a line recognition model to recognize the drawing elements and their position information, whereby the drawing elements include functional areas, symbols, and lines; constructing association relationships between drawing elements based on preset relationships and position information between functional areas, symbols, and lines to obtain multiple structured data sets to be verified; performing mutual verification between corresponding two drawing elements based on each structured data set to be verified; and determining the drawing recognition result based on the drawing elements if the verification passes. This method can improve the efficiency and accuracy of drawing recognition.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a drawing recognition method, apparatus, device, computer storage medium, and computer program product. Background Technology

[0002] Two-dimensional industrial drawings, serving as the core medium for engineering design, manufacturing, and project review, are widely used in numerous industrial fields such as machinery manufacturing, aerospace, electronics and electrical engineering, and automotive engineering. These drawings contain key elements such as functional areas (e.g., title blocks, technical requirements areas, parts lists), special symbols (e.g., roughness symbols, geometric tolerance symbols, and important characteristic markings), and special lines (e.g., centerlines, dimension lines, and section lines). These elements collectively constitute complete design semantic information. Accurately identifying and understanding these elements and their interrelationships is a crucial foundation for achieving intelligent parsing, automated review, and structured archiving of industrial drawings.

[0003] In recent years, with the development of computer vision and deep learning technologies, intelligent technologies have been introduced into the field of drawing recognition. Existing technical solutions mainly include semi-automated assisted recognition tools based on image processing and template matching, which can extract line outlines or identify frequently occurring standard symbols in drawings. However, when faced with complex scenarios such as overlapping areas, non-standard symbols, and damaged drawings, significant manual intervention is still required, and data between different tools is difficult to share, failing to form a complete recognition chain. Another mainstream approach is to use a single deep learning model (such as a convolutional neural network-based object detection model) to uniformly recognize multiple elements in drawings. However, functional areas, special symbols, and special lines in industrial drawings exhibit significant differences in visual features—areas are typically regular rectangular boxes, symbols are complex vector graphics, and lines are thin linear structures—a single model cannot simultaneously meet the feature extraction needs of all three types of elements, often leading to problems such as loss of symbol details, broken lines, or blurred area boundaries. The recognition accuracy, especially for small symbols and complex lines, is insufficient to meet the requirements of industrial applications.

[0004] Therefore, how to provide a drawing recognition method to improve the efficiency and accuracy of drawing recognition has become an urgent problem to be solved. Summary of the Invention

[0005] This application provides a drawing recognition method, apparatus, device, computer storage medium, and computer program product, which can improve the efficiency and accuracy of drawing recognition.

[0006] Firstly, this application provides a drawing recognition method, the method comprising: Obtain the drawing to be identified; The drawing to be identified is input into a pre-trained functional area recognition model, symbol recognition model, and line recognition model, respectively, to identify the drawing elements of the drawing and the position information of each drawing element. The drawing elements include: functional areas, symbols, and lines. Based on the preset relationships between functional areas, symbols, and lines, and the location information, the relationships between the drawing elements are constructed to obtain multiple structured data to be verified. Based on each structured data to be verified, the corresponding two drawing elements are mutually verified. If the verification passes, the drawing recognition result is obtained based on the drawing elements.

[0007] In some possible implementations, the association between the drawing elements is constructed based on the preset functional area-symbol-line relationship and the position information, resulting in multiple structured data to be verified, including: Based on the preset relationship between functional areas, symbols, and lines, and the position information, detect whether the symbols and lines are located in the preset functional areas; When symbols and lines are located in preset functional areas, the association between symbols and functional areas and between lines and functional areas is constructed to obtain multiple structured data to be verified.

[0008] In some possible implementations, the mutual verification of corresponding two drawing elements based on each structured data to be verified includes: The functional region and the first structured data to be verified obtained based on the symbol-functional region are input into the symbol recognition model so that the symbol recognition model can identify whether there is a corresponding symbol in the functional region; If the corresponding symbol exists in the functional area, it is determined that the first structured data to be verified has passed the verification. or, The functional area and the second structured data to be verified obtained based on the line-functional area are input into the line recognition model so that the line recognition model can identify whether there are corresponding lines in the functional area; If a corresponding line exists in the functional area, the second structured data to be verified is determined to have passed the verification.

[0009] In some possible implementations, the method further includes: If no corresponding symbol is found in the functional area, it is determined that the first structured data to be verified has failed the verification. or, If no corresponding line exists in the functional area, it is determined that the second structured data to be verified has failed the verification. If the verification fails, the relevant functional areas will be displayed separately and an alarm message will be output.

[0010] In some possible implementations, the functional area recognition model, the symbol recognition model, and the line recognition model are trained in the following manner: Obtain a drawing training sample dataset, wherein the drawing training sample data in the drawing training sample dataset includes drawing samples and their corresponding functional area labels, symbol labels and line labels; The drawing sample is input into the functional area recognition model, symbol recognition model, and line recognition model respectively to obtain the functional area, symbol, and line recognition results; Based on the functional region recognition results, symbol recognition results, and line recognition results, as well as their corresponding labels, the functional region recognition model, symbol recognition model, and line recognition model are trained to obtain the trained functional region recognition model, symbol recognition model, and line recognition model.

[0011] In some possible implementations, the functional area recognition model, the symbol recognition model, and the line recognition model include an attention network and a recognition network; The step of inputting the drawing to be identified into a pre-trained functional area recognition model, symbol recognition model, and line recognition model to identify the drawing elements and position information of each drawing element includes: By using a recognition network, the drawing elements of the drawing to be recognized and the position information of each drawing element are identified; The drawing to be identified is enhanced with dual attention through an attention network; The attention network includes a channel attention subnetwork and a spatial attention subnetwork; the attention network processes the drawing to be identified in the following manner: The drawing to be identified is weighted according to the channel dimension by a channel attention subnetwork to generate a first intermediate feature map; The first intermediate feature map is weighted by spatial dimensions using a spatial attention sub-network to generate a second intermediate feature map, which serves as the output of the attention network.

[0012] In some possible implementations, the step of weighting the drawing to be identified by the channel dimension through a channel attention subnetwork to generate a first intermediate feature map includes: The drawing to be identified is subjected to global average pooling and global max pooling respectively to obtain the first pooling feature and the second pooling feature; The first pooling feature and the second pooling feature are input into a shared multilayer perceptron network to generate a first channel weight vector and a second channel weight vector, respectively. The first channel weight vector and the second channel weight vector are added element by element and then fused together, and processed by an activation function to obtain the channel attention weights. The channel attention weights are multiplied channel by channel with the drawing to be identified to obtain the first intermediate feature map.

[0013] In some possible implementations, the step of generating a second intermediate feature map by performing spatial dimension weighting processing on the first intermediate feature map through a spatial attention sub-network includes: The first intermediate feature map is subjected to global average pooling and global max pooling in the channel dimension to obtain two two-dimensional spatial feature maps. The two two-dimensional spatial feature maps are spliced ​​together along the channel dimension to obtain the spliced ​​spatial feature map. The stitched spatial feature map is input into a convolutional layer for convolution processing and then processed by an activation function to generate spatial attention weights. The spatial attention weights are multiplied element-wise with the first intermediate feature map to obtain the second intermediate feature map.

[0014] In some possible implementations, the method further includes: If the size of the drawing to be identified is greater than a preset size threshold, a sliding window cutting strategy is used to cut the drawing into multiple blocks; the overlap rate between the blocks is greater than a preset overlap threshold, so that functional areas, symbols, and lines are at least completely contained in one block; The obtained multiple blocks are input into the functional area recognition model, the symbol recognition model, and the line recognition model respectively for recognition, and the recognition results are obtained.

[0015] In some possible implementations, the method further includes: The nonmaximum suppression algorithm is used to remove redundant detection boxes in the recognition results.

[0016] In some possible implementations, the use of a non-maximum suppression algorithm to remove redundant detection boxes in the recognition result includes: Obtain the category, confidence level, and location information of the detection box corresponding to each patch in the recognition result; Detection boxes of the same category are sorted according to confidence level, and the detection boxes corresponding to the confidence level that meet the preset conditions are used as the benchmark detection boxes. Detect the intersection-union ratio (IoU) of each of the detection boxes with the reference detection box; If the cross-union ratio is greater than a preset threshold, the detection box is determined to be a redundant detection box; Based on the location information, the redundant detection boxes are removed.

[0017] In some possible implementations, the association between the functional area, symbol, and line includes at least one of the following: The relationship between functional areas and symbols; The relationship between functional areas and lines; The relationships between functional areas; The relationships between symbols.

[0018] Secondly, this application also provides a drawing recognition device, the device comprising: The acquisition module is used to acquire the drawing to be recognized; The input module is used to input the drawing to be identified into the pre-trained functional area recognition model, symbol recognition model and line recognition model respectively, and to identify the drawing elements of the drawing to be identified and the position information of each drawing element. The drawing elements include: functional areas, symbols and lines. The construction module is used to construct the association relationship between the drawing elements based on the preset association relationship between functional areas, symbols and lines and the position information, so as to obtain multiple structured data to be verified. The verification module is used to perform mutual verification between two corresponding drawing elements based on each structured data to be verified. The determination module is used to determine the drawing recognition result based on the drawing elements if the verification passes.

[0019] Thirdly, this application also provides a drawing recognition device, the device comprising: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the drawing recognition method as described in the first aspect above.

[0020] Fourthly, this application also provides a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the drawing recognition method as described in the first aspect above.

[0021] Fifthly, this application also provides a computer program product in which instructions, when executed by a processor of an electronic device, enable the electronic device to perform the drawing recognition method as described in the first aspect above.

[0022] The drawing recognition method provided in this application, after acquiring the drawing to be recognized, can, on the one hand, input the drawing into pre-trained functional area recognition models, symbol recognition models, and line recognition models for specialized recognition. Employing a separate model-specific training strategy, each model focuses on learning the features of its own target elements, avoiding recognition bias caused by multi-element feature confusion. Compared to single-model multi-element hybrid recognition methods, the recognition accuracy of each type of element is significantly improved. Simultaneously, the models are specifically optimized for industrial drawing recognition scenarios, accurately capturing features of easily missed elements such as micro-symbols, fine lines, and overlapping functional areas. This effectively solves the problems of missed and false recognition in traditional recognition methods and existing technologies, ensuring the completeness and accuracy of drawing element recognition. On the other hand, by combining the relationships between functional areas, symbols, and lines, the scattered single-element recognition results are transformed into associated data with industrial logic, realizing collaborative recognition of drawing information. Furthermore, based on the construction of cross-element relationships, bidirectional mutual verification is carried out for each set of structured data to be verified, which can effectively correct the misjudgment problem in the single model recognition stage, eliminate false relationships caused by model recognition deviations, and ensure that each set of verified relationships has actual drawing information support, fundamentally improving the reliability of drawing recognition results and reducing the workload of subsequent manual review. Attached Figure Description

[0023] The features, advantages, and technical effects of exemplary embodiments of this application will now be described with reference to the accompanying drawings.

[0024] Figure 1 A flowchart illustrating a drawing recognition method provided in one embodiment of this application; Figure 2 A flowchart of S130 provided in one embodiment of this application; Figure 3 A flowchart of model training provided for one embodiment of this application; Figure 4 This is a schematic diagram of the structure of a drawing recognition device provided in another embodiment of this application; Figure 5 This is a schematic diagram of the structure of a drawing recognition device provided in another embodiment of this application. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0027] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0028] In this application, the term "embodiment" is used to mean that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0029] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0030] In recent years, with the development of computer vision and deep learning technologies, intelligent technologies have been introduced into the field of drawing recognition. Existing technical solutions mainly include semi-automated assisted recognition tools based on image processing and template matching, which can extract line outlines or identify frequently occurring standard symbols in drawings. However, when faced with complex scenarios such as overlapping areas, non-standard symbols, and damaged drawings, significant manual intervention is still required, and data between different tools is difficult to share, failing to form a complete recognition chain. Another mainstream approach is to use a single deep learning model (such as a convolutional neural network-based object detection model) to uniformly recognize multiple elements in drawings. However, functional areas, special symbols, and special lines in industrial drawings exhibit significant differences in visual features—areas are typically regular rectangular boxes, symbols are complex vector graphics, and lines are thin linear structures—a single model cannot simultaneously meet the feature extraction needs of all three types of elements, often leading to problems such as loss of symbol details, broken lines, or blurred area boundaries. The recognition accuracy, especially for small symbols and complex lines, is insufficient to meet the requirements of industrial applications.

[0031] Therefore, how to provide a drawing recognition method to improve the efficiency and accuracy of drawing recognition has become an urgent problem to be solved.

[0032] To address the problems of the prior art, embodiments of the present invention provide a drawing recognition method, apparatus, device, computer storage medium, and computer program product.

[0033] The following is combined Figure 1 The drawing recognition method provided in the embodiments of the present invention will be described in detail.

[0034] Figure 1 A schematic flowchart of a drawing recognition method according to an embodiment of the present invention is shown, as follows: Figure 1 As shown, the method may include the following steps: S110. Obtain the drawing to be identified.

[0035] In S110, the drawing to be identified refers to a two-dimensional industrial drawing that requires recognition of functional areas, symbols, and lines. It covers industrial drawings from various industries such as mechanical manufacturing, aerospace, electronics and electrical engineering, and automotive engineering. It supports mainstream industrial drawing formats such as AutoCAD, SolidWorks, and Pro / E, and includes various core elements of industrial drawings such as parts list, technical requirements area, roughness symbols, and center lines.

[0036] In practice, digitized image data of the drawings to be recognized can be obtained through compliant channels such as industrial scanning equipment, industrial drawing management systems, local storage devices, and cloud storage platforms. The acquired drawing data is then subjected to basic format normalization processing to ensure its readability, forming a unified drawing input file to prepare for subsequent input model recognition.

[0037] S120. Input the drawing to be recognized into the pre-trained functional area recognition model, symbol recognition model and line recognition model respectively, and recognize the drawing elements of the drawing to be recognized and the position information of each drawing element. The drawing elements include: functional areas, symbols and lines.

[0038] In S120, the functional area recognition model refers to a deep learning model trained on a dataset of tens of thousands of industrial drawings, specifically designed to identify various functional areas in industrial drawings. The core recognition targets are eight types of functional areas in industrial drawings, including parts list, change list, reference list, and technical requirements area. The model optimization focuses on the accuracy of the area bounding box regression to ensure the positional accuracy of functional area recognition.

[0039] The symbol recognition model is a deep learning model trained on a dataset of tens of thousands of industrial-specific drawings. It is specifically designed to recognize various industrial symbols in industrial drawings. The core recognition targets are industrial symbols such as surface roughness, geometric tolerances, important characteristic markings, and material symbols. The model incorporates an attention mechanism to enhance the ability to extract detailed features of the symbols.

[0040] The line recognition model is a deep learning model trained on a dataset of tens of thousands of industrial drawings. It is specifically designed to recognize various functional lines in industrial drawings. The core recognition targets are center lines, dashed lines, dimension lines, break lines, section lines, etc. The model introduces a line segmentation detection and extension recognition module to improve the ability to recognize line continuity.

[0041] Drawing elements refer to the core components of industrial drawings that possess specific industrial meaning. They are divided into three categories: functional areas, symbols, and lines. These elements are the core carriers of information expression in industrial drawings and the main identification objects of this method. Functional areas include eight key areas: parts list (including serial number, part name, date, etc.), change list (including change serial number, date, change content, etc.), datum list (including datum symbol, datum surface / line number, etc.), and technical requirements area (including process requirements, inspection standards, etc.). Symbols include surface roughness, important characteristic markings, material symbols, and shape tolerance symbols. Lines include center lines, dashed lines, dimension lines, break lines, and section lines.

[0042] Location information refers to the spatial coordinates of various drawing elements in the digitized image of the drawing to be identified. It is used to characterize the specific location of the drawing elements in the drawing and is the core spatial data basis for constructing the relationship between different drawing elements.

[0043] In practice, the digitized image of the drawing to be recognized can first be preprocessed, including image scaling, rotation correction, and grayscale standardization, to eliminate the influence of factors such as drawing tilt and brightness differences on the recognition results, forming image data that meets the model input requirements. Then, the preprocessed image data is input into the pre-trained functional area recognition model, symbol recognition model, and line recognition model, respectively, and the three models perform specialized recognition simultaneously. After that, the functional area recognition model outputs the category and corresponding location information of the recognized functional areas, the symbol recognition model outputs the category and corresponding location information of the recognized industrial symbols, and the line recognition model outputs the category and corresponding location information of the recognized functional lines. Finally, the output results of the three models are integrated to form a basic recognition dataset containing the category and location information of all drawing elements, thus completing the extraction of drawing elements.

[0044] The method of using a sub-model-specific recognition approach replaces the traditional single-model multi-element recognition mode, avoiding feature confusion between different types of drawing elements and improving the recognition accuracy of various elements. At the same time, it obtains element position information, providing spatial data support for the subsequent construction of the relationship between elements, and solving the problems of low accuracy and lack of spatial position reference in the existing technology for drawing element recognition.

[0045] S130. Based on the preset functional area-symbol-line relationship and position information, construct the relationship between drawing elements to obtain multiple structured data to be verified.

[0046] In S130, the relationship between functional areas, symbols, and lines refers to the spatial logical constraint relationship between functional areas, symbols, and lines, which is extracted based on industrial drawing design specifications, industry standards, and actual engineering application requirements. It includes four core relationships: the relationship between functional areas and symbols, the relationship between functional areas and lines, the relationship between functional areas, and the relationship between symbols. It is the core basis for constructing the relationship between drawing elements.

[0047] For example, the relationship between functional areas and symbols defines that special symbols must be located within specific areas, such as roughness symbols, which must be located within the part outline area (not in the title block); special symbols must conform to the text description of the area, for example, if the change column indicates several modifications, the corresponding change symbols on the drawing must also correspond one-to-one. The relationship between functional areas and lines defines that dimension lines must connect area boundaries, and center lines must pass through the center of symmetry of the area; the relationship between functional areas defines, for example, what the drawing mark in the title block is, and the drawing mark on all subsequent pages must be consistent; the relationship between symbols defines, for example, that two different types of characteristic symbols cannot exist simultaneously in the same set of drawings (company-specific rule).

[0048] Structured data refers to data formed by standardizing and encapsulating the category, location information, and association type of two related drawing elements. It has a unified data format and clear logical relationship, which facilitates subsequent cross-validation.

[0049] Structured data to be verified refers to structured data that has not yet undergone logical verification after the initial establishment of relationships. It is the core processing object for subsequent cross-model interactive verification.

[0050] In practice, a pre-defined database of relationships between functional areas, symbols, and lines can be retrieved to obtain spatial logical constraint rules between various drawing elements. Based on the positional information of each drawing element, it is checked whether the symbols and lines are within their corresponding pre-defined functional areas to verify the spatial position matching between elements. For drawing elements with matching spatial positions, corresponding relationships are constructed according to the rules in the database, including the relationship between symbols and functional areas, and the relationship between lines and functional areas. The relevant information of each pair of drawing elements with a relationship is standardized and encapsulated to form a single structured data set to be verified. All structured data sets that meet the conditions are integrated to form a dataset to be verified. For example, based on the relationship between functional areas and symbols, after associating the drawing elements in the recognition results, the structured data to be verified, which associates "part contour area ID_123" with "roughness symbol ID_456", can be obtained.

[0051] Based on the relationships extracted from industrial drawing design specifications, and combined with element location information, spatial relationships between elements of different types of drawings are constructed. The scattered single element identification results are transformed into structured data with logical relationships, providing verification objects for subsequent cross-validation and solving the problem that the drawing element identification results are independent and have no logical relationship in the existing technology.

[0052] S140. Based on each structured data to be verified, perform mutual verification on the corresponding two drawing elements.

[0053] In S140, cross-verification refers to using the structured data to be verified as a basis, inputting the information of one drawing element into a dedicated model for recognizing another drawing element, and then having the model perform high-precision re-recognition on a specified area to verify whether the two drawing elements conform to the preset association relationship. This cross-verification between different models is the core method for correcting misjudgments by a single model.

[0054] In practice, each structured data point in the dataset to be verified can be traversed, and verification processing can be performed sequentially. For structured data formed by associating symbols with functional areas, the location information of the functional areas and the associated symbol types are input into the symbol recognition model. The symbol recognition model focuses on the functional area to perform high-precision re-identification, improving the detection confidence threshold and verifying whether the corresponding symbol exists within the functional area. For structured data formed by associating lines with functional areas, the location information of the functional areas and the associated line types are input into the line recognition model. The line recognition model focuses on the functional area to perform high-precision re-identification, improving the detection confidence threshold and verifying whether the corresponding line exists within the functional area. Finally, the verification result for each structured data point is recorded, distinguishing between structured data that pass and fail verification, providing a basis for subsequent steps.

[0055] By using cross-model interactive verification, drawing elements with related relationships are mutually verified, enabling secondary verification of the recognition results of a single model. This effectively corrects misjudgments in the recognition of a single model, solves the problems of no verification of single model recognition results and failure to detect misjudgments in a timely manner in existing technologies, and improves the reliability of the overall recognition results.

[0056] If the verification passes, execute S150; if the verification fails, execute S160.

[0057] S150. Determine the drawing recognition result based on the drawing elements.

[0058] Drawing recognition results refer to standardized and structured industrial drawing recognition results formed after integrating the category, location information and interrelationships of all verified drawing elements. These results can be directly applied to industrial processes such as industrial design review, manufacturing, and drawing archive management.

[0059] In practice, all verified structured data to be verified can be integrated, and the categories, locations, and relationships between drawing elements can be extracted. The integrated information is then standardized and organized to form a structured data file with a unified format. This structured data file is then used as the final drawing recognition result, and can be used to output visualized and annotated drawings or structured information data tables according to industrial application needs.

[0060] S160, differentiate display function areas and output alarm information.

[0061] Distinctive display refers to using a different visual display method than the normal recognition area to mark the functional areas that failed the verification, thus visually distinguishing abnormal areas and facilitating staff to quickly locate and identify them. Alarm information refers to the output of prompts containing the location and type of abnormal area when verification fails, providing clear information for staff to review manually.

[0062] In practice, for structured data that fails verification, the location information of the corresponding functional area is first extracted. Then, in the visualization of the drawing to be identified, a special visual style is used to distinguish and display this functional area, achieving accurate location of the abnormal area. An alarm message containing the location of the functional area, the type of associated element, and the reason for the verification failure is generated and output. This can then be output through pop-ups, text prompts, system messages, etc. Finally, the distinguished drawing and alarm message are simultaneously pushed to relevant personnel, prompting them to conduct manual review.

[0063] Based on the verification results, differentiated processing is performed. When the verification passes, the results are integrated to form standardized identification results, providing usable drawing information for industrial applications. When the verification fails, anomaly marking and alarms are issued to achieve visual feedback on identification anomalies, prompting staff to conduct manual review. This solves the problems of no feedback on identification anomalies and no clear direction for manual review in existing technologies, and achieves effective connection between automated identification and manual review.

[0064] The above is the overall process of the drawing recognition method according to the embodiments of this application. After acquiring the drawing to be recognized, the drawing provided by this application can, on the one hand, input the drawing to be recognized into pre-trained functional area recognition models, symbol recognition models, and line recognition models for specialized recognition. By adopting a separate model-specific training strategy, the functional area, symbol, and line recognition models can focus on learning the features of their respective target elements, avoiding recognition bias caused by the confusion of multiple element features. Compared with the single-model multi-element hybrid recognition method, the recognition accuracy of all types of elements is significantly improved. Simultaneously, the models are specifically optimized for industrial drawing recognition scenarios, accurately capturing the features of easily missed elements such as micro-symbols, fine lines, and overlapping functional areas, effectively solving the problems of missed and misjudged judgments in traditional recognition methods and existing technologies, ensuring the completeness and accuracy of drawing element recognition. On the other hand, by combining the relationships between functional areas, symbols, and lines, the scattered single-element recognition results are transformed into associated data with industrial logic, realizing collaborative recognition of drawing information. Furthermore, based on the construction of cross-element relationships, bidirectional mutual verification is carried out for each set of structured data to be verified, which can effectively correct the misjudgment problem in the single model recognition stage, eliminate false relationships caused by model recognition deviations, and ensure that each set of verified relationships has actual drawing information support, fundamentally improving the reliability of drawing recognition results and reducing the workload of subsequent manual review.

[0065] In some embodiments, such as Figure 2 As shown, S130 may include: S1301. Based on the preset functional area-symbol-line relationship and position information, detect whether the symbol and line are located in the preset functional area.

[0066] In practice, the pre-defined relationships between functional areas, symbols, and lines are stored in a spatial association library. Based on these relationships, the pre-defined functional area type for each type of symbol and line can be clearly identified. For each matched symbol-functional area and line-functional area, the positional information is compared to determine whether the overall area of ​​the symbol falls within the corresponding pre-defined functional area and whether the effective area of ​​the line falls within the corresponding pre-defined functional area.

[0067] If the symbol or line is located in the preset function area, execute S1302; if the symbol or line is not located in the preset function area, execute S1303.

[0068] S1302. Construct the association between symbols and functional areas, and lines and functional areas to obtain multiple structured data to be verified.

[0069] In practice, when a symbol is located within a preset functional area, the association between the symbol and the functional area can be obtained; when a line is located within a preset functional area, the association between the line and the functional area can be obtained. Then, the categories, location information, association types, and corresponding identifiers of both parties can be integrated and encapsulated to form an independent structured data set to be verified.

[0070] For example, after the symbol model identifies the roughness symbol, it queries the "region-symbol association" rule in the spatial association library and finds the part contour region (identified by the region model) located within the boundary coordinate range of the symbol based on the position information of both parties. It can automatically bind the "roughness symbol-part contour region" association relationship. Then, the bound association relationship is stored in the result database to form a structured record of "target ID (part contour region ID_123) - associated target ID (roughness symbol ID_456) - association rule".

[0071] S1303, Remove mismatched associations.

[0072] If the symbols or lines are not located in the preset function area, mismatched relationships can be directly removed without further processing.

[0073] By adopting a standardized process of first verifying spatial matching and then constructing associations, it can be ensured that the constructed associations of drawing elements conform to the spatial logic specifications of industrial drawing design, avoid invalid associations caused by mismatched spatial positions of elements, provide a reasonable basis for subsequent cross-element mutual verification, improve the accuracy and effectiveness of association construction from the source, and further ensure the reliability of the overall drawing recognition results.

[0074] Since a single recognition model may make misjudgments (such as a symbol model misidentifying a line as a symbol), in order to improve the accuracy of recognition, the recognition results can be verified through cross-model interactive verification.

[0075] In some embodiments, mutual verification of two corresponding drawing elements based on each structured data to be verified may include: inputting a functional area and a first structured data to be verified obtained based on a symbol-functional area into a symbol recognition model, so that the symbol recognition model can identify whether a corresponding symbol exists in the functional area; if a corresponding symbol exists in the functional area, the first structured data to be verified is determined to have passed verification. Alternatively, inputting a functional area and a second structured data to be verified obtained based on a line-functional area into a line recognition model, so that the line recognition model can identify whether a corresponding line exists in the functional area; if a corresponding line exists in the functional area, the second structured data to be verified is determined to have passed verification.

[0076] In specific implementation, taking the first set of structured data to be verified based on symbol-functional region as an example, firstly, all the first set of structured data to be verified constructed based on the symbol-functional region association relationship are extracted from the structured dataset to be verified, and special verification processing is carried out sequentially to ensure that each set of data can be verified in a targeted manner. For a single set of first set of structured data to be verified, the location information of the functional region, the category of the functional region, and the category of the symbol to be associated are extracted and integrated to form the verification association information of the set of data. Then, the above-mentioned integrated complete information of the functional region, together with the set of first set of structured data to be verified, is input into a pre-trained symbol recognition model, transmitting precise recognition and verification instructions to the model, clarifying the model's recognition focus range and target symbol type. After receiving the input information, the symbol recognition model no longer performs full-domain recognition on the entire drawing to be recognized, but focuses on the specified functional region range to perform high-precision re-recognition, while increasing the detection confidence threshold of the target symbol type in the region, thereby improving the accuracy of recognition and verification. After completing the focused recognition, the symbol recognition model outputs the symbol recognition result in the functional region, and the model will determine whether there are symbols in the region that are consistent with the type to be associated in the first set of structured data to be verified. If the model recognition results show that the corresponding category of symbols truly exists within the specified functional area, then the symbol-functional area association relationship of the first set of structured data to be verified is determined to be true and valid, and the verification of the first set of structured data to be verified is confirmed to be passed. Finally, the first set of structured data to be verified that has passed can be marked. Subsequently, the drawing elements and association relationships corresponding to this type of data will be integrated and included in the final drawing recognition result. At the same time, the next set of first set of structured data to be verified will continue to be processed in sequence until all data of this type has been verified.

[0077] The purpose of validating the structured data to be verified based on the symbol-functional area association is to feed the functional area information and the corresponding structured data back to the symbol recognition model. This allows the model to move beyond the limitations of global recognition and focus on specific functional areas for targeted, high-precision re-identification. This verifies whether the symbols to be associated actually exist within those areas, thus determining whether the previously constructed symbol-functional area association matches the actual drawing. This method achieves a secondary verification of the recognition results of a single model, effectively eliminating false associations caused by biases in global model recognition, improving the accuracy of symbol-functional area associations, and ensuring the reliability of subsequent drawing recognition results by verifying each association through actual recognition.

[0078] It should be noted that if the corresponding symbol does not exist in the functional area, the first structured data to be verified is determined to have failed verification; or, if the corresponding line does not exist in the functional area, the second structured data to be verified is determined to have failed verification. In the case of verification failure, the functional area is displayed separately and an alarm message is output.

[0079] Taking the interactive verification between the region model and the symbol model as an example (taking the structured data to be verified as a "special region - special symbol" association as an example), firstly, the system initiates a verification command. After the region model identifies the "important characteristic region" (the region containing key dimension annotations), it queries the spatial association library to determine that the region needs to match the "important characteristic symbol". It then pushes the command "important characteristic symbol to be identified" to the symbol model, along with the boundary coordinates of the region. The symbol model focuses on the region pushed by the command and performs high-precision identification again (increasing the detection confidence threshold of the region from 0.6 to 0.7). If the important characteristic symbol is identified, it outputs "Special symbol exists, verification passed"; if the important characteristic symbol is not identified, it outputs "No special symbol detected, verification failed" and marks the region as "awaiting manual review". At the same time, the verification result is fed back to the region model. If the verification passes, the association between the region and the symbol is retained; if the verification fails, the functional region is displayed separately (for example, the region is marked with a red border in the recognition result image) and an alarm is output.

[0080] In some embodiments, such as Figure 3 As shown, the functional area recognition model, symbol recognition model, and line recognition model can be trained in the following ways: S310. Obtain the drawing training sample dataset. The drawing training sample dataset includes drawing samples and their corresponding functional area labels, symbol labels, and line labels.

[0081] In practice, a multi-channel data collection approach can be adopted to construct the dataset. This involves integrating publicly available industrial drawing datasets, actual production drawings obtained in cooperation with enterprises, and diverse self-generated drawing samples. Simultaneously, standard compliance management is implemented to ensure all samples meet industry standards and data usage requirements. Next, the collected drawing samples undergo standardized preprocessing, such as image scaling, rotation correction (uniformly aligning horizontally to eliminate angular differences caused by drawing tilt), and grayscale normalization (normalizing pixel values ​​to the [0,1] range to eliminate brightness differences). This eliminates the impact of drawing format and visual effect differences on training. Semantic feature normalization is also performed, constructing an industrial terminology ontology containing multiple industrial-specific terms and synonym mappings (e.g., "KPC" uniformly mapped to "key special characteristics of the product"). Unit unification and format standardization are also implemented. Finally, precise annotation is carried out. For example, using an "automatic pre-annotation + manual review and correction" approach, each drawing sample is labeled with corresponding functional area labels, symbol labels, and line labels, ensuring that the label information completely matches the actual elements in the sample, forming a complete drawing training sample dataset.

[0082] S320. Input the drawing sample into the functional area recognition model, symbol recognition model and line recognition model respectively to obtain the functional area, symbol and line recognition results.

[0083] In practice, drawing samples can be extracted in batches from the completed drawing training sample dataset to ensure that each batch of samples covers different industries and different element types, thus ensuring the balance of training. Each drawing sample in a batch is then input into the functional area recognition model, symbol recognition model, and line recognition model to be trained. The three models simultaneously perform initial recognition. The functional area recognition model outputs the recognition results of the functional areas in the sample, the symbol recognition model outputs the recognition results of the symbols in the sample, and the line recognition model outputs the recognition results of the lines in the sample. The output information of all models is recorded to form the initial recognition result set.

[0084] S330. Based on the functional area recognition results, symbol recognition results, and line recognition results, as well as their corresponding labels, train the functional area recognition model, symbol recognition model, and line recognition model to obtain the trained functional area recognition model, symbol recognition model, and line recognition model.

[0085] In practice, the recognition results of the three models can be compared with their corresponding labels first. The errors between functional area recognition results and functional area labels, symbol recognition results and symbol labels, and line recognition results and line labels can be calculated. The error calculation covers dimensions such as element category recognition deviation and position recognition deviation. Based on the calculated errors, optimization algorithms such as stochastic gradient descent can be used to adjust the model parameters: for the functional area recognition model, the focus is on optimizing parameters related to region bounding box regression; for the symbol recognition model, the focus is on optimizing parameters related to symbol detail feature extraction; and for the line recognition model, the focus is on optimizing parameters related to line continuity recognition. Then, the process of "sample input - result output - error calculation - parameter adjustment" is repeated: continuous iterative training continues until the errors between the recognition results of the three models and their corresponding labels converge to a preset threshold, and the model recognition accuracy meets the requirements of industrial applications. Finally, the iteration is stopped and the trained model parameters are saved to obtain the trained functional area recognition model, symbol recognition model, and line recognition model, completing the entire training process.

[0086] To improve the recognition accuracy of small target elements in drawings, as examples, functional area recognition models, symbol recognition models, and line recognition models include attention networks and recognition networks. The attention network, an integrated feature enhancement network module within the recognition model, consists of channel attention sub-networks and spatial attention sub-networks. Its core function is to perform dual weighting processing on the drawing image in both channel and spatial dimensions, strengthening key element features, weakening invalid information, and improving the accuracy of subsequent recognition. The recognition network, the core recognition module in the three models, is the main network for classifying and locating drawing elements. Based on the input image features, it outputs the category and location information of the drawing elements, serving as the core carrier for the model to complete the recognition task.

[0087] The drawing to be recognized is input into a pre-trained functional area recognition model, symbol recognition model, and line recognition model, respectively, to identify the drawing elements and their position information. This can include: identifying the drawing elements and their position information through a recognition network; and performing dual attention enhancement on the drawing through an attention network.

[0088] As an example, an attention network may include a channel attention subnetwork and a spatial attention subnetwork. The attention network processes the drawing to be recognized in the following way: the channel attention subnetwork performs channel-dimension weighting on the drawing to be recognized to generate a first intermediate feature map; the spatial attention subnetwork performs spatial-dimension weighting on the first intermediate feature map to generate a second intermediate feature map, which is the output of the attention network.

[0089] In practice, after preprocessing, the drawing to be identified is first input into the model's recognition network. Following a pre-defined recognition logic, the network performs preliminary extraction and analysis of the drawing's global features, outputting the preliminary identified element categories (functional areas, symbols, lines) and the positional information of each element, forming the basic recognition results. Simultaneously, while the recognition network performs preliminary recognition, the attention network concurrently enhances the features of the input drawing.

[0090] As an example, a channel attention subnetwork is used to perform channel-dimensional weighted processing on the image to be recognized to generate a first intermediate feature map. This can include: performing global average pooling and global max pooling on the image to be recognized to obtain first pooling features and second pooling features; this process is used to extract global statistical features of the image in the channel dimension. Then, the first and second pooling features are input into a shared multilayer perceptron network to generate first and second channel weight vectors, respectively; the first and second channel weight vectors are then fused element-wise and processed through an activation function to obtain channel attention weights; the channel attention weights are then multiplied channel-wise with the image to be recognized. By weighting and enhancing the features of different channels, key channel features can be strengthened and ineffective channel features weakened, thus obtaining the first intermediate feature map.

[0091] As an example, generating a second intermediate feature map by weighting the first intermediate feature map in terms of spatial dimensions through a spatial attention sub-network can include: performing global average pooling and global max pooling on the first intermediate feature map in the channel dimension to obtain two two-dimensional spatial feature maps; concatenating the two two-dimensional spatial feature maps in the channel dimension to obtain a concatenated spatial feature map; inputting the concatenated spatial feature map into a convolutional layer for convolution processing and processing with an activation function to generate spatial attention weights; and multiplying the spatial attention weights element-wise with the first intermediate feature map to obtain the second intermediate feature map.

[0092] Through a two-layer sub-module of the attention network, the features of the drawing image are weighted and enhanced from both channel and spatial dimensions: the channel dimension focuses on "which feature channels are important," and the spatial dimension focuses on "which location areas are important." This dual enhancement allows the model to accurately capture features of easily missed elements such as small symbols, fine lines, and overlapping functional areas. The recognition network performs recognition based on the enhanced high-quality feature map, avoiding interference from invalid information and significantly improving recognition accuracy. This architectural design allows the three types of models to retain basic recognition capabilities while also possessing the ability to focus on key elements, effectively solving the industry pain points of missed key element detection and low recognition accuracy in traditional industrial drawing recognition, and ensuring data quality in subsequent element association construction and cross-verification stages.

[0093] To avoid missing small target drawing elements, when the size of the drawing to be identified is larger than a preset size threshold, a sliding window cutting strategy can be used to cut the drawing into multiple blocks. The overlap rate between the blocks is greater than a preset overlap threshold, so that functional areas, symbols, and lines are completely contained in at least one block. The drawing to be identified is then input into a pre-trained functional area recognition model, symbol recognition model, and line recognition model to identify the drawing elements and their position information. Specifically, the multiple blocks are input into the functional area recognition model, symbol recognition model, and line recognition model respectively for recognition to obtain the recognition results.

[0094] For example, the preset size threshold refers to a pre-set critical value for the size of the drawing. This value is determined based on the optimal input size of the recognition model and is the core basis for determining whether the drawing to be recognized needs to be cut. Drawings exceeding this threshold will trigger the cutting process, while drawings within the threshold can be directly input into the model for recognition. The sliding window cutting strategy refers to a processing method that uses a fixed-size window as a basis and slides across the drawing to be recognized area by area according to a preset step size to cut a large-size drawing into multiple equal-sized blocks. This is the core method for splitting large-size drawings. A block refers to a small-size image unit formed after a large-size drawing is cut by the sliding window. Its size matches the optimal input size of the recognition model and is the basic unit for the model to perform recognition. The overlap rate refers to the proportion of the area of ​​the overlapping region between two adjacent blocks to the total area of ​​a single block. This is the core control indicator for ensuring the integrity of drawing elements. The preset overlap threshold is a pre-set critical value for the overlap rate, ensuring that the overlapping area of ​​adjacent blocks is sufficient to cover the drawing elements that may be cut and split, so that each functional area, symbol, and line appears completely in at least one block.

[0095] In practice, the first step is to determine whether the drawing to be recognized needs to be cut. Specifically, the size of the drawing to be recognized (including pixel size and physical size) can be detected, and the detection result can be compared with a preset size threshold. If the size of the drawing to be recognized is less than or equal to the preset size threshold, the entire drawing is directly input into the model for recognition; if the size of the drawing to be recognized is greater than the preset size threshold, the sliding window cutting process is started.

[0096] When using a sliding window cutting strategy to cut drawings, the cutting parameters are first determined. Specifically, the fixed size of the blocks can be set based on the optimal input size of the recognition model. A preset overlap threshold (usually 20%-30%) is set based on the common sizes of drawing elements, and the window sliding step size is calculated based on the block size (step size = block size - block size × preset overlap threshold). Then, sliding cutting is performed. Specifically, the set block size can be used as the window size, and the calculated step size as the sliding distance. Starting from the upper left corner of the drawing to be recognized, the window slides horizontally and vertically region by region, sequentially capturing the corresponding area's image and generating multiple blocks. Finally, integrity verification is performed: all cut blocks are checked to ensure that each functional area, symbol, and line is completely contained within at least one block. If any element is split and does not appear completely in any block, the sliding step size is adjusted and the cutting is repeated until the integrity requirements are met.

[0097] All the cut blocks are arranged in order to form a block recognition queue, ensuring that each block can be processed one by one. Each block in the queue is then input into a pre-trained functional area recognition model, symbol recognition model, and line recognition model. The three models independently recognize each block and output the category and position information of the functional area, symbol, and line within the block. The recognition result of each block is recorded, and the corresponding block number and its position offset information in the original drawing are labeled to form a set of recognition results for each block, thus completing the cutting and recognition process for large-size drawings.

[0098] For large-size industrial drawing recognition scenarios, issues arise due to the high image resolution and limited receptive field of the model, leading to missed detections or low recognition accuracy for elements such as tiny symbols, fine lines, and small functional areas. A sliding window segmentation strategy is employed to divide drawings exceeding a threshold size into standardized blocks. Overlap rate control ensures the integrity of drawing elements. Then, a block-based recognition approach addresses the issues of missed detections and low recognition accuracy for small target elements in full-domain recognition of large-size drawings. This ensures that all types of drawing elements can be accurately recognized by the model, providing complete element recognition data for subsequent association construction and cross-verification.

[0099] To address the issue of the same drawing element being detected and labeled multiple times due to overlapping and cutting of drawing blocks, a non-maximum suppression (NMS) algorithm is needed to remove redundant detection boxes after obtaining the recognition results. NMS is a post-processing algorithm for target detection results. Its core logic is to suppress suboptimal detection boxes, retaining only the detection boxes with the highest built-in confidence within the same target region. It is a mainstream algorithm for solving the problems of duplicate detection and redundant labeling. Redundant detection boxes refer to detection boxes generated when the model repeatedly detects the same drawing element in different blocks due to overlapping and cutting of drawing blocks. These detection boxes point to the same element but have differences in position or confidence, and are invalid labels that need to be removed. A detection box is a rectangular label box output by the model after recognizing a drawing element. It contains the element's category, confidence level, and location information, and is the core carrier representing the position and category of the drawing element.

[0100] As an example, using a non-maximum suppression algorithm to remove redundant detection boxes in the recognition results can include: obtaining the category, confidence level, and location information of the detection box corresponding to each patch in the recognition results; sorting the detection boxes of the same category according to their confidence level, and using the detection boxes corresponding to the confidence levels that meet preset conditions as the baseline detection boxes; detecting the intersection-union ratio (IUR) of each detection box with the baseline detection box; determining the detection box as a redundant detection box if the IUR is greater than a preset threshold; and removing the redundant detection boxes based on the location information.

[0101] In practice, after the results are cut and identified from the large-size drawing, the detection box data corresponding to all blocks are first extracted and integrated to form a unified detection box information set. Each data item contains the element category, confidence value, and position information (coordinates x1, y1, x2, y2) corresponding to the detection box, ensuring that the information is complete and in a uniform format.

[0102] Next, the detection boxes are grouped according to their element categories, with functional area, symbol, and line detection boxes categorized separately to ensure that detection boxes of the same category are processed together. For each category of detection boxes, they are sorted from high to low confidence levels to clarify the confidence priority of each category. Confidence preset conditions are set (e.g., confidence level greater than 0.5), and detection boxes that meet the conditions are selected. The detection box with the highest confidence level is determined as the baseline detection box for that category. If there are multiple category branches, a corresponding baseline detection box is determined for each branch.

[0103] Then, using the baseline detection frame as a reference, the Intersection over Union (IoU) ratio of all other detection frames in this category with the baseline detection frame is calculated sequentially: IoU = overlapping area / total area of ​​the two frames. The calculated IoU ratio is compared with a preset threshold (usually set to 0.5-0.7). If the IoU ratio of a certain detection frame with the baseline detection frame is greater than the preset threshold, it is determined that the detection frame and the baseline detection frame point to the same drawing element and are marked as redundant detection frames. If the IoU ratio is less than or equal to the preset threshold, it is determined that the detection frame points to an independent drawing element and is retained as a valid detection frame.

[0104] Finally, the location information of all marked redundant detection boxes is extracted, their coordinate range in the original drawing is checked, and their overlapping area with the baseline detection box is confirmed. All redundant detection box data are removed from the detection box information set, and only the baseline detection boxes and valid detection boxes with an intersection-union ratio not exceeding the threshold under each category are retained. The remaining valid detection box data are integrated to form the final recognition result set after redundancy removal. In this result set, each drawing element corresponds to only one detection box, completing the entire non-maximum suppression process.

[0105] When large-size drawings are cut and overlapped using a sliding window, the same functional area, symbol, or line may appear in multiple overlapping blocks, causing the model to output multiple detection boxes for the same element. To address this, a non-maximum suppression algorithm is used to select the detection boxes with the highest confidence and most accurate location, eliminating redundant and duplicate detection boxes. This ensures that only one optimal detection box is retained for each drawing element, allowing the recognition results to accurately reflect the actual number and location of drawing elements. This prevents redundant detection boxes from interfering with subsequent association construction and cross-verification processes, improving the overall accuracy and usability of the recognition results.

[0106] The drawing recognition method provided in this application, after acquiring the drawing to be recognized, can, on the one hand, input the drawing into pre-trained functional area recognition models, symbol recognition models, and line recognition models for specialized recognition. Employing a separate model-specific training strategy, each model focuses on learning the features of its own target elements, avoiding recognition bias caused by multi-element feature confusion. Compared to single-model multi-element hybrid recognition methods, the recognition accuracy of each type of element is significantly improved. Simultaneously, the models are specifically optimized for industrial drawing recognition scenarios, accurately capturing features of easily missed elements such as micro-symbols, fine lines, and overlapping functional areas. This effectively solves the problems of missed and false recognition in traditional recognition methods and existing technologies, ensuring the completeness and accuracy of drawing element recognition. On the other hand, by combining the relationships between functional areas, symbols, and lines, the scattered single-element recognition results are transformed into associated data with industrial logic, realizing collaborative recognition of drawing information. Furthermore, based on the construction of cross-element relationships, bidirectional mutual verification is carried out for each set of structured data to be verified, which can effectively correct the misjudgment problem in the single model recognition stage, eliminate false relationships caused by model recognition deviations, and ensure that each set of verified relationships has actual drawing information support, fundamentally improving the reliability of drawing recognition results and reducing the workload of subsequent manual review.

[0107] Based on the drawing recognition method provided in the above embodiments, this application also provides specific implementations of the drawing recognition device. Please refer to the following embodiments.

[0108] First see Figure 4 The drawing recognition device 400 provided in this application embodiment is applied to an intelligent agent. The drawing recognition device 400 includes: an acquisition module 410 for acquiring the drawing to be recognized; an input module 420 for inputting the drawing to be recognized into a pre-trained functional area recognition model, a symbol recognition model, and a line recognition model to identify the drawing elements and position information of each drawing element, the drawing elements including: functional areas, symbols, and lines; a construction module 430 for constructing the association relationship between drawing elements based on the preset association relationship and position information between functional areas, symbols, and lines to obtain multiple structured data to be verified; a verification module 440 for performing mutual verification on the corresponding two drawing elements based on each structured data to be verified; and a determination module 450 for determining the drawing recognition result based on the drawing elements if the verification is successful.

[0109] In some possible implementations, the construction module 430 is also used to: detect whether a symbol or line is located in a preset functional area based on the association relationship and position information between the preset functional area, symbol, and line; and, if the symbol or line is located in the preset functional area, construct the association relationship between the symbol and functional area and the line and functional area to obtain multiple structured data to be verified.

[0110] In some possible implementations, the verification module 440 is further configured to: input the functional area and the first structured data to be verified obtained based on the symbol-functional area into the symbol recognition model, so that the symbol recognition model can identify whether there is a corresponding symbol in the functional area; if there is a corresponding symbol in the functional area, determine that the first structured data to be verified has passed the verification; or, input the functional area and the second structured data to be verified obtained based on the line-functional area into the line recognition model, so that the line recognition model can identify whether there is a corresponding line in the functional area; if there is a corresponding line in the functional area, determine that the second structured data to be verified has passed the verification.

[0111] In some possible implementations, the determining module 450 is also used to: determine that the first structured data to be verified has failed verification if there is no corresponding symbol in the functional area; or determine that the second structured data to be verified has failed verification if there is no corresponding line in the functional area; and, if the verification fails, distinguish and display the functional area and output alarm information.

[0112] In some possible implementations, the functional area recognition model, symbol recognition model, and line recognition model are trained as follows: A drawing training sample dataset is obtained, which includes drawing samples and their corresponding functional area labels, symbol labels, and line labels; the drawing samples are input into the functional area recognition model, symbol recognition model, and line recognition model respectively to obtain the functional area, symbol, and line recognition results; based on the functional area recognition results, symbol recognition results, and line recognition results, and their corresponding labels, the functional area recognition model, symbol recognition model, and line recognition model are trained to obtain the trained functional area recognition model, symbol recognition model, and line recognition model.

[0113] In some possible implementations, the functional area recognition model, symbol recognition model, and line recognition model include an attention network and a recognition network; the input module is further used to: identify the drawing elements of the drawing to be recognized and the positional information of each drawing element through the recognition network; and to perform dual attention enhancement on the drawing to be recognized through the attention network; the attention network includes: a channel attention subnetwork and a spatial attention subnetwork; the attention network processes the drawing to be recognized in the following way: the channel attention subnetwork performs channel-dimensional weighted processing on the drawing to be recognized to generate a first intermediate feature map; the spatial attention subnetwork performs spatial-dimensional weighted processing on the first intermediate feature map to generate a second intermediate feature map, which is the output of the attention network.

[0114] In some possible implementations, a channel attention subnetwork is used to perform channel-dimensional weighted processing on the drawing to be recognized to generate a first intermediate feature map. This includes: performing global average pooling and global max pooling on the drawing to be recognized to obtain a first pooling feature and a second pooling feature; inputting the first pooling feature and the second pooling feature into a shared multilayer perceptron network to generate a first channel weight vector and a second channel weight vector, respectively; fusing the first channel weight vector and the second channel weight vector element-wise and processing them through an activation function to obtain channel attention weights; and multiplying the channel attention weights with the drawing to be recognized channel-wise to obtain the first intermediate feature map.

[0115] In some possible implementations, a spatial attention subnetwork is used to perform spatial dimension weighting on the first intermediate feature map to generate a second intermediate feature map. This includes: performing global average pooling and global max pooling on the first intermediate feature map along the channel dimension to obtain two two-dimensional spatial feature maps; concatenating the two two-dimensional spatial feature maps along the channel dimension to obtain a concatenated spatial feature map; inputting the concatenated spatial feature map into a convolutional layer for convolution processing and then processing it through an activation function to generate spatial attention weights; and multiplying the spatial attention weights element-wise with the first intermediate feature map to obtain the second intermediate feature map.

[0116] In some possible implementations, the device further includes: a cutting module; the cutting module is used to cut the drawing to be identified into multiple blocks using a sliding window cutting strategy when the size of the drawing to be identified is greater than a preset size threshold; the overlap rate between the blocks is greater than a preset overlap threshold, so that the functional area, symbol, and line are at least completely contained in one block; the multiple blocks are respectively input into the functional area recognition model, the symbol recognition model, and the line recognition model for recognition, and the recognition result is obtained.

[0117] In some possible implementations, the apparatus further includes: a removal module; the removal module is used to remove redundant detection boxes in the recognition results using a non-maximum suppression algorithm.

[0118] In some possible implementations, the removal module is also used to: obtain the category, confidence level, and location information of the detection box corresponding to each patch in the recognition result; sort the detection boxes of the same category according to their confidence level, and take the detection boxes corresponding to the confidence level that meet the preset conditions as the reference detection boxes; detect the intersection-union ratio (IUR) of each detection box with the reference detection box; determine the detection box as a redundant detection box if the IUR is greater than a preset threshold; and remove the redundant detection boxes based on the location information.

[0119] In some possible implementations, the relationship between functional areas, symbols, and lines includes at least one of the following: the relationship between functional areas and symbols; the relationship between functional areas and lines; the relationship between functional areas; and the relationship between symbols.

[0120] The various modules of the drawing recognition device provided in this application embodiment can achieve Figure 1 It provides functions for each step of the drawing recognition method and achieves the corresponding technical effects. For the sake of brevity, it will not be described in detail here.

[0121] See Figure 5 The drawing recognition method described in the above embodiments can be further described in this embodiment of the invention as a drawing recognition device 500, which includes a processor 510 and a memory 520 storing computer program instructions; when the processor 510 executes the computer program instructions, it implements any of the drawing recognition methods described in the above embodiments.

[0122] The drawing recognition method in the above embodiments can be implemented using a computer storage medium. This computer storage medium stores computer program instructions; when these instructions are executed by a processor, they implement any of the drawing recognition methods described in the above embodiments.

[0123] This application also provides a computer program product, including a computer program, which, when executed, implements any of the drawing recognition methods described in the above embodiments.

[0124] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0125] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0126] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0127] The foregoing flowcharts and / or block diagrams describing the method for determining the open-circuit voltage of a battery, the battery management system, and the power-consuming device according to embodiments of this application have described various aspects of this application. It should be understood that each block in the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to create a machine such that these instructions, executable by the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowcharts and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by dedicated hardware performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0128] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application, and they should all be covered within the scope of the claims and specification of this application. In particular, as long as there is no structural conflict, the various technical features mentioned in the embodiments can be combined in any way. This application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A drawing recognition method, characterized in that, The method includes: Obtain the drawing to be identified; The drawing to be identified is input into a pre-trained functional area recognition model, symbol recognition model, and line recognition model, respectively, to identify the drawing elements of the drawing and the position information of each drawing element. The drawing elements include: functional areas, symbols, and lines. Based on the preset relationships between functional areas, symbols, and lines, and the location information, the relationships between the drawing elements are constructed to obtain multiple structured data to be verified. Based on each structured data to be verified, the corresponding two drawing elements are mutually verified. If the verification passes, the drawing recognition result is determined based on the drawing elements.

2. The method according to claim 1, characterized in that, Based on the preset relationships between functional areas, symbols, and lines, and the position information, the relationships between the drawing elements are constructed, resulting in multiple structured data to be verified, including: Based on the preset relationship between functional areas, symbols, and lines, and the position information, detect whether the symbols and lines are located in the preset functional areas; When symbols and lines are located in preset functional areas, the association between symbols and functional areas and between lines and functional areas is constructed to obtain multiple structured data to be verified.

3. The method according to claim 1, characterized in that, The mutual verification of the corresponding two drawing elements based on each structured data to be verified includes: The functional region and the first structured data to be verified obtained based on the symbol-functional region are input into the symbol recognition model so that the symbol recognition model can identify whether there is a corresponding symbol in the functional region; If the corresponding symbol exists in the functional area, it is determined that the first structured data to be verified has passed the verification. or, The functional area and the second structured data to be verified obtained based on the line-functional area are input into the line recognition model so that the line recognition model can identify whether there are corresponding lines in the functional area; If a corresponding line exists in the functional area, the second structured data to be verified is determined to have passed the verification.

4. The method according to claim 1, characterized in that, The functional area recognition model, the symbol recognition model, and the line recognition model include an attention network and a recognition network; The step of inputting the drawing to be identified into a pre-trained functional area recognition model, symbol recognition model, and line recognition model to identify the drawing elements and position information of each drawing element includes: By using a recognition network, the drawing elements of the drawing to be recognized and the position information of each drawing element are identified; The drawing to be identified is enhanced with dual attention through an attention network; The attention network includes a channel attention subnetwork and a spatial attention subnetwork; the attention network processes the drawing to be identified in the following manner: The drawing to be identified is weighted according to the channel dimension by a channel attention subnetwork to generate a first intermediate feature map; The first intermediate feature map is weighted by spatial dimensions using a spatial attention sub-network to generate a second intermediate feature map, which serves as the output of the attention network.

5. The method according to claim 4, characterized in that, The step of performing channel-dimensional weighted processing on the drawing to be identified through a channel attention subnetwork to generate a first intermediate feature map includes: The drawing to be identified is subjected to global average pooling and global max pooling respectively to obtain the first pooling feature and the second pooling feature; The first pooling feature and the second pooling feature are input into a shared multilayer perceptron network to generate a first channel weight vector and a second channel weight vector, respectively. The first channel weight vector and the second channel weight vector are added element by element and then fused together, and processed by an activation function to obtain the channel attention weights. The channel attention weights are multiplied channel by channel with the drawing to be identified to obtain the first intermediate feature map.

6. The method according to claim 4, characterized in that, The step of generating a second intermediate feature map by performing spatial dimension weighting processing on the first intermediate feature map through a spatial attention sub-network includes: The first intermediate feature map is subjected to global average pooling and global max pooling in the channel dimension to obtain two two-dimensional spatial feature maps. The two two-dimensional spatial feature maps are spliced ​​together along the channel dimension to obtain the spliced ​​spatial feature map. The stitched spatial feature map is input into a convolutional layer for convolution processing and then processed by an activation function to generate spatial attention weights. The spatial attention weights are multiplied element-wise with the first intermediate feature map to obtain the second intermediate feature map.

7. The method according to claim 1, characterized in that, The method further includes: If the size of the drawing to be identified is greater than a preset size threshold, a sliding window cutting strategy is used to cut the drawing into multiple blocks; the overlap rate between the blocks is greater than a preset overlap threshold, so that functional areas, symbols, and lines are at least completely contained in one block; The obtained multiple blocks are input into the functional area recognition model, the symbol recognition model, and the line recognition model respectively for recognition, and the recognition results are obtained.

8. The method according to claim 7, characterized in that, The method further includes: The nonmaximum suppression algorithm is used to remove redundant detection boxes in the recognition results.

9. The method according to any one of claims 1-8, characterized in that, The relationship between the functional area, symbol, and line includes at least one of the following: The relationship between functional areas and symbols; The relationship between functional areas and lines; The relationships between functional areas; The relationships between symbols.

10. A drawing recognition device, characterized in that, include: The acquisition module is used to acquire the drawing to be recognized; The input module is used to input the drawing to be identified into the pre-trained functional area recognition model, symbol recognition model and line recognition model respectively, and to identify the drawing elements of the drawing to be identified and the position information of each drawing element. The drawing elements include: functional areas, symbols and lines. The construction module is used to construct the association relationship between the drawing elements based on the preset association relationship between functional areas, symbols and lines and the position information, so as to obtain multiple structured data to be verified. The verification module is used to perform mutual verification between two corresponding drawing elements based on each structured data to be verified. The determination module is used to determine the drawing recognition result based on the drawing elements if the verification passes.

Images