Wiring diagram graph element identification method and device, computer device and storage medium

Abstract

The application discloses a kind of wiring diagram graph element identification method, device, computer equipment and storage medium, it is related to image processing technical field.The method comprises: according to wiring diagram graph file and its corresponding graph element category mark and graph element angle mark, target detection network is trained, and the target detection network after training is obtained;Based on multiple preset clipping scales, the picture to be detected is clipped, and multiple sub-pictures to be detected corresponding to each preset clipping scale are obtained;For each preset clipping scale, the sub-pictures to be detected are predicted by the target detection network after training, and the sub-picture detection result corresponding to each preset clipping scale is obtained;Based on enhanced weighted frame fusion algorithm, the sub-picture detection result corresponding to each preset clipping scale is fused, and the graph element detection result is obtained, to improve the wiring diagram graph element identification precision;And further utilize multi-scale template matching algorithm to detect frame correction, to improve the accuracy of target graph element detection result.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, computer device, and storage medium for identifying wiring diagram elements. Background Technology

[0002] In recent years, many deep learning-based detection and recognition technologies have been actively promoted and applied in the field of document and drawing inspection. Among the various recognition technologies, object detection networks are favored due to their fast detection speed and accurate recognition results. However, object detection networks have limited capabilities in locating and recognizing wiring diagram elements, which restricts the level of wiring diagram element recognition. Summary of the Invention

[0003] In view of this, the purpose of this invention is to overcome the shortcomings of the prior art and provide a wiring diagram element recognition method, apparatus, computer device and storage medium for using an enhanced weighted bounding box fusion algorithm to output precise rectangular regions of elements. The proposed method improves the ability to locate, detect and recognize elements of various categories in wiring diagrams, and further enhances the overall recognition ability of the system for wiring diagrams. In addition, the detection box is corrected by using a multi-scale template matching method, which improves the accuracy of the recognition results.

[0004] This invention provides the following technical solution: In a first aspect, the present invention proposes a method for identifying wiring diagram elements, comprising: Construct an object detection dataset based on the wiring diagram graphic file and its corresponding primitive category and primitive angle annotations; The target detection network is trained based on the target detection dataset to obtain the trained target detection network; The image to be detected is cropped based on multiple preset cropping scales to obtain multiple sub-images to be detected corresponding to each preset cropping scale. For each of the preset cropping scales, the trained target detection network predicts each of the sub-images to be detected, and obtains the sub-image detection results corresponding to each preset cropping scale; The detection results of the sub-images corresponding to each preset clipping scale are fused based on the enhanced weighted bounding box fusion algorithm to obtain the initial primitive detection results; The initial primitive detection results are corrected based on multi-scale template matching to obtain the target primitive detection results.

[0005] In one embodiment, the step of correcting the initial primitive detection result based on multi-scale template matching to obtain the target primitive detection result includes: Construct a primitive image template library based on the primitive category annotation and the primitive angle annotation; The template image corresponding to the initial primitive detection result is obtained from the primitive image template library; The initial primitive detection result is corrected by performing detection box correction based on the template image to obtain the target primitive detection result.

[0006] In one embodiment, the step of fusing the sub-image detection results corresponding to each preset clipping scale based on the enhanced weighted bounding box fusion algorithm to obtain the initial primitive detection result includes: Based on the sub-image detection results corresponding to each preset clipping scale, determine the sub-image detection results under all preset clipping scales corresponding to multiple primitive categories; For each of the aforementioned primitive categories, hierarchical clustering is performed based on the sub-image detection results under all the preset clipping scales corresponding to the primitive category to obtain the cluster label corresponding to each of the aforementioned primitive categories; For each cluster label, the original coordinate values ​​of multiple detection boxes corresponding to the cluster label are sorted in ascending order to obtain a detection box index sort; the detection box weights corresponding to the original coordinate values ​​of each detection box are sorted according to the detection box index sort to obtain a weight sequence. The fused coordinate values ​​are determined based on the weight sequence and the original coordinate values ​​of multiple detection boxes corresponding to the cluster labels; The fused coordinate values ​​are used as the initial primitive detection results.

[0007] In one embodiment, the weight sequence includes N bounding box weights, and the step of determining the fused coordinate values ​​based on the weight sequence and the original coordinate values ​​of multiple bounding boxes corresponding to the cluster labels includes: Calculate the total weight based on the weight sequence, and obtain the weight threshold based on the preset ratio and the total weight. The current cumulative weights of the first k detection boxes are calculated sequentially based on the weight sequence. If the current cumulative weight of the weights of the first k detection boxes is greater than or equal to the weight threshold, then the index of the detection box corresponding to the weight of the kth detection box is used as the index of the target detection box. The original coordinate values ​​of the detection box corresponding to the target detection box index are used as the fused coordinate values.

[0008] In one embodiment, the step of performing hierarchical clustering based on the detection results of all sub-images under the preset clipping scale corresponding to the primitive category to obtain cluster labels includes: Calculate the intersection-union ratio and the Euclidean distance between the detection box centers of all sub-image detection results under the preset clipping scale corresponding to the primitive category; The mixed similarity matrix is ​​calculated based on the intersection-union ratio and the Euclidean distance between the centers of the detection boxes. The formula for calculating the mixed similarity matrix is ​​as follows: In the formula, The mixed similarity matrix, and For hyperparameters, The intersection-union ratio is... The Euclidean distance is the center of the detection frame. Hierarchical clustering is performed based on the hybrid similarity matrix to obtain the cluster labels.

[0009] In one embodiment, the step of performing hierarchical clustering based on the mixed similarity matrix to obtain the cluster labels includes: Calculate the agglomeration distance based on the hybrid similarity matrix; Hierarchical clustering is performed based on the agglomeration distance to obtain the clustering results; The cluster labels are obtained based on the clustering results.

[0010] In one embodiment, the step of correcting the detection box based on the template image and the initial primitive detection result to obtain the target detection box result includes: The template image and the initial primitive detection results are preprocessed to obtain a binarized edge template image and a binary image to be detected; The binary image to be detected is scaled sequentially according to n preset scaling ratios to obtain the scaled image to be detected corresponding to each preset scaling ratio; wherein the scaling order is the sorting order of the n preset scaling ratios from largest to smallest. Determine whether the image size of the i-th scaled image to be detected is greater than or equal to the image size of the binarized edge template image, 1≤i≤n; If the image size of the i-th scaled image to be detected is greater than or equal to the image size of the binarized edge template image, then edge detection is performed on the i-th scaled image to be detected to generate the i-th edge detection image; The similarity response map between the i-th edge detection image and the binarized edge template image is calculated based on the normalized correlation matching method. Extract the current matching score from the similarity response map; Determine whether the current matching score is higher than the historical matching score; If the current matching score is lower than or equal to the historical matching score, then determine whether the image size of the (i+1)th scaled image to be detected is greater than or equal to the image size of the binarized edge template image; If the current matching score is higher than the historical matching score, then the matching position coordinates corresponding to the current matching score are taken as the current matching result; The target primitive detection result is obtained based on the current matching result.

[0011] Secondly, the present invention provides a wiring diagram element identification device, comprising: The module is used to build an object detection dataset based on the wiring diagram graphic file and its corresponding element category labeling and element angle labeling; The training module is used to train the object detection network based on the object detection dataset to obtain the trained object detection network. The processing module is used to crop the image to be detected based on multiple preset cropping scales to obtain multiple sub-images to be detected corresponding to each preset cropping scale; The prediction module is used to predict each of the sub-images to be detected through the trained target detection network for each of the preset cropping scales, so as to obtain the sub-image detection results corresponding to each preset cropping scale. The fusion module is used to fuse the detection results of the sub-images corresponding to each preset clipping scale based on the enhanced weighted box fusion algorithm to obtain the initial primitive detection results; The correction module is used to correct the initial primitive detection results based on multi-scale template matching to obtain the target primitive detection results.

[0012] Thirdly, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the wiring diagram element recognition method as described in the first aspect.

[0013] Fourthly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the wiring diagram element recognition method as described in the first aspect.

[0014] This invention discloses a wiring diagram element recognition method, apparatus, computer device, and storage medium. The method involves constructing a target detection dataset based on a wiring diagram graphic file and its corresponding element category and angle annotations; training a target detection network using the target detection dataset to obtain a trained target detection network; cropping the image to be detected based on multiple preset cropping scales to obtain multiple sub-images to be detected corresponding to each preset cropping scale; predicting each sub-image to be detected using the trained target detection network for each preset cropping scale to obtain a sub-image detection result corresponding to each preset cropping scale; fusing the sub-image detection results corresponding to each preset cropping scale using an enhanced weighted bounding box fusion algorithm to obtain an initial element detection result; and correcting the initial element detection result using multi-scale template matching to obtain the target element detection result. In this way, by covering targets of different sizes through multi-scale cropping, the enhanced weighted bounding box fusion algorithm balances the detection results of different scales through weight allocation, avoiding large-scale sub-images from suppressing small-scale detections and improving the accuracy of wiring diagram primitive recognition; and further, the multi-scale template matching algorithm is used to correct the detection boxes to improve the accuracy of target primitive detection results. Attached Figure Description

[0015] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope of protection of the present invention. In the various drawings, similar components are numbered similarly.

[0016] Figure 1 A flowchart illustrating the wiring diagram element identification method proposed in this embodiment is shown. Figure 2 Another flowchart of the wiring diagram element recognition method proposed in this embodiment is shown; Figure 3 This illustration shows another flowchart of the wiring diagram element recognition method proposed in this embodiment; Figure 4 This diagram illustrates another step in the wiring diagram element identification method proposed in this embodiment. Figure 5 This illustration shows another flowchart of the wiring diagram element identification method proposed in this embodiment; Figure 6 A schematic diagram of the wiring diagram element identification device proposed in this embodiment is shown.

[0017] Explanation of reference numerals in the attached diagram: 600-Wiring diagram element recognition device; 601-Construction module; 602-Training module; 603-Processing module; 604-Prediction module; 605-Fusion module; 606-Correction module. Detailed Implementation

[0018] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0019] The components of the embodiments of the invention described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0020] In the following, the terms “comprising,” “having,” and their cognates, which may be used in various embodiments of the invention, are intended only to indicate a particular feature, number, step, operation, element, component, or combination thereof, and should not be construed as excluding, firstly, the presence of one or more other features, numbers, steps, operations, elements, components, or combinations thereof, or adding the possibility of one or more features, numbers, steps, operations, elements, components, or combinations thereof.

[0021] Furthermore, the terms "first," "second," and "third" are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.

[0022] Unless otherwise specified, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of the invention pertain. Terms (such as those defined in commonly used dictionaries) shall be interpreted as having the same meaning as in their contextual meaning in the relevant technical field and shall not be interpreted as having an idealized or overly formal meaning, unless clearly defined in the various embodiments of the invention.

[0023] Example 1 This disclosure provides a wiring diagram element recognition method, which uses an enhanced weighted bounding box fusion algorithm to output precise rectangular regions of elements. The proposed method improves the ability to locate, detect, and recognize elements of various categories in wiring diagrams, further enhancing the system's overall recognition capability for wiring diagrams. In addition, a multi-scale template matching method is used to correct the detection boxes, improving the accuracy of the recognition results.

[0024] Please see Figure 1 The wiring diagram element identification method includes steps S101 to S105, and each step is described in detail below.

[0025] Step S101: Construct an object detection dataset based on the wiring diagram graphic file and its corresponding element category labeling and element angle labeling.

[0026] In this embodiment, wiring diagram graphic files containing various graphic element types are obtained. As shown in Table 1, some graphic element categories are used. The labelImg annotation tool is used to annotate the graphic element category and angle corresponding to each graphic element in the wiring diagram graphic file. Then, a target detection dataset is constructed based on the wiring diagram graphic file and its corresponding graphic element category and angle annotations. Specifically, the original wiring diagram image is cropped into smaller sub-images using a sliding window cropping technique at a certain ratio, for example, by 1 / 3, 1 / 5, and 1 / 7 of the original image's width and height, resulting in approximately 10,000 cropped sub-images. Finally, these sub-images are randomly divided into training and validation sets at an 8:2 ratio.

[0027] Table 1:

[0028] Step S102: Train the target detection network based on the target detection dataset to obtain the trained target detection network.

[0029] In this embodiment, the target detection dataset is trained using a target network algorithm based on deep learning. Selectable target detection networks include Faster R-CNN, Cascade R-CNN, YOLO series, etc.

[0030] The specific training process includes: data augmentation of the object detection data (random rotation, flipping, scaling); modification of the number of output classes in the model, i.e., the number of labeled classes; configuration of the loss function of Cascade R-CNN, for example, classification loss for object detection class prediction and bounding box regression loss for adjusting the position of bounding boxes; setting hyperparameters, for example, a learning rate of 0.001, which is gradually adjusted during training, and selecting an appropriate value for the batch size based on the available GPU memory (e.g., 8, 16, 32), using the Adam optimizer, and setting the number of training epochs to 24; model evaluation metrics include mean accuracy (mAP) for evaluating primitive detection performance, intersection-over-union ratio (IoU) for evaluating the degree of overlap between the detection results and the ground truth boxes, and for evaluating the accuracy of the detection results; and model tuning: adjusting the model network parameters based on the validation results during training to further improve the detection accuracy.

[0031] Step S103: The image to be detected is cropped based on multiple preset cropping scales to obtain multiple sub-images to be detected corresponding to each preset cropping scale.

[0032] In this embodiment, the image to be detected is cropped according to multiple preset image cropping scales in the target detection dataset to obtain multiple sub-images to be detected corresponding to each preset cropping scale. These preset image cropping scales may include a first scale, a second scale, and a third scale, with the first, second, and third scales decreasing sequentially. Smaller scales can be used to capture microscopic features and address the problem of "missed detections due to insufficient pixels," medium scales can be used to balance the relationship between the target and its context, and larger scales can be used to maintain the integrity of the topological structure.

[0033] Step S104: For each of the preset cropping scales, the trained target detection network is used to predict each of the sub-images to be detected, and the sub-image detection results corresponding to each preset cropping scale are obtained.

[0034] In this embodiment, for each preset cropping scale, the trained object detection network predicts each sub-image to be detected, obtaining the sub-image detection results corresponding to each sub-image under each preset cropping scale. The detection results for the same primitive will differ for different preset cropping scales.

[0035] Step S105: The detection results of the sub-images corresponding to each preset clipping scale are fused based on the enhanced weighted bounding box fusion algorithm to obtain the initial primitive detection results.

[0036] In this embodiment, the detection results of subgraphs corresponding to each preset clipping scale are fused based on the enhanced weighted bounding box fusion algorithm to obtain the initial primitive detection results, thereby realizing the fusion of subgraph detection results at large, medium and small scales, reconstructing the global complex shape, and improving the recognition accuracy of wiring diagram primitives.

[0037] Please see Figure 2 In one specific embodiment, step S105 includes steps S1051 to S1054, and each step is described in detail below.

[0038] Step S1051: Based on the sub-image detection results corresponding to each preset clipping scale, determine the sub-image detection results for each of the multiple primitive categories under all the preset clipping scales.

[0039] In this embodiment, the sub-image detection results corresponding to each preset clipping scale are divided according to the primitive category to obtain the sub-image detection results under each preset clipping scale corresponding to each primitive category, thereby providing a basis for subsequent fusion of the same primitive detection results.

[0040] Step S1052: For each of the primitive categories, hierarchical clustering is performed based on the sub-image detection results under all the preset clipping scales corresponding to the primitive category to obtain the cluster label corresponding to each primitive category.

[0041] In this embodiment, the detection results are processed independently for each primitive category. Specifically, for the detection results of sub-images at all preset clipping scales corresponding to each primitive category, cross-scale hierarchical clustering is performed to obtain the cluster labels corresponding to each primitive category.

[0042] Please see Figure 3 In one specific embodiment, step S1052 includes steps S301 to S303, and each step is described in detail below.

[0043] Step S301: Calculate the intersection-union ratio and the Euclidean distance between the detection box centers of all sub-image detection results under the preset clipping scale corresponding to the primitive category.

[0044] In this embodiment, the intersection-union ratio (CIRR) and the Euclidean distance between the detection box centers are calculated for the sub-image detection results under all preset clipping scales corresponding to the primitive category. The CIRR is used to measure the degree of overlap of the detection boxes, and the Euclidean distance between the detection box centers is used to measure the spatial distance between the center points of the detection boxes.

[0045] Step S302: Calculate the hybrid similarity matrix based on the intersection-union ratio and the Euclidean distance between the centers of the detection boxes.

[0046] In this embodiment, based on the intersection-union ratio Euclidean distance from the center of the detection frame Calculate the mixed similarity matrix The formula for calculating the mixed similarity matrix is: In the formula, and The hyperparameters are used to simultaneously consider overlapping regions and center positions by mixing similarity matrices, providing a more robust feature representation for hierarchical clustering and significantly improving the fusion quality of multi-scale detection boxes.

[0047] Step S303: Perform hierarchical clustering based on the hybrid similarity matrix to obtain the cluster labels.

[0048] In this embodiment, the detection boxes are gradually merged into clusters by mixing similarity matrices to obtain cluster labels.

[0049] In one specific embodiment, step S303 includes: calculating the agglomeration distance based on the mixed similarity matrix; performing hierarchical clustering based on the agglomeration distance to obtain clustering results; and obtaining the cluster label based on the clustering results.

[0050] In this embodiment, the agglomeration distance is calculated based on the hybrid similarity matrix; hierarchical clustering is performed based on the agglomeration distance to obtain the clustering results; and cluster labels are obtained based on the clustering results.

[0051] Specifically, based on " "", "Extract the indices of the upper triangular portion (excluding the diagonal) of the similarity matrix to avoid repeatedly calculating the distance," "Convert the similarity matrix into cohesive distance; the higher the similarity, the smaller the distance." It is a one-dimensional array containing the aggregation distance of all pairs of detection boxes.

[0052] Furthermore, based on " ", the compressed cohesion distance matrix Input to hierarchical clustering function ， use Method (i.e.) The algorithm performs hierarchical clustering to obtain the clustering results. Z .

[0053] Furthermore, based on " " according to Z The clustering results are used to assign the detection boxes to different clusters: This means that during the merging process, merging stops when the distance between two clusters exceeds a threshold t, resulting in a flattened cluster. For example, if... Then the distance threshold t= 0.5, only The detection boxes will be grouped into the same cluster. Finally, an array of length N is output. , which represents the cluster label (an integer starting from 1) for each detection box.

[0054] Step S1053: For each cluster label, sort the original coordinate values ​​of multiple detection boxes corresponding to the cluster label in ascending order to obtain the detection box index sort; sort the detection box weights corresponding to the original coordinate values ​​of each detection box according to the detection box index sort to obtain the weight sequence.

[0055] In this embodiment, for each cluster label, the original values ​​of the coordinates of multiple detection boxes corresponding to the cluster label are ( Sort in ascending order. ), to obtain the sorted index of the detection boxes ( ); sorted by detection box index ( The weights of the detection boxes corresponding to the original coordinate values ​​of each detection box are sorted to obtain a weight sequence. The detection box weight is the product of the scale weight of the detection box at the corresponding scale and the score of the detection box at the current scale.

[0056] Step S1054: Determine the fused coordinate value based on the weight sequence and the original coordinate values ​​of multiple detection boxes corresponding to the cluster label; use the fused coordinate value as the primitive detection result.

[0057] In this embodiment, a fused coordinate value is determined based on the original coordinate values ​​of multiple detection boxes corresponding to the weight sequence and cluster labels, and the fused coordinate value is used as the primitive detection result. The fused coordinate value is the weighted median of the original coordinate values ​​of the multiple detection boxes obtained from the weight sequence.

[0058] It should be noted that hierarchical clustering is used, combined with IOU The hybrid similarity metric method using the center distance reduces false merging and improves primitive detection performance in dense scenes. At the same time, the use of weighted median avoids using only the weighted score of the detection box, reduces the impact of outlier boxes on merging, and makes the fusion result insensitive to outlier boxes.

[0059] It should be further explained that the score of the detection box corresponding to the current cluster is obtained, and the sum of the detection box weights of the current cluster is also obtained. The fused score C is then calculated based on the detection box scores and the sum of the detection box weights of the current cluster. In the formula, N The number of detection boxes, The score for each detection box, with a corresponding weight, is... The fused score C can be used for filtering and fusion between different primitive categories.

[0060] In one specific embodiment, step S1054 includes: calculating a weight sum based on the weight sequence; obtaining a weight threshold based on a preset ratio and the weight sum; calculating the current cumulative weight of the first k detection box weights sequentially based on the weight sequence; if the current cumulative weight of the first k detection box weights is greater than or equal to the weight threshold, then using the detection box index corresponding to the kth detection box weight as the target detection box index; and using the original value of the detection box coordinates corresponding to the target detection box index as the fused coordinate value.

[0061] In this embodiment, the weight sum is calculated based on the weight sequence, and a weight threshold is obtained based on a preset ratio and the weight sum. The preset ratio can be 1 / 2, in which case the weight threshold is calculated. The calculation formula is: , Let be the weight of the i-th detection box in the weight sequence.

[0062] Furthermore, the current cumulative weights of the first k detection boxes are calculated sequentially based on the weight sequence. .

[0063] Furthermore, find the smallest detection box index k such that the current cumulative weight of the weights of the first k detection boxes is greater than or equal to the weight threshold, and use the detection box index corresponding to the weight of the kth detection box as the target detection box index, and use the original value of the detection box coordinates corresponding to the target detection box index as the fused coordinate value.

[0064] Please see again Figure 1 Step S106: Correct the initial primitive detection result based on multi-scale template matching to obtain the target primitive detection result.

[0065] In this embodiment, using the template corresponding to the initial primitive detection result, matching at different scales is performed near the candidate region provided by the initial primitive detection result to find the position and scale most similar to the template, thereby correcting the positional deviation, size deviation, or false detection / missed detection that may exist in the initial primitive detection result, and obtaining a more accurate and robust target primitive detection result.

[0066] Please see Figure 4 In one specific embodiment, step S106 includes steps S1061 to S1063, and each step is described in detail below.

[0067] Step S1061: Construct a graphic element image template library based on the graphic element category label and the graphic element angle label.

[0068] In this embodiment, a primitive image template library is constructed based on primitive category annotation and primitive angle annotation. The annotation category is used as the index, and the primitive bounding rectangle clipping image of the corresponding annotation rectangle is used as the value of the index.

[0069] Step S1062: Obtain the template image corresponding to the initial primitive detection result from the primitive image template library.

[0070] In this embodiment, the template image corresponding to the primitive detection result is found in the primitive image template library.

[0071] Step S1063: Correct the detection box based on the template image and the initial primitive detection result to obtain the target primitive detection result.

[0072] In this embodiment, by combining prior knowledge of the template with the initial primitive detection results, detection box correction can significantly improve the accuracy of the detection results, especially in scenarios where the target is deformed or occluded.

[0073] In one specific embodiment, step S1063 includes: preprocessing the template image and the initial primitive detection result to obtain a binarized edge template image and a binary image to be detected; scaling the binary image to be detected sequentially according to n preset scaling ratios to obtain scaled images to be detected corresponding to each preset scaling ratio; wherein the scaling order is the sorting order of the n preset scaling ratios from largest to smallest; determining whether the image size of the i-th scaled image to be detected is greater than or equal to the image size of the binarized edge template image, 1≤i≤n; if the image size of the i-th scaled image to be detected is greater than or equal to the image size of the binarized edge template image, then scaling the i-th scaled image to be detected... Edge detection is performed on the measured image to generate the i-th edge detection image; a similarity response map between the i-th edge detection image and the binarized edge template image is calculated based on the normalized correlation matching method; the current matching score of the similarity response map is extracted; it is determined whether the current matching score is higher than the historical matching score; if the current matching score is lower than or equal to the historical matching score, it is determined whether the image size of the (i+1)-th scaled detection image is greater than or equal to the image size of the binarized edge template image; if the current matching score is higher than the historical matching score, the matching position coordinates corresponding to the current matching score are taken as the current matching result; the target primitive detection result is obtained based on the current matching result.

[0074] In this embodiment, the colored template image is converted into a binary template image, and edge detection is performed on the binary template image to generate a binarized edge template image. The height and width of the binarized edge template image need to be recorded. Simultaneously, the initial primitive detection results are binarized to obtain a binary image to be detected, and an initial matching score is generated.

[0075] The binary image to be detected is sequentially scaled according to n preset scaling ratios to obtain the scaled image to be detected corresponding to each preset scaling ratio; wherein, the scaling order is the sorting order of the n preset scaling ratios from largest to smallest. Larger scaling ratios are prioritized to improve efficiency.

[0076] Determine whether the image size of the i-th scaled image to be detected is greater than or equal to the image size of the binarized edge template image, where 1 ≤ i ≤ n.

[0077] If the image size of the i-th scaled image to be detected is greater than or equal to the image size of the binarized edge template image, then edge detection is performed on the i-th scaled image to be detected to generate the i-th edge detection image.

[0078] The similarity response map between the i-th edge detection image and the binarized edge template image is calculated based on the normalized correlation matching method; the current matching score of the similarity response map is extracted.

[0079] Determine if the current match score is higher than the historical match score.

[0080] If the current matching score is lower than or equal to the historical matching score, then the current matching score is used as the historical matching score, and it is determined whether the image size of the (i+1)th scaled image to be detected is greater than or equal to the image size of the binarized edge template image.

[0081] If the current match score is higher than the historical match scores, then the coordinates of the match position corresponding to the current match score will be used as the current match result. maxLoc And map the current matching result from the scaled image coordinate system back to the original image coordinate system: Corrected primitive bounding box top-left corner coordinates: The coordinates of the lower right corner of the corrected element frame: ,in r The scale ratio r is the ratio of the original primitive detection result to the size of the scaled image to be detected.

[0082] For example, please see Figure 5The original wiring diagram of the plant has a high resolution, and the graphic elements occupy a small proportion of the diagram. The original wiring diagram is cropped into multiple sub-diagrams to be detected according to a preset cropping scale. These sub-diagrams are then detected by a target detection network to obtain the detection results of multiple sub-diagrams corresponding to the original wiring diagram. The detection results of multiple sub-diagrams are then fused using an enhanced weighted bounding box fusion algorithm to obtain the target detection model detection result as the initial graphic element detection result. At this point, the detection box for the surge arrester area is too large, covering short connecting lines, so the detection box needs to be corrected. Based on the graphic element categories and angles detected in the initial graphic element detection results, template images of corresponding graphic element templates are extracted from the graphic element template image library. Furthermore, the size of the graphic element detection boxes in the initial graphic element detection results is adjusted based on a multi-scale template matching algorithm to obtain the target graphic element detection result. The background color of the graphic element box area is set, and the connecting lines are binarized. Finally, the final image rendering (CIM / G) file is obtained.

[0083] The wiring diagram primitive recognition method proposed in this embodiment constructs a target detection dataset based on the wiring diagram graphic file and its corresponding primitive category and angle annotations; trains a target detection network based on the target detection dataset to obtain a trained target detection network; crops the image to be detected based on multiple preset cropping scales to obtain multiple sub-images to be detected corresponding to each preset cropping scale; predicts each sub-image to be detected using the trained target detection network for each preset cropping scale to obtain the sub-image detection result corresponding to each preset cropping scale; fuses the sub-image detection results corresponding to each preset cropping scale based on an enhanced weighted bounding box fusion algorithm to obtain an initial primitive detection result; and corrects the initial primitive detection result based on multi-scale template matching to obtain the target primitive detection result. In this way, multi-scale cropping covers targets of different sizes, and the enhanced weighted bounding box fusion algorithm balances the detection results of different scales through weight allocation, avoiding large-scale sub-images suppressing small-scale detections and improving the accuracy of wiring diagram primitive recognition; further, the multi-scale template matching algorithm is used for detection box correction to improve the accuracy of the target primitive detection result.

[0084] Example 2 Furthermore, this disclosure provides a wiring diagram element recognition device 600, please refer to [link to relevant documentation]. Figure 6 ,include: Module 601 is used to construct an object detection dataset based on the wiring diagram graphic file and its corresponding element category annotation and element angle annotation; The training module 602 is used to train the object detection network based on the object detection dataset to obtain the trained object detection network. Processing module 603 is used to crop the image to be detected based on multiple preset cropping scales to obtain multiple sub-images to be detected corresponding to each preset cropping scale; Prediction module 604 is used to predict each of the sub-images to be detected through the trained target detection network for each of the preset cropping scales, so as to obtain the sub-image detection result corresponding to each preset cropping scale. The fusion module 605 is used to fuse the sub-image detection results corresponding to each preset clipping scale based on the enhanced weighted box fusion algorithm to obtain the initial primitive detection results.

[0085] The correction module 606 is used to correct the initial primitive detection result based on multi-scale template matching to obtain the target primitive detection result.

[0086] Optionally, the correction module 606 is further configured to construct a primitive image template library based on the primitive category label and the primitive angle label; obtain a template image corresponding to the primitive detection result in the primitive image template library; and perform detection box correction on the initial primitive detection result based on the template image to obtain the target primitive detection result.

[0087] Optionally, the fusion module 605 is further configured to: determine, based on the sub-image detection results corresponding to each preset clipping scale, all sub-image detection results under the preset clipping scales corresponding to multiple primitive categories; perform hierarchical clustering for each primitive category based on the sub-image detection results under the preset clipping scales corresponding to the primitive category to obtain cluster labels corresponding to each primitive category; sort the original values ​​of multiple detection box coordinates corresponding to each cluster label in ascending order to obtain a detection box index sort; sort the detection box weights corresponding to the original values ​​of each detection box coordinate based on the detection box index sort to obtain a weight sequence; determine fused coordinate values ​​based on the weight sequence and the original values ​​of multiple detection box coordinates corresponding to the cluster labels; and use the fused coordinate values ​​as the initial primitive detection results.

[0088] Optionally, the fusion module 605 is further configured to calculate a weight sum based on the weight sequence, obtain a weight threshold based on a preset ratio and the weight sum; calculate the current cumulative weight of the weights of the first k detection boxes sequentially based on the weight sequence; if the current cumulative weight of the weights of the first k detection boxes is greater than or equal to the weight threshold, then the detection box index corresponding to the weight of the kth detection box is used as the target detection box index; and the original value of the detection box coordinates corresponding to the target detection box index is used as the fused coordinate value.

[0089] Optionally, the fusion module 605 is further configured to calculate the intersection-union ratio (IU) and the Euclidean distance between the detection box centers of all sub-images under the preset clipping scale corresponding to the primitive category; and to calculate a hybrid similarity matrix based on the IU and the Euclidean distance between the detection box centers, wherein the formula for calculating the hybrid similarity matrix is: In the formula, The mixed similarity matrix, and For hyperparameters, The intersection-union ratio is... The Euclidean distance to the center of the detection box is used; hierarchical clustering is performed based on the hybrid similarity matrix to obtain the cluster label.

[0090] Optionally, the fusion module 605 is further configured to calculate the agglomeration distance based on the hybrid similarity matrix; perform hierarchical clustering based on the agglomeration distance to obtain clustering results; and obtain the cluster label based on the clustering results.

[0091] Optionally, the correction module 606 is further configured to preprocess the template image and the initial primitive detection result to obtain a binarized edge template image and a binary image to be detected; to scale the binary image to be detected sequentially according to n preset scaling ratios to obtain scaled images to be detected corresponding to each preset scaling ratio; wherein the scaling order is the sorting order of the n preset scaling ratios from largest to smallest; to determine whether the image size of the i-th scaled image to be detected is greater than or equal to the image size of the binarized edge template image, 1≤i≤n; if the image size of the i-th scaled image to be detected is greater than or equal to the image size of the binarized edge template image, then the i-th scaled image to be detected is... For example, edge detection is performed to generate the i-th edge to be detected image; a similarity response map between the i-th edge to be detected image and the binarized edge template image is calculated based on the normalized correlation matching method; the current matching score of the similarity response map is extracted; it is determined whether the current matching score is higher than the historical matching score; if the current matching score is lower than or equal to the historical matching score, it is determined whether the image size of the (i+1)-th scaled to be detected image is greater than or equal to the image size of the binarized edge template image; if the current matching score is higher than the historical matching score, the matching position coordinates corresponding to the current matching score are taken as the current matching result; the target primitive detection result is obtained based on the current matching result.

[0092] The apparatus provided in this embodiment can perform the steps of the wiring diagram element identification method provided in Embodiment 1. To avoid repetition, it will not be described again.

[0093] The wiring diagram primitive recognition device proposed in this embodiment constructs a target detection dataset based on the wiring diagram graphic file and its corresponding primitive category and angle annotations; trains a target detection network based on the target detection dataset to obtain a trained target detection network; crops the image to be detected based on multiple preset cropping scales to obtain multiple sub-images to be detected corresponding to each preset cropping scale; predicts each sub-image to be detected using the trained target detection network for each preset cropping scale to obtain the sub-image detection result corresponding to each preset cropping scale; fuses the sub-image detection results corresponding to each preset cropping scale based on an enhanced weighted bounding box fusion algorithm to obtain an initial primitive detection result; and corrects the initial primitive detection result based on multi-scale template matching to obtain the target primitive detection result. In this way, multi-scale cropping covers targets of different sizes, and the enhanced weighted bounding box fusion algorithm balances the detection results of different scales through weight allocation, avoiding large-scale sub-images suppressing small-scale detections and improving the accuracy of wiring diagram primitive recognition; further, the multi-scale template matching algorithm is used for detection box correction to improve the accuracy of the target primitive detection result.

[0094] Example 3 Furthermore, this disclosure provides a computer device including a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements the wiring diagram element recognition method described in Embodiment 1.

[0095] The device provided in this embodiment can perform the steps of the wiring diagram element identification method provided in Embodiment 1. To avoid repetition, it will not be described again.

[0096] Example 4 This disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the wiring diagram element recognition method described in Embodiment 1.

[0097] In this embodiment, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0098] The computer-readable storage medium provided in this embodiment can implement the wiring diagram element recognition method provided in Embodiment 1. To avoid repetition, it will not be described again here.

[0099] In all examples shown and described herein, any specific values ​​should be interpreted as merely exemplary and not as limitations; therefore, other examples of exemplary embodiments may have different values.

[0100] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0101] The above-described embodiments are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A method for identifying wiring diagram elements, characterized in that, include: Construct an object detection dataset based on the wiring diagram graphic file and its corresponding primitive category and primitive angle annotations; The target detection network is trained based on the target detection dataset to obtain the trained target detection network; The image to be detected is cropped based on multiple preset cropping scales to obtain multiple sub-images to be detected corresponding to each preset cropping scale. For each of the preset cropping scales, the trained target detection network predicts each of the sub-images to be detected, and obtains the sub-image detection results corresponding to each preset cropping scale; The detection results of the sub-images corresponding to each preset clipping scale are fused based on the enhanced weighted bounding box fusion algorithm to obtain the initial primitive detection results; The initial primitive detection results are corrected based on multi-scale template matching to obtain the target primitive detection results. The enhanced weighted bounding box fusion algorithm is used to fuse the sub-image detection results corresponding to each preset clipping scale to obtain the initial primitive detection results, including: Based on the sub-image detection results corresponding to each preset clipping scale, determine the sub-image detection results under all preset clipping scales corresponding to multiple primitive categories; For each of the aforementioned primitive categories, hierarchical clustering is performed based on the sub-image detection results under all the preset clipping scales corresponding to the primitive category to obtain the cluster label corresponding to each of the aforementioned primitive categories; For each cluster label, the original coordinate values ​​of multiple detection boxes corresponding to the cluster label are sorted in ascending order to obtain a detection box index sort; the detection box weights corresponding to the original coordinate values ​​of each detection box are sorted according to the detection box index sort to obtain a weight sequence. The fused coordinate values ​​are determined based on the weight sequence and the original coordinate values ​​of multiple detection boxes corresponding to the cluster labels; the fused coordinate values ​​are used as the initial primitive detection results. The step of performing hierarchical clustering based on the sub-image detection results at all preset clipping scales corresponding to the primitive category to obtain cluster labels corresponding to each primitive category includes: Calculate the intersection-union ratio and the Euclidean distance between the detection box centers of all sub-image detection results under the preset clipping scale corresponding to the primitive category; The mixed similarity matrix is ​​calculated based on the intersection-union ratio and the Euclidean distance between the centers of the detection boxes. The formula for calculating the mixed similarity matrix is ​​as follows: In the formula, The mixed similarity matrix, and For hyperparameters, The intersection-union ratio is... The Euclidean distance is the center of the detection frame. Hierarchical clustering is performed based on the hybrid similarity matrix to obtain the cluster labels.

2. The wiring diagram element recognition method according to claim 1, characterized in that, The weight sequence includes N detection box weights. Determining the fused coordinate values ​​based on the weight sequence and the original coordinate values ​​of multiple detection boxes corresponding to the cluster labels includes: Calculate the total weight based on the weight sequence, and obtain the weight threshold based on the preset ratio and the total weight. The current cumulative weights of the first k detection boxes are calculated sequentially based on the weight sequence. If the current cumulative weight of the weights of the first k detection boxes is greater than or equal to the weight threshold, then the index of the detection box corresponding to the weight of the kth detection box is used as the index of the target detection box. The original coordinate values ​​of the detection box corresponding to the target detection box index are used as the fused coordinate values.

3. The wiring diagram element recognition method according to claim 1, characterized in that, The step of performing hierarchical clustering based on the mixed similarity matrix to obtain the cluster labels includes: Calculate the agglomeration distance based on the hybrid similarity matrix; Hierarchical clustering is performed based on the agglomeration distance to obtain the clustering results; The cluster labels are obtained based on the clustering results.

4. The wiring diagram element recognition method according to claim 1, characterized in that, The step of correcting the initial primitive detection result based on multi-scale template matching to obtain the target primitive detection result includes: Construct a primitive image template library based on the primitive category annotation and the primitive angle annotation; The template image corresponding to the primitive detection result is obtained from the primitive image template library; The initial primitive detection result is corrected by performing detection box correction based on the template image to obtain the target primitive detection result.

5. The wiring diagram element recognition method according to claim 4, characterized in that, The step of correcting the detection bounding box of the initial primitive detection result based on the template image to obtain the target primitive detection result includes: The template image and the initial primitive detection results are preprocessed to obtain a binarized edge template image and a binary image to be detected; The binary image to be detected is scaled sequentially according to n preset scaling ratios to obtain the scaled image to be detected corresponding to each preset scaling ratio; wherein the scaling order is the sorting order of the n preset scaling ratios from largest to smallest. Determine whether the image size of the i-th scaled image to be detected is greater than or equal to the image size of the binarized edge template image, 1≤i≤n; If the image size of the i-th scaled image to be detected is greater than or equal to the image size of the binarized edge template image, then edge detection is performed on the i-th scaled image to be detected to generate the i-th edge detection image; The similarity response map between the i-th edge detection image and the binarized edge template image is calculated based on the normalized correlation matching method. Extract the current matching score from the similarity response map; Determine whether the current matching score is higher than the historical matching score; If the current matching score is lower than or equal to the historical matching score, then determine whether the image size of the (i+1)th scaled image to be detected is greater than or equal to the image size of the binarized edge template image; If the current matching score is higher than the historical matching score, then the matching position coordinates corresponding to the current matching score are taken as the current matching result; The target primitive detection result is obtained based on the current matching result.

6. A wiring diagram element recognition device, characterized in that, include: The module is used to build an object detection dataset based on the wiring diagram graphic file and its corresponding element category labeling and element angle labeling; The training module is used to train the object detection network based on the object detection dataset to obtain the trained object detection network. The processing module is used to crop the image to be detected based on multiple preset cropping scales to obtain multiple sub-images to be detected corresponding to each preset cropping scale; The prediction module is used to predict each of the sub-images to be detected through the trained target detection network for each of the preset cropping scales, so as to obtain the sub-image detection results corresponding to each preset cropping scale. The fusion module is used to fuse the detection results of the sub-images corresponding to each preset clipping scale based on the enhanced weighted box fusion algorithm to obtain the initial primitive detection results; The correction module is used to correct the initial primitive detection result based on multi-scale template matching to obtain the target primitive detection result; The fusion module is further configured to determine, based on the sub-image detection results corresponding to each preset clipping scale, all sub-image detection results under the preset clipping scales corresponding to multiple primitive categories; for each primitive category, hierarchical clustering is performed based on all sub-image detection results under the preset clipping scales corresponding to the primitive category to obtain cluster labels corresponding to each primitive category; for each cluster label, the original values ​​of multiple detection box coordinates corresponding to the cluster label are sorted in ascending order to obtain detection box index sorting; The detection box weights corresponding to the original coordinate values ​​of each detection box are sorted according to the detection box index to obtain a weight sequence; the fused coordinate values ​​are determined according to the weight sequence and the original coordinate values ​​of multiple detection boxes corresponding to the cluster labels; the fused coordinate values ​​are used as the initial primitive detection results. The fusion module is further configured to calculate the intersection-union ratio (CUI) and the Euclidean distance between the detection box centers of all sub-images under the preset clipping scale corresponding to the primitive category; and to calculate a hybrid similarity matrix based on the CUI and the Euclidean distance between the detection box centers, wherein the formula for calculating the hybrid similarity matrix is: In the formula, The mixed similarity matrix, and For hyperparameters, The intersection-union ratio is... The Euclidean distance to the center of the detection box is used; hierarchical clustering is performed based on the hybrid similarity matrix to obtain the cluster label.

7. A computer device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program, and the computer program, when executed by the processor, implements the wiring diagram element recognition method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the wiring diagram element recognition method as described in any one of claims 1 to 5.

Images