Wiring diagram primitive identification method and device, electronic equipment and storage medium

Abstract

The invention provides a primitive recognition method and device for a wiring diagram, electronic equipment and a storage medium, and relates to the technical field of image recognition, and the method comprises the steps: constructing a training data set according to the wiring diagram; training the residual network model and the semantic segmentation network model by adopting the training data set to obtain a primitive segmentation network model; detecting a to-be-identified wiring diagram according to the primitive segmentation network model to obtain a primitive mask area; and extracting edge point coordinates according to the primitive mask area to obtain a primitive segmentation result. According to the method, the advanced semantic segmentation network technology is combined, pixel-level fine segmentation is carried out on the primitive area of the line graph, the positioning detection and recognition capability of all types of primitives of the line graph is remarkably improved, and powerful support is provided for subsequent tasks such as primitive relation analysis and connection line optimization.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a method, a device, an electronic device and a storage medium for identifying graphic elements of a wiring diagram. Background Art

[0002] Wiring diagram recognition technology has a wide range of application needs in the fields of power, industrial automation, etc. The key lies in accurately and efficiently identifying the primitives, texts and connecting lines in the diagram. As the first step in wiring diagram recognition, primitive detection is not only crucial, but its detection effect directly affects the subsequent connecting line detection and overall recognition accuracy. However, wiring diagrams usually have high resolution, and the proportion of primitives in them is relatively small, which brings considerable challenges to primitive detection.

[0003] Traditional deep learning-based object detection network models often face problems such as low primitive detection recall and inaccurate detection frame regression when processing such high-resolution images with small object ratios. The coordinate accuracy of the detection frame is crucial for subsequent connection line detection, because any slight deviation may lead to failure of connection line detection or error accumulation. Due to the diversity and complexity of wiring diagram primitives and the interference of connecting lines on primitive detection, it is difficult for the object detection network to accurately capture and locate primitives. Therefore, how to improve the detection accuracy of wiring diagram primitives has become a key factor restricting the improvement of wiring diagram recognition level. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the present invention provides a wiring diagram element recognition method, device, electronic device and storage medium, which effectively solve the problem that the existing target detection network is difficult to accurately capture and locate the elements, resulting in low detection accuracy of wiring diagram elements.

[0005] In a first aspect, the present invention provides a method for identifying graphic elements in a wiring diagram, the method comprising: Construct a training dataset based on the wiring diagram; The training data set is used to train the residual network model and the semantic segmentation network model to obtain a primitive segmentation network model; Detecting the wiring diagram to be identified according to the primitive segmentation network model to obtain a primitive mask area; The edge point coordinates are extracted according to the primitive mask area to obtain the primitive segmentation result.

[0006] Further, constructing a training data set according to the wiring diagram includes: Annotating the graphic element categories of the wiring diagram to obtain an annotation result; Convert the annotation results into a semantic segmentation dataset format file; Perform sliding window cropping on the wiring diagram to obtain a sub-diagram; The training dataset is constructed according to the semantic segmentation dataset format file and the subgraph.

[0007] Furthermore, the training data set is used to train the residual network model and the semantic segmentation network model to obtain the primitive segmentation network model, including: Using the ImageNet dataset to perform transfer learning training on the residual network model and the semantic segmentation network model to obtain an initialized network model; Using the training data set to train the initialized network model to obtain an initial primitive segmentation network model; Parameters of the initial primitive segmentation network model are adjusted to obtain the primitive segmentation network model.

[0008] Furthermore, the using the training data set to train the initialized network model to obtain an initial primitive segmentation network model includes: Performing data augmentation on the training data set; Modify the number of output categories of the initialized network model, and configure the loss function and hyperparameters of the semantic segmentation network in the initialized network model; Using the data-enhanced training data set to train the initialized network model; Performing model evaluation on the trained initialized network model to obtain a model evaluation result; The parameters of the initialized network model are adjusted according to the model evaluation result to obtain the initial primitive segmentation network model.

[0009] Furthermore, the detecting the wiring diagram to be identified according to the primitive segmentation network model to obtain the primitive mask area includes: Perform sliding window cropping on the wiring diagram to be identified to obtain a sub-diagram to be identified; Inputting the sub-image to be identified into the primitive segmentation network model to perform primitive recognition to obtain a sub-image segmentation result; The sub-image segmentation result is mapped to obtain the primitive mask area.

[0010] Furthermore, mapping the sub-image segmentation result to obtain the primitive mask area includes: Obtaining the coordinate position of the coordinate frame of the sub-graph to be identified in the wiring diagram to be identified; Obtaining the pixel coordinate position of each pixel in the mask area of ​​the sub-image segmentation result; The position of the sub-image segmentation result in the wiring diagram to be identified is adjusted according to the coordinate position of the coordinate frame and the coordinate position of the pixel point to obtain the primitive mask area.

[0011] Furthermore, extracting edge point coordinates according to the primitive mask area to obtain a primitive segmentation result includes: Preprocessing the primitive mask area; Using an edge detection algorithm to perform edge detection on the preprocessed primitive mask area to obtain edge point pixel coordinates; A primitive segmentation region is constructed according to the pixel coordinates of the edge points, and the wiring diagram to be identified is shielded by the primitive segmentation region to obtain the primitive segmentation result.

[0012] In a second aspect, the present invention provides a wiring diagram element recognition device, the device comprising: A data acquisition module, used to construct a training data set based on the wiring diagram; A model processing module, used to train the residual network model and the semantic segmentation network model using the training data set to obtain a primitive segmentation network model; A graphics detection module, used to detect the wiring diagram to be identified according to the graphics primitive segmentation network model to obtain a graphics primitive mask area; The primitive segmentation module is used to extract edge point coordinates according to the primitive mask area to obtain a primitive segmentation result.

[0013] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for identifying graphic elements of a wiring diagram as described in the first aspect of the present invention.

[0014] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method for identifying graphic elements of a wiring diagram as described in the first aspect of the present invention.

[0015] The method, device, electronic device and storage medium for identifying the primitives of the wiring diagram provided by the present invention, combined with advanced semantic segmentation network technology, can perform pixel-level fine segmentation on the primitive area of ​​the wiring diagram, and can accurately distinguish the primitives from the background, as well as the boundaries between primitives of different categories. At the same time, more accurate and robust primitive recognition is achieved through the learning ability of the deep learning network, which significantly improves the positioning detection and recognition capabilities of the various categories of primitives in the wiring diagram. Through the precise segmentation and recognition of the primitive area, the overall structure and information of the wiring diagram can be better understood, providing strong support for subsequent tasks such as primitive relationship analysis and connection line optimization. Not only does it improve the intelligence level and overall effect of wiring diagram processing, but it also reduces the need for manual intervention and improves work efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 It is a first schematic diagram of the process of the method for identifying graphic elements of a wiring diagram provided by an embodiment of the present invention; Figure 2 It is a second schematic diagram of the process of the method for identifying graphic elements of a wiring diagram provided by an embodiment of the present invention; Figure 3 It is a third schematic diagram of the process of the method for identifying graphic elements of a wiring diagram provided by an embodiment of the present invention; Figure 4 It is a fourth schematic diagram of the process of the method for identifying graphic elements of a wiring diagram provided by an embodiment of the present invention; Figure 5 is a fifth schematic diagram of a method for identifying graphic elements in a wiring diagram provided by an embodiment of the present invention; Figure 6 is a sixth schematic diagram of the process of the method for identifying graphic elements of a wiring diagram provided by an embodiment of the present invention; Figure 7 is a seventh schematic diagram of a method for identifying graphic elements in a wiring diagram provided by an embodiment of the present invention; Figure 8 It is a schematic diagram comparing the results of the graphic element recognition of a certain plant wiring diagram in an embodiment of the present invention; in, Figure 8 (a) is the original wiring diagram of the plant; Figure 8 (b) is the primitive segmentation result image output by the primitive segmentation network model; Figure 8 (c) is the primitive segmentation result map output by the traditional target detection network model; Fig. 9 It is a schematic diagram comparing the connection line detection results of a plant wiring diagram in an embodiment of the present invention; in, Fig. 9 (a) is the original wiring diagram of the plant; Fig. 9 (b) is the primitive segmentation result image output by the primitive segmentation network model; Fig. 9 (c) is the primitive segmentation result map output by the traditional target detection network model; Fig. 9 (e) in the figure is the connection line detection result map based on the detection area output by the primitive segmentation network model; Fig. 9 (f) in the figure is the connection line detection result diagram based on the detection area output by the traditional target detection network model; Fig.10 It is a schematic diagram of the structure of a graphic element recognition device for a wiring diagram provided by an embodiment of the present invention; Fig.11 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention.

[0018] Description of main component symbols: 100. A graphic element recognition device for a wiring diagram; 110. A data acquisition module; 120. A model processing module; 130. A graphic detection module; 140. A graphic element segmentation module; 200. An electronic device; 210. A processor; 220. A communication interface; 230. A memory; 240. A communication bus. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be further clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. It should be noted that the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0020] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0022] Traditional deep learning-based object detection network models often face problems such as low primitive detection recall and inaccurate detection frame regression when processing such high-resolution images with small object ratios. The coordinate accuracy of the detection frame is crucial for subsequent connection line detection, because any slight deviation may lead to failure of connection line detection or error accumulation. Due to the diversity and complexity of wiring diagram primitives and the interference of connecting lines on primitive detection, it is difficult for the object detection network to accurately capture and locate primitives. Therefore, how to improve the detection accuracy of wiring diagram primitives has become a key factor restricting the improvement of wiring diagram recognition level.

[0023] Example 1 The embodiment of the present invention provides a method for identifying graphic elements of a wiring diagram, which effectively solves the problem that the existing target detection network is difficult to accurately capture and locate graphic elements, resulting in low detection accuracy of graphic elements of the wiring diagram. Figure 1 FIG. 1 is a first schematic diagram of a method for identifying graphic elements in a wiring diagram according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps: S100, constructing a training data set according to the wiring diagram.

[0024] In the embodiment of the present invention, a training data set is constructed based on 110 plant wiring diagram files provided by a power company. Figure 2 is a second schematic diagram of the process of the method for identifying the graphic element of the wiring diagram provided by the embodiment of the present invention, such as Figure 2 As shown in Figure 1, the construction of the training set specifically includes the following steps: S110, labeling the graphic element categories of the wiring diagram to obtain labeling results.

[0025] In the embodiment of the present invention, the related graphic element categories included in the wiring diagram include, but are not limited to, knife switches, switches, grounding knife switches, three-winding transformers, two-winding transformers, autotransformers, capacitors, reactors, mutual inductors and other components. Optionally, the labelme image annotation tool is used to annotate the graphic element categories corresponding to each graphic element in all wiring diagrams to obtain the annotation results, that is, a json file containing all the annotation information.

[0026] S120, converting the annotation result into a semantic segmentation dataset format file.

[0027] In an embodiment of the present invention, the annotation result is converted into a json file in the format of the COCO (Common Objects in Context) semantic segmentation dataset. The COCO format is a standard data format for tasks such as object detection, segmentation, and key point detection. It uses JSON files to store images, annotations, and category information. By reading the json file and parsing its content, the image information, annotation information, and primitive category information are extracted and added to the corresponding fields in the COCO format to obtain a json file in the format of the semantic segmentation dataset.

[0028] S130: Perform sliding window cropping on the wiring diagram to obtain a sub-diagram.

[0029] In an embodiment of the present invention, a plurality of smaller sub-graphs that may contain key information of the wiring diagram are extracted from the original large graph by a sliding window cropping technique based on a certain ratio of the length and width of each wiring diagram. Optionally, the cropping ratios are selected to be 1 / 3, 1 / 5, and 1 / 7 of the width and height of the original graph, respectively, to perform sliding window cropping, and a total of about 10,000 sub-graphs are obtained after cropping.

[0030] S140, constructing a training dataset according to the semantic segmentation dataset format file and the subgraph.

[0031] The semantic segmentation dataset format file contains the annotation results of each graphic element at the specific location of the wiring diagram. The annotation results contained in the file can be directly read when training the model. At the same time, the cropped sub-images are randomly divided into training sets and validation sets in a ratio of 8:2 to obtain the training dataset.

[0032] S200, using the training data set to train the residual network model and the semantic segmentation network model to obtain a primitive segmentation network model.

[0033] In the embodiment of this aspect, a residual network model and a semantic segmentation network model are used to form a primitive segmentation network. The residual network model can select a ResNet-50 model, which is mainly used for image classification tasks. The ResNet-50 model mainly includes an input layer, an initial convolution layer, a maximum pooling layer, a residual block, a global average pooling layer, a fully connected layer, and a softmax activation function, so that it can handle the training problem of a deep network and alleviate the gradient vanishing problem through residual connections. Wherein: The input layer is used to receive image data, usually a 224x224 pixel RGB image. The initial convolutional layer is a 7x7 convolutional layer with a stride of 2, which is used to extract initial features. After the convolutional layer, there is usually a 3x3 maximum pooling layer with a stride of 2 for downsampling. The residual block is the core part of ResNet-50. There are 4 stages, namely Conv2_x, Conv3_x, Conv4_x and Conv5_x. Each stage contains multiple residual blocks. Conv2_x includes 3 residual blocks and the number of output channels is 256. Conv3_x includes 4 residual blocks and the number of output channels is 512. Conv4_x includes 6 residual blocks and the number of output channels is 1024. Conv5_x includes 3 residual blocks and the number of output channels is 2048. After all residual blocks, a global average pooling layer is used to reduce the dimensionality. The final fully connected layer is used for classification, which usually outputs the probabilities of 1000 categories and uses the softmax activation function to convert the output into a probability distribution.

[0034] The semantic segmentation network model can adopt network models such as Mask R-CNN, U-net and DeepLab. In the embodiment of the present invention, the Mask R-CNN model is used as the semantic segmentation network model. The Mask R-CNN model is mainly used for target detection and semantic segmentation tasks. It adds a branch on the basis of Faster R-CNN to predict the precise mask of the target instance, which enables the model to not only locate and classify the target, but also accurately segment the contour of the target. The Mask R-CNN model includes a backbone structure, a region generation network, a RoI module, a classification head, a bounding box regression head and a Mask Head module. Among them: The backbone structure is used to extract image features from the backbone network, the region generation network is used to generate candidate target boxes, that is, regions that may contain targets, the RoI module is used to extract features on RoIs (Region of Interest) of different sizes, the classification head is used to predict the category of the target, the bounding box regression head is used to adjust the generated bounding box to more accurately wrap the target, and the Mask Head module is used to predict the precise mask of the target, so that the Mask R-CNN model can achieve instance segmentation based on target detection.

[0035] Figure 3 3 is a third schematic diagram of the method for identifying the wiring diagram element according to an embodiment of the present invention. Figure 3 As shown in Figure 1, the training steps of the primitive segmentation network model are as follows: S210. Use the ImageNet dataset to perform transfer learning training on the residual network model and the semantic segmentation network model to obtain an initialized network model.

[0036] The ImageNet dataset is a large-scale image dataset, containing more than 14 million images, divided into thousands of categories, commonly used in benchmarks for image recognition tasks, and is an important resource for deep learning model training and evaluation. The residual network model and semantic segmentation network model trained on the ImageNet dataset have learned many common image features, such as edges, colors, and textures, etc. These common image features have certain universality between different datasets. Although the training dataset in the embodiment of the present invention is quite different from the ImageNet dataset, these low-level features can still be effectively utilized. By using the ImageNet dataset for transfer learning to obtain an initialized network model, training time and data requirements can be reduced, and the performance of the model in primitive segmentation can be effectively improved.

[0037] S220, using the training data set to train the initialized network model to obtain an initial primitive segmentation network model.

[0038] Figure 44 is a fourth schematic diagram of the method for identifying the wiring diagram element according to an embodiment of the present invention. Figure 4 As shown, the training of the initialization network model specifically includes the following steps: S221. Perform data enhancement on the training data set.

[0039] In an embodiment of the present invention, data enhancement can be performed by performing operations such as random rotation, flipping, and scaling on the training data set, thereby improving the generalization ability and performance of the initialized network model.

[0040] S222. Modify the number of output categories of the initialized network model, and configure the loss function and hyperparameters of the semantic segmentation network in the initialized network model.

[0041] In an embodiment of the present invention, since the number of output categories of the initialized network model obtained by transfer learning using the ImageNet dataset may be thousands, it is necessary to modify the number of output categories according to the training dataset in an embodiment of the present invention, and modify the number of output categories to the number of labeled primitive categories. The loss function of Mask R-CNN includes classification loss, bounding box regression loss and mask segmentation loss, wherein the classification loss is used for category prediction of target detection, the bounding box regression loss is used to adjust the position of the bounding box, and the mask segmentation loss is used for the cross entropy loss of pixel-level segmentation. Hyperparameters include learning rate, batch size, optimizer and number of training rounds. Optionally, the learning rate can be set to 0.001 and gradually adjusted during the training process. The batch size selects a suitable value according to the size of the video memory. The optimizer uses the Adam optimizer and the number of training rounds is set to 24.

[0042] S223, using the data-enhanced training data set to train the initialization network model; Use the training set in the training data set after data augmentation to train the initialized network model with configured parameters, and use the validation set that is not involved in the training to evaluate the performance of the model and help adjust the hyperparameters of the model.

[0043] S224: Perform model evaluation on the trained initialized network model to obtain a model evaluation result.

[0044] In an embodiment of the present invention, the model evaluation results mainly include average accuracy, intersection-over-union (IoU) and pixel accuracy, wherein the average accuracy is used to evaluate the performance of primitive detection and segmentation, the IoU is used to evaluate the degree of overlap between the segmentation result and the true mask, and the pixel accuracy is used to evaluate the accuracy of segmentation.

[0045] S225. Adjust the parameters of the initialized network model according to the model evaluation result to obtain an initial primitive segmentation network model.

[0046] The parameters of the initialized network model are adjusted according to the average accuracy, intersection-over-union ratio and pixel accuracy to obtain the initial primitive segmentation network model.

[0047] S230, adjusting parameters of the initial primitive segmentation network model to obtain a primitive segmentation network model.

[0048] In an embodiment of the present invention, the parameters of the initial primitive segmentation network model are adjusted according to the verification results in the training process. The verification results include but are not limited to the loss function value, accuracy, recall rate, F1 score and visualization results. The model architecture can be adjusted by increasing or decreasing the number of layers, changing the convolution kernel size or step size, and introducing jump connections. The model can also be optimized by adjusting hyperparameters to obtain the final primitive segmentation network model.

[0049] S300 , detecting the wiring diagram to be identified according to the primitive segmentation network model to obtain a primitive mask area.

[0050] Figure 5 5 is a fifth schematic diagram of a method for identifying graphic elements in a wiring diagram according to an embodiment of the present invention. Figure 5 As shown, the detection of the wiring diagram to be identified specifically includes the following steps: S310, performing sliding window cropping on the wiring diagram to be identified to obtain a sub-diagram to be identified.

[0051] In the embodiment of the present invention, sliding window cropping is performed on the wiring diagram to be identified based on the same cropping method as the training data set to obtain multiple sub-diagrams to be identified.

[0052] S320 , input the sub-image to be identified into the primitive segmentation network model for primitive recognition to obtain a sub-image segmentation result.

[0053] Multiple sub-images to be identified are input into the trained primitive segmentation network model. The model forward propagates the sub-images through its network architecture, calculates the probability that each pixel or region belongs to a different primitive, and outputs the segmentation results. The segmentation results are analyzed for connected regions to remove small noise regions or merge adjacent similar regions, and then the segmentation results are visualized to obtain the sub-image segmentation results.

[0054] S330 , mapping the sub-image segmentation result to obtain a primitive mask area.

[0055] Figure 6 6 is a schematic diagram of a method for identifying graphic elements in a wiring diagram according to an embodiment of the present invention. Figure 6 As shown, the mapping of the sub-graph segmentation results specifically includes the following steps: S331, obtaining the coordinate position of the coordinate frame of the sub-diagram to be identified in the wiring diagram to be identified.

[0056] In the embodiment of the present invention, if there aren subgraphs to be identified, then the coordinate frame coordinate positions of all subgraphs to be identified in the wiring diagram to be identified are obtained. Optionally, the coordinate frame coordinate position may be the coordinate frame coordinate position of the upper left corner, and the obtained coordinate frame coordinate position list of the upper left corner may be expressed as: ,in Indicates n The horizontal coordinate of the upper left corner of the coordinate box of the sub-graph to be identified in the wiring diagram to be identified, Indicates n The vertical coordinate of the upper left corner of the coordinate box of the sub-graph to be identified in the wiring diagram to be identified.

[0057] S332, obtaining the pixel coordinate position of each pixel in the mask area of ​​the sub-image segmentation result.

[0058] In the embodiment of the present invention, for each sub-image segmentation result, the pixel coordinate position of each pixel in the mask area of ​​the sub-image segmentation result can be obtained. For example, a mask area has m pixels, the list of pixel coordinate positions can be expressed as: ,in x m Indicates m The horizontal coordinate of the pixel point, y m Indicates m The vertical coordinate of the pixel.

[0059] S333. Adjust the position of the sub-image segmentation result in the wiring diagram to be identified according to the coordinate position of the coordinate frame and the coordinate position of the pixel point to obtain a primitive mask area.

[0060] In the embodiment of the present invention, the detection position of the sub-image segmentation result relative to the wiring diagram to be identified is adjusted according to the coordinate position of each pixel point in the mask area of ​​the sub-image segmentation result and the coordinate position of the upper left corner coordinate frame of the sub-image to be identified in the wiring diagram to be identified. r The sub-graphs to be identified are detected c The target mask mask , then in the wiring diagram to be identified c The target mask masks The calculation formula is as follows:

[0061] In the above formula, mask i It is expressed as:

[0062] In the above formula, m express mask iThe number of pixels in the mask area, Indicates r The coordinate position of the sub-diagram to be identified in the coordinate box in the upper left corner of the wiring diagram to be identified.

[0063] S400: extract edge point coordinates according to the primitive mask area to obtain a primitive segmentation result.

[0064] Figure 7 7 is a seventh schematic diagram of a method for identifying graphic elements in a wiring diagram according to an embodiment of the present invention. Figure 7 As shown, obtaining the primitive segmentation result includes the following steps: S410: Preprocess the primitive mask area.

[0065] Since the primitive mask area may contain noise or irregular edges, preprocessing, such as denoising and morphological operations, is required to optimize the edge detection effect.

[0066] S420: Use an edge detection algorithm to perform edge detection on the preprocessed primitive mask area to obtain edge point pixel coordinates.

[0067] In an embodiment of the present invention, an edge detection algorithm such as a Canny edge detector can be used to perform edge detection on the preprocessed mask image. The Canny edge detector is an effective edge detection method that uses the first-order differential of a Gaussian function to calculate the gradient and finds the local maximum of the gradient as the edge point. The pixel coordinates of the edge point can be obtained through the edge detection algorithm, and these coordinates represent the boundary position of the target primitive mask area.

[0068] S430 , constructing a primitive segmentation region according to the pixel coordinates of the edge points, and using the primitive segmentation region to shield the wiring diagram to be identified, to obtain a primitive segmentation result.

[0069] According to the extracted edge point coordinates, the edge points are connected into lines to form a closed contour, thereby defining the range of the target primitive and constructing the primitive segmentation region. After obtaining the primitive segmentation region, the target primitive in the original image can be masked according to the primitive segmentation region, that is, it is removed from the image or replaced with the background to obtain the primitive segmentation result.

[0070] In order to verify the feasibility of the method for identifying primitives in a wiring diagram provided by an embodiment of the present invention, primitive segmentation network model in an embodiment of the present invention and traditional target detection network model are used to perform primitive recognition on a wiring diagram of a plant. Figure 8 is a schematic diagram comparing the results of the graphic element recognition of a plant wiring diagram in an embodiment of the present invention, such as Figure 8As shown, (a) is the original image (partial) of the plant wiring diagram, (b) is the primitive segmentation result image output by the primitive segmentation network model, and (c) is the primitive segmentation result image output by the traditional target detection network model. Fig. 9 is a schematic diagram comparing the connection line detection results of a plant wiring diagram in an embodiment of the present invention, such as Fig. 9 As shown, (a) is the original image (partial) of the plant wiring diagram, (b) is the image of the image segmentation result output by the image segmentation network model, (c) is the image of the image segmentation result output by the traditional target detection network model, (e) is the connection line detection result based on the detection area output by the image segmentation network model, and (f) is the connection line detection result based on the detection area output by the traditional target detection network model. Due to the inaccurate regression of the target box (the target box is too large or too small), the traditional target detection network model will cause too much or too little pixel shielding in the connection area before the model detects the connection line, thereby causing missed detection and false detection of the connection line. The image segmentation network model in the embodiment of the present invention, when identifying the connection line of the wiring diagram, performs pixel-level segmentation on the image element, outputs the precise mask area of ​​the image element, and shields the image element on the original image (the image element area is set to the background color) based on the detected precise mask area (the minimum circumscribed rectangle of the detection area), so as to detect short connection lines.

[0071] The wiring diagram primitive recognition method provided by the embodiment of the present invention combines advanced semantic segmentation network technology to perform pixel-level fine segmentation of wiring diagram primitive areas, and can accurately distinguish primitives from backgrounds, as well as boundaries between primitives of different categories. At the same time, more accurate and robust primitive recognition is achieved through the learning ability of the deep learning network, significantly improving the positioning detection and recognition capabilities of various categories of primitives in wiring diagrams.

[0072] Example 2 Based on the same technical concept as the above method embodiment 1, the embodiment of the present invention provides a wiring diagram primitive recognition device, Fig.10 is a schematic diagram of the structure of a wiring diagram primitive recognition device provided by an embodiment of the present invention, such as Fig.10 As shown, the wiring diagram graphic element recognition device 100 includes: A data acquisition module 110, for constructing a training data set according to the wiring diagram; A model processing module 120 is used to train the residual network model and the semantic segmentation network model using a training data set to obtain a primitive segmentation network model; A graphic detection module 130, used to detect the wiring diagram to be identified according to the graphic element segmentation network model to obtain a graphic element mask area; The primitive segmentation module 140 is used to extract edge point coordinates according to the primitive mask area to obtain a primitive segmentation result.

[0073] The wiring diagram primitive recognition device provided by the embodiment of the present invention can better understand the overall structure and information of the wiring diagram through accurate segmentation and recognition of primitive regions, and provide strong support for subsequent primitive relationship analysis, connection line optimization and other tasks. It not only improves the intelligent level and overall effect of wiring diagram processing, but also reduces the need for manual intervention, and improves work efficiency and accuracy.

[0074] It can be understood that the implementation of the wiring diagram primitive recognition method described in the above-mentioned embodiment 1 is also applicable to this embodiment and can achieve the same technical effect, so it will not be described repeatedly here.

[0075] Example 3 Based on the same concept, an embodiment of the present invention further provides an electronic device, Fig.11 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, such as Fig.11 As shown, the electronic device 200 may include: a processor 210, a communication interface 220, a memory 230 and a communication bus 240, wherein the processor 210, the communication interface 220 and the memory 230 communicate with each other through the communication bus 240. The processor 210 may call the logic instructions in the memory 230 to execute the steps of the method for identifying the graphic element of the wiring diagram as described in the above embodiments. For example, it includes: S100, constructing a training data set according to the wiring diagram; S200, using the training data set to train the residual network model and the semantic segmentation network model to obtain a primitive segmentation network model; S300, detecting the wiring diagram to be identified according to the primitive segmentation network model to obtain a primitive mask area; S400: extract edge point coordinates according to the primitive mask area to obtain a primitive segmentation result.

[0076] The processor 210 may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0077] In addition, the logic instructions in the above-mentioned memory 230 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0078] The memory 230 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0079] Example 4 Based on the same concept, an embodiment of the present invention further provides a computer-readable storage medium, which stores a computer program, and the computer program includes at least one code segment, which can be executed by a main control device to control the main control device to implement the steps of the method for identifying the graphic element of the wiring diagram as described in the above embodiments. For example, it includes: S100, constructing a training data set according to the wiring diagram; S200, using the training data set to train the residual network model and the semantic segmentation network model to obtain a primitive segmentation network model; S300, detecting the wiring diagram to be identified according to the primitive segmentation network model to obtain a primitive mask area; S400: extract edge point coordinates according to the primitive mask area to obtain a primitive segmentation result.

[0080] Based on the same technical concept, an embodiment of the present invention further provides a computer program, which is used to implement the above method embodiment when executed by a main control device.

[0081] The computer program may be stored in whole or in part on a computer-readable storage medium packaged with the processor, or may be stored in whole or in part on a memory not packaged with the processor.

[0082] Based on the same technical concept, an embodiment of the present invention further provides a processor, which is used to implement the above method embodiment. The above processor can be a chip.

[0083] In summary, the method, device, electronic device and storage medium for identifying the primitives of the wiring diagram provided by the present invention, combined with advanced semantic segmentation network technology, can perform pixel-level fine segmentation on the primitive area of ​​the wiring diagram, and can accurately distinguish the primitives from the background, as well as the boundaries between primitives of different categories. At the same time, more accurate and robust primitive recognition is achieved through the learning ability of the deep learning network, which significantly improves the positioning detection and recognition capabilities of the primitives of various categories in the wiring diagram. Through the precise segmentation and recognition of the primitive area, the overall structure and information of the wiring diagram can be better understood, providing strong support for subsequent tasks such as primitive relationship analysis and connection line optimization. Not only does it improve the intelligence level and overall effect of wiring diagram processing, but it also reduces the need for manual intervention and improves work efficiency and accuracy.

[0084] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0085] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying graphic elements of a wiring diagram, characterized in that: The method comprises: Construct a training dataset based on the wiring diagram; The training data set is used to train the residual network model and the semantic segmentation network model to obtain a primitive segmentation network model; Detecting the wiring diagram to be identified according to the primitive segmentation network model to obtain a primitive mask area; The edge point coordinates are extracted according to the primitive mask area to obtain the primitive segmentation result.

2. The method for identifying graphic elements of a wiring diagram according to claim 1, characterized in that: The step of constructing a training data set according to the wiring diagram comprises: Annotating the graphic element categories of the wiring diagram to obtain an annotation result; Convert the annotation results into a semantic segmentation dataset format file; Perform sliding window cropping on the wiring diagram to obtain a sub-diagram; The training dataset is constructed according to the semantic segmentation dataset format file and the subgraph.

3. The method for identifying graphic elements of a wiring diagram according to claim 1, characterized in that: The method of using the training data set to train the residual network model and the semantic segmentation network model to obtain the primitive segmentation network model includes: Using the ImageNet dataset to perform transfer learning training on the residual network model and the semantic segmentation network model to obtain an initialized network model; Using the training data set to train the initialized network model to obtain an initial primitive segmentation network model; Parameters of the initial primitive segmentation network model are adjusted to obtain the primitive segmentation network model.

4. The method for identifying graphic elements of a wiring diagram according to claim 3, characterized in that: The step of training the initialized network model using the training data set to obtain an initial primitive segmentation network model includes: Performing data augmentation on the training data set; Modify the number of output categories of the initialized network model, and configure the loss function and hyperparameters of the semantic segmentation network in the initialized network model; Using the data-enhanced training data set to train the initialized network model; Performing model evaluation on the trained initialized network model to obtain a model evaluation result; The parameters of the initialized network model are adjusted according to the model evaluation result to obtain the initial primitive segmentation network model.

5. The method for identifying graphic elements of a wiring diagram according to claim 1, characterized in that: The detecting the wiring diagram to be identified according to the primitive segmentation network model to obtain the primitive mask area includes: Perform sliding window cropping on the wiring diagram to be identified to obtain a sub-diagram to be identified; Inputting the sub-image to be identified into the primitive segmentation network model to perform primitive recognition to obtain a sub-image segmentation result; The sub-image segmentation result is mapped to obtain the primitive mask area.

6. The method for identifying graphic elements of a wiring diagram according to claim 5, characterized in that: Mapping the sub-image segmentation result to obtain the primitive mask area includes: Obtaining the coordinate position of the coordinate frame of the sub-graph to be identified in the wiring diagram to be identified; Obtaining the pixel coordinate position of each pixel in the mask area of ​​the sub-image segmentation result; The position of the sub-image segmentation result in the wiring diagram to be identified is adjusted according to the coordinate position of the coordinate frame and the coordinate position of the pixel point to obtain the primitive mask area.

7. The method for identifying graphic elements of a wiring diagram according to claim 1, characterized in that: The step of extracting edge point coordinates according to the primitive mask area to obtain a primitive segmentation result includes: Preprocessing the primitive mask area; Using an edge detection algorithm to perform edge detection on the preprocessed primitive mask area to obtain edge point pixel coordinates; A primitive segmentation region is constructed according to the pixel coordinates of the edge points, and the wiring diagram to be identified is shielded by the primitive segmentation region to obtain the primitive segmentation result.

8. A wiring diagram element recognition device, characterized in that: The device comprises: A data acquisition module, used to construct a training data set based on the wiring diagram; A model processing module, used to train the residual network model and the semantic segmentation network model using the training data set to obtain a primitive segmentation network model; A graphics detection module, used to detect the wiring diagram to be identified according to the graphics primitive segmentation network model to obtain a graphics primitive mask area; The primitive segmentation module is used to extract edge point coordinates according to the primitive mask area to obtain a primitive segmentation result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: The processor executes the computer program to implement the method for identifying graphic elements in a wiring diagram according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for identifying graphic elements in a wiring diagram according to any one of claims 1 to 7 is implemented.

Images