An automatic text detection and recognition method for plant wiring diagrams based on improved PP-OCRv3

Abstract

The present invention discloses a method for automatic detection and recognition of text in plant wiring diagrams based on an improved PP‑OCRv3. Aiming at the limitations of existing deep learning-based text detection and recognition methods for power grid drawings in processing compact text with multiple scales, multiple directions, and irregular shapes, the present invention proposes a method for automatic detection and recognition of text in plant wiring diagrams based on an improved PP‑OCRv3. Using a region re-segmentation module based on pixel row clustering, the detection errors of irregularly shaped text containing vertical and horizontal characters are corrected. By introducing a channel attention mechanism and depthwise separable convolution in the BiFPN module, the robustness of the recognition model to input images of different scales is improved. Using a character re-recognition module based on region expansion and cutting, the adverse effects of simple characters and dense text on the model are reduced. Experimental results show that the present invention can significantly improve the detection and recognition accuracy of text in plant wiring diagrams through a series of improvements to PP‑OCRv3.

Description

Technical Field

[0001] The present invention relates to the fields of smart grid and computer vision, and in particular to a method for automatic detection and recognition of text in a plant wiring diagram based on improved PP-OCRv3. Background Art

[0002] Automating the management and search of power grid drawings is extremely urgent. As the scale of power grid drawings increases, traditional manual identification of drawing content is not only inefficient but also prone to errors. Due to the complex information contained in power grid wiring diagrams, detecting the position of text annotations and identifying the text content in power drawings becomes particularly important to achieve automated processing.

[0003] Text annotations in power plant wiring diagrams often come in a variety of shapes, orientations, and sizes. These characters include not only Arabic numerals and letters, but also numerous Chinese characters. Therefore, directly using general-purpose optical character recognition models to recognize text in power plant wiring diagrams is impractical and often results in poor detection accuracy. Research is needed to develop text detection and recognition models that effectively adapt to the characteristics of text annotations in power plant wiring diagrams to improve detection and recognition accuracy.

[0004] In recent years, deep learning-based methods have achieved remarkable success in the field of optical character recognition. Deep learning-based OCR technology has become a potential path toward automated and intelligent recognition of power grid wiring diagrams. Applying deep learning to text recognition in engineering design drawings can effectively achieve intelligent and automatic recognition of power grid wiring diagrams.

[0005] Closest existing technology and its evaluation:

[0006] Laura Jamieson et al. (Jamieson, L., Moreno-Garcia, CF, Elyan, E.: Deeplearning for text detection and recognition in complex engineering diagrams. In: 2020 International Joint Conference on Neural Networks (IJCNN). pp. 1–7. IEEE (2020)) used EAST and LSTM networks to detect and recognize text in complex engineering drawings. The results showed that although the selected deep learning method can detect and recognize text in simple scenes, more complex text detection and recognition, including those located near primitives and mixed horizontal and vertical text, still pose challenges. Li et al. (Shanbin, L., Haoyu, W., Junhao, Z.: Electrical cabinet wiring detection method based on improved yolov5 and pp-ocrv3. In: 2022 ChinaAutomation Congress (CAC). pp. 6503–6508. IEEE (2022)) used YOLOv5 and PP-OCRv3 to improve the accuracy of text detection and recognition in electrical cabinet wiring diagrams. PP-OCRv3 is one of the representatives of two-stage text recognition models. In the text location detection process, DBNet is introduced to detect the text area. On this basis, text proposals are proposed. In the text feature extraction process, the cropped text image is input into MobileNetv1 for feature extraction based on the text proposal. The feature map of each text proposal is obtained. s. In the text recognition process, a recognition head based on SVTR is proposed to obtain text recognition results. However, PP-OCRv3 performs poorly in multi-scale electrical diagrams with irregular shapes and compactly distributed text. The text annotation forms in power grid wiring diagrams are often diverse, with different shapes, orientations, and sizes. These Chinese characters include not only numbers and letters, but also Chinese characters. When a general OCR model is directly used to recognize text in electrical diagrams, its accuracy is usually poor. Summary of the Invention

[0007] The object of the present invention is to provide a method for automatic detection and recognition of text in plant wiring diagrams based on improved PP-OCRv3.

[0008] Key points of technical solution:

[0009] In response to the limitations of existing deep learning-based text detection and recognition methods for power grid drawings in handling compact text with multiple scales, multiple directions, and irregular shapes, the present invention proposes an automatic detection and recognition method for text in power station wiring diagrams based on an improved PP-OCRv3. Detection errors of irregularly shaped text containing vertical and horizontal characters are corrected by utilizing a region resegmentation module based on pixel row clustering. By introducing a channel attention mechanism and depthwise separable convolution in the BiFPN module, the robustness of the recognition model to input images of different scales is improved. A character re-identification module based on region expansion and cutting is utilized to reduce the adverse effects of simple characters and dense text on the model. Experimental results show that the present invention can significantly improve the detection and recognition accuracy of text in power station wiring diagrams through a series of improvements to PP-OCRv3.

[0010] The technical solution to be protected in the present invention is characterized by:

[0011] A method for automatic detection and recognition of text in a plant wiring diagram based on improved PP-OCRv3, characterized by comprising the following steps:

[0012] Step 1: Divide the power grid plant wiring diagram dataset into a training set and a test set, which are used for training and testing the text detection and recognition models respectively.

[0013] Step 2: Detect text positions on the input image and build a region re-segmentation module to adjust the text detection results, solve the problem of accurate detection of irregular text containing both horizontal and vertical text, and obtain text candidate boxes based on this.

[0014] Step 3: Construct a feature extraction and fusion network based on the improved BiFPN to extract feature information from the cropped text image and obtain sequence features.

[0015] Step 4: Encode and decode the sequence features and build a character re-recognition module. By fine-tuning the recognition results of simple Chinese characters and multi-line numeric text, the overall accuracy is improved and the text recognition results are obtained accordingly.

[0016] Step 5: Calculate the losses of the detection and recognition models by comparing the text candidate boxes and text recognition results with the corresponding true values.

[0017] Step 6: Based on the loss, use the Adam optimizer to optimize the detection and recognition network models respectively.

[0018] Step 7: Repeat steps 2-6 until both networks converge and save the two model files.

[0019] Step 8: Input the test drawing into the detection model to predict the text candidate box, and then input the cropped text image into the recognition model to predict the text recognition result.

[0020] Step 2:

[0021] The text detection is divided into preliminary detection based on the original DBNet network model and refinement based on the region re-segmentation module. First, the test drawing to be detected is input into the DBNet text detection network model for preliminary text box position detection. The input image undergoes feature extraction and fusion based on the pyramid structure, and performs differentiable binarization (DB) to obtain a binary map of pixel segmentation. Then, after a series of post-processing, the kernel outline and kernel boundary are obtained (the kernel reflects the central area of ​​the text candidate box. Compared with the text candidate box, the smaller central area kernel can clearly separate multiple compact texts, which is very helpful for the pixel-level segmentation of dense text). Next, the obtained output is sent to the region re-segmentation module for refinement of the text candidate box.

[0022] The algorithm of the region re-segmentation module is a text position detection post-processing method based on pixel row clustering. A region re-segmentation module based on pixel row clustering is designed based on DBNet. The specific approach is as follows:

[0023] S2.1 Abnormal area filtering steps:

[0024] Calculate the kernel outline and bounding box of each text , is the area of ​​the region surrounded by the kernel outline, It is the area of ​​the region enclosed by the kernel bounding box; the region segmentation module is used to re-segment the "problem area";

[0025] S2.2 Pixel row clustering step:

[0026] Record the width of each pixel row within the kernel outline of the "problem area" and perform row-based clustering on the outline area; convert the binary image of the "problem area" into a list The length of is the height of the "problem area" ; Indicates the rowpixelrow; By a binary Composition, representing the vertical coordinate of the pixel point in the "problem area" When , the corresponding minimum value of the horizontal axis and maximum value ;according to , get the horizontal coordinates of the starting and ending positions of each line of text pixels in the "problem area"; according to , we can get the starting and ending positions of the horizontal coordinates of each pixel row in the "problem area"; next, randomly select the text pixel rows that have not been clustered As cluster center ,Right now:

[0027]

[0028] Then from Expand to the upper and lower pixel rows respectively; if the expanded pixel row The distance to the cluster center is less than the threshold , is a hyperparameter derived from the statistical law of irregular text in the power grid drawings, and it is classified into the current cluster. The above process is expressed as:

[0029]

[0030]

[0031] Then, the cluster center is recalculated based on the newly added pixel rows as follows:

[0032]

[0033] in is the current cluster, is the number of elements in the cluster; after that, continue to expand upward and downward until the distance between the pixel row and the cluster center is greater than or equal to the threshold , or a pixel row that has been clustered, or reaches the pixel row boundary; the clustering is performed cyclically until all pixel rows are clustered;

[0034] S2.3 threshold filtering steps:

[0035] set up As the threshold of the area; if the area of ​​the class cluster is less than , then filter the class clusters.

[0036] S2.4 Text candidate box formation steps:

[0037] The fine-tuned text kernel is expanded to generate text candidate boxes.

[0038] Step 3:

[0039] The feature extraction and fusion network based on the improved BiFPN is designed based on MobileNetv1 of PP-OCRv3;

[0040] After MobileNetv1, an improved BiFPN feature fusion module was introduced to extract multi-scale features of text of different shapes in plant wiring diagrams; a channel attention block was added at the end of BiFPN to enhance the information in high-level feature maps and improve the global feature fitting ability; depthwise separable convolution was used to replace all convolutions in the network to overcome the efficiency reduction problem caused by the increase in parameters; and Hardswish was used as the activation function.

[0041] Feature Map 、 、 and is the output of different layers of MobileNetv1, and the scales are 、 、 、 ; Normalize the above feature map to 256 channels as follows:

[0042]

[0043] in From 1 to 4, is the activation function Hardswish; then, upsampling is performed using nearest neighbor interpolation, and the upsampling results are weighted summed from top to bottom; the top-down fusion process is as follows:

[0044]

[0045] in From 1 to 3, is a depth-wise separable convolution, and In the top-down fusion process The trainable weights of each component of the layer are then used for downsampling, and the downsampling results are weighted and summed from bottom to top; the bottom-up fusion process is as follows:

[0046]

[0047] in From 2 to 4, 、 and In the bottom-up fusion process The trainable weights of each component of the layer; then, through The highest level feature map of the aggregation Downsample to reduce the scale to and change the number of channels from 256 to 512;

[0048] Finally, the feature map is passed through the channel attention module; first, adaptive global average pooling is performed. Then it is fed into two convolutional layers with kernel size Nonlinearity is implemented in the fully connected layer of , and a scale of The weight sequence is used to represent the contribution of each channel of the input feature map. Finally, the weight sequence is multiplied by the input feature map. The result is a feature map improved by the channel attention mechanism. The specific calculation process is as follows:

[0049]

[0050] After downsampling the result, we get a scale of Sequence feature mapping .

[0051] Step 4:

[0052] The sequence feature encoding and decoding continues to use the SVTR-based recognition head in PP-OCRv3 to output text content recognition results. The feature sequence output from step 3 is input into step 4. After a series of encoding and decoding operations by the recognition head, the predicted text content recognition result corresponding to each feature sequence is output. To address the omission of simple Chinese characters and the misrecognition of multi-line numbers, these recognition results are input into the character re-recognition module for fine-tuning to improve overall accuracy.

[0053] The character re-identification module is a post-processing method based on region expansion and cutting, and its specific approach is as follows:

[0054] S4.1 Abnormal result filtering steps:

[0055] Filter the results output by the recognition head. If there is only one Chinese character in the recognition result, perform vertical extension;

[0056] If the recognition result contains only numbers and punctuation marks, and the text candidate box corresponding to the recognition result is a square, horizontal cutting is performed;

[0057] Other recognition results are considered correct and without abnormalities;

[0058] S4.2 Vertical extension steps:

[0059] Starting from the position of the recognized single Chinese character, according to the original image and the binary map, it extends vertically towards the direction of "pixels with text" but "no text suggestions".

[0060] S4.3 Horizontal cutting steps:

[0061] In order to correct the situation where two lines of numbers are detected as a whole, the text candidate box corresponding to the incorrect recognition result is cut into two parts horizontally.

[0062] S4.4 Character re-recognition steps:

[0063] The text region corrected by vertical extension and horizontal cutting is fed into the text feature extraction and recognition head for re-recognition. Finally, the output re-recognition result is combined with the previous correct result to obtain the final recognition result.

[0064] The longitudinal extension steps of S4.2 are as follows:

[0065] Starting from the position of the recognized single Chinese character, traverse the binary image in units of pixels, both upwards and downwards;

[0066] If a direction is 0- If there is a binary image with a corresponding pixel value of 1 within the range of pixels, the traversal in this direction is abandoned;

[0067] If there is no detected text area within the range, then query whether there is a set of black pixels with high connectivity in the pixel area within the corresponding range of the original input image; if so, extend in that direction pixels, forming a new larger text area; among them, the hyperparameter Set to the original text area height * 0.5, and set the hyperparameter Set to original text area height * 0.8.

[0068] The loss function of the detection model in step 5 is as follows:

[0069]

[0070] in, Represents the relevant prediction value of the text candidate box, Represents the actual text bounding box coordinates; 、 and They represent the losses of the probability map, binary map, and threshold map prediction results in DBNet, using binary cross entropy loss, dice loss, and L1 loss respectively; and is a hyperparameter;

[0071] The loss function of the recognition model is as follows:

[0072]

[0073] in, Represents the predicted text recognition content, Represents actual text content. is the CTC loss.

[0074] By adopting the above scheme, the beneficial effects of the present invention are:

[0075] 1. The present invention designs a region re-segmentation module based on pixel row clustering, which solves the position detection problem of irregular shaped text containing vertical and horizontal characters.

[0076] 2. This paper proposes an improved BiFPN module based on channel attention mechanism and depthwise separable convolution for text feature extraction, which effectively improves the extraction accuracy of text features of different scales.

[0077] 3. The present invention introduces a character re-recognition module based on region expansion and cutting, which effectively corrects the misrecognition of simple Chinese characters and multi-line digital texts. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 These are some text instances that are difficult to detect and recognize using PP-OCRv3.

[0079] Figure 2 This is a basic flow chart of the method for automatically detecting and recognizing text in plant wiring diagrams based on the improved PP-OCRv3 of the present invention.

[0080] Figure 3 It is the overall network structure of the present invention.

[0081] Figure 4 This is a structural diagram of text position detection in the present invention (step 2).

[0082] Figure 5 This is a structural diagram of feature extraction and fusion of the present invention (step 3).

[0083] Figure 6 This is the text recognition structure diagram of the present invention (step 4).

[0084] Figure 7 This is an example result of the comparison between the present invention and other OCR models.

[0085] Figure 8 This is a quantitative comparison of the performance of the present invention and other OCR models. DETAILED DESCRIPTION

[0086] Figure 1Here are some text instances that are difficult to detect and recognize using PP-OCRv3: (a) irregularly shaped text containing horizontal and vertical characters; (b) text of various scales and shapes; (c) text containing simple Chinese characters that are easily missed by the detector; and (d) text containing dense, multi-line numbers.

[0087] Key points of technical solution:

[0088] The present invention aims to solve the limitation problem of existing deep learning-based power grid drawing text detection and recognition methods in processing multi-scale, multi-directional, and irregularly shaped compact text (such as Figure 1 As shown in FIG), a method for automatic detection and recognition of plant wiring diagram text based on improved PP-OCRv3 is proposed. The basic flow chart of the present invention is as follows: Figure 2 As shown;

[0089] The overall network structure is as follows Figure 3 As shown, the blue boxes are the components of the original PP-OCRv3, and the green boxes are the modules proposed in this invention.

[0090] The detection errors of irregular shaped texts containing vertical and horizontal characters are corrected by using the region re-segmentation module based on pixel row clustering. Figure 4 shown.

[0091] By introducing the channel attention mechanism and depth-separable convolution into the BiFPN module, the robustness of the recognition model to input images of different scales is improved. The module structure is as follows: Figure 5 shown.

[0092] The character re-identification module based on region expansion and cutting is used to reduce the adverse effects of simple characters and dense text on the model. The module structure is as follows: Figure 6 shown.

[0093] When training the model, ICDAR2019-LSVT, ICDAR2017-RCTW and Chinese Datasets (a total of 127,727 text images) were used as training sets, and ICDAR2015 (a total of 5,230 text images) was used as a validation set for general scene text recognition. In order to improve the recognition ability of the specific scene text of the power station wiring diagram, 23,377 State Grid Shanghai Electric Power Company electrical diagram text images were added during the training and validation process. During the training process, the Adam optimizer was used with a learning rate of 0.001 and a batch size of 64. Each text instance was reshaped into Pixels. Threshold for text position detection and Set to 0.6 and 3.0 respectively.

[0094] The comparative experimental results of the present invention and other OCR models are as follows: Figure 7 The quantitative analysis results of the model accuracy are shown in Figure 8 Experimental results show that the present invention can significantly improve the detection and recognition accuracy of text in plant wiring diagrams through a series of improvements to PP-OCRv3.

[0095] The following combination Figure 1 The flowchart shown illustrates the technical solution of the present invention, and the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0096] Step 1: Divide the power grid plant wiring diagram dataset into a training set and a test set, which are used for training and testing the text detection and recognition models respectively.

[0097] Step 2: Detect text positions on the input image and construct a region resegmentation module to adjust the text detection results, addressing the accurate detection of irregular text containing both horizontal and vertical text, thereby obtaining text candidate boxes. This text detection is divided into preliminary detection based on the original DBNet network model and refinement based on the region resegmentation module. First, the test drawing to be detected is input into the DBNet text detection network model for preliminary text box position detection. The input image undergoes pyramid-based feature extraction and fusion, and performs Differentiable Binarization (DB) to obtain a pixel-wise binary map. Then, through a series of post-processing, the kernel outline and kernel boundary are obtained. (The kernel represents the center area of ​​the text candidate box. Compared to the text candidate box, the smaller center area kernel can clearly separate multiple compact texts, which is very helpful for pixel-level segmentation of dense text.) Next, the resulting output is fed into the region resegmentation module for refinement of the text candidate boxes.

[0098] The algorithm of the region re-segmentation module is a text position detection post-processing method based on pixel row clustering; a region re-segmentation module based on pixel row clustering is designed based on DBNet, such as Figure 4 The specific steps are as follows:

[0099] S2.1 Abnormal area filtering steps:

[0100] Calculate the kernel outline and bounding box of each text , is the area of ​​the region surrounded by the kernel outline, It is the area of ​​the region enclosed by the kernel bounding box; the region segmentation module is used to re-segment the "problem area";

[0101] S2.2 Pixel row clustering step:

[0102] Record the width of each pixel row within the kernel outline of the "problem area" and perform row-based clustering on the outline area; convert the binary image of the "problem area" into a list The length of is the height of the "problem area" ; Indicates the rowpixelrow; By a binary Composition, representing the vertical coordinate of the pixel point in the "problem area" When , the corresponding minimum value of the horizontal axis and maximum value ;according to , get the horizontal coordinates of the starting and ending positions of each line of text pixels in the "problem area"; according to , we can get the starting and ending positions of the horizontal coordinates of each pixel row in the "problem area"; next, randomly select the text pixel rows that have not been clustered As cluster center ,Right now:

[0103]

[0104] Then from Expand to the upper and lower pixel rows respectively; if the expanded pixel row The distance to the cluster center is less than the threshold , is a hyperparameter derived from the statistical law of irregular text in the power grid drawings, and it is classified into the current cluster. The above process is expressed as:

[0105]

[0106]

[0107] Then, the cluster center is recalculated based on the newly added pixel rows as follows:

[0108]

[0109] in is the current cluster, is the number of elements in the cluster; after that, continue to expand upward and downward until the distance between the pixel row and the cluster center is greater than or equal to the threshold , or a pixel row that has been clustered, or reaches the pixel row boundary; the clustering is performed cyclically until all pixel rows are clustered;

[0110] S2.3 threshold filtering steps:

[0111] set up As the threshold of the area; if the area of ​​the class cluster is less than , then filter the class clusters.

[0112] S2.4 Text candidate box formation steps:

[0113] The fine-tuned text kernel is expanded to generate text candidate boxes.

[0114] Step 3: Construct a feature extraction and fusion network based on improved BiFPN ( Figure 5 As shown in the figure, feature information is extracted from the cropped text image to obtain sequence features. The feature extraction and fusion network based on the improved BiFPN is designed based on MobileNetv1 of PP-OCRv3;

[0115] After MobileNetv1, an improved BiFPN feature fusion module was introduced to extract multi-scale features of text of different shapes in plant wiring diagrams; a channel attention block was added at the end of BiFPN to enhance the information in high-level feature maps and improve the global feature fitting ability; depthwise separable convolution was used to replace all convolutions in the network to overcome the efficiency reduction problem caused by the increase in parameters; and Hardswish was used as the activation function.

[0116] Feature Map 、 、 and is the output of different layers of MobileNetv1, and the scales are 、 、 、 ; Normalize the above feature map to 256 channels as follows:

[0117]

[0118] in From 1 to 4, is the activation function Hardswish; then, upsampling is performed using nearest neighbor interpolation, and the upsampling results are weighted summed from top to bottom; the top-down fusion process is as follows:

[0119]

[0120] in From 1 to 3, is a depth-wise separable convolution, and In the top-down fusion process The trainable weights of each component of the layer are then used for downsampling, and the downsampling results are weighted and summed from bottom to top; the bottom-up fusion process is as follows:

[0121]

[0122] in From 2 to 4, 、 and In the bottom-up fusion process The trainable weights of each component of the layer; then, through The highest level feature map of the aggregation Downsample to reduce the scale to and change the number of channels from 256 to 512;

[0123] Finally, the feature map is passed through the channel attention module; first, adaptive global average pooling is performed. Then it is fed into two convolutional layers with kernel size Nonlinearity is implemented in the fully connected layer of , and a scale of The weight sequence is used to represent the contribution of each channel of the input feature map. Finally, the weight sequence is multiplied by the input feature map. The result is a feature map improved by the channel attention mechanism. The specific calculation process is as follows:

[0124]

[0125] After downsampling the result, we get a scale of Sequence feature mapping .

[0126] Step 4: Encode and decode the sequence features, and construct a character re-recognition module. By fine-tuning the recognition results for simple Chinese characters and multi-line numeric text, overall accuracy is improved, and text recognition results are obtained. The sequence feature encoding and decoding continues to use the SVTR-based recognition head in PP-OCRv3 to output text content recognition results. The feature sequence output from Step 3 is input into Step 4. After a series of encoding and decoding operations by the recognition head, the predicted text content recognition results corresponding to each feature sequence are output. To address the omission of simple Chinese characters and the incorrect recognition of multi-line numeric text, these recognition results are input into the character re-recognition module for fine-tuning to improve overall accuracy.

[0127] The character re-identification module is a post-processing method based on region expansion and cutting, and its specific approach is as follows:

[0128] S4.1 Abnormal result filtering steps:

[0129] Filter the results output by the recognition head. If there is only one Chinese character in the recognition result, perform vertical extension;

[0130] If the recognition result contains only numbers and punctuation marks, and the text candidate box corresponding to the recognition result is a square, horizontal cutting is performed;

[0131] Other recognition results are considered correct and without abnormalities;

[0132] S4.2 Vertical extension steps:

[0133] Starting from the position of the recognized single Chinese character, according to the original image and the binary map, it extends vertically towards the direction of "pixels with text" but "no text suggestions".

[0134] S4.3 Horizontal cutting steps:

[0135] In order to correct the situation where two lines of numbers are detected as a whole, the text candidate box corresponding to the incorrect recognition result is cut into two parts horizontally.

[0136] S4.4 Character re-recognition steps:

[0137] The text region corrected by vertical extension and horizontal cutting is fed into the text feature extraction and recognition head for re-recognition. Finally, the output re-recognition result is combined with the previous correct result to obtain the final recognition result.

[0138] The longitudinal extension steps of S4.2 are as follows:

[0139] Starting from the position of the recognized single Chinese character, traverse the binary image in units of pixels, both upwards and downwards;

[0140] If a direction is 0- If there is a binary image with a corresponding pixel value of 1 within the range of pixels, the traversal in this direction is abandoned;

[0141] If there is no detected text area within the range, then query whether there is a set of black pixels with high connectivity in the pixel area within the corresponding range of the original input image; if so, extend in that direction pixels, forming a new larger text area; among them, the hyperparameter Set to the original text area height * 0.5, and set the hyperparameter Set to original text area height * 0.8.

[0142] Step 5: Calculate the losses of the detection and recognition models by comparing the text candidate boxes and text recognition results with the corresponding true values.

[0143] The loss function of the detection model is as follows:

[0144]

[0145] in, Represents the relevant prediction value of the text candidate box, Represents the true text bounding box coordinates. 、 and They represent the losses of the probability map, binary map, and threshold map prediction results in DBNet, respectively, using binary cross entropy loss, dice loss, and L1 loss. and are hyperparameters, here set to 5 and 10.

[0146] The loss function of the recognition model is as follows:

[0147]

[0148] in, Represents the predicted text recognition content, Represents actual text content. is the CTC loss.

[0149] Step 6: Based on the loss, use the Adam optimizer to optimize the detection and recognition network models respectively.

[0150] Step 7: Repeat steps 2-6 until both networks converge and save the two model files.

[0151] Step 8: Input the test drawing into the detection model to predict the text candidate box, and then input the cropped text image into the recognition model to predict the text recognition result.

Claims

1. A method for automatic detection and recognition of text in plant wiring diagrams based on improved PP-OCRv3, characterized in that: The steps include: Step 1: Divide the power grid plant wiring diagram dataset into a training set and a test set for training and testing the text detection and recognition models, respectively. Step 2: Detect text positions in the input image and build a region re-segmentation module to adjust the text detection results. This solves the problem of accurately detecting irregular text containing both horizontal and vertical text, and thus obtains text candidate boxes. Step 3: Construct a feature extraction and fusion network based on the improved BiFPN to extract feature information from the cropped text image and obtain sequence features; The improved BiFPN-based feature extraction and fusion network is designed based on MobileNetv1 in PP-OCRv3. Following MobileNetv1, an improved BiFPN feature fusion module is introduced to extract multi-scale features of text of different shapes in plant wiring diagrams. A channel attention module is added to the end of the BiFPN to enhance the information in high-level feature maps and improve global feature fitting capabilities. Depthwise separable convolutions are used to replace all convolutions in the network to overcome the efficiency loss caused by the increase in parameters. Hardswish is used as the activation function. Step 4: Encode and decode the sequence features and build a character re-recognition module. Fine-tune the recognition results of simple Chinese characters and multi-line numeric text to improve the overall accuracy and obtain the text recognition results. Step 5: Calculate the loss of the two network models for detection and recognition by comparing the text candidate boxes and text recognition results with the corresponding true values. Step 6: Based on the loss, use the Adam optimizer to optimize the detection and recognition network models respectively; Step 7: Repeat steps 2-6 until both network models converge, and save the two model files; Step 8: Input the test drawing into the detection model to predict the text candidate box, and then input the cropped text image into the recognition model to predict the text recognition result.

2. The method for automatic detection and recognition of text in plant wiring diagrams based on improved PP-OCRv3 as claimed in claim 1, characterized in that: Step 2: The text detection process involves preliminary detection based on the original DBNet network model and refinement based on the region resegmentation module. First, the test drawing to be detected is input into the DBNet text detection network model for preliminary text box position detection. The input image undergoes pyramid-based feature extraction and fusion, and is then subjected to differentiable binarization to obtain a pixel-segmented binary image. This image is then post-processed to obtain kernel contours and kernel boundaries. The resulting output is then fed into the region resegmentation module for refinement of the text candidate boxes. The algorithm of the region re-segmentation module is a text position detection post-processing method based on pixel row clustering. A region re-segmentation module based on pixel row clustering is designed based on DBNet. The specific approach is as follows: S2.1 Abnormal area filtering steps: Calculate the kernel outline and bounding box of each text , is the area of ​​the region surrounded by the kernel outline, It is the area of ​​the region enclosed by the kernel bounding box; the region segmentation module is used to re-segment the "problem area"; S2.2 Pixel row clustering step: Record the width of each pixel row within the kernel outline of the "problem area" and perform row-based clustering on the outline area; convert the binary image of the "problem area" into a list The length of is the height of the "problem area" ; Indicates the rowpixelrow; By a binary Composition, representing the vertical coordinate of the pixel point in the "problem area" When , the corresponding minimum value of the horizontal axis and maximum value ;according to , get the horizontal coordinates of the starting and ending positions of each line of text pixels in the "problem area"; according to , we can get the starting and ending positions of the horizontal coordinates of each pixel row in the "problem area"; next, we randomly select the text pixel rows that have not been clustered As cluster center ,Right now: Then from Expand to the upper and lower pixel rows respectively; if the expanded pixel row The distance to the cluster center is less than the threshold , is a hyperparameter derived from the statistical law of irregular text in the power grid drawings, and it is classified into the current cluster. The above process is expressed as: Then, the cluster center is recalculated based on the newly added pixel rows as follows: in is the current cluster, is the number of elements in the cluster; after that, continue to expand upward and downward until the distance between the pixel row and the cluster center is greater than or equal to the threshold , or a pixel row that has been clustered, or reaches the pixel row boundary; the clustering is performed cyclically until all pixel rows are clustered; S2.3 threshold filtering steps: set up as the threshold value for the region; If the area of ​​the cluster is less than , then filter the clusters; S2.4 Text candidate box formation steps: The fine-tuned text kernel is expanded to generate text candidate boxes.

3. The method for automatic detection and recognition of text in plant wiring diagrams based on improved PP-OCRv3 as claimed in claim 1, characterized in that: Step 3: Feature Map 、 、 and is the output of different layers of MobileNetv1, and the scales are 、 、 、 ; Normalize the above feature map to 256 channels as follows: in From 1 to 4, is the activation function Hardswish; then, upsampling is performed using nearest neighbor interpolation, and the upsampling results are weighted summed from top to bottom; the top-down fusion process is as follows: in From 1 to 3, is a depth-wise separable convolution, and In the top-down fusion process The trainable weights of each component of the layer are then used for downsampling, and the downsampling results are weighted and summed from bottom to top; the bottom-up fusion process is as follows: in From 2 to 4, 、 and In the bottom-up fusion process The trainable weights of each component of the layer; then, through The highest level feature map of the aggregation Downsample to reduce the scale to and change the number of channels from 256 to 512; Finally, the feature map is passed through the channel attention module; first, adaptive global average pooling is performed; then it is fed into two convolutional layers with kernel size Nonlinearity is implemented in the fully connected layer of , and a scale of The weight sequence is used to represent the contribution of each channel of the input feature map; Finally, the weight sequence is multiplied by the input feature map; The result is a feature map improved by the channel attention mechanism; the specific calculation process is as follows: After downsampling the result, we get a scale of Sequence feature mapping .

4. The method for automatic detection and recognition of text in plant wiring diagrams based on improved PP-OCRv3 as claimed in claim 1, characterized in that: Step 4: The sequence feature encoding and decoding continues to use the SVTR-based recognition head in PP-OCRv3 to output the text content recognition results. The feature sequence output from step 3 is input into step 4. After a series of encoding and decoding operations by the recognition head, the predicted text content recognition results corresponding to each feature sequence are output. In order to solve the omission of simple Chinese characters and the incorrect recognition of multi-line numbers, the above recognition results are input into the character re-recognition module for fine-tuning to improve the overall accuracy. The character re-identification module is a post-processing method based on region expansion and cutting, and its specific approach is as follows: S4.1 Abnormal result filtering steps: Filter the results output by the recognition head. If there is only one Chinese character in the recognition result, perform vertical extension; If the recognition result contains only numbers and punctuation marks, and the text candidate box corresponding to the recognition result is a square, horizontal cutting is performed; Other recognition results are considered correct and without abnormalities; S4.2 Vertical extension steps: Starting from the position of the recognized single Chinese character, according to the original image and binary image, vertically extend to the direction of "text pixels" but "no text suggestions"; S4.3 Horizontal cutting steps: In order to correct the situation where two lines of numbers are detected as a whole, the text candidate box corresponding to the incorrect recognition result is cut into two parts horizontally; S4.4 Character re-recognition steps: The text area corrected by vertical extension and horizontal cutting is fed into the text feature extraction and recognition head for re-recognition; finally, the output re-recognition result is combined with the previous correct result to obtain the final recognition result.

5. The method for automatic detection and recognition of text in plant wiring diagrams based on improved PP-OCRv3 as claimed in claim 4, characterized in that: The longitudinal extension steps of S4.2 are as follows: Starting from the position of the recognized single Chinese character, traverse the binary image in units of pixels, both upwards and downwards; If a direction is 0- If there is a binary image with a corresponding pixel value of 1 within the range of pixels, the traversal in this direction is abandoned; If there is no detected text area within the range, then query whether there is a set of black pixels with high connectivity in the pixel area within the corresponding range of the original input image; If so, it will be extended in that direction pixels, forming a new larger text area; among them, the hyperparameter Set to the original text area height * 0.5, and set the hyperparameter Set to original text area height * 0.

8.

6. The method for automatic detection and recognition of text in plant wiring diagrams based on improved PP-OCRv3 as claimed in claim 1, characterized in that: The loss function of the detection model in step 5 is as follows: in, Represents the relevant prediction value of the text candidate box, Represents the actual text bounding box coordinates; 、 and They represent the losses of the probability map, binary map, and threshold map prediction results in DBNet, using binary cross entropy loss, dice loss, and L1 loss respectively; and is a hyperparameter; The loss function of the recognition model is as follows: in, Represents the predicted text recognition content, Indicates the actual text content; is the CTC loss.

Images