A jetson nano-based power secondary system terminal strip image recognition method and system

Abstract

The application discloses a power secondary system terminal strip image recognition method based on Jetson Nano, relates to the technical field of computer vision, and comprises the following steps: a team photographs terminal strip wiring pictures and labels the terminal strip data set by using LabelMe; a SegLink model is trained on the terminal strip data set, and position information of a labeled frame is obtained after the training is completed; a unique character segmentation network structure is used to segment text in the labeled frame, and position information of each independent character is obtained; a text feature category and coordinate sorting algorithm are used to sort the labeled frame, and a sorting result is transmitted to a Jetson Nano development board; the midpoint distance of two labeled frames is calculated, and a secondary inspection is conducted on a text recognition result by filtering the length of the labeled frame and calculating a difference value. The improved SegLink detection algorithm is applied, text detection and recognition are realized through an integrated end-to-end structure, and the error transmission risk of a pipeline system is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and in particular to a power secondary system terminal strip image recognition method and system based on JetsonNano. BACKGROUND

[0002] With the rapid development of artificial intelligence technology in the 21st century, the process of intelligentization of power equipment is accelerating. In the "Twelfth Five-Year Plan", the concept of "smart grid and power" was first proposed, and the optimization of the power supply system was clearly proposed. In the "Fourteenth Five-Year Plan", the power automation plan is more clear, and the task of intelligent transformation is proposed. In the process of maintenance and inspection of power equipment, inspection and investigation is a crucial link. However, the traditional inspection method relies on the naked eye observation and paper record of the inspection personnel to carry out investigation and feedback, which has shown serious shortcomings, and there are problems of hidden dangers caused by human subjective factors and a large amount of resource waste.

[0003] The continuous progress of computer vision technology has promoted the improvement of image recognition and text recognition technology, making it have higher accuracy and robustness. In this field, the OCR (Optical Character Recognition) technology can scan the text data and perform image analysis processing to extract the text and layout information. The current OCR algorithm performs well in processing noise-free interference or clearly printed text, and has very high accuracy. However, in the experiment, the accuracy will decrease in the case of blurred images or being blocked by a sleeve. SUMMARY

[0004] In view of the problem that the terminal strip recognition algorithm of OCR has low recognition accuracy in dealing with image blur, shielding and other situations, the present application is proposed.

[0005] Therefore, the problem to be solved by the present application is how to improve the robustness and accuracy of the terminal strip image text detection and recognition algorithm based on deep learning in complex environments.

[0006] To solve the above technical problems, the present application provides the following technical solutions:

[0007] In a first aspect, the embodiments of the present application provide a terminal block image recognition method for power secondary system based on Jetson Nano, which comprises the following steps: a team shoots a terminal block wiring picture and uses LabelMe to label, and a terminal block dataset is constructed; a SegLink model is trained on the terminal block dataset, and position information of a labeled box is obtained after the training is completed; a single-character segmentation network structure is used to segment the text in the labeled box to obtain position information of each independent character; a text feature category and coordinate sorting algorithm is used to sort the labeled box, and the sorting result is transmitted to a Jetson Nano development board; the midpoint distance of two labeled boxes is calculated, and a secondary test is performed on the text recognition result by screening the length of the labeled box and calculating the difference.

[0008] As a preferred scheme of the terminal block image recognition method for power secondary system based on Jetson Nano, wherein: the step of training the SegLink model on the terminal block dataset comprises the following steps: traversing each sample image of the terminal block training set and inputting it into the backbone network of the SegLink model for feature extraction; for each generated feature map, calculating the scaling ratio thereof relative to the original image according to the size and step length of the feature map; for each position of each feature map, generating a corresponding default box on the original image according to the calculated scaling ratio and the size of the default box, and matching it with the labeled box; using the matching information of the labeled box and the default box, calculating and updating the offset of the default box; defining a loss function as the weighted sum of position error and confidence error; iteratively processing the entire terminal block training set to obtain the position information of the labeled box, and updating the model parameters using the defined loss function.

[0009] As a preferred scheme of the terminal block image recognition method for power secondary system based on Jetson Nano, wherein: the specific formula of the scaling ratio is as follows:

[0010]

[0011] wherein s k is the scaling ratio of the feature map, m is the number of feature maps, s min is the scaling ratio of the bottommost layer, and s max is the scaling ratio of the highest layer.

[0012]

[0013] wherein s k is the scaling ratio of the feature map, a r is different aspect ratio.

[0014] As a preferred scheme of the power secondary system terminal strip image recognition method based on the Jetson Nano, wherein: the matching with the mark box comprises the following steps: calculating the IoU intersection ratio between each default box and each mark box; if the default box and the mark box have the maximum intersection ratio, it is considered that they match successfully, and the default box is set as a positive sample; for the remaining unmatched default boxes, if the maximum intersection ratio IoU of the default box and any one mark box is greater than 0.6, it is considered that they match successfully, and the default box is set as a positive sample; if the IoU of the default box and a certain mark box is less than 0.3, it is considered that the matching fails, and the default box is set as a negative sample; if the default box is not matched, the default box is set as a background sample; the negative samples are sampled, and when sampling, the confidence error is arranged in descending order, and the top-k with small confidence is selected as the training negative sample, the positive sample and the negative sample are set to be in a ratio of 1:3, and the other negative samples are discarded; the specific formula of the IoU intersection ratio is as follows:

[0015]

[0016] Wherein, A and B are the default box and the mark box.

[0017] The specific formula of the loss function is as follows:

[0018]

[0019] Wherein, N is the number of positive samples of the default box, L loc is the position error, L conf is the confidence error, c is the category confidence prediction value, l is the position prediction value of the mark box corresponding to the default box, g is the position parameter of the mark box, alpha is the weight coefficient, and x is the matching condition of the default box and the real mark box.

[0020] As a preferred scheme of the power secondary system terminal strip image recognition method based on the Jetson Nano, wherein: the mark box is sorted using a text feature category and coordinate sorting algorithm, comprising the following steps: obtaining the position information of each independent character, including the position and size of the boundary box; the boundary box according to the position information is sorted in coordinates to ensure that it is traversed in the order from left to right and from top to bottom; the sorted boundary box is traversed, the character image in each boundary box is extracted, and the characters are classified according to the character features; for the boundary box of each type of character, the midpoint coordinates of the mark box are extracted, and the mark box is sorted according to the midpoint coordinates using the coordinate sorting algorithm; the sorting result of each type of character is obtained, and the sorting result is transmitted to the Jetson Nano development board.

[0021] As a preferred scheme of the power secondary system terminal strip image recognition method based on Jetson Nano, wherein: the secondary inspection of the text recognition result comprises the following steps: calculating the midpoint distance Δd of the two labeled boxes; defining the maximum error T1 and the minimum error T2 allowed by the midpoint distance Δd of the two labeled boxes; judging whether there is missing text according to the maximum error and the minimum error of the midpoint distance Δd, and performing secondary inspection judgment; outputting the corrected labeled box and the recognized text result based on the OCR recognition and the supplemented results of the labeling; defining the maximum error T1 and the minimum error T2 allowed by the midpoint distance Δd of the two labeled boxes comprises the following steps: if the maximum error T1 of Δd does not exceed y9-y2 and the minimum error T2 of Δd does not exceed y7-y4, there is no missing target in the middle; if the maximum error T1 of Δd exceeds y9-y2 or the minimum error T2 of Δd exceeds y7-y4, and there is another labeled box above the labeled box, there is missing detection in the middle, and secondary inspection is entered.

[0022] As a preferred scheme of the power secondary system terminal strip image recognition method based on Jetson Nano, wherein: the secondary inspection of the text recognition result further comprises the following steps: reviewing the detection result, establishing a plane coordinate system with the lower left corner of the picture as the origin by loading the detected picture; defining the upper left of the labeled box as A n , the upper right as B n , the lower left as C n , and the lower right as D n , and traversing each box in the picture; selecting the box with AB length greater than AC, calculating A n -A n-1 , and obtaining a value S n ; if the value S n changes in a small range, it is a non-missing detection picture, and if it changes by 1.5 to 2 times, it is recorded as a missing detection picture, and a suffix symbol “_F” is added to the file name; the picture with the suffix “_F” is read by integrating part of the model for secondary detection, and the missing part is detected.

[0023] In a second aspect, an embodiment of the present application provides a terminal strip image recognition system for a power secondary system based on Jetson Nano, which comprises a data collection and labeling module for taking pictures of terminal strip wiring and labeling using LabelMe to build a terminal strip dataset; a SegLink model training module for training the SegLink model using the terminal strip dataset to obtain position information of the labeled boxes; a unique character segmentation network structure module for segmenting the text in the labeled boxes using the unique character segmentation network structure to obtain position information of each independent character; a labeled box sorting and classification module for sorting the labeled boxes and classifying them according to text features, including terminal serial numbers, grouping marks and sleeve marks; and a recognition result secondary inspection module for calculating the midpoint distance of two labeled boxes, judging whether there is any missed text according to the defined maximum error and minimum error, and performing secondary inspection.

[0024] In a third aspect, an embodiment of the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the steps of the terminal strip image recognition method for a power secondary system based on Jetson Nano are implemented.

[0025] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the terminal strip image recognition method for a power secondary system based on Jetson Nano are implemented.

[0026] The present application has the following advantages: the present application independently constructs a labeled dataset for a terminal strip scene; an improved SegLink detection algorithm is applied to realize text detection and recognition through an integrated end-to-end structure, thereby reducing the error propagation risk of a pipeline system; a loss function is defined to improve the detection effect of the model in the terminal strip scene; a unique character segmentation network is used to perform pixel-level segmentation on the text in the detection box, thereby improving the accuracy of character positioning and recognition in the terminal strip; the text is classified into three categories according to the visual features of the terminal strip text, and the coordinates are sorted respectively, which improves the matching accuracy of the sorting method combined with visual information; a labeled box midpoint distance algorithm is proposed to set an allowed error range to judge the missed detection and perform secondary inspection, thereby avoiding recognition errors caused by missed detection. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings. Among them:

[0028] Figure 1 Flow chart of the terminal strip image recognition method for the power secondary system based on Jetson Nano.

[0029] Figure 2 Unique character segmentation network structure of the terminal strip image recognition method for the power secondary system based on Jetson Nano.

[0030] Figure 3 Jetson Nano expansion board core module of the terminal strip image recognition method for the power secondary system based on Jetson Nano.

[0031] Figure 4 Labeling frame data of the terminal strip image recognition method for the power secondary system based on Jetson Nano.

[0032] Figure 5 Terminal strip prediction result graph of the terminal strip image recognition method for the power secondary system based on Jetson Nano. DETAILED DESCRIPTION

[0033] In order to make the above-mentioned purposes, features and advantages of the present application more apparent and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0034] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, but the present application can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the scope of the present application, therefore the present application is not limited to the specific embodiments disclosed below.

[0035] Secondly, the "one embodiment" or "embodiment" referred to herein means that the specific features, structures or characteristics can be included in at least one implementation of the present application. "In one embodiment" appearing in different places in the specification does not mean the same embodiment, nor is it an independent or alternative embodiment.

[0036] Embodiment 1

[0037] Reference Figures 1-5For the first embodiment of the present application, the embodiment provides a power secondary system terminal strip image recognition method based on Jetson Nano, comprising,

[0038] S1: The team takes pictures of terminal strip wiring and uses LabelMe for labeling to build a terminal strip dataset.

[0039] Specifically, after the team investigated various dataset websites and open source projects at home and abroad, no available power dataset was found that met the requirements. Therefore, the team decided to take pictures of terminal strip wiring by itself and used LabelMe labeling tool to label these pictures to build a terminal strip dataset.

[0040] It should be noted that in this embodiment, more than 1000 pictures are taken and labeled, and these terminal strip datasets are divided into 80% terminal strip training set and 20% terminal strip test set.

[0041] S2: Train the SegLink model on the terminal strip dataset, and obtain the position information of the labeled box after training.

[0042] Specifically, the following steps are included:

[0043] S2.1: Traverse each sample image of the terminal strip training set and input it into the backbone network of the SegLink model for feature extraction.

[0044] S2.2: For each generated feature map, calculate its scaling ratio relative to the original image according to the size and step length of the feature map.

[0045] Preferably, the process of obtaining the position information of the default box in the original image is as follows:

[0046]

[0047] Where s k is the scaling ratio of the feature map, m is the number of feature maps, s min is the scaling ratio of the bottom layer, and s max is the scaling ratio of the highest layer.

[0048]

[0049] Where s k is the scaling ratio of the feature map, and a r is different aspect ratio.

[0050] S2.3: For each position of each feature map, generate the corresponding default box on the original image according to the calculated scaling ratio and the size of the default box, and match it with the labeled box.

[0051] Specifically, the IoU intersection over union between each default frame and each labeled frame is calculated; if the default frame has the maximum intersection over union with the labeled frame, it is considered to be a successful match, and this default frame is set as a positive sample; for the remaining unmatched default frames, if the maximum intersection over union IoU of the default frame with any labeled frame is greater than 0.6, it is considered to be a successful match, and this default frame is set as a positive sample; if the IoU of the default frame with a certain labeled frame is less than 0.3, it is considered to be a failed match, and this default frame is set as a negative sample; if the default frame is not matched, it is set as a background sample; the negative samples are sampled, and the sampling is performed in descending order according to the confidence error, and the top-k with small confidence is selected as the training negative sample, the ratio of positive samples to negative samples is set to 1:3, and the other negative samples are discarded.

[0052] Specifically, the specific formula of the IoU intersection over union is as follows:

[0053]

[0054] Where A and B are default frames.

[0055] S2.4: Using the matching information of the labeled frame and the default frame, the offset of the default frame is calculated and updated.

[0056] S2.5: The loss function is defined as the weighted sum of the position error and the confidence error.

[0057] Further, the specific formula is as follows:

[0058]

[0059] Where N is the number of positive samples of the default frame, L loc is the position error, L conf is the confidence error, c is the class confidence prediction value, l is the position prediction value of the labeled frame corresponding to the default frame, g is the position parameter of the labeled frame, a is the weight coefficient, and x is the matching condition of the default frame and the real labeled frame.

[0060] S2.6: The entire terminal row training set is iteratively processed to obtain the position information of the labeled frame, and the defined loss function is used to update the model parameters.

[0061] S3: The text in the labeled frame is segmented using the unique character segmentation network structure to obtain the position information of each independent character.

[0062] Further, the unique character segmentation network structure includes a main network for text detection and a branch network for segmentation, wherein the main network is responsible for generating a text detection frame, and the branch network performs pixel-level segmentation on the text in each detection frame.

[0063] Specifically, the image or the label frame subjected to text detection is taken as input, the backbone network performs text detection on the input to generate a text detection frame covering the text in each label frame; the branch network performs pixel-level segmentation on the text in each text detection frame to segment the text into independent characters; the pixels belonging to the same character are connected together according to the pixel labels to form a character connection component; the upper left corner coordinates, width and height of the bounding box are calculated according to the pixel positions in the character connection component; the bounding box is post-processed, such as removing small character connection components or merging adjacent character connection components; and the position information of each independent character including the position and size of the bounding box is output.

[0064] S4: using a text feature category and coordinate sorting algorithm to sort the label frame, and transmitting the sorting result to the Jetson Nano development board.

[0065] Specifically, the following steps are included:

[0066] S4.1: obtaining the position information of each independent character including the position and size of the bounding box.

[0067] Preferably, the position information of each independent character including the position and size of the bounding box is obtained in the step S3.

[0068] S4.2: performing coordinate sorting on the bounding box according to the position information to ensure traversal in the order from left to right and from top to bottom.

[0069] Specifically, all the bounding boxes are sorted in the order of increasing left boundary x coordinate, and if the x coordinates are the same, the upper boundary y coordinate is sorted in the order of increasing y coordinate.

[0070] S4.3: traversing the sorted bounding boxes, extracting the character image in each bounding box, and classifying the characters according to the text features of the characters.

[0071] Specifically, the sorted bounding boxes are traversed, the character image in each bounding box is extracted, and the aspect ratio ratio of the character image is calculated; if the ratio is in the range of [0.2, 0.6], the character is marked as a pure number (terminal serial number); if the ratio is in the range of [0.6, 2.0], the character is marked as a letter; for a character image containing a number, the inclination angle is calculated, and it is checked whether it exceeds 15 degrees, and if it exceeds 15 degrees, vertical correction is performed; for a character image containing a letter, it is checked whether it contains a special symbol, and if it contains a special symbol, the character is marked as a number + letter + symbol (sleeve marker), otherwise it is marked as a number + letter (grouping marker).

[0072] Further, according to the character features, the characters can be divided into three categories, including pure numbers (terminal serial numbers), numbers + letters (group markers), and numbers + letters + symbols (sleeve markers).

[0073] S4.4: For the bounding box of each type of character, the midpoint coordinates of the labeling box are extracted, and the labeling box is sorted according to the midpoint coordinates using a coordinate sorting algorithm.

[0074] Specifically, the midpoint coordinates are sorted in ascending order from top to bottom according to the vertical coordinates of the midpoints.

[0075] S4.5: The sorting results of each type of character are obtained and transmitted to the Jetson Nano development board.

[0076] Preferably, the core module of the Jetson Nano expansion board is as shown in Figure 3 .

[0077] S5: Calculate the midpoint distance Δd of the two labeling boxes, and perform secondary verification on the text recognition result by filtering the labeling box length and calculating the difference.

[0078] Specifically, the following steps are included:

[0079] S5.1: Calculate the midpoint distance Δd of the two labeling boxes.

[0080] Preferably, in this embodiment, as shown in Figure 4 , the coordinate information of labeling box A1 is A(x1, y1), B(x2, y2), C(x3, y3), and D(x4, y4), so the midpoint coordinate data of labeling box A1 is F1(x5, y5); the coordinate information of labeling box A2 is E(x6, y6), F(x7, y7), G(x8, y8), and H(x9, y9), so the midpoint coordinate data of labeling box A2 is F2(x 10 , y 10 ), then the specific formula of the midpoint distance Δd of labeling box A1 and labeling box A2 is as follows:

[0081] Δd = y 10 -y5

[0082] Where y5 is the midpoint coordinate data of labeling box A1, and y 10 is the midpoint coordinate data of labeling box A2.

[0083] S5.2: Define the maximum error T1 and the minimum error T2 allowed for the midpoint distance Δd of the two labeling boxes.

[0084] If the maximum error T1 of Δd does not exceed y9-y2 (or y8-y1) and the minimum error T2 of Δd does not exceed y7-y4 (or y6-y3), then the target object is not missed in the middle.

[0085] If the maximum error T1 of the midpoint distance Δd exceeds y9-y2(or y8-y1) or the minimum error T2 of the midpoint distance Δd exceeds y7-y4(or y6-y3), and there is another bounding box above (or below) the bounding box, there is a missing detection, and secondary detection is entered.

[0086] S5.3: Determine whether there is missing text according to the maximum error and minimum error of the midpoint distance Δd, and perform secondary detection.

[0087] Further, the detection result is reviewed, the picture detected by loading the model is established as the origin of the plane coordinate system at the lower left corner of the picture, the measurement is determined by the size of the picture, the left upper corner of the bounding box is defined as A n , the right upper corner is defined as B n , the left lower corner is defined as C n , and the right lower corner is defined as D n , and each box in the picture is traversed; the boxes with AB length greater than AC are selected, A n -A n-1 is calculated, and a value S n is obtained; if the value S n changes in a small range, it is a non-missing detection picture, and if it changes by 1.5 to 2 times, it is recorded as a missing detection picture, and the file name suffix is added with the symbol “_F”; the pictures with the suffix “_F” are read by integrating part of the model for secondary detection, and the missing part is detected.

[0088] S5.4: Output the corrected bounding box and the recognized text result based on the results of OCR recognition and annotation supplement.

[0089] Specifically, the OCR result and the bounding box coordinates after supplement recognition in S5.3 are collected; the OCR text result is updated to the corresponding supplemented recognition bounding box; the original bounding box coordinates and text result are combined with the supplemented result; all bounding boxes are sorted according to the coordinate order, and the sorted bounding box and the corresponding OCR recognized text result are output.

[0090] Further, the embodiment also provides a terminal block image recognition system based on Jetson Nano for power secondary system, which comprises a data collection and labeling module, a SegLink model training module, a single character segmentation network structure module, a labeled box sorting and classification module, and a recognition result secondary inspection module.

[0091] The embodiment also provides a computer device suitable for the terminal block image recognition method based on Jetson Nano for power secondary system, which comprises a memory and a processor.

[0092] The computer device can be a terminal, which comprises a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner. The wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0093] The embodiment also provides a storage medium, which stores a computer program, and the program is executed by a processor to implement the following steps: a team photographs terminal block connection line pictures and labels the pictures using LabelMe to construct a terminal block dataset; a SegLink model is trained on the terminal block dataset, and position information of a labeling box is obtained after the training is completed; a single character segmentation network structure is used to segment text in the labeling box to obtain position information of each independent character; a text feature category and coordinate sorting algorithm is used to sort the labeling box, and a sorting result is transmitted to a Jetson Nano development board; a midpoint distance of two labeling boxes is calculated, and a secondary inspection is performed on a text recognition result by filtering a labeling box length and calculating a difference value.

[0094] To sum up, the application independently constructs a labeling dataset for a terminal block scene; an improved SegLink detection algorithm is applied to realize text detection and recognition through an integrated end-to-end structure, thereby reducing error transmission risk of a pipeline system; a loss function is defined to improve detection effect of the model in the terminal block scene; a single character segmentation network is used to perform pixel-level segmentation on text in a detection box, thereby improving accuracy of character positioning and recognition in the terminal block; according to visual features of terminal block text, the text is classified into three categories, and coordinates are sorted respectively, thereby improving matching accuracy through the sorting method combined with visual information; a labeling box midpoint distance algorithm is proposed to set an allowed error range, thereby judging a missed detection situation and performing a secondary inspection, and thereby avoiding recognition errors caused by missed detection.

[0095] Embodiment 2

[0096] Reference Figures 1-5 For the second embodiment of the application, the embodiment provides a power secondary system terminal block image recognition method based on a Jetson Nano. In order to verify the beneficial effects of the application, economic benefit calculation and simulation experiments are used for scientific demonstration.

[0097] Specifically, after investigating various dataset websites and open source projects at home and abroad, the team did not find a power dataset that meets the requirements. Therefore, the team decides to take the way of independently photographing terminal block connection line pictures, and uses LabelMe labeling tool to label these pictures to construct a terminal block dataset, and part of the text information of the dataset has an angle θ.

[0098] Further, SegLink is used as a text detection model, and the model input picture size is 2048x1536; training hyperparameter settings are as follows: a learning rate is 1e-3, a training epoch number is 30, and a batch size is 8; after training, a text detection Recall on a test set is 96%, and a Precision is 94%.

[0099] Further, a single character segmentation network model is adopted as shown in Figure 2 The model can segment the text pixel level in the detection frame, accurately obtain the coordinates of each character, and the final segmentation accuracy is: terminal serial number accuracy 98%, grouping mark accuracy 95%, and sleeve mark accuracy 92%.

[0100] Preferably, as shown in the terminal row prediction result map of Figure 3 Through coordinate sorting and text classification, the sorted character coordinates and text categories can be obtained, the allowable error is defined as ±20 pixels, and it is found that there may be a missed detection between terminal 5 and terminal 7 by calculating the midpoint distance. The midpoint distance is 110 pixels, which exceeds the maximum allowable error of 100 pixels.

[0101] Further, after loading the original picture, supplementary detection is performed to detect the missed terminal 6, the OCR recognition result is 6, the coordinates are (55, 65, 75, 85), and the final corrected detection result is output according to the coordinate order.

[0102] Specifically, the specific process data statistics are as follows: original annotation frame number 1200, original OCR detection number 1100, supplementary detection frame number 100, supplementary OCR detection number 95, final annotation frame number 1200, and final OCR detection number 1195.

[0103] Further, the comparison between the present application and the prior art is shown in Table 1.

[0104] Table 1 Comparison between the present application and the prior art

[0105] Contrast index Prior art Our invention Text positioning accuracy 80％ 90％ Character recognition accuracy 81％ 95％ Missed detection rate 10％ 2％ Segmentation accuracy 72％ 90％ Dataset Public dataset Autonomous construction

[0106] Preferably, from Table 1, it can be seen that the text positioning accuracy is improved by 10 percentage points, from 80% to 90%, the detection effect is better; the character recognition accuracy is improved by 14 percentage points, from 81% to 95%, the recognition effect is better; the missed detection rate is reduced by 8 percentage points, from 10% to 2%, the missed detection situation is greatly reduced; the segmentation accuracy is improved by 18 percentage points, from 72% to 90%, the segmentation effect is significantly improved; and the autonomous terminal row dataset is constructed, which is more in line with the scene requirements.

[0107] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit it. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present application, which should be covered in the scope of the claims of the present application.

Claims

1. A terminal strip image recognition method for power secondary system based on Jetson Nano, characterized in that: The application relates to a terminal block text recognition method based on SegLink model and a terminal block text recognition device. A team takes pictures of terminal block connection lines and labels the pictures using LabelMe to build a terminal block dataset; A SegLink model is trained on the terminal block dataset, and position information of the labeled boxes is obtained after the training is completed; A single-character segmentation network structure is used to segment the text in the labeled boxes to obtain position information of each independent character; Text feature categories and coordinate sorting algorithms are used to sort the labeled boxes, and the sorting results are transmitted to a JetsonNano development board; The midpoint distance of two labeled boxes is calculated, and a secondary test is performed on the text recognition result by screening the length of the labeled boxes and calculating the difference value; The secondary test on the text recognition result comprises the following steps: The midpoint distance Δd of two labeled boxes is calculated; The maximum error T1 and the minimum error T2 of the midpoint distance Δd of two labeled boxes are defined; Whether there is missing text is judged according to the maximum error and the minimum error of the midpoint distance Δd, and a secondary test is performed; Based on the OCR recognition and the supplementary results of the labeling, a corrected labeled box and a recognized text result are outputted; The maximum error T1 and the minimum error T2 of the midpoint distance Δd of two labeled boxes are defined, and the specific steps are as follows: If the maximum error T1 of Δd does not exceed y9-y2 and the minimum error T2 of Δd does not exceed y7-y4, there is no missing target in the middle; wherein y2 represents the vertical coordinate of the upper right corner of the first labeled box, y4 represents the vertical coordinate of the lower right corner of the first labeled box, y7 represents the vertical coordinate of the upper right corner of the second labeled box, and y9 represents the vertical coordinate of the lower right corner of the second labeled box; If the maximum error T1 of Δd exceeds y9-y2 or the minimum error T2 of Δd exceeds y7-y4, and there is another labeled box above the labeled box, there is missing detection in the middle, and a secondary test is performed; The secondary test on the text recognition result further comprises the following steps: The detection result is reviewed, and a plane coordinate system is established with the lower left corner of the picture as the origin; Define the top left of the bounding box as A n , the top right as B n , the bottom left as C n , and the bottom right as D n , traverse each box in the picture; filter out the boxes whose AB length is greater than AC, calculate A n -A n-1 , and get a value S n ; If the value S n If the value S is within a small range, it is a non-missed picture. If the value S is within a range of 1.5 to 2 times the size, it is recorded as a missed picture, and a suffix symbol "_F" is added to the file name. The picture with the suffix "_F" is read through the integrated part model for secondary detection, and the missing part is detected.

2. The Jetson Nano-based power secondary system terminal strip image recognition method of claim 1, wherein: The SegLink model is trained on the terminal block dataset, and the specific steps are as follows: Each sample image of the terminal block training set is traversed, and is input into the backbone network of the SegLink model for feature extraction; For each generated feature map, the scaling ratio of the feature map relative to the original image is calculated according to the size and step length of the feature map; For each position of each feature map, a corresponding default box is generated on the original image according to the calculated scaling ratio and the size of the default box, and is matched with the labeled box; The offset of the default box is calculated and updated using the matching information of the labeled box and the default box; The loss function is defined as the weighted sum of the position error and the confidence error; The entire terminal block training set is iteratively processed to obtain the position information of the labeled box, and the model parameters are updated using the defined loss function.

3. The Jetson Nano-based power secondary system terminal strip image recognition method of claim 2, wherein: The specific formula of the scaling ratio is as follows: wherein s k is the scaling ratio of the feature map, m is the number of feature maps, s min is the scaling ratio of the bottommost layer, s max is the scaling ratio of the highest layer; where s k is the scaling of the feature map, a r is the different aspect ratios.

4. The Jetson Nano-based power secondary system terminal strip image recognition method of claim 2, wherein: The matching with the labeled box comprises the following steps: The IoU intersection ratio between each default box and each labeled box is calculated; If the default frame and the labeled frame have the maximum intersection over union, it is considered that they match successfully, and this default frame is set as a positive sample; For the remaining unmatched default frame, If the maximum intersection over union IoU of the default frame and any one labeled frame is greater than 0.6, it is considered that they match successfully, and this default frame is set as a positive sample; If the IoU of the default frame and a certain labeled frame is less than 0.3, it is considered that the matching fails, and this default frame is set as a negative sample; If the default frame is not matched, this default frame is set as a background sample; The negative samples are sampled, and when sampling, the confidence error is arranged in descending order, and the top-k with small confidence is selected as the training negative sample, the ratio of positive samples to negative samples is set to 1:3, and other negative samples are discarded; The specific formula of the IoU intersection over union is as follows: Wherein, A and B are the default frame and the labeled frame respectively; The specific formula of the loss function is as follows: where N is the number of positive samples of the default box, L loc is the position error, L conf is the confidence error, c is the class confidence prediction value, l is the position prediction value of the labeled box corresponding to the default box, g is the position parameter of the labeled box, a is the weight coefficient, and x is the matching condition of the default box and the real labeled box.

5. The Jetson Nano-based power secondary system terminal strip image recognition method of claim 1, wherein: The steps of using the text feature category and coordinate sorting algorithm to sort the labeled frame include the following steps: Obtain the position information of each independent character, including the position and size of the bounding box; Coordinate sorting is performed on the bounding box according to the position information to ensure traversal in the order from left to right and from top to bottom; Traverse the sorted bounding box, extract the character image in each bounding box, and classify according to the character features; For the bounding box of each type of character, the midpoint coordinates of the labeled frame are extracted, and the coordinate sorting algorithm is used to sort the labeled frame according to the midpoint coordinates; Get the sorting result of each type of character, and transmit the sorting result to the JetsonNano development board.

6. A terminal strip image recognition system for power secondary systems based on Jetson Nano, based on the terminal strip image recognition method for power secondary systems based on Jetson Nano in any one of claims 1-5, characterized in that: Further comprising, A data collection and labeling module for shooting terminal block wiring pictures and labeling using LabelMe to build a terminal block dataset; A SegLink model training module for training the SegLink model using the terminal block dataset to obtain the position information of the labeled frame; A single character segmentation network structure module for segmenting the text in the labeled frame using the single character segmentation network structure to obtain the position information of each independent character; A labeled frame sorting and classification module for sorting the labeled frame and classifying according to the text features, including terminal number, grouping mark and sleeve mark; A secondary inspection module for calculating the midpoint distance of two labeled frames, judging whether there is missing text according to the defined maximum error and minimum error, and performing secondary inspection. 7.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is characterized in that: The processor executes the computer program to realize the steps of the power secondary system terminal block image recognition method based on Jetson Nano in any one of claims 1-5.

8. A computer readable storage medium having stored thereon a computer program, characterized in that: The computer program is executed by the processor to realize the steps of the power secondary system terminal block image recognition method based on Jetson Nano in any one of claims 1-5.

Images