A method for verifying the hard pressure plate of relay protection in smart substation based on target detection model

Abstract

The present invention discloses a verification method for the hard pressure plate of relay protection in intelligent substation based on target detection model, specifically: download the benchmark file corresponding to the work task, scan the QR code of the screen cabinet to establish a connection with the benchmark file; after completing the photo taking, the YOLOv4‑tiny model will be called and the benchmark file will be parsed synchronously, and finally the verified result will be displayed to the staff; if there is no missed detection or false detection, the staff can directly save the record locally; if missed detection or false detection occurs, the staff needs to correct the corresponding hard pressure plate. The present invention does not require additional modification of the protection screen cabinet, and uses the hard pressure plate pictures taken on site to produce a data set, studies the use of data enhancement algorithms to expand the data set, and further solves the problems of missed detection and false detection in previous inventions, and has higher universality and robustness.

Description

Technical Field

[0001] The present invention belongs to the technical field of power grid operation and maintenance, and in particular relates to a method for calibrating a relay protection hard pressure plate of an intelligent substation based on a target detection model. Background Art

[0002] Smart grid is an inevitable trend in the development of power grid, as mentioned in Reference 1: Zhang Yao, Wang Aohan, Zhang Hong. Overview of China's Smart Grid Development [J]. Power System Protection and Control, 2021, 49(5): 180-. and Reference 2: Yu Yixin. Urgency and long-term implementation of smart grid [J]. Power System Protection and Control, 2019, 47(17): 1-5. However, relay protection is a relatively weak link, especially the inspection of the protection hard pressure plate. This work is an important part of the secondary operation and maintenance of substations. Once mis-operation or leakage occurs, it may cause an accident. Therefore, this work requires high reliability, which makes it difficult to improve its intelligence level. As a result, manual inspection is still the main method at this stage, but due to the heavy workload and huge energy consumption, it will inevitably cause visual fatigue and low work efficiency. In order to improve the level of intelligence, reduce the workload of the operation and maintenance team, and avoid accidents caused by misoperation as much as possible, researchers from all parties have successively carried out research on this issue.

[0003] Starting from the hardware level, some researchers have proposed the concept of electric pressure plate, such as reference 3: Yao Qinghua, Dong Linhui, Zhou Lei, et al. Design and application research of intelligent remote electric pressure plate [J]. Power System Protection and Control, 2015, 43(20): 143-149. Some researchers have also proposed the idea of ​​intelligent pressure plate, such as reference 4: Niu Zhigang, Jia Tengfei, Xu Qinglu, et al. Application of intelligent pressure plate system in Fengtun 500kV substation in Qiqihar [J]. Power System Protection and Control, 2010, 38(23): 219-222+245. However, this method will undoubtedly increase the complexity of equipment modification. Therefore, some researchers have proposed image processing methods with the help of computer technology, such as Reference 5: Pan Chengcheng, Lu Quan, Hu Likun, et al. Research on positioning and state recognition of protective pressure plates based on color separation and morphological feature analysis [J / OL]. Electrical Measurement and Instrumentation: 1-8 [2021-03-07]. http: / / kns.cnki.net / kcms / detail / 23.1202.TH.20201127.1043.010.html., Reference 6: Fu Wenlong, Tan Jiawen, Wu Xichun, et al. State recognition of protection pressure plates in intelligent substations based on image processing and morphological feature analysis [J]. Power Automation Chemical Equipment, 2019, 39(07): 203-207., Reference 7: Lv Zhining, Ning Baifeng, Zhou Ziqiang. Research on positioning and state recognition of protective pressure plate based on image processing [J]. Digital Technology and Application, 2018, 36(06): 82-83. This method requires preprocessing of the collected image (morphological processing is generally used) to meet a certain standard, and then reference 5 uses color separation, reference 6 uses morphological feature analysis, and reference 7 uses HOG+SVM to identify the pressure plate state. However, this method has relatively high requirements on the image. In the process of standardization, many threshold selections are difficult to be universal.Due to the rise of artificial intelligence in recent years, some researchers have tried to find new methods in this field, most of which use image recognition methods, such as Reference 8: Yu Yong, Ruan Haojie, Shen Gang, et al. Application of artificial intelligence AI technology in protection plate state recognition [J]. Integrated Circuit Applications, 2020, 37(06): 122-123. Reference 9: Lv Jiawei, Liu Ruirong. Design of relay protection plate throw-in and throw-out state recognition based on color template matching [J]. Electromechanical Information, 2018(36): 115-11 6. Document 10: Cheng Zhiqiu, Peng Yongjian, Li Zhengqiang. Research on automatic scanning and recognition technology of hard pressure plate of distribution network automation switch [J]. Adhesion, 2020, 43(07): 78-81. Document 11: Ren Junjie, Jiang Lan. Image recognition system of relay protection pressure plate of power system [J]. Journal of Beijing Union University (Natural Science Edition), 2004(02): 60-64. Document 12: Deng Yingsong, Duan Qingang, Song Xiaosong. Protection pressure plate activation and deactivation state identification method based on image recognition [J]. Shaanxi Electric Power, 2015,43(10):49-53+67. Same as reference 13: Ruan Haojie, Wang Chunya, Wang Hui, et al. Research and application of protection plate state recognition technology based on image recognition [J]. Electric Power Equipment Management, 2020(01):139-140. However, the first step of image recognition still requires image preprocessing. Although reference 9 adds probabilistic Hough transform and reference 10 uses algorithms such as AdaBoost, the recognition rate is still low. Some researchers also choose clustering algorithms, such as reference 14: Xu Chao, Chen Hao, Liu Shaoqing, et al. Protection plate state recognition technology based on model clustering matching and morphological feature recognition [J]. Shaanxi Electric Power, 2017,45(01):32-36+85. Same as reference 15: Liang Chen, Sun Jianwen, Wang Lanyu, et al. Application of clustering and evidence theory to realize substation protection plate state verification [J]. Power System Technology, 2020,44(06):2343-2349. However, this method is to match by color, and the method of distinguishing by color has been tried before. In the process of collecting images, it is found that the colors of many hard pressed boards are very close to the color of the screen cabinet, which is camel-like, which makes it difficult to distinguish the hard pressed boards from the background.Some researchers have also tried deep learning methods, such as: Reference 16: Wu Di, Tang Xiaobing, Li Peng, Yang Zengli, Wen Bo, Li Hengxuan. Substation relay protection device status monitoring technology based on deep neural network [J]. Power System Protection and Control, 2020, 48(05): 81-85. DOI: 10.19783 / j.cnki.pspc.190516. Convolutional neural network + feature transformation was used; Reference 17: Wang Wei, Zhang Yanlong, Zhai Denghui, et al. Intelligent identification of substation pressure plate status based on OpenCV+SSD deep learning model [J / OL]. Electrical Measurement and Instrumentation: 1-10 [2020-09-13]. http: / / kns.cnki.net / kcms / detail / 23.1202.TH.20200827.1838.052.html. and reference 18: Zhou Ke, Yang Qianwen, Wang Yaoyi, et al. An improved SSD algorithm for pressure plate state recognition [J / OL]. Electrical Measurement and Instrumentation: 1-10 [2020-09-17]. http: / / kns.cnki.net / kcms / detail / 23.1202.TH.20200917.1717.002.html. The SSD target detection model was adopted; Reference 19: Leng Conglin. Research on substation pressure plate switch state recognition system based on machine vision [D]. Wuhan University of Technology, 2019. The YOLOv3 target detection model was adopted, but its model accuracy was lower than the model in the article. Some researchers have taken a different approach. Reference 20: Jian Xuezhi, Liu Zijun, Wenminghao, et al. Application of AR augmented reality technology in operation and inspection of secondary equipment in substations[J]. Power System Protection and Control, 2020, 48(15): 170-176. Applying AR augmented reality technology in operation and maintenance, Reference 21: Xu Genyang, Weng Shizhuang, Sun Changxiang, et al. Accurate identification method of protection pressure plate status based on phase characteristics [J]. Journal of Anhui University (Natural Science Edition), 2020, 44(03): 38-42. Others have proposed a pressure plate status recognition method based on phase competition coding, and Reference 22: Gao Yuansheng, Chen Qiang, Xiong Xiaofu, et al. Intelligent calibration method of relay protection pressure plate [J]. Journal of Chongqing University, 2015, 38(06): 91-98. and Reference 23: Zhang Man, Feng Ning, Liu Yingtong, et al. 500kV smart substation pressure plate status monitoring and intelligent calibration technology [J]. Electrical Engineering, 2018(24): 89-90+93. Relatively special intelligent calibration methods have been proposed, but in general, these methods are difficult to promote and are slightly lacking in practicality.

[0004] Chinese patent CN113794277A (application number: 202110980430.4) invented a pressure plate status image recognition method and system. This invention is also based on deep learning and also uses mobile devices instead of manual inspections. However, the accuracy of the deep learning model of this invention is lower than that of the present invention, and it does not further solve the problems of missed detection and false detection of the model. Although the missed alarm rate and false alarm rate of its model are not higher than 3%, when such problems occur, this invention can only repeat the operation again, which will reduce work efficiency.

[0005] Chinese patent CN113255827A (application number: 202110669583.7) proposes a system and method for identifying the state of a relay pressure plate based on the YOLO Nano algorithm. The invention is also based on the YOLO algorithm and a mobile device, but the mobile device in the invention only completes the identification of the pressure plate state, and the core verification work in the inspection work is not completed on the mobile device. Therefore, the invention requires frequent data exchange between the mobile device and the server, and the amount of data exchanged at a single time is larger.

[0006] The key problems to be solved by the present invention are as follows:

[0007] 1) The accuracy of the recognition algorithm is insufficient. The recognition model designed in the invention with patent publication number (CN) 113794277A still has a nearly 3% missed alarm rate and false alarm rate, which is particularly obvious when the imaging angle is tilted or the light is uneven. The occurrence of missed alarms and false alarms will cause the inspectors to take multiple images, reducing work efficiency and is an urgent problem to be solved.

[0008] 2) Correction and completion of false detection and missed detection. The target detection algorithm based on deep learning cannot guarantee 100% recognition accuracy, that is, false detection and missed detection cannot be completely eliminated. For example, the invention with the patent publication number (CN) 113255827A can only take an image again and re-detect when false detection and missed detection occur. In many cases, secondary detection cannot solve false detection and missed detection, so it is necessary to solve this problem from other angles. Summary of the invention

[0009] In order to solve the above problems, the present invention provides a method for calibrating a relay protection hard pressure plate of an intelligent substation based on a target detection model.

[0010] A method for verifying a relay protection hard pressure plate of an intelligent substation based on a target detection model of the present invention comprises the following steps:

[0011] Step 1: Download the benchmark file corresponding to the work task, and scan the screen cabinet QR code to establish a connection with the benchmark file.

[0012] The staff needs to first download the benchmark file corresponding to this work task according to their work number, and then start the inspection work; during the inspection process, they need to scan the QR code of each screen cabinet first, and use the scan result to establish a connection with the benchmark file.

[0013] Step 2: Data collection and data enhancement.

[0014] Pictures of the hard pressing plate were taken on site to create the data set. The LabelImg tool was used to annotate the pictures. There are two types of annotated labels: 0-green frame and 1-orange frame, representing the two categories of "invest" and "return", respectively.

[0015] Then the original image is normalized and preprocessed.

[0016] Step 3: Parse the benchmark file.

[0017] The benchmark file is converted into an XML file, which includes the substation name, protection device name, protection device identification code, hard pressure plate name and normal operating status information; wherein, x and y in the Pad tag represent the hard pressure plate in the xth row and yth column, and the position without a hard pressure plate is a spare hard pressure plate; parsing of the benchmark file is achieved through a DOM parser.

[0018] Step 4: YOLOv4-tiny model call.

[0019] Choose to use OpenCV's Dnn module to call the model; before calling, add OpenCV to the project as a module, and then copy the OpenCV library file to the jniLibs folder.

[0020] Step 5: Missed detection determination and automatic completion.

[0021] S51. Bubble sort is performed according to the y coordinate of each bounding box, and the entire list is traversed to determine the position where a line break is required.

[0022] S52. After determining the position of the line break, bubble sort is also performed within the line according to the x coordinate of the bounding box to form a two-dimensional array; and during this traversal process, the total number of detection results, the starting coordinate of each line, the total width and height of all bounding boxes, and the total interval between two bounding boxes are counted to obtain the average starting coordinate of each line, the average width, average height, and average interval of the bounding box.

[0023] S53. Finally, traverse once more and use the same conditions as the first traversal to determine the position of the line break. According to the principle that there are 9 hard pressure plates in each row of the relay protection panel cabinet, determine whether the number of hard pressure plates in the row is sufficient to 9. If sufficient, there is no need to supplement. If insufficient, supplement.

[0024] S54, when completing, first determine whether completion is required based on the difference between the average starting coordinates of the row and the coordinates of the beginning of the row; round down the quotient of the difference and the average width of the bounding box to determine the number of items that need to be completed; and complete from the current beginning of the row to the front; then determine whether completion is required within the row based on the difference between the bounding boxes within the row; finally, if the number of bounding boxes of the row is still less than 9 after completion of both the beginning of the row and the inside of the row, complete from the current end of the row to the back.

[0025] Step 6: False detection determination and convenient correction.

[0026] When the recognition result of the model is inconsistent with the reference file, it is judged as a false detection.

[0027] When it is the recognition result of the model, the user needs to change the recognition result first, and then mark the currently taken picture. After that, this type of picture and the corrected result are added to the model training set, and the model is continuously trained using this type of picture to improve the generalization of the model. When it is a benchmark file, the user needs to mark the benchmark, and this type of benchmark will be manually checked and modified later. After each inspection is completed, all work data will be transmitted back to the server, and subsequent model training and benchmark verification will be completed on the server side.

[0028] Furthermore, the normalization and preprocessing in step 2 are specifically as follows:

[0029] S21, normalize the original image to a size of 416×416.

[0030] S22. Randomly reduce the image and distort its length and width; the reduction degree is a random number in the interval [0.8, 1], and the distortion degree is a random number in the interval [0.5, 1.5].

[0031] S23, fill the remaining part after reduction with black to ensure that the image size is 416×416.

[0032] S24, flip the image with a probability of 50%.

[0033] S25. Perform Gaussian blur processing with a probability of 50%.

[0034] S26. Perform color gamut distortion in the HSV color space. Hue is randomly rotated counterclockwise or clockwise in the interval (-Hue, Hue). Saturation and Value are first randomly selected from positive numbers within 100, and then the reciprocal is selected with a probability of 50%.

[0035] Furthermore, the specific process of implementing the model call through the Dnn module in step 4 is as follows:

[0036] S41. Load OpenCV library files.

[0037] S42. Use the Dnn module to load the network configuration and training weight files to build the model network.

[0038] S43, read the image and convert it into the Bolb standard input format that can be used by the model.

[0039] S44. Input the converted image into the network to obtain the output layer of YOLO, and obtain the detection result list through forward propagation.

[0040] S45, looping through the detection result list, filtering out detection results that are less than a threshold, and obtaining a one-to-one corresponding bounding box list, category list, and confidence list.

[0041] S46. Perform non-maximum suppression to reduce the number of overlapping bounding boxes.

[0042] S47, draw a bounding box and save the result image.

[0043] Furthermore, in step S51, the position where a line break is required is determined, and the determination criteria include three criteria:

[0044] a. The y coordinate of the upper left corner of the next box is greater than the y coordinate of the lower right corner of the previous box.

[0045] b. If the bounding boxes of two rows overlap, the y coordinate of the upper left corner of the next box is smaller than the y coordinate of the lower right corner of the previous box.

[0046] c. The y coordinate of the upper left corner of the next box is greater than the y coordinate of the upper left corner of the previous box by the height of one box.

[0047] Furthermore, the configuration of the model training in step 6 is as follows: the image size of the input network is 416×416, the initial learning rate is 0.00261, the total number of iterations is 6000, and the learning rate is reduced by 1 / 10 at the 4800th and 5400th iterations respectively.

[0048] The beneficial technical effects of the present invention are:

[0049] 1) Taking into full consideration the requirements for safety and reliability in power work, the insufficient accuracy defect existing in the invention with the invention patent publication number (CN) 113794277A is overcome. Aiming at the problem of model training data for protecting hard pressure plates, a more suitable data enhancement algorithm is designed and implemented, which further improves the generalization of the model and enhances the detection accuracy of the model.

[0050] 2) Taking into account the reflection phenomenon of the glass door of the protective screen cabinet and the edge distortion of the camera imaging when using a mobile terminal for shooting during on-site work, the judgment of missed detection and false detection of the data after model prediction was studied and implemented, and the automatic completion of missed detection hard pressure plates and the convenient correction of false detection hard pressure plates were achieved, overcoming the unavoidable missed detection and false detection phenomena in deep learning detection methods such as invention patent publication numbers (CN) 113794277A and 113255827A.

[0051] Based on the above reasons, combined with the widespread popularity of mobile smart devices, the present invention does not require additional modifications to the protection screen cabinet, solves the problems of missed detection and false detection in previous inventions, and has higher universality and robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is a technical flow chart of the intelligent substation relay protection hard pressure plate calibration method of the present invention.

[0053] Figure 2 This is an example diagram of the benchmark file of the present invention.

[0054] Figure 3 This is a demonstration diagram of the missed detection completion function (a is before completion, b is after completion).

[0055] Figure 4 The figure shows the false detection situation.

[0056] Figure 5 The following diagram shows the model training process (a is the avg loss curve, b is the training result diagram).

[0057] Figure 6 This is a scene simulation display diagram (a is the initial state, b is the intermediate state, and c is the final state). DETAILED DESCRIPTION

[0058] The present invention is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0059] The process of the intelligent substation relay protection hard pressure plate calibration method based on the target detection model of the present invention is as follows: Figure 1As shown, the staff needs to download the benchmark file corresponding to this work task according to their work number, and then they can start the inspection work. During the inspection process, each screen cabinet needs to scan its QR code first, and the scan result is used to establish a connection with the benchmark file. After taking pictures, the call of the YOLOv4-tiny model and the analysis of the benchmark file will be carried out simultaneously, and finally the verified results will be displayed to the staff. At this time, if there is no missed detection or false detection, the staff can directly save this record locally; if missed detection or false detection occurs, the staff only needs to correct the corresponding hard pressure plate without repeating the previous steps. After the work record of this screen cabinet is saved, the staff can go to the next screen cabinet. When all screen cabinets are inspected, they can go to the area with smooth network and upload all the data of this work back to the server.

[0060] The present invention provides a method for verifying the relay protection hard pressure plate of an intelligent substation based on a target detection model as follows:

[0061] 1. Data Collection

[0062] Considering the actual working scene of the inspectors, the hard plate pictures collected on site are used to create the data set in order to reproduce the actual environment as much as possible. The specific reproduction scene is as follows:

[0063] Shooting angle: Take a group of pictures with the phone and the display cabinet relatively parallel, because such pictures are easier to identify. However, during the shooting process, because the height of the hard pressure plate area is different, the user needs to squat frequently to complete the shooting, which greatly reduces the user experience. Therefore, under the premise that there is no coverage or obstruction between the hard pressure plates, take another group of pictures without squatting.

[0064] Lighting problem: At first, I took a group of pictures under natural light and under lighting, but there was a problem of reflection in both cases. After continuous attempts, I found that using the flash of the mobile phone can greatly reduce the degree of reflection, so I took another group of pictures using the flash.

[0065] After the image collection is completed, the LabelImg tool is used to annotate the captured images. There are two types of annotated labels: 0 (green frame) and 1 (orange frame), representing the two categories of "cast" and "return", respectively.

[0066] 2. Data Augmentation

[0067] In deep learning, data sets are very important. Without sufficient data sets as support, the model is prone to overfitting. In other words, if the data set is insufficient, the model is likely to learn many unnecessary features. For example, when identifying a computer, the model may think that an object must not only have the appearance of a computer, but also must be at a certain angle and brightness to be a computer. Obviously, this view is very one-sided, and data enhancement can make the data set richer and more diverse, so that the model can achieve the purpose of "knowledgeable".

[0068] The data enhancement is as follows:

[0069] (1) Normalize the original image to a size of 416×416.

[0070] (2) Randomly reduce the image and distort its length and width; the reduction degree is a random number in the interval [0.8, 1], and the distortion degree is a random number in the interval [0.5, 1.5].

[0071] (3) Fill the remaining part after reduction with black to ensure that the image size is 416×416.

[0072] (4) Flip the image with a probability of 50%.

[0073] (5) Perform Gaussian blur processing with a probability of 50%.

[0074] (6) Perform color gamut distortion in the HSV color space; Hue is randomly rotated counterclockwise or clockwise in the interval (-Hue, Hue), and Saturation and Value are first randomly selected from positive numbers within 100, and then the reciprocal is taken with a probability of 50%.

[0075] 3. Benchmark file design and analysis

[0076] Inspection work is usually completed by two people, and one person will be responsible for reviewing the benchmark documents and the status of the protective hard plate on site. In order to reduce the complexity and save human resources, the article converts the benchmark files into XML files to achieve paperless operation. The operating benchmark of the protective hard plate can be obtained by parsing the XML file, and it can be compared with the model prediction results to complete the inspection and verification work.

[0077] The XML format of the benchmark file is as follows Figure 2 As shown, it includes information such as substation name, protection device name, protection device identification code, hard pressure plate name, normal operation status, etc. The x and y in the Pad tag represent the hard pressure plate in the xth row and yth column, and the position without hard pressure plate is a spare hard pressure plate.

[0078] The benchmark file needs to be downloaded from the server before each work, because all operations are completed offline during the entire inspection process. Parsing the benchmark file is achieved through the DOM parser. The essence of DOM parsing is to assemble the data into a "tree" and then parse the XML file through the relationship between nodes. The parsing difficulty is not high, but it is more cumbersome.

[0079] 4. Model building and calling

[0080] The resource files of the Android system are roughly divided into two types: compilable resource files in the res directory and native resource files in the assets directory. Files in the assets folder are not precompiled like XML and Java files, but always maintain the original file format, so the article chooses to embed the model's training files (including weight files and configuration files, etc.) in the assets folder. In addition, the absolute path of the training file is required in the process of calling the model, but the absolute path of the file in the assets folder cannot be obtained. Therefore, the article adopts a preloading strategy: when the APP is started for the first time, the training file must first be loaded into the cache folder with the same name as the APP through the byte stream method to obtain the absolute path of the training file.

[0081] After the model is built, the article chooses to use OpenCV's Dnn module to call the model, and the version of OpenCV is 3.4.11 (in order to ensure that OpenCV has good support for the YOLOv4 series, its version cannot be lower than 3.4.11 after testing). Before calling, you need to add OpenCV to the project as a module. Then copy the OpenCV library file to the jniLibs folder. The specific process of implementing model calling through the Dnn module is as follows:

[0082] (1) Load OpenCV library files;

[0083] (2) Use the Dnn module to load the network configuration and training weight files to build the model network;

[0084] (3) Read the image and convert it into the Bolb standard input format that can be used by the model;

[0085] (4) Input the converted image into the network to obtain the output layer of YOLO, and obtain the detection result list through forward propagation;

[0086] (5) Loop through the detection result list, filter out the detection results that are less than the threshold, and obtain a one-to-one corresponding bounding box list, category list, and confidence list;

[0087] (6) Perform non-maximum suppression to reduce the number of overlapping bounding boxes;

[0088] (7) Draw the bounding box and save the resulting image.

[0089] 5. Missed detection and automatic completion

[0090] In the actual testing process, due to factors such as reflection and occlusion, missed detections still occur, which requires code completion later.

[0091] When the model predicts, it will first predict the target area, that is, Figure 3 The area within each box in (a) (usually called a bounding box in current detection) will then make a prediction of the target category of the target area, which is represented in the figure as a rectangle representing "disconnected" and a circle representing "closed". After the prediction is completed, the coordinates of each vertex of the bounding box and the category of the target in the bounding box can be obtained, but the bounding box is unordered and needs to be determined based on its coordinates to determine whether it is missed. After continuous testing and improvement, the current missed detection completion function is implemented as follows:

[0092] First, bubble sort is performed according to the y coordinate of each bounding box, so that the detection results in the same row will be next to each other;

[0093] Then traverse the entire list to determine where to wrap. There are three criteria, which are in an "or" relationship:

[0094] (1) The y coordinate of the upper left corner of the next box is greater than the y coordinate of the lower right corner of the previous box;

[0095] (2) If the bounding boxes of two rows overlap (due to the large inclination of the shooting angle), the y coordinate of the upper left corner of the next box is smaller than the y coordinate of the lower right corner of the previous box, but this value will not be too large and is temporarily set to be less than a fixed value;

[0096] (3) Or there may be a special case where the y coordinate of the upper left corner of the next box is greater than the y coordinate of the upper left corner of the previous box by a value of the height of the box. However, because the box height is not a fixed value, this value is temporarily set to 3 / 4 of the average box height.

[0097] After determining the position of the line break, bubble sort is also performed within the line according to the x coordinate of the bounding box to form a two-dimensional array. In addition, during this traversal process, the total number of detection results, the starting coordinate of each line, the total width and height of all bounding boxes, and the total interval between two bounding boxes are counted to obtain the average starting coordinate of each line, the average width, average height, and average interval of the bounding box;

[0098] Finally, traverse it again and use the same conditions as the first traversal to determine the position of the line break. According to the principle that there are 9 hard pressure plates in each row of the relay protection panel cabinet, determine whether the number of hard pressure plates in the row is sufficient to 9. If it is sufficient, there is no need to supplement it. If it is not sufficient, it needs to be supplemented.

[0099] When completing, first determine whether completion is needed based on the difference between the average starting coordinates of the row and the coordinates of the beginning of the row. Round down the quotient of the difference and the average width of the bounding box to determine the number of items that need to be completed. Complete from the beginning of the row to the front. Then determine whether the row needs to be completed based on the difference between the bounding boxes within the row. Finally, if the number of bounding boxes in the row is still less than 9 after completing both the beginning of the row and the inside of the row, complete from the end of the row to the back.

[0100] When completing the bounding box, the following method is used: first, the starting point x coordinate plus the average interval is used as the starting point x coordinate of the first bounding box; the starting point coordinate plus the average width is used as the ending point x coordinate; the starting point y coordinate is used as the starting point y coordinate of the first bounding box; the starting point y coordinate plus the average height is used as the ending point y coordinate; thus, a complete bounding box is obtained. The remaining bounding boxes are drawn in the same way. Figure 3 (b) is the completed image, using Photoshop to simulate the reflection.

[0101] 6. False positive detection and convenient correction

[0102] False positives are when the model recognizes something wrong, but the model itself cannot determine whether it has recognized something wrong, so it needs data to compare it with, i.e., the hard plate benchmark. When the model's recognition result is inconsistent with the benchmark, the program determines it as a false positive.

[0103] Therefore, there are two reasons for false detection: model recognition errors and benchmark file errors. When it is the former, the user needs to change the recognition result first, and then mark the currently taken picture. After that, such pictures and the corrected results will be added to the model training set, and the model will be continuously trained using such pictures to improve the generalization of the model. When it is the latter, the user needs to mark the benchmark, and such benchmarks will be manually checked and modified later. After each inspection is completed, all work data will be transmitted back to the server, and subsequent model training and benchmark verification will be completed on the server side.

[0104] So far, there are three types of "boxes" in the figure, namely the boxes correctly identified by the model, the boxes completed after missed detection, and the boxes detected by mistake. The three sources are distinguished by three colors: green, blue, and red. The user needs to complete the boxes and the boxes detected by mistake.

[0105] exist Figure 4In the above example, the method of reproducing missed detection is the same as in the previous section. However, the current model accuracy is high, and it is difficult to reproduce the recognition error. Here, we make a wrong change to the benchmark file to achieve the same false detection effect.

[0106] When correcting, if the user clicks anywhere in the bounding box on the screen, the APP will pop up a prompt box to display the status of the hard pressure plate in the currently selected box and provide change options. The implementation of this function requires rewriting the Android touch event - onTouch method. First, in this method, the screen coordinates clicked by the user can be easily obtained, and then the getImageMatrix method of the ImageView control (the control used to display the hard pressure plate image) is called to obtain the image matrix, and then the inverse matrix is ​​calculated. Through the mapping of the inverse matrix, the actual coordinates in the image after the screen coordinates are converted can be obtained, and then the position can be determined in which bounding box. Then a prompt box pops up to let the user select the state to be changed. Finally, according to the user selection result returned by the prompt box, OpenCV is called to redraw the bounding box, thus completing a false detection correction. When all bounding boxes are corrected, other colors are "eliminated". This method of distinguishing by color can greatly relieve visual fatigue.

[0107] 7. Model training

[0108] The machine configuration used for training the model is as follows:

[0109] System: Windows 10;

[0110] CPU: Core TM i7-9700K;

[0111] GPU: NVIDIA RTX 2080 SUPER.

[0112] The hyperparameter configuration for model training is shown in Table 1:

[0113] Table 1 Parameters in training

[0114]

[0115] The image size of the input network is 416×416, the initial learning rate is 0.00261, the total number of iterations is 6000, and the learning rate is reduced by 1 / 10 at the 4800th and 5400th iterations respectively. The average loss (avgloss) of the model during training is as follows Figure 5 As shown in (a), the final avg loss is less than 1, which is 0.6771. Figure 5 (b) The mAP of the model shown reaches 99.44%.

[0116] Table 2 Model performance comparison

[0117]

[0118] Table 2 compares the model in this article with the SSD target detection model used in reference

[16] and the improved SSD algorithm model used in reference

[17] . The YOLOv3 target detection model used in reference

[18] only gives a model recognition rate of 98.95%, but does not provide a definition and calculation method for the recognition rate, nor other indicators, so Table 2 does not provide a comparison. In addition, the article has previously used YOLOv3, and after testing, the accuracy of the YOLOv3 model is only 95.45%.

[0119] After the training was completed, the actual test was carried out using a Samsung Note9 smartphone (CPU is Qualcomm Snapdragon 845, running memory is 6GB, and the overall performance belongs to the mid-range queue). The article randomly selected 25 pictures in the test set. The recognition accuracy of YOLOv4-tiny is 98.64%, and the recognition speed is about 1s. Table 3 is the detailed information of the YOLOv4-tiny model test results.

[0120] Table 3 Mobile phone test table

[0121]

[0122] 8. Work environment simulation display

[0123] Assume that there is strong reflection at the hard plate in the first row and fourth column of the captured image, so that the model cannot recognize the hard plate here, resulting in missed detection; similarly, modify the benchmark file to simulate the false detection phenomenon. Then the initial state is as follows Figure 6 As shown in (a), the bounding box in the first row and fourth column (because the color has been turned to black in the figure, the specific location of the bounding box is specially indicated) is blue, which is the completed bounding box; the bounding box in the first row and fifth column is red, which is the false detection bounding box. Click the position of the blue box and the red box respectively, and the APP will pop up a prompt box, such as Figure 6 (b). The prompt box will display the clicked location and the current state of the hard pressure plate at that location, and provide a change option. When the change state selected by the user is different from the current state, the bounding box will be redrawn, as shown in Figure 6(c). The shapes of the two bounding boxes change and the color turns green.

[0124] After modification, all boundary boxes turn green, so that the user's operation results on the screen cabinet can be saved.

[0125] Using bounding boxes of different colors to distinguish different situations helps improve recognition and is very simple to operate. It not only helps reduce the accident rate, but also greatly improves work efficiency.

[0126] The present invention applies the YOLOv4-tiny model to mobile smart terminals, completes the state identification of the protective hard pressure plate with the support of OpenCV, designs and uses the benchmark file in XML format, parses it and compares it with the model prediction result, and then assists with the missed detection completion and false detection correction strategies, so that the mobile terminal can replace manual inspection work.

[0127] Since there are two main types of protective hard pressure plates: compression type and plug-in type, and their structures are relatively simple and their activation and deactivation states are relatively easy to identify, this article proposes a new intelligent verification technology for relay protection hard pressure plates based on the lightweight and fast YOLOv4-tiny target detection model.

[0128] The mean average precision (mAP) of the model of the present invention reaches 99.44%, and the precision, recall and F1 scores are all 0.98. With the support of OpenCV, the state identification and inspection verification of the protective hard pressure plate can be completed on the mobile terminal. Although the model accuracy has reached a very high standard, the missed detection and false detection of the model are still inevitable in the actual testing process. Therefore, the present invention realizes the automatic completion of missed detection hard pressure plates (referred to as missed detection completion), and the user can directly correct the falsely detected hard pressure plate (referred to as false detection correction) by clicking on the screen of the mobile device. With the assistance of these two strategies, the user only needs to shoot once to complete the inspection of a protective screen cabinet, which will greatly improve the work efficiency. The protection hard pressure plate data on the mobile terminal is easy to use across platforms, which can effectively improve the intelligent operation and maintenance level of relay protection.

Claims

1. A method for verifying the relay protection hard plate of an intelligent substation based on a target detection model, characterized in that: The following steps are involved: Step 1: Download the benchmark file corresponding to the work task, scan the QR code of the screen cabinet to establish a connection with the benchmark file; The staff needs to download the benchmark file corresponding to this work task according to their work number, and then start the inspection work; During the inspection, each cabinet needs to have its QR code scanned first, and the scan result is used to establish a connection with the reference file; Step 2: Data collection and data enhancement; Take pictures of the hard plate on site to create the data set; use the LabelImg tool to annotate the pictures. There are two types of annotated labels: 0-green frame and 1-orange frame, representing the two categories of "invest" and "return" respectively; Then normalize and preprocess the original image; Step 3: Parse the benchmark file; The benchmark file is converted into an XML file, which includes the substation name, protection device name, protection device identification code, hard pressure plate name and normal operating status information; wherein x and y in the Pad tag represent the hard pressure plate in the xth row and yth column, and the position without hard pressure plate is the spare hard pressure plate; parsing the benchmark file is realized by a DOM parser; Step 4: YOLOv4-tiny model call; Choose to use OpenCV's Dnn module to call the model; before calling, add OpenCV to the project as a module, and then copy the OpenCV library file to the jniLibs folder; Step 5: Missed detection determination and automatic completion; S51, perform bubble sorting according to the y coordinate of each bounding box, traverse the entire list, and determine the position where a line break is required; S52, after determining the position of the line break, bubble sort is also performed in the line according to the x coordinate of the bounding box, so as to form a two-dimensional array; and in this traversal process, the total number of detection results, the starting coordinate of each line, the total width and height of all bounding boxes and the total interval between two bounding boxes are counted, so as to obtain the average starting coordinate of each line, the average width, average height and average interval of the bounding box; S53, traverse once more, and use the same conditions as the first traversal to determine the position of the line break, and determine whether the number of hard pressure plates in the row is sufficient to 9 according to the principle that each row of the relay protection panel cabinet has 9 hard pressure plates. If sufficient, no need to fill in, otherwise fill in; S54, when completing, first determine whether completion is required based on the difference between the average starting coordinates of the row and the coordinates of the beginning of the row; round down the quotient of the difference and the average width of the bounding box to determine the number of items to be completed; and complete from the current beginning of the row to the front; then determine whether completion is required within the row based on the difference between the bounding boxes within the row; finally, if the number of bounding boxes of the row is still less than 9 after completion of both the beginning of the row and the inside of the row, complete from the current end of the row to the back; Step 6: False detection determination and convenient correction; When the recognition result of the model is inconsistent with the reference file, it is judged as a false detection; When it is the recognition result of the model, the user needs to change the recognition result first, and then mark the currently taken picture. After that, this type of picture and the corrected result are added to the model training set, and the model is continuously trained using this type of picture to improve the generalization of the model. When it is a benchmark file, the user needs to mark the benchmark, and this type of benchmark will be manually checked and modified later. After each inspection is completed, all work data will be transmitted back to the server, and subsequent model training and benchmark verification will be completed on the server side.

2. According to the target detection model-based intelligent substation relay protection hard pressure plate verification method according to claim 1, it is characterized in that: The normalization and preprocessing in step 2 are specifically as follows: S21, normalize the original image to a size of 416×416; S22, randomly shrink the image and distort its length and width; the shrinkage degree is a random number in the interval [0.8, 1], and the distortion degree is a random number in the interval [0.5, 1.5]; S23, fill the remaining part after the reduction with black to ensure that the image size is 416×416; S24, flip the image with a probability of 50%; S25, perform Gaussian blur processing with a probability of 50%; S26. Perform color gamut distortion in the HSV color space. Hue is randomly rotated counterclockwise or clockwise in the interval (-Hue, Hue). Saturation and Value are first randomly selected from positive numbers within 100, and then the reciprocal is selected with a probability of 50%.

3. According to the target detection model-based intelligent substation relay protection hard pressure plate calibration method according to claim 1, it is characterized in that: The specific process of implementing the model call through the Dnn module in step 4 is as follows: S41, load OpenCV library files; S42. Use the Dnn module to load the network configuration and training weight files to build the model network; S43, read the image and convert it into the Bolb standard input format that can be used by the model; S44, input the converted image into the network, obtain the output layer of YOLO, and obtain the detection result list through forward propagation; S45, looping through the detection result list, filtering out detection results less than a threshold, and obtaining a one-to-one corresponding bounding box list, category list, and confidence list; S46, performing non-maximum suppression to reduce the number of overlapping bounding boxes; S47, draw a bounding box and save the result image.

4. According to the target detection model-based intelligent substation relay protection hard pressure plate verification method of claim 1, it is characterized in that: In step S51, the position where a line break is required is determined according to three criteria: a. The y coordinate of the upper left corner of the next box is greater than the y coordinate of the lower right corner of the previous box; b. If the bounding boxes of two rows overlap, the y coordinate of the upper left corner of the next box is smaller than the y coordinate of the lower right corner of the previous box; c. The y coordinate of the upper left corner of the next box is greater than the y coordinate of the upper left corner of the previous box by the height of one box.

5. According to the target detection model-based intelligent substation relay protection hard pressure plate calibration method of claim 1, it is characterized in that: The configuration of the model training in step 6 is: the image size of the input network is 416×416, the initial learning rate is 0.00261, the total number of iterations is 6000, and the learning rate is reduced by 1 / 10 at the 4800th and 5400th iterations respectively.

Images