A power plant key facility anomaly identification method based on YOLOA

Abstract

The present application relates to power plant inspection robot and fixed camera background image processing technical field, specifically, a kind of power plant key facility anomaly identification method based on YOLOA.One kind of power plant key facility anomaly identification method based on YOLOA, comprising the following steps: S1, using YOLOA network training power plant collected leakage data, then the trained model is deployed to algorithm server;S2, power inspection robot reaches specified position and rotates holder, and the video camera is aimed at identification area after interception 3s visible light and infrared video;S3, B transform is used to fuse the B component of visible light image and infrared image, to obtain the fused video;S4, the fused video is input network prediction and is displayed in front-end page.Compared with prior art, the abnormal condition of power plant can be more accurately identified;Using video detection, it is not easy to appear false detection and missed detection and can be in real time in front-end page display detection condition.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of power plant inspection robots and fixed camera background image processing technology, in particular to a power plant key facility anomaly identification method based on YOLOA. BACKGROUND

[0002] The production process environment of the power plant is complex, with characteristics such as high temperature, high pressure, high noise, dense pipelines and valves, etc. Abnormal conditions such as leakage occur from time to time, which poses a great safety hazard. Not only does it seriously threaten the health and safety of operating personnel, but it can also cause a series of faults and accidents that threaten the safe and stable operation of the generator set, and also affects the efficiency and economic benefits of power plant power production.

[0003] The abnormal conditions of the power plant include leakage of steam, oil and water in key areas such as oil depots and ammonia stations. In view of the current problems existing in power plants, the accurate detection of abnormal conditions in key areas by using robot binoculars and fixed point cameras has great practical significance for the sustainability of power supply.

[0004] In recent years, target detection algorithms have matured and are widely used in various fields. Common target detection algorithms include Faster-RCNN, SSD, YOLO series, VisionTransformer, etc. The Faster-RCNN algorithm has higher detection accuracy than SSD, but the detection speed is too slow. The SSD algorithm has fast detection speed but low detection accuracy. Recently, VisionTransformer has been very popular in the field of target detection, but its performance on small data sets is not good and is prone to overfitting. YOLOX, as a new generation of target detection algorithm of YOLO series, has made improvements on the basis of YOLOv3 and has good detection effect and detection speed in target detection. The author of T-YOLOX adds a lightweight attention mechanism module to the YOLOX algorithm, introduces a channel mixing technique, and replaces the last layer of the backbone network with a lightweight Transformer module, which has good detection effect in the scene of fire. SUMMARY

[0005] To overcome the shortcomings of the prior art, the present application provides a power plant key facility anomaly identification method based on YOLOA, which can more accurately identify abnormal conditions in power plants. Video detection is used, which is less likely to miss or miss detection and can display the detection status in real time on the front-end page. Once an abnormal condition occurs, the staff can be reminded in time.

[0006] To achieve the above purpose, a power plant key facility anomaly identification method based on YOLOA is designed, characterized by the following steps:

[0007] S1, train the leakage data collected by the power plant using the YOLOA network, and then deploy the trained model to the algorithm server;

[0008] S2, the power inspection robot reaches the specified position and rotates the holder, and the dual-view camera is aligned with the identification area to intercept 3s of visible light and infrared video;

[0009] S3, B transform is used to fuse the B component of the visible light image and the infrared image to obtain the fused video;

[0010] S4, input the fused video into the network for prediction and display on the front-end page, and count the number t of c greater than 0.8 after video detection, and finally judge the size of t and 0.6f, if t>0.6f, the power plant confirms that an abnormal condition has occurred, wherein f is the total frame number of 3s video, and c is the confidence.

[0011] In the step S1, in order to make the network converge faster, the specific process is as follows:

[0012] S11, first use the YOLOA network for large data sets to obtain pre-training weights and then use them for the data set;

[0013] S12, the parameter setting of the neural network is width=640, height=640, learning-rate=1e-3, epochs=300, and after 300 rounds of training, the loss is reduced to 0.04;

[0014] S13, the loss calculation formula is: Loss=a*Lr+Lo+Lc+L1, wherein Lr is the regression loss, Lo is the target loss, Lc is the classification loss, and L1 is the L1 regular term loss;

[0015] S14, the network post-processing is to solve the problem of target missing detection caused by directly deleting highly overlapping targets in dense scenes, and the M_NMS calculation formula is: Wherein, M represents the current highest confidence detection box, Ci represents a certain candidate detection box, and t represents the threshold;

[0016] S15, finally deploy the trained model to the algorithm server to detect the leakage video of the power plant.

[0017] In the step S2, the dual-view camera intercepts the visible light and infrared video after aligning with the identification area, and in order to facilitate the fusion of the intercepted visible light and infrared video, they must be synchronized.

[0018] In step S3, the B component of the visible light is fused with the gray scale image of the infrared image by using the B transformation method, and a B' component is obtained after fusion, and then the B', C and S components are inversely transformed into a color image, and the formula is #G'=0.3*R+0.6*G+0.1*B; B=(R+G+B+G') / 4, wherein B is brightness, C is chroma, S is saturation, and G' is gray scale.

[0019] Because the input network prediction image is a color image, after the B component of the visible light is fused with the gray scale image of the infrared image, the B', C and S components need to be inversely transformed into a color image, and the formula of the chroma C and the saturation S is

[0020] In step S4, the confidence is set to 0.8, and then each frame of the fused 3s video is detected, when the confidence is less than or equal to 0.8, the original image is pushed to the front end display through the flask framework, when the confidence is greater than 0.8, the abnormal image is pushed to the front end display, after the 3s video is detected, the number of times that the confidence is greater than 0.8 in the video is counted, if the number of times is greater than 0.6 times of the total frame number, the power plant confirms that an abnormal condition occurs, and an alarm needs to be sent at this time.

[0021] Compared with the prior art, the power plant key facility abnormality identification method based on YOLOA can more accurately identify the abnormal condition of the power plant; by using video detection, false detection and missed detection are not easy to occur, and the detection condition can be displayed in real time on the front end page, and once an abnormal condition occurs, the staff can be reminded in time. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 It is a schematic diagram of the network model of YOLOA.

[0023] Figure 2 It is a schematic diagram of Mosaic data enhancement.

[0024] Figure 3 It is a flowchart of the implementation mode of the channel attention mechanism.

[0025] Figure 4 It is a flowchart of the implementation mode of the spatial attention mechanism.

[0026] Figure 5 , Figure 6 It is a loss curve diagram of training and verification.

[0027] Figure 7 It is an image after threshold adjustment.

[0028] Figure 8 It is an image of the abnormality detection result.

[0029] Figure 9 for the fused image. DETAILED DESCRIPTION

[0030] The application will be further described below with reference to the accompanying drawings.

[0031] As Figures 1 to 6 shown, a power plant key facility anomaly identification method based on YOLOA includes the following steps:

[0032] S1, using a YOLOA network to train the leakage data collected by the power plant, and then deploying the trained model to an algorithm server;

[0033] S2, the power inspection robot arrives at the specified position to rotate the holder, and the dual-view camera is aimed at the identification area to intercept 3s of visible light and infrared video;

[0034] S3, B transform is used to fuse the B component of the visible light image and the infrared image to obtain the fused video;

[0035] S4, input the fused video into the network for prediction and display on the front-end page, count the number t of times that c is greater than 0.8 after video detection, and finally judge the size of t and 0.6f, if t>0.6f, the power plant confirms that an abnormal condition has occurred, wherein f is the total frame number of 3s video, and c is the confidence.

[0036] In step S1, in order to make the network converge faster, the specific process is as follows:

[0037] S11, first use the YOLOA network to obtain pre-training weights on a large data set and then use it on the data set;

[0038] S12, the parameter setting of the neural network is width=640, height=640, learning-rate=1e-3, epochs=300, and after 300 rounds of training, the loss is reduced to 0.04;

[0039] S13, the calculation formula of the loss is: Loss=a*Lr+Lo+Lc+L1, wherein Lr is the regression loss, Lo is the target loss, Lc is the classification loss, and L1 is the L1 regular term loss;

[0040] S14, the post-processing of the network is to solve the problem of target missing detection caused by directly deleting highly overlapping targets in dense scenes, and the calculation formula of M_NMS is: Wherein, M represents the current highest confidence detection box, Ci represents a certain candidate detection box, and t represents the threshold value;

[0041] S15, finally deploy the trained model to the algorithm server to detect the leakage video of the power plant.

[0042] In step S2, the dual-view camera captures visible light and infrared video after aligning with the identified area. The visible light and infrared video captured at this time must be synchronized for easy fusion.

[0043] In step S3, the B component of the visible light and the grayscale image of the infrared image are fused by using the B transformation method. After fusion, the B', C, and S components are inverse transformed into a color image. The formula is #G' = 0.3*R + 0.6*G + 0.1*B; B = (R + G + B + G') / 4, where B is brightness, C is chroma, S is saturation, and G' is grayscale.

[0044] Because the input network prediction image is a color image, after the B component of the visible light and the grayscale image of the infrared image are fused, the B', C, and S components are inverse transformed into a color image. The formula for chroma C and saturation S is

[0045] In step S4, the confidence is set to 0.8, and then each frame of the fused 3s video is detected. When the confidence is less than or equal to 0.8, the original image is pushed to the front end for display through the flask framework. When the confidence is greater than 0.8, the abnormal image is pushed to the front end for display. After detecting the 3s video, the number of frames with a confidence greater than 0.8 is counted. If it is greater than 0.6 times the total number of frames, the power plant confirms that an abnormal condition has occurred, and an alarm needs to be sent.

[0046] Embodiment:

[0047] As shown in Figure 1 YOLOA is divided into four parts in structure: input (Input), backbone (Backbone), feature fusion (FPN), and classification and regression (Head). In order to reduce the parameters and complexity of the network as much as possible, the CBAM module uses a smaller convolution kernel, i.e., a 3*3 convolution kernel. The YOLOA algorithm also uses a feature pyramid (Feature Pyramid Networks, FPN). The feature pyramid structure can realize multi-scale detection because FPN can fuse both low-level feature information and high-level feature information.

[0048] YOLOA network structure training process:

[0049] S1, the original image is not distorted scaling and using Mosaic and Mixup method for data enhancement. The data preprocessing is an important step of deep learning, which occupies most of the algorithm time, the preprocessing includes: data collection, data cleaning and data labeling. First, data collection, the data of the application is derived from the video taken by the power plant; second, data cleaning, the collected video needs to be converted into pictures and delete blurred, repeated pictures; finally, data labeling, the data set adopts the data format of PascalVOC, and the collected data is labeled by using the labeling software labelImg, including three categories: 1000 water leakage, 1500 oil leakage and 2000 steam leakage. In view of the problem of class imbalance, the method of oversampling and undersampling is adopted, and the data amount after balancing the categories is: 1000 water leakage, 1000 oil leakage and 1000 steam leakage. The total data set of this experiment is 3000, 2800 pictures are divided into training set, validation set and test set according to the proportion of 7:2:1, and the remaining 200 are background pictures. Data enhancement can increase the size of the data set and the diversity of the sample, can solve the problem of overfitting and improve the generalization ability of the model, common data enhancement has flip, scaling, cropping, affine transformation, etc., Mosaic data enhancement can greatly enrich the background of the object, but Mosaic also has the disadvantage that the enhanced data is far away from the real distribution of natural pictures. Therefore, the application uses Mosaic data enhancement in the first 80% of the epoch in the experiment, and no longer uses the data enhancement method in the last 20%. The result of Mosaic data enhancement in the experiment is shown in Figure 2 . Mixup data enhancement sample is the linear interpolation of two random samples and labels. The training process of Mixup data enhancement: first, two samples and corresponding labels are randomly selected from the training samples for weighted summation, then the loss of the prediction result and the label is calculated, and finally the parameters are updated in the reverse direction. The advantages of Mixup data enhancement: can eliminate the memory of the network to the wrong label; reduce the instability of adversarial training.

[0050] S2, the picture after data enhancement is input into the improved network model for training. The attention module can make up for the problem of local overstrength and global deficiency of CNN and obtain global context information. CBAM attention mechanism adopts channel and spatial double attention mechanism, so the detection effect is better than that of using channel or spatial attention mechanism alone in the network structure. The implementation method of channel attention mechanism is as follows Figure 3 , first, the input feature layer is subjected to maximum pooling (MaxPool) and average pooling (AvgPool), and then feature extraction is performed by using full connection; second, the two results output by full connection are added; finally, the added result is subjected to nonlinear activation (Sigmoid). The implementation method of spatial attention mechanism is as follows Figure 4, first, the maximum value and the average value of each feature point on the input feature layer are taken on the channel, then the two results are added, and finally the added results are convolved (Conv) and nonlinearly activated (Sigmoid).

[0051] S3, the dynamic matching of SimOTA and the output end of the network is used for regression, classification and target loss calculation of three parts. SimOTA is a simplified and improved version of OTA. OTA algorithm is a method to solve the optimal transmission problem. In OTA, the real box is regarded as the supply end and the predicted box is regarded as the demand end in the transmission problem, so if the cost from the supply end to the demand end can be calculated, the idea of OTA can be used to solve the positive and negative sample problems. SimOTA working part: first, the predicted points are screened according to whether they are in the real box, and the remaining boxes are called candidate boxes; then the cost of the candidate boxes is calculated to establish the optimal transmission model, and the IOU loss and classification loss of the predicted box and the real box in the candidate region are calculated, and the cost of the predicted box is obtained by adding the IOU loss and the classification loss; finally, for each real box, the top 10 IOU largest target boxes are selected for the next calculation. For the 10 predicted boxes in each real box, the IOU is added to get the number k of objects to be allocated. The first k predicted boxes are treated as positive samples. Loss calculation: the IOU loss between the real box and the predicted box is calculated to get the regression loss; the cross-entropy loss is calculated according to the prediction results of whether the positive and negative samples and the feature points contain objects to get the target loss; the cross-entropy loss is calculated according to the category prediction results of the real box and the feature points to get the classification loss. After 300 epochs of training and verification, the training loss is reduced to 0.04, and the verification loss is reduced to 0.06. It can be seen that the final loss basically converges. The final training loss (loss) and verification loss (val_loss) of the experiment are shown in Figure 5 , Figure 6 .

[0052] S4, set a suitable M_NMS threshold to obtain the best target box. The role of NMS is to select the box with the highest score in a certain region. The trained model is used to detect the abnormal situation of the on-site leakage. Since the threshold of M_NMS is not suitable, there will be redundant detection boxes, so the appropriate M_NMS threshold needs to be adjusted. After setting M_NMS to 0.5, the detection result is shown in Figure 7 , Figure 7 From left to right, the pictures to be detected, the pictures detected when the threshold is small, and the pictures detected with the appropriate threshold.

[0053] S5, set M_NMS to 0.5 and the confidence conf to 0.8, and the detection result is shown in Figure 8, the left side is the picture to be detected, and the right side is the detected picture, where (a) is the water leakage detection when the power plant is abnormal, (b) is the steam leakage detection, and (c) is the oil leakage detection. It can be seen that the improved network has good detection effect. In order to avoid false leakage, a video detection method is used, that is, a 3s video is intercepted at a fixed point, and after each frame is detected and the number of abnormal frames is counted, it is accurately judged whether the abnormal condition occurs.

[0054] S6, compare the algorithm with other algorithms. In order to compare the detection effect of each model, three evaluation indexes of mean average precision (mAP), detection speed (FPS) and model parameter quantity (params) are adopted, and the test set is tested, and the results are compared as shown in Table 1. YOLOv5 algorithm also has good effect in industrial practical application, YOLOv5 also has s, m, l, x and other models, and the following verification experiment results are all s models. YOLOv5 model has 7.2(M) parameters, YOLOX has 9.0(M) parameters, and YOLOA has 9.2(M) parameters. From the parameters, it can be seen that YOLOA only has 0.2(M) more than the original model, which shows that the network is not too complicated due to the increase of CBAM; when YOLOv5 algorithm is verified, the mAP value is 51.3%, the mAP of YOLOX is 57.4%, which is slightly improved. The mAP of YOLOA is increased by 2.7% compared with YOLOX, which shows that the accuracy of the model is improved in the detection of power plant abnormalities. The FPS in Table 1 is the speed of the network camera of the power plant transmitting to the webpage in real time, and the real-time detection of YOLOv5 is about 30.1 frames, and the real-time detection of YOLOX is about 47.3 frames. The real-time detection speed of YOLOA is obviously faster than YOLOv5, and only 1.4 frames slower than the original model YOLOX. Therefore, the YOLOA model can not only accurately detect the abnormal condition of the power plant in terms of accuracy, but also has fast detection speed, and can realize real-time detection on robots and cameras.

[0055] Table 1

[0056] Methods mAP@0.5:0.95(% 0.5:0.95 FPS params(M) YOLOv5 51.3 30.1 7.2 YOLOX 57.4 47.3 9.0 YOLOA 60.1 45.9 9.0

[0057] S7, the B transformation method fuses infrared images and visible light images, and the fused image is as shown in Figure 9 Figure 9 ​From left to right are visible light image, infrared image, and fused image. Visible light image can provide the texture and structural details of the scene through the high spatial resolution and clarity of the human visual system. Infrared image can distinguish targets and backgrounds according to the difference in thermal radiation between targets and backgrounds all day round and regardless of light conditions. Therefore, fusing infrared image and visible light image can combine the advantages of both images to improve the accuracy of target detection. The fused video is detected using the YOLOA network, and the detection status is displayed in real time on the front-end page through the flask framework. When the detection result is normal, the original image is displayed; when the detection result is abnormal, the abnormal image is displayed. The front-end can display in real time and timely and effectively remind the staff.

Claims

1.A method for identifying anomalies of key facilities in a power plant based on YOLOA, characterized in that: It comprises the following steps: S1, training the leakage data collected by the power plant using the YOLOA network, and then deploying the trained model to the algorithm server; S2, the power inspection robot reaches the specified position and rotates the holder, and the dual-view camera is aligned with the identification area to intercept 3s visible light and infrared video; S3, B transform is used to fuse the B component of the visible light image and the infrared image to obtain the fused video; S4, the fused video is input into the network for prediction and displayed on the front-end page, the number t of frames with a confidence level greater than 0.8 is counted after video detection, and finally the size of t and 0.6f is judged, if t>0.6f, the power plant confirms that an abnormal condition has occurred, wherein f is the total number of frames of 3s video, and c is the confidence level; In the step S1, in order to make the network converge faster, the specific process is as follows: S11, first use the YOLOA network to obtain pre-training weights on a large data set and then use it on the data set; S12, the parameter setting for training the neural network is width=640, height=640, learning-rate=1e-3, epochs=300, and the loss is reduced to 0.04 after 300 rounds of training; S13, the loss calculation formula is: Loss=a*Lr+Lo+Lc+L1, wherein Lr is the regression loss, Lo is the target loss, Lc is the classification loss, and L1 is the L1 regular term loss; S15, finally, the trained model is deployed to the algorithm server to detect the leakage video of the power plant; in the step S2, the dual-view camera intercepts the visible light and infrared video after aligning with the identification area, and the visible light and infrared video must be synchronized at this time for easy fusion; S14, the post-processing of the network is to solve the problem of missing detection of targets caused by directly deleting highly overlapped targets in dense scenes, and the M_NMS calculation formula is: wherein M represents the detection frame with the highest current confidence, Ci represents a certain candidate detection frame, and t represents a threshold. In the step S3, the B transform method is used to fuse the B component of the visible light and the gray scale of the infrared image, and the B', C and S components are inversely transformed into color images, and the formula is G′=0.3*R+0.6*G+0.1*B; B=(R+G+B+G') / 4, wherein B is the brightness, C is the chroma, S is the saturation, and G' is the gray scale; In the step S4, the confidence level is set to 0.8, and then each frame of the fused 3s video is detected, when the confidence level is less than or equal to 0.8, the original image is pushed to the front-end display through the flask framework, when it is greater than 0.8, the abnormal image is pushed to the front-end display, the number of frames with a confidence level greater than 0.8 in the video is counted after detecting the 3s video, and if it is greater than 0.6 times the total number of frames, the power plant confirms that an abnormal condition has occurred, and an alarm needs to be sent at this time. ​ 2. The power plant key facility anomaly identification method based on YOLOA according to claim 1, characterized in that: Because the input network predicted image is a color image, after the B component of the visible light and the gray image of the infrared image are fused, the B', C, and S components need to be inversely transformed into a color image, wherein the formulas of the chroma C and the saturation S are

Images