On-off state identification method based on improved YOLOv7 algorithm in industrial environment
By improving the YOLOv7 algorithm, combining attention mechanism and Mobilenetv3 network, the problem of poor switching equipment identification in industrial environments is solved, accurate identification and automated supervision in complex contexts are achieved, and the supervision efficiency and stability of power plant equipment are improved.
Patent Information
- Application Number
- CN202310745098.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-21
- Publication Date
- 2024-12-24
AI Technical Summary
In industrial environments, traditional manual inspection methods have problems such as unstable information recording, easy loss and inaccurate operation. Especially in complex power plant environments, the identification of switch equipment is poor and it is difficult to achieve automated supervision.
The improved YOLOv7 algorithm is adopted to acquire switch images through wearable intelligent terminals, perform perspective transformation correction and feature learning, and introduce attention mechanisms ECA and Mobilenetv3 backbone networks to improve the accuracy and efficiency of small object detection and realize automatic identification of switch status.
It realizes accurate identification of switch status in complex contexts, reduces the error rate and operation intensity of manual inspections, and improves the supervision efficiency and stability of power plant equipment.
Smart Images
Figure CN119181036A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of digital image processing technology, and in particular to a switch state recognition method based on an improved YOLOv7 algorithm in an industrial environment. Background Art
[0002] In the country's long-term development plan, artificial intelligence technology has become one of the priority development goals. With the development of artificial intelligence technology, artificial intelligence technology has begun to be introduced in more and more industrial production scenarios. In the power production system of the pumped-storage power station, it is necessary to set the optimal working state of the pumped-storage power station periodically according to the load demand of the power grid to obtain the maximum benefit of the pumped-storage power station in the power system. The setting of the optimal working state depends entirely on manual operation. The entire business process needs to go through the steps of reading the operation workflow, repeating the operation workflow, manual operation, and manual inspection. However, there are many types of switches and instruments at the industrial production site, and they are complex. Manual supervision leads to increased work intensity, and the correctness of their operation cannot be objectively and effectively supervised.
[0003] The traditional way to inspect various equipment in substations is to use manual inspection methods, arrange specific personnel to inspect the switchgear in the designated area every day, and summarize the inspection results into a record book for later review. Considering manpower and material resources, the manual inspection method always has many disadvantages: first, the inspection information that can be recorded in the paper record book is limited, and it is easy to lose or even lose; second, the inspection record information is greatly affected by the physical and mental conditions of the inspectors, and is unstable and prone to omissions. Therefore, the application of intelligent inspection systems in substations is of great significance to ensure the stable and efficient operation of substations.
[0004] With the development of image processing technology, deep learning has made good progress in the field of target detection. At present, the classic target detection algorithms are mainly divided into two categories: single-stage and dual-stage. The single-stage ones include YOLO, SDD, Retina-Net, etc., and the dual-stage ones include R-CNN, Fast R-CNN, Mask R-CNN, etc.; Among them, the YOLO algorithm runs the fastest and has higher accuracy. It is more suitable for the recognition and detection of various switches and indicator lights in the switch cabinet in the power station. However, due to the complex background of different power station images, the existence of occlusion, and small switches, the recognition effect is not very good, and further research is still needed. Summary of the invention
[0005] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art and provide a switch state recognition method based on the improved YOLOv7 algorithm in an industrial environment. The present invention collects switch images and uses the improved YOLOv7 algorithm to prepare for recognition and evaluation of the switch state under the condition of complex background and small target. It has good detection and recognition effect and provides technical support for the intelligent construction of distribution room.
[0006] In order to achieve the above object, the present invention adopts the following technical solutions:
[0007] A switch state recognition method based on an improved YOLOv7 algorithm in an industrial environment comprises the following steps:
[0008] The images of industrial equipment switches are collected through wearable smart terminals to obtain the original image dataset;
[0009] Perform perspective transformation correction on the images in the original image dataset and divide them into training set and test set;
[0010] Improve the original YOLOv7 network model, including: first improve its network model, introduce the attention mechanism ECA, and replace the Backbone backbone network with the Mobilenetv3 backbone network;
[0011] Iteratively train the improved YOLOv7 network model on the prepared training set until convergence, and retain the best network weights;
[0012] The improved YOLOv7 network model is tested using the test set, and the corresponding switch status information is output.
[0013] The wearable intelligent terminal is a 5G-based intelligent safety helmet, specifically:
[0014] It can automatically obtain switch information images in complex industrial environments, and multiple smart safety helmets can work simultaneously to monitor multiple switch states, and can achieve 1080p video image transmission.
[0015] The perspective transformation correction of the switch image dataset adopts the correction of auxiliary labels, specifically:
[0016] Four easily located auxiliary labels are attached to the four corners of the rectangular switch panel in the image. The center points of the auxiliary labels are used as the four vertices of the original rectangular image before perspective transformation. After perspective transformation, a positive-view image containing only the rectangular switch panel is obtained.
[0017] The improved YOLOv7 network model includes input, backbone and head;
[0018] Add adaptive attention module ECA based on FPN structure;
[0019] Replace the Backbone backbone network with the Mobilenetv3 backbone network;
[0020] The ECA attention is used to reduce feature channels and reduce the number of parameters;
[0021] The improved backbone network is used to reduce the amount of network calculation;
[0022] As a preferred technical solution, the iterative training on the training set to improve the yolov7 network model is specifically as follows:
[0023] The training set is input into the Backbone network through the input layer, and then the feature graph of the training set is obtained through the Neck network;
[0024] Make predictions in the prediction layer based on the feature graph of the training set, calculate the loss, and update the model parameters;
[0025] Retrain until the loss function converges or the maximum number of iterations is reached, and save the best network weights.
[0026] The adaptive attention module ECA is specifically:
[0027] First, input the feature map, whose dimension is H*W*C;
[0028] Compress the input feature map in space. Use global average pooling (GAP) in the spatial dimension to obtain a 1*1*C feature map.
[0029] For the compressed feature map, channel feature learning is performed; the importance of different channels is learned through 1*1 convolution, and the output dimension is still 1*1*C;
[0030] Finally, the channel attention is combined to Figure 1 *1*C, the original input feature map H*W*C, perform channel-by-channel multiplication, and finally output a feature map with channel attention.
[0031] The backbone network of Mobilenetv3 is specifically:
[0032] The network architecture is based on MnasNet implemented by NAS, and the parameters are obtained by NAS search;
[0033] It also introduces the depth-wise separable convolution of MobileNetV1, the inverted residual structure with linear bottleneck of MobileNetV2, and a new activation function h-swish(x).
[0034] The activation function is defined as:
[0035]
[0036] As a preferred technical solution, the method further includes:
[0037] After obtaining the switch state recognition results, the network model performance is evaluated using the accuracy, recall rate, and average precision mean indicators;
[0038] The accuracy calculation formula is:
[0039] Precision=TP / (TP+FP)=TP / all detections;
[0040] The recall rate calculation formula is:
[0040] Recall=TP / (TP+FN)=TP / all ground trusts;
[0041] The average accuracy mean calculation formula is:
[0042]
[0043] Among them, TP is the area predicted correctly by the algorithm detection box, FP is the area predicted incorrectly by the algorithm detection box, and FN is the area where the actual annotation box is correct but not predicted by the algorithm detection box; all detections is the predicted area of the algorithm detection box, all ground trusts is the actual area of the actual annotation box, r represents the recall rate, ρ(r) is the precision value of the recall rate r, and ρ interp (r n+1 ) is the maximum precision value in the corresponding precision value ρ(r) when the recall rate is greater than or equal to r.
[0044] Compared with the prior art, the present invention has the following advantages and beneficial effects:
[0045] 1. The present invention uses positioning auxiliary tags to perform perspective transformation correction, thereby facilitating the subsequent positioning and segmentation tasks of the switch unit.
[0046] 2. Improve the original YOLOv7 network model, including: first improve its network model and introduce the attention mechanism ECA to reduce feature channels and reduce the number of parameters; and replace the Backbone backbone network with the Mobilenetv3 backbone network to reduce the amount of network calculation and improve the detection effect of small targets. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] Figure 1It is a flow chart of a switch state recognition method based on an improved YOLOv7 algorithm in an industrial environment in an embodiment of the present invention;
[0048] Figure 2 is a schematic diagram of an auxiliary label used in an embodiment of the present invention;
[0049] Figure 3 This is a schematic diagram of the structure of the attention mechanism ECA module in an embodiment of the present invention;
[0050] Figure 4 This is a schematic diagram of the backbone network structure of Mobilenetv3 in an embodiment of the present invention;
[0051] Figure 5 Schematic diagram of the activation function h-swish(x) in an embodiment of the present invention;
[0052] Figure 6 Schematic diagram of the recognition result in the embodiment of the present invention. DETAILED DESCRIPTION
[0053] The present invention will be further described in detail below in conjunction with the accompanying drawings.
[0054] S1: Collect images of industrial equipment switches through wearable smart terminals to obtain original image datasets;
[0055] The wearable smart terminal is a 5G-based smart safety helmet, specifically:
[0056] It can automatically acquire switch information images in complex industrial environments, and multiple smart safety helmets can work simultaneously to monitor multiple switch states and achieve 1080p video image transmission.
[0057] The present invention obtains more than 1258 switch cabinet images from the camera under different backgrounds and different cabinet surface targets, thereby maximizing the richness.
[0058] S2: Perform perspective transformation correction on the images in the original image dataset and divide them into training set and test set; The perspective transformation correction of the switch image dataset adopts the correction of auxiliary labels, specifically:
[0059] Four easily located auxiliary labels are attached to the four corners of the rectangular switch panel in the image. The center points of the auxiliary labels are used as the four vertices of the original rectangular image before perspective transformation. After perspective transformation, a positive-view image containing only the rectangular switch panel is obtained.
[0060] The auxiliary label is a circular label with a diameter of 3 cm. Figure 2 As shown, specifically:
[0061] It has a strong Harr-like feature, with only black and white brightness, and has the strongest contrast;
[0062] After the image is filtered, the four largest local peaks are found and the center positions of the four auxiliary labels can be obtained.
[0063] S3: Improve the original YOLOv7 network model, including: first improve its network model, introduce the attention mechanism ECA, and replace the Backbone backbone network with the Mobilenetv3 backbone network;
[0064] The improved YOLOv7 network model includes input, backbone and head;
[0065] Add adaptive attention module ECA based on FPN structure;
[0066] Replace the Backbone backbone network with the Mobilenetv3 backbone network;
[0067] The ECA attention is used to reduce feature channels and reduce the number of parameters;
[0068] The improved backbone network is used to reduce the amount of network calculation;
[0069] The adaptive attention module ECA has the following structure: Figure 3 As shown, specifically:
[0070] First, input the feature map, whose dimension is H*W*C;
[0071] Compress the input feature map in space. Use global average pooling (GAP) in the spatial dimension to obtain a 1*1*C feature map.
[0072] For the compressed feature map, channel feature learning is performed; the importance of different channels is learned through 1*1 convolution, and the output dimension is still 1*1*C;
[0073] Finally, the channel attention is combined to Figure 1 *1*C, the original input feature map H*W*C, perform channel-by-channel multiplication, and finally output a feature map with channel attention.
[0074] The backbone network of Mobilenetv3 is as follows: Figure 4 As shown, specifically:
[0075] The network architecture is based on MnasNet implemented by NAS, and the parameters are obtained by NAS search;
[0076] And introduce the depth of MobileNetV1 separable convolution, MobileNetV2 with linear bottleneck inverted residual structure, introduce a new activation function h-swish (x), the principle is as follows Figure 5 shown.
[0077] The activation function is defined as:
[0078] S4: Iteratively train the improved YOLOv7 network model on the prepared training set until convergence, and retain the best network weights;
[0079] Iterative training on the training set improves the yolov7 network model, specifically:
[0080] The training set is input into the Backbone network through the input layer, and then the feature graph of the training set is obtained through the Neck network;
[0081] Make predictions in the prediction layer based on the feature graph of the training set, calculate the loss, and update the model parameters;
[0082] Retrain until the loss function converges or the maximum number of iterations is reached, and save the best network weights.
[0083] S5: Use the test set to test the improved YOLOv7 network model and output the corresponding switch status information. The schematic diagram of the recognition result is as follows: Figure 6 shown.
[0084] The switch state recognition method based on the improved YOLOv7 algorithm according to claim 1 is characterized in that the method further comprises: after obtaining the switch state recognition result, using the accuracy, recall rate, and average precision mean index to evaluate the network model performance;
[0085] The accuracy calculation formula is: Precision = TP / (TP+FP) = TP / all detections;
[0086] The recall rate calculation formula is: Recall = TP / (TP+FN) = TP / all ground trusts;
[0087] The average accuracy mean calculation formula is:
[0088] Among them, TP is the area predicted correctly by the algorithm detection box, FP is the area predicted incorrectly by the algorithm detection box, and FN is the area where the actual annotation box is correct but not predicted by the algorithm detection box; all detections is the predicted area of the algorithm detection box, all ground trusts is the actual area of the actual annotation box, r represents the recall rate, ρ(r) is the precision value of the recall rate r, and ρ interp (r n+1 ) is the maximum precision value in the corresponding precision value ρ(r) when the recall rate is greater than or equal to r.
Claims
1. A switch state recognition method based on an improved YOLOv7 algorithm in an industrial environment, characterized in that: The steps include: S1: Collect images of industrial equipment switches through wearable smart terminals to obtain original image datasets; S2: Perform perspective transformation correction on the images in the original image dataset and divide them into training set and test set; S3: Improve the original YOLOv7 network model, including: first improve its network model, introduce the attention mechanism ECA, and replace the Backbone backbone network with the Mobilenetv3 backbone network; S4: Iteratively train the improved YOLOv7 network model on the prepared training set until convergence, and retain the best network weights; S5: Use the test set to test the improved YOLOv7 network model and output corresponding switch status information.
2. The switch state recognition method based on the improved YOLOv7 algorithm according to claim 1 is characterized in that The wearable smart terminal of S1 is a 5G-based smart safety helmet, which can automatically obtain switch information images in complex industrial environments, and multiple smart safety helmets can work at the same time to monitor multiple switch states, and can realize 1080p video image transmission.
3. The switch state recognition method based on the improved YOLOv7 algorithm according to claim 1 is characterized in that The S2 uses auxiliary label correction to correct the perspective transformation of the switch image data set, specifically: four easily located auxiliary labels are attached to the four corners of the rectangular switch panel in the image, and the center points of the auxiliary labels are used as the four vertices of the original rectangular image before the perspective transformation. After the perspective transformation, a positive-view image containing only the rectangular switch panel is obtained.
4. The switch state recognition method based on the improved YOLOv7 algorithm according to claim 3 is characterized in that The auxiliary tag is a circular tag with a diameter of 3 cm. Specifically, it has a strong Harr-like feature, only black and white brightness, and the strongest contrast. After image filtering, the four largest local peaks are found to obtain the center positions of the four auxiliary tags.
5. The switch state recognition method based on the improved YOLOv7 algorithm according to claim 1 is characterized in that The S3 improved YOLOv7 network model includes input, backbone and head; an adaptive attention module ECA is added on the basis of the FPN structure; the Backbone backbone network is replaced with the Mobilenetv3 backbone network; the ECA attention is used to reduce feature channels and reduce the number of parameters; the improved backbone network is used to reduce the amount of network calculations.
6. The switch state recognition method based on the improved YOLOv7 algorithm according to claim 5 is characterized in that: Iterative training is performed on the training set to improve the yolov7 network model. Specifically, the training set is input into the Backbone network through the input layer, and then the feature map of the training set is obtained through the Neck network; prediction is performed in the prediction layer based on the feature map of the training set, the loss is calculated, and the model parameters are updated; retraining is performed until the loss function converges or the maximum number of iterations is reached, and the optimal network weights are saved.
7. The switch state recognition method based on the improved YOLOv7 algorithm according to claim 5, characterized in that: The adaptive attention module ECA is specifically: S301: First, input the feature map, whose dimension is H*W*C; S302: Perform spatial feature compression on the input feature map; use global average pooling GAP in the spatial dimension to obtain a 1*1*C feature map; S303: Perform channel feature learning on the compressed feature map; Through 1*1 convolution, we can learn the importance of different channels. At this time, the output dimension is still 1*1*C. S304: Finally, the channel attention is combined, and the feature map of the channel attention 1*1*C and the original input feature map H*W*C are multiplied channel by channel, and finally the feature map with channel attention is output.
8. The switch state recognition method based on the improved YOLOv7 algorithm according to claim 5, characterized in that: The backbone network of the Mobilenetv3 is specifically as follows: the network architecture is based on MnasNet implemented by NAS, and parameters are obtained by NAS search; and the depth separable convolution of MobileNetV1 and the inverted residual structure with linear bottleneck of MobileNetV2 are introduced, and a new activation function h-swish(x) is introduced.
9. The switch state recognition method based on the improved YOLOv7 algorithm according to claim 1, characterized in that: The method further comprises: after obtaining the switch state recognition result, evaluating the network model performance using the accuracy, recall rate, and average accuracy mean index; The accuracy calculation formula is: Precision = TP / (TP+FP) = TP / all detections; The recall rate calculation formula is: Recall = TP / (TP+FN) = TP / all ground trusts; The average accuracy mean calculation formula is: Among them, TP is the area predicted correctly by the algorithm detection box, FP is the area predicted incorrectly by the algorithm detection box, and FN is the area where the actual annotation box is correct but not predicted by the algorithm detection box; all detections is the predicted area of the algorithm detection box, allground trusts is the actual area of the actual annotation box, r represents the recall rate, ρ(r) is the precision value of the recall rate r, and ρ interp (r n+1 ) is the maximum precision value in the corresponding precision value ρ(r) when the recall rate is greater than or equal to r.