Power equipment heating defect detection method based on infrared image

Abstract

The invention provides a power equipment heating defect detection method based on an infrared image. The method comprises the following steps: acquiring an inspection aerial image of a power transmission line; constructing a heating defect image data set of the power equipment; building a deep learning network model based on a PyTorch framework; carrying out feature extraction, feature fusion and target prediction on a preset deep learning network model; optimizing a preset deep learning network model; the unmanned aerial vehicle carries an optimized and trained deep learning network model, and image acquisition and power equipment heating defect online detection are carried out according to an inspection task; or the video image of the unmanned aerial vehicle polling the power transmission line is input to the deep learning network model, and the deep learning network model carries out offline detection on the heating power equipment in the infrared data and outputs a detection result. According to the method, the infrared image is taken as a research object, the deep learning network model is taken as a tool, the trained deep learning network model is deployed on the unmanned aerial vehicle, and intelligent detection of the heating defect of the power equipment in the power transmission line is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for detecting heating defects of electric power equipment based on infrared images. Background Art

[0002] The power grid is an important infrastructure related to the national economy, people's livelihood and national energy security. The transmission lines are the link of power transmission. Their safe and stable operation is a necessary guarantee for social production and people's lives. In recent years, with the continuous increase in electricity demand, the distribution of transmission lines has become more and more widespread, and their total length has also increased rapidly. Since the transmission lines are erected in various natural environments, they are exposed to wind, sun and rain all year round, and are sometimes affected by severe weather such as ice and snow, which inevitably causes the loss or damage of power equipment. Therefore, regular inspections have become an important task to ensure the continuous supply of electricity and the safe operation of transmission lines.

[0003] Drone inspection has become an important inspection method for major power grids. "Drone inspection as the main method, supplemented by manual inspection" has become the main operation and maintenance mode for power transmission line inspection in my country. The mature application of drone power inspection technology in power transmission lines has produced a large number of aerial inspection photos of power transmission lines. The main method for finding defects in massive images is manual reading and interpretation, which is time-consuming, labor-intensive and easy to miss. Using computers to automatically analyze inspection image defects is the future development direction.

[0004] As an indispensable part of overhead transmission lines, power fittings and other equipment are responsible for connecting power components such as ground wires and towers, transmission wires and insulators, and towers and insulators in the power system, and play an important role in the safe and stable operation of the system. However, since most power equipment not only has to work outdoors in harsh environments, but also has to withstand the external mechanical load tension and the power load inside the power system for a long time, power equipment is prone to heating defects, affecting the stable operation of the power grid. However, the heating defects of power equipment are mainly detected manually, which is not only time-consuming and labor-intensive, but also has low efficiency in the face of massive drone inspection images, and is also prone to false detection or missed detection. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In response to the problem of detecting heating defects of power equipment in the above-mentioned drone aerial images, the present invention provides a method for detecting heating defects of power equipment based on infrared images. The infrared image is taken as the research object and the deep learning network model is used as the tool. The trained deep learning network model is deployed on the drone to realize intelligent detection of heating defects of power equipment in transmission lines.

[0007] (II) Technical solution

[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0009] A method for detecting heating defects of electric power equipment based on infrared images comprises the following steps:

[0010] Step S1, the drone is equipped with an infrared camera to collect aerial images of power transmission line inspection;

[0011] Step S2, cropping the infrared image of the heating of the electric equipment to construct a data set of heating defect images of the electric equipment;

[0012] Step S3: Building a deep learning network model based on the PyTorch framework;

[0013] Step S4, inputting the power equipment heating defect data set of step S2 into the deep learning network model preset in step S3 for feature extraction, feature fusion and target prediction;

[0014] Step S5: Optimizing the preset deep learning network model;

[0015] Step S6: The UAV is equipped with an optimized and trained deep learning network model to perform image acquisition and online detection of heating defects in power equipment according to the inspection task; or the video image of the UAV inspecting the transmission line is input into the deep learning network model, and the deep learning network model performs offline detection of the heating power equipment in the infrared data and outputs the detection results.

[0016] Furthermore, the resolution of the heating defect image of the electric equipment in step S2 is 640×640, and the labeling tool of the heating defect image data set of the electric equipment is labelimg.

[0017] Furthermore, 70% of the images in the data set in step S2 are used for training, and the remaining 30% of the images are used for testing.

[0018] Furthermore, the deep learning network model in step S3 is a cascade detection model.

[0019] Furthermore, the front stage of the cascade detection model is YOLOv7, and the back stage of the cascade detection model is YOLOv7-tiny.

[0020] Furthermore, the feature extraction in step S4 is based on the backbone network Backbone of YOLOv7 to perform feature extraction at different scales, the feature fusion in step S4 is based on the neck of YOLOv7 to perform feature fusion at different scales, and the target prediction in step S4 is based on the head of YOLOv7 to perform target detection at three scales.

[0021] Furthermore, the step S5 optimizes the preset deep learning network model, including:

[0022] Step S5.1, introduce the SimAM attention mechanism module into YOLOv7 to enhance the model's adaptability to multi-scale and small target features;

[0023] Step S5.2: Introduce a cross-level weighted feature pyramid network into the neck network of YOLOv7.

[0024] Step S5.3: In the neck network of YOLOv7, the spatial pyramid dilated convolution SPD is used to improve the maximum pooling structure MP downsampling operation.

[0025] Furthermore, in step S5.1, three SimAM attention mechanism modules are introduced between the effective feature layers C3, C4, and C5 of the YOLOv7 backbone network and the neck input of YOLOv7.

[0026] Furthermore, in step S5.1, three SimAM attention mechanism modules are introduced between the neck network output of YOLOv7 and the head input of YOLOv7.

[0027] Furthermore, in step S5.2, the cross-level weighted feature pyramid network includes four cross-level weighted connection modules, namely CLW-Add1, CLW-Add2, CLW-Add3 and CLW-Add4.

[0028] (III) Beneficial effects

[0029] The beneficial effects of the present invention are as follows: a method for detecting heating defects of electric equipment based on infrared images, which takes infrared images as research objects and deep learning network models as tools, regards heating defects of electric equipment as a secondary target detection problem, uses the front-end improved YOLOv7 for classifying whether electric equipment has heating defects or not, uses the back-end YOLOv7-tiny for locating heating defects of electric equipment, and deploys the trained deep learning network model on a drone to realize intelligent detection of heating defects of electric equipment in transmission lines; the SimAM attention mechanism module is introduced into YOLOv7 to enhance the adaptability of the model to multi-scale and small target features, thereby improving the accuracy of classifying whether electric equipment has heating; the cross-level weighted feature pyramid network CLW-FPN is applied to the neck part of YOLOv7, further adjusts the size and position of the receptive field during the convolution process, and improves the maximum pooling structure MP downsampling by using the spatial pyramid dilated convolution to further improve the accuracy of detecting heating defects of electric equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0031] Figure 1 This is a flow chart of the method for detecting heating defects in electric power equipment according to the present invention;

[0032] Figure 2 Improved YOLOv7 network structure diagram for the present invention;

[0033] Figure 3 This is the structural diagram of the SimAM attention mechanism module of the present invention;

[0034] Figure 4 This is a cross-level weighted feature pyramid network structure diagram of the present invention;

[0035] Figure 5 This is the SPD Conv structure diagram of the present invention;

[0036] Figure 6 This is the YOLOv7-tiny network structure diagram of the present invention. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] Combination Figure 1 , a method for detecting heating defects of power equipment based on infrared images, comprising the following steps:

[0039] Step S1: The drone is equipped with an infrared camera to collect aerial images of power transmission line inspections.

[0040] Step S2: cropping the infrared image of heating of electric equipment and constructing a dataset of heating defect images of electric equipment.

[0041] Furthermore, the resolution of the heating defect image of the electric equipment in step S2 is 640×640, and the labeling tool of the heating defect image data set of the electric equipment is labelimg.

[0042] Furthermore, 70% of the images in the data set in step S2 are used for training, and the remaining 30% of the images are used for testing.

[0043] Step S3: Build a deep learning network model based on the PyTorch framework.

[0044] Furthermore, since the heating defects of power equipment in the infrared images taken by the drone during the inspection of the power line are relatively small, they can be regarded as small target detection. Therefore, the deep learning network model in step S3 is a cascade detection model.

[0045] Furthermore, the cascade detection model adopts a cascade YOLO model, the front stage is YOLOv7, and the back stage of the cascade detection model is YOLOv7-tiny.

[0046] Step S4: input the heating defect data set of the power equipment in step S2 into the deep learning network model preset in step S3 for feature extraction, feature fusion and target prediction.

[0047] Furthermore, the feature extraction in step S4 is based on the backbone network Backbone of YOLOv7 to perform feature extraction at different scales, the feature fusion in step S4 is based on the neck of YOLOv7 to perform feature fusion at different scales, and the target prediction in step S4 is based on the head of YOLOv7 to perform target detection at three scales.

[0048] Step S5: Optimize the preset deep learning network model.

[0049] Furthermore, the step S5 optimizes the preset deep learning network model, including:

[0050] Step S5.1: Introduce the SimAM attention mechanism module into YOLOv7 to enhance the model's adaptability to multi-scale and small target features.

[0051] The attention mechanism focuses on the important positions of the target to obtain its key information. It plays a huge role in the field of deep learning. In the target detection task, adding an attention module can enhance the representation ability of the target network model to a certain extent, thereby effectively reducing the interference of invalid targets and improving the detection effect of key targets. In machine learning, the attention mechanism can be divided into channel attention mechanism, spatial attention mechanism and self-attention mechanism. Figure 3,Structure diagram of SimAM attention mechanism module,In order to further improve the performance of the target detection algorithm, the present invention introduces the SimAM attention module into the neck network of the YOLOv7 model (between the effective feature layers C3, C4, C5 of the backbone network and the neck input of YOLOv7) to optimize the features extracted from the backbone network while taking into account the width, depth and detection speed of the network, thereby using fewer network parameters to improve the classification accuracy of heating defective power equipment.

[0052] Compared with other attention mechanisms, SimAM's operation is simpler and can effectively avoid the problem of increasing model parameters due to structural adjustment. The energy function of each neuron is shown in formula (1).

[0053]

[0054] t, i, and x i They represent the target neuron on a single channel in the input X, the index of the spatial dimension, and other neurons. M is the number of all neurons on a certain channel, and y is the label, indicating whether it is an important neuron. The weight w t and deviation b t As shown in formula (2-3).

[0055]

[0056] μ t , δ t They respectively represent the mean and variance of the channel except the target neuron, as shown in formula (4-5).

[0057]

[0058] Through the calculation of the energy function, it can be found that when the energy of a neuron is lower, the greater the difference between it and other neurons, the higher its importance. Therefore, the SimAM module can accurately capture the key information in the image features without adding additional parameters, and has high practical value. At the same time, for the specific task of detecting heating defects in power equipment, due to the different background light brightness, color type, size and shape of power equipment, the complex and changeable environment puts high demands on the positioning and detection capabilities of the model. In order to improve the classification ability of the network, three SimAM attention mechanism modules are introduced between the output of the neck network of YOLOv7 and the input of the head of YOLOv7 to enhance the small target features of heating defects in power equipment, reduce background interference, and improve the detection accuracy to a certain extent while maintaining efficient detection.

[0059] Step S5.2, introduce a cross-level weighted feature pyramid network CLW-FPN (cross-level weighted feature pyramid network) into the neck network of YOLOv7. In the deep learning calculation process, as the convolution layer deepens, a certain degree of feature loss will occur in most cases. To solve this problem, multi-scale feature fusion is used in the neck network. In order to improve the neck network, the present invention designs a cross-level weighted feature pyramid network, such as Figure 4 As shown in the figure, the difference from the PAN network is that it not only has bidirectional feature fusion, but also adopts a cross-scale connection mode, that is, it adds information fusion from P51 to P3 and from P31 to P52, and connects P4 with P42.

[0060] CLW-FPN contains 4 cross-level weighted connection modules, namely CLW-Add1, CLW-Add2, CLW-Add3 and CLW-Add4. After upsampling, P51 is fused with P4 through the CLW-Add1 convolution module; after upsampling, P51 is fused with P3 through the CLW-Add2 convolution module after twice upsampling; after downsampling, P4 is fused with P41 through the CLW-Add3 convolution module; after downsampling, P42 is fused with P31 through the CLW-Add4 convolution module after twice downsampling. The CLW-Add module adopts the fusion mode of BiFPN, multiplies the input feature map by the corresponding weights for feature fusion, and then outputs the feature map through the depth-separable convolution (DSConv).

[0061] Step S5.3: In the neck network of YOLOv7, the spatial pyramid dilated convolution SPD is used to improve the maximum pooling structure MP downsampling operation.

[0062] In the case of complex background and blurred image, it is difficult to distinguish redundant pixels and small targets in the feature map of the neck network. Multiple applications of SConv (3×3 filter, step size = 2) to filter redundant information will also filter out a large number of key features of small targets. In order to reduce the feature loss of small targets, the SConv in the downsampling MP structure of the neck network is replaced with SPD convolution. The splitting principle of the SPD convolution layer is applied to map the global spatial information of small targets to the channel dimension, so as to retain more feature information of small targets in complex scenes. SPD Conv can reduce information loss. It includes a SPD layer from space to depth and a strideless convolution layer. Combined with Figure 5,First, SPD Conv slices the input feature map to obtain 4 downsampled sub-maps, which contain the global spatial information of the original image. Then these sub-maps are concatenated along the channel dimension. Finally, the channel dimension is adjusted through the non-strided convolution layer, so that the global spatial feature information is retained in the channel dimension.

[0063] The YOLOv7-tiny algorithm is simplified from YOLOv7, retaining the cascade-based model scaling strategy and improving the efficient long-range aggregation network (ELAN+), which ensures detection accuracy based on fewer parameters and faster detection speed, and is suitable for real-time thermal defect detection of power equipment. Figure 6 As shown in the figure, the YOLOv7-tiny model is divided into four parts: input, backbone network, neck and head. Specifically, in the input part, mosaic data enhancement and adaptive anchor box calculation are used to preprocess the input image. The backbone network consists of several CBS modules (convolutional layer, batch normalization layer and SiLU function), ELAN layer (several CBS modules) and MP layer (CBS module and maximum pooling). In the neck part, SPPCSPC and PAN structures are used to integrate the features of each layer to detect targets of different scales, among which CBS module, MP layer, SPPCSPC module and ELAN+ are used for structural connection. In the head part, 3 REP layers and CBM modules are used for prediction of 3 different scales. The REP module has different network structures during training and inference, while the CBM module consists of convolutional layer, batch normalization layer and Sigmoid function.

[0064] Step S6: The UAV is equipped with an optimized and trained deep learning network model to perform image acquisition and online detection of heating defects in power equipment according to the inspection task; or the video image of the UAV inspecting the transmission line is input into the deep learning network model, and the deep learning network model performs offline detection of the heating power equipment in the infrared data and outputs the detection results.

[0065] In summary, the embodiments of the present invention, the method for detecting heating defects of power equipment based on infrared images, takes infrared images as the research object, and uses deep learning network models as tools. The heating defects of power equipment are regarded as a secondary target detection problem. The front-end improved YOLOv7 is used for classifying whether the power equipment has heating defects, and the back-end YOLOv7-tiny is used for locating heating defects of power equipment. The trained deep learning network model is deployed on a drone to realize intelligent detection of heating defects of power equipment in transmission lines. The present invention introduces the SimAM attention mechanism module into YOLOv7, which enhances the adaptability of the model to multi-scale and small target features, thereby improving the accuracy of classifying whether the power equipment is heated; the cross-level weighted feature pyramid network CLW-FPN is applied to the neck part of YOLOv7, and the size and position of the receptive field are further adjusted during the convolution process. The spatial pyramid dilated convolution improves the maximum pooling structure MP downsampling, which further improves the accuracy of detecting heating defects of power equipment.

[0066] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting heating defects of power equipment based on infrared images, characterized in that: The following steps are involved: Step S1, the drone is equipped with an infrared camera to collect aerial images of power transmission line inspection; Step S2, cropping the infrared image of the heating of the electric equipment to construct a data set of heating defect images of the electric equipment; Step S3: Building a deep learning network model based on the PyTorch framework; Step S4, inputting the power equipment heating defect data set of step S2 into the deep learning network model preset in step S3 for feature extraction, feature fusion and target prediction; Step S5: Optimizing the preset deep learning network model; Step S6: The UAV is equipped with an optimized and trained deep learning network model to perform image acquisition and online detection of heating defects in power equipment according to the inspection task; or the video image of the UAV inspecting the transmission line is input into the deep learning network model, and the deep learning network model performs offline detection of the heating power equipment in the infrared data and outputs the detection results.

2. The method for detecting heating defects of electric power equipment based on infrared images according to claim 1, characterized in that: The resolution of the heating defect image of the electric equipment in step S2 is 640×640, and the labeling tool of the heating defect image dataset of the electric equipment is labelimg.

3. A method for detecting heating defects of electric power equipment based on infrared images as claimed in claim 2, characterized in that: In step S2, 70% of the images in the data set are used for training, and the remaining 30% of the images are used for testing.

4. The method for detecting heating defects of electric power equipment based on infrared images according to claim 1, characterized in that: The deep learning network model in step S3 is a cascade detection model.

5. The method for detecting heating defects of electric power equipment based on infrared images as claimed in claim 4, characterized in that: The front stage of the cascade detection model is YOLOv7, and the back stage of the cascade detection model is YOLOv7-tiny.

6. A method for detecting heating defects of electric power equipment based on infrared images as claimed in claim 5, characterized in that: The feature extraction in step S4 is based on the backbone network Backbone of YOLOv7 to perform feature extraction at different scales, the feature fusion in step S4 is based on the neck of YOLOv7 to perform feature fusion at different scales, and the target prediction in step S4 is based on the head of YOLOv7 to perform target detection at three scales.

7. A method for detecting heating defects of electric power equipment based on infrared images as claimed in claim 6, characterized in that: The step S5 optimizes the preset deep learning network model, including: Step S5.1, introduce the SimAM attention mechanism module into YOLOv7 to enhance the model's adaptability to multi-scale and small target features; Step S5.2: Introduce a cross-level weighted feature pyramid network into the neck network of YOLOv7. Step S5.3: In the neck network of YOLOv7, the spatial pyramid dilated convolution SPD is used to improve the maximum pooling structure MP downsampling operation.

8. The method for detecting heating defects of electric power equipment based on infrared images as claimed in claim 7, characterized in that: In the step S5.1, three SimAM attention mechanism modules are introduced between the effective feature layers C3, C4, and C5 of the YOLOv7 backbone network and the neck input of YOLOv7.

9. The method for detecting heating defects of electric power equipment based on infrared images according to claim 7, characterized in that: In step S5.1, three SimAM attention mechanism modules are introduced between the neck network output of YOLOv7 and the head input of YOLOv7.

10. The method for detecting heating defects of electric power equipment based on infrared images according to claim 7, characterized in that: In step S5.2, the cross-level weighted feature pyramid network includes four cross-level weighted connection modules.

Images