Wire clamp heating fault detection method based on infrared image

Abstract

The invention provides a wire clamp heating fault detection method based on an infrared image, and relates to the technical field of machine vision detection. Comprising the following steps: acquiring an infrared image of an aerial power transmission line of an unmanned aerial vehicle, constructing a heating infrared image data set of the strain clamp and the suspension clamp, constructing a Yo-v7 network model, training and optimizing the Yo-v7 network model, and predicting the heating strain clamp and the suspension clamp in the infrared data by using the Yo-v7 network model. According to the wire clamp heating fault detection method based on the infrared image, the Yo-v7 network model and the computer vision technology are combined to detect the wire clamp heating fault in the infrared image or video, automatic sensing and analysis of the heating state of a strain clamp and a suspension clamp are achieved, and the automation and intelligence level of power transmission line inspection is effectively improved.

Description

Technical Field

[0001] The invention relates to the technical field of machine vision detection, and in particular to a method for detecting heating faults of wire clamps based on infrared images. Background Art

[0002] Transmission lines are an important part of the power system. Regular inspection of transmission line channels is an important measure to ensure the stable operation of the power system. The inspection content mainly includes the appearance defect detection of power equipment in transmission lines and the prediction of dangerous objects around power lines. Transmission lines are composed of many power equipment. Due to the long span of transmission lines, the environment in which power equipment is located is relatively complex. They are exposed to the outdoors for a long time and are exposed to wind and sun. Damage and rust are inevitable, which brings great safety hazards to the stable operation of the power grid. Suspension clamps and tension clamps are important power equipment for transmission lines. Among them, suspension clamps are used to suspend or support power lines, and are mainly hardware that bears vertical loads; tension clamps are used to fix power lines and lightning conductors on the insulator strings of tension, corner, and terminal towers, and are mainly hardware that bears the tension of power lines.

[0003] As the coverage of high-voltage overhead transmission lines becomes more and more extensive, and power grids are often built between mountains, transmission line channel inspection has become a heavy daily task for the power department. Power supply companies mostly use manual inspection as the main method to diagnose faults of power equipment. However, this inspection mode has the following problems: 1) High work risk for inspectors: The environment of transmission lines in some parts of my country is relatively steep. If the staff loses contact with the outside world due to signal loss, it is difficult to ask for help in the first time and face life-threatening danger. 2) Low inspection efficiency: Due to the complex environment of the transmission line corridor, which is generally built on the mountain, the environment is generally harsh. When inspecting, the staff spends a lot of time on overcoming the problems of the natural environment. 3) Poor inspection quality and failure to solve faults in time: Low inspection efficiency and heavy workload lead to long manual inspection work cycles. Often, a part has not been inspected yet, and the normal part that has been inspected has faults again. This situation not only increases the workload, but also leads to the failure to deal with transmission line faults in a timely manner. In recent years, drones have become light in weight, small in size, easy to carry, low in cost, and can fly automatically. They have the advantages of fast operation speed, high measurement accuracy, large measurement data, and high degree of automation. In order to overcome the shortcomings of the current manual inspection mode, the use of drones to inspect transmission lines has become a new way of line inspection. With the development of drone technology, intelligent inspection mobile platforms have emerged. In the field of power inspection, drones, with their own flexible and compact characteristics, carry infrared cameras to identify equipment and detect defects on transmission lines. Compared with manual inspections, there is no need to consider traffic obstacles. It not only reduces the difficulty of inspectors to overcome complex natural environments, but also can more effectively find the location of power equipment fault points, quickly repair and restore power supply. However, drone inspections will generate a large number of inspection images and videos. At present, the main method of manual inspection is to detect heating faults of tension clamps and suspension clamps, which not only consumes a lot of human resources, but also easily causes missed inspections or misjudgments. Summary of the invention

[0004] 1. Technical issues to be solved

[0005] In view of the problem that manual detection of wire clamp heating faults in the infrared images or videos of the transmission lines taken by drones is prone to defects such as missed detection or false detection, the present invention provides a wire clamp heating fault detection method based on infrared images, which uses a combination of deep learning and computer vision technology to detect wire clamp heating faults in infrared images or videos, realizes automatic perception and analysis of the heating status of tension wire clamps and suspension wire clamps, and effectively improves the automation and intelligence level of transmission line inspection.

[0006] (II) Technical solution

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0008] A method for detecting heating faults of wire clamps based on infrared images comprises the following steps:

[0009] S1. Obtain infrared images of power transmission lines taken by drones, and expand the number of samples through image data enhancement processing;

[0010] S2, constructing a dataset of heating infrared images of tension clamps and suspension clamps;

[0011] S3. Build the Yolo-v7 network model using the PyTorch deep learning framework;

[0012] S4, using the heating infrared image data set of the tension clamp and the suspension clamp in step S2 to train and optimize the deep learning network model of step S3, and finally obtaining the Yolo-v7 network model for automatic detection of heating images of tension clamps and suspension clamps;

[0013] S5. Input the infrared image or video data obtained by the UAV inspection of the transmission line into the Yolo-v7 network model optimized in step S4, use the Yolo-v7 network model to predict the heating tension clamps and suspension clamps in the infrared data, and output the prediction results.

[0014] According to an embodiment of the present invention, the image data enhancement process in step S1 adopts a generative adversarial network model, and the generative adversarial network model adopts a CycleGAN network model, and the network model includes a first generator G, a first discriminator D X , second generator F, second discriminator D Y The hardware heating infrared image data set is divided into X-domain images and Y-domain images, the X-domain images contain real samples X, and the Y-domain images contain real samples Y; the first generator G generates generated samples G(X) and reconstructed samples G(F(Y)), and the second generator F generates generated samples F(Y) and reconstructed samples F(G(X)); the real samples X and generated samples F(Y) are connected to the first generator G to generate generated samples G(X), and the generated samples G(X) and the real samples Y are connected to the second discriminator D Y It is used to judge the authenticity of the generated samples and obtain the loss function L GAN2 The real sample Y and the generated sample G(X) are connected to the second generator F to generate the generated sample F(Y), and the generated sample F(Y) is connected to the real sample X and the first discriminator D X It is used to judge the authenticity of the generated samples and obtain the loss function L GAN1 The reconstructed sample F(G(X)) and the real sample X, the reconstructed sample G(F(Y)) and the real sample Y constitute a cyclic loss function L cyc .

[0015] According to one embodiment of the present invention, the step S2 first uniformly adjusts the image resolution obtained in step S1 to a size of 416×416, and then uses the LabelImg tool to annotate the heating wire clip target, wherein 70% of the images in the data set are used for training and the remaining 30% of the images are used for testing.

[0016] According to one embodiment of the present invention, the Yolo-v7 network model in step S3 includes a backbone network, a feature fusion network and a prediction network, the backbone network includes a first feature extraction module, a second feature extraction module, a third feature extraction module, a fourth feature extraction module, a fifth feature extraction module, and an SPP module, and the first feature extraction module, the second feature extraction module, the third feature extraction module, the fourth feature extraction module, the fifth feature extraction module, and the SPP module are connected in sequence; the feature fusion module includes a first effective feature layer, a second effective feature layer, a third effective feature layer, a fourth effective feature layer, a fifth effective feature layer, and a sixth effective feature layer, the input end of the first effective feature layer is connected to the output end of the SPP module, and the input end of the second effective feature layer is connected to the output end of the fourth feature extraction module and the output end of the first effective feature layer through 2 times upsampling. The input end of the third effective feature layer is connected to the output end of the third feature extraction module and the output end of the second effective feature layer through 2 times upsampling, the input end of the fourth effective feature layer is connected to the output end of the third effective feature layer, the input end of the fifth effective feature layer is connected to the output end of the second effective feature layer and the output end of the fourth effective feature layer through 2 times downsampling, and the input end of the sixth effective feature layer is connected to the output end of the first effective feature layer and the output end of the fifth effective feature layer through 2 times downsampling; the prediction network includes a first Yolo-Head, a second Yolo-Head, and a third Yolo-Head, the first Yolo-Head is connected to the sixth effective feature layer, the second Yolo-Head is connected to the fifth effective feature layer, and the third Yolo-Head is connected to the fourth effective feature layer.

[0017] According to one embodiment of the present invention, the output end of the SPP module is connected to the input end of the sixth effective feature layer, the output end of the fourth feature extraction module is connected to the input end of the fifth effective feature layer, and the output end of the third feature extraction module is connected to the input end of the fourth effective feature layer.

[0018] According to one embodiment of the present invention, the first feature extraction module, the second feature extraction module, the third feature extraction module, the fourth feature extraction module, and the fifth feature extraction module respectively include a first 1*1 convolutional layer, a 3*3 depth-separable convolutional layer, a second 1*1 convolutional layer, and a feature fusion layer, and the first 1*1 convolutional layer, the 3*3 depth-separable convolutional layer, the second 1*1 convolutional layer, and the feature fusion layer are connected in sequence.

[0019] According to one embodiment of the present invention, the number of the first feature extraction modules is 1, the number of the second feature extraction modules is 2, the number of the third feature extraction modules is 8, the number of the fourth feature extraction modules is 8, and the number of the fifth feature extraction modules is 4.

[0020] (III) Beneficial effects

[0021] The beneficial effects of the present invention are as follows: a method for detecting heating faults of wire clamps based on infrared images, which detects heating faults of wire clamps in infrared images or videos by combining the Yolo-v7 network model with computer vision technology; the feature fusion network adopts a more efficient and fast BiFPN structure for feature fusion, which reduces the amount of calculation compared with PANet, adds jump connections, and integrates more features; the present invention realizes automatic perception and analysis of the heating status of tension wire clamps and suspension wire clamps, effectively improving the automation and intelligence level of transmission line inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 This is a flow chart of the method for detecting heating fault of a wire clamp of the present invention;

[0024] Figure 2 The CycleGAN network model of the present invention;

[0025] Figure 3 This is a structural diagram of the Yolo-v7 network model of the present invention;

[0026] Figure 4 This is a structural diagram of the feature extraction module of the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] Combination Figure 1, a method for detecting heating faults of wire clamps based on infrared images, comprising the following steps:

[0029] S1. Obtain infrared images of transmission lines taken by drones and expand the number of samples through image data enhancement processing.

[0030] Furthermore, the image data enhancement processing in step S1 adopts a generative adversarial network model, and the generative adversarial network model adopts a CycleGAN network model.

[0031] Combination Figure 2 The generative adversarial network model in step S1 is a CycleGAN network model, which includes a first generator G, a first discriminator D X , second generator F, second discriminator D Y The hardware heating infrared image data set is divided into X-domain images and Y-domain images. The X-domain images contain real samples X, and the Y-domain images contain real samples Y. The first generator G generates generated samples G(X) and reconstructed samples G(F(Y)), and the second generator F generates generated samples F(Y) and reconstructed samples F(G(X)). The real samples X and generated samples F(Y) are connected to the first generator G to generate generated samples G(X). The generated samples G(X) and the real samples Y are connected to the second discriminator D. Y It is used to judge the authenticity of the generated samples and obtain the loss function L GAN2 The real sample Y and the generated sample G(X) are connected to the second generator F to generate the generated sample F(Y), and the generated sample F(Y) is connected to the real sample X and the first discriminator D X It is used to judge the authenticity of the generated samples and obtain the loss function L GAN1 The reconstructed sample F(G(X)) and the real sample X, the reconstructed sample G(F(Y)) and the real sample Y constitute a cyclic loss function L cyc .

[0032] To further constrain the training process of the generator, a consistency loss function L is introduced ide , the real sample image and the generated sample image are input into the corresponding generator respectively, and the image output by the generator should remain unchanged. The cycle consistency loss is the consistency loss between the reconstructed image and the real image of the generated image obtained by reconstructing the images generated by the first generator G and the second generator F, and the L1 distance is used for measurement. The loss function L of the generative adversarial network model oss This is achieved by weighted summation, and only the values ​​of λ and β need to be adjusted to control the importance of reconstruction consistency. The loss function L of the generative adversarial network model is oss Including the first discriminator loss function L GAN1 , the second discriminator loss function LGAN2 , the cycle loss function L cyc , consistency loss function L ide , whose formulas are shown in (1)-(5):

[0033] L GAN1 =E X～Pdata(X) [logP X ]+E Y～Pdata(Y) [log(1-P X (F(Y)))] (1)

[0034] L GAN2 =E Y～Pdata(Y) [logP Y ]+E X～Pdata(X) [log(1-P Y (G(X)))] (2)

[0035] L cyc =E X～Pdata(X) [F(G(X))-X1]+E Y～Pdata(Y) [G(F(Y))-Y1] (3)

[0036] L ide =E X～Pdata(X) [G(X)-X1]+E Y～Pdata(Y) [F(Y)-Y1] (4)

[0037] L oss =L GAN1 +L GAN2 +λL cyc +βL ide (5)

[0038] Among them, P X , P Y is the probability that the data input to the discriminant model is the real data; X~Pdata(X) represents that X follows the real data distribution, and Y~Pdata(Y) represents that Y follows the real data distribution; E(·) is the expectation; ||·||1 represents the L1 norm. λ and β are weighting coefficients, λ can be equal to 10, and β can be equal to 5.

[0039] S2. Construct a dataset of infrared images of heating wire clamps and suspension wire clamps. First, adjust the image resolution obtained in step S1 to 416×416, and then use the LabelImg tool to annotate the heating wire clamps. 70% of the images in the dataset are used for training, and the remaining 30% are used for testing.

[0040] S3. Build the Yolo-v7 network model using the PyTorch deep learning framework.

[0041] like Figure 3 As shown, the Yolo-v7 network model includes a backbone network, a feature fusion network and a prediction network, the backbone network includes a first feature extraction module, a second feature extraction module, a third feature extraction module, a fourth feature extraction module, a fifth feature extraction module, and an SPP module, the first feature extraction module, the second feature extraction module, the third feature extraction module, the fourth feature extraction module, the fifth feature extraction module, and the SPP module are connected in sequence; the feature fusion module includes a first effective feature layer, a second effective feature layer, a third effective feature layer, a fourth effective feature layer, a fifth effective feature layer, and a sixth effective feature layer, the input end of the first effective feature layer is connected to the output end of the SPP module, the input end of the second effective feature layer is connected to the output end of the fourth feature extraction module and the output end of the first effective feature layer through 2 times upsampling, and the third The input end of the effective feature layer is connected to the output end of the third feature extraction module and the output end of the second effective feature layer through 2 times upsampling, the input end of the fourth effective feature layer is connected to the output end of the third effective feature layer, the input end of the fifth effective feature layer is connected to the output end of the second effective feature layer and the output end of the fourth effective feature layer through 2 times downsampling, and the input end of the sixth effective feature layer is connected to the output end of the first effective feature layer and the output end of the fifth effective feature layer through 2 times downsampling; the prediction network includes a first Yolo-Head, a second Yolo-Head, and a third Yolo-Head, the first Yolo-Head is connected to the sixth effective feature layer, the second Yolo-Head is connected to the fifth effective feature layer, and the third Yolo-Head is connected to the fourth effective feature layer.

[0042] The feature fusion structure used by the original Yolo-v7 network model is PANet, which aggregates features at different levels through a top-down and bottom-up bidirectional fusion network. In order to further improve the reasoning speed of the network model, the feature fusion network uses a more efficient and fast BiFPN structure for feature fusion. Compared with PANet, it reduces the amount of calculation, adds skip connections, and integrates more features.

[0043] Combination Figure 4 , the first feature extraction module, the second feature extraction module, the third feature extraction module, the fourth feature extraction module, and the fifth feature extraction module respectively include a first 1*1 convolution layer, a 3*3 depth-separable convolution layer, a second 1*1 convolution layer, and a feature fusion layer, and the first 1*1 convolution layer, the 3*3 depth-separable convolution layer, the second 1*1 convolution layer, and the feature fusion layer are connected in sequence. The number of the first feature extraction modules is 1, the number of the second feature extraction modules is 2, the number of the third feature extraction modules is 8, the number of the fourth feature extraction modules is 8, and the number of the fifth feature extraction modules is 4.

[0044] S4. Use the heating infrared image data set of the tension clamp and suspension clamp in step S2 to train and optimize the deep learning network model of step S3, and finally obtain the Yolo-v7 network model for automatic detection of heating images of tension clamps and suspension clamps.

[0045] S5. Input the infrared image or video data obtained by the UAV inspection of the transmission line into the Yolo-v7 network model optimized in step S4, use the Yolo-v7 network model to predict the heating tension clamps and suspension clamps in the infrared data, and output the prediction results.

[0046] To sum up, in the embodiments of the present invention, the wire clamp heating fault detection method based on infrared images of the present invention utilizes the Yolo-v7 network model and computer vision technology to detect the wire clamp heating fault in infrared images or videos; the feature fusion network adopts a more efficient and fast BiFPN structure for feature fusion, which reduces the amount of calculation compared with PANet, adds jump connections, and integrates more features; the present invention realizes automatic perception and analysis of the heating status of tension wire clamps and suspension wire clamps, effectively improving the automation and intelligence level of transmission line inspection.

[0047] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting heating faults of wire clamps based on infrared images, which is characterized by the following steps: S1. Obtain infrared images of power transmission lines taken by drones, and expand the number of samples through image data enhancement processing; S2, constructing a dataset of heating infrared images of tension clamps and suspension clamps; S3. Build the Yolo-v7 network model using the PyTorch deep learning framework; S4, using the heating infrared image data set of the tension clamp and the suspension clamp in step S2 to train and optimize the deep learning network model of step S3, and finally obtaining the Yolo-v7 network model for automatic detection of heating images of tension clamps and suspension clamps; S5. Input the infrared image or video data obtained by the UAV inspection of the transmission line into the Yolo-v7 network model optimized in step S4, use the Yolo-v7 network model to predict the heating tension clamps and suspension clamps in the infrared data, and output the prediction results.

2. A method for detecting heating faults of wire clamps based on infrared images as claimed in claim 1, characterized in that: The image data enhancement process in step S1 adopts a generative adversarial network model, and the generative adversarial network model adopts a CycleGAN network model. The network model includes a first generator G, a first discriminator D X , second generator F, second discriminator D Y The hardware heating infrared image data set is divided into X-domain images and Y-domain images, the X-domain images contain real samples X, and the Y-domain images contain real samples Y; the first generator G generates generated samples G(X) and reconstructed samples G(F(Y)), and the second generator F generates generated samples F(Y) and reconstructed samples F(G(X)); the real samples X and generated samples F(Y) are connected to the first generator G to generate generated samples G(X), and the generated samples G(X) and the real samples Y are connected to the second discriminator D Y It is used to judge the authenticity of the generated samples and obtain the loss function L GAN2 The real sample Y and the generated sample G(X) are connected to the second generator F to generate the generated sample F(Y), and the generated sample F(Y) is connected to the real sample X and the first discriminator D X It is used to judge the authenticity of the generated samples and obtain the loss function L GAN1 The reconstructed sample F(G(X)) and the real sample X, the reconstructed sample G(F(Y)) and the real sample Y constitute a cyclic loss function L cyc .

3. A method for detecting heating faults of wire clamps based on infrared images as claimed in claim 1, characterized in that: In step S2, the image resolution obtained in step S1 is first uniformly adjusted to a size of 416×416, and then the heating wire clip target is labeled using the LabelImg tool, wherein 70% of the images in the data set are used for training and the remaining 30% of the images are used for testing.

4. The method for detecting heating faults of wire clamps based on infrared images according to claim 1, characterized in that: In the step S3, the Yolo-v7 network model includes a backbone network, a feature fusion network and a prediction network, wherein the backbone network includes a first feature extraction module, a second feature extraction module, a third feature extraction module, a fourth feature extraction module, a fifth feature extraction module and an SPP module, wherein the first feature extraction module, the second feature extraction module, the third feature extraction module, the fourth feature extraction module, the fifth feature extraction module and the SPP module are connected in sequence; the feature fusion module includes a first effective feature layer, a second effective feature layer, a third effective feature layer, a fourth effective feature layer, a fifth effective feature layer and a sixth effective feature layer, wherein the input end of the first effective feature layer is connected to the output end of the SPP module, the input end of the second effective feature layer is connected to the output end of the fourth feature extraction module and the output end of the first effective feature layer through 2 times upsampling, and the first effective feature layer is connected to the output end of the fourth feature extraction module and the output end of the first effective feature layer through 2 times upsampling. The input end of the three effective feature layers is connected to the output end of the third feature extraction module and the output end of the second effective feature layer through 2 times upsampling, the input end of the fourth effective feature layer is connected to the output end of the third effective feature layer, the input end of the fifth effective feature layer is connected to the output end of the second effective feature layer and the output end of the fourth effective feature layer through 2 times downsampling, and the input end of the sixth effective feature layer is connected to the output end of the first effective feature layer and the output end of the fifth effective feature layer through 2 times downsampling; the prediction network includes a first Yolo-Head, a second Yolo-Head, and a third Yolo-Head, the first Yolo-Head is connected to the sixth effective feature layer, the second Yolo-Head is connected to the fifth effective feature layer, and the third Yolo-Head is connected to the fourth effective feature layer.

5. A method for detecting heating faults of wire clamps based on infrared images as claimed in claim 4, characterized in that: The output end of the SPP module is connected to the input end of the sixth effective feature layer, the output end of the fourth feature extraction module is connected to the input end of the fifth effective feature layer, and the output end of the third feature extraction module is connected to the input end of the fourth effective feature layer.

6. A method for detecting heating faults of wire clamps based on infrared images as claimed in claim 4, characterized in that: The first feature extraction module, the second feature extraction module, the third feature extraction module, the fourth feature extraction module, and the fifth feature extraction module respectively include a first 1*1 convolutional layer, a 3*3 depth-separable convolutional layer, a second 1*1 convolutional layer, and a feature fusion layer, and the first 1*1 convolutional layer, the 3*3 depth-separable convolutional layer, the second 1*1 convolutional layer, and the feature fusion layer are connected in sequence.

7. A method for detecting heating faults of wire clamps based on infrared images as claimed in claim 6, characterized in that: The number of the first feature extraction modules is 1, the number of the second feature extraction modules is 2, the number of the third feature extraction modules is 8, the number of the fourth feature extraction modules is 8, and the number of the fifth feature extraction modules is 4.

Images