Overhead line insulator defect detection method in foggy day scene

Abstract

The invention provides an overhead line insulator defect detection method in a foggy day scene, which is suitable for the field of line inspection and fault identification in a power system. The method comprises the following steps: firstly, preprocessing an insulator image collected by an unmanned aerial vehicle, constructing a foggy day insulator defect data set, then inputting image data into a foggy day insulator defect detection model, and finally outputting a defogged insulator defect detection result. According to the invention, organic combination of two models of an image defogging algorithm and a target detection network is realized, and on the premise of ensuring image processing quality, insulator faults in special weather scenes can still be rapidly and accurately identified.

Description

Technical Field

[0001] The present invention relates to the field of computer vision and image recognition, and specifically to a method for detecting defects of overhead line insulators in foggy weather scenarios. Background Art

[0002] China's transmission lines are characterized by a wide distribution range. Insulators on overhead transmission lines are long-term exposed to complex natural environments and are vulnerable to various defects caused by factors such as lightning, hail, and pollution, which seriously affect the power transmission link in the overall operation of the power system. To ensure the safe and stable operation of transmission lines and timely detect various defects of insulators and factors endangering the safe operation of the lines, it is necessary to strengthen the inspection and fault troubleshooting efficiency of transmission lines. Traditional manual inspections have a high labor intensity and are prone to incomplete inspections. Therefore, in recent years, drone inspections have become an important development direction of inspection technologies. However, extreme weather such as haze in the real environment will cause a serious decline in the quality of photos taken by cameras. Haze consists of suspended particles and aerosols, and its optical properties will cause multiple interferences to detection equipment. For example, the contrast between the target object and the background is reduced, the contours of insulators and the boundaries of defects are unclear, etc., making it more difficult to locate and identify insulator defects in foggy weather.

[0003] Drone inspections are often combined with deep learning and computer vision technologies. Regarding the research on target detection algorithms, it mainly focuses on classic algorithms based on convolutional neural networks such as YOLO, SSD, and R-CNN, and many researchers have achieved fruitful results. Existing dehazing algorithms include two categories: traditional dehazing methods and dehazing methods based on deep learning. Among traditional dehazing methods, they are further divided into image enhancement-based methods represented by histogram equalization and Retinex algorithms, and physical model-based methods represented by dark channel prior and color prior. In complex scenarios, dehazing methods based on deep learning have significant advantages, such as the DehazeNet dehazing network and the AOD-Net dehazing network. However, the training of the model depends on a large amount of data, and the running speed is slow and the real-time performance is insufficient. Summary of the Invention

[0004] For the above research background and current situation, the present invention proposes a method for detecting defects of overhead line insulators in foggy weather scenarios. The present invention divides the detection of insulator defects in foggy weather into two major tasks: image dehazing and target detection, and conducts research around these two tasks. First, an improved dark channel dehazing algorithm is proposed for image dehazing. The fused dark channel is calculated to generate a more accurate dark channel, and image pyramid decomposition and reconstruction are introduced to accelerate the processing speed. Then, the dehazing algorithm is embedded into the target detection model. Here, a single-stage detection model is selected for the target detection model, which has high detection efficiency and is convenient for fusion with the dehazing algorithm. The following steps are included in the process of using this method:

[0005] Step 1: Collect the insulator defect images captured by the drone during the transmission line inspection, and preprocess and augment the data. The specific process is as follows:

[0006] Control the drone to take pictures of the insulators during the transmission line inspection. Uniformly process the insulator photos collected by the drone into a size of 640×640×3. Use the data annotation tool labelimg to perform feature annotation on all the photos, and then perform fogging operations on the photos with completed feature annotation based on the atmospheric scattering model. To further expand the sample quantity, perform data augmentation on the fogged images using rotation operations such as horizontal flipping, vertical flipping, 30° clockwise, and 60° clockwise. Divide the insulator defect photos of each category into a training set, a validation set, and a test set according to a ratio of 8:1:1. After combining the training sets, validation sets, and test sets of each defect category, form a foggy-day insulator defect detection data set, and complete the fogging operation of the images and the preprocessing of the data set.

[0007] When taking pictures in a foggy environment, the light source received by the device will be interfered by the haze, resulting in a decrease in image quality. On the one hand, the reflected light energy of the target object will be absorbed by the suspended particles in the air during transmission. On the other hand, the presence of impurity particles in the air causes the ambient light to scatter and form many interfering light sources. The atmospheric scattering model establishes a mathematical model based on the above principles to describe the generation process of haze in the natural environment. The formula for using the atmospheric scattering model to simulate the haze effect in the real state is: I(x) = J(x)t(x) + A(1 - t(x))

[0008] Among them, I(x) represents the generated foggy-day image, t(x) represents the transmittance, J(x) represents the haze-free image, and A represents the atmospheric light intensity. t(x) can be specifically expressed as: t(x) = e -βd(x)

[0009] Among them, β is the atmospheric scattering coefficient. By controlling the value of β, simulate the insulator images captured under three haze states of mild, moderate, and severe, and further construct an insulator defect data set for various foggy-day scenarios.

[0010] Step 2: Obtain a foggy-day insulator defect detection model combined with an improved dark channel dehazing algorithm, and use the dehazed image data to train the model to obtain an optimal detection model. The steps before step (2) also include: improving the dark channel dehazing algorithm and embedding the dehazing algorithm into the target detection model. The specific process is as follows:

[0011] (1) First, use the fusion dark channel calculation to generate a more accurate dark channel, and then introduce image pyramid decomposition and reconstruction to speed up the processing speed.

[0012] (2) Calculate the dark channel using a small window (neighborhood size of 3×3) and a large window (neighborhood size of 15×15) respectively, and obtain and The calculation formula for the dark channel is:

[0013] where Ω(x) represents the window centered at x, I dark (x) represents the dark channel value of pixel x, and I c (y) represents the luminance value of pixel y in channel c. Therefore, we have:

[0014] After obtaining the dark channels under the two windows, calculate the morphological gradient of the image, and use the morphological gradient to compensate the dark channel values at the image edges. The morphological gradient is defined as the difference between image dilation and erosion: G(x,y) = Dilate(x,y) - Erode(x,y)

[0015] where Dilate(x,y) represents the dilation operation value of pixel point (x,y), and Erode(x,y) represents the erosion operation value of pixel point (x,y). After normalizing the gradient values to the interval [0,1], perform the next weighted operation. The normalization formula is as follows:

[0016] max(G) and min(G) are the maximum and minimum values of the image gradient respectively. After obtaining the normalized G norm then combine the morphological gradient information to assign weights to the dark channels of the large window and the small window. When calculating the weights by combining the gradient information, introduce the HardSigmoid function to limit the output within the range of [0,1]. At the same time, the ξ coefficient ensures that the weights are adjusted smoothly with the gradient change without drastic fluctuations. The following are the weight calculation formula and the weighted fusion formula for the dark channels of the large window and the small window:

[0017] By designing the above weighting method, the weight of the dark channel of the small window is higher in the area with a large morphological gradient, and the weight of the dark channel of the large window is higher in the area with a small morphological gradient. The above dynamic weight assignment enables the algorithm to adaptively adjust the calculation strategy in different regions of the image, obtain more accurate dark channel values, retain more image details while reducing noise interference.

[0018] (3) The original image is continuously downsampled as the bottom layer to obtain a Gaussian pyramid, and a defogging algorithm is applied to the lowest-resolution image generated by the Gaussian pyramid. To retain more image details, a 3×3 convolutional kernel is used. Each time the Gaussian pyramid downsamples the image, the width and height are reduced by half. Construct a Gaussian pyramid with K layers. If the number of rows and columns of the image is M and N, respectively, the expression for the k-th layer of the Gaussian pyramid is: (0 < k < K, 0 < x < M, 0 < y < N)

[0019] Then, a Laplacian pyramid is constructed to store the high-frequency details of each layer of the image. The low-layer image is upsampled to restore the resolution of the upper-layer image:

[0020] Each layer of the Laplacian pyramid can be obtained through differential operations: L k =G k -I k

[0021] After generating the Gaussian pyramid and the Laplacian pyramid, the Laplacian pyramid is reconstructed. The reconstruction starts from the lowest resolution and superimposes the high-frequency details to gradually restore the high-resolution defogged image layer by layer: I k =G k +L k

[0022] In this invention, by constructing an image pyramid and applying a defogging algorithm to the lowest-resolution image, the computational complexity is significantly reduced. The image details are retained through layer-by-layer reconstruction of the Laplacian pyramid, achieving a balance between defogging speed and detail retention.

[0023] (4) After obtaining the improved dark channel defogging algorithm and the target detection model, this invention applies the improved dark channel defogging algorithm to the data loading stage of the YOLO series algorithms. The data loading stage mainly includes obtaining the image file and the corresponding label file. This invention applies the improved dark channel defogging algorithm during the process of obtaining the image file. Finally, the image input into the target detection model is the clear image after defogging processing. This not only avoids additional data preprocessing but also adapts to the single-stage target detection algorithm used, completing the tasks of image defogging and target detection successively without affecting other working processes.

[0024] Step 3: Input the foggy insulator image data to be detected into the trained optimal detection model, output the insulator defect detection result, and complete the positioning and type recognition of the insulator defect. Description of the Drawings

[0025] Figure 1 is a flowchart of a method for detecting defects of overhead line insulators in a foggy scenario according to the present invention

[0026] Figure 2 is an architecture diagram of a foggy insulator defect detection model combining an improved dark channel dehazing algorithm in the present invention

[0027] Figure 3 is a structural diagram of an image pyramid in the present invention Detailed Description of the Invention

[0028] The present invention will be described in detail below with reference to the accompanying drawings:

[0029] Figure 1 is a flowchart of a method for detecting defects of overhead line insulators in a foggy scenario according to the present invention. As shown in the flowchart, the detection method includes the following steps:

[0030] Step 1: Collect the insulator defect images captured by the drone during the inspection of the transmission line, and preprocess and enhance the data of the images. The specific process is as follows:

[0031] Control the drone to take pictures of the insulators during the inspection of the transmission line, uniformly process the insulator photos collected by the drone into a size of 640×640×3, use the data annotation tool labelimg to perform feature annotation on all the photos, and then perform a fogging operation on the photos with completed feature annotation based on the atmospheric scattering model. In order to further expand the sample quantity, perform data enhancement on the fogged images using rotation operations such as horizontal flipping, vertical flipping, 30° clockwise, and 60° clockwise. Divide the insulator defect photos of each category into a training set, a validation set, and a test set according to a ratio of 8:1:1. After combining the training sets, validation sets, and test sets of each defect category, a foggy insulator defect detection data set is formed, and the fogging operation of the images and the preprocessing of the data set are completed.

[0032] When taking pictures in a foggy environment, the light source received by the device will be interfered by the haze, resulting in a reduction in image quality. On the one hand, the reflected light energy of the target object will be absorbed by the suspended particles in the air during transmission. On the other hand, the existence of impurity particles in the air causes the environmental light to scatter and form many interfering light sources. The atmospheric scattering model establishes a mathematical model based on the above principles to describe the generation process of haze in the natural environment. The formula for using the atmospheric scattering model to simulate the haze effect in the real state is: I(x) = J(x)t(x) + A(1 - t(x))

[0033] Among them, I(x) represents the generated foggy image, t(x) represents the transmittance, J(x) represents the fog-free image, A represents the atmospheric light intensity, and t(x) can be specifically expressed as: t(x) = e -βd(x)

[0034] Among them, β is the atmospheric scattering coefficient. By controlling the value of β, insulator images captured under three haze states of mild, moderate, and severe are simulated, and an insulator defect dataset with multiple foggy scenarios is further constructed.

[0035] Step 2: Obtain a foggy insulator defect detection model combined with an improved dark channel defogging algorithm, and use the defogged image data to train the model to obtain an optimal detection model. The steps before step (2) also include: improving the dark channel defogging algorithm and embedding the defogging algorithm into the object detection model. The specific process is as follows:

[0036] (1) First, use the fusion dark channel calculation to generate a more accurate dark channel, and then introduce image pyramid decomposition and reconstruction to accelerate the processing speed.

[0037] (2) Calculate the dark channel using a small window (neighborhood size is 3×3) and a large window (neighborhood size is 15×15) respectively to obtain and The calculation formula of the dark channel is:

[0038] Among them, Ω(x) represents the window centered on x, and I dark (x) represents the dark channel value of pixel x, and I c (y) represents the brightness value of pixel y in channel c. Therefore, there is:

[0039] After obtaining the dark channels under the two windows, calculate the morphological gradient of the image, and use the morphological gradient to compensate the dark channel values at the image edges. The morphological gradient is defined as the difference between image dilation and erosion: G(x,y) = Dilate(x,y) - Erode(x,y)

[0040] Among them, Dilate(x,y) represents the dilation operation value of pixel point (x,y), and Erode(x,y) represents the erosion operation value of pixel point (x,y). After normalizing the gradient value to the interval [0,1], the following weighted operation is performed. The normalization formula is as follows:

[0041] max(G) and min(G) are the maximum and minimum values of the image gradient respectively. After obtaining the normalized G normAfter combining the morphological gradient information, weights are assigned to the dark channels of the large window and the small window. When calculating the weights by combining the gradient information, the HardSigmoid function is introduced to limit the output within the range of [0, 1]. At the same time, the ξ coefficient ensures that the weights are smoothly adjusted with the change of the gradient and there will be no drastic fluctuations. The following are the weight calculation formula and the weighted fusion formula for the dark channels of the large window and the small window:

[0042] By designing the above-mentioned weighted method, the weights of the dark channels of the small window are higher in the areas with large morphological gradients, and the weights of the dark channels of the large window are higher in the areas with small morphological gradients. The above dynamic weight assignment enables the algorithm to adaptively adjust the calculation strategy in different regions of the image, obtain more accurate dark channel values, reduce noise interference while retaining more image details.

[0043] (3) The original image is continuously downsampled as the bottom layer to obtain a Gaussian pyramid, and the dehazing algorithm is applied to the lowest-resolution image generated by the Gaussian pyramid. A 3×3 convolutional kernel is used to retain more image details. Each time the image is downsampled in the Gaussian pyramid, the width and height are reduced by half. To construct a Gaussian pyramid with K layers, if the number of rows and columns of the image is M and N, the expression of the k-th layer of the Gaussian pyramid is: (0 < k < K, 0 < x < M, 0 < y < N)

[0044] Then a Laplacian pyramid is constructed to store the high-frequency details of each layer of the image. The low-layer image is upsampled to restore the resolution of the upper-layer image:

[0045] Each layer of the Laplacian pyramid can be obtained through differential operations: L k =G k -I k

[0046] After generating the Gaussian pyramid and the Laplacian pyramid, the Laplacian pyramid is reconstructed. The reconstruction starts from the lowest resolution and superimposes the high-frequency details to gradually restore the high-resolution dehazed image layer by layer: I k =G k +L k

[0047] In the present invention, by constructing an image pyramid and applying the dehazing algorithm to the lowest-resolution image, the computational complexity is greatly reduced. The details of the image are retained through layer-by-layer reconstruction of the Laplacian pyramid, and a balance is constructed between the dehazing speed and the retention of details.

[0048] (4) After obtaining the improved dark channel dehazing algorithm and the object detection model, the present invention applies the improved dark channel dehazing algorithm to the data loading stage of the YOLO series algorithms. The data loading stage mainly includes obtaining the image file and the corresponding label file. The present invention applies the improved dark channel dehazing algorithm during the process of obtaining the image file. The image finally input into the object detection model is the clear image after dehazing processing. In this way, it not only avoids additional data preprocessing but also can adapt to the single-stage object detection algorithm used, and completes the image dehazing and object detection tasks successively without affecting other working processes.

[0049] Input the divided training set into the network model with various training hyperparameters already set for iterative training. After each iterative training, use the validation set to verify the performance of the model. Finally, obtain the optimal network model after multiple iterative trainings, and use the test set to test the optimal network model to view the test results.

[0050] Step 3: Input the foggy insulator image data to be detected into the trained optimal detection model, output the insulator defect detection result, and complete the positioning and type identification of the insulator defect.

Claims

1. An overhead line insulator defect detection method in a foggy weather scenario, characterized in that, The method includes the following steps: Step 1: Collect the insulator defect images captured by the drone during the inspection of the transmission line, and preprocess and enhance the data of the images. Step 2: Obtain a foggy-day insulator defect detection model combined with an improved dark channel defogging algorithm, and use the defogged image data to train the model to obtain an optimal detection model. Step 3: Input the foggy-day insulator image data to be detected into the trained optimal detection model, output the insulator defect detection result, and complete the positioning and type identification of the insulator defect.

2. The method for detecting defects of overhead line insulators in a foggy weather scenario according to claim 1, wherein Collect the insulator defect images captured by the drone during the inspection of the transmission line, and preprocess and enhance the data of the images. Specifically: Control the drone to take pictures of the insulators during the inspection of the transmission line, uniformly process the insulator photos collected by the drone into a size of 640×640×3, use the data annotation tool labelimg to perform feature annotation on all the photos, and then perform a fogging operation on the photos with completed feature annotation based on the atmospheric scattering model. To further expand the sample quantity, perform data enhancement on the fogged images using rotation operations such as horizontal flipping, vertical flipping, 30° clockwise, and 60° clockwise. Divide the insulator defect photos of each category into a training set, a validation set, and a test set according to a ratio of 8:1:

1. After combining the training sets, validation sets, and test sets of each defect category, a foggy-day insulator defect detection data set is formed, and the fogging operation of the images and the preprocessing of the data set are completed.

3. A method for detecting defects of overhead line insulators in a foggy weather scenario according to claim 1, characterized in that The obtained foggy-day insulator defect detection model combined with an improved dark channel defogging algorithm is specifically: First, obtain the improved dark channel defogging algorithm, obtain the object detection model, and embed the defogging algorithm into the object detection model to obtain a complete foggy-day insulator defect detection model, which can be used to complete the training and detection tasks.

4. A method for detecting defects of overhead line insulators in foggy weather according to claim 2, characterized in that Perform a fogging operation on the picture based on the atmospheric scattering model. Specifically: When taking pictures in a foggy environment, the light source received by the device will be interfered by the haze, resulting in a reduction in image quality. On the one hand, the reflected light energy of the target object will be absorbed by the suspended particles in the air during the transmission process. On the other hand, the existence of impurity particles in the air causes the ambient light to scatter to form many interfering light sources. The atmospheric scattering model establishes a mathematical model based on the above principles to describe the generation process of haze in the natural environment. The formula for using the atmospheric scattering model to simulate the haze effect in the real state is: I(x) = J(x)t(x) + A(1 - t(x)) Among them, I(x) represents the generated foggy-day image, t(x) represents the transmittance, J(x) represents the haze-free image, A represents the atmospheric light intensity, and t(x) can be specifically expressed as: t(x) = e -βd(x) Among them, β is the atmospheric scattering coefficient. By controlling the value of β, simulate the insulator images taken in three haze states of mild, moderate, and severe, and further construct an insulator defect data set for various foggy-day scenarios.

5. The method for detecting defects of overhead line insulators in a foggy scenario according to claim 3, wherein The obtained improved dark channel defogging algorithm is specifically: Compared with the original dark channel defogging algorithm, the present invention makes the following improvements: First, use the fusion dark channel calculation to generate a more accurate dark channel, and then introduce image pyramid decomposition and reconstruction to speed up the processing speed.

6. The method for detecting defects of overhead line insulators in a foggy weather scenario according to claim 3, characterized in that Generate a more accurate dark channel using the fusion dark channel calculation, specifically as follows: Calculate the dark channel using a small window (neighborhood size of 3×3) and a large window (neighborhood size of 15×15) respectively, and obtain and The calculation formula for the dark channel is: where Ω(x) represents the window centered at x, and I dark (x) represents the dark channel value of pixel x, and I c (y) represents the luminance value of pixel y in channel c. Therefore, we have: After obtaining the dark channels under two types of windows, calculate the morphological gradient of the image, and use the morphological gradient to compensate the dark channel values at the edges of the image. The morphological gradient is defined as the difference between image dilation and erosion: G(x,y) = Dilate(x,y) - Erode(x,y) where Dilate(x, y) represents the dilation operation value of the pixel point (x, y), and Erode(x, y) represents the erosion operation value of the pixel point (x, y). After normalizing the gradient values to the interval [0, 1], perform the next weighted operation. The normalization formula is as follows: max(G) and min(G) are the maximum and minimum values of the image gradient respectively. The normalized G is obtained. norm After that, the morphological gradient information is combined to assign weights to the dark channels of the large window and the small window. When calculating the weights by combining the gradient information, the HardSigmoid function is introduced to limit the output within the range of [0, 1]. At the same time, the ξ coefficient ensures that the weights are smoothly adjusted with the change of the gradient and there will be no violent fluctuations. The following are the weight calculation formula and the weighted fusion formula for the dark channels of the large window and the small window: By designing the above weighting method, the weight of the small-window dark channel is higher in the area with a large morphological gradient, and the weight of the large-window dark channel is higher in the area with a small morphological gradient. The above dynamic weight allocation enables the algorithm to adaptively adjust the calculation strategy in different regions of the image, obtain a more accurate dark channel value, retain more image details while reducing noise interference.

7. A method for detecting defects of overhead line insulators in foggy weather scenarios according to claim 3, characterized in that Introduce image pyramid decomposition and reconstruction to accelerate the processing speed, specifically as follows: Use the original image as the bottom layer and continuously downsample to obtain the Gaussian pyramid. Apply the defogging algorithm to the lowest-resolution image generated by the Gaussian pyramid. Use a 3×3 convolutional kernel to retain more image details. Each time the Gaussian pyramid downsamples the image, the width and height are reduced by half. Build a Gaussian pyramid with K layers. If the number of rows and columns of the image is M and N, then the expression of the k-th layer of the Gaussian pyramid is: Then, a Laplacian pyramid is constructed to store the high-frequency details of each layer of the image. The low-layer image is upsampled to restore the resolution of the upper-layer image: Each layer of the Laplacian pyramid can be obtained through differential operations: L k = G k - I k After generating the Gaussian pyramid and the Laplacian pyramid, reconstruct the Laplacian pyramid. The reconstruction starts from the lowest resolution and superimposes high-frequency details to gradually restore the high-resolution defogged image: I k = G k + L k Through constructing an image pyramid, applying the defogging algorithm to the lowest-resolution image in this invention greatly reduces the computational complexity. By reconstructing layer by layer with the Laplacian pyramid, image details are retained, and a balance is constructed between the defogging speed and detail retention.

8. A method for detecting defects of overhead line insulators in a foggy weather scenario according to claim 3, characterized in that Obtain the object detection model, specifically as follows: This patent is applied to the field of real-time object detection. Therefore, the obtained object detection model is a single-stage object detection algorithm of the YOLO series (YOLOv5, YOLOv8, YOLOv11, etc.). When the YOLO algorithm is used to complete the object detection task and the improved dark channel defogging algorithm is used to complete the image defogging task, both algorithms can ensure a high processing speed and robustness.

9. The method for detecting defects of overhead line insulators in a foggy weather scenario according to claim 3, wherein Embed the defogging algorithm into the object detection model, specifically as follows: After obtaining the improved dark channel defogging algorithm and the object detection model, this invention applies the improved dark channel defogging algorithm to the data loading stage of the YOLO series algorithms. The data loading stage mainly includes obtaining the image file and the corresponding label file. This invention applies the improved dark channel defogging algorithm during the process of obtaining the image file. Finally, the image input into the object detection model is the clear image after defogging processing. This not only avoids additional data preprocessing but also adapts to the used single-stage object detection algorithm, and completes the image defogging and object detection tasks successively without affecting other working processes.

Images