Image processing method and device for power transmission line in rain and fog weather conditions

Abstract

The application provides a power transmission line image processing method and device under rain and fog weather conditions, and belongs to the field of image processing. The method comprises the following steps: obtaining an original image corresponding to a to-be-tested power transmission line; inputting the original image into a multi-scale feature fusion network to obtain a dehazed image output by the multi-scale feature fusion network; and using the dehazed image for image recognition to determine a fault detection result corresponding to the to-be-tested power transmission line; wherein the multi-scale feature fusion network comprises a feature extraction module, a feature interaction module and a fusion output module; an input end of the feature extraction module is used for receiving the original image, a plurality of output ends of the feature extraction module are respectively connected to a plurality of input ends of the feature interaction module, and a plurality of output ends of the feature interaction module are all connected to the fusion output module. The application can better image detail information and improve the fault detection precision of the power transmission line.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and apparatus for image processing of power transmission lines under rainy and foggy weather conditions. Background Technology

[0002] With the continuous increase in the mileage of power transmission lines in my country, coupled with the influence of complex geographical terrain and climate, traditional manual line inspection methods can no longer meet the current needs of power transmission line inspection. Therefore, the highly efficient and accurate drone inspection method has emerged. Drone inspection, through image recognition of images of power transmission lines collected by drones, can achieve fault detection of power transmission lines.

[0003] However, in rainy and foggy weather conditions, images of power transmission lines captured by drones are prone to fogging, which degrades image quality and consequently reduces the accuracy of fault detection. Therefore, it is necessary to perform defogging processing on power transmission line images taken in rainy and foggy weather to improve image quality and ensure the accuracy of fault detection.

[0004] In related technologies, a model framework for fog imaging is typically constructed, and existing prior knowledge is used to estimate model parameters, thereby inversely generating a clear, fog-free image. For example, a fog-free image can be estimated using a dark channel prior and an atmospheric scattering model, or a fog-free image can be recovered by estimating transmittance based on fog line prior theory.

[0005] Although the above methods can remove fog from transmission line images to some extent, they are prone to losing image details during the defogging process, resulting in blurry and color-distorted images, which in turn reduces the accuracy of transmission line fault detection. Summary of the Invention

[0006] This invention provides a method and apparatus for image processing of power transmission lines under rainy and foggy weather conditions, in order to solve the problem that existing image defogging methods easily lose image detail information, thereby reducing the accuracy of fault detection of power transmission lines.

[0007] In a first aspect, embodiments of the present invention provide a method for image processing of power transmission lines under rainy and foggy weather conditions, including:

[0008] Acquire the original image corresponding to the transmission line under test;

[0009] The original image is input into a multi-scale feature fusion network to obtain a fog-free image output by the multi-scale feature fusion network; the fog-free image is used for image recognition to determine the fault detection result corresponding to the transmission line under test.

[0010] The multi-scale feature fusion network includes a feature extraction module, a feature interaction module, and a fusion output module. The input of the feature extraction module is used to receive the original image, and the multiple outputs of the feature extraction module are respectively connected to the multiple inputs of the feature interaction module. The multiple outputs of the feature interaction module are all connected to the fusion output module.

[0011] The feature extraction module extracts features from the original image at different scales to obtain the first feature image, the second feature image, and the third feature image, and outputs them to the feature interaction module.

[0012] The feature interaction module determines the fourth feature image based on the third feature image; determines the fifth feature image based on the third feature image and the second feature image; determines the sixth feature image based on the fifth feature image and the first feature image, and outputs the fourth feature image, the fifth feature image and the sixth feature image to the fusion output module;

[0013] The fusion output module determines the dehazed image based on the feature image output by the feature interaction module.

[0014] In one possible implementation, the feature extraction module includes: a first feature extraction unit, a second feature extraction unit, and a third feature extraction unit;

[0015] The input end of the first feature extraction unit is used to receive the original image, the first output end is used to output the first feature image, and the second output end is used to output the intermediate feature variables.

[0016] The input terminal of the second feature extraction unit is used to receive the first feature image, and the output terminal is used to output the second feature image;

[0017] The input end of the third feature extraction unit is used to receive the second feature image, and the output end is used to output the third feature image;

[0018] The feature interaction module includes: a first scale fusion unit, a second scale fusion unit, and a third scale fusion unit;

[0019] The input of the third scale fusion unit is connected to the output of the third feature extraction unit, and the output of the third scale fusion unit is connected to the input of the fusion output module. The third feature image is convolved to obtain a fourth feature image, which is then input to the fusion output module.

[0020] The first input terminal of the second scale fusion unit is connected to the output terminal of the third feature extraction unit, the second input terminal of the second scale fusion unit is connected to the output terminal of the second feature extraction unit, and the output terminal of the second scale fusion unit is connected to the first input terminal of the first scale fusion unit and the input terminal of the fusion output module, respectively, for processing the third feature image and the second feature image to obtain a fifth feature image, which is then input to the first scale fusion unit and the fusion output module, respectively.

[0021] The first input terminal of the first scale fusion unit is connected to the output terminal of the second scale fusion unit, the second input terminal of the first scale fusion unit is connected to the first output terminal of the first feature extraction unit, the third input terminal of the first scale fusion unit is connected to the second output terminal of the first feature extraction unit, and the output terminal of the first scale fusion unit is connected to the input terminal of the fusion output module. This is used to process the fifth feature image, the first feature image, and the intermediate feature variables to obtain a sixth feature image, which is then input to the fusion output module.

[0022] In one possible implementation, the second scale fusion unit includes: a first upsampling layer, a first feature fusion layer, and a first convolutional layer;

[0023] The input of the first upsampling layer is connected to the output of the third feature extraction unit, and the output of the first upsampling layer is connected to the input of the first feature fusion layer.

[0024] The input of the first feature fusion layer is also connected to the output of the second feature extraction unit, and the output of the first feature fusion layer is connected to the input of the first convolutional layer.

[0025] The output of the first convolutional layer is connected to the first input of the first scale fusion unit and the input of the fusion output module, respectively.

[0026] In one possible implementation, the first scale fusion unit includes: a second upsampling layer, a second feature fusion layer, a second convolutional layer, a first downsampling layer, and a first feature extraction layer;

[0027] The input of the second upsampling layer is connected to the output of the second scale fusion unit, and the output of the second upsampling layer is connected to the input of the second feature fusion layer.

[0028] The input of the second feature fusion layer is also connected to the first output of the first feature extraction unit, and the output of the second feature fusion layer is connected to the input of the second convolutional layer.

[0029] The input of the second convolutional layer is also connected to the second output of the first feature extraction unit, and the output of the second convolutional layer is connected to the input of the first downsampling layer.

[0030] The output of the first downsampling layer is connected to the input of the first feature extraction layer;

[0031] The output of the first feature extraction layer is connected to the input of the fusion output module.

[0032] In one possible implementation, the first feature extraction unit includes: a second feature extraction layer and a second downsampling layer;

[0033] The input end of the second feature extraction layer is used to receive the original image, and the output end is connected to the input end of the second downsampling layer and the third input end of the first scale fusion unit, respectively.

[0034] The output of the second downsampling layer is connected to the input of the second feature extraction unit and the second input of the first scale fusion unit.

[0035] In one possible implementation, the second feature extraction layer comprises a third convolutional layer and a convolutional attention block connected in sequence.

[0036] The convolutional attention block includes: a spatial depth transformation convolutional layer, a third feature fusion layer, a detail enhancement convolutional layer, a fourth convolutional layer, a CBAM attention mechanism layer, and a fourth feature fusion layer;

[0037] The input of the spatial depth transformation convolutional layer and the input of the third feature fusion layer are both connected to the third convolutional layer;

[0038] The input of the third feature fusion layer is also connected to the output of the spatial depth transformation convolutional layer, and the output of the third feature fusion layer is connected to the input of the detail enhancement convolutional layer.

[0039] The output of the detail enhancement convolutional layer is connected to the input of the fourth convolutional layer and the input of the fourth feature fusion layer, respectively.

[0040] The output of the fourth convolutional layer is connected to the input of the CBAM attention mechanism layer, and the output of the CBAM attention mechanism layer is connected to the input of the fourth feature fusion layer.

[0041] The output of the fourth feature fusion layer is connected to the input of the second downsampling layer and the third input of the first scale fusion unit.

[0042] In one possible implementation, the third feature extraction unit includes: a third feature extraction layer and an improved SPPF submodule;

[0043] The input end of the third feature extraction layer is connected to the output end of the second feature extraction unit;

[0044] The output of the third feature extraction layer is connected to the input of the improved SPPF submodule, and the output of the improved SPPF submodule is connected to the first input of the second scale fusion unit and the input of the third scale fusion unit, respectively.

[0045] In one possible implementation, the improved SPPF submodule includes: a fifth convolutional layer, a first pooling layer, a second pooling layer, a third pooling layer, a connection layer, an LSKA mechanism layer, and a sixth convolutional layer;

[0046] The input of the fifth convolutional layer is connected to the output of the third feature extraction layer, and the output of the fifth convolutional layer is connected to the input of the first pooling layer and the first input of the connection layer.

[0047] The output of the first pooling layer is connected to the input of the second pooling layer and the first input of the connection layer, respectively.

[0048] The output of the second pooling layer is connected to the input of the third pooling layer and the first input of the connection layer, respectively.

[0049] The output of the third pooling layer is connected to the second input of the connection layer;

[0050] The output of the connection layer is connected to the input of the LSKA mechanism layer, and the output of the LSKA mechanism layer is connected to the input of the sixth convolutional layer.

[0051] The output of the sixth convolutional layer is connected to the first input of the second scale fusion unit and the input of the third scale fusion unit, respectively.

[0052] In one possible implementation, the fusion output module includes: a fifth feature fusion layer, a seventh convolutional layer, and an activation function layer;

[0053] The input of the fifth feature fusion layer is connected to multiple outputs of the feature interaction module, and the output of the fifth feature fusion layer is connected to the input of the seventh convolutional layer.

[0054] The output of the seventh convolutional layer is connected to the input of the activation function layer, and the output of the activation function layer is used to output a dehazed image.

[0055] Secondly, embodiments of the present invention provide an image processing apparatus for power transmission lines under rainy and foggy weather conditions, comprising:

[0056] The acquisition module is used to acquire the original image corresponding to the transmission line under test;

[0057] The processing module is used to input the original image into a multi-scale feature fusion network to obtain a fog-free image output by the multi-scale feature fusion network; the fog-free image is used for image recognition to determine the fault detection result corresponding to the transmission line under test;

[0058] The multi-scale feature fusion network includes a feature extraction module, a feature interaction module, and a fusion output module. The input of the feature extraction module is used to receive the original image, and the multiple outputs of the feature extraction module are respectively connected to the multiple inputs of the feature interaction module. The multiple outputs of the feature interaction module are all connected to the fusion output module.

[0059] The feature extraction module extracts features from the original image at different scales to obtain the first feature image, the second feature image, and the third feature image, and outputs them to the feature interaction module.

[0060] The feature interaction module determines the fourth feature image based on the third feature image; determines the fifth feature image based on the third feature image and the second feature image; determines the sixth feature image based on the fifth feature image and the first feature image, and outputs the fourth feature image, the fifth feature image and the sixth feature image to the fusion output module;

[0061] The fusion output module determines the dehazed image based on the feature image output by the feature interaction module.

[0062] This invention provides a method and apparatus for image processing of transmission lines under rainy and foggy weather conditions. By using a multi-scale feature fusion network to defog the original image of the transmission line under test, a fog-free image can be generated, thereby determining the fault detection result of the transmission line under test. Specifically, the feature extraction module in the multi-scale feature fusion network can extract features from the original image at different scales to determine a first feature image, a second feature image, and a third feature image. The feature interaction module can determine a fourth feature image based on the third feature image, a fifth feature image based on the third and second feature images, and a sixth feature image based on the fifth and first feature images. This achieves information fusion between feature images at different scales, while preserving color, texture, and semantic information in the original image, thus better preserving image details and avoiding color distortion and image blurring. Furthermore, when the fusion output module determines the defog image based on the multiple feature images output by the feature interaction module, it not only avoids feature overlap and loss effects caused by the feature interaction module during information fusion, thus enhancing the defogging effect, but also aggregates rich feature information from various scales, further preserving image details and improving the fault detection accuracy of transmission lines. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 This is a flowchart illustrating the implementation of the image processing method for power transmission lines under rainy and foggy weather conditions provided in this embodiment of the invention.

[0065] Figure 2 This is a schematic diagram of the structure of the multi-scale feature fusion network provided in an embodiment of the present invention;

[0066] Figure 3 This is a schematic diagram of the structure of the convolutional attention block provided in an embodiment of the present invention;

[0067] Figure 4 This is a schematic diagram of the structure of the CBAM attention mechanism layer provided in an embodiment of the present invention;

[0068] Figure 5 This is a schematic diagram of the channel attention module provided in an embodiment of the present invention;

[0069] Figure 6 This is a schematic diagram of the spatial attention module provided in an embodiment of the present invention;

[0070] Figure 7 This is a schematic diagram of the structure of the improved SPPF submodule provided in an embodiment of the present invention;

[0071] Figure 8 This is a schematic diagram of the structure of the image processing device for power transmission lines under rainy and foggy weather conditions provided in an embodiment of the present invention. Detailed Implementation

[0072] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0073] Currently, defogging techniques for transmission line images under rainy and foggy weather conditions are mainly divided into two categories: methods that rely on prior knowledge and methods that apply deep learning.

[0074] Prior knowledge methods: These methods construct a model framework for fog imaging and use existing prior knowledge to estimate model parameters, thereby inversely generating a clear, fog-free image. For example, a fog-free image can be estimated using a dark channel prior and an atmospheric scattering model. Alternatively, a fog-free image can be recovered by estimating transmittance based on fog line prior theory.

[0075] Deep learning methods: Deep learning methods learn the transformation rules from foggy to fog-free images by training on a large amount of image data. For example, DCPPDN uses a generative adversarial network framework that simultaneously learns transport maps and atmospheric light values ​​to recover fog-free images. GFN uses a gated fusion network to recover fog-free images by taking the original blurred image and its preprocessed image as input.

[0076] Although the above methods can remove fog from transmission line images to some extent, they are prone to losing image details during the defogging process, resulting in blurry and color-distorted images, which in turn reduces the accuracy of transmission line fault detection.

[0077] To better preserve image details and reduce image blur and color distortion during dehazing, this application proposes a novel multi-scale feature fusion network, comprising a feature extraction module, a feature interaction module, and a fusion output module. The feature extraction module extracts feature images at different scales, the feature interaction module exchanges information between these feature images to obtain new feature images, and the fusion output module fuses these new feature images. Specifically, feature images at shallow scales contain rich color and texture information, while feature images at deeper scales contain rich semantic information. By exchanging information between feature images at different scales, the feature interaction module enables the generated new feature images to simultaneously retain shallow color and texture information as well as deep semantic information, thereby better preserving image details and reducing image blur and color distortion.

[0078] To make the objectives, technical solutions, and advantages of the present invention clearer, specific embodiments will be described below in conjunction with the accompanying drawings.

[0079] Figure 1 The implementation flowchart of the image processing method for power transmission lines under rainy and foggy weather conditions provided in this embodiment of the invention is described in detail below:

[0080] Step 101: Obtain the original image corresponding to the transmission line under test.

[0081] Here, the original image can be an image of the power transmission line under test acquired by an image acquisition device under rainy or foggy weather conditions.

[0082] Step 102: Input the original image into the multi-scale feature fusion network to obtain the fog-free image output by the multi-scale feature fusion network.

[0083] Considering that rainy or foggy weather can introduce fog into the original image, affecting subsequent fault detection results, this embodiment of the invention pre-inputs the original image into a multi-scale feature fusion network to perform dehazing processing, obtaining a fog-free image. Image recognition is then performed based on the fog-free image to achieve fault detection for the transmission line under test.

[0084] See Figure 2 The multi-scale feature fusion network includes a feature extraction module 21, a feature interaction module 22, and a fusion output module 23. The input of the feature extraction module 21 is used to receive the original image. The multiple outputs of the feature extraction module 21 are respectively connected to the multiple inputs of the feature interaction module 22, and the multiple outputs of the feature interaction module 22 are all connected to the fusion output module 23.

[0085] The feature extraction module 21 performs feature extraction at different scales on the original image to obtain a first feature image, a second feature image, and a third feature image, and outputs them to the feature interaction module 22.

[0086] The feature interaction module 22 determines the fourth feature image based on the third feature image; determines the fifth feature image based on the third feature image and the second feature image; determines the sixth feature image based on the fifth feature image and the first feature image, and outputs the fourth feature image, the fifth feature image and the sixth feature image to the fusion output module 23.

[0087] The fusion output module 23 determines the dehazed image based on the feature image output by the feature interaction module 22.

[0088] In this embodiment of the invention, the feature extraction module 21 performs feature extraction at three scales on the original image in sequence to obtain the first feature image, the second feature image and the third feature image in sequence.

[0089] In some embodiments, see Figure 2 The feature extraction module 21 includes: a first feature extraction unit 211, a second feature extraction unit 212 and a third feature extraction unit 213.

[0090] The first output terminal of the first feature extraction unit 211 is connected to the input terminal of the second feature extraction unit and the feature interaction module, respectively. The output terminal of the second feature extraction unit is connected to the feature interaction module of the input terminal of the third feature extraction unit. The output terminal of the third feature extraction unit is connected to the feature interaction module.

[0091] The first feature extraction unit 211 receives the original image at its input and outputs a first feature image at its output. The second feature extraction unit 212 receives the first feature image at its input and outputs a second feature image at its output. The third feature extraction unit 213 receives the second feature image at its input and outputs a third feature image at its output.

[0092] It should be noted that when the feature extraction module sequentially extracts features from the image at different scales, the shallow-scale feature extraction units extract shallow high-resolution features containing rich color and texture information. The deep-scale feature extraction units extract deep low-resolution features containing rich semantic information. The feature interaction module, by preserving the color and detail information in the high-resolution features and allowing information exchange across parallel streams, integrates high-resolution features with the help of low-resolution features, thereby better preserving image details.

[0093] In other words, the semantic information contained in the first, second, and third feature images increases sequentially. The color and texture information contained in the first, second, and third feature images decreases sequentially.

[0094] The feature interaction module determines the fifth feature image based on the third and second feature images, which can make the fifth feature image contain both the rich semantic information in the third feature image and the rich color and texture information in the second feature image.

[0095] Similarly, the feature interaction module also determines the sixth feature image based on the fifth feature image and the first feature image, so that the sixth feature image can simultaneously contain the color and texture information in the first feature image and the feature information in the fifth feature image (i.e., semantic information, color and texture information).

[0096] The feature interaction module exchanges information between the first feature image, the second feature image, and the third feature image, which enables the output fourth feature image, fifth feature image, and sixth feature image to contain more semantic information, color information, and texture information, thereby better preserving image details and avoiding image blurring and color distortion.

[0097] The fusion output module 23 can perform feature convergence, convolution, and activation processing on the fourth, fifth, and sixth feature images to generate a dehazed image.

[0098] In some embodiments, see Figure 2 The fusion output module includes: a fifth feature fusion layer, a seventh convolutional layer, and an activation function layer.

[0099] The input of the fifth feature fusion layer is connected to multiple outputs of the feature interaction module, and the output of the fifth feature fusion layer is connected to the input of the seventh convolutional layer.

[0100] The output of the seventh convolutional layer is connected to the input of the activation function layer, and the output of the activation function layer is used to output the dehazed image.

[0101] Here, the fifth feature fusion layer can perform feature fusion on the fourth, fifth, and sixth feature images output by the feature interaction module. This not only avoids the feature overlap and loss effects caused by the feature interaction module during information fusion, thus enhancing the dehazing effect, but also aggregates rich feature information at various scales to preserve image details.

[0102] In this embodiment of the invention, after feature fusion, convolution processing is performed through a seventh convolutional layer, followed by activation processing through an activation function layer to obtain a dehazed image. The dehazed image is then used for image recognition to determine the fault detection result corresponding to the transmission line under test.

[0103] For example, the seventh convolutional layer can be a 1×1 convolutional structure. The activation function of the activation function layer can be the Sigmoid activation function.

[0104] This invention employs a multi-scale feature fusion network to dehaze the original image of the transmission line under test, generating a haze-free image to determine the fault detection result of the transmission line. Specifically, the feature extraction module in the multi-scale feature fusion network extracts features from the original image at different scales, determining a first, second, and third feature image. The feature interaction module determines a fourth feature image based on the third feature image, a fifth feature image based on the third and second feature images, and a sixth feature image based on the fifth and first feature images. This achieves information fusion between feature images at different scales while preserving color, texture, and semantic information in the original image, thus better retaining image details and avoiding color distortion and image blurring. Furthermore, when the fusion output module determines the dehazed image based on the multiple feature images output by the feature interaction module, it not only avoids feature overlap and loss effects caused by the feature interaction module during information fusion, enhancing the dehazing effect, but also aggregates rich feature information from various scales, further preserving image details.

[0105] In some embodiments, see Figure 2 The first feature extraction unit 211 is also provided with a second output terminal for outputting intermediate feature variables.

[0106] The feature interaction module 22 includes: a first scale fusion unit 221, a second scale fusion unit 222 and a third scale fusion unit 223.

[0107] The input of the third scale fusion unit 223 is connected to the output of the third feature extraction unit 213, and the output of the third scale fusion unit 223 is connected to the input of the fusion output module 23. It is used to perform convolution processing on the third feature image P3 to obtain the fourth feature image P4, and input it to the fusion output module 23.

[0108] Here, the third scale fusion unit can use a 1×1 convolution structure to convolve the third feature image to obtain the fourth feature image.

[0109] The first input terminal of the second scale fusion unit 222 is connected to the output terminal of the third feature extraction unit 213, the second input terminal of the second scale fusion unit 222 is connected to the output terminal of the second feature extraction unit 212, and the output terminal of the second scale fusion unit 222 is connected to the first input terminal of the first scale fusion unit 221 and the input terminal of the fusion output module 23, respectively, for processing the third feature image P3 and the second feature image P2 to obtain the fifth feature image P5, which is then input to the first scale fusion unit 221 and the fusion output module 23, respectively.

[0110] The first input terminal of the first scale fusion unit 221 is connected to the output terminal of the second scale fusion unit 222. The second input terminal of the first scale fusion unit 221 is connected to the first output terminal of the first feature extraction unit 211. The third input terminal of the first scale fusion unit 221 is connected to the second output terminal of the first feature extraction unit 211. The output terminal of the first scale fusion unit 221 is connected to the input terminal of the fusion output module 23. The first feature image P5, the first feature image P1, and the intermediate feature variables are processed to obtain the sixth feature image P6, which is then input to the fusion output module.

[0111] In this embodiment of the invention, features are extracted sequentially using a first feature extraction unit, a second feature extraction unit, and a third feature extraction unit to achieve a top-down feature pyramid. Simultaneously, a bottom-up information fusion is performed using a feature interaction module, thereby preserving rich color and texture information, as well as semantic information.

[0112] In some embodiments, see Figure 2 The second scale fusion unit includes: a first upsampling layer, a first feature fusion layer, and a first convolutional layer.

[0113] The input of the first upsampling layer is connected to the output of the third feature extraction unit, and the output of the first upsampling layer is connected to the input of the first feature fusion layer.

[0114] The input of the first feature fusion layer is also connected to the output of the second feature extraction unit, and the output of the first feature fusion layer is connected to the input of the first convolutional layer.

[0115] The output of the first convolutional layer is connected to the first input of the first scale fusion unit and the input of the fusion output module, respectively.

[0116] Here, the first sampling layer performs an upsampling operation, the first feature fusion layer performs a feature fusion operation, and the first convolutional layer performs a convolution operation. The second scale fusion unit first performs an upsampling operation on the third feature image output by the third feature extraction unit, then performs feature fusion on the upsampled feature image and the second feature image, and finally performs convolution processing on the feature image after feature fusion to obtain the fifth feature image. For example, the first convolutional layer can be a 3×3 convolutional structure.

[0117] In some embodiments, see Figure 2 The first scale fusion unit includes: a second upsampling layer, a second feature fusion layer, a second convolutional layer, a first downsampling layer, and a first feature extraction layer.

[0118] The input of the second upsampling layer is connected to the output of the second scale fusion unit, and the output of the second upsampling layer is connected to the input of the second feature fusion layer.

[0119] The input of the second feature fusion layer is also connected to the first output of the first feature extraction unit, and the output of the second feature fusion layer is connected to the input of the second convolutional layer.

[0120] The input of the second convolutional layer is also connected to the second output of the first feature extraction unit, and the output of the second convolutional layer is connected to the input of the first downsampling layer.

[0121] The output of the first downsampling layer is connected to the input of the first feature extraction layer;

[0122] The output of the first feature extraction layer is connected to the input of the fusion output module.

[0123] The first scale fusion unit first upsamples the fifth feature image P5, and then fuses the upsampled feature image with the first feature image P1. Next, it performs convolution processing on the fused feature image and the intermediate feature variables output by the first feature extraction unit. Subsequently, it downsamples and extracts features from the convolutional feature image to generate the sixth feature image P6. For example, the second convolutional layer can be a 3×3 convolutional structure.

[0124] In some embodiments, see Figure 2 The first feature extraction unit includes: a second feature extraction layer and a second downsampling layer;

[0125] The input of the second feature extraction layer is used to receive the original image, and the output is connected to the input of the second downsampling layer and the third input of the first scale fusion unit, respectively.

[0126] The output of the second downsampling layer is connected to the input of the second feature extraction unit and the second input of the first scale fusion unit, respectively.

[0127] The second feature extraction unit is used to extract features from the original image, obtain intermediate feature variables, and perform downsampling processing on the intermediate feature variables to obtain the first feature image P1.

[0128] The second feature extraction layer comprises a third convolutional layer and a convolutional attention block connected in sequence. The third convolutional layer performs preliminary feature extraction on the original image through convolution operations. Based on this, the convolutional attention block further processes the preliminary feature extraction results, making the output intermediate feature variables richer and more refined in texture and detail. For example, the third convolutional layer can be a 3×3 convolutional structure.

[0129] See Figure 3 The convolutional attention block includes: a spatial depth transformation convolutional layer, a third feature fusion layer, a detail enhancement convolutional layer, a fourth convolutional layer, a CBAM attention mechanism layer, and a fourth feature fusion layer;

[0130] The inputs of the spatial depth transformation convolutional layer and the third feature fusion layer are both connected to the third convolutional layer;

[0131] The input of the third feature fusion layer is also connected to the output of the spatial depth transformation convolutional layer, and the output of the third feature fusion layer is connected to the input of the detail enhancement convolutional layer.

[0132] The output of the detail enhancement convolutional layer is connected to the input of the fourth convolutional layer and the input of the fourth feature fusion layer, respectively; here, the fourth convolutional layer can be a 3×3 convolutional structure.

[0133] The output of the fourth convolutional layer 34 is connected to the input of the CBAM attention mechanism layer 35, and the output of the CBAM attention mechanism layer 35 is connected to the input of the fourth feature fusion layer 36.

[0134] The output of the fourth feature fusion layer 36 is connected to the input of the second downsampling layer and the third input of the first scale fusion unit.

[0135] The following sections will introduce the detail enhancement convolutional layer, the spatial depth transformation convolutional layer, and the CBAM attention mechanism layer in turn.

[0136] Detail-Enhanced Convolution (DEConv) layers can enhance the expressiveness and adaptability of a model by integrating prior knowledge into traditional convolutions.

[0137] Images captured in rainy or foggy conditions often suffer from low visibility, unclear details, and blurred outlines of small objects, and also contain a large number of redundant pixels. Traditional convolutional neural networks typically use strided convolution and pooling operations to reduce the resolution of feature maps and thus reduce computational cost. However, these operations may lose important spatial information, especially in rainy or foggy conditions, where the loss of detail can affect dehazing results. To address this issue, this invention introduces a Space-to-Depth Convolution (SPD-Conv) layer.

[0138] SPD-Conv, by replacing traditional strided convolution and pooling operations, can more effectively handle blur and small objects in foggy images without significantly increasing computational resource requirements. Its working principle is to use a pyramid structure for multi-scale image analysis, while simultaneously using differential convolution to capture local features and enhance the ability to perceive details. This method not only optimizes the network structure but also improves the model's performance and robustness when processing foggy images.

[0139] Specifically, SPD-Conv is achieved through the following steps:

[0140] Multi-scale analysis: Decompose the input image into feature maps of different scales, each scale corresponding to a different resolution.

[0141] Differential convolution: Applying differential convolution operations at each scale captures local changes in an image and enhances the perception of details.

[0142] Feature fusion: fusing feature maps of different scales to retain rich details and semantic information.

[0143] This design not only reduces the reliance on computing resources, but also improves the model's real-time performance when processing foggy images, making it more suitable for real-time inspection and rapid response applications.

[0144] See Figure 4 The CBAM (Convolutional Block Attention Module) attention mechanism layer comprises two parts: a channel attention module and a spatial attention module. These modules emphasize channel and spatial features respectively, thus providing a more comprehensive feature extraction capability. This mechanism assigns a unique spatial importance mapping to each channel, enhancing the model's perception of image features.

[0145] The CBAM workflow is as follows: First, it receives the input feature map and then processes it through the channel attention module. In the channel attention module, the input feature map undergoes a one-dimensional convolution operation to generate a channel attention weight. This weight is then used to adjust the channel response of the original feature map, enhancing important features and suppressing unimportant features. Next, the processed feature map is fed into the spatial attention module. In the spatial attention module, the feature map undergoes a two-dimensional convolution to generate a spatial attention weight. This weight is used to adjust the spatial response of the feature map, further highlighting salient regions and reducing the impact of background noise. Finally, the feature map processed by the dual channel and spatial attention mechanisms not only retains key information but also improves the expressive power of the features, thus providing richer feature support for subsequent network tasks.

[0146] See Figure 5 After the feature maps enter the channel attention module, they first undergo two parallel aggregation operations: GlobalMaxPool and GlobalAvgPool. These operations transform the original C×H×W dimensional feature maps into a C×1×1 size, aggregating the features of each channel into a single value. These aggregated features are then fed into a shared multilayer perceptron module. In this module, the feature maps undergo two linear transformations: first, the number of channels is compressed to 1 / r of the original value, and then the number of channels is expanded back to the original value. During this process, the features are non-linearly processed using a ReLU activation function.

[0147] The two feature maps obtained through these two steps are summed element-wise to integrate information from different pooling methods. Finally, this synthesized feature map is activated using the sigmoid function to generate attention weights for each channel. These weights are then used to adjust the channel responses of the original feature map by multiplying the attention weights with the original feature map, thereby weighting the channel dimensions and restoring the original C×H×W size.

[0148] See Figure 6The spatial attention module of CBAM aims to maintain the spatial resolution of the feature maps while optimizing the channel dimensions. This process begins with the output of the channel attention module, where max pooling and average pooling operations compress each channel of the feature map into a single spatial dimension, generating two 1×H×W feature maps. These two feature maps are then concatenated to form a comprehensive feature representation. This concatenated feature map is convolved with a 7×7 kernel, a crucial step that allows the network to capture important locational information over a wider spatial range. Experimental results show that the 7×7 kernel is more effective than the 3×3 kernel in capturing spatial information. The output of the convolution is processed by a sigmoid function to generate a spatial attention weight map, which highlights the importance of each spatial location in the feature map. Finally, this spatial attention weight map is multiplied by the original feature map to restore its original C×H×W dimensions. In this way, each channel of the feature map is enhanced or suppressed according to its spatial importance, enabling the network to more accurately locate and identify targets.

[0149] In addition, a residual connection mechanism is set in the convolutional attention module. That is, the input end of the spatial depth transformation convolutional layer and the input end of the third feature fusion layer are both connected to the third convolutional layer. The input end of the third feature fusion layer 32 is also connected to the output end of the spatial depth transformation convolutional layer 31. This residual connection mechanism can avoid the loss of information during gradient propagation, thereby accelerating the convergence speed of the model.

[0150] The convolutional attention module further processes the preliminary feature extraction results output by the third convolutional layer by employing different convolutional structures and CBAM attention mechanism layers. This makes the intermediate feature variables output richer and more refined in texture and detail, thereby improving the image dehazing effect and preserving more local details.

[0151] In this embodiment of the invention, the internal connection structure of the second feature extraction unit is the same as that of the first feature extraction unit. For details, please refer to the description of the first feature extraction unit, which will not be repeated here.

[0152] In some embodiments, see Figure 2 The third feature extraction unit includes: a third feature extraction layer and an improved SPPF submodule;

[0153] The input of the third feature extraction layer is connected to the output of the second feature extraction unit;

[0154] The output of the third feature extraction layer is connected to the input of the improved SPPF submodule, and the output of the improved SPPF submodule is connected to the first input of the second scale fusion unit and the input of the third scale fusion unit, respectively.

[0155] In this embodiment of the invention, the internal connection structures of the first feature extraction layer, the third feature extraction layer and the second feature extraction layer are the same. For details, please refer to the description of the second feature extraction layer, which will not be repeated here.

[0156] In some embodiments, see Figure 7 The improved SPPF sub-modules include: a fifth convolutional layer, a first pooling layer, a second pooling layer, a third pooling layer, a connection layer, an LSKA mechanism layer, and a sixth convolutional layer.

[0157] The input of the fifth convolutional layer is connected to the output of the third feature extraction layer, and the output of the fifth convolutional layer is connected to the input of the first pooling layer and the first input of the connection layer.

[0158] The output of the first pooling layer is connected to the input of the second pooling layer and the first input of the connection layer, respectively.

[0159] The output of the second pooling layer is connected to the input of the third pooling layer and the first input of the connection layer, respectively.

[0160] The output of the third pooling layer is connected to the second input of the connection layer;

[0161] The output of the connection layer is connected to the input of the LSKA mechanism layer, and the output of the LSKA mechanism layer is connected to the input of the sixth convolutional layer.

[0162] The output of the sixth convolutional layer is connected to the first input of the second scale fusion unit and the input of the third scale fusion unit, respectively.

[0163] This invention improves the SPPF submodule by introducing the LSKA (Large Separable Kernel Attention) mechanism layer into the SPPF (Spatial Pyramid Pooling) module. In this improved SPPF submodule, input data is first processed through a first pooling layer, a second pooling layer, and a third pooling layer. The outputs of each layer are concatenated in parallel to form a multi-scale feature set. To further optimize this process, the LSKA mechanism is introduced. It splits the traditional k×k convolutional kernel into 1×k and k×1 separable convolutional kernels, processing the input in a cascaded manner, effectively reducing the parameter inflation problem caused by large convolutional kernels.

[0164] The first, second, and third pooling layers can be 5x5 in size. The LSKA mechanism layers can contain large separable convolutional kernels, such as 11x11 kernels.

[0165] In the LSKA mechanism layer, large separable convolutional kernels are used to capture long-range dependencies, providing a larger receptive field and thus enhancing the feature representation capability. Subsequently, these features are further fused through standard convolutional operations to optimize the output feature vector of the model's backbone network.

[0166] This approach not only retains SPPF's advantages in multi-scale feature extraction but also enhances the richness and effectiveness of features through the LSKA mechanism, thereby improving the overall model's ability in feature fusion and representation, while the increase in parameters is relatively small. This fusion strategy enables the model to more effectively handle complex visual tasks, capturing and integrating key information from different scales and spatial locations.

[0167] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0168] The following are device embodiments of the present invention. For details not described in detail, please refer to the corresponding method embodiments described above.

[0169] Figure 8 A schematic diagram of the image processing device for power transmission lines under rainy and foggy weather conditions provided in an embodiment of the present invention is shown. For ease of explanation, only the parts related to the embodiment of the present invention are shown, and are described in detail below:

[0170] like Figure 8 As shown, the image processing device 8 for power transmission lines under rainy and foggy weather conditions includes: an acquisition module 81 and a processing module 82.

[0171] The acquisition module 81 is used to acquire the original image corresponding to the transmission line under test;

[0172] Processing module 82 is used to input the original image into a multi-scale feature fusion network to obtain a fog-free image output by the multi-scale feature fusion network; the fog-free image is used for image recognition to determine the fault detection result corresponding to the transmission line under test;

[0173] The multi-scale feature fusion network includes a feature extraction module, a feature interaction module, and a fusion output module. The input of the feature extraction module is used to receive the original image, and the multiple outputs of the feature extraction module are respectively connected to the multiple inputs of the feature interaction module. The multiple outputs of the feature interaction module are all connected to the fusion output module.

[0174] The feature extraction module extracts features from the original image at different scales to obtain the first feature image, the second feature image, and the third feature image, and outputs them to the feature interaction module.

[0175] The feature interaction module determines the fourth feature image based on the third feature image; determines the fifth feature image based on the third feature image and the second feature image; determines the sixth feature image based on the fifth feature image and the first feature image, and outputs the fourth feature image, the fifth feature image and the sixth feature image to the fusion output module;

[0176] The fusion output module determines the dehazed image based on the feature image output by the feature interaction module.

[0177] Optionally, the feature extraction module includes: a first feature extraction unit, a second feature extraction unit, and a third feature extraction unit;

[0178] The input end of the first feature extraction unit is used to receive the original image, the first output end is used to output the first feature image, and the second output end is used to output the intermediate feature variables.

[0179] The input end of the second feature extraction unit is used to receive the first feature image, and the output end is used to output the second feature image;

[0180] The input end of the third feature extraction unit is used to receive the second feature image, and the output end is used to output the third feature image;

[0181] The feature interaction module includes: a first-scale fusion unit, a second-scale fusion unit, and a third-scale fusion unit;

[0182] The input of the third scale fusion unit is connected to the output of the third feature extraction unit, and the output of the third scale fusion unit is connected to the input of the fusion output module. It is used to perform convolution processing on the third feature image to obtain the fourth feature image, which is then input to the fusion output module.

[0183] The first input of the second scale fusion unit is connected to the output of the third feature extraction unit, the second input of the second scale fusion unit is connected to the output of the second feature extraction unit, and the output of the second scale fusion unit is connected to the first input of the first scale fusion unit and the input of the fusion output module, respectively, to process the third feature image and the second feature image to obtain the fifth feature image, which is then input to the first scale fusion unit and the fusion output module, respectively.

[0184] The first input terminal of the first scale fusion unit is connected to the output terminal of the second scale fusion unit, the second input terminal of the first scale fusion unit is connected to the first output terminal of the first feature extraction unit, the third input terminal of the first scale fusion unit is connected to the second output terminal of the first feature extraction unit, and the output terminal of the first scale fusion unit is connected to the input terminal of the fusion output module. This is used to process the fifth feature image, the first feature image, and the intermediate feature variables to obtain the sixth feature image, which is then input to the fusion output module.

[0185] Optionally, the second scale fusion unit includes: a first upsampling layer, a first feature fusion layer, and a first convolutional layer;

[0186] The input of the first upsampling layer is connected to the output of the third feature extraction unit, and the output of the first upsampling layer is connected to the input of the first feature fusion layer.

[0187] The input of the first feature fusion layer is also connected to the output of the second feature extraction unit, and the output of the first feature fusion layer is connected to the input of the first convolutional layer.

[0188] The output of the first convolutional layer is connected to the first input of the first scale fusion unit and the input of the fusion output module, respectively.

[0189] Optionally, the first scale fusion unit includes: a second upsampling layer, a second feature fusion layer, a second convolutional layer, a first downsampling layer, and a first feature extraction layer;

[0190] The input of the second upsampling layer is connected to the output of the second scale fusion unit, and the output of the second upsampling layer is connected to the input of the second feature fusion layer.

[0191] The input of the second feature fusion layer is also connected to the first output of the first feature extraction unit, and the output of the second feature fusion layer is connected to the input of the second convolutional layer.

[0192] The input of the second convolutional layer is also connected to the second output of the first feature extraction unit, and the output of the second convolutional layer is connected to the input of the first downsampling layer.

[0193] The output of the first downsampling layer is connected to the input of the first feature extraction layer;

[0194] The output of the first feature extraction layer is connected to the input of the fusion output module.

[0195] Optionally, the first feature extraction unit includes: a second feature extraction layer and a second downsampling layer;

[0196] The input of the second feature extraction layer is used to receive the original image, and the output is connected to the input of the second downsampling layer and the third input of the first scale fusion unit, respectively.

[0197] The output of the second downsampling layer is connected to the input of the second feature extraction unit and the second input of the first scale fusion unit, respectively.

[0198] Optionally, the second feature extraction layer includes a third convolutional layer and a convolutional attention block connected in sequence;

[0199] The convolutional attention block includes: a spatial depth transformation convolutional layer, a third feature fusion layer, a detail enhancement convolutional layer, a fourth convolutional layer, a CBAM attention mechanism layer, and a fourth feature fusion layer;

[0200] The inputs of the spatial depth transformation convolutional layer and the third feature fusion layer are both connected to the third convolutional layer;

[0201] The input of the third feature fusion layer is also connected to the output of the spatial depth transformation convolutional layer, and the output of the third feature fusion layer is connected to the input of the detail enhancement convolutional layer.

[0202] The output of the detail enhancement convolutional layer is connected to the input of the fourth convolutional layer and the input of the fourth feature fusion layer, respectively.

[0203] The output of the fourth convolutional layer is connected to the input of the CBAM attention mechanism layer, and the output of the CBAM attention mechanism layer is connected to the input of the fourth feature fusion layer.

[0204] The output of the fourth feature fusion layer is connected to the input of the second downsampling layer and the third input of the first scale fusion unit.

[0205] Optionally, the third feature extraction unit includes: a third feature extraction layer and an improved SPPF submodule;

[0206] The input of the third feature extraction layer is connected to the output of the second feature extraction unit;

[0207] The output of the third feature extraction layer is connected to the input of the improved SPPF submodule, and the output of the improved SPPF submodule is connected to the first input of the second scale fusion unit and the input of the third scale fusion unit, respectively.

[0208] Optional, the improved SPPF sub-module includes: a fifth convolutional layer, a first pooling layer, a second pooling layer, a third pooling layer, a connection layer, an LSKA mechanism layer, and a sixth convolutional layer;

[0209] The input of the fifth convolutional layer is connected to the output of the third feature extraction layer, and the output of the fifth convolutional layer is connected to the input of the first pooling layer and the first input of the connection layer.

[0210] The output of the first pooling layer is connected to the input of the second pooling layer and the first input of the connection layer, respectively.

[0211] The output of the second pooling layer is connected to the input of the third pooling layer and the first input of the connection layer, respectively.

[0212] The output of the third pooling layer is connected to the second input of the connection layer;

[0213] The output of the connection layer is connected to the input of the LSKA mechanism layer, and the output of the LSKA mechanism layer is connected to the input of the sixth convolutional layer.

[0214] The output of the sixth convolutional layer is connected to the first input of the second scale fusion unit and the input of the third scale fusion unit, respectively.

[0215] Optionally, the fusion output module includes: a fifth feature fusion layer, a seventh convolutional layer, and an activation function layer;

[0216] The input of the fifth feature fusion layer is connected to multiple outputs of the feature interaction module, and the output of the fifth feature fusion layer is connected to the input of the seventh convolutional layer.

[0217] The output of the seventh convolutional layer is connected to the input of the activation function layer, and the output of the activation function layer is used to output the dehazed image.

[0218] The image processing device for power transmission lines under rainy and foggy weather conditions provided in this embodiment of the invention can be used to implement the above-mentioned image processing method for power transmission lines under rainy and foggy weather conditions. Its technical principle and implementation effect are the same as those of the above-mentioned method embodiments, and will not be repeated here.

[0219] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0220] Those skilled in the art will recognize that the templates, units, and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0221] If the module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the above embodiments of the transmission line image processing method under rainy and foggy weather conditions. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory, a random access memory, an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0222] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for image processing of power transmission lines under rainy and foggy weather conditions, characterized in that, include: Acquire the original image corresponding to the transmission line under test; The original image is input into a multi-scale feature fusion network to obtain a fog-free image output by the multi-scale feature fusion network; The fog-free image is used for image recognition to determine the fault detection result corresponding to the transmission line under test. The multi-scale feature fusion network includes a feature extraction module, a feature interaction module, and a fusion output module. The input of the feature extraction module is used to receive the original image, and the multiple outputs of the feature extraction module are respectively connected to the multiple inputs of the feature interaction module. The multiple outputs of the feature interaction module are all connected to the fusion output module. The feature extraction module extracts features from the original image at different scales to obtain the first feature image, the second feature image, and the third feature image, and outputs them to the feature interaction module. The feature interaction module determines the fourth feature image based on the third feature image; determines the fifth feature image based on the third feature image and the second feature image; determines the sixth feature image based on the fifth feature image and the first feature image, and outputs the fourth feature image, the fifth feature image and the sixth feature image to the fusion output module; The fusion output module determines the dehazed image based on the feature image output by the feature interaction module; The feature extraction module includes: a first feature extraction unit, a second feature extraction unit, and a third feature extraction unit; The feature interaction module includes: a first scale fusion unit, a second scale fusion unit, and a third scale fusion unit; The first feature extraction unit includes: a second feature extraction layer and a second downsampling layer; The second feature extraction layer comprises a third convolutional layer and a convolutional attention block connected in sequence; The convolutional attention block includes: a spatial depth transformation convolutional layer, a third feature fusion layer, a detail enhancement convolutional layer, a fourth convolutional layer, a CBAM attention mechanism layer, and a fourth feature fusion layer; The input of the spatial depth transformation convolutional layer and the input of the third feature fusion layer are both connected to the third convolutional layer; The input of the third feature fusion layer is also connected to the output of the spatial depth transformation convolutional layer, and the output of the third feature fusion layer is connected to the input of the detail enhancement convolutional layer. The output of the detail enhancement convolutional layer is connected to the input of the fourth convolutional layer and the input of the fourth feature fusion layer, respectively. The output of the fourth convolutional layer is connected to the input of the CBAM attention mechanism layer, and the output of the CBAM attention mechanism layer is connected to the input of the fourth feature fusion layer. The output of the fourth feature fusion layer is connected to the input of the second downsampling layer and the third input of the first scale fusion unit.

2. The image processing method for power transmission lines under rainy and foggy weather conditions according to claim 1, characterized in that, The input end of the first feature extraction unit is used to receive the original image, the first output end is used to output the first feature image, and the second output end is used to output the intermediate feature variables. The input terminal of the second feature extraction unit is used to receive the first feature image, and the output terminal is used to output the second feature image; The input end of the third feature extraction unit is used to receive the second feature image, and the output end is used to output the third feature image; The input of the third scale fusion unit is connected to the output of the third feature extraction unit, and the output of the third scale fusion unit is connected to the input of the fusion output module. The third feature image is convolved to obtain a fourth feature image, which is then input to the fusion output module. The first input terminal of the second scale fusion unit is connected to the output terminal of the third feature extraction unit, the second input terminal of the second scale fusion unit is connected to the output terminal of the second feature extraction unit, and the output terminal of the second scale fusion unit is connected to the first input terminal of the first scale fusion unit and the input terminal of the fusion output module, respectively, for processing the third feature image and the second feature image to obtain a fifth feature image, which is then input to the first scale fusion unit and the fusion output module, respectively. The first input terminal of the first scale fusion unit is connected to the output terminal of the second scale fusion unit, the second input terminal of the first scale fusion unit is connected to the first output terminal of the first feature extraction unit, the third input terminal of the first scale fusion unit is connected to the second output terminal of the first feature extraction unit, and the output terminal of the first scale fusion unit is connected to the input terminal of the fusion output module. This is used to process the fifth feature image, the first feature image, and the intermediate feature variables to obtain a sixth feature image, which is then input to the fusion output module.

3. The image processing method for power transmission lines under rainy and foggy weather conditions according to claim 2, characterized in that, The second scale fusion unit includes: a first upsampling layer, a first feature fusion layer, and a first convolutional layer; The input of the first upsampling layer is connected to the output of the third feature extraction unit, and the output of the first upsampling layer is connected to the input of the first feature fusion layer. The input of the first feature fusion layer is also connected to the output of the second feature extraction unit, and the output of the first feature fusion layer is connected to the input of the first convolutional layer. The output of the first convolutional layer is connected to the first input of the first scale fusion unit and the input of the fusion output module, respectively.

4. The image processing method for transmission lines under rainy and foggy weather conditions according to claim 2 or 3, characterized in that, The first scale fusion unit includes: a second upsampling layer, a second feature fusion layer, a second convolutional layer, a first downsampling layer, and a first feature extraction layer; The input of the second upsampling layer is connected to the output of the second scale fusion unit, and the output of the second upsampling layer is connected to the input of the second feature fusion layer. The input of the second feature fusion layer is also connected to the first output of the first feature extraction unit, and the output of the second feature fusion layer is connected to the input of the second convolutional layer. The input of the second convolutional layer is also connected to the second output of the first feature extraction unit, and the output of the second convolutional layer is connected to the input of the first downsampling layer. The output of the first downsampling layer is connected to the input of the first feature extraction layer; The output of the first feature extraction layer is connected to the input of the fusion output module.

5. The image processing method for transmission lines under rainy and foggy weather conditions according to claim 2 or 3, characterized in that, The input end of the second feature extraction layer is used to receive the original image, and the output end is connected to the input end of the second downsampling layer and the third input end of the first scale fusion unit, respectively. The output of the second downsampling layer is connected to the input of the second feature extraction unit and the second input of the first scale fusion unit.

6. The image processing method for transmission lines under rainy and foggy weather conditions according to claim 2 or 3, characterized in that, The third feature extraction unit includes: a third feature extraction layer and an improved SPPF submodule; The input end of the third feature extraction layer is connected to the output end of the second feature extraction unit; The output of the third feature extraction layer is connected to the input of the improved SPPF submodule, and the output of the improved SPPF submodule is connected to the first input of the second scale fusion unit and the input of the third scale fusion unit, respectively.

7. The image processing method for transmission lines under rainy and foggy weather conditions according to claim 6, characterized in that, The improved SPPF submodule includes: a fifth convolutional layer, a first pooling layer, a second pooling layer, a third pooling layer, a connection layer, an LSKA mechanism layer, and a sixth convolutional layer; The input of the fifth convolutional layer is connected to the output of the third feature extraction layer, and the output of the fifth convolutional layer is connected to the input of the first pooling layer and the first input of the connection layer. The output of the first pooling layer is connected to the input of the second pooling layer and the first input of the connection layer, respectively. The output of the second pooling layer is connected to the input of the third pooling layer and the first input of the connection layer, respectively. The output of the third pooling layer is connected to the second input of the connection layer; The output of the connection layer is connected to the input of the LSKA mechanism layer, and the output of the LSKA mechanism layer is connected to the input of the sixth convolutional layer. The output of the sixth convolutional layer is connected to the first input of the second scale fusion unit and the input of the third scale fusion unit, respectively.

8. The image processing method for transmission lines under rainy and foggy weather conditions according to any one of claims 1-3, characterized in that, The fusion output module includes: a fifth feature fusion layer, a seventh convolutional layer, and an activation function layer; The input of the fifth feature fusion layer is connected to multiple outputs of the feature interaction module, and the output of the fifth feature fusion layer is connected to the input of the seventh convolutional layer. The output of the seventh convolutional layer is connected to the input of the activation function layer, and the output of the activation function layer is used to output a dehazed image.

9. An image processing device for power transmission lines under rainy and foggy weather conditions, characterized in that, The apparatus for implementing the image processing method for power transmission lines under rainy and foggy weather conditions according to any one of claims 1-8 comprises: The acquisition module is used to acquire the original image corresponding to the transmission line under test; The processing module is used to input the original image into a multi-scale feature fusion network to obtain a fog-free image output by the multi-scale feature fusion network; the fog-free image is used for image recognition to determine the fault detection result corresponding to the transmission line under test; The multi-scale feature fusion network includes a feature extraction module, a feature interaction module, and a fusion output module. The input of the feature extraction module is used to receive the original image, and the multiple outputs of the feature extraction module are respectively connected to the multiple inputs of the feature interaction module. The multiple outputs of the feature interaction module are all connected to the fusion output module. The feature extraction module extracts features from the original image at different scales to obtain the first feature image, the second feature image, and the third feature image, and outputs them to the feature interaction module. The feature interaction module determines the fourth feature image based on the third feature image; determines the fifth feature image based on the third feature image and the second feature image; determines the sixth feature image based on the fifth feature image and the first feature image, and outputs the fourth feature image, the fifth feature image and the sixth feature image to the fusion output module; The fusion output module determines the dehazed image based on the feature image output by the feature interaction module.

Images