Power transmission line image processing method and device under rainy and foggy weather conditions

Abstract

The invention provides a power transmission line image processing method and device under rainy and foggy weather conditions, and belongs to the field of image processing. The method comprises the following steps: acquiring an original image corresponding to a to-be-detected power transmission line; inputting the original image into a multi-scale feature fusion network to obtain a fogless image output by the multi-scale feature fusion network; the fogless image is used for image recognition to determine a fault detection result corresponding to the to-be-detected power transmission line; the multi-scale feature fusion network comprises a feature extraction module, a feature interaction module and a fusion output module. The input end of the feature extraction module is used for receiving an original image, the multiple output ends of the feature extraction module are connected with the multiple input ends of the feature interaction module respectively, and the multiple output ends of the feature interaction module are connected with the fusion output module. According to the invention, image detail information can be better obtained, and the fault detection precision of the power transmission line is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and device for processing images of power transmission lines under rainy and foggy weather conditions. Background Art

[0002] With the continuous increase in the mileage of power transmission lines in my country, coupled with the influence of complex geographical terrain and climate environment, the traditional manual line inspection method can no longer meet the current power transmission line inspection needs. For this reason, the high-efficiency and high-accuracy drone inspection method has emerged. Drone inspection can detect power transmission line faults by performing image recognition on the transmission line images collected by drones.

[0003] However, under rainy and foggy weather conditions, fog is prone to appear in the transmission line images collected by drones, which reduces the image quality and thus reduces the fault detection accuracy of the transmission line. Therefore, it is necessary to perform defogging on the transmission line images under rainy and foggy weather to improve the image quality and ensure the fault detection accuracy of the transmission line.

[0004] In related technologies, a model framework for foggy imaging is usually constructed, and the model parameters are estimated using existing prior knowledge, so as to reversely generate a clear fog-free image. For example, the fog-free image is estimated by using the dark channel prior and the atmospheric scattering model, or the fog-free image is restored by estimating the transmittance based on the fog line prior theory.

[0005] Although the above method can remove fog in the transmission line image to a certain extent, it is easy to lose image detail information during the defogging process, resulting in blurriness and color distortion in the defogged image, which in turn reduces the fault detection accuracy of the transmission line. Summary of the invention

[0006] The embodiments of the present invention provide a method and device for processing images of power transmission lines under rainy and foggy weather conditions, so as to solve the problem that the existing image defogging processing method is prone to lose image detail information, thereby reducing the fault detection accuracy of the power transmission line.

[0007] In a first aspect, an embodiment of the present invention provides a method for processing a power transmission line image under rainy and foggy weather conditions, comprising:

[0008] Obtaining an original image corresponding to the transmission line to be tested;

[0009] Inputting the original image into a multi-scale feature fusion network to obtain a fog-free image output by the multi-scale feature fusion network; the fog-free image is used for image recognition to determine a fault detection result corresponding to the power transmission line to be tested;

[0010] The multi-scale feature fusion network includes: a feature extraction module, a feature interaction module and a fusion output module; the input end of the feature extraction module is used to receive the original image, the multiple output ends of the feature extraction module are respectively connected to the multiple input ends of the feature interaction module, and the multiple output ends of the feature interaction module are all connected to the fusion output module;

[0011] The feature extraction module extracts features of different scales from the original image to obtain a first feature image, a second feature image and a third feature image, and outputs the images to the feature interaction module;

[0012] The feature interaction module determines a fourth feature image according to the third feature image; determines a fifth feature image according to the third feature image and the second feature image; determines a sixth feature image according to the fifth feature image and the first feature image, and outputs the fourth feature image, the fifth feature image and the sixth feature image to the fusion output module;

[0013] The fusion output module determines the dehazed image based on the feature image output by the feature interaction module.

[0014] In a possible implementation, the feature extraction module includes: a first feature extraction unit, a second feature extraction unit and a third feature extraction unit;

[0015] The input end of the first feature extraction unit is used to receive the original image, the first output end is used to output the first feature image, and the second output end is used to output the intermediate feature variable;

[0016] The input end of the second feature extraction unit is used to receive the first feature image, and the output end is used to output the second feature image;

[0017] The input end of the third feature extraction unit is used to receive the second feature image, and the output end is used to output the third feature image;

[0018] The feature interaction module includes: a first scale fusion unit, a second scale fusion unit and a third scale fusion unit;

[0019] The input end of the third scale fusion unit is connected to the output end of the third feature extraction unit, and the output end of the third scale fusion unit is connected to the input end of the fusion output module, so as to perform convolution processing on the third feature image to obtain a fourth feature image, and input the fourth feature image into the fusion output module;

[0020] The first input end of the second scale fusion unit is connected to the output end of the third feature extraction unit, the second input end of the second scale fusion unit is connected to the output end of the second feature extraction unit, and the output end of the second scale fusion unit is respectively connected to the first input end of the first scale fusion unit and the input end of the fusion output module, for processing the third feature image and the second feature image to obtain a fifth feature image, and respectively inputting the fifth feature image to the first scale fusion unit and the fusion output module;

[0021] A first input end of the first scale fusion unit is connected to an output end of the second scale fusion unit, a second input end of the first scale fusion unit is connected to a first output end of the first feature extraction unit, a third input end of the first scale fusion unit is connected to a second output end of the first feature extraction unit, and an output end of the first scale fusion unit is connected to an input end of the fusion output module, for processing the fifth feature image, the first feature image, and the intermediate feature variable to obtain a sixth feature image, and input the sixth feature image into the fusion output module.

[0022] In a possible implementation, the second scale fusion unit includes: a first upsampling layer, a first feature fusion layer, and a first convolution layer;

[0023] The input end of the first upsampling layer is connected to the output end of the third feature extraction unit, and the output end of the first upsampling layer is connected to the input end of the first feature fusion layer;

[0024] The input end of the first feature fusion layer is also connected to the output end of the second feature extraction unit, and the output end of the first feature fusion layer is connected to the input end of the first convolutional layer;

[0025] The output end of the first convolutional layer is respectively connected to the first input end of the first scale fusion unit and the input end of the fusion output module.

[0026] In a possible implementation, the first scale fusion unit includes: a second upsampling layer, a second feature fusion layer, a second convolution layer, a first downsampling layer, and a first feature extraction layer;

[0027] An input end of the second upsampling layer is connected to an output end of the second scale fusion unit, and an output end of the second upsampling layer is connected to an input end of the second feature fusion layer;

[0028] The input end of the second feature fusion layer is also connected to the first output end of the first feature extraction unit, and the output end of the second feature fusion layer is connected to the input end of the second convolutional layer;

[0029] The input end of the second convolutional layer is also connected to the second output end of the first feature extraction unit, and the output end of the second convolutional layer is connected to the input end of the first downsampling layer;

[0030] The output end of the first downsampling layer is connected to the input end of the first feature extraction layer;

[0031] The output end of the first feature extraction layer is connected to the input end of the fusion output module.

[0032] In a possible implementation, the first feature extraction unit includes: a second feature extraction layer and a second downsampling layer;

[0033] The input end of the second feature extraction layer is used to receive the original image, and the output end is respectively connected to the input end of the second down-sampling layer and the third input end of the first scale fusion unit;

[0034] The output end of the second down-sampling layer is respectively connected to the input end of the second feature extraction unit and the second input end of the first scale fusion unit.

[0035] In a possible implementation, the second feature extraction layer includes a third convolutional layer and a convolutional attention block connected in sequence;

[0036] The convolutional attention block includes: a spatial depth conversion convolution layer, a third feature fusion layer, a detail enhancement convolution layer, a fourth convolution layer, a CBAM attention mechanism layer and a fourth feature fusion layer;

[0037] The input end of the spatial depth conversion convolution layer and the input end of the third feature fusion layer are both connected to the third convolution layer;

[0038] The input end of the third feature fusion layer is also connected to the output end of the spatial depth conversion convolution layer, and the output end of the third feature fusion layer is connected to the input end of the detail enhancement convolution layer;

[0039] The output end of the detail enhancement convolution layer is respectively connected to the input end of the fourth convolution layer and the input end of the fourth feature fusion layer;

[0040] The output end of the fourth convolutional layer is connected to the input end of the CBAM attention mechanism layer, and the output end of the CBAM attention mechanism layer is connected to the input end of the fourth feature fusion layer;

[0041] The output end of the fourth feature fusion layer is respectively connected to the input end of the second down-sampling layer and the third input end of the first scale fusion unit.

[0042] In a possible implementation, the third feature extraction unit includes: a third feature extraction layer and an improved SPPF submodule;

[0043] The input end of the third feature extraction layer is connected to the output end of the second feature extraction unit;

[0044] The output end of the third feature extraction layer is connected to the input end of the improved SPPF submodule, and the output end of the improved SPPF submodule is respectively connected to the first input end of the second scale fusion unit and the input end of the third scale fusion unit.

[0045] In a possible implementation, the improved SPPF submodule includes: a fifth convolutional layer, a first pooling layer, a second pooling layer, a third pooling layer, a connection layer, an LSKA mechanism layer and a sixth convolutional layer;

[0046] The input end of the fifth convolutional layer is connected to the output end of the third feature extraction layer, and the output end of the fifth convolutional layer is respectively connected to the input end of the first pooling layer and the first input end of the connection layer;

[0047] The output end of the first pooling layer is respectively connected to the input end of the second pooling layer and the first input end of the connection layer;

[0048] The output end of the second pooling layer is respectively connected to the input end of the third pooling layer and the first input end of the connection layer;

[0049] The output end of the third pooling layer is connected to the second input end of the connection layer;

[0050] The output end of the connection layer is connected to the input end of the LSKA mechanism layer, and the output end of the LSKA mechanism layer is connected to the input end of the sixth convolutional layer;

[0051] The output end of the sixth convolutional layer is respectively connected to the first input end of the second scale fusion unit and the input end of the third scale fusion unit.

[0052] In a possible implementation, the fusion output module includes: a fifth feature fusion layer, a seventh convolution layer and an activation function layer;

[0053] The input end of the fifth feature fusion layer is respectively connected to the multiple output ends of the feature interaction module, and the output end of the fifth feature fusion layer is connected to the input end of the seventh convolutional layer;

[0054] The output end of the seventh convolutional layer is connected to the input end of the activation function layer, and the output end of the activation function layer is used to output a defogged image.

[0055] In a second aspect, an embodiment of the present invention provides a transmission line image processing device under rainy and foggy weather conditions, comprising:

[0056] An acquisition module, used to acquire an original image corresponding to the transmission line to be tested;

[0057] A processing module, used for inputting the original image into a multi-scale feature fusion network to obtain a fog-free image output by the multi-scale feature fusion network; the fog-free image is used for image recognition to determine a fault detection result corresponding to the power transmission line to be tested;

[0058] The multi-scale feature fusion network includes: a feature extraction module, a feature interaction module and a fusion output module; the input end of the feature extraction module is used to receive the original image, the multiple output ends of the feature extraction module are respectively connected to the multiple input ends of the feature interaction module, and the multiple output ends of the feature interaction module are all connected to the fusion output module;

[0059] The feature extraction module extracts features of different scales from the original image to obtain a first feature image, a second feature image and a third feature image, and outputs the images to the feature interaction module;

[0060] The feature interaction module determines a fourth feature image according to the third feature image; determines a fifth feature image according to the third feature image and the second feature image; determines a sixth feature image according to the fifth feature image and the first feature image, and outputs the fourth feature image, the fifth feature image and the sixth feature image to the fusion output module;

[0061] The fusion output module determines the dehazed image based on the feature image output by the feature interaction module.

[0062] The embodiment of the present invention provides a method and device for processing transmission line images under rainy and foggy weather conditions. The original image corresponding to the transmission line to be tested is defogged by a multi-scale feature fusion network to generate a fog-free image, thereby determining the fault detection result of the transmission line to be tested. Among them, the feature extraction module in the multi-scale feature fusion network can perform feature extraction of different scales on the original image to determine the first feature image, the second feature image and the third feature image. The feature interaction module can determine the fourth feature image according to the third feature image, determine the fifth feature image according to the third feature image and the second feature image, and determine the sixth feature image according to the fifth feature image and the first feature image, thereby realizing information fusion between feature images at different scales, while retaining the color and texture information and semantic information in the original image, thereby better retaining the image details and avoiding the problems of color distortion and image blur. In addition, when the fusion output module determines the defogging image according to the multiple feature images output by the feature interaction module, it can not only avoid the feature overlap and loss effect caused by the feature interaction module in the information fusion process to enhance the defogging effect, but also aggregate the rich feature information of each scale, thereby further retaining the image details, thereby improving the fault detection accuracy of the transmission line. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0064] Figure 1 is a flow chart of an implementation method of a power transmission line image processing method under rainy and foggy weather conditions provided by an embodiment of the present invention;

[0065] Figure 2 is a schematic diagram of the structure of a multi-scale feature fusion network provided by an embodiment of the present invention;

[0066] Figure 3 is a schematic diagram of the structure of a convolutional attention block provided by an embodiment of the present invention;

[0067] Figure 4 It is a schematic diagram of the structure of the CBAM attention mechanism layer provided by an embodiment of the present invention;

[0068] Figure 5 is a schematic diagram of the structure of a channel attention module provided by an embodiment of the present invention;

[0069] Figure 6 is a schematic diagram of the structure of a spatial attention module provided by an embodiment of the present invention;

[0070] Figure 7 It is a schematic diagram of the structure of an improved SPPF submodule provided in an embodiment of the present invention;

[0071] Figure 8 It is a structural schematic diagram of a transmission line image processing device under rainy and foggy weather conditions provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0072] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0073] At present, the dehazing technologies for transmission line images under rainy and foggy weather conditions are mainly divided into two categories: methods that rely on prior knowledge and methods that apply deep learning.

[0074] Prior knowledge methods: This type of method builds a model framework for foggy imaging and uses existing prior knowledge to estimate model parameters, thereby inversely generating a clear fog-free image. For example, the fog-free image is estimated through the dark channel prior and the atmospheric scattering model. Alternatively, the fog-free image is restored by estimating the transmittance based on the fog line prior theory.

[0075] Deep learning methods: Deep learning methods learn the transformation rules from foggy images to fog-free images by training a large amount of image data. For example, DCPPDN uses a generative adversarial network framework that simultaneously learns the transmission map and atmospheric light values ​​to restore fog-free images. GFN uses a gated fusion network to take the original blurred image and its preprocessed image as input to restore the fog-free image.

[0076] Although the above method can remove fog in the transmission line image to a certain extent, it is easy to lose image detail information during the defogging process, resulting in blurriness and color distortion in the defogged image, which in turn reduces the fault detection accuracy of the transmission line.

[0077] In order to better preserve image details and reduce image blur and color distortion during the dehazing process, a new multi-scale feature fusion network is designed in the implementation of the present application, including a feature extraction module, a feature interaction module and a fusion output module. The feature extraction module can extract feature images at different scales, the feature interaction module can exchange information with feature images at different scales to obtain new feature images, and the fusion output module fuses the new feature images. Among them, the feature images at the shallow scale have rich color and texture information, and the feature images at the deep scale contain rich semantic information. The feature interaction module exchanges information with feature images at different scales, so that the generated new feature images can retain the shallow color and texture information and the deep semantic information at the same time, thereby better preserving image details and reducing image blur and color distortion.

[0078] In order to make the purpose, technical solutions and advantages of the present invention more clear, specific embodiments will be described below in conjunction with the accompanying drawings.

[0079] Figure 1 The implementation flow chart of the method for processing power line images under rainy and foggy weather conditions provided by the embodiment of the present invention is described in detail as follows:

[0080] Step 101: Acquire an original image corresponding to the power transmission line to be tested.

[0081] Here, the original image may be an image of the transmission line to be tested acquired by an image acquisition device under rainy and foggy weather conditions.

[0082] Step 102: input the original image into a multi-scale feature fusion network to obtain a haze-free image output by the multi-scale feature fusion network.

[0083] Considering that foggy weather may cause fog in the original image, which may affect the subsequent fault detection results, the embodiment of the present invention pre-inputs the original image into a multi-scale feature fusion network to perform defogging on the original image to obtain a fog-free image. Image recognition is then performed based on the fog-free image to achieve fault detection of the transmission line to be tested.

[0084] See also Figure 2 The multi-scale feature fusion network includes: a feature extraction module 21, a feature interaction module 22 and a fusion output module 23. The input end of the feature extraction module 21 is used to receive the original image, the multiple output ends of the feature extraction module 21 are respectively connected to the multiple input ends of the feature interaction module 22, and the multiple output ends of the feature interaction module 22 are all connected to the fusion output module 23.

[0085] The feature extraction module 21 extracts features of different scales on the original image to obtain a first feature image, a second feature image and a third feature image, and outputs the images to the feature interaction module 22 .

[0086] The feature interaction module 22 determines the fourth feature image based on the third feature image; determines the fifth feature image based on the third feature image and the second feature image; determines the sixth feature image based on the fifth feature image and the first feature image, and outputs the fourth feature image, the fifth feature image and the sixth feature image to the fusion output module 23.

[0087] The fusion output module 23 determines the defogging image according to the feature image output by the feature interaction module 22 .

[0088] In the embodiment of the present invention, the feature extraction module 22 sequentially extracts features of three levels of scales on the original image to obtain a first feature image, a second feature image and a third feature image.

[0089] In some embodiments, see Figure 2 The feature extraction module 21 includes: a first feature extraction unit 211, a second feature extraction unit 212 and a third feature extraction unit 213.

[0090] The first output end of the first feature extraction unit 211 is respectively connected to the input end and the feature interaction module of the second feature extraction unit, and the output end of the second feature extraction unit is respectively connected to the feature interaction module of the input end of the third feature extraction unit. The output end of the third feature extraction unit is connected to the feature interaction module.

[0091] The input end of the first feature extraction unit 211 is used to receive the original image, and the first output end is used to output the first feature image. The input end of the second feature extraction unit 212 is used to receive the first feature image, and the output end is used to output the second feature image. The input end of the third feature extraction unit 213 is used to receive the second feature image, and the output end is used to output the third feature image.

[0092] It should be noted that when the feature extraction module extracts features of different scales from the image in sequence, the shallow high-resolution features extracted by the shallow-scale feature extraction unit contain rich color and texture information. The deep low-resolution features extracted by the deep-scale feature extraction unit contain rich semantic information. The feature interaction module integrates high-resolution features with the help of low-resolution features by maintaining the color and detail information in high-resolution features and allowing information exchange across parallel streams, thereby better preserving image details.

[0093] That is, the semantic information contained in the first feature image, the second feature image, and the third feature image increases in sequence, while the color and texture information contained in the first feature image, the second feature image, and the third feature image decreases in sequence.

[0094] The feature interaction module determines a fifth feature image based on the third feature image and the second feature image, so that the fifth feature image can contain both the rich semantic information in the third feature image and the rich color and texture information in the second feature image.

[0095] Similarly, the feature interaction module also determines the sixth feature image based on the fifth feature image and the first feature image, so that the sixth feature image can simultaneously include the color and texture information in the first feature image and the feature information in the fifth feature image (i.e., semantic information, color and texture information).

[0096] The feature interaction module performs information interaction on the first feature image, the second feature image and the third feature image, so that the output fourth feature image, the fifth feature image and the sixth feature image contain more semantic information, color information and texture information, thereby better retaining image details and avoiding image blur and color distortion.

[0097] The fusion output module 23 may perform feature fusion, convolution, and activation processing on the fourth feature image, the fifth feature image, and the sixth feature image, so as to generate a defogging image.

[0098] In some embodiments, see Figure 2 ,The fusion output module includes: the fifth feature fusion layer, the seventh convolution layer and the activation function layer.

[0099] The input end of the fifth feature fusion layer is respectively connected to the multiple output ends of the feature interaction module, and the output end of the fifth feature fusion layer is connected to the input end of the seventh convolutional layer;

[0100] The output end of the seventh convolutional layer is connected to the input end of the activation function layer, and the output end of the activation function layer is used to output the dehazed image.

[0101] Here, the fifth feature fusion layer can perform feature fusion on the fourth feature image, the fifth feature image and the sixth feature image output by the feature interaction module, which can not only avoid the feature overlap and loss effect caused by the feature interaction module in the information fusion process to enhance the dehazing effect, but also aggregate rich feature information at various scales to retain image details.

[0102] After feature fusion, the embodiment of the present invention performs convolution processing through the seventh convolution layer and activation processing through the activation function layer to obtain a defogging image. The defogging image is used for image recognition to determine the fault detection result corresponding to the transmission line to be tested.

[0103] Exemplarily, the seventh convolution layer may be a 1×1 convolution structure. The activation function of the activation function layer may be a Sigmoid activation function.

[0104] The embodiment of the present invention performs defogging processing on the original image corresponding to the transmission line to be tested through a multi-scale feature fusion network, and can generate a fog-free image, thereby determining the fault detection result of the transmission line to be tested. Among them, the feature extraction module in the multi-scale feature fusion network can perform feature extraction of different scales on the original image, determine the first feature image, the second feature image and the third feature image, and the feature interaction module can determine the fourth feature image according to the third feature image, determine the fifth feature image according to the third feature image and the second feature image, and determine the sixth feature image according to the fifth feature image and the first feature image, thereby realizing information fusion between feature images at different scales, while retaining the color and texture information and semantic information in the original image, so as to better retain the image details and avoid the problems of color distortion and image blur. In addition, when the fusion output module determines the defogging image according to the multiple feature images output by the feature interaction module, it can not only avoid the feature overlap and loss effect caused by the feature interaction module in the information fusion process to enhance the defogging effect, but also aggregate the rich feature information of each scale, thereby further retaining the image details.

[0105] In some embodiments, see Figure 2 The first feature extraction unit 211 also sets a second output terminal for outputting the intermediate feature variable.

[0106] The feature interaction module 22 includes: a first scale fusion unit 221 , a second scale fusion unit 222 and a third scale fusion unit 223 .

[0107] The input end of the third scale fusion unit 223 is connected to the output end of the third feature extraction unit 213, and the output end of the third scale fusion unit 223 is connected to the input end of the fusion output module 23, so as to perform convolution processing on the third feature image P3 to obtain a fourth feature image P4, and input the fourth feature image P4 to the fusion output module 23;

[0108] Here, the third-scale fusion unit may use a 1×1 convolution structure to perform convolution processing on the third feature image to obtain a fourth feature image.

[0109] The first input end of the second scale fusion unit 222 is connected to the output end of the third feature extraction unit 213, the second input end of the second scale fusion unit 222 is connected to the output end of the second feature extraction unit 212, and the output end of the second scale fusion unit 222 is respectively connected to the first input end of the first scale fusion unit 221 and the input end of the fusion output module 23, for processing the third feature image P3 and the second feature image P2 to obtain a fifth feature image P5, and inputting the fifth feature image P5 to the first scale fusion unit 221 and the fusion output module 23 respectively;

[0110] The first input end of the first scale fusion unit 221 is connected to the output end of the second scale fusion unit 222, the second input end of the first scale fusion unit 221 is connected to the first output end of the first feature extraction unit 211, the third input end of the first scale fusion unit 221 is connected to the second output end of the first feature extraction unit 211, and the output end of the first scale fusion unit 221 is connected to the input end of the fusion output module 23, which is used to process the fifth feature image P5, the first feature image P1 and the intermediate feature variable to obtain the sixth feature image P6, and input it to the fusion output module.

[0111] In the embodiment of the present invention, the first feature extraction unit, the second feature extraction unit and the third feature extraction unit are used to extract features in sequence to realize a top-down feature pyramid. At the same time, the feature interaction module is used to perform bottom-up information fusion, thereby retaining rich color and texture information, as well as semantic information.

[0112] In some embodiments, see Figure 2 , the second scale fusion unit includes: a first upsampling layer, a first feature fusion layer and a first convolutional layer.

[0113] An input end of the first upsampling layer is connected to an output end of the third feature extraction unit, and an output end of the first upsampling layer is connected to an input end of the first feature fusion layer;

[0114] The input end of the first feature fusion layer is also connected to the output end of the second feature extraction unit, and the output end of the first feature fusion layer is connected to the input end of the first convolutional layer;

[0115] The output end of the first convolutional layer is respectively connected to the first input end of the first scale fusion unit and the input end of the fusion output module.

[0116] Here, the first sampling layer performs an upsampling operation, the first feature fusion layer performs a feature fusion operation, and the first convolution layer performs a convolution operation. The second scale fusion unit first performs an upsampling operation on the third feature image output by the third feature extraction unit, then performs feature fusion on the upsampled feature image and the second feature image, and finally performs convolution processing on the feature fused feature image to obtain a fifth feature image. Exemplarily, the first convolution layer can be a 3×3 convolution structure.

[0117] In some embodiments, see Figure 2 The first scale fusion unit includes: a second upsampling layer, a second feature fusion layer, a second convolutional layer, a first downsampling layer, and a first feature extraction layer.

[0118] The input end of the second upsampling layer is connected to the output end of the second scale fusion unit, and the output end of the second upsampling layer is connected to the input end of the second feature fusion layer;

[0119] The input end of the second feature fusion layer is also connected to the first output end of the first feature extraction unit, and the output end of the second feature fusion layer is connected to the input end of the second convolutional layer;

[0120] The input end of the second convolutional layer is also connected to the second output end of the first feature extraction unit, and the output end of the second convolutional layer is connected to the input end of the first downsampling layer;

[0121] The output end of the first downsampling layer is connected to the input end of the first feature extraction layer;

[0122] The output end of the first feature extraction layer is connected to the input end of the fusion output module.

[0123] The first scale fusion unit first upsamples the fifth feature image P5, and performs feature fusion on the upsampled feature image and the first feature image P1, then performs convolution processing on the fused feature image and the intermediate feature variable output by the first feature extraction unit, and then performs downsampling and feature extraction on the convolution-processed feature image to generate a sixth feature image P6. Exemplarily, the second convolution layer can be a 3×3 convolution structure.

[0124] In some embodiments, see Figure 2 , the first feature extraction unit includes: a second feature extraction layer and a second downsampling layer;

[0125] The input end of the second feature extraction layer is used to receive the original image, and the output end is respectively connected to the input end of the second down-sampling layer and the third input end of the first scale fusion unit;

[0126] The output end of the second down-sampling layer is respectively connected to the input end of the second feature extraction unit and the second input end of the first scale fusion unit.

[0127] The second feature extraction unit is used to extract features from the original image to obtain intermediate feature variables, and to perform downsampling processing on the intermediate feature variables to obtain a first feature image P1.

[0128] The second feature extraction layer includes a third convolution layer and a convolutional attention block connected in sequence. Among them, the third convolution layer can perform preliminary feature extraction on the original image through convolution operation. On this basis, the convolutional attention block further processes the preliminary feature extraction results, which can make the output intermediate feature variables richer and more refined in texture and details. Exemplarily, the third convolution layer can be a 3×3 convolution structure.

[0129] See also Figure 3 ,The convolutional attention block includes: spatial depth conversion convolution layer, the third feature fusion layer, detail enhancement convolution layer, the fourth convolution layer, CBAM attention mechanism layer and the fourth feature fusion layer;

[0130] The input end of the spatial depth conversion convolution layer and the input end of the third feature fusion layer are both connected to the third convolution layer;

[0131] The input end of the third feature fusion layer is also connected to the output end of the spatial depth conversion convolution layer, and the output end of the third feature fusion layer is connected to the input end of the detail enhancement convolution layer;

[0132] The output end of the detail enhancement convolution layer is respectively connected to the input end of the fourth convolution layer and the input end of the fourth feature fusion layer; here, the fourth convolution layer can be a 3×3 convolution structure.

[0133] The output end of the fourth convolutional layer 34 is connected to the input end of the CBAM attention mechanism layer 35, and the output end of the CBAM attention mechanism layer 35 is connected to the input end of the fourth feature fusion layer 36;

[0134] The output end of the fourth feature fusion layer 36 is respectively connected to the input end of the second down-sampling layer and the third input end of the first scale fusion unit.

[0135] The detail enhancement convolution layer, spatial depth conversion convolution layer and CBAM attention mechanism layer are introduced below.

[0136] The detail-enhanced convolution (DEConv) layer can enhance the expression and adaptability of the model by integrating prior knowledge into traditional convolution.

[0137] Considering that images taken under rainy and foggy weather conditions often face problems such as low visibility, unclear details, and blurred outlines of small objects, and there are also a large number of redundant pixels. When processing images, traditional convolutional neural networks usually use stride convolution and pooling operations to reduce the resolution of feature maps, thereby reducing the amount of calculation. However, these operations may lose important spatial information, especially in rainy and foggy weather conditions, the loss of detail information will affect the defogging effect. To solve this problem, an embodiment of the present invention introduces a space-to-Depth Convolution (SPD-Conv) layer.

[0138] SPD-Conv replaces traditional stride convolution and pooling operations to more effectively handle blur and tiny objects in foggy images without significantly increasing computing resource requirements. It works by using a pyramid structure to perform multi-scale analysis of images while using differential convolution to capture local features and enhance the perception of details. This approach not only optimizes the network structure, but also improves the performance and robustness of the model when processing foggy images.

[0139] Specifically, SPD-Conv is implemented through the following steps:

[0140] Multi-scale analysis: Decompose the input image into feature maps of different scales, each scale corresponding to a different resolution.

[0141] Differential convolution: A differential convolution operation is applied at each scale to capture local changes in the image and enhance the perception of details.

[0142] Feature fusion: Fuse feature maps of different scales to retain rich detail and semantic information.

[0143] This design not only reduces the dependence on computing resources, but also improves the real-time performance of the model when processing foggy images, making it more suitable for real-time inspection and rapid response application scenarios.

[0144] See also Figure 4 ,The CBAM (Convolutional Block Attention Module) attention mechanism layer contains two parts: channel attention module and spatial attention module, which respectively pay attention to channel and spatial features, thereby providing more comprehensive feature extraction capabilities. This mechanism can assign a unique spatial importance map to each channel, enhancing the model's perception of image features.

[0145] The workflow of CBAM is as follows: first, the input feature map is received and then processed by the channel attention module. In the channel attention module, the input feature map undergoes a one-dimensional convolution operation to generate a channel attention weight. This weight is then used to adjust the channel response of the original feature map, enhancing important features and suppressing unimportant features. Next, the processed feature map is sent to the spatial attention module. In the spatial attention module, the feature map undergoes a two-dimensional convolution to generate a spatial attention weight. This weight is used to adjust the spatial response of the feature map, further highlighting the salient areas and reducing the impact of background noise. Finally, the feature map processed by the dual attention mechanism of channel and space not only retains the key information, but also improves the expressiveness of the features, thereby providing richer feature support for subsequent network tasks.

[0146] See also Figure 5 After the feature map enters the channel attention module, it first passes through two parallel aggregation operations, namely the global maximum pooling operation (GlobalMaxPool) and the global average pooling operation (GlobalAvgPool). These two operations convert the original C×H×W dimensional feature map into a size of C×1×1, so that the features of each channel are aggregated into a single value. These aggregated features are then fed into a shared multi-layer perceptron module. In this multi-layer perceptron module, the feature map undergoes two linear transformations: first, the number of channels is compressed to the original 1 / r, and then the number of channels is expanded back to the original value. In this process, the features are nonlinearly processed through a ReLU activation function.

[0147] The two feature maps obtained by these two steps are added element by element to integrate the information from different pooling methods. Finally, this combined feature map is activated by a sigmoid function to generate attention weights for each channel. These weights are then used to adjust the channel response of the original feature map by multiplying the attention weights with the original feature map to achieve weighted adjustment of the channel dimension and restore it to the original C×H×W size.

[0148] See also Figure 6, the spatial attention module of CBAM aims to keep the spatial resolution of the feature map unchanged while optimizing the channel dimension. This process starts with the output of the channel attention module, compressing each channel of the feature map into a single spatial dimension through the maximum pooling (MaxPool) and average pooling (AvgPool) operations, generating two 1×H×W feature maps. The two feature maps are then concatenated together to form a comprehensive feature representation. This concatenated feature map is convolved with a 7×7 convolution kernel, which is a key step because it allows the network to capture important location information in a wider spatial range. Experimental results show that the 7×7 convolution kernel is more effective in capturing spatial information than the 3×3 convolution kernel. The output of the convolution is processed by the sigmoid function to generate a spatial attention weight map, which highlights the importance of each spatial location in the feature map. Finally, this spatial attention weight map is multiplied with the original feature map to restore its original C×H×W size. In this way, each channel of the feature map is enhanced or suppressed accordingly according to its spatial importance, enabling the network to locate and identify targets more accurately.

[0149] In addition, a residual connection mechanism is set up in the convolutional attention module, that is, the input end of the spatial depth conversion convolution layer and the input end of the third feature fusion layer are both connected to the third convolution layer, and the input end of the third feature fusion layer 32 is also connected to the connection structure of the output end of the spatial depth conversion convolution layer 31. This residual connection mechanism can avoid the loss of information during gradient propagation, thereby accelerating the convergence speed of the model.

[0150] The convolutional attention module further processes the preliminary feature extraction results output by the third convolutional layer by adopting different convolutional structures and CBAM attention mechanism layers, making the output intermediate feature variables richer and more refined in texture and details, thereby improving the image dehazing effect and retaining more local details.

[0151] In the embodiment of the present invention, the internal connection structure of the second feature extraction unit is the same as that of the first feature extraction unit. For details, please refer to the description of the first feature extraction unit, which will not be repeated here.

[0152] In some embodiments, see Figure 2 , the third feature extraction unit includes: a third feature extraction layer and an improved SPPF submodule;

[0153] An input end of the third feature extraction layer is connected to an output end of the second feature extraction unit;

[0154] The output end of the third feature extraction layer is connected to the input end of the improved SPPF submodule, and the output end of the improved SPPF submodule is respectively connected to the first input end of the second scale fusion unit and the input end of the third scale fusion unit.

[0155] In the embodiment of the present invention, the internal connection structures of the first feature extraction layer, the third feature extraction layer and the second feature extraction layer are the same, and the details may be referred to the description of the second feature extraction layer, which will not be repeated here.

[0156] In some embodiments, see Figure 7 ,The improved SPPF sub-module includes: the fifth convolution layer, the first pooling layer, the second pooling layer, the third pooling layer, the connection layer, the LSKA mechanism layer and the sixth convolution layer;

[0157] The input end of the fifth convolutional layer is connected to the output end of the third feature extraction layer, and the output end of the fifth convolutional layer is respectively connected to the input end of the first pooling layer and the first input end of the connection layer;

[0158] The output end of the first pooling layer is respectively connected to the input end of the second pooling layer and the first input end of the connection layer;

[0159] The output end of the second pooling layer is respectively connected to the input end of the third pooling layer and the first input end of the connection layer;

[0160] The output end of the third pooling layer is connected to the second input end of the connection layer;

[0161] The output end of the connection layer is connected to the input end of the LSKA mechanism layer, and the output end of the LSKA mechanism layer is connected to the input end of the sixth convolutional layer;

[0162] The output end of the sixth convolutional layer is respectively connected to the first input end of the second scale fusion unit and the input end of the third scale fusion unit.

[0163] The embodiment of the present invention obtains an improved SPPF submodule by introducing an LSKA (Large Separable Kernel Attention) mechanism layer into the SPPF (Spatial Pyramid Pooling) module. In the improved SPPF submodule, the input data is first processed by the first pooling layer, the second pooling layer, and the third pooling layer, and the output of each layer is spliced ​​in parallel to form a multi-scale feature set. In order to further optimize this process, the LSKA mechanism is introduced, which effectively reduces the parameter expansion problem caused by large convolution kernels by splitting the traditional k×k convolution kernel into 1×k and k×1 separable convolution kernels and processing the input in a cascade manner.

[0164] The sizes of the first pooling layer, the second pooling layer, and the third pooling layer may be 5*5. The LSKA mechanism layer may include a large-sized separable convolution kernel, for example, a 11×11 convolution kernel.

[0165] The large-sized separable convolution kernels in the LSKA mechanism layer are used to capture long-distance dependencies and provide a larger receptive field, thereby enhancing the ability of feature representation. Subsequently, these features are further fused through standard convolution operations to optimize the output feature vector of the model backbone network.

[0166] This method not only retains the advantages of SPPF in multi-scale feature extraction, but also enhances the richness and effectiveness of features through the LSKA mechanism, thereby improving the overall model's ability in feature fusion and representation, while the amount of parameter increase is relatively small. This fusion strategy enables the model to handle complex visual tasks more effectively and capture and integrate key information at different scales and spatial locations.

[0167] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0168] The following is an embodiment of the device of the present invention. For details not described in detail therein, reference may be made to the corresponding method embodiment described above.

[0169] Figure 8 The structure diagram of the transmission line image processing device under rainy and foggy weather conditions provided by an embodiment of the present invention is shown. For the convenience of description, only the part related to the embodiment of the present invention is shown, which is described in detail as follows:

[0170] like Figure 8 As shown, the transmission line image processing device 8 under rainy and foggy weather conditions includes: an acquisition module 81 and a processing module 82.

[0171] An acquisition module 81 is used to acquire an original image corresponding to the power transmission line to be tested;

[0172] The processing module 82 is used to input the original image into the multi-scale feature fusion network to obtain a fog-free image output by the multi-scale feature fusion network; the fog-free image is used for image recognition to determine the fault detection result corresponding to the transmission line to be tested;

[0173] The multi-scale feature fusion network includes: a feature extraction module, a feature interaction module and a fusion output module; the input end of the feature extraction module is used to receive the original image, the multiple output ends of the feature extraction module are respectively connected to the multiple input ends of the feature interaction module, and the multiple output ends of the feature interaction module are all connected to the fusion output module;

[0174] The feature extraction module extracts features of different scales from the original image to obtain a first feature image, a second feature image and a third feature image, and outputs the images to the feature interaction module;

[0175] The feature interaction module determines a fourth feature image according to the third feature image; determines a fifth feature image according to the third feature image and the second feature image; determines a sixth feature image according to the fifth feature image and the first feature image, and outputs the fourth feature image, the fifth feature image and the sixth feature image to the fusion output module;

[0176] The fusion output module determines the dehazed image based on the feature image output by the feature interaction module.

[0177] Optionally, the feature extraction module includes: a first feature extraction unit, a second feature extraction unit and a third feature extraction unit;

[0178] The input end of the first feature extraction unit is used to receive the original image, the first output end is used to output the first feature image, and the second output end is used to output the intermediate feature variable;

[0179] The input end of the second feature extraction unit is used to receive the first feature image, and the output end is used to output the second feature image;

[0180] The input end of the third feature extraction unit is used to receive the second feature image, and the output end is used to output the third feature image;

[0181] The feature interaction module includes: a first scale fusion unit, a second scale fusion unit and a third scale fusion unit;

[0182] The input end of the third scale fusion unit is connected to the output end of the third feature extraction unit, and the output end of the third scale fusion unit is connected to the input end of the fusion output module, so as to perform convolution processing on the third feature image to obtain a fourth feature image, and input the fourth feature image into the fusion output module;

[0183] The first input end of the second scale fusion unit is connected to the output end of the third feature extraction unit, the second input end of the second scale fusion unit is connected to the output end of the second feature extraction unit, and the output end of the second scale fusion unit is respectively connected to the first input end of the first scale fusion unit and the input end of the fusion output module, for processing the third feature image and the second feature image to obtain a fifth feature image, and respectively inputting the fifth feature image to the first scale fusion unit and the fusion output module;

[0184] A first input end of the first scale fusion unit is connected to an output end of the second scale fusion unit, a second input end of the first scale fusion unit is connected to a first output end of the first feature extraction unit, a third input end of the first scale fusion unit is connected to a second output end of the first feature extraction unit, and an output end of the first scale fusion unit is connected to an input end of a fusion output module, for processing the fifth feature image, the first feature image, and the intermediate feature variable to obtain a sixth feature image, and input the sixth feature image into the fusion output module.

[0185] Optionally, the second scale fusion unit includes: a first upsampling layer, a first feature fusion layer and a first convolution layer;

[0186] An input end of the first upsampling layer is connected to an output end of the third feature extraction unit, and an output end of the first upsampling layer is connected to an input end of the first feature fusion layer;

[0187] The input end of the first feature fusion layer is also connected to the output end of the second feature extraction unit, and the output end of the first feature fusion layer is connected to the input end of the first convolutional layer;

[0188] The output end of the first convolutional layer is respectively connected to the first input end of the first scale fusion unit and the input end of the fusion output module.

[0189] Optionally, the first scale fusion unit includes: a second upsampling layer, a second feature fusion layer, a second convolution layer, a first downsampling layer, and a first feature extraction layer;

[0190] The input end of the second upsampling layer is connected to the output end of the second scale fusion unit, and the output end of the second upsampling layer is connected to the input end of the second feature fusion layer;

[0191] The input end of the second feature fusion layer is also connected to the first output end of the first feature extraction unit, and the output end of the second feature fusion layer is connected to the input end of the second convolutional layer;

[0192] The input end of the second convolutional layer is also connected to the second output end of the first feature extraction unit, and the output end of the second convolutional layer is connected to the input end of the first downsampling layer;

[0193] The output end of the first downsampling layer is connected to the input end of the first feature extraction layer;

[0194] The output end of the first feature extraction layer is connected to the input end of the fusion output module.

[0195] Optionally, the first feature extraction unit includes: a second feature extraction layer and a second downsampling layer;

[0196] The input end of the second feature extraction layer is used to receive the original image, and the output end is respectively connected to the input end of the second down-sampling layer and the third input end of the first scale fusion unit;

[0197] The output end of the second down-sampling layer is respectively connected to the input end of the second feature extraction unit and the second input end of the first scale fusion unit.

[0198] Optionally, the second feature extraction layer includes a third convolutional layer and a convolutional attention block connected in sequence;

[0199] The convolutional attention block includes: spatial depth conversion convolution layer, third feature fusion layer, detail enhancement convolution layer, fourth convolution layer, CBAM attention mechanism layer and fourth feature fusion layer;

[0200] The input end of the spatial depth conversion convolution layer and the input end of the third feature fusion layer are both connected to the third convolution layer;

[0201] The input end of the third feature fusion layer is also connected to the output end of the spatial depth conversion convolution layer, and the output end of the third feature fusion layer is connected to the input end of the detail enhancement convolution layer;

[0202] The output end of the detail enhancement convolution layer is connected to the input end of the fourth convolution layer and the input end of the fourth feature fusion layer respectively;

[0203] The output end of the fourth convolutional layer is connected to the input end of the CBAM attention mechanism layer, and the output end of the CBAM attention mechanism layer is connected to the input end of the fourth feature fusion layer;

[0204] The output end of the fourth feature fusion layer is respectively connected to the input end of the second downsampling layer and the third input end of the first scale fusion unit.

[0205] Optionally, the third feature extraction unit includes: a third feature extraction layer and an improved SPPF submodule;

[0206] An input end of the third feature extraction layer is connected to an output end of the second feature extraction unit;

[0207] The output end of the third feature extraction layer is connected to the input end of the improved SPPF submodule, and the output end of the improved SPPF submodule is respectively connected to the first input end of the second scale fusion unit and the input end of the third scale fusion unit.

[0208] Optionally, the improved SPPF submodule includes: a fifth convolutional layer, a first pooling layer, a second pooling layer, a third pooling layer, a connection layer, an LSKA mechanism layer and a sixth convolutional layer;

[0209] The input end of the fifth convolutional layer is connected to the output end of the third feature extraction layer, and the output end of the fifth convolutional layer is respectively connected to the input end of the first pooling layer and the first input end of the connection layer;

[0210] The output end of the first pooling layer is respectively connected to the input end of the second pooling layer and the first input end of the connection layer;

[0211] The output end of the second pooling layer is respectively connected to the input end of the third pooling layer and the first input end of the connection layer;

[0212] The output end of the third pooling layer is connected to the second input end of the connection layer;

[0213] The output end of the connection layer is connected to the input end of the LSKA mechanism layer, and the output end of the LSKA mechanism layer is connected to the input end of the sixth convolutional layer;

[0214] The output end of the sixth convolutional layer is respectively connected to the first input end of the second scale fusion unit and the input end of the third scale fusion unit.

[0215] Optionally, the fusion output module includes: a fifth feature fusion layer, a seventh convolution layer and an activation function layer;

[0216] The input end of the fifth feature fusion layer is respectively connected to the multiple output ends of the feature interaction module, and the output end of the fifth feature fusion layer is connected to the input end of the seventh convolutional layer;

[0217] The output end of the seventh convolutional layer is connected to the input end of the activation function layer, and the output end of the activation function layer is used to output the dehazed image.

[0218] The transmission line image processing device under rainy and foggy weather conditions provided in an embodiment of the present invention can be used to implement the above-mentioned transmission line image processing method under rainy and foggy weather conditions. Its technical principle and implementation effect are the same as those of the above-mentioned method embodiment, and will not be repeated here.

[0219] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0220] Those of ordinary skill in the art will appreciate that the templates, units, and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0221] If the module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned power transmission line image processing method embodiments under rainy and foggy weather conditions can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory, random access memory, electric carrier signal, telecommunication signal and software distribution medium.

[0222] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for processing transmission line images under rainy and foggy weather conditions, characterized in that: include: Obtaining an original image corresponding to the transmission line to be tested; Inputting the original image into a multi-scale feature fusion network to obtain a haze-free image output by the multi-scale feature fusion network; The fog-free image is used for image recognition to determine the fault detection result corresponding to the power transmission line to be tested; The multi-scale feature fusion network includes: a feature extraction module, a feature interaction module and a fusion output module; the input end of the feature extraction module is used to receive the original image, the multiple output ends of the feature extraction module are respectively connected to the multiple input ends of the feature interaction module, and the multiple output ends of the feature interaction module are all connected to the fusion output module; The feature extraction module extracts features of different scales from the original image to obtain a first feature image, a second feature image and a third feature image, and outputs the images to the feature interaction module; The feature interaction module determines a fourth feature image according to the third feature image; determines a fifth feature image according to the third feature image and the second feature image; determines a sixth feature image according to the fifth feature image and the first feature image, and outputs the fourth feature image, the fifth feature image and the sixth feature image to the fusion output module; The fusion output module determines the dehazed image based on the feature image output by the feature interaction module.

2. The method for processing power line images under rainy and foggy weather conditions according to claim 1, characterized in that: The feature extraction module includes: a first feature extraction unit, a second feature extraction unit and a third feature extraction unit; The input end of the first feature extraction unit is used to receive the original image, the first output end is used to output the first feature image, and the second output end is used to output the intermediate feature variable; The input end of the second feature extraction unit is used to receive the first feature image, and the output end is used to output the second feature image; The input end of the third feature extraction unit is used to receive the second feature image, and the output end is used to output the third feature image; The feature interaction module includes: a first scale fusion unit, a second scale fusion unit and a third scale fusion unit; The input end of the third scale fusion unit is connected to the output end of the third feature extraction unit, and the output end of the third scale fusion unit is connected to the input end of the fusion output module, so as to perform convolution processing on the third feature image to obtain a fourth feature image, and input the fourth feature image into the fusion output module; The first input end of the second scale fusion unit is connected to the output end of the third feature extraction unit, the second input end of the second scale fusion unit is connected to the output end of the second feature extraction unit, and the output end of the second scale fusion unit is respectively connected to the first input end of the first scale fusion unit and the input end of the fusion output module, for processing the third feature image and the second feature image to obtain a fifth feature image, and inputting the fifth feature image to the first scale fusion unit and the fusion output module respectively; A first input end of the first scale fusion unit is connected to an output end of the second scale fusion unit, a second input end of the first scale fusion unit is connected to a first output end of the first feature extraction unit, a third input end of the first scale fusion unit is connected to a second output end of the first feature extraction unit, and an output end of the first scale fusion unit is connected to an input end of the fusion output module, for processing the fifth feature image, the first feature image, and the intermediate feature variable to obtain a sixth feature image, and input the sixth feature image into the fusion output module.

3. The method for processing power line images under rainy and foggy weather conditions according to claim 2, characterized in that: The second scale fusion unit includes: a first upsampling layer, a first feature fusion layer and a first convolution layer; The input end of the first upsampling layer is connected to the output end of the third feature extraction unit, and the output end of the first upsampling layer is connected to the input end of the first feature fusion layer; The input end of the first feature fusion layer is also connected to the output end of the second feature extraction unit, and the output end of the first feature fusion layer is connected to the input end of the first convolutional layer; The output end of the first convolutional layer is respectively connected to the first input end of the first scale fusion unit and the input end of the fusion output module.

4. The method for processing power transmission line images under rainy and foggy weather conditions according to claim 2 or 3, characterized in that: The first scale fusion unit includes: a second upsampling layer, a second feature fusion layer, a second convolution layer, a first downsampling layer, and a first feature extraction layer; An input end of the second upsampling layer is connected to an output end of the second scale fusion unit, and an output end of the second upsampling layer is connected to an input end of the second feature fusion layer; The input end of the second feature fusion layer is also connected to the first output end of the first feature extraction unit, and the output end of the second feature fusion layer is connected to the input end of the second convolutional layer; The input end of the second convolutional layer is also connected to the second output end of the first feature extraction unit, and the output end of the second convolutional layer is connected to the input end of the first downsampling layer; The output end of the first downsampling layer is connected to the input end of the first feature extraction layer; The output end of the first feature extraction layer is connected to the input end of the fusion output module.

5. The method for processing power line images under rainy and foggy weather conditions according to claim 2 or 3, characterized in that: The first feature extraction unit includes: a second feature extraction layer and a second downsampling layer; The input end of the second feature extraction layer is used to receive the original image, and the output end is respectively connected to the input end of the second down-sampling layer and the third input end of the first scale fusion unit; The output end of the second down-sampling layer is respectively connected to the input end of the second feature extraction unit and the second input end of the first scale fusion unit.

6. The method for processing power line images under rainy and foggy weather conditions according to claim 5, characterized in that: The second feature extraction layer comprises a third convolutional layer and a convolutional attention block connected in sequence; The convolutional attention block includes: a spatial depth conversion convolution layer, a third feature fusion layer, a detail enhancement convolution layer, a fourth convolution layer, a CBAM attention mechanism layer and a fourth feature fusion layer; An input end of the spatial depth conversion convolution layer and an input end of the third feature fusion layer are both connected to the third convolution layer; The input end of the third feature fusion layer is also connected to the output end of the spatial depth conversion convolution layer, and the output end of the third feature fusion layer is connected to the input end of the detail enhancement convolution layer; The output end of the detail enhancement convolution layer is respectively connected to the input end of the fourth convolution layer and the input end of the fourth feature fusion layer; The output end of the fourth convolutional layer is connected to the input end of the CBAM attention mechanism layer, and the output end of the CBAM attention mechanism layer is connected to the input end of the fourth feature fusion layer; The output end of the fourth feature fusion layer is respectively connected to the input end of the second down-sampling layer and the third input end of the first scale fusion unit.

7. The method for processing power line images under rainy and foggy weather conditions according to claim 2 or 3, characterized in that: The third feature extraction unit includes: a third feature extraction layer and an improved SPPF submodule; The input end of the third feature extraction layer is connected to the output end of the second feature extraction unit; The output end of the third feature extraction layer is connected to the input end of the improved SPPF submodule, and the output end of the improved SPPF submodule is respectively connected to the first input end of the second scale fusion unit and the input end of the third scale fusion unit.

8. The method for processing power line images under rainy and foggy weather conditions according to claim 7, characterized in that: The improved SPPF submodule includes: a fifth convolutional layer, a first pooling layer, a second pooling layer, a third pooling layer, a connection layer, an LSKA mechanism layer and a sixth convolutional layer; The input end of the fifth convolutional layer is connected to the output end of the third feature extraction layer, and the output end of the fifth convolutional layer is respectively connected to the input end of the first pooling layer and the first input end of the connection layer; The output end of the first pooling layer is respectively connected to the input end of the second pooling layer and the first input end of the connection layer; The output end of the second pooling layer is respectively connected to the input end of the third pooling layer and the first input end of the connection layer; The output end of the third pooling layer is connected to the second input end of the connection layer; The output end of the connection layer is connected to the input end of the LSKA mechanism layer, and the output end of the LSKA mechanism layer is connected to the input end of the sixth convolutional layer; The output end of the sixth convolutional layer is respectively connected to the first input end of the second scale fusion unit and the input end of the third scale fusion unit.

9. The method for processing power line images under rainy and foggy weather conditions according to any one of claims 1 to 3, characterized in that: The fusion output module includes: a fifth feature fusion layer, a seventh convolution layer and an activation function layer; The input end of the fifth feature fusion layer is respectively connected to the multiple output ends of the feature interaction module, and the output end of the fifth feature fusion layer is connected to the input end of the seventh convolutional layer; The output end of the seventh convolutional layer is connected to the input end of the activation function layer, and the output end of the activation function layer is used to output a defogged image.

10. A transmission line image processing device under rainy and foggy weather conditions, characterized in that: include: An acquisition module, used for acquiring an original image corresponding to the transmission line to be tested; A processing module, used for inputting the original image into a multi-scale feature fusion network to obtain a haze-free image output by the multi-scale feature fusion network; The fog-free image is used for image recognition to determine the fault detection result corresponding to the power transmission line to be tested; The multi-scale feature fusion network includes: a feature extraction module, a feature interaction module and a fusion output module; the input end of the feature extraction module is used to receive the original image, the multiple output ends of the feature extraction module are respectively connected to the multiple input ends of the feature interaction module, and the multiple output ends of the feature interaction module are all connected to the fusion output module; The feature extraction module extracts features of different scales from the original image to obtain a first feature image, a second feature image and a third feature image, and outputs the images to the feature interaction module; The feature interaction module determines a fourth feature image according to the third feature image; determines a fifth feature image according to the third feature image and the second feature image; determines a sixth feature image according to the fifth feature image and the first feature image, and outputs the fourth feature image, the fifth feature image and the sixth feature image to the fusion output module; The fusion output module determines the dehazed image based on the feature image output by the feature interaction module.

Images