A power transmission line monitoring image enhancement method and device

Abstract

The application discloses a power transmission line monitoring image enhancement method and device, comprising: adopting the received training set and test set to construct two initial image generation models and two initial image pair models to carry out bidirectional training and model verification, and determine a target image generation model; adopting an encoder in the target image generation model to carry out multi-layer feature extraction on a target power transmission line monitoring image, and output a preliminary feature map, a cascaded feature map and an encoded feature map; inputting the encoded feature map into a receptive field residual module and a light attention module to carry out feature extraction and feature weighting and construct a weight feature map; and carrying out multi-scale feature extraction and feature fusion on the preliminary feature map, the cascaded feature map and the weight feature map through the light attention module and a decoder to determine a target enhanced image. The enhanced image obtained through the target image generation model improves the image texture detail definition, and the enhanced image quality is better.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and apparatus for enhancing images of power transmission line monitoring. Background Technology

[0002] Because overhead transmission lines are exposed to the outdoor high-altitude environment for extended periods, power companies use remote cameras installed on transmission towers to collect and analyze images of the transmission lines in real time to monitor their status. However, under low light conditions such as cloudy days or nighttime, the images captured by these remote cameras are of poor quality, with significant loss of detail, reducing the accuracy of defect detection. Therefore, it is necessary to enhance the monitoring images of transmission lines under low light conditions to improve image quality.

[0003] Existing image enhancement methods under low illumination can be divided into three categories: histogram equalization-based methods, Retinex-based methods, and deep learning-based methods. Among them, deep learning-based methods offer better image enhancement quality and versatility compared to histogram equalization and Retinex-based methods.

[0004] Generative adversarial networks are currently being used for image enhancement in low-light conditions. Although the enhanced images are of better quality than those based on conventional deep learning models, the texture details of the enhanced images are still somewhat blurred, and some areas are overexposed or underexposed, which affects the accuracy of power transmission line defect detection. Further improvements are needed to enhance the image quality. Summary of the Invention

[0005] This invention provides a method and apparatus for enhancing transmission line monitoring images, which solves the technical problem that the enhanced images generated by existing image enhancement technologies are of low quality.

[0006] The first aspect of this invention provides a method for enhancing images of power transmission line monitoring, comprising:

[0007] When the training set and the test set are received, two initial image generation models and two initial image adversarial models are constructed.

[0008] The training set and the test set are used to perform bidirectional training and model validation on all the initial image generation models and all the initial image adversarial models to determine the target image generation model; wherein, the target image generation model includes an encoder, a receptive field residual module, an illumination attention module, and a decoder;

[0009] When a target transmission line monitoring image is received, the encoder is used to perform multi-layer feature extraction on the target transmission line monitoring image, and outputs a preliminary feature map, a cascaded feature map and an encoded feature map;

[0010] The residual feature map is generated by extracting features from the encoded feature map through the receptive field residual module.

[0011] The residual feature map is input into the illumination attention module for feature weighting to construct a weighted feature map;

[0012] The illumination attention module and the decoder are used to perform multi-scale feature extraction and feature fusion on the preliminary feature map, the cascaded feature map and the weighted feature map to determine the target enhancement image.

[0013] Optionally, the test set includes low-light test images; the step of using the training set and the test set to perform bidirectional training and model validation on all the initial image generation models and all the initial image adversarial models to determine the target image generation model; wherein, the target image generation model includes an encoder, a receptive field residual module, an illumination attention module, and a decoder, comprising:

[0014] The training set is used to perform bidirectional training on all the initial image generation models and all the initial image adversarial models to determine the total loss function value;

[0015] If the total loss function value does not converge, then the gradient descent method is used to update the model parameters of all the initial image generation models and all the initial image adversarial models, and then the process jumps to the step of using the training set to perform bidirectional training on all the initial image generation models and all the initial image adversarial models to determine the total loss function value.

[0016] If the total loss function value converges, the optimized image generation model is determined based on the current model parameters;

[0017] The low-light test image is input into the optimized image generation model for image enhancement, and the enhanced test image is output.

[0018] Calculate the natural image quality evaluation index and the no-reference image evaluation index based on meta-learning for the enhanced test image;

[0019] When the natural image quality evaluation index value is less than the first preset ideal value, and the meta-learning-based no-reference image evaluation index value is greater than the second preset ideal value, a target image generation model is determined; wherein, the target image generation model includes an encoder, a receptive field residual module, an illumination attention module, and a decoder.

[0020] Optionally, the training set includes normal lighting training images and low lighting training images; the two initial image generation models include a forward image generation model and a reverse image generation model; the two initial image adversarial models include a forward image adversarial model and a reverse image adversarial model; the step of using the training set to perform bidirectional training on all the initial image generation models and all the initial image adversarial models to determine the total loss function value includes:

[0021] The low-light training image and the normal-light training image are respectively input into the forward image generation model and the reverse image generation model for image enhancement, and the first enhanced image and the second enhanced image are output.

[0022] The first enhanced image and the second enhanced image are respectively input into the reverse image generation model and the forward image generation model to perform image transformation, and a first reconstructed image and a second reconstructed image are constructed.

[0023] The positive image adversarial model performs image discrimination based on the first enhanced image and the normal light training image to generate a first discrimination result;

[0024] The reverse image adversarial model is used to perform image discrimination on the second enhanced image and the low-light training image, and a second discrimination result is output.

[0025] Using the low-light training image, the normal-light training image, the first enhanced image, the second enhanced image, the first reconstructed image, the second reconstructed image, the first discrimination result, and the second discrimination result, calculate the adversarial loss function value, the cycle consistency loss function value, the identity consistency loss function value, and the unsupervised perception loss function value;

[0026] The total loss function value is determined by adding the adversarial loss function value, the cycle consistency loss function value, the identity consistency loss function value, and the unsupervised perception loss function value.

[0027] Optionally, the initial image adversarial model includes convolutional layers with LeakyReLU activation functions, downsampling layers, upsampling layers, and ordinary convolutional layers; the execution process of the initial image adversarial model includes:

[0028] The adversarial input feature image of the input adversarial model is used to extract local features through a convolutional layer with a LeakyReLU activation function to generate a first adversarial map;

[0029] The first adversarial graph is constructed by performing max pooling downsampling and convolution operations on the first adversarial graph using cascaded downsampling layers and convolutional layers with LeakyReLU activation function;

[0030] After max pooling downsampling is performed on the second adversarial graph through a downsampling layer, continuous feature extraction is performed using multiple convolutional layers with LeakyReLU activation function to output the third adversarial graph;

[0031] After the third adversarial graph is input into the upsampling layer and upsampled by bilinear interpolation, it is added element-wise with the second adversarial graph to determine the fourth adversarial graph.

[0032] After performing convolutional filtering and bilinear interpolation upsampling on the fourth adversarial graph based on cascaded convolutional layers and upsampling layers with LeakyReLU activation function, it is added element-wise to the first adversarial graph to generate the fifth adversarial graph.

[0033] After continuously extracting features from the fifth adversarial graph using multiple convolutional layers with LeakyReLU activation functions, image discrimination is performed using ordinary convolutional layers to determine the discrimination result.

[0034] Optionally, the encoder includes a convolutional layer with a LeakyReLU activation function, a contextual feature extraction module, and a cascaded module; the step of using the encoder to perform multi-layer feature extraction on the target transmission line monitoring image when a target transmission line monitoring image is received, and outputting a preliminary feature map, a cascaded feature map, and an encoded feature map, includes:

[0035] When a monitoring image of the target transmission line is received, a convolutional layer with the LeakyReLU activation function is used to perform preliminary feature extraction on the monitoring image of the target transmission line to generate a preliminary feature map;

[0036] The context information of the preliminary feature map is extracted by the context feature extraction module to construct a deep feature map.

[0037] The deep feature map is input into the cascade module for downsampling and refined feature extraction, and the cascaded feature map is output.

[0038] A cascaded module is used to downsample and extract global features from the cascaded feature map to generate an encoded feature map; wherein, the cascaded module includes a cascaded residual D module and a context feature extraction module.

[0039] Optionally, the context feature extraction module includes a first convolutional layer, a second convolutional layer, and a third convolutional layer, wherein both the first and second convolutional layers have a LeakyReLU activation function; the image processing procedure of the context feature extraction module includes:

[0040] The context input feature map input to the context feature extraction module is subjected to a first convolutional layer for feature extraction to generate a first feature map;

[0041] The first feature map is convolved by a second convolutional layer and multiple cascaded second convolutional layers to construct a second feature map and a third feature map respectively.

[0042] After performing channel dimensionality reduction on the context input feature map using a third convolutional layer, it is added element-wise to the second feature map and the third feature map to output the context output feature map.

[0043] Optionally, the receptive field residual module includes a first dilated convolutional layer, a second dilated convolutional layer, and a third dilated convolutional layer with multi-level residual connections, as well as a standard convolutional layer; the step of extracting features from the encoded feature map through the receptive field residual module to generate a residual feature map includes:

[0044] The encoded feature map is subjected to dilated convolution filtering using a first dilated convolution layer to output a first dilated map;

[0045] The encoded feature map is added element-wise to the first hole map to generate a second hole map;

[0046] After extracting features from the second dilated convolutional layer, the third dilated map is constructed by adding it element-wise to the encoded feature map and the second dilated map.

[0047] The third hole map is input into the third hole convolutional layer for hole convolution operation to determine the fourth hole map;

[0048] The encoded feature map is reduced in channel dimension using a standard convolutional layer, and then added element-wise with the second and fourth hole maps to output a residual feature map.

[0049] Optionally, the illumination attention module includes a cascaded channel attention module and a spatial attention module; the channel attention module includes an average pooling layer, a max pooling layer, a first channel convolutional layer with a LeakyReLU activation function, a second channel convolutional layer, and a sigmoid function; the image processing procedure of the illumination attention module includes:

[0050] The illumination input feature map input to the illumination attention module is subjected to average pooling and max pooling operations respectively using the average pooling layer and the max pooling layer to generate a first channel map and a second channel map respectively.

[0051] The first channel image and the second channel image are added element by element to construct the third channel image;

[0052] The first channel image, the second channel image, and the third channel image are extracted in parallel using cascaded first channel convolutional layers and second channel convolutional layers respectively, and then the fourth channel image is output by adding them element by element.

[0053] The channel feature weights of the fourth channel map are extracted using the Sigmoid function, and the channel feature weights are multiplied element-wise with the illumination input feature map to determine the channel output feature map.

[0054] The channel output feature map is input into the spatial attention module to deepen the spatial weights, and the illumination output feature map is output.

[0055] Optionally, the spatial attention module includes an average pooling layer, a max pooling layer, a first spatial convolutional layer, a second spatial convolutional layer, a third spatial convolutional layer, a fourth spatial convolutional layer, and a sigmoid function, wherein the first spatial convolutional layer, the second spatial convolutional layer, and the third spatial convolutional layer all have a LeakyReLU activation function; the step of inputting the channel output feature map into the spatial attention module to deepen the spatial weights and outputting an illumination output feature map includes:

[0056] After performing average pooling and max pooling operations on the channel output feature map respectively, the channels are stitched together to generate the first spatial map.

[0057] The first spatial graph is input into a cascaded first spatial convolutional layer and a fourth spatial convolutional layer for feature extraction to construct a second spatial graph;

[0058] The first spatial graph is subjected to convolution operation by a cascaded second spatial convolutional layer and a fourth spatial convolutional layer to output a third spatial graph;

[0059] The first spatial graph is convolutionally filtered by cascaded third and fourth spatial convolutional layers to determine the fourth spatial graph.

[0060] The second spatial diagram, the third spatial diagram, and the fourth spatial diagram are added element by element to generate the fifth spatial diagram;

[0061] The spatial feature weights are determined by activating the fifth spatial map using the Sigmoid function, and then the spatial feature weights are multiplied element-wise with the channel output feature map to output the illumination output feature map.

[0062] Optionally, the decoder includes an upsampling layer, a convolutional layer with a LeakyReLU activation function, a contextual feature extraction module, and a convolutional layer with a Tanh activation function; the step of using the illumination attention module and the decoder to perform multi-scale feature extraction and feature fusion on the preliminary feature map, the cascaded feature map, and the weighted feature map to determine the target enhanced image includes:

[0063] After the illumination attention module performs feature weighting on the preliminary feature map and the cascaded feature map respectively, it outputs the first illumination map and the second illumination map accordingly.

[0064] After the weighted feature map is input into the upsampling layer and upsampled by bilinear interpolation, it is concatenated with the second illumination map to generate the first decoded map.

[0065] The first decoded image is constructed by using a cascaded convolutional layer with LeakyReLU activation function, a context feature extraction module and an upsampling layer to perform channel compression, feature extraction and bilinear interpolation upsampling, and then channel-by-channel concatenation with the first illumination image.

[0066] The second decoded map is subjected to channel dimensionality reduction and multi-scale feature extraction by cascaded convolutional layers with LeakyReLU activation function and context feature extraction module to determine the third decoded map;

[0067] The third decoded image is subjected to feature extraction mapping using a convolutional layer with Tanh activation function, and the target enhancement image is output.

[0068] A second aspect of the present invention provides a transmission line monitoring image enhancement device, comprising:

[0069] The model building module is used to build two initial image generation models and two initial image adversarial models when the training set and test set are received.

[0070] The model training module is used to perform bidirectional training and model validation on all the initial image generation models and all the initial image adversarial models using the training set and the test set to determine the target image generation model; wherein, the target image generation model includes an encoder, a receptive field residual module, an illumination attention module, and a decoder;

[0071] The encoder processing module is used to perform multi-layer feature extraction on the target transmission line monitoring image using the encoder when the target transmission line monitoring image is received, and output a preliminary feature map, a cascaded feature map and an encoded feature map.

[0072] The residual feature map generation module is used to extract features from the encoded feature map through the receptive field residual module to generate a residual feature map;

[0073] The weighted feature map construction module is used to input the residual feature map into the illumination attention module for feature weighting and to construct a weighted feature map.

[0074] The target enhancement image output module is used to perform multi-scale feature extraction and feature fusion on the preliminary feature map, the cascaded feature map and the weighted feature map using the illumination attention module and the decoder to determine the target enhancement image.

[0075] As can be seen from the above technical solutions, the present invention has the following advantages:

[0076] This invention utilizes received training and testing sets to bidirectionally train and validate two initial image generation models and two initial image adversarial models, thereby determining the target image generation model. When a target transmission line monitoring image is received, the encoder in the target image generation model performs multi-layer feature extraction on the image, outputting a preliminary feature map, a cascaded feature map, and an encoded feature map. The encoded feature map is then input into a cascaded receptive field residual module and an illumination attention module for feature extraction and weighting, constructing a weighted feature map. The illumination attention module and decoder then perform multi-scale feature extraction and feature fusion on the preliminary feature map, the cascaded feature map, and the weighted feature map to determine the target enhanced image. The enhanced image obtained through this target image generation model improves the clarity of image texture details while avoiding overexposure or underexposure, resulting in better image quality and providing more accurate and reliable image information for transmission line defect detection. Attached Figure Description

[0077] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0078] Figure 1 This is a flowchart of the steps of a transmission line monitoring image enhancement method provided in Embodiment 1 of the present invention;

[0079] Figure 2 This is a flowchart of the steps of a transmission line monitoring image enhancement method provided in Embodiment 2 of the present invention;

[0080] Figure 3 This is a diagram illustrating the overall framework for model training provided in Embodiment 2 of the present invention.

[0081] Figure 4 This is a schematic diagram of the image generation model and residual D module provided in Embodiment 2 of the present invention;

[0082] Figure 5 This is a schematic diagram of the context feature extraction module provided in Embodiment 2 of the present invention;

[0083] Figure 6 This is a schematic diagram of the receptive field residual module provided in Embodiment 2 of the present invention;

[0084] Figure 7 This is a schematic diagram of the illumination attention module provided in Embodiment 2 of the present invention;

[0085] Figure 8 This is a schematic diagram of the image adversarial model provided in Embodiment 2 of the present invention;

[0086] Figure 9 This is a structural block diagram of a power transmission line monitoring image enhancement device provided in Embodiment 3 of the present invention. Detailed Implementation

[0087] This invention provides a method and apparatus for enhancing transmission line monitoring images, which addresses the technical problem that existing image enhancement technologies produce low-quality enhanced images when processing transmission line monitoring images.

[0088] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0089] Please see Figure 1 , Figure 1 This is a flowchart illustrating the steps of a transmission line monitoring image enhancement method provided in Embodiment 1 of the present invention.

[0090] This invention provides a method for enhancing monitoring images of power transmission lines, comprising:

[0091] Step 101: When the training set and test set are received, construct two initial image generation models and two initial image adversarial models.

[0092] The training set refers to the collection of power transmission line monitoring images used to train the model.

[0093] The test set refers to the collection of power transmission line monitoring images used to test the trained model.

[0094] An initial image generation model refers to an image generation model that uses initial model parameters. This model can be used to enhance images.

[0095] An initial image adversarial model refers to an image adversarial model that uses initial model parameters. Image adversarial models can be used to determine the degree of consistency in the distribution of image data.

[0096] In this embodiment of the invention, any demand-side platform that supports the application of the transmission line monitoring image enhancement method constructs a training set and a test set by performing image preprocessing on the transmission line monitoring images collected from the overhead transmission line monitoring system. When the training set and test set are received from the demand-side platform, an initial image generation model and an initial image adversarial model are constructed according to the preset initial model parameters.

[0097] Optionally, the training set includes normal light training images and low light training images, and the test set includes low light test images. The process of constructing the training and test sets by preprocessing the transmission line monitoring images acquired from the overhead transmission line monitoring system includes:

[0098] Multiple sets of constant light sample images of different scenarios are collected by the overhead transmission line monitoring system. All constant light sample images are cropped according to the preset size to generate constant light cropped sample images.

[0099] According to the first preset ratio, all normal light cropping sample images are divided into normal light training images and normal light cropping sample images to be processed.

[0100] By changing the pixel values ​​of the normal light cropping sample image to be processed, the corresponding low light cropping sample image is determined.

[0101] All low-light cropped sample images are divided into low-light training images and low-light test images according to the second preset ratio.

[0102] The training set consists of normal light training images and low light training images, and the low light test images are used as the test set.

[0103] Normal light sample images refer to transmission line monitoring images directly acquired from overhead transmission line monitoring systems. Normal light training images refer to transmission line monitoring images used for training under normal lighting conditions. Low light training images refer to transmission line monitoring images used for training under low lighting conditions. Low light test images refer to transmission line monitoring images used for testing under low lighting conditions.

[0104] Optionally, the preset size is set to 600×400. It is understood that the first and second preset ratios can be specifically set according to the number of images required for model training and testing. For example: 8000 sets of normal light sample images for different scenarios are selected from the transmission line monitoring images collected by the overhead transmission line monitoring system, and all normal light sample images are cropped into normal light cropped sample images of size 600×400; 5000 sets of normal light cropped sample images are used as normal light cropped sample images to be processed, and low light images are simulated by changing pixel values, serving as low light cropped sample images; 4000 sets of low light cropped sample images and another 3000 sets of normal light cropped sample images are selected to form the training set, and the remaining 1000 sets of low light cropped sample images are used as the test set.

[0105] Preferably, the corresponding low-light cropping sample image is determined by changing the pixel values ​​of the normal light cropping sample image to be processed using the following formula:

[0106]

[0107] Where α is the brightness scaling factor, I in For the input image; I out The generated low-light nighttime image uses γ as a brightness scaling factor.

[0108] It is understandable that α can take the value 1, and γ can be set in the range of [0.6, 0.8].

[0109] Step 102: Use the training set and test set to perform bidirectional training and model validation on all initial image generation models and all initial image adversarial models to determine the target image generation model; wherein, the target image generation model includes an encoder, a receptive field residual module, an illumination attention module, and a decoder.

[0110] A target image generation model refers to an image generation model obtained by training and iteratively optimizing an initial image generation model to achieve a predetermined goal and then validating the model.

[0111] In this embodiment of the invention, after bidirectional training of all initial image generation models and all image adversarial models using the training set, the test set is input into the trained image generation model for model verification. When the verification reaches the predetermined target, the target image generation model is determined. The target image generation model includes an encoder, a receptive field residual module, an illumination attention module, and a decoder.

[0112] Step 103: When the target transmission line monitoring image is received, the encoder is used to perform multi-layer feature extraction on the target transmission line monitoring image, and outputs a preliminary feature map, a cascaded feature map and an encoded feature map.

[0113] Target transmission line monitoring images refer to the monitoring images of transmission lines in a target scene collected by an overhead transmission line monitoring system.

[0114] In this embodiment of the invention, when a target transmission line monitoring image is received, the target transmission line monitoring image and the image after feature extraction of the target transmission line monitoring image are processed by the encoder in the target image generation model. This process can obtain the preliminary feature map, cascaded feature map and coded feature map output by the encoder in the multi-layer feature extraction.

[0115] Step 104: Extract features from the encoded feature map using the receptive field residual module to generate a residual feature map.

[0116] In this embodiment of the invention, the receptive field residual module in the target image generation model is connected in series with the encoder. The encoded feature map output by the encoder is then input into the receptive field residual module. The receptive field residual module extracts features from the encoded feature map to generate a residual feature map.

[0117] Step 105: Input the residual feature map into the illumination attention module for feature weighting to construct a weighted feature map.

[0118] In this embodiment of the invention, the receptive field residual module and the illumination attention module in the target image generation model are cascaded. After the receptive field residual module generates a residual feature map, it is input into the cascaded illumination attention module for feature weighting to construct an output weighted feature map.

[0119] Step 106: Use the illumination attention module and decoder to perform multi-scale feature extraction and feature fusion on the preliminary feature map, cascaded feature map and weighted feature map to determine the target enhancement image.

[0120] Target-enhanced images refer to images that have been enhanced from monitoring images of target power transmission lines using a target image generation model.

[0121] In this embodiment of the invention, the decoder in the target image generation model is connected in series with the illumination attention module cascaded with the receptive field residual module. At the same time, the encoder in the target image generation model is also connected to the decoder via multiple illumination attention modules. After the illumination attention module outputs the weighted feature map, the decoder performs image processing on the weighted feature map. Then, the image is combined with the image after the illumination attention module has processed the preliminary feature map and the cascaded feature map. After performing skip feature fusion and multi-scale feature extraction, the target enhancement image is determined.

[0122] In this embodiment of the invention, the received training and test sets are used to perform bidirectional training and model validation on two initial image generation models and two initial image adversarial models to determine the target image generation model. When a target transmission line monitoring image is received, the encoder in the target image generation model performs multi-layer feature extraction on the target transmission line monitoring image, outputting a preliminary feature map, a cascaded feature map, and an encoded feature map. The encoded feature map is input into a cascaded receptive field residual module and an illumination attention module for feature extraction and feature weighting to construct a weighted feature map. The illumination attention module and decoder then perform multi-scale feature extraction and feature fusion on the preliminary feature map, the cascaded feature map, and the weighted feature map to determine the target enhanced image. The enhanced image obtained through this target image generation model improves the clarity of image texture details while avoiding overexposure or underexposure, resulting in better image quality and providing more accurate and reliable image information for transmission line defect detection.

[0123] Please see Figure 2 , Figure 2 This is a flowchart illustrating the steps of a transmission line monitoring image enhancement method provided in Embodiment 2 of the present invention.

[0124] This invention provides a method for enhancing monitoring images of power transmission lines, comprising:

[0125] Step 201: When the training set and test set are received, construct two initial image generation models and two initial image adversarial models.

[0126] In this embodiment of the invention, the specific implementation process of step 201 is similar to that of step 101, and will not be repeated here.

[0127] Step 202: Use the training set to train both the initial image generation model and the initial image adversarial model bidirectionally to determine the total loss function value.

[0128] Optionally, the training set includes normal lighting training images and low lighting training images; the two initial image generation models include a forward image generation model and a reverse image generation model; the two initial image adversarial models include a forward image adversarial model and a reverse image adversarial model; step 202 includes the following sub-steps:

[0129] Low-light training images and normal-light training images are input into the forward image generation model and the reverse image generation model, respectively, for image enhancement, and the first enhanced image and the second enhanced image are output.

[0130] The first and second enhancement maps are input into the reverse image generation model and the forward image generation model, respectively, to perform image transformation and construct the first and second reconstruction maps.

[0131] Image discrimination is performed based on the first enhanced image and the normal light training image using a positive image adversarial model, generating the first discrimination result;

[0132] The second enhanced image and the low-light training image are discriminated using an inverse image adversarial model, and the second discrimination result is output.

[0133] Using low-light training images, normal-light training images, first augmented images, second augmented images, first reconstructed images, second reconstructed images, first discrimination results, and second discrimination results, we calculate the adversarial loss function value, cycle consistency loss function value, identity consistency loss function value, and unsupervised perception loss function value.

[0134] The total loss function value is determined by adding the adversarial loss function value, the cycle consistency loss function value, the identity consistency loss function value, and the unsupervised perception loss function value.

[0135] In this embodiment of the invention, reference is made to Figure 3 The two initial image generation models are divided into a forward image generation model and a reverse image generation model, and the two initial image adversarial models are also divided into a forward image adversarial model and a reverse image adversarial model. Low-light training images are selected from the training set and input into the forward image generation model for image enhancement, outputting the first enhanced image. Normal-light training images and the first enhanced image are selected from the training set. Figure 1 The first image is input into the forward image adversarial model for image discrimination, generating a first discrimination result. Normal lighting training images from the training set are selected, and a second enhanced image is generated by processing these images using a reverse image generation model. Low-light training images and the second enhanced image from the training set are then input into the reverse image adversarial model for image discrimination, outputting a second discrimination result. The first enhanced image is then transformed using the reverse image generation model to generate a first reconstructed image. The second enhanced image is then transformed using the forward image generation model to construct a second reconstructed image. Combining the low-light training image, normal lighting training image, first enhanced image, second enhanced image, first discrimination result, and second discrimination result, the adversarial loss function, cycle consistency loss function, identity consistency loss function, and unsupervised perception loss function are calculated. The calculated loss function values ​​are then added together to determine the total loss function value.

[0136] Optionally, the process of determining the total loss function value includes:

[0137] L = L Adv (G X→Y D X→Y ,X,Y)+L Adv (G Y→X D Y→X (,X,Y)+

[0138] LCyc (G X→Y G Y→X )+L Identity (G X→Y G Y→X )+L Perceptual (G X→Y G Y→X );

[0139] Where L is the total loss function, L Adv () represents the adversarial loss function, L Cyc () is the cycle consistency loss function, L Identity () is the identity consistency loss function, L Perceptual () represents the unsupervised perception loss function.

[0140] The calculation process for the adversarial loss function value includes:

[0141]

[0142]

[0143] Where x is the low-light training image, X is the image domain of the low-light training image, y is the normal-light training image, Y is the image domain of the normal-light training image, and G... X→Y For a forward image generation model, D X→Y For a positive image adversarial model, D X→Y (y) represents the discrimination result obtained by using the normal light training image through the positive image adversarial model, G X→Y (x) represents the enhanced image output by the positive image generation model from the low-light training image, D. X→Y (G X→Y (x) represents the discrimination result obtained by the enhanced image corresponding to the low-light training image through the positive image adversarial model, G Y→X For the inverse image generation model, D Y→X For reverse image adversarial models, D Y→X (x) represents the discrimination result obtained from the low-light training image through the inverse image adversarial model, G Y→X (y) represents the enhanced image output by the inverse image generation model from the constant light training image, D. Y→X (G Y→X (y) represents the discrimination result obtained by using the inverse image adversarial model on the enhanced image corresponding to the normal light training image. data(x) P represents the sample distribution of the low-light training images. data(y) Let E be the sample distribution of the training images of Changguang, and E be the mathematical expectation.

[0144] The calculation process of the cycle consistency loss function value includes:

[0145]

[0146] Among them, G Y→X (G X→Y (x) represents the reconstructed image obtained by using the inverse image generation model from the enhanced image corresponding to the low-light training image, G. X→Y (G Y→X (y) is the reconstructed image obtained by using the enhanced image corresponding to the normal light training image through the forward image generation model.

[0147] The calculation process of the identity consistency loss function value includes:

[0148]

[0149] The calculation process of the unsupervised perceptual loss function includes:

[0150]

[0151] Where W is the image width, H is the image height, i is the i-th column pixel of the enhanced image and the training image, and j is the j-th row pixel of the enhanced image and the training image. This is the first convolutional layer after the fifth pooling layer in the trained fully convolutional VGG16 feature extraction model.

[0152] It's understandable that the fully convolutional VGG16 feature extraction model refers to a VGG16 feature extraction model based on the regular VGG16 model, where the last three fully connected layers are replaced with convolutional layers. After obtaining a trained fully convolutional VGG16 feature extraction model through ImageNet pre-training, the model is used to extract features from low-light training images, normal-light training images, the first augmented image, and the second augmented image, respectively, outputting multiple perceptual feature maps. These perceptual feature maps are used to calculate the unsupervised perceptual loss function.

[0153] Optionally, the initial image adversarial model includes convolutional layers with LeakyReLU activation functions, downsampling layers, upsampling layers, and ordinary convolutional layers; the execution process of the initial image adversarial model includes:

[0154] The adversarial input feature image of the adversarial model is processed by a convolutional layer with LeakyReLU activation function to extract local features and generate the first adversarial map.

[0155] A second adversarial graph is constructed by performing max pooling downsampling and convolution operations on the first adversarial graph using cascaded downsampling layers and convolutional layers with LeakyReLU activation function.

[0156] After max pooling downsampling the second adversarial graph through a downsampling layer, continuous feature extraction is performed using multiple convolutional layers with LeakyReLU activation function to output the third adversarial graph;

[0157] After the third adversarial graph is input into the upsampling layer and upsampled by bilinear interpolation, it is added element-wise with the second adversarial graph to determine the fourth adversarial graph.

[0158] The fourth adversarial graph is generated by performing convolutional filtering and bilinear interpolation upsampling on a cascaded convolutional and upsampling layer with LeakyReLU activation function, and then adding it element-wise to the first adversarial graph.

[0159] After extracting continuous features from the fifth adversarial map using multiple convolutional layers with LeakyReLU activation function, image discrimination is performed using ordinary convolutional layers to determine the discrimination result.

[0160] Adversarial input feature maps refer to the feature maps of an adversarial model that inputs an image. In essence, they can be understood as feature maps that are input as image pairs into the adversarial model during model training for image discrimination.

[0161] The first adversarial graph, the second adversarial graph, the third adversarial graph, the fourth adversarial graph, and the fifth adversarial graph refer to intermediate graphs generated in the image adversarial model.

[0162] In this embodiment of the invention, reference is made to Figure 8The initial image adversarial model includes convolutional layers with LeakyReLU activation, downsampling layers, upsampling layers, and ordinary convolutional layers, including a multi-scale discriminant module composed of these layers. The adversarial input feature image input to the initial image adversarial model first undergoes multi-scale discrimination and feature fusion by the multi-scale discriminant module, outputting a fifth adversarial map. This fifth adversarial map is then subjected to three consecutive feature extractions through multiple layers of convolutional layers with LeakyReLU activation, followed by image discrimination using ordinary convolutional layers to determine the discrimination result. The steps for outputting the fifth adversarial map after the adversarial input feature image undergoes multi-scale discrimination and feature fusion by a multi-scale discrimination module include: first, local feature extraction is performed on the adversarial input feature image through a convolutional layer with a LeakyReLU activation function to generate a first adversarial map; second, the first adversarial map is downsampled by a downsampling layer using max pooling, and then convolved by a convolutional layer with a LeakyReLU activation function to construct a second adversarial map; next, the second adversarial map is input into a downsampling layer for max pooling downsampling, and then feature extraction is performed three times consecutively through multiple convolutional layers with LeakyReLU activation functions to output a third adversarial map; then, the third adversarial map is upsampled by a bilinear interpolation layer, and then element-wise added to the second adversarial map with skip connections to determine a fourth adversarial map; finally, the fourth adversarial map is input into a convolutional layer with a LeakyReLU activation function for convolutional filtering, then upsampled by a bilinear interpolation layer, and then element-wise added to the first adversarial map with skip connections to generate the fifth adversarial map.

[0163] Step 203: If the total loss function value does not converge, then use gradient descent to update the model parameters of all initial image generation models and all initial image adversarial models, and jump to the step of bidirectional training of all initial image generation models and all initial image adversarial models using the training set to determine the total loss function value.

[0164] In this embodiment of the invention, the convergence of the total loss function value obtained in each training is judged. If the total loss function value does not converge, the model parameters of all initial image generation models and all initial image adversarial models are updated by gradient descent based on the gradient of the loss function value contained in the total loss function value, and then the process jumps to step 202.

[0165] Understandably, the specific process of updating model parameters using gradient descent can be found in existing techniques and will not be elaborated here.

[0166] Step 204: If the total loss function value converges, then the optimized image generation model is determined based on the current model parameters.

[0167] Optimizing an image generation model refers to training an initial image generation model and iteratively optimizing it until the total loss function value converges, resulting in the final image generation model.

[0168] In this embodiment of the invention, if the total loss function value obtained in the current training converges, the optimized image generation model and the optimized image adversarial model are determined based on the current model parameters.

[0169] Step 205: Input the low-light test image into the optimized image generation model for image enhancement, and output the enhanced test image.

[0170] In this embodiment of the invention, after the optimized image generation model is determined by the convergence of the total loss function value, the low-light test images in the test set are input into the optimized image generation model for image processing, and the corresponding enhanced test images are output.

[0171] Step 206: Calculate the natural image quality evaluation index value and the meta-learning-based no-reference image evaluation index value of the enhanced test image.

[0172] In this embodiment of the invention, for the enhanced test image corresponding to the low-light test image, the Natural Image Quality Evaluation (NIQE) index algorithm is used to calculate the Natural Image Quality Evaluation (NIQE) index value of the enhanced test image, and the Meta-learning-based No-Reference Image Evaluation (MetaIQA) index algorithm is used to calculate the Meta-learning-based No-Reference Image Evaluation (MetaIQA) index value of the enhanced test image, so as to verify the optimized image generation model.

[0173] It is understandable that both the Natural Image Quality Evaluation (NIQE) metric and the Meta-learning-based No-Reference Image Evaluation (MetaIQA) metric are evaluations of the quality of a single image. For details, please refer to existing technologies, which will not be elaborated here.

[0174] Step 207: When the natural image quality evaluation index value is less than the first preset ideal value, and the evaluation index value of the no-reference image based on meta-learning is greater than the second preset ideal value, determine the target image generation model; wherein, the target image generation model includes an encoder, a receptive field residual module, an illumination attention module, and a decoder.

[0175] The first preset ideal value refers to the ideal value set for the Natural Image Quality Evaluation (NIQE) index.

[0176] The second preset ideal value refers to the ideal value set for the Meta-learning-based no-reference image evaluation (MetaIQA) metric.

[0177] In this embodiment of the invention, the image quality index value of the enhanced test image is compared with the corresponding ideal value to determine whether the natural image quality evaluation index value is less than the first preset ideal value and the no-reference image evaluation index value based on meta-learning is greater than the second preset ideal value. If yes, the target image generation model is determined, wherein the target image generation model includes an encoder, a receptive field residual module, an illumination attention module, and a decoder. If no, the model parameters of the optimized image generation model are adjusted and optimized, and the updated optimized image generation model is used as the new initial image generation model, and the optimized image adversarial model is used as the new initial image adversarial model. Then, step 202 is executed to continue training the model.

[0178] Step 208: When the target transmission line monitoring image is received, the encoder is used to perform multi-layer feature extraction on the target transmission line monitoring image, and outputs a preliminary feature map, a cascaded feature map and an encoded feature map.

[0179] Optionally, the encoder includes convolutional layers with LeakyReLU activation, a contextual feature extraction module, and a cascaded module; step 208 includes the following sub-steps:

[0180] When a monitoring image of the target transmission line is received, a convolutional layer with the LeakyReLU activation function is used to perform preliminary feature extraction on the monitoring image of the target transmission line to generate a preliminary feature map;

[0181] The contextual feature extraction module extracts contextual information from the preliminary feature map to construct a deep feature map.

[0182] The deep feature map is input into the cascaded module for downsampling and refined feature extraction, and the cascaded feature map is output.

[0183] A cascaded module is used to downsample and extract global features from the cascaded feature map to generate an encoded feature map; the cascaded module includes a cascaded residual D module and a context feature extraction module.

[0184] Preferably, the context feature extraction module includes a first convolutional layer, a second convolutional layer, and a third convolutional layer, wherein both the first and second convolutional layers have the LeakyReLU activation function; the image processing procedure of the context feature extraction module includes:

[0185] The context input feature map of the input context feature extraction module is used to extract features using the first convolutional layer to generate the first feature map;

[0186] The first feature map is convolved by the second convolutional layer and multiple cascaded second convolutional layers respectively, thereby constructing the second and third feature maps.

[0187] After performing channel dimensionality reduction on the context input feature map using the third convolutional layer, it is added element-wise with the second and third feature maps to output the context output feature map.

[0188] The context input feature map refers to the feature map input to the context feature extraction module, which can correspond to the feature map processed by any of the context feature extraction modules during model training. The first, second, and third feature maps refer to the intermediate maps generated in the context feature extraction module. The context output feature map refers to the feature map output by the context feature extraction module, which can correspond to the feature map output after image processing by the context feature extraction module during model training.

[0189] It is understandable that the context feature extraction module includes three extraction branches, each consisting of a convolutional layer with different kernels. The first extraction branch includes a first convolutional layer and multiple second convolutional layers, the second extraction branch includes a second convolutional layer, and the third extraction branch includes a third convolutional layer.

[0190] Furthermore, the residual D module includes a first residual convolutional layer, a second residual convolutional layer, and an average pooling layer; the image processing procedure of the residual D module includes:

[0191] The residual input feature map of the input residual D module is used to extract features by cascading first residual convolutional layer, second residual convolutional layer and first residual convolutional layer to construct the first residual map;

[0192] After performing average pooling on the input feature map through the average pooling layer, the first residual convolutional layer is used to perform convolution operation to generate the second residual map.

[0193] The first residual map and the second residual map are concatenated by channels to output the residual output feature map.

[0194] The residual input feature map refers to the feature map input to the residual D module, which can be considered as the feature map of any input residual D module after image processing during model training. The first residual map and the second residual map refer to the intermediate maps generated in the residual D module. The residual output feature map refers to the feature map output by the residual D module, which can be considered as the feature map output by the residual D module after image processing during model training.

[0195] In this embodiment of the invention, reference is made to Figure 4-5When a monitoring image of a target transmission line is received, a convolutional layer with a kernel of 3 and a Leaky ReLU activation function can be used to initially extract features from the image. The number of channels N is increased from 3 to 32. The generated preliminary feature map is then input into the first context feature extraction module. In the first context feature extraction module, a 1×1 first convolutional layer is used to extract features from the preliminary feature map to generate a first feature map. The first feature map is then convolved with a 3×3 second convolutional layer and multiple cascaded 3×3 second convolutional layers to construct the second and third feature maps. The preliminary feature map is then input into a 1×1 third convolutional layer for channel dimensionality reduction, and then element-wise added to the second and third feature maps to output a deep feature map. The number of channels is increased from 32 to 64.

[0196] The deep feature map is input into the residual D module of the first cascaded module. In the residual D module, a cascaded 1×1 first residual convolutional layer, a 3×3 second residual convolutional layer with a stride of 2, and a 1×1 first residual convolutional layer are used to extract features from the deep feature map, constructing the first residual map. After average pooling, the deep feature map is convolved by an average pooling layer, followed by convolution by the 1×1 first residual convolutional layer to generate the second residual map. The first and second residual maps are concatenated and then input into the context feature extraction module of the first cascaded module to obtain receptive fields of different sizes, outputting a cascaded feature map. The feature map size is halved, and the number of channels increases to 128. The residual D module and the context feature extraction module of the second cascaded module are used to downsample and extract global features from the cascaded feature map to generate an encoded feature map. The feature map size is reduced to one-quarter of the original, and the number of channels increases to 256.

[0197] Step 209: Extract features from the encoded feature map using the receptive field residual module to generate a residual feature map.

[0198] Optionally, the receptive field residual module includes a first dilated convolutional layer, a second dilated convolutional layer, and a third dilated convolutional layer with multi-level residual connections, as well as a standard convolutional layer; step 209 includes the following sub-steps:

[0199] The encoded feature map is subjected to dilated convolutional filtering using the first dilated convolutional layer to output the first dilated map.

[0200] The encoded feature map is added element-wise to the first hole map to generate the second hole map;

[0201] After extracting features from the second dilated convolutional layer, the third dilated map is constructed by adding it element-wise to the encoded feature map and the second dilated map.

[0202] The third hole map is input into the third hole convolutional layer for hole convolution operation to determine the fourth hole map;

[0203] The encoded feature map is reduced in dimensionality by channel using a standard convolutional layer, and then added element-wise with the second and fourth dilated maps to output a residual feature map.

[0204] It is understandable that the first, second, and third holed convolutional layers are holed convolutional layers with different void ratios, and their void ratios increase in sequence.

[0205] In this embodiment of the invention, reference is made to Figure 6 For the encoded feature map of the input receptive field residual module, a 3×3 first dilated convolutional layer with a dilation rate of 1 is used for dilated convolution filtering to output a first dilated map. The encoded feature map and the first dilated map are then fused element-wise to generate a second dilated map. Features are extracted from the second dilated map using a 3×3 second dilated convolutional layer with a dilation rate of 2, and then element-wise added to the skip-connected encoded feature map and the second dilated map to construct a third dilated map. A 3×3 third dilated convolutional layer with a dilation rate of 4 is used to perform dilated convolution operations on the third dilated map to determine a fourth dilated map. A 1×1 standard convolutional layer is used to reduce the channel dimensionality of the encoded feature map, and then element-wise added to the skip-connected second and fourth dilated maps to output a residual feature map. The receptive field residual module effectively increases the receptive field by using dilated convolutions with different dilation rates in a multi-level residual manner to further acquire low-frequency information from the feature map.

[0206] Step 210: Input the residual feature map into the illumination attention module for feature weighting to construct a weighted feature map.

[0207] Optionally, the illumination attention module includes a cascaded channel attention module and a spatial attention module; the image processing procedure of the illumination attention module includes:

[0208] S1. Use a channel attention module to deepen the channel weights of the illumination input feature map of the input illumination attention module, and generate a channel output feature map;

[0209] S2. Input the channel output feature map into the spatial attention module to deepen the spatial weights, and output the illumination output feature map.

[0210] It is understandable that the illumination input feature map can correspond to the residual feature map and other feature maps that are processed by the illumination attention module, and the illumination output feature map can correspond to the weighted feature map and other feature maps that are output by the illumination attention module after image processing.

[0211] Preferably, the channel attention module includes an average pooling layer, a max pooling layer, a first-channel convolutional layer with a LeakyReLU activation function, a second-channel convolutional layer, and a sigmoid function; step S1 includes:

[0212] The illumination input feature map of the illumination attention module is subjected to average pooling and max pooling operations respectively, generating the first channel map and the second channel map accordingly.

[0213] The first and second channel images are added element by element to construct the third channel image;

[0214] The first, second, and third channel images are extracted in parallel using cascaded first and second channel convolutional layers, and then the fourth channel image is output by adding them element by element.

[0215] The channel feature weights of the fourth channel image are extracted using the Sigmoid function, and then the channel feature weights are multiplied element-wise with the illumination input feature image to determine the channel output feature image.

[0216] Preferably, the spatial attention module includes an average pooling layer, a max pooling layer, a first spatial convolutional layer, a second spatial convolutional layer, a third spatial convolutional layer, a fourth spatial convolutional layer, and a sigmoid function, wherein the first spatial convolutional layer, the second spatial convolutional layer, and the third spatial convolutional layer all have a LeakyReLU activation function; step S2 includes:

[0217] After performing average pooling and max pooling operations on the channel output feature maps respectively using average pooling and max pooling layers, the channels are stitched together to generate the first spatial map.

[0218] The first spatial graph is input into the cascaded first and fourth spatial convolutional layers for feature extraction, and a second spatial graph is constructed.

[0219] The first spatial graph is subjected to convolution operations using cascaded second and fourth spatial convolutional layers to output the third spatial graph;

[0220] The first spatial graph is convolutionally filtered by cascaded third and fourth spatial convolutional layers to determine the fourth spatial graph.

[0221] The fifth spatial diagram is generated by adding elements one by one using the second, third, and fourth spatial diagrams.

[0222] The spatial feature weights are determined by activating the fifth spatial map using the Sigmoid function, and then the spatial feature weights are multiplied element-wise with the channel output feature map to output the illumination output feature map.

[0223] In this embodiment of the invention, reference is made to Figure 7 The illumination input feature map of the input channel attention module is subjected to average pooling and max pooling respectively to generate the first channel map and the second channel map. The first channel map and the second channel map are then fused element-wise by summing their features to construct the third channel map. Through three parallel channel branches consisting of cascaded 3×3 first channel convolutional layers and 1×1 second channel convolutional layers, the first channel map, the second channel map, and the third channel map are extracted in parallel, and then element-wise summed to determine the fourth channel map. The channel feature weights of the fourth channel map are extracted using the Sigmoid function. The channel feature weights are then multiplied element-wise with the illumination input feature map to output the channel output feature map. The channel output feature maps of the input spatial attention module are subjected to average pooling and max pooling respectively, and then concatenated to generate the first spatial map. The first spatial map is then used as input to three parallel spatial branches for convolution operations, generating the second, third, and fourth spatial maps. These are then element-wise added to construct the fifth spatial map. The first spatial branch consists of cascaded 1×1 first and fourth spatial convolutional layers; the second spatial branch consists of cascaded 3×3 second and 1×1 fourth spatial convolutional layers; and the third spatial branch consists of cascaded 5×5 third and 1×1 fourth spatial convolutional layers. The fifth spatial map is activated using the Sigmoid function to determine the spatial feature weights. These spatial feature weights are then element-wise multiplied with the channel output feature maps to output the illumination output feature map.

[0224] Step 211: Use the illumination attention module and decoder to perform multi-scale feature extraction and feature fusion on the preliminary feature map, cascaded feature map and weighted feature map to determine the target enhanced image.

[0225] Optionally, the decoder includes an upsampling layer, a convolutional layer with LeakyReLU activation, a contextual feature extraction module, and a convolutional layer with Tanh activation; step 211 includes the following sub-steps:

[0226] After the initial feature map and the cascaded feature map are weighted by the illumination attention module, the first illumination map and the second illumination map are output respectively.

[0227] After the weighted feature map is input into the upsampling layer and upsampled by bilinear interpolation, it is concatenated with the second illumination map to generate the first decoded map.

[0228] A cascaded convolutional layer with LeakyReLU activation function, a context feature extraction module, and an upsampling layer are used to perform channel compression, feature extraction, and bilinear interpolation upsampling on the first decoded image, and then the channel is concatenated with the first illumination image to construct the second decoded image;

[0229] The third decoding map is determined by performing channel dimensionality reduction and multi-scale feature extraction on the second decoding map through cascaded convolutional layers with LeakyReLU activation function and context feature extraction module;

[0230] The third decoded image is processed by a convolutional layer with Tanh activation function to extract and map features, and the target enhancement image is output.

[0231] In this embodiment of the invention, reference is made to Figure 4 After the weighted feature map enters the decoder, it is first upsampled by bilinear interpolation in an upsampling layer, and then concatenated with the first illumination map (which has been weighted by the illumination attention module) from the initial feature map output from the encoder, skipping channels to generate the first decoded map. The first decoded map is then compressed by a convolutional layer with LeakyReLU activation, reducing the number of channels to 128. After multi-scale feature extraction by the context feature extraction module, the number of channels is reduced to 64. It is then upsampled again by bilinear interpolation in an upsampling layer, and concatenated with the second illumination map (which has been weighted by the illumination attention module) from the cascaded feature map output from the encoder to construct the second decoded map. The second decoded map is then compressed by a convolutional layer with LeakyReLU activation, and then input into the context feature extraction module for feature extraction to determine the third decoded map, reducing the number of channels to 32. Finally, the third decoded map is used for feature extraction by a convolutional layer with Tanh activation, and then mapped to output the target enhancement image.

[0232] In this embodiment of the invention, the received training set is used to bidirectionally train two initial image generation models and two initial image adversarial models, and the total loss function value is determined. Based on the total loss function value, the model is iteratively optimized, and the received test set combined with image quality evaluation metrics is used to validate the model, thus determining the target image generation model. When a target transmission line monitoring image is received, the encoder in the target image generation model performs multi-layer feature extraction on the target transmission line monitoring image, outputting a preliminary feature map, a cascaded feature map, and an encoded feature map. The encoded feature map is input into a cascaded receptive field residual module and an illumination attention module for feature extraction and feature weighting, constructing a weighted feature map. The illumination attention module and decoder then perform multi-scale feature extraction and feature fusion on the preliminary feature map, the cascaded feature map, and the weighted feature map to determine the target enhanced image. The enhanced image obtained through this target image generation model improves the clarity of image texture details while avoiding overexposure or underexposure, resulting in better image quality and providing more accurate and reliable image information for transmission line defect detection.

[0233] Please see Figure 9 , Figure 9This is a structural block diagram of a power transmission line monitoring image enhancement device provided in Embodiment 3.

[0234] The present invention provides an image enhancement device for monitoring power transmission lines, comprising:

[0235] Model building module 901 is used to build two initial image generation models and two initial image adversarial models when a training set and a test set are received;

[0236] The model training module 902 is used to perform bidirectional training and model validation on all initial image generation models and all initial image adversarial models using training and test sets to determine the target image generation model; wherein, the target image generation model includes an encoder, a receptive field residual module, an illumination attention module, and a decoder;

[0237] The encoder processing module 903 is used to perform multi-layer feature extraction on the target transmission line monitoring image using an encoder when the target transmission line monitoring image is received, and output a preliminary feature map, a cascaded feature map and an encoded feature map.

[0238] The residual feature map generation module 904 is used to extract features from the encoded feature map through the receptive field residual module and generate a residual feature map.

[0239] The weighted feature map construction module 905 is used to input the residual feature map into the illumination attention module for feature weighting and to construct the weighted feature map.

[0240] The target enhancement image output module 906 is used to perform multi-scale feature extraction and feature fusion on the preliminary feature map, cascaded feature map and weighted feature map using the illumination attention module and decoder to determine the target enhancement image.

[0241] Optionally, the test set includes low-light test images; the model training module 902 includes:

[0242] The total loss function value determination submodule is used to perform bidirectional training on all initial image generation models and all initial image adversarial models using the training set to determine the total loss function value.

[0243] The total loss function convergence judgment submodule is used to update the model parameters of all initial image generation models and all initial image adversarial models using gradient descent if the total loss function value does not converge, and then jump to the step of bidirectional training of all initial image generation models and all initial image adversarial models using the training set to determine the total loss function value; if the total loss function value converges, then the optimized image generation model is determined based on the current model parameters.

[0244] The model testing submodule is used to input the low-light test image into the optimized image generation model for image enhancement and output the enhanced test image.

[0245] The image evaluation index calculation submodule is used to calculate the natural image quality evaluation index value and the meta-learning-based no-reference image evaluation index value of the enhanced test image.

[0246] The target image generation model determination submodule is used to determine the target image generation model when the natural image quality evaluation index value is less than the first preset ideal value and the meta-learning-based no-reference image evaluation index value is greater than the second preset ideal value; wherein, the target image generation model includes an encoder, a receptive field residual module, an illumination attention module, and a decoder.

[0247] Optionally, the training set includes normal lighting training images and low lighting training images; the two initial image generation models include a forward image generation model and a backward image generation model; the two initial image adversarial models include a forward image adversarial model and a backward image adversarial model; the total loss function value convergence judgment submodule is specifically used for:

[0248] Low-light training images and normal-light training images are input into the forward image generation model and the reverse image generation model, respectively, for image enhancement, and the first enhanced image and the second enhanced image are output.

[0249] The first and second enhancement maps are input into the reverse image generation model and the forward image generation model, respectively, to perform image transformation and construct the first and second reconstruction maps.

[0250] Image discrimination is performed based on the first enhanced image and the normal light training image using a positive image adversarial model, generating the first discrimination result;

[0251] The second enhanced image and the low-light training image are discriminated using an inverse image adversarial model, and the second discrimination result is output.

[0252] Using low-light training images, normal-light training images, first augmented images, second augmented images, first reconstructed images, second reconstructed images, first discrimination results, and second discrimination results, we calculate the adversarial loss function value, cycle consistency loss function value, identity consistency loss function value, and unsupervised perception loss function value.

[0253] The total loss function value is determined by adding the adversarial loss function value, the cycle consistency loss function value, the identity consistency loss function value, and the unsupervised perception loss function value.

[0254] Optionally, the initial image adversarial model includes convolutional layers with LeakyReLU activation functions, downsampling layers, upsampling layers, and ordinary convolutional layers; the execution process of the initial image adversarial model includes:

[0255] The adversarial input feature image of the adversarial model is processed by a convolutional layer with LeakyReLU activation function to extract local features and generate the first adversarial map.

[0256] A second adversarial graph is constructed by performing max pooling downsampling and convolution operations on the first adversarial graph using cascaded downsampling layers and convolutional layers with LeakyReLU activation function.

[0257] After max pooling downsampling the second adversarial graph through a downsampling layer, continuous feature extraction is performed using multiple convolutional layers with LeakyReLU activation function to output the third adversarial graph;

[0258] After the third adversarial graph is input into the upsampling layer and upsampled by bilinear interpolation, it is added element-wise with the second adversarial graph to determine the fourth adversarial graph.

[0259] The fourth adversarial graph is generated by performing convolutional filtering and bilinear interpolation upsampling on a cascaded convolutional and upsampling layer with LeakyReLU activation function, and then adding it element-wise to the first adversarial graph.

[0260] After extracting continuous features from the fifth adversarial map using multiple convolutional layers with LeakyReLU activation function, image discrimination is performed using ordinary convolutional layers to determine the discrimination result.

[0261] Optionally, the encoder includes a convolutional layer with a LeakyReLU activation function, a contextual feature extraction module, and a cascaded module; the encoder processing module 903 is specifically used for:

[0262] When a monitoring image of the target transmission line is received, a convolutional layer with the LeakyReLU activation function is used to perform preliminary feature extraction on the monitoring image of the target transmission line to generate a preliminary feature map;

[0263] The contextual feature extraction module extracts contextual information from the preliminary feature map to construct a deep feature map.

[0264] The deep feature map is input into the cascaded module for downsampling and refined feature extraction, and the cascaded feature map is output.

[0265] A cascaded module is used to downsample and extract global features from the cascaded feature map to generate an encoded feature map; the cascaded module includes a cascaded residual D module and a context feature extraction module.

[0266] Furthermore, the context feature extraction module includes a first convolutional layer, a second convolutional layer, and a third convolutional layer, wherein both the first and second convolutional layers have the LeakyReLU activation function; the image processing procedure of the context feature extraction module includes:

[0267] The context input feature map of the input context feature extraction module is used to extract features using the first convolutional layer to generate the first feature map;

[0268] The first feature map is convolved by the second convolutional layer and multiple cascaded second convolutional layers respectively, thereby constructing the second and third feature maps.

[0269] After performing channel dimensionality reduction on the context input feature map using the third convolutional layer, it is added element-wise with the second and third feature maps to output the context output feature map.

[0270] Optionally, the receptive field residual module includes a first dilated convolutional layer, a second dilated convolutional layer, and a third dilated convolutional layer with multi-level residual connections, as well as a standard convolutional layer; the residual feature map generation module 904 is specifically used for:

[0271] The encoded feature map is subjected to dilated convolutional filtering using the first dilated convolutional layer to output the first dilated map.

[0272] The encoded feature map is added element-wise to the first hole map to generate the second hole map;

[0273] After extracting features from the second dilated convolutional layer, the third dilated map is constructed by adding it element-wise to the encoded feature map and the second dilated map.

[0274] The third hole map is input into the third hole convolutional layer for hole convolution operation to determine the fourth hole map;

[0275] The encoded feature map is reduced in dimensionality by channel using a standard convolutional layer, and then added element-wise with the second and fourth dilated maps to output a residual feature map.

[0276] Optionally, the illumination attention module includes a cascaded channel attention module and a spatial attention module; the channel attention module includes an average pooling layer, a max pooling layer, a first-channel convolutional layer with a LeakyReLU activation function, a second-channel convolutional layer, and a sigmoid function; the image processing procedure of the illumination attention module includes:

[0277] The illumination input feature map of the illumination attention module is subjected to average pooling and max pooling operations respectively, generating the first channel map and the second channel map accordingly.

[0278] The first and second channel images are added element by element to construct the third channel image;

[0279] The first, second, and third channel images are extracted in parallel using cascaded first and second channel convolutional layers, and then the fourth channel image is output by adding them element by element.

[0280] The channel feature weights of the fourth channel image are extracted using the Sigmoid function, and then the channel feature weights are multiplied element-wise with the illumination input feature image to determine the channel output feature image.

[0281] The channel output feature map is input into the spatial attention module to deepen the spatial weights, and the illumination output feature map is output.

[0282] Furthermore, the spatial attention module includes an average pooling layer, a max pooling layer, a first spatial convolutional layer, a second spatial convolutional layer, a third spatial convolutional layer, a fourth spatial convolutional layer, and a sigmoid function, wherein the first spatial convolutional layer, the second spatial convolutional layer, and the third spatial convolutional layer all have the LeakyReLU activation function; the step of inputting the channel output feature map into the spatial attention module to deepen the spatial weights and outputting the illumination output feature map includes:

[0283] After performing average pooling and max pooling operations on the channel output feature maps respectively using average pooling and max pooling layers, the channels are stitched together to generate the first spatial map.

[0284] The first spatial graph is input into the cascaded first and fourth spatial convolutional layers for feature extraction, and a second spatial graph is constructed.

[0285] The first spatial graph is subjected to convolution operations using cascaded second and fourth spatial convolutional layers to output the third spatial graph;

[0286] The first spatial graph is convolutionally filtered by cascaded third and fourth spatial convolutional layers to determine the fourth spatial graph.

[0287] The fifth spatial diagram is generated by adding elements one by one using the second, third, and fourth spatial diagrams.

[0288] The spatial feature weights are determined by activating the fifth spatial map using the Sigmoid function, and then the spatial feature weights are multiplied element-wise with the channel output feature map to output the illumination output feature map.

[0289] Optionally, the decoder includes an upsampling layer, a convolutional layer with LeakyReLU activation, a contextual feature extraction module, and a convolutional layer with Tanh activation; the target enhancement image output module 906 is specifically used for:

[0290] After the initial feature map and the cascaded feature map are weighted by the illumination attention module, the first illumination map and the second illumination map are output respectively.

[0291] After the weighted feature map is input into the upsampling layer and upsampled by bilinear interpolation, it is concatenated with the second illumination map to generate the first decoded map.

[0292] A cascaded convolutional layer with LeakyReLU activation function, a context feature extraction module, and an upsampling layer are used to perform channel compression, feature extraction, and bilinear interpolation upsampling on the first decoded image, and then the channel is concatenated with the first illumination image to construct the second decoded image;

[0293] The third decoding map is determined by performing channel dimensionality reduction and multi-scale feature extraction on the second decoding map through cascaded convolutional layers with LeakyReLU activation function and context feature extraction module;

[0294] The third decoded image is processed by a convolutional layer with Tanh activation function to extract and map features, and the target enhancement image is output.

[0295] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described apparatus and unit can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0296] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0297] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0298] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0299] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for enhancing images during power transmission line monitoring, characterized in that, include: When the training set and the test set are received, two initial image generation models and two initial image adversarial models are constructed. The training set and the test set are used to perform bidirectional training and model validation on all the initial image generation models and all the initial image adversarial models to determine the target image generation model; wherein, the target image generation model includes an encoder, a receptive field residual module, an illumination attention module, and a decoder; When a target transmission line monitoring image is received, the encoder is used to perform multi-layer feature extraction on the target transmission line monitoring image, and outputs a preliminary feature map, a cascaded feature map and an encoded feature map; The residual feature map is generated by extracting features from the encoded feature map through the receptive field residual module. The residual feature map is input into the illumination attention module for feature weighting to construct a weighted feature map; The illumination attention module and the decoder are used to perform multi-scale feature extraction and feature fusion on the preliminary feature map, the cascaded feature map and the weighted feature map to determine the target enhanced image; The receptive field residual module includes a first dilated convolutional layer, a second dilated convolutional layer, and a third dilated convolutional layer with multi-level residual connections, as well as a standard convolutional layer; the step of extracting features from the encoded feature map through the receptive field residual module to generate a residual feature map includes: The encoded feature map is subjected to dilated convolution filtering using a first dilated convolution layer to output a first dilated map; The encoded feature map is added element-wise to the first hole map to generate a second hole map; After extracting features from the second dilated convolutional layer, the third dilated map is constructed by adding it element-wise to the encoded feature map and the second dilated map. The third hole map is input into the third hole convolutional layer for hole convolution operation to determine the fourth hole map; The encoded feature map is reduced in channel dimension using a standard convolutional layer, and then added element-wise with the second and fourth hole maps to output a residual feature map. The illumination attention module includes cascaded channel attention modules and spatial attention modules; the channel attention module includes an average pooling layer, a max pooling layer, a first channel convolutional layer with LeakyReLU activation, a second channel convolutional layer, and a sigmoid function; the spatial attention module includes an average pooling layer, a max pooling layer, a first spatial convolutional layer, a second spatial convolutional layer, a third spatial convolutional layer, a fourth spatial convolutional layer, and a sigmoid function, wherein the first spatial convolutional layer, the second spatial convolutional layer, and the third spatial convolutional layer all have LeakyReLU activation functions; The image processing procedure of the illumination attention module includes: The illumination input feature map input to the illumination attention module is subjected to average pooling and max pooling operations respectively using the average pooling layer and the max pooling layer to generate a first channel map and a second channel map respectively. The first channel image and the second channel image are added element by element to construct the third channel image; The first channel image, the second channel image, and the third channel image are extracted in parallel using cascaded first channel convolutional layers and second channel convolutional layers respectively, and then the fourth channel image is output by adding them element by element. The channel feature weights of the fourth channel map are extracted using the Sigmoid function, and the channel feature weights are multiplied element-wise with the illumination input feature map to determine the channel output feature map. After performing average pooling and max pooling operations on the channel output feature map respectively, the channels are stitched together to generate the first spatial map. The first spatial graph is input into a cascaded first spatial convolutional layer and a fourth spatial convolutional layer for feature extraction to construct a second spatial graph; The first spatial graph is subjected to convolution operation by a cascaded second spatial convolutional layer and a fourth spatial convolutional layer to output a third spatial graph; The first spatial graph is convolutionally filtered by cascaded third and fourth spatial convolutional layers to determine the fourth spatial graph. The second spatial diagram, the third spatial diagram, and the fourth spatial diagram are added element by element to generate the fifth spatial diagram; The spatial feature weights are determined by activating the fifth spatial map using the Sigmoid function, and then the spatial feature weights are multiplied element-wise with the channel output feature map to output the illumination output feature map.

2. The image enhancement method for power transmission line monitoring according to claim 1, characterized in that, The test set includes low-light test images; the step of using the training set and the test set to perform bidirectional training and model validation on all the initial image generation models and all the initial image adversarial models to determine the target image generation model; wherein, the target image generation model includes an encoder, a receptive field residual module, an illumination attention module, and a decoder, comprising: The training set is used to perform bidirectional training on all the initial image generation models and all the initial image adversarial models to determine the total loss function value; If the total loss function value does not converge, then the gradient descent method is used to update the model parameters of all the initial image generation models and all the initial image adversarial models, and then the process jumps to the step of using the training set to perform bidirectional training on all the initial image generation models and all the initial image adversarial models to determine the total loss function value. If the total loss function value converges, the optimized image generation model is determined based on the current model parameters; The low-light test image is input into the optimized image generation model for image enhancement, and the enhanced test image is output. Calculate the natural image quality evaluation index and the no-reference image evaluation index based on meta-learning for the enhanced test image; When the natural image quality evaluation index value is less than the first preset ideal value, and the meta-learning-based no-reference image evaluation index value is greater than the second preset ideal value, a target image generation model is determined; wherein, the target image generation model includes an encoder, a receptive field residual module, an illumination attention module, and a decoder.

3. The image enhancement method for power transmission line monitoring according to claim 2, characterized in that, The training set includes normal light training images and low light training images; the two initial image generation models include a forward image generation model and a reverse image generation model; the two initial image adversarial models include a forward image adversarial model and a reverse image adversarial model; the step of using the training set to perform bidirectional training on all the initial image generation models and all the initial image adversarial models to determine the total loss function value includes: The low-light training image and the normal-light training image are respectively input into the forward image generation model and the reverse image generation model for image enhancement, and the first enhanced image and the second enhanced image are output. The first enhanced image and the second enhanced image are respectively input into the reverse image generation model and the forward image generation model to perform image transformation, and a first reconstructed image and a second reconstructed image are constructed. The positive image adversarial model performs image discrimination based on the first enhanced image and the normal light training image to generate a first discrimination result; The reverse image adversarial model is used to perform image discrimination on the second enhanced image and the low-light training image, and a second discrimination result is output. Using the low-light training image, the normal-light training image, the first enhanced image, the second enhanced image, the first reconstructed image, the second reconstructed image, the first discrimination result, and the second discrimination result, calculate the adversarial loss function value, the cycle consistency loss function value, the identity consistency loss function value, and the unsupervised perception loss function value; The total loss function value is determined by adding the adversarial loss function value, the cycle consistency loss function value, the identity consistency loss function value, and the unsupervised perception loss function value.

4. The image enhancement method for power transmission line monitoring according to claim 1, characterized in that, The initial image adversarial model includes convolutional layers with LeakyReLU activation, downsampling layers, upsampling layers, and ordinary convolutional layers; the execution process of the initial image adversarial model includes: The adversarial input feature image of the input adversarial model is used to extract local features through a convolutional layer with a LeakyReLU activation function to generate a first adversarial map; The first adversarial graph is constructed by performing max pooling downsampling and convolution operations on the first adversarial graph using cascaded downsampling layers and convolutional layers with LeakyReLU activation function; After max pooling downsampling is performed on the second adversarial graph through a downsampling layer, continuous feature extraction is performed using multiple convolutional layers with LeakyReLU activation function to output the third adversarial graph; After the third adversarial graph is input into the upsampling layer and upsampled by bilinear interpolation, it is added element-wise with the second adversarial graph to determine the fourth adversarial graph. After performing convolutional filtering and bilinear interpolation upsampling on the fourth adversarial graph based on cascaded convolutional layers and upsampling layers with LeakyReLU activation function, it is added element-wise to the first adversarial graph to generate the fifth adversarial graph. After continuously extracting features from the fifth adversarial graph using multiple convolutional layers with LeakyReLU activation functions, image discrimination is performed using ordinary convolutional layers to determine the discrimination result.

5. The image enhancement method for power transmission line monitoring according to claim 1, characterized in that, The encoder includes a convolutional layer with a LeakyReLU activation function, a contextual feature extraction module, and a cascaded module; the step of using the encoder to perform multi-layer feature extraction on the target transmission line monitoring image when a target transmission line monitoring image is received, and outputting a preliminary feature map, a cascaded feature map, and an encoded feature map includes: When a monitoring image of the target transmission line is received, a convolutional layer with the LeakyReLU activation function is used to perform preliminary feature extraction on the monitoring image of the target transmission line to generate a preliminary feature map; The context information of the preliminary feature map is extracted by the context feature extraction module to construct a deep feature map. The deep feature map is input into the cascade module for downsampling and refined feature extraction, and the cascaded feature map is output. A cascaded module is used to downsample and extract global features from the cascaded feature map to generate an encoded feature map; wherein, the cascaded module includes a cascaded residual D module and a context feature extraction module.

6. The image enhancement method for power transmission line monitoring according to claim 5, characterized in that, The context feature extraction module includes a first convolutional layer, a second convolutional layer, and a third convolutional layer, wherein both the first and second convolutional layers have the LeakyReLU activation function; the image processing procedure of the context feature extraction module includes: The context input feature map input to the context feature extraction module is subjected to a first convolutional layer for feature extraction to generate a first feature map; The first feature map is convolved by a second convolutional layer and multiple cascaded second convolutional layers to construct a second feature map and a third feature map respectively. After performing channel dimensionality reduction on the context input feature map using a third convolutional layer, it is added element-wise to the second feature map and the third feature map to output the context output feature map.

7. The method for enhancing transmission line monitoring images according to claim 1, characterized in that, The decoder includes an upsampling layer, a convolutional layer with LeakyReLU activation, a contextual feature extraction module, and a convolutional layer with Tanh activation; the step of using the illumination attention module and the decoder to perform multi-scale feature extraction and feature fusion on the preliminary feature map, the cascaded feature map, and the weighted feature map to determine the target enhanced image includes: After the illumination attention module performs feature weighting on the preliminary feature map and the cascaded feature map respectively, it outputs the first illumination map and the second illumination map accordingly. After the weighted feature map is input into the upsampling layer and upsampled by bilinear interpolation, it is concatenated with the second illumination map to generate the first decoded map. The first decoded image is subjected to channel compression, feature extraction and bilinear interpolation upsampling by cascaded convolutional layers with LeakyReLU activation function, context feature extraction module and upsampling layer, and then channel-wise concatenated with the first illumination image to construct the second decoded image; The second decoded map is subjected to channel dimensionality reduction and multi-scale feature extraction by cascaded convolutional layers with LeakyReLU activation function and context feature extraction module to determine the third decoded map; The third decoded image is subjected to feature extraction mapping using a convolutional layer with Tanh activation function, and the target enhancement image is output.

8. A transmission line monitoring image enhancement device, characterized in that, include: The model building module is used to build two initial image generation models and two initial image adversarial models when the training set and test set are received. The model training module is used to perform bidirectional training and model validation on all the initial image generation models and all the initial image adversarial models using the training set and the test set to determine the target image generation model; wherein, the target image generation model includes an encoder, a receptive field residual module, an illumination attention module, and a decoder; The encoder processing module is used to perform multi-layer feature extraction on the target transmission line monitoring image using the encoder when the target transmission line monitoring image is received, and output a preliminary feature map, a cascaded feature map and an encoded feature map. The residual feature map generation module is used to extract features from the encoded feature map through the receptive field residual module to generate a residual feature map; The weighted feature map construction module is used to input the residual feature map into the illumination attention module for feature weighting and to construct a weighted feature map. The target enhancement image output module is used to perform multi-scale feature extraction and feature fusion on the preliminary feature map, the cascaded feature map and the weighted feature map using the illumination attention module and the decoder to determine the target enhancement image; The receptive field residual module includes a first dilated convolutional layer, a second dilated convolutional layer, and a third dilated convolutional layer with multi-level residual connections, as well as a standard convolutional layer; the step of extracting features from the encoded feature map through the receptive field residual module to generate a residual feature map includes: The encoded feature map is subjected to dilated convolution filtering using a first dilated convolution layer to output a first dilated map; The encoded feature map is added element-wise to the first hole map to generate a second hole map; After extracting features from the second dilated convolutional layer, the third dilated map is constructed by adding it element-wise to the encoded feature map and the second dilated map. The third hole map is input into the third hole convolutional layer for hole convolution operation to determine the fourth hole map; The encoded feature map is reduced in channel dimension using a standard convolutional layer, and then added element-wise with the second and fourth hole maps to output a residual feature map. The illumination attention module includes cascaded channel attention modules and spatial attention modules; the channel attention module includes an average pooling layer, a max pooling layer, a first channel convolutional layer with LeakyReLU activation, a second channel convolutional layer, and a sigmoid function; the spatial attention module includes an average pooling layer, a max pooling layer, a first spatial convolutional layer, a second spatial convolutional layer, a third spatial convolutional layer, a fourth spatial convolutional layer, and a sigmoid function, wherein the first spatial convolutional layer, the second spatial convolutional layer, and the third spatial convolutional layer all have LeakyReLU activation functions; The image processing procedure of the illumination attention module includes: The illumination input feature map input to the illumination attention module is subjected to average pooling and max pooling operations respectively using the average pooling layer and the max pooling layer to generate a first channel map and a second channel map respectively. The first channel image and the second channel image are added element by element to construct the third channel image; The first channel image, the second channel image, and the third channel image are extracted in parallel using cascaded first channel convolutional layers and second channel convolutional layers respectively, and then the fourth channel image is output by adding them element by element. The channel feature weights of the fourth channel map are extracted using the Sigmoid function, and the channel feature weights are multiplied element-wise with the illumination input feature map to determine the channel output feature map. After performing average pooling and max pooling operations on the channel output feature map respectively, the channels are stitched together to generate the first spatial map. The first spatial graph is input into a cascaded first spatial convolutional layer and a fourth spatial convolutional layer for feature extraction to construct a second spatial graph; The first spatial graph is subjected to convolution operation by a cascaded second spatial convolutional layer and a fourth spatial convolutional layer to output a third spatial graph; The first spatial graph is convolutionally filtered by cascaded third and fourth spatial convolutional layers to determine the fourth spatial graph. The second spatial diagram, the third spatial diagram, and the fourth spatial diagram are added element by element to generate the fifth spatial diagram; The spatial feature weights are determined by activating the fifth spatial map using the Sigmoid function, and then the spatial feature weights are multiplied element-wise with the channel output feature map to output the illumination output feature map.

Images