A low-light image enhancement network and enhancement method based on zero-element feature guidance

Abstract

The application discloses a low-light image enhancement network and method based on zero element feature guidance, which comprises the following steps: firstly, data preprocessing is performed, and a low-light image is input; then, a zero element image of the low-light image is calculated; a joint attention Transformer coding network and a reversible network are constructed; the low-light image and the zero element image are input into the joint attention Transformer coding network to extract conditional features; a standard normal distribution is sampled; the standard normal distribution and the extracted conditional features are input into the reversible network to perform reversible transformation, and an enhanced image is output. In the joint attention Transformer coding network, a zero element attention module is designed based on the original Transformer self-attention module; in the reversible network, a cross- affine coupling layer is designed to realize cross learning of image features. The application strengthens the learning of missing information in the low-light image based on the guidance of the zero element feature, improves the visual effect of the enhanced image, and improves the peak signal-to-noise ratio of the image.

Description

Technical Field

[0001] This invention belongs to the technical field of low-light image enhancement, and mainly relates to a low-light image enhancement network and enhancement method based on zero-element feature guidance. Background Technology

[0002] Images captured in low-light environments often suffer from severe degradation, characterized by low visibility and contrast, severe color distortion, and unpredictable image noise. These low-light images have poor aesthetic quality, and the information they contain is often biased and incomplete. When used as input images for advanced computer vision tasks such as object recognition and instance segmentation, the biased and incomplete input information negatively impacts performance. Therefore, low-light image enhancement can improve the aesthetic quality of images, enhancing subjective visual perception, and recover missing information, thereby improving the overall performance of related computer vision tasks.

[0003] Current low-light image enhancement methods mostly focus on enhancing the appearance characteristics of low-light images, such as increasing brightness and suppressing noise. However, few methods consider designing enhancement methods based on the numerical characteristics of low-light images. For RGB images, pixels have values ​​in three channels: red (R), green (G), and blue (B). Low-light conditions often cause pixel value loss, resulting in a large number of pixels having one or more channels with zero values; these pixels are defined as zero elements. Low-light images often contain a large number of zero elements. Therefore, marking these zero elements to form a binary zero-element map can reflect the overall lack of image information. However, how to use this zero-element map to enhance low-light images has become a problem that those skilled in the art are eager to solve. Summary of the Invention

[0004] This invention addresses the problems of existing technologies by providing a low-light image enhancement network and method based on zero-element feature guidance. First, data preprocessing is performed, inputting a low-light image and obtaining its zero-element map. Then, a joint attention Transformer coding network and a reversible network are constructed to form a low-light image enhancement network based on zero-element feature guidance, integrating the low-light image with its zero-element map. Figure 1The input image is fed into a joint attention Transformer encoding network to extract conditional features, which are sampled from a standard normal distribution. These features, along with the extracted conditional features, are then fed into a reversible network for a reversible transformation, outputting the enhanced image. Specifically, in the joint attention Transformer encoding network, a zero-element attention module is designed based on the original Transformer's self-attention module, utilizing zero-element features for joint attention computation. In the reversible network, a cross-affine coupling layer is designed to achieve cross-learning of image features. This invention, guided by zero-element features, enhances the learning of missing information in low-light images, improves the visual effect of the enhanced image, and simultaneously increases the peak signal-to-noise ratio (PSNR).

[0005] To achieve the above objectives, the technical solution adopted by the present invention is: a low-light image enhancement network based on zero-element feature guidance, comprising a joint attention Transformer coding network and a reversible network;

[0006] The joint attention Transformer coding network includes a shallow feature extraction block, a joint attention deep feature coding block, and a conditional feature output block; wherein...

[0007] The shallow feature extraction block consists of a convolutional layer C1, an activation function layer R1, and a convolutional layer C2 connected in sequence.

[0008] The joint attention deep feature encoding block consists of 6 joint attention Transformer sub-blocks. Each joint attention Transformer sub-block is composed of a joint attention computation network and a feedforward network connected in sequence. The joint attention computation network is composed of a self-attention module and a zero-element attention module connected in parallel. The feedforward network is composed of a normalization layer LN6, a linear layer Linear8, an activation function layer R2, and a Linear9 cascaded together.

[0009] The conditional feature output block consists of a convolutional layer C3, an interpolation sampling layer I1, a convolutional layer C4, and an interpolation sampling layer I2 connected in sequence.

[0010] The reversible network is divided into 1 to 3 levels, each level consisting of 1 scale transformation layer and 8 reversible transformation blocks connected in sequence.

[0011] As an improvement of the present invention, the self-attention module in the joint attention computation network includes three branches, whose inputs are q respectively. sl k sl Both q and v are image features output from the shallow feature extraction block; where the input is q. sl The branch consists of a normalized layer LN1 and a linear layer Linear1 cascaded together, with input k. slThe branch with input v consists of a normalized layer LN2 and a linear layer Linear2 cascaded together. The branch with input v consists of a normalized layer LN3 and a linear layer Linear3 cascaded together. Attention is calculated in all three branches, and their outputs will be used as the input to the linear layer Linear4.

[0012] As another improvement of the present invention, the zero-element attention module includes two branches, whose inputs are q respectively. sp and k sp All of them come from zero-element features; where the input is q sp The branch consists of a cascaded normalized layer LN4 and a linear layer Linear5, with input k. sl The branch consists of a normalized layer LN5 and a linear layer Linear6 cascaded together. The two branches perform attention calculations with the branch with input v in the self-attention part, and their outputs will be used as the input of the linear layer Linear7.

[0013] As another improvement of the present invention, the reversible transformation block of the reversible network includes four cascaded reversible transformation layers: a reversible activation layer, a 1×1 convolutional layer, an affine injection layer, and a cross-affine coupling layer.

[0014] As another improvement of the present invention, in the cross-affine coupling layer of the reversible network, the input features are cross-separated in their channel dimension according to the parity of the number of channels to form two parts with the same number of channels. The features undergo a two-stage affine transformation, and the transformed features are cross-stitched together to restore the original channel order for output.

[0015] To achieve the above objectives, the technical solution adopted by this invention is as follows: a low-light image enhancement method based on zero-element feature guidance, comprising the following steps:

[0016] S1, Data preprocessing: Obtain low-light images and obtain their zero-element maps;

[0017] S2, Constructing a joint attention Transformer encoding network and a reversible network and training the network: The joint attention Transformer encoding network and the reversible network constitute a low-light image enhancement network based on zero-element feature guidance. The low-light image and its zero-element map in step S1 are input into the enhancement network, and the negative log-likelihood loss function is used to train the network to obtain the optimized network.

[0018] S3, Output: Combine the low-light image from step S1 with its zero elements. Figure 1The input is a Joint Attention Transformer encoding network to extract conditional features, and a standard normal distribution is sampled. These features, along with the conditional features extracted by the Joint Attention Transformer encoding network, are used as inputs to the invertible network in step S2 to perform an invertible transformation and finally generate an enhanced image.

[0019] As an improvement to the present invention, in step S1, the zero-element diagram is specifically as follows:

[0020]

[0021] in, Let i and j represent the pixel values ​​of the red channel R (Red), green channel G (Green), and blue channel B (Blue) of the pixel at coordinates (i,j), respectively, where i and j represent the height index and width index of the image, respectively.

[0022] As another improvement of the present invention, the loss function for network training is:

[0023]

[0024] Among them, L(I L ,I GT ) represents the negative log-likelihood loss function, I L and I GT These represent the low-light image and its corresponding normal-light image, respectively. P y (y|I L P represents the distribution of a normal illumination image conditioned on a low-light image. z (z) represents the potential spatial distribution. φ represents the Jacobian determinant value calculated during the transformation of the invertible network. i and represents the reversible transform block and the image features it processes, respectively, and ft represents the conditional features input to the reversible transform layer.

[0025] Compared with existing technologies, the present invention has the following advantages: The present invention defines zero elements based on the numerical characteristics exhibited by low-light images, and reflects the missing image information through the zero-element map corresponding to the low-light image. The present invention discloses a low-light image enhancement network and method guided by zero-element features, which integrates low-light images and zero elements... Figure 1The input is fed into a joint attention Transformer encoding network to participate in the calculation of joint attention, thereby enhancing the network's learning of severely missing image information regions. Conditional features extracted by the joint attention Transformer encoding network are input into a reversible network to guide the generation of enhanced images using a standard normal distribution. A cross-affine coupling layer is designed in the reversible network to achieve cross-learning of image features, further improving the overall network's expressive power. The low-light image enhancement network designed in this invention can effectively repair missing image information, and the enhanced images have good visual effects and a high peak signal-to-noise ratio (PSNR). Attached Figure Description

[0026] Figure 1 This is a flowchart illustrating the steps of the low-light image enhancement method based on zero-element feature guidance of the present invention.

[0027] Figure 2 This is a schematic diagram of the low-light image enhancement network based on zero-element feature guidance of the present invention;

[0028] Figure 3 This is a schematic diagram of the shallow feature extraction block in the Joint Attention Transformer coding network of this invention;

[0029] Figure 4 This is a schematic diagram of the structure of the joint attention Transformer sub-block in the joint attention Transformer coding network of the present invention;

[0030] Figure 5 This is a schematic diagram of the structure of the conditional feature output block in the joint attention Transformer coding network of this invention;

[0031] Figure 6 This is a schematic diagram of the reversible network structure of the present invention;

[0032] Figure 7 This is a schematic diagram of the cross-affine coupling layer in the reversible network of the present invention;

[0033] Figure 8 This is a schematic diagram of the enhanced image in Embodiment 2 of the present invention. Detailed Implementation

[0034] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are for illustrative purposes only and are not intended to limit the scope of the present invention.

[0035] Example 1

[0036] A low-light image enhancement network guided by zero-element features, its workflow is as follows: Figure 1As shown, the specific steps include:

[0037] Step S1: Data preprocessing;

[0038] Acquire a low-light image and obtain its zero-element map. The process of obtaining the zero-element map is represented by the following formula:

[0039] Let R, G, and B represent the pixel values ​​of the red, green, and blue channels of the pixel at coordinates (i,j), respectively, where i and j represent the height and width indices of the image.

[0040] The above formula indicates that by performing pixel-by-pixel analysis on a low-light image, if a pixel has at least one zero value in its R, G, and B channels, then that pixel is marked as a zero element. The resulting zero-element map is a binary image, and pixels with a value of 1 are the zero elements. The zero-element map marks the zero elements in the low-light image, reflecting the information loss in the image. The low-light image and the zero-element map are used as input to the network for learning and recovering information loss in the image.

[0041] In this embodiment, the input low-light image size is (3, 400, 600), that is, the number of channels is 3, the width and height are 400 and 600 respectively, and the obtained zero-element image size is 1 in terms of the number of channels, and the width and height are the same as the low-light image.

[0042] Step S2: Construct a joint attention Transformer encoding network and an invertible network respectively to form a low-light image enhancement network guided by zero-element features, and complete the network training.

[0043] The joint attention Transformer coding network is as follows: Figure 2 As shown, the network includes: a shallow feature extraction block, a joint attention deep feature encoding block, and a conditional feature output block.

[0044] Specifically, the shallow feature extraction block is as follows: Figure 3As shown, this part consists of a convolutional layer C1, an activation function layer R1, and a convolutional layer C2 connected in sequence. The convolutional layer C1 has a kernel size of 3×3, 3 input channels, 64 output channels, a stride of 1, and padding of 1. The activation function layer R1 uses the Leaky ReLU activation function with a negative slope coefficient of 0.1. The convolutional layer C2 has a kernel size of 3×3, 64 input channels, 64 output channels, a stride of 2, and padding of 1. The shallow feature extraction block designed in this invention utilizes the characteristics of the convolutional structure to extract local information from low-light images. The obtained shallow features are input to the joint attention deep feature encoding block for further processing.

[0045] In this embodiment, the feature size output by the shallow feature extraction block is (64, 200, 300), that is, the number of channels is 64, and the width and height are 200 and 300 respectively.

[0046] The joint attention deep feature encoding block is composed of 6 concatenated joint attention Transformer sub-blocks, as shown in the figure below. Figure 4 As shown, the input to this sub-block is image features and zero-element features. The image features are shallow features obtained from the shallow feature extraction block, and the zero-element features are derived from the zero-element map corresponding to the low-light image obtained in step S1. In this embodiment, the zero-element map is interpolated and downsampled, and then copied along the channel dimension. The size of the original zero-element map is adjusted from (1, 400, 600) to (64, 200, 300) to form the zero-element features, which are the same size as the shallow features. Simultaneously, to adapt to the shape requirements of the Transformer structure for the input, both the image features and the zero-element features are adjusted to a sequence shape, specifically (1, 3750, 1024), i.e., the sequence length is 3750 and the dimension is 1024. The adjusted image features and zero-element features are first processed by a joint attention computation network, which consists of two modules: self-attention and zero-element attention. The image features are copied to q... sl k sl The three features, v ... sl K sl And V, zero-element feature copied to Q sp and K sp The two features are processed sequentially through a normalization layer and a linear layer, respectively, to output feature Q. sp and K sp :

[0047]

[0048] The normalization layers LN1 to LN5 have the same parameters, and their input dimension is 1024. The linear layers Linear1 to Linear3 and Linear5 to Linear6 have the same parameters, their input dimension is 1024, their output dimension is 512, and the weight matrix size is (512, 1024). No bias vector is used.

[0049] Next, feature Q sl K sl V and V are used as the query vector, key vector, and numerical vector, respectively, as inputs to the self-attention part for self-attention computation. Simultaneously, feature Q... sp and K sp The query vector and key vector, which are inputs to the zero-element attention part, are used in the zero-element attention calculation along with the numerical vector V in the self-attention part. The calculations for self-attention and zero-element attention are as follows:

[0050]

[0051]

[0052] The |||| symbol represents the L2 norm calculation. The formula represents the calculation of the cosine similarity between the query vector and the key vector, which is then processed by the Softmax function to generate weight values ​​in the range of 0-1. These weight values ​​are then multiplied by the numerical vector to obtain the feature V. SL and V SP The features obtained through self-attention and zero-element attention are then processed by linear layers Linear4 and Linear7 respectively, summed, and then introduced into the original image features v to form a residual connection, resulting in the output feature V. out :

[0053] V out =Linear4(V SL )+Linear7(V SP )+v (5)

[0054] Linear4 and Linear7 have the same parameters, with an input dimension of 512 and an output dimension of 1024. The weight matrix size is (1024, 512), and no bias vector is used.

[0055] Based on the self-attention computation in the Transformer structure, this invention designs zero-element attention by inputting zero-element features. This, together with the self-attention part, forms a joint attention mechanism, which enhances the network's ability to learn image features. In particular, the guidance of zero-element attention makes the network more focused on learning and recovering information from missing regions of the image.

[0056] Output features V of the joint attention computation network out Finally, the input is fed into a feedforward network for fully connected transformation to enhance feature extraction. The structure of the feedforward network is as follows: Figure 4 As shown, the deep feature encoding block consists of a normalized layer LN6, a linear layer Linear8, an activation function layer R2, and a cascaded Linear9, with residual connections added to the network input. The input dimension of the normalized layer LN6 is 1024, the input dimension of the linear layer Linear8 is 1024, and the output dimension is 2048, with a weight matrix of size (2048, 1024) using a bias vector. The activation function layer R2 uses the ReLU activation function, and the input dimension of the linear layer Linear9 is 2048, the output dimension is 1024, and the weight matrix of size (1024, 2048) uses a bias vector. The final output features of the joint attention deep feature encoding block are converted from sequence shape to image features, i.e., size (64, 200, 300), for further processing by subsequent network modules.

[0057] The conditional feature output block is as follows Figure 5 As shown, this part consists of a convolutional layer C3, an interpolation sampling layer I1, a convolutional layer C4, and an interpolation sampling layer I2 connected in sequence. Convolutional layers C3 to C4 have the same parameters: a kernel size of 1×1, 64 input channels, 64 output channels, a stride of 1, and padding of 1. Interpolation sampling layers I1 and I2 have the same parameters, all using a bilinear interpolation function with a scaling factor of 0.5. The deep image features extracted by the joint attention deep feature coding block are then processed by the conditional feature output block, sequentially outputting conditional features Ft1, Ft2, and Ft3.

[0058] In this embodiment, the dimensions of the conditional features Ft1, Ft2 and Ft3 are (64,200,300), (64,100,150) and (64,50,75) respectively. The conditional features will guide the reversible network to transform the standard normal distribution into an enhanced image.

[0059] The reversible network is as follows Figure 6 As shown, the network includes a scaling layer and a reversible scaling block.

[0060] The reversible network progressively changes the size of the features it processes through scaling layers. Therefore, the entire network is divided into 1 to 3 levels, each consisting of one scaling layer and eight reversible transformation blocks connected sequentially. The scaling layer reversibly adjusts the feature size. During the forward transformation, it readjusts the feature size, halving the width and height and quadrupling the number of channels. In this embodiment, since the final generated normal illumination image size is (3, 400, 600), the feature sizes processed by levels 1 to 3 of the reversible network are (12, 200, 300), (48, 100, 150), and (192, 50, 75), respectively.

[0061] The reversible transform block contains four cascaded reversible transform layers: a reversible activation layer, a 1×1 convolutional layer, an affine injection layer, and a cross-affine coupling layer. The affine injection layer and the cross-affine coupling layer take conditional features Ft1, Ft2, or Ft3 as one input to the current feature being processed, depending on the size of the current feature. The reversible network achieves the transformation between normal illumination image distribution and standard normal distribution through these reversible transform layers.

[0062] Reversible activation layers achieve affine transformations of features through custom learnable parameters. The following equation describes the forward and inverse transformation processes of a reversible activation layer:

[0063] Y = S1⊙X + T1 (6)

[0064] X=(Y-T1) / S1 (7)

[0065] Where ⊙ represents element-wise multiplication, X and Y represent the features being processed, and S1 and T1 are learnable parameters. After the features undergo a forward transformation in this layer, the Jacobian determinant is calculated:

[0066]

[0067] Where f1 and x1 represent the transformation function of formula (6) and its input features, respectively, and h and w represent the width and height of the features, respectively.

[0068] A reversible 1×1 convolutional layer transforms features using a convolutional layer with a kernel size of 1×1. Equations (9) and (10) describe the forward and inverse transformation processes of the reversible 1×1 convolutional layer:

[0069] Y = WX (9)

[0070] X = W -1 Y (10)

[0071] Where W is the convolution kernel matrix. After the features undergo a forward transformation in this layer, the Jacobian determinant value is calculated using formula (11):

[0072]

[0073] Where f2 and x2 represent the transformation function of formula (9) and its input features, respectively, and h and w represent the width and height of the features, respectively.

[0074] The affine injection layer takes the conditional features output by the joint attention Transformer encoding network as input and outputs transformation coefficients to achieve a reversible affine transformation of the features. Equations (12) and (13) describe the forward and inverse transformation processes of the reversible 1×1 convolutional layer:

[0075]

[0076]

[0077] Where Ft represents the conditional feature, which outputs transformation coefficients S3 and T3 through the parameter extraction network NN. NN can be any neural network.

[0078] In this embodiment, the NN is composed of cascaded modules with three convolutional layers and ReLU activation functions connected in sequence. In the first module, the convolutional kernel size of the convolutional layer is 3×3, the number of input channels is the number of channels of the conditional features, the number of output channels is 64, the stride is 1, and the padding is 1. In the second module, the convolutional kernel size of the convolutional layer is 1×1, the number of input channels is 64, the number of output channels is 64, the stride is 1, and the padding is 1. In the third module, the convolutional kernel size of the convolutional layer is 3×3, the number of input channels is 64, the number of output channels is the number of channels of the features to be processed in the current layer, the stride is 1, and the padding is 1. After the features undergo a positive transformation in this layer, the Jacobian determinant value is calculated using formula (14):

[0079]

[0080] Where f3 and x3 represent the transformation function of formula (12) and the characteristics of its input, respectively.

[0081] The cross-affine coupling layer achieves reversible feature transformation based on a cross-learning mechanism, further enhancing the network's expressive power. Its structure is as follows: Figure 7As shown. Taking the forward transformation process of this layer as an example, firstly, the input features are cross-separated along their channel dimension according to the parity of the number of channels to form two parts with the same number of channels. Then, the features undergo a two-stage affine transformation. In the first stage, the features of the upper half undergo an affine transformation while the features of the lower half remain unchanged. The parameters for the affine transformation are obtained by concatenating the features of the lower half and the conditional features along the channel dimension and inputting them into the parameter extraction network. In the second stage, the features of the upper half remain unchanged while the features of the lower half undergo an affine transformation. The parameters for the affine transformation are obtained by concatenating the features of the upper half and the conditional features along the channel dimension and inputting them into the parameter extraction network. Finally, the transformed features are cross-concatenated to restore the original channel order for output. Formulas (15) and (16) describe the forward and inverse transformation processes of the cross-affine coupling layer:

[0082]

[0083]

[0084] Where Ft represents the conditional feature, (S a ,T a ) and (S b ,T b All are transformation coefficients, output by the parameter extraction network NN. In this embodiment, the NN network used in the cross-affine coupling layer is structurally the same as the NN network used in the affine injection layer. The features undergo a positive transformation in this layer, and the Jacobian determinant value is calculated using formula (17):

[0085]

[0086] Where f4 and x4 represent the transformation function of formula (15) and the characteristics of its input, respectively.

[0087] For each invertible transformation block φ in the invertible network i (i = 1, 2, ..., K), whose input features pass sequentially through a reversible activation layer, a reversible 1×1 convolutional layer, an affine injection layer, and a cross-affine coupling layer, and output features. Based on formulas (8), (11), (14), and (17), the features undergo a forward transformation by the invertible transform block, and the Jacobian determinant value is calculated using formula (18):

[0088]

[0089] Step S3: Combine the low-light image from step S1 with its zero elements. Figure 1The input is a Joint Attention Transformer encoding network to extract conditional features and sample a standard normal distribution. These features are then input together with the conditional features into the reversible network in step S2, which generates the enhanced image.

[0090] Example 2

[0091] A low-light image enhancement method based on zero-element feature guidance is proposed. The training process of the low-light image enhancement network based on zero-element feature guidance is as follows: The training image dataset contains pairs of low-light and normal-light images. During the data import stage, the low-light images and their corresponding normal-light images used for training are randomly cropped and rotated at the same position to improve the diversity of the training data, obtaining image pairs of size H×W for network training. During the model training stage, the zero-element map of the low-light image is obtained, and the low-light image and its zero-element map are used as input to the joint attention Transformer encoding network to extract conditional features based on the zero-element information. At the same time, the normal-light image is input to the invertible network. Guided by the extracted conditional features, the forward transformation of the network is performed to map the distribution of the normal-light image to the latent space. The network parameters are optimized based on maximum likelihood estimation to make the latent space distribution fit a standard normal distribution. After the network is trained, the sampled standard normal distribution is used as input to the invertible network along with the conditional features extracted by the joint attention Transformer coding network, and its invertible transformation is performed to finally generate the enhanced image.

[0092] In this embodiment, the image dataset used is LOL-v2-real, which consists of low-light to normal-light image pairs taken in real-world scenes. The image size is (3, 400, 600), of which 689 pairs are used for model training and 100 pairs are used for model testing. In actual training, the image size is randomly cropped to (3, 128, 128). The loss function is:

[0093]

[0094] Among them, L(I L ,I GT ) represents the negative log-likelihood loss function, I L and I GT These represent the low-light image and its corresponding normal-light image, respectively. P y (y|I L P represents the distribution of a normal illumination image conditioned on a low-light image. z (z) represents the potential spatial distribution. φ represents the Jacobian determinant value calculated during the transformation of the invertible network. i and ...

[0095] The low-light image enhancement network trained in this embodiment, guided by zero-element features, enhances 100 test images from the LOL-v2-real dataset and calculates the average PSNR of the 100 enhanced images. The result is 24.06 dB, which is superior to most existing low-light image enhancement methods. An illustration of the enhanced image is shown below. Figure 8 As shown, Figure 8 As can be seen, the enhanced image has appropriate light intensity, normal color, and rich detail, and the severe loss of image information caused by the low light environment has been significantly restored.

[0096] In summary, the low-light image enhancement network based on zero-element feature guidance designed in this invention considers low-light images and their zero-element maps as inputs to the enhancement network to guide the network's learning and training, thereby improving the enhancement network's ability to repair information loss caused by low-light environments.

[0097] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0098] It should be noted that the above content merely illustrates the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. For those skilled in the art, various improvements and modifications can be made without departing from the principle of the present invention, and all such improvements and modifications fall within the scope of protection of the claims of the present invention.

Claims

1. A low-light image enhancement network guided by zero-element features, characterized in that... This includes joint attention Transformer coding networks and invertible networks; The joint attention Transformer coding network includes a shallow feature extraction block, a joint attention deep feature coding block, and a conditional feature output block; wherein, The shallow feature extraction block consists of sequentially connected convolutional layers. Activation function layer and convolutional layers constitute; The joint attention deep feature encoding block consists of 6 joint attention Transformer sub-blocks. Each joint attention Transformer sub-block is composed of a joint attention computation network and a feedforward network connected sequentially. The joint attention computation network is composed of a self-attention module and a zero-element attention module connected in parallel, and the feedforward network consists of a normalization layer. Linear layer Activation function layer and Cascaded structure; the self-attention module contains 3 branches, whose inputs are respectively , and All features are derived from the image features extracted from the shallow feature extraction blocks. Attention is calculated in the three branches, and their outputs will be used as linear layers. The zero-element attention module contains two branches, whose inputs are respectively... and All of them come from zero-element features, and the inputs in the two branches and the self-attention part are... Attention is calculated on the branch, and its output will be used as the linear layer. The input; the features obtained after self-attention and zero-element attention are then processed by linear layers respectively. and After processing, the images are added together, and then the original image features are introduced. By constructing residual connections, output features are obtained. ; The conditional feature output block consists of sequentially connected convolutional layers. Interpolation sampling layer Convolutional layer and interpolation sampling layer constitute; The reversible network is divided into 1 to 3 levels, each level consisting of 1 scale transformation layer and 8 reversible transformation blocks connected in sequence.

2. The low-light image enhancement network based on zero-element feature guidance as described in claim 1, characterized in that: The three branches of the self-attention module in the joint attention computation network have the following inputs: The branch is formed by the normalization layer and linear layer Cascaded, with input as The branch is formed by the normalization layer and linear layer Cascaded, with input as The branch is formed by the normalization layer and linear layer Cascaded structure.

3. The low-light image enhancement network based on zero-element feature guidance as described in claim 1, characterized in that: The zero-element attention module has two branches, with the input being... The branch is formed by the normalization layer and linear layer Cascaded, with input as The branch is formed by the normalization layer and linear layer Cascaded structure.

4. A low-light image enhancement network based on zero-element feature guidance as described in claim 2 or 3, characterized in that: The reversible transformation block of the reversible network contains four cascaded reversible transformation layers: a reversible activation layer, a 1×1 convolutional layer, an affine injection layer, and a cross-affine coupling layer.

5. The low-light image enhancement network based on zero-element feature guidance as described in claim 4, characterized in that: In the cross-affine coupling layer of the reversible network, the input features are cross-separated along their channel dimension based on the parity of the number of channels to form two parts with the same number of channels. The features undergo a two-stage affine transformation, and the transformed features are then cross-joined to restore the original channel order for output.

6. A low-light image enhancement method based on zero-element feature guidance using the network as described in claim 1, characterized in that, Includes the following steps: S1, Data preprocessing: Obtain low-light images and obtain their zero-element maps; S2, Constructing a joint attention Transformer encoding network and a reversible network, and training the network: The joint attention Transformer encoding network and the reversible network constitute a low-light image enhancement network based on zero-element feature guidance. The low-light image and its zero-element map from step S1 are input into the enhancement network, and the network is trained using a negative log-likelihood loss function to obtain the optimized network; S3, Output: Input the low-light image and its zero-element image from step S1 into the joint attention Transformer coding network to extract conditional features and sample a standard normal distribution. Use this and the conditional features extracted by the joint attention Transformer coding network as input to the invertible network in step S2 to perform an invertible transformation and finally generate the enhanced image.

7. A low-light image enhancement method based on zero-element feature guidance according to the network described in claim 6, characterized in that: In step S1, the zero-element diagram is specifically as follows: ； , , These represent the graph in coordinates. At this location, the pixel values ​​of the red channel (R), green channel (G), and blue channel (B) of a pixel. These represent the height index and width index of the image, respectively.

8. The low-light image enhancement method based on zero-element feature guidance as described in claim 6, characterized in that: The loss function for training the network is: ； in, This represents the negative log-likelihood loss function. and These represent the low-light image and its corresponding normal-light image, respectively. This represents the distribution of a normal illumination image conditioned on a low-light image. Indicates the potential spatial distribution. This represents the Jacobian determinant value calculated during the transformation process of the invertible network. and These represent the reversible transform block and the image features it processes, respectively. This represents the conditional features of the input to the reversible transformation layer.

Images