Unsupervised low-light image enhancement method based on frequency domain perception and illumination self-adaptation

Abstract

The application discloses a kind of based on frequency domain perception and illumination self-adaptive unsupervised low-light image enhancement method, comprising: extracting the basic feature of low-light picture by basic feature encoder;The illumination feature of low-light picture is extracted by illumination feature encoder;By frequency domain feature extraction module, the frequency domain feature that is focused on detail texture is converted to basic feature;By color histogram feature extraction module, the local color feature of low-light picture is obtained;By illumination feature decoder, illumination information distribution is learned;By basic feature decoder, the detail texture information of enhanced picture is recovered.The application makes full use of the spatial basic feature and frequency domain detail feature of low-light image, while using adaptive convolution mechanism to learn the illumination condition of different regions, effectively improve the visual quality of enhanced picture.

Description

Technical Field

[0001] This invention belongs to the field of image enhancement technology, specifically relating to an unsupervised low-light image enhancement method based on frequency domain awareness and illumination adaptation. Background Technology

[0002] Visual perception is one of the most important ways for humans to acquire information, and images are a primary carrier of information. With the rapid development of society, people's demand for information is increasing. However, in real-world applications, images often suffer varying degrees of quality degradation due to unpredictable circumstances. Among these degradation issues, addressing low-light degradation remains a pressing problem. This can greatly assist in advanced visual tasks such as object detection, tracking, classification, and recognition, thus enabling applications in many fields, such as urban traffic, surveillance video, and medical assistance. Low-light images are often caused by insufficient light intensity or short exposure time, resulting in most pixels being concentrated in low grayscale areas, while high grayscale areas have very few pixels. Furthermore, the small difference between pixels and their neighbors leads to very low contrast. Dark areas in low light often contain a large amount of noise, making it difficult to distinguish noise from detailed features. This results in low contrast, severe loss of detail, and significant color distortion, making it impossible to extract effective information from the image.

[0003] In existing technologies, low-light image enhancement can be divided into two categories: traditional methods and deep learning-based methods. Traditional enhancement methods are based on histogram equalization or Retinex theory. The former focuses on expanding the dynamic range of image pixels and improving image contrast, while the latter treats the image as a combination of reflection and illumination components, considering the reflection component as the enhancement result, and then estimating the illumination component to obtain the final enhancement result. Although it achieved good results at the time, the decomposition is an ill-posed problem, and there is insufficient constraint on reflectivity, often resulting in overexposure and whitening of the enhanced image. Compared with traditional methods, deep learning-based solutions have higher accuracy and robustness. For example, Lore et al. proposed LLNet, which was the first to apply deep neural networks to low-light image enhancement and proposed an algorithm based on an autoencoder structure to brighten and denoise low-light images. Liang et al. proposed a Transformer-based image restoration method. These deep learning-based enhancement methods have achieved good results, but there are still several major challenges for low-light image enhancement: 1) In the image restoration process, high-frequency texture details of the image are often easily ignored by the network model during training. 2) Existing algorithms often result in overexposure of images, failing to adaptively perceive lighting conditions and adjust enhancement intensity for areas with varying levels of illumination. 3) Deep learning enhancement algorithms based on Retinex theory typically cannot effectively address the precision loss caused by element-wise division (R = I / L). 4) Supervised learning enhancement algorithms cannot adapt well to complex lighting conditions in real life and cannot effectively handle various low-light situations.

[0004] In summary, the methods described above in the existing technology have problems such as difficulty in obtaining illumination pairing datasets, inability to adequately enhance images, unnatural images obtained after color enhancement, and unclear details in the enhanced and restored images. Summary of the Invention

[0005] The purpose of this invention is to propose an unsupervised low-light image enhancement method based on frequency domain awareness and illumination adaptation, in order to solve the problems of difficulty in obtaining paired datasets, unclear restoration of local details in low-light images by low-light enhancement models, and insensitivity to image illumination conditions.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0007] An unsupervised low-light image enhancement method based on frequency domain awareness and illumination adaptation includes:

[0008] The low-light image is processed by an illumination feature encoder, a local color histogram feature extraction module, and a basic feature encoder to obtain illumination features, local color histogram features, and basic features at different scales.

[0009] The frequency domain feature extraction module extracts the basic features at different scales and reconstructs the extracted frequency domain features to obtain the reconstructed frequency domain features corresponding to the basic features at different scales.

[0010] The illumination feature decoder decodes the encoded illumination features at different scales into illumination decoding features for different regional illumination conditions, guided by local color histogram features at the same scale.

[0011] The basic feature decoder decodes basic features at different scales. The basic decoded features output by each decoding block of the basic feature decoder are combined with the illumination decoded features at the same scale through the context-sensitive aggregation module to obtain aggregated features. Guided by the reconstructed frequency domain features at the same scale, the aggregated features are corrected by the detail correction module. The corrected result is used as the input of the next decoding block. When the basic decoded features output by the last decoding block are combined with the illumination decoded features at the same scale, the enhanced image with improved illumination conditions is obtained.

[0012] Furthermore, feature extraction is performed on the low-light image using a basic feature encoder to obtain basic features at different scales, including:

[0013] Low-light images are downsampled through multiple convolutions to obtain basic features at different scales. The basic features obtained from each convolution downsampling are fused with the low-frequency feature components of the basic features at the previous scale and then input into the next convolution block for further convolution downsampling to obtain the basic features at the next scale.

[0014] Furthermore, the step of extracting frequency domain features from basic features at different scales using the frequency domain feature extraction module, and reconstructing the extracted frequency domain features to obtain reconstructed frequency domain features corresponding to basic features at different scales, includes:

[0015] The basic features of the first N-1 scales generated by the basic feature encoder are input into the corresponding frequency domain feature extraction module. For each basic feature, the feature components of four frequencies, LL, HL, LH and HH, are extracted by wavelet transform. The three high-frequency feature components, HL, LH and HH, are combined to obtain the hybrid high-frequency feature.

[0016] The mixed high-frequency features are downsampled into C×1×1 feature vectors through global pooling, and then further processed by 1×1 convolution for dimensionality reduction, PReLU activation function, and 1×1 convolution to generate global feature vectors. Simultaneously, the mixed high-frequency features are processed by 3×3 convolution, PReLU activation function, and 3×3 convolution to generate local feature vectors. The global feature vector and local feature vector are summed to form a weight map. The weight map is then processed by three 3×3 convolutions and a Sigmoid activation function to obtain the weight maps of the corresponding three frequency features. Based on the weight maps of the three frequency features, three high-frequency feature components are selected, and the selected three high-frequency feature components are aggregated along the channels to obtain the final mixed high-frequency features.

[0017] The low-frequency feature component LL is normalized, and the unprocessed LL is used as Q and K of self-attention. The final mixed high-frequency feature is used as V. The cross-attention feature map is calculated. The cross-attention feature map is passed through LayerNorm and then the mean μ and variance σ are learned by two 3x3 convolutions. The mean μ and variance σ are injected into the normalized LL to obtain the final frequency domain feature.

[0018] Furthermore, the low-light image is feature extracted using a local color histogram feature extraction module to obtain local color histogram features at different scales, including:

[0019] The low-light image pixels are set to the range of [0,1], and then the low-light image is divided into [h / K]×[w / K] pixel blocks according to the K value. By statistically analyzing the pixel values ​​in each pixel block, the color distribution of that pixel block is obtained, which serves as a token for the final color feature. The K value determines the range of the statistical pixel block represented by the token, and the value of K is determined according to the scale of the local color histogram feature.

[0020] Furthermore, the step of decoding the encoded illumination features at different scales using an illumination feature decoder, guided by local color histogram features of the same scale, into illumination decoding features that adaptively learn illumination conditions for different regions, includes:

[0021] The local color histogram features at different scales are convolved and the argmax() function is used to obtain the corresponding guiding mask, thereby dividing the illumination decoding features at the corresponding scale into n different illumination regions;

[0022] The basic features of the last scale are introduced into the lighting features of the last scale and input into the lighting feature decoder. The lighting features of each scale are first processed by adaptive average pooling, and then by convolution, sigmoid activation function and convolution to obtain the parameters of n convolution kernels.

[0023] Guided by the guiding mask, different convolution kernels are applied to n different lighting regions at corresponding scales to learn to distinguish the lighting conditions of different regions.

[0024] Furthermore, the basic decoding features output by each decoding block of the basic feature decoder are combined with the illumination decoding features of the same scale through a context-sensitive aggregation module to obtain aggregated features. Guided by the reconstructed frequency domain features of the same scale, the obtained aggregated features are corrected by a detail correction module, including:

[0025] The basic decoding features at each scale are combined with the illumination decoding features at the same scale through a context-sensitive aggregation module to obtain aggregated features.

[0026] The aggregated features are input into the detail correction module. They first pass through LayerNorm and then through three fully connected layers to obtain Q, K, and V respectively. V is multiplied by the reconstructed frequency domain features of the same scale to generate a new V1. Q and K are multiplied to obtain the corresponding attention map map. The map is then multiplied by V1 and combined with position encoding to obtain the attention features. The attention features are summed with the aggregated features to obtain the first feature map. The first feature map is then passed through LayerNorm and FFN to obtain the second feature map. The first feature map and the second feature map are summed to obtain the final enhanced features.

[0027] Furthermore, the unsupervised low-light image enhancement method based on frequency domain awareness and illumination adaptation also includes:

[0028] The generated enhanced image and the ideal lighting image are input together into the discriminator to determine whether the enhanced image meets the normal lighting conditions.

[0029] Furthermore, the step of inputting the generated enhanced image and the ideal lighting image together into the discriminator to determine whether the enhanced image meets the normal lighting conditions includes:

[0030] The enhanced image and the ideal lighting image are input into the global discriminator. The global discriminator compares the enhanced image and the ideal lighting image to determine whether the enhanced image belongs to normal lighting.

[0031] The same number of pixel blocks are randomly cropped from both the enhanced image and the ideal lighting image. The cropped pixel blocks are then compared using a local discriminator to determine whether the enhanced image belongs to normal lighting.

[0032] The final judgment result is obtained by weighted summing of the judgment results of the global discriminator and the local discriminator.

[0033] Furthermore, the illumination feature encoder, the local color histogram feature extraction module, the basic feature encoder, the frequency domain feature extraction module, the basic feature decoder, the illumination feature decoder, and the detail correction module constitute a low-light image enhancement model. The training method for the low-light image enhancement model is as follows:

[0034] Obtain a dataset of low-light images and a dataset of images under normal lighting conditions;

[0035] Based on a low-light image dataset, low-light images are input into a low-light image enhancement model, and the low-light image enhancement model outputs an enhanced image after improving lighting conditions.

[0036] A normal lighting condition image is randomly selected from the normal lighting condition image dataset. The enhanced image and the randomly selected normal lighting condition image are input into the discriminator to determine whether the enhanced image meets the normal lighting condition.

[0037] The low-light image enhancement model is trained by combining the illumination decoding features output by the illumination feature decoder and the grayscale image of the low-light image as a set of illumination smoothing losses, and the enhanced image and the image under normal lighting conditions as a set of adversarial losses of the GAN. The two are combined into the final total loss function, with the goal of reducing the value of the total loss function.

[0038] Compared with the prior art, the beneficial effects of the present invention are:

[0039] This invention presents an unsupervised low-light image enhancement method based on frequency domain awareness and illumination adaptation. It learns and estimates the illumination conditions of low-light images through an illumination feature encoder, a local color histogram feature extraction module, and an illumination feature decoder. Combining the appearance information obtained from the local color histogram features helps the network model distinguish different illumination regions, guiding the model to apply different convolutional kernels to different illumination areas and adaptively perceive illumination changes in the image. Image restoration is performed through a basic feature encoder, a frequency domain feature extraction module, a basic feature decoder, and a detail correction module. The method leverages the frequency domain information's focus on detail and texture, helping the network recover more fine details from the image under the guidance of frequency domain features. Simultaneously, the restored features and illumination features are connected through a context-sensitive aggregation module, aggregating contextual dependency information. The detail correction module also helps the model better fit the details, giving the restored features better representation and thus achieving better enhanced visual effects. This effectively improves the visual quality of the enhanced image, demonstrating a reasonable design and significant results. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the method flow of an embodiment of the present invention;

[0041] Figure 2This is a schematic diagram of the frequency domain feature extraction module structure according to an embodiment of the present invention;

[0042] Figure 3 This is a schematic diagram of the structure of the local color histogram guidance module according to an embodiment of the present invention;

[0043] Figure 4 This is a schematic diagram of the detailed correction module structure according to an embodiment of the present invention;

[0044] Figure 5 This is an enhanced example diagram of an embodiment of the present invention. Detailed Implementation

[0045] The present invention will be further described below with reference to specific embodiments. These embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0046] To address the issue that existing low-light enhancement models are not ideal for restoring image details, this invention designs a frequency domain feature extraction module in the lower branch to extract frequency domain features from basic features. Leveraging its focus on texture characteristics, this module compensates for the lack of detail restoration in enhanced images, improving the visual perception quality of the enhanced images. To address the common overexposure / underexposure issues in existing low-light enhancement models, this invention extracts local color histogram features from the image and uses their information distribution to predict adaptive convolutional kernels, helping the network distinguish and learn the illumination of different regions of the image, thus improving the network's sensitivity to lighting conditions. To address the potential data overflow issues caused by element-wise division in the Retinex algorithm, this invention replaces the original method with element-wise multiplication and a detail correction module to aggregate semantic information. Finally, to address the difficulty in obtaining paired data, the model employs an unsupervised training method.

[0047] like Figure 1 As shown, this embodiment of the invention provides an unsupervised low-light image enhancement method based on frequency domain awareness and illumination adaptation. This method is implemented based on a low-light image enhancement model, which is designed as a two-stream estimation method. One stream adaptively learns the illumination conditions of the image, while the other is responsible for detail information recovery. The illumination estimation branch includes an illumination feature encoder, a local color histogram feature extraction module, and an illumination feature decoder; the other branch includes a basic feature encoder, a frequency domain feature extraction module (FPFM), a basic feature decoder, and a detail restoration module (Restore).

[0048] Specifically, the method includes the following steps:

[0049] Step 1: Extract features from low-light images using a basic feature encoder to obtain basic features at different scales;

[0050] The low-light image is randomly cropped to a size of 256×256, and then subjected to one convolution and three convolution downsampling to obtain features of four different scales: 256×256, 128×128, 64×64, and 32×32. At the same time, the low-frequency feature components of the basic features of the previous scale are introduced into the basic features of different scales as the input of the next convolution block.

[0051] Step 2: Extract features from low-light images using a lighting feature encoder to obtain lighting features at different scales;

[0052] The illumination feature encoder is a feature extraction module that uses multiple convolutional blocks stacked together.

[0053] The cropped 256×256 low-light image is then subjected to one convolution and three convolution downsampling operations to obtain illumination feature pyramids with scales of 256×256, 128×128, 64×64, and 32×32, respectively.

[0054] Step 3: Extract features from the low-light image using the local color histogram feature extraction module to obtain local color histogram features at different scales;

[0055] The local color distribution histogram features at different scales are extracted using the Bilateral Grid algorithm, specifically including:

[0056] The pixels of the cropped 256×256 low-light image are set to the interval [0,1]. The low-light image is then divided into [h / K]×[w / K] pixel blocks according to the K value. By statistically analyzing the pixel values ​​in each pixel block, the color distribution of that pixel block is obtained, which serves as a token for the final color feature. The K value determines the range of statistical pixel blocks represented by the token, with K taking values ​​of {1, 2, 4, 8}. This yields local color histogram feature pyramids for 256×256, 128×128, 64×64, and 32×32 pixels.

[0057] Step 4: Extract frequency domain features from basic features at different scales using the frequency domain feature extraction module, and reconstruct the extracted frequency domain features to obtain the reconstructed frequency domain features corresponding to basic features at different scales.

[0058] like Figure 2 As shown, this step specifically includes:

[0059] Step 41: Input the basic features of the three scales of 256×256, 128×128 and 64×64 generated by the basic feature encoder into the corresponding frequency domain feature extraction module respectively. Extract the feature components of four frequencies, LL, HL, LH and HH, through wavelet transform. Then, add HL, LH and HH into a mixed high frequency feature through three learnable parameters α1, α2 and α3.

[0060] Step 42: The mixed high-frequency features are downsampled into C×1×1 feature vectors (C varies for different depths) through global pooling. Then, dimensionality is restored using 1×1 convolution, PReLU activation function, and 1×1 convolution to generate a C×1×1 global feature vector, learning global information. Simultaneously, another branch generates local feature vectors from the mixed high-frequency features through 3×3 convolution, PReLU activation function, and 3×3 convolution, learning local information. The global and local feature vectors from both branches are summed to form a weight map. The weight map is then passed through three 3×3 convolutions and a Sigmoid activation function to obtain the weight maps of the corresponding three-frequency features. Based on the weight maps of the three-frequency features, three high-frequency feature components are selected. The selected three high-frequency feature components are then aggregated along the channels to obtain the final mixed high-frequency features.

[0061] Step 43: Normalize the low-frequency feature component LL, and use the unprocessed LL as Q and K of self-attention. Use the final mixed high-frequency feature obtained in step 32 as V to calculate the cross-attention feature map. After passing the cross-attention feature map through LayerNorm, learn the mean μ and variance σ through two 3x3 convolutions respectively. Inject the mean μ and variance σ into the normalized LL to obtain the final frequency domain feature.

[0062] Step 5: Using the illumination feature decoder, the encoded illumination features at different scales are decoded into illumination decoding features for different regional illumination conditions under the guidance of local color histogram features at the same scale.

[0063] Combination Figure 3 As shown, this step specifically includes:

[0064] Step 51: The local color histogram features (LCDM) of different scales generated in step 3 are convolved and the argmax() function is used to obtain the corresponding guided mask, thereby dividing the illumination decoding features of the corresponding scale into n different illumination regions;

[0065] Step 52: Introduce basic features of the same scale into the 32×32 illumination features and input them into the illumination feature decoder. The illumination features of each scale are first processed by adaptive average pooling, and then by convolution, sigmoid activation function and convolution to obtain the parameters of n convolution kernels {W1, W2, ... Wn}.

[0066] Step 53: Guided by the generated guide mask, different convolutional kernels are applied to n different lighting regions at the corresponding scale to learn different lighting distributions, helping the network learn to distinguish the lighting conditions of different lighting regions, such as overexposed and underexposed regions.

[0067] Step 6: The basic features at different scales are decoded by the basic feature decoder. The basic decoded features output by each decoding block of the basic feature decoder are combined with the illumination decoded features at the same scale through the context-sensitive aggregation module to obtain aggregated features. Under the guidance of the reconstructed frequency domain features at the same scale, the aggregated features are corrected by the detail correction module. The corrected result is used as the input of the next decoding block. When the basic decoded features output by the last decoding block are combined with the illumination decoded features at the same scale, the enhanced image with improved illumination conditions is obtained.

[0068] Combination Figure 1 and Figure 4 As shown, this step specifically includes:

[0069] Step 61: The 32×32 basic decoding features and the illumination decoding features of the same scale are semantically aggregated through a context-sensitive aggregation module. This involves multiplying the basic decoding features and illumination decoding features of the same scale element-wise to aggregate their semantic information, allowing the basic features to locally perceive the distribution of illumination, thereby obtaining the aggregated feature F. in ;

[0070] Step 62, aggregate feature F in The input is fed into the detail correction module, first passing through a normalization layer (LayerNorm), then through three fully connected layers to obtain Q, K, and V respectively. Finally, V is compared with the reconstructed frequency domain features F at the same scale. Fin Multiply Q and K to generate a new V1; multiply Q and K to obtain the corresponding attention map map, then multiply the map and V1 and combine them with positional encoding to obtain the attention features. Combine the attention features with the aggregated features F. in The first feature map is obtained by summing the features. The first feature map is then passed through LayerNorm and Feedforward Neural Network (FFN) to obtain the second feature map. The first feature map and the second feature map are summed to obtain the final enhanced feature.

[0071] By introducing aggregated features into the detail correction module, and simultaneously introducing frequency domain features of the same scale, more refined details can be learned under the guidance of the frequency domain features, thereby correcting the aggregated features.

[0072] Step 63: The obtained enhanced features are used as input for the next decoding block to obtain 64×64 basic decoding features. The 64×64 basic decoding features are then combined with illumination decoding features of the same scale and semantic information is aggregated through the context-sensitive aggregation module. Step 62 is then executed, but in step 62, 64×64 reconstructed frequency domain features are introduced to obtain new enhanced features as input for the next decoding block. This process continues until 256×256 basic decoding features are obtained. These 256×256 basic decoding features are then combined with illumination decoding features of the same scale to finally obtain an enhanced image with complete details.

[0073] Furthermore, the method also includes:

[0074] Step 7: Input the generated enhanced image and the ideal lighting image into the discriminator to determine whether the enhanced image meets the normal lighting conditions;

[0075] ,like Figure 1 As shown, this step specifically includes:

[0076] Step 71: Input the enhanced image and the ideal lighting image into the global discriminator. The global discriminator compares the enhanced image and the ideal lighting image to determine whether the enhanced image belongs to normal lighting.

[0077] Step 72: Randomly crop 5 pixel blocks from both the enhanced image and the ideal lighting image, and compare the cropped pixel blocks using a local discriminator to determine whether the enhanced image belongs to normal lighting.

[0078] Step 73: The judgment results of the global discriminator and the judgment results of the local discriminator are weighted and summed to obtain the final judgment result.

[0079] The low-light image enhancement model can be trained through the following steps:

[0080] Step ss1: Obtain the low-light image dataset and the image dataset under normal lighting conditions;

[0081] The low-light image dataset and the normal-light image dataset do not need to be in one-to-one correspondence.

[0082] Step ss2: Based on the low-light image dataset, input the low-light images into the low-light image enhancement model, and the low-light image enhancement model outputs the enhanced image after improving the lighting conditions.

[0083] Step ss3: Randomly select an image from the image dataset under normal lighting conditions, and input the enhanced image and the randomly selected image under normal lighting conditions into the discriminator to determine whether the enhanced image meets the normal lighting conditions.

[0084] Step ss4: The illumination decoding features and the grayscale image of the low-light image are used as a set of illumination smoothing losses, and the enhanced image and the image under normal lighting conditions are used as a set of adversarial losses of GAN. The two are merged into the objective function, and the model parameters are updated through the backpropagation algorithm.

[0085] The batch size was set to 1 and the training epochs to 300. The Adam algorithm was used to optimize the network. After the network was trained, PSNR and SSIM were used to evaluate the model performance.

[0086] After the model is trained, the new low-light image is input into the low-light image enhancement model to obtain the enhanced result after improving the lighting conditions. For example... Figure 5 As shown in the figure, the comparison results between the input low-light image and the enhanced image output by the model can be seen. It can be seen that the model of the present invention can effectively improve the lighting conditions of low-light images and effectively improve the visual quality of the enhanced images.

[0087] This invention modifies U-Net and uses it as the network backbone. A frequency domain feature extraction module obtains reconstructed frequency domain features of different sizes, and then uses these reconstructed frequency domain features in skip connections to guide the network in correcting and restoring the detailed texture information of the enhanced image. During continuous downsampling of U-Net, low-frequency features are used for bridging to maintain the structural stability of the features and mitigate information loss during continuous downsampling. A local color histogram pyramid is used to guide the network to learn the lighting conditions of different regions of the image, thereby helping the network adapt to different lighting conditions. The enhanced image obtained through these improvements is then input into the discriminator for end-to-end training of the entire network, resulting in an unsupervised low-light image enhancement model.

[0088] This invention makes full use of the spatial general features and frequency domain detail features of low-light images, and at the same time uses an adaptive convolution mechanism to learn the lighting conditions of different regions, effectively improving the visual quality of the enhanced images.

[0089] The present invention has been disclosed above with reference to preferred embodiments, but it is not intended to limit the present invention. All technical solutions obtained by adopting equivalent substitutions or equivalent transformations fall within the protection scope of the present invention.

Claims

1. An unsupervised low-light image enhancement method based on frequency domain sensing and illumination adaptation, characterized in that, include: The low-light image is processed by an illumination feature encoder, a local color histogram feature extraction module, and a basic feature encoder to obtain illumination features, local color histogram features, and basic features at different scales. The frequency domain feature extraction module extracts the basic features at different scales and reconstructs the extracted frequency domain features to obtain the reconstructed frequency domain features corresponding to the basic features at different scales. The illumination feature decoder decodes the encoded illumination features at different scales into illumination decoding features for different regional illumination conditions, guided by local color histogram features at the same scale. The basic feature decoder decodes basic features at different scales. The basic decoded features output by each decoding block of the basic feature decoder are combined with the illumination decoded features at the same scale through the context-sensitive aggregation module to obtain aggregated features. Guided by the reconstructed frequency domain features at the same scale, the aggregated features are corrected by the detail correction module. The corrected result is used as the input of the next decoding block. When the basic decoded features output by the last decoding block are combined with the illumination decoded features at the same scale, the enhanced image with improved illumination conditions is obtained.

2. The unsupervised low-light image enhancement method based on frequency domain sensing and illumination adaptation according to claim 1, characterized in that, Feature extraction is performed on low-light images using a basic feature encoder to obtain basic features at different scales, including: Low-light images are downsampled through multiple convolutions to obtain basic features at different scales. The basic features obtained from each convolution downsampling are fused with the low-frequency feature components of the basic features at the previous scale and then input into the next convolution block for further convolution downsampling to obtain the basic features at the next scale.

3. The unsupervised low-light image enhancement method based on frequency domain sensing and illumination adaptation according to claim 1, characterized in that, The step of extracting frequency domain features from basic features at different scales using a frequency domain feature extraction module, and reconstructing the extracted frequency domain features to obtain reconstructed frequency domain features corresponding to basic features at different scales, includes: The basic features of the first N-1 scales generated by the basic feature encoder are input into the corresponding frequency domain feature extraction module. For each basic feature, the feature components of four frequencies, LL, HL, LH and HH, are extracted by wavelet transform. The three high-frequency feature components, HL, LH and HH, are combined to obtain the hybrid high-frequency feature. The mixed high-frequency features are downsampled into C×1×1 feature vectors through global pooling, and then further processed by 1×1 convolution for dimensionality reduction, PReLU activation function, and 1×1 convolution to generate global feature vectors. Simultaneously, the mixed high-frequency features are processed by 3×3 convolution, PReLU activation function, and 3×3 convolution to generate local feature vectors. The global feature vector and local feature vector are summed to form a weight map. The weight map is then processed by three 3×3 convolutions and a Sigmoid activation function to obtain the weight maps of the corresponding three frequency features. Based on the weight maps of the three frequency features, three high-frequency feature components are selected, and the selected three high-frequency feature components are aggregated along the channels to obtain the final mixed high-frequency features. The low-frequency feature component LL is normalized, and the unprocessed LL is used as Q and K of self-attention. The final mixed high-frequency feature is used as V. The cross-attention feature map is calculated. The cross-attention feature map is passed through LayerNorm and then the mean μ and variance σ are learned by two 3x3 convolutions. The mean μ and variance σ are injected into the normalized LL to obtain the final frequency domain feature.

4. The unsupervised low-light image enhancement method based on frequency domain sensing and illumination adaptation according to claim 1, characterized in that, The local color histogram feature extraction module extracts features from low-light images to obtain local color histogram features at different scales, including: The low-light image pixels are set to the range of [0,1], and then the low-light image is divided into [h / K]×[w / K] pixel blocks according to the K value. By statistically analyzing the pixel values ​​in each pixel block, the color distribution of that pixel block is obtained, which serves as a token for the final color feature. The K value determines the range of the statistical pixel block represented by the token, and the value of K is determined according to the scale of the local color histogram feature.

5. The unsupervised low-light image enhancement method based on frequency domain sensing and illumination adaptation according to claim 1, characterized in that, The process of decoding encoded illumination features at different scales using an illumination feature decoder, guided by local color histogram features of the same scale, into adaptively learned illumination decoding features for different regional illumination conditions, includes: The local color histogram features at different scales are convolved and the argmax() function is used to obtain the corresponding guiding mask, thereby dividing the illumination decoding features at the corresponding scale into n different illumination regions; The basic features of the last scale are introduced into the lighting features of the last scale and input into the lighting feature decoder. The lighting features of each scale are first processed by adaptive average pooling, and then by convolution, sigmoid activation function and convolution to obtain the parameters of n convolution kernels. Guided by the guiding mask, different convolution kernels are applied to n different lighting regions at corresponding scales to learn to distinguish the lighting conditions of different regions.

6. The unsupervised low-light image enhancement method based on frequency domain sensing and illumination adaptation according to claim 1, characterized in that, The basic decoding features output by each decoding block of the basic feature decoder are combined with the illumination decoding features of the same scale through a context-sensitive aggregation module to obtain aggregated features. Guided by the reconstructed frequency domain features of the same scale, the aggregated features are corrected by a detail correction module, including: The basic decoding features at each scale are combined with the illumination decoding features at the same scale through a context-sensitive aggregation module to obtain aggregated features. The aggregated features are input into the detail correction module. They first pass through LayerNorm and then through three fully connected layers to obtain Q, K, and V respectively. V is multiplied by the reconstructed frequency domain features of the same scale to generate a new V1. Q and K are multiplied to obtain the corresponding attention map map. The map is then multiplied by V1 and combined with position encoding to obtain the attention features. The attention features are summed with the aggregated features to obtain the first feature map. The first feature map is then passed through LayerNorm and FFN to obtain the second feature map. The first feature map and the second feature map are summed to obtain the final enhanced features.

7. The unsupervised low-light image enhancement method based on frequency domain sensing and illumination adaptation according to claim 1, characterized in that, Also includes: The generated enhanced image and the ideal lighting image are input together into the discriminator to determine whether the enhanced image meets the normal lighting conditions.

8. The unsupervised low-light image enhancement method based on frequency domain sensing and illumination adaptation according to claim 7, characterized in that, The step of inputting the generated enhanced image and the ideal lighting image together into the discriminator to determine whether the enhanced image meets the normal lighting conditions includes: The enhanced image and the ideal lighting image are input into the global discriminator. The global discriminator compares the enhanced image and the ideal lighting image to determine whether the enhanced image belongs to normal lighting. The same number of pixel blocks are randomly cropped from both the enhanced image and the ideal lighting image. The cropped pixel blocks are then compared using a local discriminator to determine whether the enhanced image belongs to normal lighting. The final judgment result is obtained by weighted summing of the judgment results of the global discriminator and the local discriminator.

9. The unsupervised low-light image enhancement method based on frequency domain sensing and illumination adaptation according to claim 1, characterized in that, The illumination feature encoder, local color histogram feature extraction module, basic feature encoder, frequency domain feature extraction module, basic feature decoder, illumination feature decoder, and detail correction module constitute a low-light image enhancement model. The training method for the low-light image enhancement model is as follows: Obtain a dataset of low-light images and a dataset of images under normal lighting conditions; Based on a low-light image dataset, low-light images are input into a low-light image enhancement model, and the low-light image enhancement model outputs an enhanced image after improving lighting conditions. A normal lighting condition image is randomly selected from the normal lighting condition image dataset. The enhanced image and the randomly selected normal lighting condition image are input into the discriminator to determine whether the enhanced image meets the normal lighting condition. The low-light image enhancement model is trained by combining the illumination decoding features output by the illumination feature decoder and the grayscale image of the low-light image as a set of illumination smoothing losses, and the enhanced image and the image under normal lighting conditions as a set of adversarial losses of the GAN. The two are combined into the final total loss function, with the goal of reducing the value of the total loss function.

Images