Low-illumination image enhancement method based on Fourier frequency domain information

Abstract

The invention discloses a low-illumination image enhancement method based on Fourier frequency domain information. The low-illumination image enhancement method comprises the following steps: constructing a training set by using a reference data set containing a normal illumination image and a low-illumination image which are paired; constructing a low-illumination image enhancement model comprising an amplitude illumination estimation module and a phase enhancement module, and inputting the training set into the model for training to obtain an enhanced image; constructing a loss function of the enhanced image and the normal illumination image based on mean absolute error loss, structure loss and perception loss, and optimizing a low-illumination image enhancement model according to the loss function; and enhancing the low-illumination image according to the optimized model. Through the synergistic effect of the amplitude illumination estimation module and the phase enhancement module, the brightness of the image is improved while details and definition are still kept, so that the enhancement effect of the low-illumination image is more prominent; and the problems of illumination distribution nonuniformity, color distortion and texture detail loss in a low-illumination environment can be better solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of low-light image enhancement, and particularly relates to a low-light image enhancement method based on Fourier frequency domain information. Background Art

[0002] With the rapid development of computer vision, high-quality images not only record daily life but also support scientific research. However, due to factors such as varying light intensity and limitations of imaging technology, everyday images often exhibit low contrast, artifacts, and significant noise, making it difficult to effectively convey real-world information. This adversely impacts advanced visual tasks such as autonomous driving, security, and biomedicine. However, environmental complexity, technical limitations, and various uncontrollable physical factors can affect the image capture process, resulting in the generation of low-light images. Images captured under low-light conditions typically exhibit low contrast, unclear content and texture, and noise. These issues significantly impact the accuracy and effectiveness of subsequent image processing tasks, making it extremely difficult to extract valid information from images. Typically, only weak, effective information can be obtained, which complicates image processing and can negatively impact more complex visual tasks such as semantic segmentation, object detection, and surveillance systems.

[0003] Low-light image enhancement methods can be categorized into traditional methods and deep learning-based methods. Traditional methods can be broadly divided into histogram equalization and Retinex theory methods. While these methods are effective in increasing overall brightness, their performance relies heavily on hand-crafted constraints and designs, which limits their adaptability and generalization. Compared to traditional methods, deep learning-based methods offer improved generalization, feasibility, and robustness. Although deep learning-based methods have made significant progress in performance in recent years, some existing transformer-based methods, while highly competitive, often rely on expensive hardware resources and large amounts of memory. Most existing methods are based on spatial information and rarely consider Fourier frequency domain information. Some recent low-light image enhancement algorithms have incorporated Fourier frequency and spatial domain information into neural networks, achieving significant improvements. For example, some research methods have explored Fourier and wavelet transform frequency domain information for low-light image enhancement. Li et al. proposed the ultra-high-definition Fourier low-light enhancement network (UHDFour). This method constructs a Fourier convolution module that adjusts the brightness of low-light images in the Fourier frequency domain by separating the amplitude and phase maps. The fused enhanced image effectively avoids noise amplification during the brightness enhancement process. Wang et al. proposed the FourLLIE method, demonstrating that Fourier analysis can extract global features without increasing model parameters. However, this method focuses primarily on amplitude characteristics and ignores the important role of phase information in detail enhancement in the frequency domain, resulting in suboptimal performance in detail enhancement. Xue et al. proposed a CLIP Fourier-guided wavelet diffusion method for low-light image enhancement. This method combines wavelets and Fourier transforms to construct a perceptually guided high-frequency mixing space, effectively reducing the diversity interference in wavelet diffusion. Similarly, Tan et al. proposed a wavelet-based Mamba and Fourier conditioning model. This model combines wavelet and Fourier frequency domain techniques to enhance global brightness in the wavelet domain while refining local details in the Fourier frequency domain. Summary of the Invention

[0004] The purpose of the present invention is to provide a low-light image enhancement method based on Fourier frequency domain information, which solves the problem that existing low-light image enhancement methods improve brightness but perform poorly in detail enhancement.

[0005] The technical solution adopted by the present invention is: a low-light image enhancement method based on Fourier frequency domain information, comprising the following steps: Step 1: Use a benchmark dataset containing paired normal-light images and low-light images to build a training set; Step 2: Build a low-light image enhancement model including an amplitude illumination estimation module and a phase enhancement module, and input the training set into the low-light image enhancement model for training to obtain an enhanced image; Step 3: Construct a loss function between the enhanced image and the normal light image based on the mean absolute error loss, structural loss, and perceptual loss, and optimize the low-light image enhancement model according to the loss function. Step 4: Enhance the low-light image based on the optimized low-light image enhancement model.

[0006] The present invention is also characterized in that: Step 2 specifically includes the following steps: Step 2.1: Low-light images in the training set Perform Fourier transform from spatial domain to frequency domain, and then decompose the frequency domain feature matrix to obtain the amplitude graph A and phase diagram P ; Step 2.2: The amplitude graph A and phase diagram P The amplitude illumination estimation module and phase enhancement module are used for enhancement respectively to obtain the enhanced amplitude component. and phase components ; Step 2.3: The enhanced amplitude component and phase components By inverse Fourier transform Convert back to the spatial domain to generate the enhanced image .

[0007] Step 2.1 is as follows: From the spatial domain through Fourier transform F Convert to frequency domain X , and get the frequency domain feature matrix:

[0008] Where, 、 Represents the input low-light image height and width; h 、 w are coordinates in the spatial domain; 、 are coordinates in Fourier space; is an imaginary unit, and the inverse Fourier transform process is expressed as ; is a complex component, using the amplitude component and phase components express:

[0009] Where, and Respectively The real and imaginary parts of Independently calculate the Fourier transform of each channel of the low-light image to obtain the corresponding amplitude map A and phase diagram P .

[0010] Amplitude component in step 2.2 The calculation method is as follows: A First, the amplitude illumination estimation module estimates the amplitude transformation map by inputting six Fourier processing blocks with skip connections. M , and then the amplitude diagram A Divide by Amplitude Transformation M The enhanced amplitude component ,in A constant to avoid division by zero.

[0011] Amplitude transformation diagram M The specific estimation method is: each Fourier processing block uses a Convolutional layer, two LeakyReLU and one The deformable convolution layer extracts features, and then the amplitude transformation map is limited to the range of (0,1) through the sigmoid activation function to obtain the amplitude transformation map M .

[0012] Phase component in step 2.2 The calculation method is as follows: Step 2.2.1, the phase diagram P First use two Convolutional layer and a LeakyReLU activation function extract features to obtain phase component features ; Step 2.2.2: Low-light images in the training set First, the infrared image is processed by a pre-trained RGB to TIR image conversion model, and then the phase component of the infrared image is extracted by fast Fourier transform. , and then the phase component Use two The convolution layer and a LeakyReLU activation function extract features to obtain the phase information of the enhanced infrared image. ; Step 2.2.3: The phase information obtained in step 2.2.2 and the phase component characteristics obtained in step 2.2.1 Perform fusion to generate phase attention map :

[0013] Step 2.2.4: The phase attention map obtained in step 2.2.3 The phase component characteristics obtained in step 2.2.1 Multiply and add through the residual connection to obtain the enhanced phase component : .

[0014] Step 3 is as follows: construct the enhanced image based on mean absolute error loss, structural loss and perceptual loss Loss function compared to normal lighting images :

[0015] Where, 、 is the weight factor, represents the mean absolute error loss, Indicates structural loss, Indicates perceived loss.

[0016] Mean absolute error loss Expressed as:

[0017] Where, GT Real normal lighting images, represents the total number of pixels in the image, and Represents absolute value operation.

[0018] Structural loss Expressed as:

[0019] Where, Defined as:

[0020] Where, GT Real normal lighting images, 、 Represents images and The local mean of 、 is an image 、 The local variance of Representing an image 、 The local covariance of 、 is a constant to avoid zero.

[0021] Perceptual loss Expressed as:

[0022] Where, GT Real normal lighting images, It is the first The feature map of the layer, This is for the The weights assigned to the layers, Refers to the L1 norm.

[0023] The beneficial effects of the present invention are as follows: the low-light image enhancement method based on Fourier frequency domain information of the present invention enables the image to retain details and clarity while improving brightness through the synergistic effect of the amplitude illumination estimation module and the phase enhancement module; by introducing infrared prior information in the phase enhancement module and using cross-modal infrared image information to guide the structural information of the phase component, the enhancement effect of the low-light image is more prominent, and the problems of uneven illumination distribution, color distortion and loss of texture details in low-light environments can be better handled. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 1 is a flow chart of a low-light image enhancement method based on Fourier frequency domain information according to the present invention; Figure 2 2 is a schematic diagram of the network structure of the low-light image enhancement model in the low-light image enhancement method based on Fourier frequency domain information of the present invention; Figure 3 Schematic diagram of the network structure of the Fourier processing block in the low-light image enhancement method based on Fourier frequency domain information of the present invention; Figure 4 This is a visual comparison chart of the present invention and other methods on the LOL-v2-real dataset; Figure 5 This is a visual comparison chart of the present invention and other methods on the LOL-v2-synthetic dataset; Figure 6 This is a visual comparison chart of the present invention and other methods on the LSRW-Huawei dataset; Figure 7 This is a visual comparison chart of the present invention and other methods on the NPE dataset; Figure 8 This is a visual comparison chart of the present invention and other methods on the MEF dataset; Figure 9 This is a visual comparison chart of the present invention and other methods on the LIME dataset; Figure 10 This is a visual comparison chart of the present invention and other methods on the DICM dataset; Figure 11 This is a visual comparison chart of the present invention and other methods on the VV dataset. DETAILED DESCRIPTION

[0025] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0026] Example 1 The present invention provides a low-light image enhancement method based on Fourier frequency domain information, such as Figure 1 As shown in the figure, it focuses on the reconstruction of Fourier frequency domain information, while enhancing the amplitude and phase components. The two modules work together to obtain complementary information. Specifically, to enhance the amplitude component in low-light images, an amplitude illumination estimation module is designed to improve the brightness of low-light images by estimating the amplitude transform map in Fourier space. For the phase component, a phase enhancement module is designed. By introducing infrared prior information and using cross-modal infrared image information to guide the structural information of the phase component, the image retains details and clarity while improving brightness, making the enhancement effect of low-light images more prominent and better able to deal with the problems of uneven illumination distribution, color distortion, and loss of texture details in low-light environments.

[0027] Example 2 The present invention provides a low-light image enhancement method based on Fourier frequency domain information, which is specifically implemented according to the following steps: Step 1: Use a public benchmark containing paired normal-light images and low-light images to build a training set. Each paired image is from the same scene and is used for low-light image enhancement model training.

[0028] Step 2: Construct a low-light image enhancement model, including an amplitude illumination estimation module and a phase enhancement module. The structure is as follows: Figure 2 As shown, the training set is input into the low-light image enhancement model for training to obtain the enhanced image. The specific steps include: Step 2.1: Low-light images in the training set Perform Fourier transform from spatial domain to frequency domain, and then decompose the frequency domain feature matrix to obtain the amplitude graph A and phase diagram P ; Step 2.2: The amplitude graph A and phase diagram P The amplitude illumination estimation module and phase enhancement module are used for enhancement respectively to obtain the enhanced amplitude component. and phase components ; Step 2.3: The enhanced amplitude component and phase components By inverse Fourier transform Convert back to the spatial domain to generate the enhanced image .

[0029] Step 3: Low-light image enhancement aims to restore image illumination, detail, and color distortion, while also reducing noise introduced during the illumination restoration process. To this end, a loss function is designed consisting of three components: a mean absolute error loss, a structural loss, and a perceptual loss, which construct a loss function between the detail-enhanced image and the normal-light image. This loss function is then used to optimize the low-light image enhancement model, ensuring that the enhanced image has good visual quality.

[0030] Step 4: Enhance the low-light image based on the optimized low-light image enhancement model.

[0031] Example 3 The present invention provides a low-light image enhancement method based on Fourier frequency domain information. Based on Example 2, the preferred process of step 2.1 is as follows: the low-light images in the training set are first converted to the frequency domain space by Fourier transform, and then the amplitude map is obtained by decomposing the frequency domain feature matrix. A and phase diagram P The specific process is as follows: Figure 2 As shown in the upper middle box, the amplitude illumination estimation module mainly consists of six Fourier processing blocks. Figure 3 As shown, a skip connection is used between the two Fourier processing blocks. First, the input low-light image is transformed from the spatial domain features through Fourier transform F Convert to frequency domain X , we get the frequency domain feature matrix, and the Fourier transform formula is:

[0032] in, represents the inverse Fourier transform, 、 Represents the input image The height and width, 、 are coordinates in Fourier space, is an imaginary unit, It is composed of complex values, that is, the amplitude components can be used and phase components express:

[0033] in, and Respectively For RGB images, the Fourier transform of each channel is calculated independently to obtain the corresponding amplitude A and phase P .

[0034] Example 4 The present invention provides a low-light image enhancement method based on Fourier frequency domain information. Based on Example 2, the preferred process of step 2.2 is: For the amplitude A , first input to the amplitude illumination estimation module with six Fourier processing blocks with skip connections to estimate the amplitude transform map M Each Fourier processing block then uses a Convolutional layer, two LeakyReLU and one The deformable convolution layer extracts features. Compared with ordinary convolution, the deformable convolution improves the flexibility of feature extraction and can further improve the brightness information. Finally, the sigmoid activation function is used to limit the amplitude transformation map to the range of (0,1) to obtain the amplitude transformation map. M ; Finally, the amplitude component of the input image A Divide by Amplitude Transformation M The enhanced amplitude , where to avoid division by zero, let .

[0035] For phase P ,like Figure 2 As shown in the lower half of the box, first, use two Convolutional layer and a LeakyReLU activation function are used to further extract features and obtain ; Then, the input low-light image is first processed through a pre-trained RGB to TIR image conversion model to obtain an infrared image. The model focuses on preserving edge details. The model has been trained on the VIPER, FLIRADAS and STheReO datasets and is able to generate infrared images from low-light scenes, thereby improving the quality of infrared images in dark environments. Then, the phase component of the infrared image is extracted by fast Fourier transform (FFT). ,Will Use two The convolution layer and a LeakyReLU activation function are used to further extract features and finally obtain the phase information of the enhanced infrared image. ; Then, the phase information of the enhanced infrared image is Fusion with the phase component features of the visible light image to generate a phase attention map , phase attention map As shown below:

[0036] at last, and Multiply and add through the residual connection to obtain the enhanced phase component :

[0037] Example 5 The present invention provides a low-light image enhancement method based on Fourier frequency domain information. Based on Example 2, the preferred process of step 3 is: constructing an enhanced image based on mean absolute error loss, structural loss and perceptual loss Loss function compared to normal lighting images :

[0038] Among them, 、 are weight factors set to 0.03 and 0.02 respectively based on experience, represents the mean absolute error loss, Indicates structural loss, Indicates perceived loss.

[0039] Mean absolute error loss Defined as:

[0040] Among them, GT is the real normal lighting image, represents the enhanced image, represents the total number of pixels in the image, and Represents absolute value operation.

[0041] Structural loss Defined as:

[0042] in, Defined as:

[0043] in, 、 Represents images and The local mean of 、 is an image 、 The local variance of Representing an image 、 The local covariance of 、 is a constant to avoid zero.

[0044] Perceptual loss Defined as:

[0045] in, It is the first The feature map of the layer is used to capture high-level semantic features. This is for the The weights assigned to the layers, Refers to the L1 norm.

[0046] Example 6 Experimental analysis and performance evaluation: The proposed model is validated on three datasets, including League of Legends (LOL) (v2) and LRSW-Huawei. LOL-v2 is divided into real and synthetic subsets. In addition, the proposed method is evaluated on five unpaired datasets, including DICM, NPE, LIME, MEF, and VV.

[0047] from Figure 4 、 Figure 5 、 Figure 6 、 Figure 7 、 Figure 8 、 Figure 9 、 Figure 10 and Figure 11 From the visualization results, the results of the method of the present invention show better visual quality, higher contrast and color consistency, and achieve the best visual effect in terms of details and brightness.

Claims

1. A low-light image enhancement method based on Fourier frequency domain information, characterized in that: The following steps are involved: Step 1: Use a benchmark dataset containing paired normal-light images and low-light images to build a training set; Step 2: Build a low-light image enhancement model including an amplitude illumination estimation module and a phase enhancement module, and input the training set into the low-light image enhancement model for training to obtain an enhanced image; Step 3: Construct a loss function between the enhanced image and the normal light image based on the mean absolute error loss, structural loss, and perceptual loss, and optimize the low-light image enhancement model according to the loss function. Step 4: Enhance the low-light image based on the optimized low-light image enhancement model.

2. The low-light image enhancement method based on Fourier frequency domain information according to claim 1, characterized in that: The step 2 specifically includes the following steps: Step 2.1: Low-light images in the training set Perform Fourier transform from spatial domain to frequency domain, and then decompose the frequency domain feature matrix to obtain the amplitude graph A and phase diagram P ; Step 2.2: The amplitude graph A and phase diagram P The amplitude illumination estimation module and phase enhancement module are used for enhancement respectively to obtain the enhanced amplitude component. and phase components ; Step 2.3: The enhanced amplitude component and phase components By inverse Fourier transform Convert back to the spatial domain to generate the enhanced image .

3. The low-light image enhancement method based on Fourier frequency domain information according to claim 2, wherein: The step 2.1 is specifically as follows: From the spatial domain through Fourier transform F Convert to frequency domain X , and get the frequency domain feature matrix: Where, 、 Represents the input low-light image height and width; h 、 w are coordinates in the spatial domain; 、 are coordinates in Fourier space; is an imaginary unit, and the inverse Fourier transform process is expressed as ; is a complex component, using the amplitude component and phase components express: Where, and Respectively The real and imaginary parts of Independently calculate the Fourier transform of each channel of the low-light image to obtain the corresponding amplitude map A and phase diagram P .

4. The low-light image enhancement method based on Fourier frequency domain information according to claim 2, wherein: The amplitude component in step 2.2 The calculation method is as follows: A First, the amplitude illumination estimation module estimates the amplitude transformation map by inputting six Fourier processing blocks with skip connections. M , and then the amplitude diagram A Divide by Amplitude Transformation M The enhanced amplitude component ,in A constant to avoid division by zero.

5. The low-light image enhancement method based on Fourier frequency domain information according to claim 4, characterized in that: The amplitude transformation diagram M The specific estimation method is: each Fourier processing block uses a Convolutional layer, two LeakyReLU and one The deformable convolution layer extracts features, and then the amplitude transformation map is limited to the range of (0,1) through the sigmoid activation function to obtain the amplitude transformation map M .

6. The low-light image enhancement method based on Fourier frequency domain information according to claim 2, wherein: The phase component in step 2.2 The calculation method is as follows: Step 2.2.1, the phase diagram P First use two Convolutional layer and a LeakyReLU activation function extract features to obtain phase component features ; Step 2.2.2: Low-light images in the training set First, the infrared image is processed by a pre-trained RGB to TIR image conversion model, and then the phase component of the infrared image is extracted by fast Fourier transform. , and then the phase component Use two The convolution layer and a LeakyReLU activation function extract features to obtain the phase information of the enhanced infrared image. ; Step 2.2.3: The phase information obtained in step 2.2.2 and the phase component characteristics obtained in step 2.2.1 Perform fusion to generate phase attention map : Step 2.2.4: The phase attention map obtained in step 2.2.3 The phase component characteristics obtained in step 2.2.1 Multiply and add through the residual connection to obtain the enhanced phase component : 。 7. The low-light image enhancement method based on Fourier frequency domain information according to claim 2, wherein: The step 3 is specifically as follows: constructing an enhanced image based on mean absolute error loss, structural loss and perceptual loss Loss function compared to normal lighting images : Where, 、 is the weight factor, represents the mean absolute error loss, Indicates structural loss, Indicates perceived loss.

8. The low-light image enhancement method based on Fourier frequency domain information according to claim 7, wherein: The mean absolute error loss Expressed as: Where, GT Real normal lighting images, represents the total number of pixels in the image, and Represents absolute value operation.

9. The low-light image enhancement method based on Fourier frequency domain information according to claim 7, wherein: The structural loss Expressed as: Where, Defined as: Where, GT Real normal lighting images, 、 Represents images and The local mean of 、 is an image 、 The local variance of Representing an image 、 The local covariance of 、 is a constant to avoid zero.

10. The low-light image enhancement method based on Fourier frequency domain information according to claim 7, wherein: The perceptual loss Expressed as: Where, GT Real normal lighting images, It is the first The feature map of the layer, This is for the The weights assigned to the layers, Refers to the L1 norm.

Images