Low-illumination image enhancement method and system based on multi-domain fusion and lightweight design

Abstract

The invention discloses a low-illumination image enhancement method and system based on multi-domain fusion and lightweight design. Firstly, a multi-domain feature fusion module fusing wavelet transform and Fourier transform is provided, and the global structure and detail information of a low-illumination image can be jointly extracted and enhanced; secondly, correlation modeling among multi-scale features is enhanced through a cross-scale attention module, and self-adaptive and multi-scale feature fusion is achieved; furthermore, a gating attention alignment module is provided, the accuracy of feature alignment of the encoder and the decoder is improved, and effective transmission of detailed information is ensured. According to the method, on data sets such as LOLv1, LOLv2-real, LOLv2-syn and the like, the performance expressions of 25.03 dB PSNR and 0.863 SSIM are achieved with the calculation expenditure not exceeding 1.2 M parameters and 1.41 G FLOPS, and the enhancement effect and capability of the model on a low-illumination image are remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of low-light image enhancement, specifically relating to a low-light image enhancement method and system based on multi-domain fusion and lightweight design. Background Technology

[0002] With the rapid development of computer vision technology, Low-Light Image Enhancement (LLIE), as a key technology, is widely used in fields such as nighttime surveillance, autonomous driving, medical image diagnosis, and mobile phone photography. LLIE processes low-brightness, high-noise image data, significantly improving image brightness, detail, and color fidelity, suppressing noise interference, and providing high-quality data for subsequent computer vision tasks such as object detection and image segmentation, ensuring the stable operation of various intelligent systems. However, low-light images often suffer from insufficient brightness, blurred details, color distortion, and noise interference, greatly affecting visual effects and the accuracy of subsequent tasks. Therefore, how to efficiently and accurately perform low-light image enhancement has become an important research direction in the field of computer vision.

[0003] Traditional LLIE methods, such as histogram equalization, Retinex theory, and gamma correction, effectively improve brightness and contrast in certain simple scenes by manually adjusting the image brightness distribution or decomposing illumination and reflection components. However, in complex low-light environments, these methods often lead to color distortion, loss of detail, and a lack of adaptability, making them unsuitable for the needs of modern vision systems. In recent years, deep learning methods have driven the advancement of LLIE technology. For example, EnlightenGAN, based on generative adversarial networks (GANs), can improve image quality under unsupervised conditions; Restormer enhances the processing capability of complex image information by combining global self-attention of Transformers and local features of convolutional neural networks (CNNs); and methods such as DMFourLLIE and Wave-Mamba improve image brightness and detail reconstruction through Fourier and wavelet transforms, achieving better enhancement results. However, these deep learning methods still face problems such as high computational cost, slow inference speed, and incomplete optimization of fusion strategies, especially in the optimization and fusion of frequency domain components, where many challenges remain and further improvements are urgently needed.

[0004] Patent (CN120689257A) discloses a multispectral fusion method for enhancing and compensating for occlusion in low-light nighttime images. This method collaboratively acquires multimodal data using a visible light camera, infrared sensor, thermal imaging sensor, and millimeter-wave radar. It combines a multi-scale transformation algorithm, a U-Net network, and edge fusion and illumination smoothing techniques to enhance low-light nighttime images and compensate for occlusion information. On one hand, the multimodal data encompasses the base image, dark light texture, temperature distribution, and the contours and motion information of objects behind occlusions, containing more comprehensive scene information than data from a single sensor. On the other hand, the multi-scale transformation algorithm dynamically balances the weights of visible light and infrared features, the U-Net network accurately predicts the texture of occluded areas, and edge fusion and illumination smoothing techniques ensure image consistency. This multi-technology collaboration not only solves the problems of high image noise and lack of detail in low-light environments but also compensates for the inability of traditional methods to penetrate occlusions, significantly improving the clarity and integrity of nighttime images.

[0005] Patent (CN118333867A) discloses a low-light image enhancement method guided by pixel RGB value imbalance features. This method acquires a low-light image and its RGB value imbalance point map, inputs it into a Transformer-Unet enhancement network guided by RGB value imbalance points, and outputs an enhanced image after training. On one hand, the RGB value imbalance point map can accurately locate distorted areas of image information, providing a clear target for enhancement; on the other hand, the Transformer-Unet network integrates local and global feature encoding / decoding modules. Local Transformer blocks repair distorted details based on imbalance points, while global Transformer blocks compensate for global information through self-attention. The parallel operation of these two modules not only improves the problem of traditional models' poor learning of distorted areas but also ensures the global consistency and local clarity of the enhanced image.

[0006] Patent (CN120746845A) discloses a low-light binocular stereo image enhancement method based on multi-scale wavelet domain decoupling. This method preprocesses the binocular stereo image dataset for low-light conditions, constructs a low-light stereo image enhancement network based on wavelet decoupling, and outputs enhanced images after training and testing. On the one hand, binocular stereo images contain spatial correlation information of the left and right views, providing richer scene dimensions than monocular images; on the other hand, multi-scale wavelet transform can accurately separate low-frequency illumination information and high-frequency texture information. Combined with illumination adjustment, high-frequency guided stereo matching, and high-frequency feature enhancement modules, it achieves synergistic optimization of low-frequency color correction and high-frequency detail cross-view interaction, solving the problems of insufficient image brightness and poor color recovery under low light conditions, and improving the detail integrity and matching accuracy of stereo images.

[0007] Existing low-light image enhancement methods mostly rely on complex network architectures and feature extraction mechanisms. However, these methods generally suffer from high computational complexity, high resource consumption, and limited detail recovery in low-light environments. Therefore, there is an urgent need for a method that can achieve higher-quality image enhancement while ensuring computational efficiency, especially in resource-constrained application scenarios. This method should effectively improve the brightness and detail of low-light images while balancing the needs of noise suppression and detail recovery. Summary of the Invention

[0008] To address the issues of high computational complexity and difficulty in adapting to resource-constrained scenarios, this invention constructs a lightweight architecture. First, it employs a small-dimensional feature channel and streamlined module design, retaining only three levels in the end-to-end encoder-decoder structure, with each level containing only 1-4 modules. This reduces redundant computation and avoids the high complexity of multi-head self-attention, instead using local attention and lightweight gating mechanisms to optimize the modules. Second, it proposes a multi-domain feature fusion module that integrates wavelet transform and Fourier transform, enabling joint extraction and enhancement of global structure and detail information in low-light images, addressing the problem of insufficient utilization of frequency domain information in existing methods. Third, it proposes a cross-scale attention module to strengthen the correlation modeling between multi-scale features, achieving adaptive, multi-scale feature fusion. Finally, it proposes a gated attention alignment module to improve the accuracy of feature alignment between the encoder and decoder, ensuring the effective transmission of detail information.

[0009] The objective of this invention is achieved through the following technical solution:

[0010] Step 1: Construct an encoder-decoder pyramid structure based on an end-to-end neural network, which includes downsampling encoder and upsampling decoder paths for realizing feature interactions at different spatial scales;

[0011] Step 2: Introduce a multi-domain feature fusion module that integrates wavelet transform and Fourier transform at each stage of the encoder and decoder. After performing wavelet transform and Fourier transform on the input image features respectively, fuse them to achieve adaptive enhancement of low-frequency global structure and high-frequency details.

[0012] Step 3: Introduce a cross-scale attention module among the features at different scales of the pyramid. By mapping the features at each scale to the same dimension and adaptively assigning weights, enhance the correlation modeling between features at multiple scales and achieve adaptive feature fusion.

[0013] Step 4: In the decoder's progressive upsampling and skip connection stages, a gated attention alignment module is introduced. This module combines cross-scale convolution with spatial-channel attention mechanisms to achieve precise alignment of encoder and decoder features and transmission of detail information through gating.

[0014] Step 5: Train the lightweight low-light image enhancement method that integrates multi-domain features and cross-scale attention using the LOLv1, LOLv2-real, and LOLv2-syn low-light image datasets, configure the training parameters, and set the training rounds;

[0015] Step 6: Input the low-light image dataset into the trained network model. The model performs brightness enhancement and detail enhancement on the input image through multi-scale encoding, multi-domain feature extraction, cross-scale attention fusion, and gated attention alignment, and finally outputs a high-quality enhanced image, achieving efficient adaptive enhancement of low-light images.

[0016] Step 1 addresses the limitations of traditional low-light image enhancement methods in handling complex lighting and preserving details by constructing an encoder-decoder pyramid structure based on an end-to-end neural network. This structure includes downsampling encoder and upsampling decoder paths for realizing feature interaction at different spatial scales. Through the stepwise extraction and fusion of multi-scale features, the network's adaptability to features at different spatial resolutions is effectively improved, achieving global modeling from shallow details to deep semantic information, and providing high-quality input representation for subsequent feature transformation and enhancement.

[0017] Step 2 introduces a multi-domain feature fusion module that integrates wavelet transform and Fourier transform. This module aims to overcome the shortcomings of single spatial or frequency domain features in maintaining both the integrity of image structure and the authenticity of details. By performing wavelet transform on the input features, low-frequency and high-frequency sub-bands are obtained, and convolution operations are used to enhance the information of different frequency bands. At the same time, the global structure and detail features extracted by Fourier transform are combined, and the above features are spliced ​​and convolved in the channel dimension to form a multi-scale representation that is complementary in the spatial and frequency domains. This effectively improves the detail restoration and noise suppression capabilities of low-light images.

[0018] Step 3 establishes a cross-scale attention module, which unifies features from different scales to the same channel dimension and adjusts them to a consistent spatial size, thus resolving the resolution inconsistency issue during multi-level feature fusion. Then, all scale features are weighted and fused. This mechanism enhances global contextual understanding, enabling the network to flexibly aggregate and utilize useful features for multi-scale targets and complex scenes under low-light conditions, achieving significantly improved enhancement effects.

[0019] Step 4 involves introducing a gated attention alignment module during the decoder's step-by-step upsampling and skip connections. By designing cross-scale convolutional branches with different receptive fields, a rich representation of features at the same dimension of the encoder and decoder is achieved. This module also combines channel and spatial attention mechanisms to activate key features and suppress irrelevant features. Through a gating mechanism, it adaptively adjusts the fusion ratio based on information distribution to achieve a dynamic balance between the two types of features. This effectively addresses the problems of difficult feature alignment and easy loss of details under low light conditions, improving support for detail recovery and edge fidelity, and ensuring that the enhancement results have high realism and naturalness.

[0020] The specific method for step 5 is as follows:

[0021] Step 5.1: Input the low-light image that has undergone normalization and enhancement preprocessing into the encoder-decoder pyramid backbone network. First, the input image is projected into features through the initial feature embedding module to obtain the primary spatial feature representation. Then, multi-scale feature layers are constructed step by step using downsampling operations to lay the foundation for multi-scale information fusion.

[0022] Step 5.2 introduces a multi-domain feature fusion module that integrates wavelet transform and Fourier transform in each coding stage. Low-frequency and high-frequency features are extracted by wavelet transform decomposition followed by convolution enhancement and Fourier transform frequency domain reconstruction. The information in the spatial domain and frequency domain is spliced ​​and fused, and then reorganized and normalized by convolution operation, which effectively enhances the robustness to details, brightness and noise.

[0023] Step 5.3: Collect and normalize the encoded features at different scales, and input them into the cross-scale attention module. The module first maps the features at each scale to the same channel dimension and upsamples to align the spatial size. Then, it fuses multi-layer features at different resolutions and dynamically allocates weights through statistical and attention mechanisms to achieve adaptive combination and enhancement of global and local information, thereby improving the model's generalization ability to cope with complex low-light scenes.

[0024] Step 5.4 inputs the high-level features fused with multi-scale attention into the latent spatial processing and decoder path, and restores spatial details through upsampling operations in sequence. During decoding, gated attention alignment modules are inserted at each level of skip connections. Through parallel multi-receptive field convolutions, channel and spatial attention branches, dynamic alignment and detail compensation of deep and shallow features are achieved. The gating mechanism can adjust the fusion ratio of treaties and main features according to semantic relevance, ensuring fine feature reconstruction and suppressing noise and artifact spread.

[0025] Step 5.5 concludes the image reconstruction process by using convolutional fusion and residual enhancement to output the network. This enhances the brightness and detail of low-light images, ensuring the naturalness and high fidelity of the output image in terms of structure, color, and subjective perception. The entire training and inference process is jointly optimized using multiple loss terms, including L1, SSIM, and TV losses, to further enhance the model's generalization and robustness, ensuring excellent PSNR and SSIM performance metrics on the LOLv1, LOLv2-real, and LOLv2-syn datasets.

[0026] This application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the low-light image enhancement method based on multi-domain fusion and lightweight design. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 A flowchart of the low-light image enhancement method based on multi-domain fusion and lightweight design provided by the present invention;

[0029] Figure 2 A diagram of a lightweight low-light image enhancement network architecture that integrates multi-domain features, provided by the present invention.

[0030] Figure 3 This is a structural diagram of the multi-domain feature fusion module provided by the present invention;

[0031] Figure 4 The structural diagram of cross-scale attention provided by this invention;

[0032] Figure 5 This is a structural diagram of the gating attention module provided by the present invention. Detailed Implementation

[0033] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] This application proposes a low-light image enhancement method based on multi-domain fusion and lightweight design, such as... Figure 1 As shown, the low-light image enhancement method based on multi-domain fusion and lightweight design provided in this application includes the following steps:

[0035] S1: Combined with appendix Figure 1 As shown, the above steps will be implemented as follows:

[0036] Step 1.1: Preprocess the original training data, including performing various transformations on the images such as padding, random cropping, random flipping, and random rotation, to enhance the richness of the dataset and the generalization ability of the model. After these processes, the images are normalized into a uniform tensor format, providing a high-quality input foundation for model training.

[0037] Step 1.2: Input the low-light image after normalization and enhancement preprocessing into the encoder-decoder pyramid backbone network. First, the input image is projected into features through the initial feature embedding module to obtain the primary spatial feature representation. Then, multi-scale feature layers are constructed step by step using downsampling operations to lay the foundation for multi-scale information fusion.

[0038] Step 1.3: Introduce a multi-domain feature fusion module that integrates wavelet transform and Fourier transform in each coding stage. By combining wavelet transform decomposition and convolution enhancement with Fourier transform frequency domain reconstruction, low-frequency structure and high-frequency detail features are extracted respectively. The information in the spatial domain and frequency domain is spliced ​​and fused, and then reorganized and normalized by convolution operation, which effectively enhances the robustness to details, brightness and noise.

[0039] Step 1.4: Collect and normalize encoded features at different scales, and input them into the cross-scale attention module. The module first maps the features at each scale to the same channel dimension and upsamples to align the spatial size. Then, it fuses multi-layer features at different resolutions and dynamically allocates weights through statistical and attention mechanisms to achieve adaptive combination and enhancement of global and local information, thereby improving the model's generalization ability to cope with complex low-light scenes.

[0040] Step 1.5: Input the high-level features fused with multi-scale attention into the latent spatial processing and decoder path, and restore spatial details sequentially through upsampling operations. During decoding, gated attention alignment modules are inserted at each level of skip connections. Through parallel multi-receptive field convolutions, channel and spatial attention branches, dynamic alignment and detail compensation of deep and shallow features are achieved. The gating mechanism can adjust the fusion ratio of treaties and main features according to semantic relevance, ensuring fine feature reconstruction and suppressing noise and artifact spread.

[0041] Step 1.6: Finally, the image is reconstructed through convolutional fusion and residual enhancement output networks, achieving brightness improvement and detail enhancement in low-light images, ensuring the naturalness and high fidelity of the output image in terms of structure, color, and subjective perception. The entire training and inference process is jointly optimized through multiple loss terms, including L1, SSIM, and TV losses, to further enhance the model's generalization and robustness, ensuring excellent PSNR and SSIM performance metrics on the LOLv1, LOLv2-real, and LOLv2-syn datasets.

[0042] S2: Combined with appendix Figure 2 Therefore, the multi-domain feature fusion module of this invention aims to achieve efficient joint spatial structure decomposition and frequency feature enhancement in deep neural networks, thereby significantly improving the overall brightness recovery, detail preservation, and noise suppression capabilities of low-light images. This module first performs wavelet decomposition on the input features, then reassembles them after spatial domain enhancement, and further feeds them into a Fourier transform path for frequency domain enhancement. Finally, residual superposition is performed, achieving efficient complementarity between the spatial and frequency domains. This sequential separation-enhancement-fusion design not only ensures complete modeling of global and local key information under low-light conditions but also provides high-quality input features for subsequent networks. Simultaneously, by employing a small convolutional network and reusing DWT / IWT and FFT / IFFT basic transform operators to reduce redundant parameters, a lightweight design is achieved. This module can be represented as:

[0043]

[0044] Where I represents the input feature of the module, and I1 and I2 are intermediate variables. WDFFM For the enhanced features after multi-domain fusion, WDAB is the wavelet domain adjustment module, LayerNorm is the layer normalization operation, FFN is the feedforward network, and FDAB is the Fourier frequency domain adjustment module.

[0045] In the wavelet transform adjustment module, the input features are first decomposed into low-frequency I0 through a two-dimensional discrete wavelet transform (DWT). LL and high frequency I LH ,I HL ,I HH Quantity:

[0046] I LL ,{I LH ,I HL ,I HH}=DWT(I) (2)

[0047] I LL I is responsible for global brightness and large structural information. LH ,I HL ,I HHThis is responsible for rendering details such as texture and edges. The model then applies independent convolution operations to these two branches, effectively enhancing their respective feature attributes. All enhanced components I... LL ′,I LH ′,I HL ′,I HH The feature maps I' are reconstructed by concatenating the data by channel and then inputting them into the inverse wavelet transform (IWT).

[0048] I′=IWT(cat(I LL ′,{I LH ′,I HL ′,I HH ′}))+I (3)

[0049] The features are further integrated through residual connections and a feedforward network (FFN) to obtain the final output I″:

[0050] I″=FFN(LN(I′))+I′ (4)

[0051] This process ensures the synergistic enhancement of global illumination information and local details, enabling better recovery of the main structure while suppressing detail loss and artifacts in low-light conditions.

[0052] The Fourier transform adjustment module takes the spatial enhancement output I″ as input and first performs a two-dimensional real Fourier transform to convert the features to the frequency domain, represented as I. freq , where mag = |I freq | represents the amplitude spectrum, pha = ∠I freq The phase spectrum is shown. The frequency domain adjustment module performs nonlinear processing on mag and pha using small convolutional networks to obtain enhanced spectra mag′ and pha′. The module then recombines the adjusted amplitude and phase into new frequency domain features, which are returned to the spatial domain via inverse Fourier transform (IFFT).

[0053] I freq ′=mag′cos(pha′)+j·mag′sin(pha′),I″=IFFT(I freq ′) (5)

[0054] Finally, by fusing the frequency domain enhancement results with the original spatial features through residual connections, the Fourier enhancement process can highlight the wide range of brightness, periodic variations, and microstructural rhythms in low-light enhancement, effectively supplementing the global characteristics that are difficult to model in spatial convolution. This module design balances parameter efficiency and comprehensive representation, so that the final output integrates the structural details under wavelet decomposition and the frequency attributes enhanced by Fourier, providing a better feature foundation for image enhancement in complex low-light scenes.

[0055] S3: Combined with appendix Figure 3 The cross-scale attention module of this invention aims to efficiently integrate multi-layer features from different scales, improving the multi-scale representation and generalization ability of low-light image enhancement models. This module first projects all input features F0, F1, and F2 from different scales onto a unified output channel dimension through a convolutional layer, and then upsamples them to the target spatial resolution using bilinear interpolation, i.e.:

[0056]

[0057] Where C is the number of channels, H×W is the spatial size, and Conv(·,4C) represents a 1×1 convolution operation that projects features onto 4C channels. This indicates a bilinear interpolation upsampling operation that unifies the feature size to... F i ′ represents the multi-scale feature after unifying the scale.

[0058] Then, the multi-scale features F0′, F1′, and F2′ after unification are concatenated into a single holistic feature tensor F. cat =Concat(F0′,F1′,F2′). Next, the cross-scale attention module uses a lightweight attention network to perform global statistics and weighting on the channel dimensions of the concatenated features, automatically learning the weight coefficient α of each scale in the final fusion output. Finally, the output is a weighted sum of features from each scale, achieving dynamic fusion.

[0059]

[0060] Among them, F fused This represents the final output feature of the cross-scale attention module.

[0061] By adaptively weighting and fusing multi-scale information, the model can simultaneously take into account brightness restoration, detail restoration and noise resistance, thereby improving the overall image enhancement effect in low-light environments.

[0062] S4: Combined with appendix Figure 4 The gated attention alignment module of this invention mainly addresses the challenges of deep and shallow feature fusion and skip connection alignment during network decoding, solving the problems of spatial fine-grained semantic consistency and low-light noise suppression. First, the module achieves feature alignment through cross-scale attention, channel attention, and spatial attention, obtaining aligned features F. align :

[0063] F align =AttentionAlignment(F enc ,F dec (8)

[0064] Among them, F enc For encoder features, Fdec For decoder features, AttentionAlignment(·) is the attention alignment operation, including cross-scale attention, spatial attention and channel attention. Cross-scale attention is used to characterize the multi-scale context in deep and shallow features. Spatial attention extracts key region responses through spatial pooling and convolution. Channel attention performs global average pooling and max pooling on channel features to learn global importance weights respectively.

[0065] Next, based on F enc Generate a gating weight g (value 0-1) to control the fusion ratio between aligned features and original features:

[0066] g=Sigmoid(Conv(ReLU(Conv(Avgpool(F enc ))))) (9)

[0067] Here, Avgpool(·) is global average pooling, Conv(·) is convolution operation, ReLU(·) is activation function, and Sigmoid(·) is normalization function.

[0068] Finally, g is used to align the feature F align The original feature residuals are weighted and fused to obtain the output feature F. out :

[0069] F out =g·F align +(1-g)·(F enc +F dec (10)

[0070] The three features are finally concatenated in the channel dimension, and the gating weights are generated through the aforementioned gating unit. This enables the module to adaptively adjust and compensate for high-resolution spatial details and deep semantics based on the image content when fusing deep and shallow features. This effectively achieves feature alignment, noise suppression, and detail restoration, improving the model's structural restoration and output naturalness in low-light enhancement tasks.

[0071] S5: Build a low-light image enhancement method architecture based on multi-domain fusion and lightweight design for training and validation. The steps are as follows:

[0072] Step 5.1: The example model of this invention uses a Python 3.8 environment and the PyTorch 1.12 deep learning framework. Model training and testing are performed on a computing platform equipped with an NVIDIA GeForce RTX 3090 GPU. Experimental data undergoes uniform preprocessing of input image size, with a batch size set to 12. During training, the Adam optimizer is used, with an initial learning rate set to 1e-6, and combined with a 5-round warm-up and cosine annealing scheduling mechanism, gradually decreasing to a minimum learning rate of 1e-4. The total number of training iterations is 3000 epochs. All experiments are implemented based on Python and PyTorch environments. To ensure computational efficiency and stability, the GPU-accelerated environment is equipped with a suitable CUDA driver version.

[0073] Step 5.2: After training, model performance is evaluated from multiple dimensions. Pixel consistency is measured by Peak Signal-to-Noise Ratio (PSNR), and structural fidelity is evaluated by the Structural Similarity Index (SSIM, ranging from 0 to 1), with values ​​closer to 1 indicating better structural reconstruction. The Perceptual Similarity Index (LPIPS) is used to capture high-level visual differences such as semantics and texture; lower values ​​indicate stronger perceptual consistency between the generated image and the reference image. In the absence of a reference, Natural Image Quality Evaluation (NIQE) is used to reflect the naturalness of the image as perceived by the human eye; lower values ​​indicate higher naturalness. Furthermore, model efficiency is evaluated using floating-point operations (FLOPs / G) and the number of learnable parameters (Params / M) to achieve a balance between performance and computational cost.

[0074] Table 1 shows the ablation experiments in LOL-v1, LOL-v2-real, and LOL-v2-syn.

[0075]

[0076] Ablation experiments were conducted on three mainstream low-light image enhancement datasets: LoL-v1, LoL-v2-real, and LoL-v2-syn. The results are shown in Table 1. On the LoL-v1 dataset, the baseline model, i.e., the proposed method, achieved a PSNR of 25.029 dB and an SSIM of 0.863. In Experiment #1, removing the wavelet domain adjustment module from the multi-domain feature fusion module resulted in a 1.069 dB decrease in PSNR and a 0.007 decrease in SSIM. In Experiment #2, removing the spatial convolution from the wavelet domain adjustment module resulted in a 1.475 dB decrease in PSNR and a 0.012 decrease in SSIM. In Experiment #3, removing the frequency domain adjustment module from the multi-domain feature fusion module resulted in a significant 2.451 dB decrease in PSNR and a 0.012 decrease in SSIM, representing the largest performance degradation among the component removal experiments. In Experiment #4, replacing DWT / IWT with ordinary convolution resulted in a 0.889 dB decrease in PSNR, while SSIM remained unchanged. In Experiment #5, replacing the cross-scale attention module with global average pooling resulted in… PSNR decreased by 1.304 dB, while SSIM remained unchanged. In Experiment #6, after replacing the gated attention alignment module with simple stitching, PSNR decreased by 1.079 dB, while SSIM remained unchanged. In Experiment #7, when only the multi-domain feature fusion module was retained, PSNR decreased by 1.794 dB, and SSIM decreased by 0.012. In Experiment #8, when both the multi-domain feature fusion module and the gated attention alignment module were retained, PSNR decreased by 1.861 dB, and SSIM decreased by 0.014. In Experiment #9, when both the multi-domain feature fusion module and the cross-scale attention module were retained, PSNR decreased by 1.106 dB, and SSIM decreased by 0.005, making it the group with the closest performance to the baseline model among all ablation experiments.

[0077] On the LoL-v2-real dataset, the proposed method achieves a PSNR of 26.128 dB and an SSIM of 0.945. In Experiment #1, removing the wavelet domain adjustment module from the multi-domain feature fusion module decreased PSNR by 0.051 dB and SSIM by 0.003. In Experiment #2, removing the spatial convolution from the wavelet domain adjustment module decreased PSNR by 0.043 dB and SSIM by 0.001. In Experiment #3, removing the frequency domain adjustment module from the multi-domain feature fusion module decreased PSNR by 0.765 dB and SSIM by 0.005. In Experiment #4, replacing DWT / IWT with ordinary convolution decreased PSNR by 0.305 dB and SSIM by 0.002. In Experiment #5, replacing the cross-scale attention module with global average pooling decreased PSNR by 0.404 dB and SSIM by [missing value]. 0.001; Experiment #6, after replacing the gated attention alignment module with simple stitching, saw the most significant performance degradation, with PSNR decreasing by 0.702dB and SSIM decreasing by 0.003; Experiment #7, when only the multi-domain feature fusion module was retained, saw PSNR decrease by 0.989dB and SSIM decrease by 0.007; Experiment #8, after retaining both the multi-domain feature fusion module and the gated attention alignment module, saw PSNR decrease by 0.617dB and SSIM decrease by 0.005; Experiment #9, after retaining both the multi-domain feature fusion module and the cross-scale attention module, saw PSNR decrease by 0.434dB and SSIM decrease by 0.003. Overall, the performance degradation of the component removal and replacement experiments was generally smaller than that of the component retention experiments.

[0078] On the LoL-v2-syn dataset, the proposed method achieved a PSNR of 26.263 dB and an SSIM of 0.946. In Experiment #1, removing the wavelet domain adjustment module from the multi-domain feature fusion module resulted in a PSNR decrease of 1.078 dB and an SSIM decrease of 0.009. In Experiment #2, removing the spatial convolution from the wavelet domain adjustment module resulted in a PSNR decrease of 0.980 dB and an SSIM decrease of 0.006. In Experiment #3, removing the frequency domain adjustment module from the multi-domain feature fusion module resulted in a PSNR decrease of 0.851 dB and an SSIM decrease of 0.005. In Experiment #4, replacing DWT / IWT with ordinary convolution resulted in a PSNR decrease of only 0.208 dB, while the SSIM remained unchanged, making it the most stable group among all ablation experiments. In Experiment #5, replacing the cross-scale attention module with global average pooling resulted in a PSNR decrease of... The PSNR decreased by 0.696 dB, and the SSIM decreased by 0.004. In Experiment #6, after replacing the gated attention alignment module with simple stitching, the PSNR decreased by 0.879 dB, and the SSIM decreased by 0.004. In Experiment #7, when only the multi-domain feature fusion module was retained, the PSNR decreased by 0.685 dB, and the SSIM decreased by 0.004. In Experiment #8, after retaining both the multi-domain feature fusion module and the gated attention alignment module, the PSNR decreased by 0.468 dB, and the SSIM decreased by 0.004. In Experiment #9, after retaining both the multi-domain feature fusion module and the cross-scale attention module, the PSNR decreased by only 0.350 dB, and the SSIM decreased by 0.001, making it the group whose performance was closest to the baseline model among all ablation experiments.

[0079] In ablation experiments on three mainstream low-light image enhancement datasets—LoL-v1, LoL-v2-real, and LoL-v2-syn—the comparative results, using the proposed method as a benchmark, fully validate the necessity and synergistic effectiveness of the core components of the proposed model. Regardless of whether components were removed, replaced, or partially retained in the experimental settings, the PSNR and SSIM metrics for all ablation groups did not surpass the benchmark model, and many groups showed significant performance degradation. This demonstrates the scientific validity of the overall architecture design, which comprises the multi-domain feature fusion module (including wavelet domain adjustment, frequency domain adjustment sub-modules, and spatial convolution), DWT / IWT transform, cross-scale attention module, and gated attention alignment module. Consistent performance across different datasets: In LoL-v1, removing the frequency domain adjustment module resulted in a PSNR reduction of 2.451 dB; in LoL-v2-real, replacing the gated attention alignment module with simple stitching reduced PSNR by 4.101 dB; and in LoL-v2-syn, retaining the multi-domain feature fusion module and the cross-scale attention module yielded performance closest to the benchmark. This not only demonstrates the model's good generalization ability in both synthetic and real low-light scenes but also confirms the crucial role of each component in improving image pixel consistency and structural fidelity. Experimental results show that the proposed model achieves comprehensive optimization of low-light image enhancement performance through the collaborative work of its core components, further validating the effectiveness and superiority of the overall design approach.

[0080] S6: After inputting the low-light image into the trained network, it goes through multi-scale encoding, feature fusion, attention mechanism and other steps to output the enhanced high-quality image. The results are quantitatively compared on the LOL-v1, LOL-v2-real and LOL-v2-syn datasets for low-light image enhancement task. The results are shown in Table 2.

[0081] As shown in Table 2, the proposed method exhibits comprehensive performance advantages. On the LOL-v1 dataset, the proposed method achieves a PSNR of 25.03dB and an SSIM of 0.863, significantly higher than mainstream models such as LLFormer and CIDNet. On the LOL-v2-real dataset, the proposed method ranks first with an absolute advantage of 26.13dB PSNR and 0.943 SSIM, far exceeding the second-place CIDNet. In particular, the SSIM metric is significantly improved compared to similar models, demonstrating stronger structure fidelity. On the LOL-v2-syn dataset, the proposed method has the highest PSNR and SSIM, and its overall performance is at a high level.

[0082] Table 2 shows a quantitative comparison of the LOL-v1, LOL-v2-real, and LOL-v2-syn datasets. Results highlighted in bold are the best, followed by those underlined. Up indicates a larger value is better, and down indicates a smaller value is better.

[0083]

[0084] The quantitative comparison results on the LSRW-Huawei and LSRW-Nikon datasets are shown in Table 3. As can be seen from Table 3, the proposed method exhibits superior performance on the LSRW-Huawei and LSRW-Nikon datasets.

[0085] Table 3. Quantitative comparison of LSRW-Huawei and LSRW-Nikon. The results highlighted in bold are the best, followed by the results highlighted in underline.

[0086]

[0087] On the LSRW-Huawei dataset, the proposed method achieves a PSNR of 21.16dB and an SSIM of 0.642, surpassing models such as CodeEnhance and CIDNet. It demonstrates the best image quality and structure fidelity in real-world low-light scenes captured by Huawei devices. On the LSRW-Nikon dataset, the proposed method achieves a PSNR of 17.67dB and an SSIM of 0.534, higher than models such as Retinexmamba and RetinexFormer. It also leads in low-light scenes captured by Nikon devices, fully demonstrating the model's adaptability to low-light images captured by different devices.

[0088] The quantitative comparison results on the MIT-Adobe FiveK dataset are shown in Table 4. As can be seen from Table 4, the proposed method ranks first with a comprehensive performance of PSNR 25.52dB, SSIM 0.929, and LPIPS 0.042. The PSNR metric surpasses CodeEnhance and LIEDNet, the SSIM metric is only slightly lower than LIEDNet, and the LPIPS metric is the lowest among all the comparison models, indicating that the proposed method has strong practicality in large-scale real image enhancement tasks.

[0089] Table 4. Quantitative comparison of the MIT-Adobe FiveK dataset. The results highlighted in bold are the best, followed by the results highlighted in underline.

[0090]

[0091] The quantitative comparison results on the SID, SDID-in, and SDID-out datasets are shown in Table 5. As can be seen from Table 5, the proposed method performs particularly well on the SDID-in and SDID-out datasets, with PSNRs of 30.62dB and 29.34dB, and SSIMs of 0.910 and 0.887, respectively, significantly outperforming models such as CIDNet. On the SID dataset, the proposed method achieves an SSIM of 0.659, which is higher than LLFlow and CIDNet, demonstrating that the model can stably output high-quality enhancement results on different types of low-light imaging datasets and has extremely strong robustness.

[0092] Table 5 shows a quantitative comparison of the SID, SDSD-in, and SDID-out datasets. Results highlighted in bold are the best, followed by those underlined.

[0093]

[0094] Table 6 shows the quantitative comparison results of the unpaired datasets DICM, LIME, MEF, NPE, and VV. As can be seen from Table 6, the proposed method ranks first with an excellent mean NIQE of 3.47. Specifically, the NIQE on the VV dataset is only 2.58, the lowest among all the compared models. The NIQE on the DICM, LIME, MEF, and NPE datasets is also among the top. This indicates that the proposed method can still generate augmented images that conform to the natural perception of the human eye in scenarios without reference image guidance, solving the technical pain point of lacking paired reference images in real-world scenarios and demonstrating outstanding generalization ability.

[0095] Table 6 shows the quantitative comparisons of the unpaired datasets DICM, LIME, MEF, NPE, and VV. Results highlighted in bold are the best, followed by those underlined.

[0096]

[0097] The results of the noise level estimation experiments are shown in Table 7. The proposed method outperforms the comparison model in estimation results at preset noise levels of 5, 15, 20, 30, and 40. Especially at high noise levels, the estimated values ​​are almost identical to the true values. This indicates that the proposed method can accurately capture the noise distribution characteristics in low-light images, providing reliable support for subsequent noise suppression and detail enhancement, further ensuring the final image enhancement effect, and consolidating the performance advantage of the model from the underlying technical level.

[0098] Table 7. Experimental Results of Noise Level Estimation

[0099]

[0100] Based on the quantitative comparisons across multiple datasets, ablation experiments, and computational complexity analysis presented above, the low-light image enhancement model proposed in this invention achieves a dual breakthrough in performance and efficiency. Through the collaborative design of a multi-domain feature fusion module, a cross-scale attention module, and a gated attention alignment module, the model significantly outperforms other models in PSNR and SSIM metrics on multiple datasets, including the LoL series, LSRW series, and MIT-Adobe FiveK. The performance drops dramatically after removing or replacing components in the ablation experiments, fully demonstrating the scientific validity of this architecture design in balancing structural fidelity and pixel consistency. While maintaining performance, the model reduces redundant parameters by employing a small convolutional network and reusing basic transformation operators such as DWT / IWT and FFT / IFFT, unifying channel dimensions with 1×1 convolutional projection to reduce channel redundancy, and simplifying parameter size by introducing grouped convolutions. These lightweight designs control the number of parameters to 1.2M and reduce FLOPs to 1.41G, far lower than mainstream models such as KinD++ and LLFlow. This successfully addresses the industry pain point that "high performance is inevitably accompanied by high complexity" in traditional enhancement models and possesses strong potential for edge deployment.

[0101] Compared with existing technologies, this application has the following advantages: 1. Existing technologies rely on single-domain features or single-scale information in low-light image enhancement, failing to fully explore the synergistic gains of multi-domain and multi-scale features. This application reuses basic transformation operators and adopts small convolutional networks through a multi-domain feature fusion module to fuse spatial domain structural details and frequency domain global characteristics. At the same time, it uses a cross-scale attention module to unify the channel dimension with 1×1 convolution and a gated attention alignment module to simplify parameters through grouped convolution. While enriching feature expression, it achieves extreme lightweighting with only 1.2M parameters and as low as 1.41G FLOPs, which is far superior to mainstream models. 2. Existing technologies are prone to problems such as scale mismatch and insufficient alignment of deep and shallow features during feature fusion, and have limited generalization ability. This application adaptively aggregates multi-scale features through a lightweight attention mechanism of the cross-scale attention module, and achieves accurate alignment of deep and shallow features and noise suppression through the three-way collaboration of the gated attention alignment module. This enables the model to maintain superior performance on multiple datasets such as LOL series, LSRW series, MIT-Adobe FiveK, and in different scenarios such as real / synthetic and with / without reference. The core indicators are significantly superior, and the robustness and practicality are greatly improved.

[0102] In one exemplary embodiment, the present invention also provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0103] In one exemplary embodiment, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0104] In one exemplary embodiment, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0105] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with relevant laws and regulations.

[0106] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided by this invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0107] The databases involved in the various embodiments provided by this invention may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchain. The processors involved in the various embodiments provided by this invention may be, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc.

[0108] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0109] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A low-light image enhancement method based on multi-domain fusion and lightweight design, characterized in that, Includes the following steps: Step 1: Construct an encoder-decoder pyramid structure based on an end-to-end neural network, which includes downsampling encoder and upsampling decoder paths for realizing feature interactions at different spatial scales; Step 2: Introduce a multi-domain feature fusion module that integrates wavelet transform and Fourier transform at each stage of the encoder and decoder. After performing wavelet transform and Fourier transform on the input image features respectively, fuse them to achieve adaptive enhancement of low-frequency global structure and high-frequency details. Step 3: Introduce a cross-scale attention module among the features at different scales of the pyramid. By mapping the features at each scale to the same dimension and adaptively assigning weights, enhance the correlation modeling between features at multiple scales and achieve adaptive feature fusion. Step 4: In the decoder's progressive upsampling and skip connection stages, a gated attention alignment module is introduced. This module combines cross-scale convolution with spatial-channel attention mechanisms to achieve precise alignment of encoder and decoder features and transmission of detail information through gating. Step 5: Train the lightweight low-light image enhancement method that integrates multi-domain features and cross-scale attention using the LOLv1, LOLv2-real, and LOLv2-syn low-light image datasets, configure the training parameters, and set the training rounds; Step 6: Input the low-light image dataset into the trained network model. The model performs a series of processes, such as multi-scale encoding, multi-domain feature extraction, cross-scale attention fusion, and gated attention alignment, to enhance the brightness and details of the input image and finally output a high-quality enhanced image, achieving efficient adaptive enhancement of low-light images.

2. The low-light image enhancement method based on multi-domain fusion and lightweight design according to claim 1, characterized in that, Step 1 addresses the limitations of traditional low-light image enhancement methods in handling complex lighting and preserving details by constructing an encoder-decoder pyramid structure based on an end-to-end neural network. This structure includes downsampling encoder and upsampling decoder paths for realizing feature interaction at different spatial scales. Through the stepwise extraction and fusion of multi-scale features, the network's adaptability to features at different spatial resolutions is effectively improved, achieving global modeling from shallow details to deep semantic information, and providing high-quality input representation for subsequent feature transformation and enhancement.

3. The low-light image enhancement method based on multi-domain fusion and lightweight design according to claim 1, characterized in that, Step 2 introduces a multi-domain feature fusion module that integrates wavelet transform and Fourier transform. This module aims to overcome the shortcomings of single spatial or frequency domain features in maintaining both the integrity of image structure and the authenticity of details. By performing wavelet transform on the input features, low-frequency and high-frequency sub-bands are obtained, and convolution operations are used to enhance the information of different frequency bands. At the same time, the global structure and detail features extracted by Fourier transform are combined, and the above features are spliced ​​and convolved in the channel dimension to form a multi-scale representation that is complementary in the spatial and frequency domains. This effectively improves the detail restoration and noise suppression capabilities of low-light images.

4. The low-light image enhancement method based on multi-domain fusion and lightweight design according to claim 1, characterized in that, Step 3 sets up a cross-scale attention module, which solves the problem of inconsistent resolution when fusing multi-level features by unifying features of different scales to the same channel dimension and adjusting them to a consistent spatial size. Then, all scale features are weighted and fused. This mechanism enhances the global context understanding capability, enabling the network to flexibly aggregate and utilize useful features for multi-scale targets and complex scenes under low light conditions, achieving a significant enhancement effect.

5. The low-light image enhancement method based on multi-domain fusion and lightweight design according to claim 1, characterized in that, Step 4 involves introducing a gated attention alignment module during the decoder's step-by-step upsampling and skip connections. By designing cross-scale convolutional branches with different receptive fields, a rich expression of features at the same dimension of the encoder and decoder is achieved. This module also combines channel and spatial attention mechanisms to activate key features and suppress irrelevant features. Through a gating mechanism, the fusion ratio is adaptively adjusted according to the information distribution to achieve a dynamic balance between the two types of features. This effectively addresses the problems of difficult feature alignment and easy loss of details under low light conditions, improves support for detail recovery and edge fidelity, and ensures that the enhancement results have high realism and naturalness.

6. The low-light image enhancement method based on multi-domain fusion and lightweight design according to claim 1, characterized in that, The specific method for step 5 is as follows: Step 6.1: Input the low-light image that has been normalized and enhanced into the encoder-decoder pyramid backbone network. First, the input image is projected into features through the initial feature embedding module to obtain the primary spatial feature representation. Then, the multi-scale feature layer is constructed step by step using downsampling operation to lay the foundation for multi-scale information fusion. Step 6.2 introduces a multi-domain feature fusion module that integrates wavelet transform and Fourier transform in each coding stage. By combining wavelet transform decomposition and convolution enhancement with Fourier transform frequency domain reconstruction, low-frequency structure and high-frequency detail features are extracted respectively. The information in the spatial domain and frequency domain is spliced ​​and fused, and then reorganized and normalized by convolution operation, which effectively enhances the robustness to details, brightness and noise. Step 6.3: Collect and normalize the encoded features at different scales, and input them into the cross-scale attention module. The module first maps the features at each scale to the same channel dimension and upsamples to align the spatial size. Then, it fuses the multi-layer features at different resolutions and dynamically allocates weights through statistical and attention mechanisms to achieve adaptive combination and enhancement of global and local information, thereby improving the model's generalization ability to cope with complex low-light scenes. Step 6.4: Input the high-level features fused with multi-scale attention into the latent spatial processing and decoder path, and restore spatial details through upsampling operations in sequence. During the decoding process, gating attention alignment modules are inserted at the skip connections of each level. Through parallel multi-receptive field convolution, channel and spatial attention branches, dynamic alignment and detail compensation of deep and shallow features are achieved. The gating mechanism can adjust the fusion ratio of the treaty and the main feature according to semantic relevance to ensure fine feature reconstruction and suppress noise and artifact diffusion. Step 6.5: Finally, the image is reconstructed through convolutional fusion and residual enhancement output network, achieving brightness improvement and detail enhancement in low-light images. This ensures the naturalness and high fidelity of the output image in terms of structure, color, and subjective perception. The entire training and inference process is jointly optimized through multiple loss terms, including the base loss L... base And TV loss L tv The specific loss function is designed as follows: L total <0.9×L base +L tv Where I′ and I are the enhanced image and the target image, respectively, and I i ′,I i Let be the i-th pixel value of the enhanced image and the target image, respectively, where N is the total number of pixels, and SSIM(I′,I) is the structural similarity index between the enhanced image and the target image.

7. The low-light image enhancement method based on multi-domain fusion and lightweight design according to claim 1, characterized in that, In step 6.5, the joint optimization process uses multiple loss terms to improve the model's robustness and detail recovery ability on low-light images. By calculating the pixel value difference between the enhanced image and the target image, the brightness, detail, and noise control of the enhancement result are further optimized to ensure that the final output image has high image quality and naturalness. At the same time, by optimizing the performance of the result similarity index (SSIM) and peak signal-to-noise ratio (PSNR), the visual effect of the image and its adaptability to complex low-light scenes are improved.

8. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the low-light image enhancement method based on multi-domain fusion and lightweight design as described in any one of claims 1-7.

Images