A lightweight low-light image enhancement method based on illumination iteration adjustment

Abstract

The present application relates to a kind of lightweight low-light image enhancement methods based on illumination iterative adjustment, HVI color space transformation is carried out to the image to be processed to obtain HVI image, separates out illumination intensity component to obtain illumination intensity feature map, and shallow feature map is obtained by convolution processing to HVI image;Feature fusion and brightness adjustment are carried out to illumination intensity feature map and shallow feature map to obtain deep feature map, and size is recovered gradually by two layers of decoder stepwise up-sampling, and channel adjustment is carried out using specified size convolution in each layer of decoder, and up-sampling is realized using bilinear interpolation;Mamba-based structure refinement module is introduced at the end of each layer of decoder to improve structure restoration capability, and the HVI feature output by the last layer of decoder is adjusted to specified value after a specified size convolution is carried out, and residual connection is carried out with initial HVI image, to obtain enhanced HVI image, and recovery image is obtained after inverse HVI transformation.In each low-light condition, adaptive enhancement is carried out, multi-directional spatial context aggregation is realized, and texture representation is improved.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and in particular relates to a lightweight low-light image enhancement method based on iterative illumination adjustment. Background Art

[0002] Low-light images, typically captured under insufficient lighting or with sensors with limited dynamic range, often exhibit typical degradation issues such as insufficient brightness, high noise, low contrast, and loss of structural details. These degradations not only severely impact the human visual experience, but also adversely affect the performance of advanced vision tasks such as target detection, autonomous driving, and target recognition. Therefore, improving the quality of low-light images has become a key research issue in academia and industry. Low-light image enhancement (LLIE) aims to enhance image brightness, suppress noise, and preserve as much texture and structural details as possible through algorithmic processing, thereby providing reliable support for subsequent visual perception and analysis tasks.

[0003] Early LLIE methods relied on classic image processing techniques such as histogram equalization, gamma correction, and Retinex theory, which enhance brightness by directly adjusting image intensity or decomposing and reconstructing illumination and reflectance components. However, these methods often struggle with complex scenes, resulting in inaccurate brightness estimates, color distortion, and texture degradation. In contrast, recent deep learning-based methods, leveraging convolutional neural networks (CNNs), transformers, and diffusion models, have achieved significant improvements by learning rich representations from large datasets. For example, Retinex-based methods such as RetinexNet decompose images into illumination and reflectance components, enhancing and denoising them separately through two sub-networks, providing an effective framework for handling uneven illumination and preserving image details. Subsequently, recursive and progressive optimization methods such as ZeroDCE and DRBN perform multi-scale illumination correction and texture enhancement, striking a balance between enhancement quality and model complexity, and offering strong advantages in lightweight design. Recently, several models (e.g., DMFourLLIE, WalMaFa, DarkIR) have demonstrated the advantages of frequency-spatial domain fusion by utilizing wavelet or Fourier transform to jointly enhance brightness, suppress noise, and recover details by coordinating frequency domain enhancement and spatial domain reconstruction.

[0004] Despite these advances, simply boosting brightness can inadvertently amplify noise, especially in darker areas, thereby compromising structural details. To alleviate this problem, SWANet first boosts brightness and then employs a wavelet-based network for noise suppression. He et al. discovered that gamma correction has good noise suppression performance and used it to reduce the noise level of enhanced images. In contrast, CIDNet proposed a learnable horizontal / vertical-intensity (HVI) space that reduces the Euclidean distance between similar colors, effectively suppressing noise at the source within the color space, thereby improving enhancement performance.

[0005] In view of this, it is necessary not only to suppress noise but also to adaptively control the degree of brightness enhancement. Studies have shown that in the frequency domain, the amplitude spectrum mainly controls the brightness, while the phase carries structural information. Further research is needed to study how amplitude modulation affects the enhancement results under different lighting conditions. Figure 2 As shown, increasing the amplitude scaling factor gradually brightens the image, indicating that brightness can be adjusted by manipulating the amplitude. Furthermore, due to differences in initial brightness levels, applying the same scaling factor to different low-light images produces different enhancement results: images with lower initial brightness may remain under-enhanced, while images with higher initial brightness may be overexposed. This suggests that adaptive amplitude scaling is required for optimal enhancement of different low-light images. Furthermore, even within a single image, pixel-level adaptive scaling is necessary to avoid overexposure in bright areas and underenhancement in dark areas.

[0006] Therefore, the present invention proposes an efficient and lightweight low-light image enhancement network - Lightweight Illumination Iterative Adjustment Network (LIIA-Net). The proposed model adopts a U-shaped architecture, in which the encoder focuses on adaptive brightness enhancement and the decoder emphasizes fine-grained spatial texture reconstruction. In the illumination adjustment process, a linear cross attention module (LCAM, Linear CrossAttention Module) is first designed to achieve effective interaction between illumination and content features with low parameter count and computational overhead. Then, the amplitude adaptive iterative adjustment (AAIA, Amplitude Adaptive Iterative Adjustmentmodule) module is adopted to iteratively and adaptively enhance the amplitude component in the frequency domain according to the brightness level of the image. This enables adaptive enhancement under various low-light conditions while effectively balancing the brightness of different regions in a single image. In order to further enhance spatial structure modeling, a Mamba-based structure refinement (MaSR, Mamba-based StructureRefinement module) module is integrated to achieve multi-directional spatial context aggregation and improve texture representation. In order to effectively suppress noise, the HVI (horizontal / vertical intensity) space from CIDNet is introduced. As Figure 3 As shown in Figure 3, the performance comparison on the LOL-v2-Synthetic dataset shows that the model is able to achieve the best results with the least number of parameters. Summary of the Invention

[0007] The purpose of the present invention is to provide a lightweight low-light image enhancement method based on iterative illumination adjustment, which can perform adaptive enhancement under various low-light conditions, effectively balance the brightness of different areas in a single image, achieve multi-directional spatial context aggregation and improve texture representation, and obtain the best results with the least parameters.

[0008] In order to solve the above technical problems, the technical solutions adopted by the present invention are as follows:

[0009] A lightweight low-light image enhancement method based on iterative illumination adjustment includes the following steps:

[0010] S1: Perform HVI color space conversion on the image to be processed, converting it from sRGB space to HVI space to obtain an HVI image;

[0011] S2: Separating the light intensity component from the HVI image to obtain a light intensity feature map of the HVI image to represent light information;

[0012] S3: Convolution processing is performed on the HVI image to obtain a shallow feature map for capturing content and structure information;

[0013] S4: Inputting the illumination intensity feature map and the shallow feature map into a two-stage encoder for feature fusion and brightness adjustment to obtain a deep feature map;

[0014] S5: The depth feature map is gradually upsampled and restored to its size through two layers of decoders. Each layer of decoders uses a specified size of convolution to adjust the channel and bilinear interpolation to achieve upsampling.

[0015] S6: A Mamba-based structure refinement module is introduced at the end of each decoder layer to improve the structure restoration capability. The HVI features output by the last decoder layer undergo a convolution of a specified size to adjust the number of channels to the specified value, and a residual connection is performed with the initial HVI image to obtain the enhanced HVI image. The restored image is obtained through the inverse HVI transformation.

[0016] Preferably, the specific process of feature fusion in step S4 is as follows:

[0017] S41: The light intensity feature F i and shallow features F hvi Flattened into a sequence of light feature tokens X i and shallow feature token sequences X hvi ,

[0018] in, , ;

[0019] , ;

[0020] H Represents the height of the feature map, W represents the width of the feature map, C Indicates the number of channels of the feature map;

[0021] S42: Combine the illumination feature token sequence Xi and the shallow feature token sequence X hvi Divided by channel dimension h The specific formula is:

[0022] ;

[0023] in, , , ;

[0024] S43: For each header, content features are used as queries , lighting features as keys Sum , each head calculates the scaled cross attention separately to obtain a deep feature map.

[0025] Preferably, in step S43, each head calculates the scaled cross attention separately to obtain the specific process of the deep feature map as follows:

[0026] S431: In each scaled cross attention unit, the key K is first transposed and then multiplied by the query Q to obtain the attention weight. The specific formula is as follows:

[0027] ;

[0028] in, , T represents matrix transpose, d h Indicates the number of channels contained in each attention head;

[0029] S432: Scale the attention weight and multiply it by the transpose of the value V. The specific formula is as follows:

[0030] ;

[0031] in, ;

[0032] S433: The outputs of all heads are transposed and concatenated in the channel dimension. After passing through the fully connected layer and deformation, they are restored to the same size as the input features and added to the position embedding to finally obtain a deep feature map containing rich position information and cross-domain interaction.

[0033] Preferably, the brightness adjustment process in step S4 is implemented by an amplitude adaptive iterative adjustment module, and the specific process is as follows:

[0034] S44: Perform Fourier transform on the initially enhanced content feature map, and apply convolution activation operation only on the amplitude component to achieve brightness adjustment;

[0035] S45: The following adaptive control mechanism is introduced under the illumination differences of different images:

[0036] First, perform global average pooling on the light intensity feature map to compress spatial information;

[0037] Then two layers of 1×1 products reduce the number of channels to r Multiply and 1 channel to extract global illumination intensity B r , combined with a learnable dilation factor EF and the maximum number of iterations, according to the global illumination intensity Br The number of iterations of the amplitude transformation is dynamically determined to adjust the enhancement amplitude.

[0038] Preferably, the specific process of step S5 is as follows:

[0039] The depth feature map is gradually upsampled and restored to size through two layers of decoders. Each layer of decoder uses 3×3 convolution for channel adjustment and bilinear interpolation to achieve upsampling. In order to maintain spatial consistency at high resolution, jump connections are used to splice the encoder features with the corresponding decoder features, and 1×1 convolution is used to compress the number of channels.

[0040] Preferably, step S6 is implemented by the MaSR module, and the specific process is as follows:

[0041] S61: The visual state space module VSSM of the MaSR module concatenates the light intensity features and shallow features in the channel dimension, and then reduces the number of channels by half through 1×1 convolution;

[0042] S62: The features are fed into two parallel branches. The first branch passes through a linear layer, a depthwise convolution, and a SiLU activation function, and then is fed into a two-dimensional selective scanning module (2D-SSM) for layer normalization. The second branch passes through a linear layer and a SiLU activation function.

[0043] S63: Perform element-wise multiplication and fusion on the outputs of the two branches and map them back to the input feature space through a linear layer.

[0044] Preferably, the specific process of the two-dimensional selective scanning module 2D-SSM performing layer normalization in step S62 is as follows:

[0045] The input image is expanded in forward and reverse sequences by row and column respectively to obtain four scanning paths, namely row forward, row reverse, column forward, and column reverse. Each path is processed by the state-space equation, and then the processing results of the four paths are fused and restored to the original two-dimensional image structure.

[0046] Preferably, it also includes a network training process, and in the training process, it is trained with the set goal of "brightness adjustment first, then structure restoration", setting the illumination adjustment loss and structure texture loss, and combining L 1 loss and VGG-based perceptual loss L p Improve network reconstruction capabilities and perception quality;

[0047] The lighting adjustment loss is calculated by outputting the intermediate result image in sRGB space in the middle of the network and downsampling it by 4 times compared to the true value image. The specific formula is as follows:

[0048] ;

[0049] in Indicates that the true value image (sRGB space) is downsampled 4 times; The intermediate result image is 4 times smaller than the final output image. L 1 means the mean absolute error loss used for the two images; L p represents the perceptual loss used on the two images; It is a hyperparameter used to control the proportion of perceptual loss used, and its value is set to 0.01.

[0050] The structural texture loss is calculated by calculating the final restored image in sRGB and HVI color spaces respectively: the specific formula is as follows:

[0051] ;

[0052] ;

[0053] The total loss is calculated as follows:

[0054] ;

[0055] in represents the true value image; Y Represents the final result image; represents the true value image; Represents the final result image, L 1 means the mean absolute error loss used for the two images; L p represents the perceptual loss used on the two images; It is a hyperparameter used to control the proportion of perceptual loss used, and its value is set to 0.01.

[0056] The beneficial effects of the present invention include:

[0057] The lightweight low-light image enhancement method based on iterative illumination adjustment provided by the present invention performs HVI color space transformation on the image to be processed to obtain an HVI image, separates the illumination intensity component to obtain an illumination intensity feature map, and performs convolution processing on the HVI image to obtain a shallow feature map. Feature fusion and brightness adjustment are performed on the illumination intensity feature map and the shallow feature map to obtain a deep feature map. The deep feature map is then gradually upsampled and restored to its original size through two layers of decoders. Each layer of decoders uses a specified size convolution to adjust the channels, and bilinear interpolation is used to achieve upsampling. A Mamba-based structure refinement module is introduced at the end of each layer of decoders to improve structural restoration capabilities. The HVI features output by the last layer of decoders undergo a specified size convolution to adjust the number of channels to a specified value, and are residually connected with the initial HVI image to obtain an enhanced HVI image. The restored image is then obtained through an inverse HVI transformation. Adaptive enhancement is performed under various low-light conditions to achieve multi-directional spatial context aggregation and improve texture representation.

[0058] The linear cross attention module (LCAM) is used to enhance the feature interaction between content and illumination with low computational and parameter costs. The amplitude adaptive iterative adjustment module (AAIA) iteratively adjusts the amplitude component in the frequency domain according to the image brightness level, enabling adaptive and balanced brightness enhancement under various complex low-light conditions. In the spatial domain, a Mamba-based structure refinement (MaSR) module is introduced to perform multi-directional context aggregation and structural texture recovery using a selective state space model, effectively preserving fine-grained spatial details and reducing the loss of structural information. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 Schematic diagram of the flowchart of the lightweight low-light image enhancement method based on iterative illumination adjustment of the present invention.

[0060] Figure 2 FIG. 1 is a schematic diagram of changing image brightness by adjusting an amplitude scaling factor in the prior art.

[0061] Figure 3 Schematic diagram of the performance comparison between the lightweight iterative illumination adjustment network LIIA-Net of the present invention and other models on the LOL-v2-Synthetic dataset.

[0062] Figure 4 Schematic diagram of the architecture of the lightweight iterative illumination adjustment network LIIA-Net of the present invention.

[0063] Figure 5 Schematic diagram of the architecture of the LCAA module of the present invention.

[0064] Figure 6 FIG. 4 is a schematic diagram of the architecture of the LCAM module of the present invention.

[0065] Figure 7 Schematic diagram of the working principle of the two-dimensional spatial state model 2D-SSM of the present invention. DETAILED DESCRIPTION

[0066] The following is combined with Figures 1 to 7 The present invention is described in further detail:

[0067] Example 1

[0068] See attached Figure 1 As shown in the figure, a lightweight low-light image enhancement method based on iterative illumination adjustment is implemented through the lightweight iterative illumination adjustment network LIIA-Net. The network architecture is shown in Figure 4 As shown, the network adopts an end-to-end encoder-decoder architecture, and the input low-light image is recorded as ,in H and W Represents the height and width of the image. The overall processing pipeline is divided into two stages: the first stage enhances brightness, while the second stage reconstructs structure and texture. An intermediate loss is introduced at the end of the first stage to monitor the effect of lighting adjustments.

[0069] The lightweight low-light image enhancement method based on iterative illumination adjustment specifically includes the following steps:

[0070] S1: Perform HVI color space conversion on the image to be processed, converting it from sRGB space to HVI space to obtain an HVI image;

[0071] S2: Separating the light intensity component from the HVI image, extracting the initial light features through 3×3 convolution to obtain a light intensity feature map of the HVI image to represent the light information;

[0072] S3: Apply 3×3 convolution to the HVI image itself to extract shallow features and obtain shallow feature maps to capture content and structure information;

[0073] S4: Inputting the illumination intensity feature map and the shallow feature map into a two-stage encoder for feature fusion and brightness adjustment to obtain a deep feature map;

[0074] S5: The depth feature map is gradually upsampled and restored to its size through two layers of decoders. Each layer of decoders uses a specified size of convolution to adjust the channel and bilinear interpolation to achieve upsampling.

[0075] S6: A Mamba-based structure refinement module is introduced at the end of each decoder layer to improve the structure restoration capability. The HVI features output by the last decoder layer undergo a convolution of a specified size to adjust the number of channels to the specified value, and a residual connection is performed with the initial HVI image to obtain the enhanced HVI image. The restored image is obtained through the inverse HVI transformation.

[0076] The lightweight low-light image enhancement method based on iterative illumination adjustment of the present invention is implemented through a lightweight iterative illumination adjustment network LIIA-Net, which includes a linear cross-attention module LCAM, an amplitude adaptive iterative adjustment module AAIA, and a Mamba-based structure refinement module MaSR. The linear cross-attention module LCAM enhances the feature interaction between content and lighting with low computational and parameter costs, and the amplitude adaptive iterative adjustment module AAIA iteratively adjusts the amplitude component in the frequency domain according to the image brightness level. This enables adaptive and balanced brightness enhancement under various complex low-light conditions. The Mamba-based structure refinement module MaSR utilizes a selective state space model for multi-directional context aggregation and structural texture recovery. This module effectively preserves fine-grained spatial details and reduces the loss of structural information.

[0077] The core goal of low-light image enhancement is to increase the brightness of low-light images while restoring texture details without introducing additional noise. With the widespread application of CNNs, a large number of CNN-based LLIE methods have been proposed. For example, a decomposition network estimates the illumination and reflectance components of an image, and two sub-networks are used to perform illumination enhancement and denoising, respectively, to obtain a higher-quality output image. In recent years, the Transformer architecture has achieved new breakthroughs in low-light image restoration by leveraging its self-attention mechanism and global modeling capabilities. SNR-Net, starting from noise modeling of low-light images, emphasizes pixel-by-pixel enhancement by improving the signal-to-noise ratio. Retinexformer, based on the retinal mechanism, separates and fuses illumination information with image content, effectively improving the enhancement effect. These methods reflect the development trend of low-light image enhancement methods from local feature extraction to global modeling.

[0078] Recently, researchers have explored collaborative frameworks between the spatial and frequency domains for low-light image enhancement. FourLLIE effectively adjusts the brightness of low-light images by estimating the amplitude map in Fourier space and introduces a signal-to-noise ratio map to fuse global frequency-domain features with local spatial information. UHDFour points out that brightness information is primarily concentrated in the amplitude component, while noise is more associated with the phase component. Therefore, it uses Fourier transforms to separate amplitude and phase, enhancing brightness while suppressing noise. DEFormer, based on the discrete cosine transform, designs a learnable frequency branch to enhance specific frequency components and achieves deep fusion of the frequency and spatial domains. WaveletMamba decomposes the image using wavelet transforms and uses position-aware Mamba blocks and residual convolution groups to process low-frequency and high-frequency components, respectively, effectively mining the frequency-domain features of the image. WalMaFa combines wavelet transforms with Fourier transforms to fully leverage the low-frequency representation capabilities of wavelets and the phase information advantages of Fourier space. These methods demonstrate the unique potential of collaboratively improving low-light image quality by combining frequency and spatial domains, opening up new research directions in this field.

[0079] Fast Fourier Transform (FFT) is an efficient frequency-domain image analysis method that can quickly decompose spatial images into amplitude and phase components, thereby effectively modeling the global frequency characteristics of the image. In recent years, Fourier transform technology has been widely used in various image restoration tasks, including low-light image enhancement, image denoising, and image dehazing. Among them, FECNet effectively improves the quality of enhancement results by reconstructing brightness and structure in the amplitude and phase components respectively. DMFourLLIE proposes a two-stage multi-branch Fourier network, which introduces infrared images in Fourier space to guide the structural reconstruction of the phase component, further enhancing the ability to restore details. These latest research advances fully demonstrate the effectiveness and application potential of FFT in low-light image enhancement tasks.

[0080] Recently, Selective State Space Models (SSMs) have become an important tool in visual image processing due to their ability to model long-range dependencies and their linear computational complexity. For example, Vmamba bridges the gap between text and image sequence processing through a scanning strategy, and U-Mamba has successfully applied this approach to medical image segmentation. In the field of low-light image enhancement, Retinexmamba, based on Retinex theory, uses the Mamba module instead of the Transformer to capture long-range dependencies. WaveMamba focuses on ultra-high-definition low-light image enhancement, achieving a balance between performance and resource consumption. LIEDNet, with Mamba at its core, designs a lightweight low-light deblurring network.

[0081] The HSV color space (hue H, saturation S, and brightness V) effectively separates color information from brightness. In the field of low-light image enhancement, traditional enhancement methods based on the HSV space, while able to improve brightness, often produce red and black artifacts. To address this issue, the HS plane is polarized to reduce red artifacts by collapsing the red coordinates. A learnable intensity collapse function is designed to compress low-light areas and suppress black artifacts. Experimental results show that the introduction of this learnable color space improves the performance of various advanced methods (FourLLIE, LEDNet, GSAD, etc.). Given its advantages in improving color consistency and suppressing artifacts, the lightweight illumination iterative adjustment network LIIA-Net of this paper also adopts this color space to improve the overall quality of low-light image enhancement.

[0082] The lightweight low-light image enhancement method based on iterative illumination adjustment of the present invention is implemented with the above-mentioned low-light image enhancement, fast Fourier transform, selective state space model, and HSV color space.

[0083] Example 2

[0084] Based on Example 1, step S4 is implemented by the LCAA module. The LCAA module architecture is shown in Figure 5 As shown in the figure, the LCAA module includes a linear cross attention module LCAM and an amplitude adaptive iterative adjustment module AAIA. The specific architecture of the LCAM module can be found in Figure 6 As shown in the figure, content features are enhanced under the guidance of illumination information. Subsequently, the enhanced features are gating-based and added to the residual of the original content features to achieve preliminary adjustments to the content features. This approach aims to address the high time complexity of traditional attention mechanisms when processing high-resolution feature maps and improve the information interaction between the dual input features of content and illumination.

[0085] The specific process of feature fusion in step S4 is as follows:

[0086] S41: The light intensity feature F i and shallow features F hvi Flattened into a sequence of light feature tokens X i and shallow feature token sequences X hvi ,

[0087] in, , ;

[0088] , ;

[0089] H Represents the height of the feature map, W represents the width of the feature map, C Indicates the number of channels of the feature map;

[0090] S42: Illuminate feature token sequence X i and shallow feature token sequences X hvi Divided by channel dimension h The specific formula is:

[0091] ;

[0092] in, , , , d h Indicates the number of channels contained in each attention head;

[0093] S43: For each header, content features are used as queries , lighting features as keys Sum , each head calculates the scaled cross attention separately to obtain a deep feature map.

[0094] In step S43, each head calculates the scaled cross attention separately to obtain the specific process of the deep feature map as follows:

[0095] S431: In each scaled cross attention unit, first transpose the key K and then multiply it by the query Q to obtain the attention weight. The specific formula is as follows:

[0096] ;

[0097] in, , T represents matrix transpose, d h Indicates the number of channels contained in each attention head.

[0098] S432: Scale the attention weight and multiply it by the transpose of the value V. The specific formula is as follows:

[0099] ;

[0100] in, ;

[0101] S433: The outputs of all heads are transposed and concatenated in the channel dimension. After passing through the fully connected layer and deformation, they are restored to the same size as the input features and added to the position embedding to finally obtain a deep feature map containing rich position information and cross-domain interaction.

[0102] Based on matrix multiplication, the complexity analysis of LCAM is as follows:

[0103] ;

[0104] in, H Represents the height of the feature map, W represents the width of the feature map, C Indicates the number of channels of the feature map, h Indicates the division h An attention head, d h Indicates the number of channels contained in each attention head;

[0105] Therefore, the time complexity of LCAM is linearly related to the feature size, and the larger the number of heads, the lower the computational complexity. It is suitable for high-resolution scenarios and significantly improves the information interaction efficiency of the model under high-resolution features.

[0106] The brightness adjustment process in step S4 is implemented by the amplitude adaptive iterative adjustment module AAIA, and the specific process is as follows:

[0107] S44: Perform Fourier transform on the initially enhanced content feature map, and apply convolution activation operation only on the amplitude component to achieve brightness adjustment;

[0108] S45: The following adaptive control mechanism is introduced under the illumination differences of different images:

[0109] First, perform global average pooling on the light intensity feature map to compress spatial information. The specific formula is as follows:

[0110] ;

[0111] in, F i Light intensity characteristic diagram, Indicates global average pooling.

[0112] Then two layers of 1×1 products reduce the number of channels to r Multiply and 1 channel to extract global illumination intensity B r , the specific formula is as follows:

[0113] ;

[0114] ;

[0115] ;

[0116] ;

[0117] That, Represents the light intensity feature map with compressed spatial information; Indicates the use of a 1×1 convolution kernel; GELU Indicates use GELU Activation function; EF is the expansion factor, the initial value is 0.5; It is the maximum number of iterations set. The subsequent ablation study shows that setting it to 4 is the best; round The brightness level L is obtained by rounding the obtained light intensity.

[0118] Combined with learnable dilation factor EF and the maximum number of iterations, according to the global illumination intensity B r Dynamically determine the number of iterations of the amplitude transformation to adjust the enhancement amplitude. The specific formula is as follows:

[0119] ;

[0120] ;

[0121] .

[0122] in, Represents the image content feature map after passing through the LCAM module; FFT represents Fourier transform, and the corresponding Amplitude Amplitude and Phase phase; LeakLU Indicates use LeakLU Activation function; iFFT represents the inverse Fourier transform.

[0123] Example 3

[0124] Based on Example 1 or Example 2, the specific process of step S5 is as follows:

[0125] The depth feature map is gradually upsampled and restored to size through two layers of decoders. Each layer of decoder uses 3×3 convolution for channel adjustment and bilinear interpolation to achieve upsampling. In order to maintain spatial consistency at high resolution, jump connections are used to splice the encoder features with the corresponding decoder features, and 1×1 convolution is used to compress the number of channels.

[0126] Step S6 is implemented by the MaSR module to reconstruct the global structure of the enhanced image features. The specific process is as follows:

[0127] S61: The visual state space module VSSM of the MaSR module concatenates the light intensity features and shallow features in the channel dimension, and then reduces the number of channels by half through 1×1 convolution;

[0128] S62: The features are fed into two parallel branches. The first branch passes through a linear layer, a depthwise convolution, and a SiLU activation function, and then is fed into a two-dimensional selective scanning module (2D-SSM) for layer normalization. The second branch passes through a linear layer and a SiLU activation function.

[0129] S63: Perform element-wise multiplication and fusion on the outputs of the two branches and map them back to the input feature space through a linear layer.

[0130] See also Figure 7 As shown, the specific process of the two-dimensional selective scanning module 2D-SSM performing layer normalization in step S62 is as follows:

[0131] The input image is expanded in forward and reverse sequences by row and column respectively to obtain four scanning paths, namely row forward, row reverse, column forward, and column reverse. Each path is processed by the state-space equation, and then the processing results of the four paths are fused and restored to the original two-dimensional image structure.

[0132] It also includes the network training process, and in the training process, it is trained with the goal of "brightness adjustment first, then structure recovery", setting the illumination adjustment loss and structure texture loss, and combining L 1 loss and VGG-based perceptual loss L p Improve network reconstruction capabilities and perception quality;

[0133] The lighting adjustment loss is calculated by outputting the intermediate result image in sRGB space in the middle of the network and downsampling it by 4 times compared to the true value image. The specific formula is as follows:

[0134] ;

[0135] The structural texture loss is calculated by calculating the final restored image in sRGB and HVI color spaces respectively: the specific formula is as follows:

[0136] ;

[0137] ;

[0138] The total loss is calculated as follows:

[0139] .

[0140] in, Indicates that the true value image (sRGB space) is downsampled 4 times; The intermediate result image (sRGB space) is 4 times smaller than the final output image. Represents the true value image (sRGB space); Y Represents the final result image (sRGB space); Represents the true value image (HVI space); Represents the final result image (HVI space), L 1 means the mean absolute error loss used for the two images; L p represents the perceptual loss used on the two images; It is a hyperparameter used to control the proportion of perceptual loss used, and its value is set to 0.01.

[0141] Experiments are designed to validate the model and evaluate the performance of the proposed model on three paired datasets (LOL-v1, LOL-v2-real and LOL-v2-synthetic) and five real-world datasets (DICM, LIME, MEF, NPE, VV). The experiment is developed based on the PyTorch framework. The version implements GPU acceleration. During the training process, it is optimized by the Adam optimizer. The default parameters β 1=0.9 and β 2 = 0.99, and training was performed for 1000 epochs. Cosine annealing was used for tuning during the experiment. Detailed hyperparameter settings are shown in Table 1. For evaluation on unpaired real-world datasets, the model trained on the LOL-v2-synthetic dataset was used directly without additional fine-tuning.

[0142] Table 1 Paired datasets used and hyperparameter settings

[0143]

[0144] Table 2 shows the quantitative comparison results of the proposed method with representative state-of-the-art methods in recent years on the League of Legends (LOL)-v1, League of Legends (LOL)-v2-real, and League of Legends (LOL)-v2-synthetic datasets. Due to the small size of the LOL-v1 test set, the results of other methods were corrected using a GT mean correction strategy, referencing methods such as LLFlow, GSAD, and CIDNet, to ensure fairness and comparability. As can be seen in Table 2, while significantly lower than most of the compared methods in terms of parameters (params) and floating-point operations (FLOPs), the proposed method achieved comparable or even superior performance to the latest state-of-the-art methods on all three datasets. Specifically, CIDNet, through its introduction of the HVI color space and meticulous architectural design, achieved outstanding performance across multiple metrics. Similarly, the proposed method, benefiting from the HVI color space, achieved comparable or even superior performance to CIDNet with only approximately one-fourth the number of parameters and a similar amount of computation, fully demonstrating the proposed method's advantage in achieving significant lightweightness while maintaining high performance.

[0145] Table 2 Comparison results on LOL and LOL-v2 datasets. The complexity is calculated on 256*256 images.

[0146]

[0147] In the experimental setting, the present invention uses the ExDark dataset to evaluate the practical effectiveness of various low-light image enhancement methods in target detection tasks. The ExDark dataset contains a total of 7,363 images taken in low-light environments, covering 12 object categories, and provides bounding box annotations for each image. Among them, 5,890 images are used for training and 1,473 images are used for testing. All methods are first pre-trained on the LOL-v2 synthetic dataset, and then the trained model is used to enhance the ExDark dataset to generate restored images. Subsequently, the enhanced images are input to the YOLO-v3 detector for fine-tuning to evaluate the improvement effect of different enhancement methods on target detection performance, thereby reflecting its practicality in actual visual tasks.

[0148] Table 3 Low-light object detection results on the ExDark dataset

[0149]

[0150] Table 3 shows that our method performs well in all indicators and achieves the highest average performance. The mean average precision (mAP) across 12 categories reaches 0.762, and it achieves the highest detection results in the bicycle, boat, chair, and cup categories.

[0151] Ablation experiments were conducted on the LOL-v2-real dataset to explore the impact of each component module on model performance. The experimental settings follow Table 1. The model architecture was evaluated from the following perspectives: (1) the effectiveness of the basic modules of the encoder and decoder in the network; (2) the contribution of each key module or step in the LCAA module to the overall performance; (3) the substitutability analysis of the MaSR module; (4) the impact of the selection of the maximum number of iterations on the model performance; and (5) the effect of different loss functions on the model learning effect. The evaluation indicators include peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and perceptual image similarity (LPIPS), and their complexity was calculated with a patch size of 256×256.

[0152] Table 4 Ablation results for basic modules of encoder and decoder

[0153]

[0154] Table 4 shows the ablation results of different backbone modules. Using the convolutional block attention module (CBAM) as the baseline module, despite its low computational overhead, performance was mediocre. (b) applies the proposed LCAA module to the encoder and bottleneck layers, replacing the CBAM block in (a). Experimental results show an approximately 1.7dB improvement in PSNR compared to the baseline, validating LCAA's prominent role in brightness adjustment and feature fusion. Scheme (c) is the opposite of (b), replacing only the decoder portion of (a) with the MaSR module. While parameters and FLOPs increase slightly, structural metrics like SSIM improve significantly, demonstrating that MaSR facilitates detailed structural modeling. Ultimately, the improved modules are used in both the encoder and decoder. The model achieves optimal comprehensive restoration of brightness and texture, achieving optimal performance while maintaining a low number of parameters and computational overhead.

[0155] Table 5 Ablation experiment results for each component within the LCAA module

[0156]

[0157] Because the LCAA module plays a key role in image brightness restoration, we further explored the impact of its internal subcomponents on overall performance. The experimental results are shown in Table 5. As can be seen, removing the LCAM, iterative processing, or AAIA modules significantly reduces the PSNR metric, demonstrating the critical importance of each subcomponent in brightness restoration. The decrease in structural similarity (SSIM) is limited. This analysis suggests that MaSR's global modeling capabilities effectively restore structural information, but brightness restoration still requires the coordinated efforts of its subcomponents. This experiment further validates the key contribution of the multi-stage mechanism within the LCAA module to low-light enhancement.

[0158] Table 6 Results after replacing the MaSR module of the decoder with other attention modules

[0159]

[0160] Analysis of the Substitutability of MaSR Modules. To evaluate MaSR's effectiveness in global structure modeling, Table 6 compares CBAM, the illumination-guided multi-head self-attention module (IG-MSA), and the lightweight crisscross attention module (LCA). The results show that MaSR, leveraging state-space equations and a selective scanning mechanism, achieves superior global structure modeling and multi-directional feature interaction while maintaining linear complexity, significantly improving low-light enhancement performance.

[0161] Table 7 Maximum number of iterations for control ablation

[0162]

[0163] Ablation analysis of the maximum number of iterations. The ablation experiment was conducted to explore its impact on the model performance. The experimental results show that when When the value of is 4, the model can achieve the best enhancement effect, which verifies the role of iterative enhancement in promoting performance.

[0164] Table 8 Design ablation of loss function

[0165]

[0166] Ablation analysis of loss function design. L 1 loss, the model performs generally well; introduce perceptual loss L p Afterwards, PSNR increased by approximately 1.1, LPIPS decreased by approximately 0.99, and visual quality was significantly improved. Further increasing the loss in the middle layer enabled effective decomposition and coordinated optimization of brightness and structure, enhancing optimal performance.

[0167] For low-light image enhancement tasks, this paper proposes an efficient and lightweight U-shaped network architecture that integrates innovative illumination-aware cross-attention and structure reconstruction mechanisms. The LCAA module fuses image and brightness information, adaptively adjusting the amplitude to enhance brightness. Combined with the Mamba-based MaSR module, the architecture effectively restores both global structure and detailed textures. Extensive experiments demonstrate that this method achieves excellent quantitative and qualitative results on multiple synthetic and real-world low-light datasets, and also demonstrates outstanding performance gains in downstream tasks such as low-light object detection.

[0168] In summary, the lightweight low-light image enhancement method based on iterative illumination adjustment provided by the present invention performs HVI color space transformation on the image to be processed to obtain an HVI image, separates the illumination intensity component to obtain an illumination intensity feature map, and performs convolution processing on the HVI image to obtain a shallow feature map; performs feature fusion and brightness adjustment on the illumination intensity feature map and the shallow feature map to obtain a deep feature map, and gradually upsamples and restores the size through two layers of decoders. Each layer of decoder uses a specified size convolution for channel adjustment, and bilinear interpolation is used to achieve upsampling; a Mamba-based structure refinement module is introduced at the end of each layer of decoder to improve the structure restoration capability. The HVI features output by the last layer of decoder are adjusted to a specified value through a specified size convolution, and a residual connection is performed with the initial HVI image to obtain an enhanced HVI image, and the restored image is obtained through an inverse HVI transformation. Adaptive enhancement is performed under various low-light conditions to achieve multi-directional spatial context aggregation and improve texture representation.

[0169] The linear cross attention module (LCAM) is used to enhance the feature interaction between content and illumination with low computational and parameter costs. The amplitude adaptive iterative adjustment module (AAIA) iteratively adjusts the amplitude component in the frequency domain according to the image brightness level, enabling adaptive and balanced brightness enhancement under various complex low-light conditions. In the spatial domain, a Mamba-based structure refinement (MaSR) module is introduced to perform multi-directional context aggregation and structural texture recovery using a selective state space model, effectively preserving fine-grained spatial details and reducing the loss of structural information.

Claims

1. A lightweight low-light image enhancement method based on iterative illumination adjustment, characterized in that: The following steps are involved: S1: Perform HVI color space conversion on the image to be processed, converting it from sRGB space to HVI space to obtain an HVI image; S2: Separating the light intensity component from the HVI image to obtain a light intensity feature map of the HVI image to represent light information; S3: Convolution processing is performed on the HVI image to obtain a shallow feature map for capturing content and structure information; S4: Inputting the illumination intensity feature map and the shallow feature map into a two-stage encoder for feature fusion and brightness adjustment to obtain a deep feature map; S5: The depth feature map is gradually upsampled and restored to its size through two layers of decoders. Each layer of decoders uses a specified size of convolution to adjust the channel and bilinear interpolation to achieve upsampling. S6: A Mamba-based structure refinement module is introduced at the end of each decoder layer to improve the structural restoration capability. The HVI features output by the last decoder layer undergo a convolution of a specified size to adjust the number of channels to a specified value. The features are then residually connected with the initial HVI image to obtain an enhanced HVI image. The restored image is then obtained through an inverse HVI transform. The feature fusion process of step S4 is: The illumination intensity features and shallow features are flattened into illumination feature token sequences and shallow feature token sequences respectively, and divided into h For each head, the content feature is used as the query, the illumination feature is used as the key and value, and each head calculates the scaled cross attention separately; The brightness adjustment process of step S4 is: Perform Fourier transform on the initially enhanced content feature map, and introduce an adaptive control mechanism to process the illumination differences of different images; The Mamba-based structure refinement module includes a VSSM module and two sequentially connected convolutional layers with 1×1 convolution kernels, and split processing is performed between the two convolutional layers.

2. The lightweight low-light image enhancement method based on iterative illumination adjustment according to claim 1, characterized in that: The specific process of feature fusion in step S4 is as follows: S41: The light intensity feature F i and shallow features F hvi Flattened into a sequence of light feature tokens X i and shallow feature token sequences X hvi ; in, , ; ， ； H Represents the height of the feature map, W represents the width of the feature map, C Indicates the number of channels of the feature map; S42: Illuminate feature token sequence X i and shallow feature token sequences X hvi Divided by channel dimension h The specific formula is: ； in, , , , d h Indicates the number of channels contained in each attention head; S43: For each header, content features are used as queries , lighting features as keys Sum , each head calculates the scaled cross attention separately to obtain a deep feature map.

3. The lightweight low-light image enhancement method based on iterative illumination adjustment according to claim 2, characterized in that: In step S43, each head calculates the scaled cross attention separately to obtain the specific process of the deep feature map as follows: S431: In each zoom cross attention unit, first key After transposition, multiply it by the query Q to get the attention weight. The specific formula is as follows: ； in, , T represents matrix transpose, d h Indicates the number of channels contained in each attention head; S432: Scale the attention weight and multiply it by the value The transpose of is as follows: ； in, ; S433: The outputs of all heads are transposed and concatenated in the channel dimension. After passing through the fully connected layer and deformation, they are restored to the same size as the input features and added to the position embedding to finally obtain a deep feature map containing rich position information and cross-domain interaction.

4. The lightweight low-light image enhancement method based on iterative illumination adjustment according to claim 1, characterized in that: The brightness adjustment process in step S4 is implemented by the amplitude adaptive iterative adjustment module, and the specific process is as follows: S44: Perform Fourier transform on the initially enhanced content feature map, and apply convolution activation operation only on the amplitude component to achieve brightness adjustment; S45: The following adaptive control mechanism is introduced under the illumination differences of different images: First, perform global average pooling on the light intensity feature map to compress spatial information; Then two layers of 1×1 convolution reduce the number of channels to r Multiply by 1 channel to extract global illumination intensity B r , combined with a learnable dilation factor EF and the maximum number of iterations, according to the global illumination intensity B r The number of iterations of the amplitude transformation is dynamically determined to adjust the enhancement amplitude.

5. The lightweight low-light image enhancement method based on iterative illumination adjustment according to claim 1, characterized in that: The specific process of step S5 is as follows: The depth feature map is gradually upsampled and restored to size through two layers of decoders. Each layer of decoder uses 3×3 convolution for channel adjustment and bilinear interpolation to achieve upsampling. In order to maintain spatial consistency at high resolution, jump connections are used to splice the encoder features with the corresponding decoder features, and 1×1 convolution is used to compress the number of channels.

6. The lightweight low-light image enhancement method based on iterative illumination adjustment according to claim 1, characterized in that: Step S6 is implemented by the MaSR module, and the specific process is as follows: S61: The visual state space module VSSM of the MaSR module concatenates the light intensity features and shallow features in the channel dimension, and then reduces the number of channels by half through 1×1 convolution; S62: The features are fed into two parallel branches. The first branch passes through a linear layer, a depthwise convolution, and a SiLU activation function, and then is fed into a two-dimensional selective scanning module (2D-SSM) for layer normalization. The second branch passes through a linear layer and a SiLU activation function. S63: Perform element-wise multiplication and fusion on the outputs of the two branches and map them back to the input feature space through a linear layer.

7. The lightweight low-light image enhancement method based on iterative illumination adjustment according to claim 6, characterized in that: The specific process of layer normalization performed by the two-dimensional selective scanning module 2D-SSM in step S62 is as follows: The input image is expanded in forward and reverse sequences by row and column respectively to obtain four scanning paths, namely row forward, row reverse, column forward, and column reverse. Each path is processed by the state-space equation, and then the processing results of the four paths are fused and restored to the original two-dimensional image structure.

8. The lightweight low-light image enhancement method based on iterative illumination adjustment according to claim 1, characterized in that: It also includes the network training process, and in the training process, it is trained with the goal of "brightness adjustment first, then structure recovery", setting the illumination adjustment loss and structure texture loss, and combining L 1 loss and VGG-based perceptual loss L p Improve network reconstruction capabilities and perception quality; The lighting adjustment loss is calculated by outputting the intermediate result image in sRGB space in the middle of the network and downsampling it by 4 times compared to the true value image. The specific formula is as follows: ； in Indicates that the true value image in sRGB space is downsampled by 4 times; The intermediate result image is 4 times smaller than the final output image. L 1 means the mean absolute error loss used for the two images; L p represents the perceptual loss used on the two images; It is a hyperparameter used to control the proportion of perceptual loss, and its value is set to 0.01; The structural texture loss is calculated by calculating the final restored image in sRGB and HVI color spaces respectively: the specific formula is as follows: ； ； The total loss is calculated as follows: ； in represents the true value image; Y Represents the final result image; represents the true value image; Represents the final result image, L 1 means the mean absolute error loss used for the two images; L p represents the perceptual loss used on the two images; It is a hyperparameter used to control the proportion of perceptual loss used, and its value is set to 0.01.

Images