A low-light image enhancement method based on diffusion Mamba

Abstract

The application discloses a low-light image enhancement method based on diffusion Mamba, belongs to the technical field of image restoration, and solves the problems that a Transformer and a diffusion model respectively have high calculation complexity and inaccurate noise estimation in image restoration.The solution is as follows: firstly, a diffusion Mamba module and a diffusion feedforward network module cooperate to make the model complete basic noise estimation and feature enhancement, and the calculation complexity is reduced through a parallel scanning mechanism; then, the model is further optimized by constraining the difference between the restoration result and an actual image; finally, a four-step sampling algorithm is adopted to generate an enhanced image, and the L1 loss and the structural similarity (SSIM) loss function are combined to optimize the result from the two dimensions of pixel-level difference and structural consistency. The application not only utilizes the progressive denoising advantage of the diffusion model to ensure the image quality, but also reduces redundant calculation through dynamic parameter adjustment and hierarchical processing, so that the low-light image enhancement is realized.

Description

Technical Field

[0001] This invention belongs to the field of image restoration in the field of computer vision and image processing technology, specifically relating to a low-light image enhancement method based on diffusion Mamba. Background Technology

[0002] In scenarios such as security monitoring, autonomous driving, and night scene photography, images acquired under low light conditions suffer from problems such as low brightness, blurred details, and significant noise, which seriously affect image quality and subsequent analysis.

[0003] Traditional low-light image enhancement methods typically include histogram equalization and Retinex-based methods, which suffer from over-enhancement or loss of detail. Specifically:

[0004] 1) Histogram equalization: can easily lead to local overexposure of the image, destroying the original color and structural information of the image;

[0005] 2) Retinex-based methods: When dealing with complex lighting scenes, the accuracy of lighting estimation is low, and the enhancement effect is not ideal;

[0006] 3) Low-light image enhancement methods based on deep learning: Although some progress has been made, the computational burden is heavy, and existing acceleration techniques often reduce performance, making it difficult to balance computational efficiency and enhancement quality.

[0007] In addition, some methods struggle to balance noise and detail restoration in low-light images, resulting in noise residue or blurred details in the enhanced image.

[0008] Diffusion models have demonstrated powerful capabilities in image generation and restoration tasks, generating high-quality images through progressive denoising. However, traditional diffusion models suffer from high computational complexity and time consumption. The Mamba model, based on a state-space model (SSM), efficiently handles long sequences, dynamically adjusts parameters, and quickly captures long-range dependencies and detailed features of images. Combining Mamba with diffusion models provides a new technical approach for low-light image enhancement. Summary of the Invention

[0009] The main objective of this invention is to address the problems of high computational complexity and inaccurate noise estimation in image restoration using Transformer and diffusion models, respectively. This invention proposes a low-light image enhancement method based on diffusion-Mamba, employing a two-stage approach. The linearly complex Mamba state-space model is introduced into the image restoration process, including a diffusion state-space model and a diffusion feedforward neural network module. Through two-stage training, noise estimation and image restoration quality are optimized, balancing computational efficiency and restoration effectiveness, thereby achieving image enhancement.

[0010] The design concept of this invention addresses the problems of high computational resource consumption and inaccurate noise estimation caused by the quadratic complexity of traditional deep learning methods (such as Transformer and diffusion models) in image restoration. It proposes a two-stage diffusion method based on Mamba. The core concept lies in integrating the linear computational efficiency of the Mamba state-space model with the generative capability of the diffusion model to construct an image restoration framework that balances efficiency and accuracy. Mamba, as a sequence model based on the S4 structure, reduces the computational complexity of image restoration from quadratic to linear through dynamic parameter adjustment and multi-directional scanning mechanisms, solving the resource bottleneck of traditional methods when processing long sequences. Based on this, a two-stage training strategy is designed: the first stage optimizes the model's ability to capture noise distribution through noise constraint training; the second stage combines L1 loss and SSIM loss to improve restoration quality from both pixel-level accuracy and structural similarity dimensions, effectively solving the problem of inaccurate noise estimation in diffusion models. In summary, this invention achieves a balance between computational efficiency and restoration quality, providing an innovative solution for scenarios with high real-time and accuracy requirements, such as security monitoring and medical imaging. It exhibits stronger robustness, especially in complex scenarios, bringing significant performance improvements to image restoration tasks and demonstrating broad application prospects.

[0011] This invention is achieved through the following technical solution: a low-light image enhancement method based on diffusion Mamba. First, this invention deeply integrates the linearly complex Mamba state-space model with the diffusion model, addressing the shortcomings of existing technologies through a progressive strategy of noise-constrained training followed by image optimization training. Mamba's dynamic parameter adjustment capability based on the S4 structure can reduce the computational complexity of image restoration from quadratic to linear. Its multi-directional scanning mechanism can also effectively capture long-distance dependencies, compensating for the efficiency deficiencies of the Transformer model. The dual-stage training strategy first allows the model to learn the noise distribution pattern, and then optimizes the pixel details and structural consistency of the restored image through joint optimization of L1 and SSIM losses, solving the problem of inaccurate noise estimation in the diffusion model. On this basis, the diffusion state-space model and the diffusion feedforward neural network module are used together to achieve the following: the diffusion state-space model embeds the Mamba state-space update mechanism into the diffusion model's reverse denoising process, and realizes dynamic denoising intensity adjustment through time-step encoding to perceive the denoising stage; the diffusion feedforward neural network module adopts a structure combining gating mechanism and deep convolution to control the information flow to strengthen local structure learning. The two form a hierarchical extraction system of global dependencies and local features. During training, the first stage optimizes the model's ability to capture noise distribution through noise-constrained loss. The second stage generates a restored image using four-step sampling, balancing L1 and SSIM losses with a weight of λ = 0.84, thus improving restoration quality from both pixel-level accuracy and structural similarity dimensions. Specifically, it includes the following steps:

[0012] S1. First, let the highlight image be x and the low-light image be y. Then the highlight-low-light image pair matrix is ​​{x,y}∈R. H×W×3 Then, Gaussian noise ∈ is added to the specular image x according to the forward process of the diffusion model. t ~N(0,I), where N is a Gaussian distribution and I is the identity matrix, to obtain noisy specular image samples x. t Finally, x t The input tensor J∈R is obtained by concatenating the low-light image y along the channel dimension. H×W×6 , as input to the DiMB (DiffusionMambaBlock);

[0013] S2. Use a 3×3 convolutional layer to convert the input tensor J∈R. H×W×6 Encode the embedded feature F0, and use DiMB hierarchical encoding and decoding. At the same time, time step t is encoded as feature T and embedded in DiMB, and details are preserved by using cross-layer skip connections.

[0014] S3. Noise-constrained training: Extract the residual image from the encoding and decoding features using a 3×3 convolutional layer, and then compare it with the noisy sample x. t Adding them together yields the estimated noise tensor.

[0015] S4. Image Optimization Training: Estimating the Noise Tensor Noise constraints are removed, and a sampling algorithm is used to generate the restored image. The weight coefficient is set as λ. The image enhancement quality is improved by optimizing the model through the weight coefficient λ, L1 pixel difference loss and SSIM structural similarity loss, thus completing the low-light image enhancement based on diffusion Mamba.

[0016] Further, step S1 includes the following steps:

[0017] S1-1. First, let the highlight image be x and the low-light image be y. Then the highlight-low-light image pair matrix is ​​{x,y}∈R. H ×W×3 Then, Gaussian noise is added to the highlight image x according to the forward process of the diffusion model. That is, weighted Gaussian noise is gradually introduced at each step. The mathematical expression of the single-step diffusion noise addition process from time step t-1 to time step t is:

[0018]

[0019] In equation (1), x t Here are the specular image samples at time step t, x t-1 For the specular image sample at time step t-1, β t For variance scheduling, I is the identity matrix, and N is a Gaussian distribution;

[0020] The noise distribution expression is:

[0021]

[0022] In equation (2), q() represents the conditional probability distribution in the forward diffusion process; T is the total number of time steps;

[0023] Finally, we obtain the noisy specular image sample x. t ∈R H×W×3 ;

[0024] S1-2, Use a 3×3 convolutional layer on the six-channel input tensor H∈R H×W×6 Encoding is performed, and spatial local features are extracted using a sliding window. During convolution, the same padding is used to keep the feature map size unchanged, and the embedded features R0∈R are output. H ×W×C , where C is the preset number of channels; this operation maps the multi-channel input to a high-dimensional feature space, providing a basic feature representation for subsequent processing;

[0025] S1-3. Input the embedded features R0 into the hierarchical encoder of DiMB: the four-level structure is used to downsample step by step, and the spatial dimension is compressed at each time through dynamic convolution or self-attention mechanism, while aggregating global contextual information; the feature map output by each layer carries semantic information at different scales, forming a hierarchical feature pyramid.

[0026] S1-4. The time step t∈[0,1] is mapped to a feature vector Q through sine and cosine position encoding or MLP, and embedded into each layer of DiMB through gating mechanism or residual connection. Before each layer input of the encoder and decoder, the feature vector Q is added or concatenated with the spatial features channel by channel, so that the model can perceive the time information of the current denoising stage and dynamically adjust the processing logic.

[0027] S1-5. In the decoder stage, the features of the same level in the encoder of DiMB are spliced ​​with the corresponding layer output of the decoder through skip connections. That is, the features of the i-th layer of the encoder are fused with the features of the i-th layer of the decoder to restore spatial details. The decoder gradually restores the feature size through upsampling operations (such as deconvolution) and finally outputs a feature map of the same size as the input, preserving the local structural information lost during the encoding process.

[0028] Further, step S2 includes the following steps:

[0029] S2-1. Use a 3×3 convolution kernel on the input tensor J∈R H×W×6 Perform convolution operations using a learnable weight matrix W∈R 3×3×6×C and bias term b∈R CGenerate embedding feature F0 while keeping the spatial size unchanged. Embedding feature F0 is:

[0030] F0 = GELU(Conv) 3×3 (H)+b); (3)

[0031] S2-2. Using a state-space model, the input sequence i(t)∈R is mapped to the hidden state h(t)∈R. N And the predicted output sequence o(t)∈R is derived, which can be expressed by a linear ordinary differential equation as:

[0032] h(t) = Ah(t) + Bi(t); (4)

[0033] o(t) = Ch(t); (5)

[0034] In equation (4), A represents the state matrix, A∈R N×N B represents the input weight matrix, B∈R N×1 , is used to define how the hidden state evolves;

[0035] In equation (5), C represents the projection parameter, C∈R 1×N This is used to define how the input signal is projected onto the latent state and how the latent state is projected onto the output;

[0036] S2-3. Use the zero-order preservation rule to convert the state matrices A and B into discrete system parameters. for:

[0037]

[0038] In equations (6) and (7), Δ represents the time scale parameter, which is used to control the step size of the discretization of the continuous system; I is the identity matrix;

[0039] S2-4. Using Discrete System Parameters The hidden state h'(t) and the output sequence o'(t) are calculated recursively as follows:

[0040]

[0041] o'(t)=Ch(t; (9)S2-5、Construct structured convolution kernel Used to perform a convolution operation with the input sequence i;

[0042]

[0043] In formula (10), M represents the length of the input sequence i;

[0044] S2-6. Scan the input image along four directions: horizontal, vertical and two diagonal directions. Through the scanning extension module, the two-dimensional image is converted into a one-dimensional sequence to realize multi-view feature decomposition and provide multi-directional feature information for subsequent processing.

[0045] S2-7. Using the S6 module in DiMB, selective feature extraction is performed on the one-dimensional sequence expanded in step S2-6 to select effective features and filter noise and redundant information. Then, the sequence in four directions is summed through the scan merging module to restore the processed features to the same size as the input image, thus completing the feature enhancement and integration.

[0046] Furthermore, the effective features include brightness restoration, detail texture, and edge contrast.

[0047] Further, step S3 includes the following steps:

[0048] S3-1. Improve learning performance by using gating mechanisms and deep convolution: Gating mechanisms are used to control the flow of information and selectively pass information through element-wise multiplication, so that each level focuses on the details that supplement other levels, thereby learning more effective local image structures in image restoration.

[0049] S3-2. Normalize the input feature F and the time step t, where The spatial height of the input tensor X, The spatial width of the input tensor X. The input tensor X is the number of channels; then it is fed into a DFNN (Diffusion Feedforward Neural Network) for feature transformation, and finally the result is residually connected to the original input.

[0050]

[0051] In equation (12), F represents the input feature, and W represents the input feature. p 1 W represents the weights of the input projection convolutional layer. d 1 Represents the weights of the deep convolutional layer. The GeLU nonlinear activation function is represented by ⊙, which represents element-wise multiplication. W p 2 This represents the weights of the output projection convolutional layer, and LN() represents the layer normalization operation.

[0052] S3-3. Apply a 3×3 convolution operation to the encoded and decoded features. Extract the residual image by operating the convolution kernel and the feature map to capture the details in the image that need to be corrected.

[0053] S3-4. Compare the extracted residual image with the original noisy sample x. t By performing an addition operation and integrating the residual information with the original noise features, the estimated noise tensor is obtained. Complete the noise constraint training.

[0054] Further, step S4 includes the following steps:

[0055] S4-1, Based on the estimated noise ∈ t ,∈ t The loss function L is the real Gaussian noise added to the image at time step t. t (θ) is:

[0056]

[0057] S4-2. Further optimize the model using the data trained in the first stage. Combining this with four-step sampling can achieve the best recovery results. This is done by using L1 loss and L... SSIM The loss function L is used to constrain the generated restored image from the real image, thereby improving the restoration quality. rec for:

[0058] L rec =λL ssim (x t0 ,x)+(1-λ)L1(x t0 ,x); (14)

[0059] Loss function L t (θ) and loss function L rec They are complementary and progressive, designed for different stages of model training, with the goal of transitioning from indirectly constraining noise to directly constraining the recovery of results. t The core of (θ) is to enable the model to grasp the underlying rules of noise removal, laying the foundation for subsequent sampling to generate clear images. rec It is a direct constraint on the recovery result, shifting from learning the noise pattern to optimizing the final output.

[0060] In equation (14), L1 loss is the loss function used to calculate the absolute error between the generated image and the real image. SSIM The loss function is used to calculate the difference in structural similarity between the generated image and the real image, where λ is the weight.

[0061] S4-3. By continuously iterating and adjusting the model parameters, the image restoration quality is improved from both pixel and structural dimensions, effectively overcoming the limitations of single noise estimation, and finally achieving high-quality low-light image enhancement. Experiments have verified that this method significantly improves image restoration performance and completes low-light image enhancement based on diffusion Mamba.

[0062] Furthermore, the value of λ is 0.84.

[0063] This invention achieves multiple beneficial effects in the field of image restoration through innovative technical architecture and training strategies:

[0064] 1) Introducing the Mamba state-space model into the diffusion framework breaks through the secondary complexity bottleneck of traditional methods, significantly improves computational efficiency, and greatly reduces computational resource consumption while ensuring model performance, making it possible to deploy edge devices.

[0065] 2) The dual-stage training mechanism and modular design work synergistically, which not only strengthens the model’s ability to capture noise distribution, but also improves the accuracy of detail restoration through dynamic feature fusion. This effectively improves the problem of inaccurate noise estimation in the diffusion model, and makes the restored image significantly optimized in terms of pixel-level accuracy and structural consistency.

[0066] 3) The model has good generalization ability and robustness, can adapt to a variety of image restoration tasks, and shows strong adaptability to degradation factors in different scenarios, providing an efficient and reliable solution for complex image restoration needs in fields such as security and medical care. Attached Figure Description

[0067] Figure 1 This is a flowchart of the present invention; (original) Figure 2 and Figure 3 (Merge to form a complete flowchart)

[0068] Figure 2 This is the diffusion feedforward network module of the present invention;

[0069] Figure 3 To distribute the Mamba module DiMB;

[0070] Figure 4 For the S6 module in DiMB;

[0071] Figure 5 This is a visual comparison of the enhancement effects of this invention on other low-light images on the LOLv1 dataset;

[0072] Figure 6 This is a visualization comparison of the enhancement effects of this invention on other low-light images on the LOLv2-real dataset;

[0073] Figure 7 This is a visualization comparison of the effects of this invention on other low-light image enhancement methods on the LOLv2-synthetic dataset. Detailed Implementation

[0074] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0075] like Figures 1 to 4 The low-light image enhancement method based on diffusion Mamba, as shown, includes the following steps:

[0076] S1. First, let the highlight image be x and the low-light image be y. Then the highlight-low-light image pair matrix is ​​{x,y}∈R. H×W×3 Then, Gaussian noise ∈ is added to the specular image x according to the forward process of the diffusion model. t ~N(0,I), where N is a Gaussian distribution and I is the identity matrix, to obtain noisy specular image samples x. t Finally, x t The input tensor J∈R is obtained by concatenating the low-light image y along the channel dimension. H×W×6 As input to the Diffusion Mamba Block (DiMB); Step S1 includes the following steps:

[0077] S1-1. First, let the highlight image be x and the low-light image be y. Then the highlight-low-light image pair matrix is ​​{x,y}∈R. H ×W×3 Then, Gaussian noise is added to the highlight image x according to the forward process of the diffusion model. That is, weighted Gaussian noise is gradually introduced at each step. The mathematical expression of the single-step diffusion noise addition process from time step t-1 to time step t is:

[0078]

[0079] In equation (1), x t Here are the specular image samples at time step t, x t-1 For the specular image sample at time step t-1, β t For variance scheduling, I is the identity matrix, and N is a Gaussian distribution;

[0080] The noise distribution expression is:

[0081]

[0082] In equation (2), q() represents the conditional probability distribution in the forward diffusion process; T is the total number of time steps;

[0083] Finally, we obtain the noisy specular image sample x. t ∈R H×W×3 ;

[0084] S1-2, Use a 3×3 convolutional layer on the six-channel input tensor H∈R H×W×6 Encoding is performed, and spatial local features are extracted using a sliding window. During convolution, the same padding is used to keep the feature map size unchanged, and the embedded features R0∈R are output.H ×W×C , where C is the preset number of channels; this operation maps the multi-channel input to a high-dimensional feature space, providing a basic feature representation for subsequent processing;

[0085] S1-3. Input the embedded features R0 into the hierarchical encoder of DiMB: the four-level structure is used to downsample step by step, and the spatial dimension is compressed at each time through dynamic convolution or self-attention mechanism, while aggregating global contextual information; the feature map output by each layer carries semantic information at different scales, forming a hierarchical feature pyramid.

[0086] S1-4. The time step t∈[0,1] is mapped to a feature vector Q through sine and cosine position encoding or MLP, and embedded into each layer of DiMB through gating mechanism or residual connection. Before each layer input of the encoder and decoder, the feature vector Q is added or concatenated with the spatial features channel by channel, so that the model can perceive the time information of the current denoising stage and dynamically adjust the processing logic.

[0087] S1-5. In the decoder stage, the features of the same level in the encoder of DiMB are spliced ​​with the corresponding layer output of the decoder through skip connections. That is, the features of the i-th layer of the encoder are fused with the features of the i-th layer of the decoder to restore spatial details. The decoder gradually restores the feature size through upsampling operations (such as deconvolution) and finally outputs a feature map of the same size as the input, preserving the local structural information lost during the encoding process.

[0088] S2. Use a 3×3 convolutional layer to convert the input tensor J∈R. H×W×6 The feature F0 is encoded and encoded using DiMB hierarchical encoding and decoding. Simultaneously, time step t is encoded as feature T and embedded into DiMB, preserving details through cross-layer skip connections. Step S2 includes the following steps:

[0089] S2-1. Use a 3×3 convolution kernel on the input tensor J∈R H×W×6 Perform convolution operations using a learnable weight matrix W∈R 3×3×6×C and bias term b∈R C Generate embedding feature F0 while keeping the spatial size unchanged. Embedding feature F0 is:

[0090] F0 = GELU(Conv) 3×3 (H)+b); (3)S2-2, Using a state-space model, the input sequence i(t)∈R is mapped to the hidden state h(t)∈R. N And the predicted output sequence o(t)∈R is derived, which can be expressed by a linear ordinary differential equation as:

[0091] h(t) = Ah(t) + Bi(t); (4)

[0092] o(t) = Ch(t); (5)

[0093] In equation (4), A represents the state matrix, A∈R N×N B represents the input weight matrix, B∈R N×1 , is used to define how the hidden state evolves;

[0094] In equation (5), C represents the projection parameter, C∈R 1×N This is used to define how the input signal is projected onto the latent state and how the latent state is projected onto the output;

[0095] S2-3. Use the zero-order preservation rule to convert the state matrices A and B into discrete system parameters. for:

[0096]

[0097] In equations (6) and (7), Δ represents the time scale parameter, which is used to control the step size of the discretization of the continuous system; I is the identity matrix;

[0098] S2-4. Using Discrete System Parameters The hidden state h'(t) and the output sequence o'(t) are calculated recursively as follows:

[0099]

[0100] o'(t)=Ch(t; (9)S2-5、Construct structured convolution kernel Used to perform a convolution operation with the input sequence i;

[0101]

[0102] In formula (10), M represents the length of the input sequence i; Represents a structured convolution kernel;

[0103] S2-6. Scan the input image along four directions: horizontal, vertical and two diagonal directions. Through the scanning extension module, the two-dimensional image is converted into a one-dimensional sequence to realize multi-view feature decomposition and provide multi-directional feature information for subsequent processing.

[0104] S2-7. Using the S6 module in DiMB, selective feature extraction is performed on the one-dimensional sequence expanded in step S2-6. Effective features are selected and retained, while noise and redundant information are filtered out. Then, the sequence in four directions is summed by the scan merging module, and the processed features are restored to the same size as the input image, thus completing the feature enhancement and integration.

[0105] S3. Noise-constrained training: Extract the residual image from the encoding and decoding features using a 3×3 convolutional layer, and then compare it with the noisy sample x. t Adding them together yields the estimated noise tensor. Step S3 includes the following steps:

[0106] S3-1. Improve learning performance by using gating mechanisms and deep convolution: Gating mechanisms are used to control the flow of information and selectively pass information through element-wise multiplication, so that each level focuses on the details that supplement other levels, thereby learning more effective local image structures in image restoration.

[0107] S3-2. Normalize the input feature F and the time step t, where The spatial height of the input tensor X, The spatial width of the input tensor X. The input tensor X is the number of channels; then it is fed into a DFNN (Diffusion Feedforward Neural Network) for feature transformation, and finally the result is residually connected to the original input.

[0108]

[0109] In equation (12), F represents the input feature, and W represents the input feature. p 1 W represents the weights of the input projection convolutional layer. d 1 Represents the weights of the deep convolutional layer. The GeLU nonlinear activation function is represented by ⊙, which represents element-wise multiplication. W p 2 This represents the weights of the output projection convolutional layer, and LN() represents the layer normalization operation.

[0110] S3-3. Apply a 3×3 convolution operation to the encoded and decoded features. Extract the residual image by operating the convolution kernel and the feature map to capture the details in the image that need to be corrected.

[0111] S3-4. Compare the extracted residual image with the original noisy sample x. t By performing an addition operation and integrating the residual information with the original noise features, the estimated noise tensor is obtained. Complete the noise constraint training.

[0112] S4. Image Optimization Training: Estimating the Noise Tensor Noise constraints are removed, and a sampling algorithm is used to generate the restored image. With a weight coefficient of λ, the model is optimized using the weight coefficient λ, L1 pixel difference loss, and SSIM structural similarity loss to improve image enhancement quality. Step S4 includes the following steps:

[0113] S4-1, Based on the estimated noise ∈ t ,∈ t The loss function L is the real Gaussian noise added to the image at time step t. t (θ) is:

[0114]

[0115] S4-2. Further optimize the model using the data trained in the first stage. Combining this with four-step sampling can achieve the best recovery results. This is done by using L1 loss and L... SSIM The loss function L is used to constrain the generated restored image from the real image, thereby improving the restoration quality. rec for:

[0116] L rec =λL ssim (x t0 ,x)+(1-λ)L1(x t0 ,x); (14)

[0117] Loss function L t (θ) and loss function L rec They are complementary and progressive, designed for different stages of model training, with the goal of transitioning from indirectly constraining noise to directly constraining the recovery of results. t The core of (θ) is to enable the model to grasp the underlying rules of noise removal, laying the foundation for subsequent sampling to generate clear images. rec It is a direct constraint on the recovery result, shifting from learning the noise pattern to optimizing the final output.

[0118] In equation (14), L1 loss is the loss function used to calculate the absolute error between the generated image and the real image. SSIM The loss is a loss function that calculates the difference in structural similarity between the generated image and the real image, where λ is the weight, and in this embodiment, the value of λ is 0.84.

[0119] S4-3. By continuously iterating and adjusting the model parameters, the image restoration quality is improved from both pixel and structural dimensions, effectively overcoming the limitations of single noise estimation, and finally achieving high-quality low-light image enhancement. Experiments have verified that this method significantly improves image restoration performance and completes low-light image enhancement based on diffusion Mamba.

[0120] Therefore, this invention introduces two innovative modules in the first stage: a diffusion Mamba module and a diffusion feedforward network module. The diffusion Mamba module, based on the dynamic parameter adjustment capability of the State Space Model (SSM), captures long-distance dependencies in the image through a hierarchical encoding and decoding structure, and can adaptively adjust the feature extraction granularity when processing low-light images. The diffusion feedforward network module, through a combination of gating mechanisms and deep convolution, strengthens the learning of local structural features and effectively suppresses noise interference. The synergistic effect of the two modules allows the model to complete basic noise estimation and feature enhancement in the first stage, while reducing computational complexity through a parallel scanning mechanism. After the first stage training is completed, the second stage further optimizes the model by constraining the difference between the restored result and the real image. Finally, a four-step sampling algorithm is used to generate the enhanced image, and the results are optimized from both pixel-level differences and structural consistency by combining L1 loss and structural similarity (SSIM) loss functions. This two-stage strategy utilizes the asymptotic denoising advantage of the diffusion model to ensure image quality, while reducing redundant computation through dynamic parameter adjustment and hierarchical processing, thereby achieving low-light image enhancement.

[0121] Preliminary experiments were conducted on the low-light image enhancement method based on diffused Mamba proposed in this invention. All experiments were performed on an NVIDIA RTX 4090 GPU. The model underwent a first-stage training followed by a second-stage training. The time step t was set to 1000. Optimization was performed using the AdamW optimizer (β1 = 0.9, β2 = 0.999), with an initial learning rate set to 3e. -4 Gradually decrease to 1e -5 The learning rate is scheduled according to cosine annealing.

[0122] In this embodiment, the LOL dataset, including LOLv1 and LOLv2, was used for training and testing on a real low-light-normal paired dataset. The LOLv2 dataset contains LOLv2-real and LOLv2-synthetic datasets, and the results are analyzed below.

[0123] Figure 5 , Figure 6 , Figure 7The qualitative comparisons of this invention with other methods on the LOLv1, LOLv1-real, and LOLv2-synthetic datasets are presented respectively. Visualization results show that this invention exhibits significant advantages in low-light environments. Compared to other methods, this invention better preserves image details and textures, especially in low-light areas, providing a more natural enhancement effect. Compared to KinD and LLFlow, this invention improves brightness while avoiding detail loss and significantly outperforms RetinexNet in noise suppression, especially in high-noise environments, effectively reducing noise impact and generating clearer images. This invention demonstrates better performance than KinD, LLFlow, and RetinexNet, particularly in preserving image details, denoising, and enhancing brightness, making it a more competitive method for low-light image enhancement tasks.

[0124] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A low-light image enhancement method based on diffuse Mamba, characterized in that, Includes the following steps: S1. First, let the highlight image be x and the low-light image be y. Then the highlight-low-light image pair matrix is ​​{x,y}∈R. H×W×3 Then, Gaussian noise ∈ is added to the specular image x according to the forward process of the diffusion model. t ~N(0,I), where N is a Gaussian distribution and I is the identity matrix, to obtain noisy specular image samples x. t Finally, x t The input tensor J∈R is obtained by concatenating the low-light image y along the channel dimension. H×W×6 , as input to the diffused Mamba block DiMB; S2. Use a 3×3 convolutional layer to convert the input tensor J∈R. H×W×6 Encode the embedded feature F0, and use DiMB hierarchical encoding and decoding. At the same time, time step t is encoded as feature T and embedded in DiMB, and details are preserved by using cross-layer skip connections. S3. Noise-constrained training: Extract the residual image from the encoding and decoding features using a 3×3 convolutional layer, and then compare it with the noisy sample x. t Adding them together yields the estimated noise tensor. S4. Image Optimization Training: Estimating the Noise Tensor Noise constraints are removed, and a sampling algorithm is used to generate the restored image. The weight coefficient is set as λ. The image enhancement quality is improved by optimizing the model through the weight coefficient λ, L1 pixel difference loss and SSIM structural similarity loss, thus completing the low-light image enhancement based on diffusion Mamba.

2. The low-light image enhancement method based on diffused Mamba according to claim 1, characterized in that, Step S1 includes the following steps: S1-1. First, let the highlight image be x and the low-light image be y. Then the highlight-low-light image pair matrix is ​​{x,y}∈R. H×W×3 Then, Gaussian noise is added to the highlight image x according to the forward process of the diffusion model. That is, weighted Gaussian noise is gradually introduced at each step. The mathematical expression of the single-step diffusion noise addition process from time step t-1 to time step t is: In equation (1), x t Here are the specular image samples at time step t, x t-1 For the specular image sample at time step t-1, β t For variance scheduling, I is the identity matrix, and N is a Gaussian distribution; The noise distribution expression is: In equation (2), q() represents the conditional probability distribution in the forward diffusion process; T is the total number of time steps; Finally, we obtain the noisy specular image sample x. t ∈R H×W×3 ; S1-2, Use a 3×3 convolutional layer on the six-channel input tensor H∈R H×W×6 Encoding is performed, and spatial local features are extracted using a sliding window. During convolution, the same padding is used to keep the feature map size unchanged, and the embedded features R0∈R are output. H×W×C , where C is the preset number of channels; this operation maps the multi-channel input to a high-dimensional feature space, providing a basic feature representation for subsequent processing; S1-3. Input the embedded features R0 into the hierarchical encoder of DiMB: the four-level structure is used to downsample step by step, and the spatial dimension is compressed at each time through dynamic convolution or self-attention mechanism, while aggregating global contextual information; the feature map output by each layer carries semantic information at different scales, forming a hierarchical feature pyramid. S1-4. The time step t∈[0,1] is mapped to a feature vector Q through sine and cosine position encoding or MLP, and embedded into each layer of DiMB through gating mechanism or residual connection. Before each layer input of the encoder and decoder, the feature vector Q is added or concatenated with the spatial features channel by channel, so that the model can perceive the time information of the current denoising stage and dynamically adjust the processing logic. S1-5. In the decoder stage, the features of the same level in the encoder of DiMB are spliced ​​with the corresponding layer output of the decoder through skip connections. That is, the features of the i-th layer of the encoder are fused with the features of the i-th layer of the decoder to restore spatial details. The decoder gradually restores the feature size through upsampling operations and finally outputs a feature map of the same size as the input, retaining the local structural information lost during the encoding process.

3. The low-light image enhancement method based on diffused Mamba according to claim 1, characterized in that, Step S2 includes the following steps: S2-1. Use a 3×3 convolution kernel on the input tensor J∈R H×W×6 Perform convolution operations using the learnable weight matrix W∈R 3 ×3×6×C and bias term b∈R C Generate embedding feature F0 while keeping the spatial size unchanged. Embedding feature F0 is: F0=GELU(Conv 3×3 (H)+b); (3) S2-2. Using a state-space model, the input sequence i(t)∈R is mapped to the hidden state h(t)∈R. N And the predicted output sequence o(t)∈R is derived, which can be expressed by a linear ordinary differential equation as: h(t) = Ah(t) + Bi(t); (4) o(t) = Ch(t); (5) In equation (4), A represents the state matrix, A∈R N×N B represents the input weight matrix, B∈R N×1 , is used to define how the hidden state evolves; In equation (5), C represents the projection parameter, C∈R 1×N This is used to define how the input signal is projected onto the latent state and how the latent state is projected onto the output; S2-3. Use the zero-order preservation rule to convert the state matrices A and B into discrete system parameters. for: In equations (6) and (7), Δ represents the time scale parameter, which is used to control the step size of the discretization of the continuous system; I is the identity matrix; S2-4. Using Discrete System Parameters The hidden state h'(t) and the output sequence o'(t) are calculated recursively as follows: o'(t) = Ch(t); (9) S2-5, Constructing Structured Convolutional Kernels Used to perform a convolution operation with the input sequence i; In formula (10), M represents the length of the input sequence i; S2-6. Scan the input image along four directions: horizontal, vertical and two diagonal directions. Through the scanning extension module, the two-dimensional image is converted into a one-dimensional sequence to realize multi-view feature decomposition and provide multi-directional feature information for subsequent processing. S2-7. Using the S6 module in DiMB, selective feature extraction is performed on the one-dimensional sequence expanded in step S2-6 to select effective features and filter noise and redundant information. Then, the sequence in four directions is summed through the scan merging module to restore the processed features to the same size as the input image, thus completing the feature enhancement and integration.

4. The low-light image enhancement method based on diffused Mamba according to claim 3, characterized in that, In steps S2-7, the effective features include brightness recovery, detail texture, and edge contrast.

5. The low-light image enhancement method based on diffused Mamba according to claim 1, characterized in that, Step S3 includes the following steps: S3-1. Improve learning performance by using gating mechanisms and deep convolution: Gating mechanisms are used to control the flow of information and selectively pass information through element-wise multiplication, so that each level focuses on the details that supplement other levels, thereby learning more effective local image structures in image restoration. S3-2. Normalize the input feature F and the time step t, where The spatial height of the input tensor X, The spatial width of the input tensor X. The input tensor X is the number of channels; then it is fed into a DFNN (Diffusion Feedforward Neural Network) for feature transformation, and finally the result is residually connected to the original input. In equation (12), F represents the input feature, and W represents the input feature. p 1 W represents the weights of the input projection convolutional layer. d 1 Represents the weights of the deep convolutional layer. The GeLU nonlinear activation function is represented by ⊙, which represents element-wise multiplication. W p 2 This represents the weights of the output projection convolutional layer, and LN() represents the layer normalization operation. S3-3. Apply a 3×3 convolution operation to the encoded and decoded features. Extract the residual image by operating the convolution kernel and the feature map to capture the details in the image that need to be corrected. S3-4. Compare the extracted residual image with the original noisy sample x. t By performing an addition operation and integrating the residual information with the original noise features, the estimated noise tensor is obtained. Complete the noise constraint training.

6. The low-light image enhancement method based on diffused Mamba according to claim 1, characterized in that, Step S4 includes the following steps: S4-1, Based on the estimated noise ∈ t ,∈ t The loss function L is the real Gaussian noise added to the image at time step t. t (θ) is: S4-2, Loss Function L rec for: L rec =λL ssim (x t0 ,x)+(1-λ)L1(x t0 ,x); (14) Loss function L t (θ) and loss function L rec They are complementary and progressive, designed for different stages of model training, with the goal of transitioning from indirectly constraining noise to directly constraining the recovery of results. t The core of (θ) is to enable the model to grasp the underlying rules of noise removal, laying the foundation for subsequent sampling to generate clear images. rec It is a direct constraint on the recovery result, shifting from learning the noise pattern to optimizing the final output. In equation (14), L1 loss is the loss function used to calculate the absolute error between the generated image and the real image. SSIM The loss function is used to calculate the difference in structural similarity between the generated image and the real image, where λ is the weight. S4-3. By continuously iterating and adjusting the model parameters, the image restoration quality is improved from both pixel and structural dimensions, thus completing the low-light image enhancement based on diffusion Mamba.

7. The low-light image enhancement method based on diffused Mamba according to claim 6, characterized in that, The value of λ is 0.84.

Images