Low-illumination image enhancement method based on double-guide prompt diffusion model

Abstract

The invention discloses a low-illumination image enhancement method based on a double-guide prompt diffusion model. The method comprises the following implementation steps: carrying out size uniformization preprocessing on a generated sample pair; inputting the preprocessed low-illumination image into a diffusion model, and extracting an illumination perception prompt from the output illumination features; extracting an illumination invariant prompt from the preprocessed low-illumination image; the diffusion model is trained through cross attention injection illumination perception prompt and illumination invariant prompt in the diffusion model; and inputting the preprocessed to-be-enhanced low-illumination image into the trained double-guidance prompt diffusion model, and outputting an enhanced natural illumination image. According to the method, the brightness of the image is improved through a double-guide prompt strategy, real textures and details of the image are more accurately reserved and recovered, the calculation complexity is reduced, and deployment of an application scene is easy.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing, and further relates to the technical field of image enhancement or restoration, and specifically relates to a low-illumination image enhancement method based on a double-guided prompt diffusion model. The present application can be used to improve the image quality taken under low-light conditions, making it closer to the image under natural lighting conditions. BACKGROUND

[0002] Under low-light conditions, image quality is severely affected, with problems such as loss of detail, low contrast, and increased noise, which greatly affects the reliability of images in subsequent processing and analysis tasks. Traditional enhancement techniques, such as histogram equalization and gamma correction, while improving image brightness, often introduce color artifacts and noise amplification problems. In recent years, deep learning-based methods have gradually become mainstream, which use deep convolutional neural networks and Vision Transformers to learn the direct mapping from low-light to natural-light images, but when dealing with complex low-light conditions, they often output overly smoothed results, resulting in loss of detail and texture blur. Methods based on the Retinex theory assume that images can be decomposed into illumination and reflection components, but this method performs poorly when dealing with non-uniform lighting or complex material scenes, and may introduce color shifts or loss of detail.

[0003] Light estimation methods based on physical models attempt to simulate real-world lighting conditions by modeling light sources, object reflections, and atmospheric propagation, separating illumination and reflection components to restore image details and contrast. However, these methods fail to fully consider the effects of complex lighting conditions in actual environments, resulting in inaccurate light estimation and affecting image restoration results.

[0004] Diffusion model-based methods, as a new image processing technology, integrate degradation and image priors into the diffusion process through dynamic modules, and the diffusion model learns the probability distribution of the image to recover the true texture and details from noise, thereby avoiding over-smoothing problems. Guided diffusion models obtain conditional guidance information from input images, which can improve image brightness and contrast to some extent, further optimizing and controlling the image enhancement process. However, this method often relies on sub-optimal lighting images for guidance, which can introduce deviations in the guidance information itself, leading to irrelevant artifacts in the enhancement process, resulting in noise amplification and unnatural distortion. Moreover, the iterative denoising process has high computational overhead, and intensive guidance can lead to noise amplification.

[0005] Xiamen Ruwei Information Technology Co., Ltd. discloses a low-light image enhancement method based on a diffusion model in its applied patent document "Low-light image enhancement method based on brightness guidance and color adjustment diffusion model" (Application No. 202510100836.7, Application Publication No. CN 120047368 A). The implementation steps of this method are as follows: (1) brightness information extraction: to extract additional information in low-light images, the average value of pixels is extracted from the grayscale image through linear mapping as the representation of image brightness information. (2) Detail and texture enhancement: to improve the detail and texture quality of the enhancement result, a residual denoising diffusion model is introduced to learn the mapping relationship between natural light images and low-light images, thereby more effectively restoring image content. (3) Color restoration: to address the color distortion problem of low-light images, an adaptive chroma adjustment algorithm is proposed to adaptively adjust the chroma of the enhancement result through trainable parameters, ensuring the accuracy and naturalness of color restoration and significantly improving the performance of the enhancement model. This method improves the enhancement performance by deeply mining the information of low-light images. However, this method still has the following shortcomings: training the diffusion model requires obtaining the brightness level from the low-light image, and since the poor lighting image contains noise and artifacts, directly using it for dense guidance will inevitably result in undesirable effects such as enhanced noise and unnatural artifacts. At the same time, the model needs to gradually denoise in the reverse diffusion process based on multiple conditions such as residual images, brightness levels, noise images, etc. Each step of sampling involves matrix operations, noise prediction, residual estimation, and other operations, which cannot be parallelized to speed up, significantly slowing down the processing speed. For real-time or near-real-time processing scenarios such as video surveillance and autonomous driving, this method is difficult to deploy.

[0006] Tianjin University of Science and Technology disclosed a physical model-based low-light image enhancement method in its patent application "A Retinex-based Low-Light Image Enhancement Method" (Application No. 202410719371.9, Publication No. CN 118570082 A). This method reveals degraded content in the image through a degraded appearance restorer. It performs Retinex decomposition using two branches consisting of convolution and sigmoid activation functions to estimate the reflection and luminance components, respectively. After obtaining the estimated reflection and luminance components, gamma correction is used to adjust the luminance component. This method decouples the low-light image into a three-channel color map and a grayscale detail map to maintain consistency with the target image in color and detail representation. Furthermore, this application provides an unsupervised loss function to constrain the solution space of the Retinex decomposition, thereby improving adaptability in unknown and complex scenes. However, the method still has shortcomings. Retinex decomposition is essentially based on filters or convolution operations to estimate illumination and reflection. This separation is heuristic and does not have physical interpretability or reversibility. It only provides a statistical mapping for the network and cannot truly separate physical components. It is prone to over-smoothing and loss of texture. Summary of the Invention

[0007] The purpose of this invention is to address the shortcomings of the existing technology by proposing a low-light image enhancement method based on a dual-guided cue diffusion model. This method aims to solve the following problems existing in the guided diffusion model for low-light image enhancement:

[0008] (1) Dense guidance information injection: Guiding information obtained directly from suboptimal illumination images may introduce artifacts that are irrelevant to the enhanced target, resulting in noise amplification and unnatural distortion.

[0009] (2) Limitations of single-type guidance: Existing guidance diffusion models usually rely on a single type of guidance, such as relying only on illumination-dependent guidance or relying only on illumination-invariant guidance. This will result in either loss of details or inability to effectively adjust global brightness during the enhancement process.

[0010] (3) Computational efficiency problem: Some existing methods have significant computational overhead in the iterative denoising process, resulting in slow processing speed and difficulty in meeting real-time requirements.

[0011] The technical idea for achieving the object of the present application is that, in order to solve the problem of dense guidance information injection, the present application avoids directly relying on suboptimal illumination images for guidance by combining light perception prompts and light invariant prompts, thereby reducing noise amplification and unnatural distortion. In order to solve the problem that existing methods usually rely on a single type of guidance (for example, only focusing on light or only focusing on content), which leads to a dilemma: either losing the fine texture and details of the image when improving brightness, or failing to effectively adjust the global light when preserving details. The present application realizes the balance between global brightness adjustment and detail preservation by simultaneously using light-dependent guidance and light-invariant guidance through a dual-guidance strategy. The present application optimizes the diffusion model and the efficient guidance mechanism, that is, the guidance information is injected into the bottleneck layer of the U-Net through the cross-attention mechanism. The present application only selects the top K prompts sorted by cosine similarity to participate in guidance, which significantly reduces the computational overhead and improves the processing speed, meeting the real-time requirement. Through the innovative guidance strategy and the optimized model structure, the present application improves the generalization and robustness of the model in non-uniform light and complex material scenes, and ensures the stability and consistency of the enhanced results under various low-light conditions.

[0012] To achieve the above object, the implementation steps of the present application are as follows:

[0013] Step 1, generating at least one pair of sample pairs composed of a low-illumination image and a corresponding natural light image, and performing size consistency preprocessing on the sample pairs;

[0014] Step 2, inputting the preprocessed low-illumination image into the diffusion model, and extracting light perception prompts from the output light features;

[0015] Step 3, extracting light-invariant prompts from the preprocessed low-illumination image;

[0016] Step 4, injecting the light perception prompts and the light-invariant prompts into the diffusion model through the cross-attention, training the diffusion model, iteratively updating the model parameters until the joint loss function of the model converges, and obtaining the trained dual-guided prompt diffusion model;

[0017] Step 5, using the same preprocessing method as step 1, preprocessing the low-light image to be enhanced and inputting it into the trained dual-guided prompt diffusion model, and outputting the enhanced natural light image.

[0018] Further, the size consistency preprocessing step is as follows:

[0019] First, the low-illumination image and its corresponding natural light image are subjected to spatial size standardization processing to unify their resolutions to 256x256 pixels;

[0020] Secondly, the image after standardization is enhanced by geometric transformation, including random rotation, horizontal or vertical flip, cropping, scaling and translation.

[0021] Further, the step of extracting the illumination perception hint is as follows:

[0022] Firstly, the pixel value in the illumination feature image is averaged in spatial position by the following formula, and then converted into a query vector:

[0023] ;

[0024] wherein, represents the query vector of the illumination feature image, represents the width of the illumination feature image, represents the length of the illumination feature image, represents the pixel value of the illumination feature image at the h, w coordinates, , ;

[0025] Secondly, the cosine similarity between the query vector representing the illumination feature and a group of learnable key vectors is calculated according to the following formula:

[0026]

[0027] wherein, represents the cosine similarity of the th learnable key vector, , represents the total number of learnable key vectors, represents the th learnable key vector;

[0028] Thirdly, all the cosine similarities are sorted in descending order, and the first K similarity values in the sequence are selected to form the illumination perception hint, wherein K is a value preset according to the peak signal-to-noise ratio of the enhanced image.

[0029] Further, the step of extracting the illumination invariant hint is as follows:

[0030] Firstly, the low-illumination image after preprocessing is physically modeled, the influence of the light source intensity term is eliminated by calculating the ratio of spectral intensity and spectral derivative, and an illumination invariant feature representation only related to material reflectivity and its derivative is obtained:

[0031] ;

[0032] wherein, represents the operation of extracting the illumination invariance feature, a low-light image representing an input diffusion model, denotes a logarithm operation with base 10, denotes a multi-scale spatial derivative feature of the low-light image representing an input diffusion model, for depicting local structure information of the image at different positions and directions, , denote the normalized mean and standard deviation, respectively, denotes a constant value , for stabilizing numerical values during calculation;

[0033] Secondly, the channel standardization and dimension mapping are performed on the illumination-invariant features to generate illumination-invariant hints for providing geometric and structural information that is not affected by brightness.

[0034] Further, the step of training the diffusion model is as follows:

[0035] Firstly, the parameters in the diffusion model are iteratively updated using an Adam optimizer, and the initial learning rate is set as: ;

[0036] Secondly, the learning rate is adjusted according to the shape of the cosine function, and is gradually decayed through a cosine annealing strategy.

[0037] The adjustment of the learning rate according to the shape of the cosine function is realized by the following formula:

[0038] ;

[0039] wherein, denotes the learning rate after the current iterative update, denotes the minimum learning rate, and its value is , denotes the maximum learning rate, and its value is equal to the initial learning rate, denotes the current iteration number of training the diffusion model, denotes the maximum iteration number of training the diffusion model, and during the training process, the learning rate will gradually decrease with the increase of , and finally tends to .

[0040] Further, the joint loss function is: wherein, denotes the overall optimization objective function of the diffusion model; denotes a residual reconstruction loss function for constraining the difference between the enhanced image and the natural light image at the pixel level; denotes a perceptual consistency loss function for maintaining the consistency of high-level features in the deep feature space between the enhanced image and the natural light image. This indicates a decoupling contrastive loss function used to separate the features of illumination-aware cues from those of illumination-invariant cues. , , Indicates the regulating factor. = = Its value range is This is used to balance the weights of various losses in the overall optimization process.

[0041] The residual reconstruction loss function is: ,in, This indicates that the expected error is averaged. Represents an image of natural light. This represents the low-light image used as the input diffusion model. Indicates sampling noise. This represents the noise predicted by the neural network in the diffusion model. , These represent predefined noise scheduling coefficients. This represents the result after fusing the illumination-invariant cue and the illumination-aware cue. , This represents the fusion operation of cross-attention. Indicates a light perception cue. This indicates a light perception cue. Indicates the time step of the diffusion process. express Norm operations.

[0042] The perceptual consistency loss function is as follows: ,in, In the diffusion model, the VGG-19 network represents the first... Feature maps generated by the layers This represents the enhanced image. This represents the L1 norm operation.

[0043] The decoupling comparison loss function is as follows:

[0044] in, This indicates the batch size of low-light images input into the diffusion model. This represents the number of illumination perception cues selected using cosine similarity. , , Indicates unchanged lighting prompt key With the average vector of the illumination feature space dissimilarity, Indicates a light-sensing prompt button key of the illumination-invariant cue of the dissimilarity, , denotes a cosine similarity function.

[0045] Compared with the prior art, the present application has the following advantages:

[0046] Firstly, the present application avoids directly extracting dense guidance information from defective low-light images by combining illumination-aware cues and illumination-invariant cues through an innovative dual-guidance cue strategy. This method effectively solves the problems of noise amplification, unnatural distortion and color artifacts caused by poor guidance information in traditional guided diffusion models, enabling the present application to improve image brightness while more accurately preserving and restoring the true texture and details of the image, achieving a balance between global brightness and local details.

[0047] Secondly, the present application simultaneously utilizes illumination-aware cues for illumination-dependent guidance and illumination-invariant cues for detail content guidance through an innovative dual-guidance strategy. This design enables the model to have a clear division of labor, with illumination-aware cues responsible for accurately adjusting the overall brightness and contrast of the image, and illumination-invariant cues focused on preserving and restoring the inherent texture and structure of objects that are not affected by changes in illumination. This successfully addresses the inherent shortcomings of single guidance, enabling the present application to achieve the best balance between brightness and detail preservation, generating high-quality images that are both bright and clear.

[0048] Thirdly, compared to the large amount of computational overhead in the iterative denoising process of existing diffusion model methods, the present application significantly reduces processing delay through an optimized model structure and efficient guidance mechanism. By avoiding the use of all cues and reducing computational complexity, the present application enables real-time or near-real-time application scenarios such as video surveillance and autonomous driving that have high requirements for processing speed, solving the deployment difficulties of existing technologies and significantly improving computational efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 is a flowchart of the present application;

[0050] Figure 2 is a structural diagram of the dual-guidance cue diffusion model of the present application. DETAILED DESCRIPTION

[0051] The present application will be further described in detail below in conjunction with the drawings and examples.

[0052] In conjunction with Figure 2 the dual-guidance cue diffusion model of the present application shown in Figure 1 The implementation steps of the embodiments of the present application are further described in detail.

[0053] The system of the present application takes low-light images as input, cooperates with the illumination perception prompt and the illumination invariant prompt, and injects conditions in the U-Net denoising network of the diffusion model in a cross-attention manner layer by layer, so as to realize global brightness recovery and local detail preservation at the same time, and significantly reduce the noise amplification and artifact risk caused by suboptimal lighting dense guidance.

[0054] Step S1: obtaining a low-light dataset, the low-light dataset comprising a plurality of low-light images and corresponding natural light images, each image pair comprising a low-light image and a natural light image of the same scene. The low-light images and natural light images in the dataset are input as a group, wherein the low-light images are input images of the model, and the natural light images are reference images of the model.

[0055] Step S2: inputting the group of paired low-light images and natural light images into the network of the present application, extracting the illumination features of the low-light images at the bottleneck layer of the U-Net in the diffusion model , wherein and represent the height and width of the image respectively, represents the dimension of the feature. It is converted into a single query vector by spatial averaging , and the specific operation is to average all spatial positions of

[0056] ;

[0057] wherein is the width of the illumination feature image, represents the length of the illumination feature image, represents the index of the image coordinates of the illumination feature, , .

[0058] A group of learnable guidance prompts are defined, each guidance prompt is paired with a learnable key . The cosine similarity between the illumination feature query vector and each learnable key is calculated by the following formula:

[0059] ;

[0060] Then, the cosine similarity between the query vector and each learnable key is calculated, and the most relevant guidance prompt is dynamically selected according to the similarity score. Specifically, the first k guidance prompts with the highest similarity scores are selected and combined along the Token dimension to form the final illumination perception prompt , and then the channel number is pressed back through a learnable linear projection layer​ , which is consistent with the feature channels in U-Net, and is sent to the subsequent cross-attention.

[0061] Step S3: Illumination-invariant cue extraction.

[0062] The illumination intensity term is algebraically eliminated in the calculation process by the Kubelka-Munk light transfer theory, so that the result is independent of the light source intensity, thereby realizing the illumination invariance. The specific operation is as follows:

[0063] In this theory, the energy of the incident light spectrum at a certain spatial position x is described by the following formula:

[0064] ;

[0065] In the formula, represents the wavelength, is the spectrum of the light source, represents the specular reflection, is the intrinsic reflectivity of the material.

[0066] In order to derive useful invariants from this general model, a series of simplifying assumptions are introduced. First, it is assumed that the surface is glossy. This means that the specular reflection can be ignored, so the assumption , the original formula can be simplified as:

[0067] ;

[0068] Second, in order to eliminate the influence of the colorimetric characteristics of the light source, an equal-energy illumination model is assumed. Under this assumption, the light source spectrum is considered to be independent of the wavelength , and can therefore be represented as a spatially varying intensity term . Thus the model becomes:

[0069] ;

[0070] In order to facilitate expression in subsequent derivation, we express the spectral intensity as , the first-order spectral derivative with respect to wavelength as , and the second-order spectral derivative as . Thus, the spectral derivative of the observed energy can be expressed as a function of the material reflectivity and its corresponding derivative:

[0071] ;

[0072] ;

[0073] In the formula, is the first derivative of the material reflectance, is the second derivative of the material reflectance.

[0074] Now these relations can be used to construct an invariant. By computing the ratio of the squared modulus of the spectral derivative to the squared intensity, the following expression is obtained:

[0075] ;

[0076] The derivation shows that the illumination intensity terms cancel each other out mathematically. This result shows that this ratio is independent of the light source intensity, and therefore, the value of the following equation can be used as a robust and illumination-agnostic guidance signal for guiding the image enhancement process, as it depends only on the intrinsic material properties and their spectral derivatives.

[0077] ;

[0078] The actual computation of these invariant features is done by the CIConv layer, a dedicated layer inspired by the Kubelka-Munk theory. The CIConv layer processes the input image and transforms it into a set of feature maps, where each map captures a specific illumination-invariant property. The computation of these invariant features by the CIConv layer is based on the Kubelka-Munk transfer theory. The layer processes the input image and transforms it into a set of feature maps, each capturing a specific illumination-invariant property. These feature maps, representing the invariant quantities related to the chroma are then passed through a global pooling layer and a linear projection, transforming them into compact length embeddings, forming the illumination-invariant cues that are suitable for integration into the cross-attention mechanism of the diffusion model, used as key-value pairs. The CIConv output shape is . The spatial dimension is first compressed by global average pooling, and then upsampled to by the Linear layer, finally forming with the same dimension as , ensuring that the subsequent dual cues can be computed in parallel in the same cross-attention layer

[0079] This operation is expressed as follows:

[0080] ;

[0081] where denotes the operation of extracting the illumination-invariant features, denotes the input low-light image, denotes the log operation with base 10, denotes the multi-scale spatial derivative features of the input image, which depicts the local structure information of the image at different positions and directions, , denote the normalized mean and standard deviation, respectively, takes the value , used for a constant value for numerical stability when calculating. This enables the neural network to learn to generate the required illumination-invariant features.

[0082] Step S4: The illumination-aware cue and the illumination-invariant cue are injected as the condition guidance of the diffusion model through cross-attention at the bottleneck layer of the U-Net of the diffusion model.

[0083] First, the illumination-aware cue is fused with the feature map . In this operation, serves as the query, provides the key and value.

[0084] The features of the U-Net bottleneck layer are first generated by a 1x1 convolution to generate a query, and the illumination-aware cue and the illumination-invariant cue are mapped into key and value vectors, respectively, which are achieved by a learnable linear projection matrix , , , which is usually represented as a 1x1 convolution layer in actual operation.

[0085] ;

[0086] ;

[0087] wherein denotes the illumination-aware cue and the illumination-invariant cue with the subscript set.

[0088] By inserting the cross-attention mechanism in the latent processing layer of the denoising network, effective integration of the condition signal is achieved. For the latent feature map derived from the low-light input, the cross-attention mechanism integrates the double-guided cues in two steps, which is implemented as follows:

[0089] ;

[0090] wherein denotes the illumination-aware cue guided enhanced intermediate feature, denotes computing attention weights, converting the dot product results of a set of numerical values into a probability distribution, ensuring that the sum of all weights is 1, denotes the query vector of the low-light image feature map, denotes the key vector of the illumination-aware hint, denotes the dimension of the key vector, denotes the value vector of the illumination-aware hint.

[0091] Subsequently, the illumination-invariant hint is integrated into the updated feature map .

[0092] ;

[0093] wherein, denotes the feature map enhanced by the illumination-invariant hint guidance, denotes the query vector updated by , denotes the key vector of the illumination-invariant hint, denotes the value vector of the illumination-invariant hint.

[0094] The final refined feature map replaces the original and is propagated to subsequent layers of the U-Net. By repeating this two-stage fusion process at multiple semantic layers, the model dynamically utilizes illumination and content guidance throughout the denoising trajectory.

[0095] Step S5: Training strategy and objective function.

[0096] In the training phase, the model is trained using low-light images and natural-light images in the training set. By adjusting the model parameters through optimization algorithms, the model can learn the effective features and mapping relationships of low-light image enhancement. Specifically, the Adam optimizer is used for parameter update, and the initial learning rate starts from and gradually decays through the cosine annealing strategy.

[0097] Cosine annealing is a method of dynamically adjusting the learning rate, which core idea is to adjust the learning rate according to the training progress, following the shape of the cosine function. The specific formula is as follows:

[0098] ;

[0099] wherein, denotes the learning rate at the current iteration update, denotes the minimum learning rate, set to , denotes the maximum learning rate, with a value of the initial learning rate, denotes the current iteration number of model training, represents the maximum number of iterations for model training, and the learning rate will gradually decrease with the increase of , and finally tends to . This strategy helps to quickly converge in the early stage of training, and fine-tune through a smaller learning rate in the later stage of training, thereby improving the training effect and stability of the model. Through the cosine annealing strategy, the model can dynamically adjust the learning rate during the training process, thereby better adapting to different training stages, improving training efficiency and model performance.

[0100] In order to make the denoising network consider brightness recovery, detail preservation and double prompt semantics at the same time, the application adopts a joint objective function composed of three sub-losses, which is calculated as follows:

[0101] ;

[0102] In the formula, represents the overall optimization objective function of the diffusion model; represents the residual reconstruction loss function, which is used to constrain the difference between the enhanced image and the natural light image at the pixel level; represents the perceptual consistency loss function, which is used to maintain the consistency of high-level features in the deep feature space between the enhanced image and the natural light image; represents the prompt decoupling contrast loss function, which is used to separate the features of the light perception prompt and the light invariant prompt; 、 、 represents the adjustment factor, = = , whose value range is , used to balance the weight of each loss in the overall optimization process.

[0103] The residual loss directly lets the network predict the "difference between the natural light image and the low light image", and strengthens the brightness and contrast recovery:

[0104] ;

[0105] In the formula, represents the expected error average operation, represents the natural light image, represents the low light image input into the diffusion model, represents the sampling noise, represents the noise estimated by the U-net network in the diffusion model, 、 respectively represent the pre-defined noise scheduling coefficients, This represents the result after fusing the illumination-invariant cue and the illumination-aware cue. , This represents the fusion operation of cross-attention. Indicates a light perception cue. This indicates a light perception cue. This indicates the time step in the diffusion process.

[0106] Using perceived loss Features are extracted from the convolutional layers ReLU1, ReLU2, ReLU3, and ReLU4 of the pre-trained VGG-19, and the higher-order semantic distance between the augmented results and the ground truth is measured.

[0107] ;

[0108] In the formula, This represents the first... Feature maps generated by the layers This represents the enhanced image. This represents the L1 norm operation. VGG-19 is a classic deep convolutional network proposed by the Visual Geometry Group. It contains 19 layers (16 convolutional layers + 3 fully connected layers), and all convolutional kernels are 3×3 with a stride of 1. It expands the receptive field by continuously stacking small convolutional kernels. The network is pre-trained on ImageNet. Its shallow features are rich in low-level visual information such as edges and textures, while its deep features contain semantic and structural information. Therefore, it is often used as a feature extractor for perceptual loss to measure the similarity of images at the visual perception level.

[0109] Light perception prompts are ensured through a "push-pull" method. With constant lighting To avoid confusion, the semantic information between illumination-independent cues and illumination-aware cues is pushed away, ensuring that illumination-aware cues convey illumination-aware information, while simultaneously allowing... Maintains high similarity with diffusion bottleneck layer features:

[0110] ;

[0111] In the formula, This indicates the batch size of low-light images input into the diffusion model. This represents the number of illumination perception cues selected using cosine similarity. , , Represents the cosine similarity function. The key that indicates unchanged lighting With the average vector of the illumination feature space dissimilarity, Key representing illumination-aware prompt Key representing illumination-invariant prompt dissimilarity. The goal of the network is to minimize .

[0112] Step S6: Inference and generation.

[0113] The low-light image to be enhanced is input into the trained and optimized dual-guided prompt diffusion model. The model first dynamically retrieves and generates the corresponding illumination-aware prompt according to the illumination features of the input image, while extracting the illumination-invariant prompt for detail preservation. Subsequently, the model injects the two types of prompt conditions in the U-Net denoising network, cooperatively adjusts the illumination and details, and outputs the enhanced natural-light image.

Claims

1. A low-light image enhancement method based on a dual-guided cue diffusion model, characterized in that, The implementation steps of this method are as follows: Step 1: Generate at least one pair of samples consisting of a low-light image and its corresponding natural light image, and perform size uniformity preprocessing on the sample pairs; Step 2: Input the preprocessed low-light image into the diffusion model and extract the light perception cue from the output light features; Step 3: Extract illumination-invariant cues from the preprocessed low-light image; Step 4: Inject illumination-aware cues and illumination-invariant cues into the diffusion model through cross-attention, train the diffusion model, iteratively update the model parameters until the joint loss function of the model converges, and obtain the trained dual-guided cue diffusion model. Step 5: Using the same preprocessing method as in Step 1, the low-light image to be enhanced is preprocessed and then input into the trained dual-guided cue diffusion model, and the enhanced natural lighting image is output.

2. The low-light image enhancement method according to claim 1, characterized in that, The steps for the size uniformity preprocessing described in step 1 are as follows: The first step is to perform spatial size standardization on the low-light image and its corresponding natural lighting image to unify their resolution to 256×256 pixels. The second step involves using geometric transformations to augment the standardized image by performing random rotation, horizontal or vertical flipping, cropping, scaling, and translation operations.

3. The low-light image enhancement method according to claim 1, characterized in that, The steps for extracting illumination-sensing cues in step 2 are as follows: The first step is to use the following formula to perform spatial location averaging on the pixel values ​​in the illumination feature image, and then convert it into a query vector; ； in, This represents the query vector of the illumination feature image. Represents the width of the illumination feature image. Indicates the length of the illumination feature image. Image representing illumination features The pixel value located at the h and w coordinates. , ; The second step is to calculate the cosine similarity between the query vector representing the illumination features and a set of learnable key vectors, according to the following formula: ； in, Indicates the first Cosine similarity of learnable key vectors , This represents the total number of learnable key vectors. Indicates the first One learnable key vector; The third step is to sort all cosine similarities from largest to smallest, and select the top K similarity values ​​from the sequence to form the illumination perception cue, where K is a preset value based on the peak signal-to-noise ratio of the enhanced image.

4. The low-light image enhancement method according to claim 1, characterized in that, The steps for extracting the illumination invariance prompt in step 3 are as follows: The first step is to perform physical modeling on the preprocessed low-light image. By calculating the ratio of spectral intensity to spectral derivative, the influence of the light source intensity term is eliminated, resulting in an illumination-invariant feature representation that is only related to the material reflectivity and its derivative. ； in, This indicates the operation for extracting illumination-invariant features. This represents a low-light image used as input to the diffusion model. This represents a logarithmic operation with base 10. This represents the multi-scale spatial derivative features of the low-light image input to the diffusion model, used to depict the local structural information of the image at different locations and directions. , These represent the normalized mean and standard deviation, respectively. Indicates the value to be A constant used for Stable values ​​during calculation; The second step is to perform channel normalization and dimension mapping on the illumination-invariant features to generate illumination-invariant cues, which provide geometric and structural information that is unaffected by brightness.

5. The low-light image enhancement method according to claim 1, characterized in that, The steps for training the diffusion model described in step 4 are as follows: The first step is to use the Adam optimizer to iteratively update the parameters in the diffusion model, setting the initial learning rate as follows: ; The second step is to adjust the learning rate according to the shape of the cosine function and gradually decay it using a cosine annealing strategy.

6. The low-light image enhancement method according to claim 5, characterized in that, The adjustment of the learning rate according to the shape of the cosine function is achieved by the following formula: ； in, This represents the learning rate after the current iteration update. This represents the minimum learning rate, and its value is... , This represents the maximum learning rate, which is equal to the initial learning rate. This indicates the current iteration number in training the diffusion model. This represents the maximum number of iterations required to train the diffusion model, and the learning rate during training. Will follow As it increases, it gradually decreases, eventually approaching... .

7. The low-light image enhancement method according to claim 1, characterized in that, The joint loss function mentioned in step 4 is: ,in, This represents the overall optimization objective function of the diffusion model; This represents the residual reconstruction loss function, used to constrain the pixel-level differences between the enhanced image and the natural light image. This represents the perceptual consistency loss function, used to maintain the consistency of high-level features between the enhanced image and the natural light image in the deep feature space; This indicates a decoupling contrastive loss function used to separate the features of illumination-aware cues from those of illumination-invariant cues. , , Indicates the regulating factor. = = Its value range is This is used to balance the weights of various losses in the overall optimization process.

8. The low-light image enhancement method according to claim 7, characterized in that, The residual reconstruction loss function is: ,in, This indicates that the expected error is averaged. Represents an image of natural light. This represents a low-light image used as input to the diffusion model. Indicates sampling noise. This represents the noise predicted by the U-net network in the diffusion model. , These represent predefined noise scheduling coefficients. This represents the result after fusing the illumination-invariant cue and the illumination-aware cue. , This represents the fusion operation of cross-attention. Indicates a light perception cue. This indicates a light perception cue. Indicates the time step of the diffusion process. express Norm operations.

9. The low-light image enhancement method according to claim 7, characterized in that, The perceptual consistency loss function is as follows: ,in, This indicates the pre-trained VGG-19 network. Feature maps generated by the layers, This represents the enhanced image. This represents the L1 norm operation.

10. The low-light image enhancement method according to claim 7, characterized in that, The decoupling comparison loss function is as follows: in, This indicates the batch size of low-light images input into the diffusion model. This represents the number of illumination perception cues selected using cosine similarity. , , Indicates unchanged lighting prompt key With the average vector of the illumination feature space dissimilarity, Indicates a light-sensing prompt button The key with the unchanged lighting prompt dissimilarity, , This represents the cosine similarity function.

Images