Low-light image enhancement method and system based on semantic domain prior and lighting gain prediction

Abstract

The application discloses a low-light image enhancement method and system based on a semantic domain prior and illumination gain prediction, and the method comprises the following steps: acquiring an initial natural light domain migration direction; acquiring a low-light image to be enhanced and extracting an input semantic prior; constructing a content subspace, removing a projection component related to the content subspace from the initial natural light domain migration direction, and obtaining a natural light domain direction after content bias suppression; generating a semantic domain conditional vector based on the natural light domain direction and the input semantic prior; constructing a conditional illumination gain prediction network, performing forward diffusion disturbance on the low-light image, and generating noise input; the conditional illumination gain prediction network outputs shared features, and respectively predicts noise prediction results and a single-channel logarithmic illumination gain map; a semantic feature modulation module is set in the decoding stage of the conditional illumination gain prediction network, spatially adaptive semantic response features are obtained based on the semantic domain conditional vector; and an enhanced image is generated through exponential mapping and expansion to an image channel dimension.

Description

Technical Field

[0001] This invention focuses on low-light image enhancement technology in the fields of computer vision and image processing, specifically involving a low-light image enhancement method and system based on semantic domain prior and illumination gain prediction. Background Technology

[0002] Image quality enhancement in low-light environments is a crucial research area in image processing and computer vision. Due to insufficient lighting, limited exposure time, limited sensor dynamic range, and complex ambient light interference often present challenges in environments such as nighttime, low-light indoor environments, backlighting, tunnels, surveillance scenes, or mobile device photography, the acquired images typically exhibit degradation phenomena such as overall low brightness, underexposure in localized areas, insufficient contrast, color shift, significant noise in dark areas, blurred texture details, and uneven illumination distribution. The goal of low-light image enhancement technology is not simply to increase image brightness, but to improve image visibility while preserving the original image's structural contours, texture details, and natural color distribution as much as possible, and to prevent noise from being amplified synchronously with increased brightness. Existing low-light image enhancement methods mainly include traditional image enhancement methods and deep learning-based image enhancement methods. Traditional methods typically rely on prior assumptions such as histogram equalization, gamma correction, Retinex theory, dark channel or illumination map estimation to adjust brightness and contrast. Deep learning methods, on the other hand, learn the mapping relationship between low-light images and normally lit images through neural networks, further including supervised learning, unsupervised learning, zero-reference learning, and enhancement methods based on generative or diffusion models developed in recent years.

[0003] Prior art closely related to this invention mainly relies on unsupervised or zero-reference learning-based low-light image enhancement methods. These methods aim to reduce dependence on paired low-light and normal-light images, typically requiring no strictly registered training samples. Instead, they guide the model to estimate illumination, reflection, or enhancement mapping relationships from low-light input images by designing loss functions such as exposure control, color constancy, spatial consistency, illumination smoothness, structure preservation, noise suppression, or adversarial learning, thereby outputting images with higher brightness and better visual quality. Some methods also incorporate Retinex theory, decomposing low-light images into reflection and illumination maps, and achieving image reconstruction by learning illumination adjustment curves or enhancement coefficients. In recent years, visual language foundation models and diffusion models have also been introduced into low-light image enhancement tasks. The former provides strong high-level semantic representation capabilities, enabling the model to obtain prior information about scene categories, target regions, and natural image distribution; the latter has strong image distribution modeling capabilities, recovering image brightness, color, and details through generative denoising processes. Therefore, low-light image enhancement methods that combine unpaired training, semantic priors, and generative modeling capabilities have become a representative technical direction.

[0004] However, existing technologies still have several shortcomings. First, unsupervised or zero-reference methods mostly rely on low-level statistical constraints such as brightness, color, gradient, and spatial smoothness, lacking effective modeling of scene semantics and natural lighting domain distribution. This makes it difficult to differentiate lighting adjustments for different semantic objects such as faces, roads, sky, buildings, indoor backgrounds, shadow areas, and light source areas, easily leading to problems such as under-enhanced foregrounds, over-enhanced backgrounds, local halos, unnatural colors, or degraded texture details. Second, existing methods based on semantic cues or visual language models typically only use semantic information as a global supervision signal or textual cue constraint. Semantic features are difficult to directly participate in the multi-scale local lighting estimation process, thus providing insufficient guidance for local exposure recovery in complex scenes. Under unpaired training conditions, high-quality natural light images and low-light images often do not have consistent scene content. Directly using the differences between the two in the semantic embedding space as the enhancement direction can easily introduce content biases unrelated to lighting. Furthermore, diffusion-based enhancement methods typically require multiple backsampling steps, resulting in high computational costs that struggle to meet the efficiency requirements of real-time vision systems or downstream preprocessing tasks. Simultaneously, in severely underexposed areas, generative sampling can lead to texture drift, false details, or structural shifts, causing the enhancement results to deviate from the true content of the input image. Therefore, existing techniques still have room for improvement in areas such as unpaired natural lighting guidance, semantically adaptive enhancement, content bias suppression, structure preservation, and efficient inference. Summary of the Invention

[0005] This invention addresses the problems of insufficient natural light domain guidance, weak semantic adaptability, structural drift caused by generative enhancement, high inference cost of diffusion sampling, and over-enhancement or under-enhancement caused by low-level constraints in unpaired training scenarios of existing low-light image enhancement techniques. It proposes a low-light image enhancement method and system based on semantic domain prior and illumination gain prediction. This low-light image enhancement scheme can integrate semantic domain prior, illumination physical constraints and noise robust training mechanism to achieve natural, stable, structurally preserved and computationally efficient image enhancement under unpaired supervision.

[0006] This invention extracts semantic priors from low-light input images and domain priors from high-quality natural light images using a frozen visual semantic coding model, and weakens scene content interference in the unpaired reference domain through a content bias suppression mechanism. Furthermore, it constructs a conditional illumination gain prediction network, injecting semantic domain conditions into multi-scale decoding features. Simultaneously, during the training phase, a diffusion denoising objective is introduced as a noise scale-aware regularization, while during the inference phase, multi-step diffusion sampling is avoided, and the illumination gain map is predicted only through a single conditional forward propagation. The enhanced image is then obtained based on a Retinex-inspired reconstruction formula.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] The first aspect is a low-light image enhancement method based on semantic domain priors and illumination gain prediction, which includes the following steps:

[0009] S1. Obtain a set of low-light images, a set of high-quality natural light images, and an initial natural light domain migration direction to characterize the average feature change from the low-light image domain to the natural light image domain; obtain the low-light image to be enhanced and extract the input semantic prior;

[0010] S2. Construct a content subspace and remove the projection components related to the content subspace from the initial natural light domain migration direction to obtain the natural light domain direction after content bias suppression; generate a semantic domain conditional vector based on the natural light domain direction and the input semantic prior.

[0011] S3. Construct a conditional illumination gain prediction network. During the training phase, perform forward diffusion perturbation on the low-light image to generate noise input. The conditional illumination gain prediction network outputs shared features based on the noise input and semantic domain conditional vectors, and predicts the noise prediction result and the single-channel logarithmic illumination gain map respectively.

[0012] S4. Set up a semantic feature modulation module in the decoding stage of the conditional illumination gain prediction network to obtain spatially adaptive semantic response features based on the semantic domain conditional vector.

[0013] S5. The single-channel logarithmic illumination gain map generates an enhanced image through exponential mapping and expansion to the image channel dimension;

[0014] S6. Construct a loss function to jointly optimize network parameters; input the low-light image to be enhanced into the optimized conditional illumination gain prediction network to obtain the enhanced image.

[0015] Preferably, S1 includes:

[0016] After normalizing the size and pixels of the low-light training image set and the high-quality natural light image set, respectively, they are input into the frozen visual semantic image encoder to obtain the low-light image embedding set and the high-quality natural light image embedding set.

[0017] The mean values ​​of the low-light image domain and the high-quality natural light image domain are calculated separately, and the difference between the two is used as the initial natural light domain migration direction to provide the global natural light enhancement direction.

[0018] Perform perceptual brightness preprocessing on the low-light image to obtain a preprocessed image; input the preprocessed image into the frozen visual semantic image encoder to obtain the input semantic prior of the preprocessed image.

[0019] As a preferred option, the content subspace constructed in S2 includes:

[0020] Several content-related text terms are preset, and the content-related text terms are encoded using a frozen text encoder to obtain content-related text embeddings; the content-related text terms include terms used to describe scene content or object categories; the content-related text embeddings are stacked into a content subspace.

[0021] Preferably, in S2, generating the semantic domain condition vector includes:

[0022] The natural light domain direction and the input semantic prior are respectively input into the learnable mapping layer. and learnable mapping layer The natural light domain conditional component and the input semantic conditional component are obtained; the natural light domain conditional component and the input semantic conditional component are concatenated to obtain the semantic domain conditional vector.

[0023] Preferably, the conditional illumination gain prediction network includes an encoder, a decoder, skip connections, a semantic feature modulation module, a noise prediction head, and an illumination gain prediction head. The encoder is used to extract multi-scale features of the low-light image or its feature representation; the decoder is used to restore spatial resolution and predict the illumination gain map; the skip connections are used to preserve local structural information; the semantic feature modulation module is used to inject semantic domain conditional vectors into the decoding process; the noise prediction head is used for diffusion denoising regularization to generate noise prediction results; and the illumination gain prediction head is used to output a single-channel logarithmic illumination gain map.

[0024] As a preferred option, in S4:

[0025] The conditional illumination gain prediction network flattens the decoded features into a spatial feature sequence and inputs the semantic domain conditional vector into a lightweight multilayer perceptron to generate conditionally related semantic lexical units.

[0026] Using spatial feature sequences as queries and condition-related semantic terms as keys and values, a cross-attention operation is performed to obtain semantic response features with the same spatial resolution as the decoded features.

[0027] As a preferred embodiment, the conditional illumination gain prediction network in S4 also includes a spatial gating branch; the spatial gating branch concatenates the decoded features and semantic response features through channels, and generates a spatial gating map through convolutional layers and a sigmoid activation function, which is used to control the injection intensity of semantic response features at different locations.

[0028] Preferably, S4 further includes: generating channel-level scaling parameters based on the semantic domain condition vector c. and bias parameters The decoded features are then subjected to channel modulation to generate the modulated output features, which are represented as follows:

[0029]

[0030] Here, ⊙ represents element-wise multiplication. For semantic response features, This is a spatial gating diagram.

[0031] As a preferred option, the loss function in S6 is expressed as:

[0032] ,

[0033] in, This is used to constrain the noise prediction results to be consistent with the actual noise, thereby improving the network's robustness to low-light noise disturbances. This is used to constrain the illumination gain map to change smoothly in flat areas, and to appropriately relax the constraints at the edges of the image; This is used to constrain the enhanced image to move along the natural light domain in the visual semantic feature space and to approach the high-quality natural light image domain; Used to suppress pixel saturation and irreversible information loss caused by excessive gain; , and The weights for each loss item.

[0034] Secondly, a low-light image enhancement system based on semantic domain priors and illumination gain prediction includes:

[0035] The image acquisition module is used to acquire a set of low-light images and a set of high-quality natural light images.

[0036] A semantic domain prior construction module is used to obtain the initial natural light domain migration direction representing the average feature change from the low-light image domain to the natural light image domain; acquire the low-light image to be enhanced and extract the input semantic prior;

[0037] The content bias suppression module is used to construct the content subspace and remove the projection components related to the content subspace from the initial natural light domain migration direction to obtain the natural light domain direction after content bias suppression.

[0038] The conditional vector generation module is used to generate semantic domain conditional vectors based on the natural light domain direction and the input semantic prior.

[0039] The diffusion-regularized illumination gain prediction module is used to construct the conditional illumination gain prediction network. During the training phase, forward diffusion perturbation is performed on the low-light image to generate noise input. The conditional illumination gain prediction network outputs shared features based on the noise input and semantic domain conditional vectors, and predicts the noise prediction result and the single-channel logarithmic illumination gain map, respectively.

[0040] A semantic feature modulation module is included in the decoding stage of the conditional illumination gain prediction network to obtain spatially adaptive semantic response features based on the semantic domain conditional vector.

[0041] The enhanced image reconstruction module is used to generate an enhanced image by exponentially mapping a single-channel logarithmic illumination gain map to the image channel dimension.

[0042] The unsupervised training optimization module is used to construct loss functions to jointly optimize network parameters;

[0043] The inference output module is used to input the low-light image to be enhanced into the optimized conditional illumination gain prediction network, thereby obtaining the enhanced image;

[0044] The low-light image enhancement system based on semantic domain prior and illumination gain prediction is used to implement the low-light image enhancement method and steps based on semantic domain prior and illumination gain prediction as described in the first aspect.

[0045] The present invention has the following advantages over the prior art:

[0046] 1. Unlike existing technologies that directly predict and enhance RGB images or rely on low-level brightness curve adjustments, this invention employs an illumination gain prediction-based low-light image enhancement scheme. This scheme confines the enhancement process within the illumination gain space, enhancing the original low-light image by predicting a single-channel illumination gain map and applying it to the image. This effectively preserves the scene structure, edge information, and color relationships of the input image, reducing the risk of erroneous modification of image content under unsupervised training conditions.

[0047] 2. This invention proposes a semantic domain prior joint modeling mechanism. This mechanism utilizes a frozen visual semantic encoder to simultaneously extract the domain prior of the high-quality natural light image domain and the semantic prior of the current low-light input image, enabling the model to obtain the natural light enhancement direction and semantic information of the input scene even without paired supervision. This overcomes the limitations of traditional unsupervised enhancement methods that rely solely on low-level constraints such as brightness, color, or smoothness.

[0048] 3. This invention introduces a content bias suppression mechanism in unpaired natural light domain guidance. By constructing a content-related text feature subspace and removing content-related components from the initial natural light domain migration direction, this invention can reduce the misleading effect caused by scene content inconsistencies between high-quality reference images and low-light images, making the natural light domain direction more focused on enhancement-related attributes such as illumination, exposure, sharpness, and visibility.

[0049] 4. This invention proposes a semantic feature modulation module for illumination gain prediction. This module achieves adaptive fusion of semantic domain conditions and multi-scale local features through cross-attention, spatial gating, and channel-level modulation, avoiding homogenized modulation of all regions by global semantic conditions, and enabling the model to handle the balance between dark area enhancement, bright area protection, and texture preservation more finely.

[0050] 5. This invention uses diffusion denoising as a noise scale-aware regularization during the training phase, rather than as an iterative generation sampling process during the inference phase. This design leverages the robust modeling capability of diffusion denoising training against perturbations of different noise scales while avoiding the computational overhead and structural drift risk associated with multi-step backsampling. It is suitable for real-time low-light enhancement or preprocessing of downstream vision tasks.

[0051] 6. This invention adopts a modular system architecture design, setting semantic domain prior construction, content bias suppression, illumination gain prediction, semantic feature modulation, unsupervised training optimization, and single forward inference as relatively independent functional modules. This allows each module to be replaced, trimmed, or extended according to different application scenarios, giving it strong engineering adaptability and application promotion value. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the method framework of Embodiment 1 of the present invention;

[0053] Figure 2 This is a schematic diagram of the semantic feature modulation module in Embodiment 1 of the present invention;

[0054] Figure 3 This is a schematic diagram showing the enhancement effect of Embodiment 1 of the present invention on a low-light image enhancement dataset;

[0055] Figure 4 This is a schematic diagram showing the enhancement effect of Embodiment 1 of the present invention on a backlight scene dataset;

[0056] Figure 5 This is a schematic diagram of the face detection effect as a downstream visual task preprocessing module in Embodiment 1 of the present invention. Detailed Implementation

[0057] The present invention will be further described below with reference to the accompanying drawings and embodiments. It should be understood that the following embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention. Without departing from the concept of the present invention, those skilled in the art can make equivalent substitutions, combinations or improvements to the frozen visual semantic image encoder, frozen text encoder, content lexical set, conditional network structure, semantic feature modulation method, diffuse noise scheduling method, loss function form, training parameters, inference time step and image reconstruction method, and such substitutions, combinations or improvements should all fall within the scope of protection of the present invention.

[0058] This invention achieves adaptive enhancement of images in low-light, non-uniform lighting, and backlit scenes by organically integrating semantic information from low-light input images, domain prior information from high-quality natural light images, content bias suppression mechanisms, diffusion denoising regularization mechanisms, and Retinex-inspired illumination gain reconstruction strategies. This provides key technical support for nighttime surveillance, autonomous driving, mobile photography, robot vision, security detection, and other low-light vision tasks.

[0059] The core idea of ​​this invention is to predict a single-channel illumination gain map instead of directly generating an enhanced RGB image, and then apply this illumination gain map to the original low-light input image, thereby improving visibility while maintaining the structure, edges, and color relationships of the input image.

[0060] The specific implementation process of this invention includes a training phase and an inference phase. The training phase is used to construct a natural light domain prior, learn the conditional illumination gain prediction network, and optimize various unsupervised losses; the inference phase is used to perform a single forward enhancement on the low-light image to be enhanced. The training phase and the inference phase share the semantic domain conditional construction mechanism, but the inference phase no longer performs random noise sampling or multi-step diffusion backsampling.

[0061] Example 1:

[0062] like Figure 1 The overall system framework shown in this invention proposes a low-light image enhancement method based on semantic domain prior and illumination gain prediction. This framework includes semantic domain prior construction, content bias suppression, conditional vector generation, diffusion-regularized illumination gain prediction, illumination gain map reconstruction, and unsupervised training constraints. Specifically, it includes the following steps:

[0063] Step 1: Semantic domain prior construction.

[0064] Obtain a low-light training image set and a high-quality natural light image set. The low-light training image set represents the image distribution under low light, weak exposure, or non-uniform lighting conditions; the high-quality natural light image set represents the image distribution under normal exposure, natural lighting, and higher perceived quality. These two image sets do not require image-level pairing; that is, the low-light training images do not need to have strictly corresponding normal lighting label images.

[0065] In a preferred embodiment, the set of low-light training images is denoted as... A set of high-quality natural light images is denoted as The low-light image to be enhanced is denoted as First of all, and The images in the image are normalized in size and pixels, and then fed into the frozen visual semantic image encoder. This yields a set of low-light image embeddings and a set of high-quality natural light image embeddings. Each image embedding preferably undergoes L2 normalization to improve feature comparability between different images.

[0066] Subsequently, the low-light image domain mean was calculated. and high-quality natural light image domain mean The difference between the two is used as the initial natural light domain migration direction. ,Right now:

[0067] .

[0068] in, This direction is used to characterize the average semantic feature change from a low-light image domain to a high-quality natural light image domain. It does not correspond to the enhancement target of a specific image, but rather provides a global natural light enhancement direction in unpaired training scenarios.

[0069] Furthermore, regarding the current low-light images that need enhancement... The system performs perceived brightness preprocessing, such as gamma correction or slight brightness enhancement, to obtain... This preprocessed image is only used to improve the frozen visual semantic image encoder's ability to recognize dark area content; it is not used as the final enhanced image, nor does it replace the original low-light input image. Subsequently, Input the frozen visual semantic image encoder to obtain the normalized semantic embedding of the current input image. This input semantic prior is used to characterize the scene content and semantic structure in the current image, enabling the subsequent lighting enhancement process to adaptively adjust according to the specific image content.

[0070] Step 2: Content bias suppression and condition vector generation.

[0071] Due to low-light image collection With a collection of high-quality natural light images They are not captured in pairs; there may be differences in scene content, object categories, and shooting environment. If used directly... As an enhancement direction, this direction may simultaneously include changes in lighting and changes in scene content. For example, a high-quality natural light image set may contain more outdoor scenes, while a low-light image set may contain more indoor scenes. Directly calculating the difference in their means may cause the network to learn a content migration direction unrelated to lighting. Therefore, this invention further introduces a content bias suppression mechanism.

[0072] Specifically, the system pre-defines several content-related text terms and utilizes a frozen text encoder. The embedded text is encoded to obtain content-related text embeddings. Content-related text terms include, but are not limited to, terms describing scene content or object categories such as wall, door, window, ceiling, table, chair, floor, room, light fixture, furniture, road, building, and sky. These text embeddings are then stacked into a content subspace matrix B. The system calculates the initial natural light domain migration direction. The projection component on the content subspace B, and the projection component from The content bias is removed to obtain the natural optical domain direction r after content bias suppression. Preferably, this process is expressed as:

[0073] ,

[0074] Where η is the numerically stable regularization term and I is the identity matrix. The natural light domain direction r is normalized and used as a fixed natural light domain prior. This mechanism does not require the content direction and illumination direction to be completely separable in the feature space. Instead, it reduces the interference of content differences in the unpaired reference domain on the enhancement direction by suppressing components strongly correlated with the preset content direction, making the natural light domain direction more focused on enhancement-related attributes such as exposure, illumination, sharpness, and visibility.

[0075] Subsequently, the system combines the natural light domain direction r with the input semantic prior. Input learnable mapping layers respectively and The natural light domain conditional component was obtained. and input semantic conditional components :

[0076] ,

[0077] After that, and The semantic domain condition vector c is obtained by concatenation:

[0078]

[0079] This conditional vector c serves as a global conditional input in subsequent networks, providing guidance on both the natural light enhancement direction and the current image's semantic content. The natural light domain conditional component... Provides a global direction for moving from the low-light domain to the high-quality natural light domain, and inputs semantic conditional components. It provides scene content information of the current image, and the two together constrain the illumination gain prediction process.

[0080] Step 3: Predict illumination gain with diffusion regularization.

[0081] A conditional illumination gain prediction network is constructed. This network preferably adopts a conditional U-Net structure, including an encoder, a decoder, skip connections, a semantic feature modulation module, a noise prediction head, and an illumination gain prediction head. The encoder extracts multi-scale features from the low-light image or its feature representation; the decoder restores spatial resolution and predicts the illumination gain map; skip connections preserve local structural information; the semantic feature modulation module injects semantic domain conditional vectors into the decoding process; the noise prediction head performs diffusion denoising regularization; and the illumination gain prediction head outputs a single-channel logarithmic illumination gain map.

[0082] During training, the system randomly samples time step t from a preset set of time steps and samples Gaussian noise ε to represent low-light images. Perform forward diffusion perturbation to obtain noise input. This process can be represented as:

[0083] ,

[0084] in, Let be the cumulative noise scheduling coefficient corresponding to time step t during the diffusion process. Conditional U-Net uses... The time-step embedding t and the semantic domain conditional vector c are taken as inputs, and the output is the shared feature. Shared features are input into the noise prediction head. and lighting gain prediction head The prediction results of the added noise are obtained. Logarithmic illumination gain plot .

[0085] In this invention, the diffusion process is used only for noise scale-aware feature learning during the training phase, and not for multi-step generation sampling during the inference phase. That is, the diffusion denoising target does not directly generate the enhanced image, but rather serves as an auxiliary regularization term to constrain shared features, enabling the network to learn stable low-light image representations at different noise scales. In this way, the illumination gain prediction network can reduce the impact of dark area noise perturbations on gain estimation, thereby mitigating the noise amplification problem commonly found in low-light enhancement.

[0086] Step 4: Semantic feature modulation and multi-scale illumination guidance.

[0087] like Figure 2 The semantic feature modulation module shown in this invention is used to inject semantic domain conditional vectors into the multi-scale decoding features of the conditional illumination gain prediction network. Figure 2The interaction process between semantic domain conditional vectors and spatial decoding features is illustrated, including generating condition-related semantic lexical units from the conditional vectors, performing cross-attention calculation with spatial features as queries and semantic lexical units as keys and values, controlling the injection intensity of semantic response features through spatial gating, and performing illumination-related calibration on the decoding features through channel-level scaling and bias modulation to obtain semantically perceptual modulation features.

[0088] Semantic feature modulation modules are set up in one or more decoding stages of the conditional U-Net. For the feature map of the s-th decoding stage... The system first flattens it into a spatial feature sequence. The semantic domain condition vector c is then input into a lightweight multilayer perceptron to generate conditionally relevant semantic lexical units. Subsequently, using... As a query, a cross-attention operation is performed using conditionally relevant semantic terms as keys and values ​​to obtain the result. Semantic response features with the same spatial resolution .

[0089] Simply adding or concatenating global conditional vectors directly into spatial features can easily lead to all regions receiving the same level of semantic modulation, resulting in over-enhanced bright areas, excessively smoothed textured areas, or insufficiently enhanced dark areas. To avoid this problem, this invention further incorporates a spatial gating branch. This branch decodes the features... With semantic response features Channel stitching is performed, and a spatial gating map is generated using convolutional layers and a sigmoid activation function. Spatial gating maps are used to control the injection intensity of semantic response features at different locations. For dark areas, foregrounds, or semantically salient regions, gating maps can enhance conditional guidance; for fully exposed areas or texture-sensitive regions, gating maps can suppress overmodulation.

[0090] Furthermore, the system generates channel-level scaling parameters based on the semantic domain condition vector c. and bias parameters The decoded features are then modulated using channel modulation. The modulated output features can be expressed as:

[0091]

[0092] Here, ⊙ represents element-wise multiplication. Through this structure, the semantic feature modulation module simultaneously achieves spatial location-level enhancement control and channel-level illumination calibration, making illumination gain prediction more adaptable to different semantic regions and lighting conditions. In other words, this module ensures that semantic domain priors are not merely global constraints on the final image, but directly participate in the multi-scale local illumination estimation process.

[0093] Step 5: Illumination gain reconstruction.

[0094] Illumination gain prediction head outputs single-channel logarithmic illumination gain graph The system obtains a positive illumination gain map through exponential mapping. :

[0095]

[0096] Then Extended to input low-light images Same number of channels, and with Pixel-by-pixel multiplication yields the enhanced image. :

[0097]

[0098] Here, Broadcast represents the channel expansion operation, and clip represents the pixel value cropping operation. Since the enhanced image is jointly determined by the original low-light image and the illumination gain map, the enhancement process is always constrained by the structure of the input image, avoiding content drift that may occur with direct RGB image generation. This illumination gain reconstruction method is consistent with the Retinex-inspired image formation concept, which adaptively adjusts the illumination component while preserving the reflective components and structural content as much as possible.

[0099] Step 6: Unsupervised training objectives.

[0100] During training, the system does not use paired normal illumination labeled images. Instead, it jointly optimizes network parameters through loss functions such as diffusion denoising regularization, edge-aware illumination smoothing, semantic domain alignment, and inverse illumination consistency. The total loss function can be expressed as:

[0101] ,

[0102] in, This is used to constrain the noise prediction results to be consistent with the actual noise, thereby improving the network's robustness to low-light noise disturbances. This is used to constrain the illumination gain map to change smoothly in flat areas, and to appropriately relax the constraints at the edges of the image; This is used to constrain the enhanced image to move along the natural light domain in the visual semantic feature space and to approach the high-quality natural light image domain; Used to suppress pixel saturation and irreversible information loss caused by excessive gain; , and The weights for each loss item.

[0103] Specifically, the edge-aware illumination smoothing term weights the illumination gain map gradient based on the gray-level gradient of the input low-light image. For flat regions, the system strengthens the smoothing constraint to avoid noise amplification caused by abrupt changes in local illumination gain; for structural edge regions, the system reduces the smoothing penalty to preserve true edge variations. This loss term can maintain image structural sharpness while suppressing noise amplification.

[0104] The semantic domain alignment term includes orientation alignment constraints and natural light domain anchoring constraints. Orientation alignment constraints encourage the direction of visual semantic feature changes in the enhanced image relative to the original low-light image to align with the orientation of the natural light domain after content bias suppression; natural light domain anchoring constraints encourage the features of the enhanced image to approximate the domain mean of the high-quality natural light image. In this way, the model can obtain natural light domain guidance even without paired labels.

[0105] The inverse illumination consistency term is obtained by comparing the enhanced image, which is inversely divided by the predicted illumination gain, with the original low-light image. When the predicted gain is too large, leading to pixel cropping or saturation, this inverse mapping struggles to recover the original low-light input, resulting in a larger loss value. Therefore, the inverse illumination consistency term effectively suppresses over-enhancement, bright area saturation, and information loss.

[0106] Step 7: Single forward reasoning.

[0107] During the inference phase, the system no longer samples random noise or performs multi-step diffusion backsampling. For the input low-light image, the system extracts its semantic prior, which, together with the pre-calculated and stored natural light domain direction, forms a semantic domain conditional vector. Subsequently, the system uses a fixed low-noise time step to feed the input image representation and conditional vector into a trained illumination gain prediction network. An illumination gain map is obtained through a single conditional forward propagation, and the enhanced image is reconstructed based on the illumination gain map. The inference phase requires no normal light reference image, no text prompts, no high-quality guiding image, and no multi-step backsampling of the diffusion model. This inference process has advantages such as high computational efficiency, strong structure preservation, and convenient deployment, making it suitable for low-light vision systems such as nighttime surveillance, autonomous driving perception, mobile photography, robot vision, and security detection.

[0108] Experimental verification:

[0109] To illustrate the effects of implementing the present invention, the following is combined with... Figures 3 to 5 Tables 1 to 3 illustrate the experimental verification results of the present invention. It should be understood that the following experimental results are only used to illustrate the effects that the technical solution of the present invention can achieve, and should not be construed as limiting the scope of protection of the present invention.

[0110] Table 1 shows the quantitative comparison results of the present invention and existing methods on the LOL and Backlit300 datasets. The LOL dataset was used for paired low-light enhancement evaluation, and the Backlit300 dataset was used for generalization evaluation in complex backlit scenes. PSNR and SSIM in Table 1 are used to evaluate the signal fidelity and structural similarity between the enhanced image and the reference image, while NIQE and MUSIQ are used to evaluate image naturalness and perceptual quality. As can be seen from Table 1, under the condition of no paired normal-light labeled image supervision, the present invention achieves higher structural similarity and better perceptual quality evaluation results on the LOL dataset, indicating that the present invention can maintain image structure and natural appearance while enhancing the visibility of dark areas. On the Backlit300 dataset, the present invention achieves better no-reference quality evaluation results, indicating that the present invention has good adaptability to spatially non-uniform lighting and backlit scenes. Backlit scenes usually have both bright backgrounds and underexposed foregrounds. If only global brightness enhancement is performed, it is easy to cause further overexposure in areas such as the sky, windows, and light sources; if the enhancement is too conservative, dark areas such as people, roads, or buildings will still be invisible. This invention, through semantic domain conditions and gated semantic feature modulation, can more rationally adjust the illumination gain of different regions. The corresponding table in the paper shows that the proposed method achieves SSIM 0.7926, NIQE 3.4270, and MUSIQ 62.4890 on LOL, and NIQE 3.0180 and MUSIQ 56.0767 on Backlit300.

[0111] Table 1: Quantitative comparison results of the present invention and existing methods on the LOL and Backlit300 datasets.

[0112]

[0113] Table 2 shows the no-reference image quality evaluation results of the present invention and existing methods on real unpaired low-light datasets such as MEF, LIME, NPE, VV, and DICM. Due to the significant differences in degradation types, shooting devices, scene content, and illumination distribution in real low-light images, single low-level constraint methods are prone to over-enhancement, color drift, or detail loss in certain scenes. As shown in Table 2, the present invention maintains relatively stable no-reference perceptual quality across multiple real-world scene datasets, rather than optimizing only a single dataset or a single metric. The present invention utilizes natural light domain priors to provide global enhancement directions and adapts the current image content using input semantic priors, thus exhibiting good cross-scene stability in multi-dataset evaluations.

[0114] Table 2: No-reference quality assessment results of the present invention and existing methods on real unpaired low-light datasets

[0115]

[0116] like Figure 3 The visualization enhancement results of this invention on the LOL dataset shown demonstrate the effectiveness of this invention in improving dark area visibility, preserving structure, and enhancing color naturalness. Figure 3 As can be seen, while some existing methods can improve overall brightness, they are prone to problems such as grayish colors, unstable local contrast, amplified noise in dark areas, or unnatural textures. Some methods based on Retinex or generative modeling can preserve certain structures, but may still exhibit halo artifacts or color shifts when the illumination estimation is inaccurate. In contrast, this invention constrains the enhancement process through a single-channel illumination gain map, making the enhancement result mainly reflect spatial illumination adjustment rather than regenerating image content. At the same time, semantic domain priors provide high-level guidance for exposure recovery, thus maintaining relatively stable structure and more natural color performance while enhancing dark areas.

[0117] Figure 4 This paper presents the visualization enhancement results of the present invention in a Backlit300 backlit scene, demonstrating the effectiveness of the invention in enhancing dark areas and protecting bright areas in non-uniform lighting and backlit scenes. Backlight enhancement is more challenging than ordinary low-light enhancement because images often contain both overly bright backgrounds and underexposed foregrounds. Figure 4 It can be seen that methods relying on global exposure adjustment tend to overexpose the sky, windows, or brightly lit backgrounds, while conservative enhancement methods tend to keep foreground figures, roads, or buildings dark. This invention can improve visibility in dark areas while better protecting the original bright areas, demonstrating that semantic domain conditions, spatial gating, and illumination gain prediction mechanisms can help the network adaptively adjust according to different scene content and local lighting conditions.

[0118] Table 3 shows the ablation experiment results of the core modules of this invention. This table examines the performance changes after removing the input semantic prior, removing the natural light domain prior, removing content bias suppression, removing diffusion denoising regularization, removing the semantic feature modulation module, and removing different training loss terms. The results show that removing any core component leads to varying degrees of performance degradation. This indicates that the input semantic prior, natural light domain prior, content bias suppression, diffusion regularization, semantic feature modulation module, and various unsupervised training objectives have complementary effects. The input semantic prior helps the model understand the current image content; the natural light domain prior helps provide global enhancement directions; content bias suppression helps reduce content interference in the unpaired reference domain; diffusion denoising regularization helps improve robustness under dark area noise perturbations; and the semantic feature modulation module helps achieve local adaptive illumination adjustment. The ablation results show that the complete method achieves PSNR 18.8538, SSIM 0.7926, NIQE 3.4270, and MUSIQ 62.4890, respectively, outperforming the variants after removing each core module.

[0119] Table 3: Ablation Experiment Results of the Core Module of the Invention

[0120]

[0121] Figure 5 A representative face detection example on the DARK FACE dataset is shown, demonstrating the auxiliary effect of the enhanced results of this invention on visual perception tasks in low-light environments. Figure 5 As can be seen, the image enhanced by this invention reveals more dark facial areas, making it easier for a fixed face detector to identify small-scale or low-light facial targets in low-light environments. Simultaneously, this invention avoids severe over-enhancement and unnatural artifacts, reducing the misleading effect of the enhancement process on the detector. This demonstrates that this invention can serve as an effective preprocessing module for visual perception systems in low-light environments.

[0122] In conclusion, Figure 1 and Figure 2 This invention explains the system structure and implementation methods of key modules. Figures 3 to 5 The visualization implementation effects of this invention in low-light enhancement, backlight enhancement, and downstream visual tasks are illustrated. Tables 1 to 3 verify the effectiveness of this invention from three aspects: main performance comparison, quality evaluation of real unpaired scenes, and core module ablation. The above results show that this invention can achieve low-light image enhancement with structure preservation, natural color, improved visibility in dark areas, and efficient inference under unpaired supervision.

[0123] Example 2:

[0124] The present invention also provides a low-light image enhancement system, including an image acquisition module, a semantic domain prior construction module, a content bias suppression module, a conditional vector generation module, a diffusion regularized illumination gain prediction module, a semantic feature modulation module, an enhanced image reconstruction module, an unsupervised training optimization module, and an inference output module.

[0125] The system comprises the following modules: an image acquisition module for acquiring low-light images, high-quality natural light images, and low-light images to be enhanced; a semantic domain prior construction module for extracting embedding features from the low-light image domain, high-quality natural light image domain, and the current input image using a frozen visual semantic image encoder, and calculating the natural light domain direction and the input semantic prior; a content bias suppression module for constructing a content subspace and suppressing content-related components in the natural light domain direction; a conditional vector generation module for mapping and concatenating the natural light domain prior and the input semantic prior into a semantic domain conditional vector; a diffusion-regularized illumination gain prediction module for performing noise scale-aware feature learning during the training phase and outputting the prediction results with added noise and an illumination gain map; a semantic feature modulation module for injecting semantic domain conditions into multi-scale decoding features; an enhanced image reconstruction module for generating enhanced images based on the illumination gain map and the original low-light image; an unsupervised training optimization module for updating network parameters based on multiple unsupervised losses; and an inference output module for performing a single forward enhancement during the inference phase and outputting the enhanced image.

[0126] Combination Figure 1 As shown in the overall framework, this invention first constructs a natural light domain orientation using the low-light image domain and the high-quality natural light image domain during the training phase, and reduces the interference of content difference between unpaired image sets through a content bias suppression mechanism; then, it extracts the input semantic prior from the low-light image to be enhanced, and forms a semantic domain conditional vector together with the natural light domain orientation; then, it inputs the low-light image representation, diffusion time step embedding, and semantic domain conditional vector into the conditional illumination gain prediction network; finally, it obtains a single-channel illumination gain map through the illumination gain prediction head, and obtains the enhanced image through a Retinex-inspired reconstruction method. Figure 1 The relationship between diffusion denoising regularization, illumination smoothing constraint, semantic domain alignment constraint and inverse illumination consistency constraint during the training phase is also shown, illustrating that the present invention does not rely on a single loss term to enhance brightness, but rather achieves natural, stable and structure-preserving image enhancement through multiple constraints.

[0127] Combination Figure 2The semantic feature modulation module shown in this invention achieves adaptive fusion between semantic domain conditions and spatial decoding features through cross-attention, spatial gating, and channel-level modulation. Specifically, cross-attention establishes the association between spatial features and conditionally related semantic terms; spatial gating controls the intensity of semantic condition injection at different locations; and channel-level scaling and bias modulation are used for global illumination calibration. This module avoids applying the same modulation to all regions based on global semantic conditions, enabling the network to achieve a better balance between dark area enhancement, bright area protection, and texture preservation.

[0128] The above embodiments are merely preferred embodiments of the present invention and do not constitute a limitation on the scope of protection of the present invention. Any equivalent substitutions, combinations, or improvements made to the visual semantic image encoder, text encoder, content lexical set, conditional network structure, semantic feature modulation method, diffuse noise scheduling method, illumination gain reconstruction method, loss function combination, inference time step, and system deployment method within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A low-light image enhancement method based on semantic domain prior and illumination gain prediction, characterized in that, Includes the following steps: S1. Obtain a set of low-light images, a set of high-quality natural light images, and an initial natural light domain migration direction to characterize the average feature change from the low-light image domain to the natural light image domain. Acquire the low-light image to be enhanced and extract the input semantic prior; S2. Construct a content subspace and remove the projection components related to the content subspace from the initial natural light domain migration direction to obtain the natural light domain direction after content bias suppression; generate a semantic domain conditional vector based on the natural light domain direction and the input semantic prior. S3. Construct a conditional illumination gain prediction network. During the training phase, perform forward diffusion perturbation on the low-light image to generate noise input. The Conditional Illumination Gain Prediction Network (CIBNR) is based on noise input and semantic domain conditional vectors, outputs shared features, and predicts noise prediction results and single-channel logarithmic illumination gain maps, respectively. The CIBNR includes an encoder, a decoder, skip connections, a semantic feature modulation module, a noise prediction head, and an illumination gain prediction head. The encoder is used to extract multi-scale features of low-light images or their feature representations. The decoder is used to recover spatial resolution and predict illumination gain maps; skip connections are used to preserve local structural information. The semantic feature modulation module is used to inject semantic domain conditional vectors into the decoding process; The noise prediction head is used for diffusion denoising regularization to generate noise prediction results; the illumination gain prediction head is used to output a single-channel logarithmic illumination gain map. S4. Set up a semantic feature modulation module in the decoding stage of the conditional illumination gain prediction network to obtain spatially adaptive semantic response features based on the semantic domain conditional vector. The Conditional Illumination Gain Prediction Network flattens the decoded features into a spatial feature sequence and inputs the semantic domain conditional vector into a lightweight multilayer perceptron to generate conditionally relevant semantic terms. Using the spatial feature sequence as the query and the conditionally relevant semantic terms as the key and value, a cross-attention operation is performed to obtain semantic response features with the same spatial resolution as the decoded features. S5. The single-channel logarithmic illumination gain map generates an enhanced image through exponential mapping and expansion to the image channel dimension; S6. Construct a loss function to jointly optimize network parameters; The low-light image to be enhanced is input into the optimized conditional illumination gain prediction network to obtain the enhanced image.

2. The low-light image enhancement method based on semantic domain prior and illumination gain prediction according to claim 1, characterized in that, S1 includes: After normalizing the size and pixels of the low-light training image set and the high-quality natural light image set, respectively, they are input into the frozen visual semantic image encoder to obtain the low-light image embedding set and the high-quality natural light image embedding set. The mean values ​​of the low-light image domain and the high-quality natural light image domain are calculated separately, and the difference between the two is used as the initial natural light domain migration direction to provide the global natural light enhancement direction. Perform perceptual brightness preprocessing on the low-light image to obtain a preprocessed image; input the preprocessed image into the frozen visual semantic image encoder to obtain the input semantic prior of the preprocessed image.

3. The low-light image enhancement method based on semantic domain prior and illumination gain prediction according to claim 1, characterized in that, The construction of the content subspace in S2 includes: Several content-related text terms are preset, and the content-related text terms are encoded using a frozen text encoder to obtain content-related text embeddings; the content-related text terms include terms used to describe scene content or object categories; the content-related text embeddings are stacked into a content subspace.

4. The low-light image enhancement method based on semantic domain prior and illumination gain prediction according to claim 3, characterized in that, In S2, generating the semantic domain condition vector includes: The natural light domain direction and the input semantic prior are respectively input into the learnable mapping layer. and learnable mapping layer The natural light domain conditional component and the input semantic conditional component are obtained; the natural light domain conditional component and the input semantic conditional component are concatenated to obtain the semantic domain conditional vector.

5. The low-light image enhancement method based on semantic domain prior and illumination gain prediction according to claim 1, characterized in that, The conditional illumination gain prediction network in S4 also includes a spatial gating branch; the spatial gating branch concatenates the decoded features with the semantic response features and generates a spatial gating map through convolutional layers and a sigmoid activation function, which is used to control the injection intensity of the semantic response features at different locations.

6. The low-light image enhancement method based on semantic domain prior and illumination gain prediction according to claim 1, characterized in that, S4 also includes: generating channel-level scaling parameters based on the semantic domain condition vector c. and bias parameters The decoded features are then subjected to channel modulation to generate the modulated output features, which are represented as follows: Here, ⊙ represents element-wise multiplication. For semantic response features, This is a spatial gating diagram.

7. The low-light image enhancement method based on semantic domain prior and illumination gain prediction according to claim 1, characterized in that, The loss function in S6 is expressed as: ， in, This is used to constrain the noise prediction results to be consistent with the actual noise, thereby improving the network's robustness to low-light noise disturbances. This is used to constrain the illumination gain map to change smoothly in flat areas, and to appropriately relax the constraints at the edges of the image; This is used to constrain the enhanced image to move along the natural light domain in the visual semantic feature space and to approach the high-quality natural light image domain; Used to suppress pixel saturation and irreversible information loss caused by excessive gain; , and The weights for each loss item.

8. A low-light image enhancement system based on semantic domain prior and illumination gain prediction, characterized in that, include: The image acquisition module is used to acquire a set of low-light images and a set of high-quality natural light images. A semantic domain prior construction module is used to obtain the initial natural light domain migration direction that represents the average feature change from the low-light image domain to the natural light image domain. Acquire the low-light image to be enhanced and extract the input semantic prior; The content bias suppression module is used to construct the content subspace and remove the projection components related to the content subspace from the initial natural light domain migration direction to obtain the natural light domain direction after content bias suppression. The conditional vector generation module is used to generate semantic domain conditional vectors based on the natural light domain direction and the input semantic prior. The diffusion-regularized illumination gain prediction module is used to construct the conditional illumination gain prediction network. During the training phase, it performs forward diffusion perturbation on the low-light image to generate noise input. The Conditional Illumination Gain Prediction Network (CIBNR) is based on noise input and semantic domain conditional vectors, outputs shared features, and predicts noise prediction results and single-channel logarithmic illumination gain maps, respectively. The CIBNR includes an encoder, a decoder, skip connections, a semantic feature modulation module, a noise prediction head, and an illumination gain prediction head. The encoder is used to extract multi-scale features of low-light images or their feature representations. The decoder is used to recover spatial resolution and predict illumination gain maps; skip connections are used to preserve local structural information. The semantic feature modulation module is used to inject semantic domain conditional vectors into the decoding process; The noise prediction head is used for diffusion denoising regularization to generate noise prediction results; the illumination gain prediction head is used to output a single-channel logarithmic illumination gain map. A semantic feature modulation module is included in the decoding stage of the conditional illumination gain prediction network to obtain spatially adaptive semantic response features based on the semantic domain conditional vector. The conditional illumination gain prediction network flattens the decoded features into a spatial feature sequence and inputs the semantic domain conditional vector into a lightweight multilayer perceptron to generate conditionally relevant semantic lexical units. Using the spatial feature sequence as the query and the conditionally relevant semantic lexical units as the key and value, a cross-attention operation is performed to obtain semantic response features with the same spatial resolution as the decoded features. The enhanced image reconstruction module is used to generate an enhanced image by exponentially mapping a single-channel logarithmic illumination gain map to the image channel dimension. The unsupervised training optimization module is used to construct loss functions to jointly optimize network parameters; The inference output module is used to input the low-light image to be enhanced into the optimized conditional illumination gain prediction network, thereby obtaining the enhanced image; The low-light image enhancement system based on semantic domain prior and illumination gain prediction is used to implement the low-light image enhancement method and steps based on semantic domain prior and illumination gain prediction as described in claim 1.

Images