A Retinex-based low-light image enhancement method

Abstract

This invention belongs to the field of image processing and relates to a Retinex-based low-light image enhancement method. It reveals degraded content in the image through a degraded appearance restorer and performs Retinex decomposition using two branches consisting of convolution and a sigmoid activation function to estimate the reflectance and luminance components, respectively. After obtaining the estimated reflectance and luminance components, gamma correction is used to adjust the luminance component. This invention decouples the low-light image into a three-channel color map and a grayscale detail map to maintain consistency with the target image in color and detail representation. Furthermore, this application provides an unsupervised loss function to constrain the solution space of the Retinex decomposition, thereby improving adaptability in unknown and complex scenes. Extensive experiments demonstrate that this method outperforms state-of-the-art methods.

Description

Technical Field

[0001] This invention belongs to the field of image processing, and in particular to a low-light image enhancement method based on Retinex. Background Technology

[0002] Images captured in dim or dark environments are often plagued by blurred details and color distortion. This degradation not only affects the human visual experience but also limits the high-quality images required for subsequent multimedia computing and computer vision tasks such as semantic segmentation and object detection in nighttime scenes. To address this challenge, previous researchers have developed many deep learning algorithms based on Retinex theory to restore low-light images to their natural brightness. However, current methods often inevitably amplify hidden artifacts because Retinex theory does not account for the various uncertain degradation patterns that may exist in the dark areas of low-light images. These degradation patterns can be categorized into detail degradation and color degradation. Furthermore, learning-based methods typically rely on high-quality, normally lit images or handcrafted prior information to guide network training, but these processes are time-consuming and have limited adaptability to unknown scenes.

[0003] To enhance the contrast of low-light images, several traditional techniques were widely used in the past. One of the most widely used methods is based on Retinex theory. These methods decompose the image into two parts: reflection and illumination, representing the physical properties of the object and the illumination properties of its interaction with its surface, respectively. In subsequent developments, many studies have treated Retinex decomposition as an optimization task and incorporated complex image priors into their models. For example, Guo et al. [GuoX.LIME:Amethod for low-light image enhancement[C] / / Proceedings of the 24th ACM international conference on Multimedia.2016:87-91] used a dark channel prior for image enhancement and combined it with a sparse gradient prior for illumination. Li et al. [Li M, Liu J, Yang W, et al. Structure-revealing low-light image enhancement via robust retinex model[J]. IEEE Transactions on Image Processing, 2018, 27(6):2828-2841.] proposed a novel structure-revealing prior to enhance reflection quality while addressing robustness issues. Ren et al. [Ren X, Yang W, Cheng WH, et al. LR3M: Robust low-light enhancement via low-rank regularized retinex model[J]. IEEE Transactions on Image Processing, 2020, 29:5862-5876] introduced a noise suppression method that integrates low-rank priors into the retinex to enhance images and effectively eliminate noise. Although these methods achieve contrast enhancement and noise reduction, they encounter some limitations in real-world applications because they rely on carefully designed handcrafted priors.

[0004] In recent years, learning-based methods have achieved impressive results in low-light image enhancement. Wei et al. [Wei C, Wang W, Yang W, et al. Deep retinex decomposition for low-light enhancement[J]. arXiv preprint arXiv:1808.04560,2018.] introduced a decomposition method that incorporates structure-aware change constraints for end-to-end training. Zhang et al. [Zhang Y, Zhang J, Guo X. Kindling the darkness: A practical low-light image enhancer[C] / / Proceedings of the 27th ACM international conference on multimedia.2019:1632-1640.] designed a similar network that connects the illumination map and the reflection map during the decomposition stage. Wang et al. [Wang R, Zhang Q, Fu CW, et al. Underexposed photo enhancement using deep illumination estimation[C] / / Proceedings of the IEEE / CVF conference on computer vision and pattern recognition.2019:6849-6857.] proposed an image-to-light network based on a bidirectional learning framework. Fu et al. [Fu H, Zheng W, Meng X, et al. You do not need additional priors or regularizers inretinex-based low-light image enhancement[C] / / Proceedings of the IEEE / CVF conference on Computer Vision and Pattern Recognition.2023:18125-18134.] designed a contrastive learning method and a self-knowledge distillation method, enabling the network to extract illumination and reflection without additional prior knowledge.Ma et al. [Ma L, Ma T, Liu R, et al. Toward fast, flexible, and robust low-light image enhancement [C] / / Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. 2022:5637-5646.] introduced a self-calibrating illumination module that utilizes unsupervised learning to recover the brightness of underexposed areas in an image. Xu et al. [Xu X, Wang R, Lu J. Low-light image enhancement via structure modeling and guidance [C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2023:9893-9903.] designed a network that guides appearance improvement through structure-aware features and is trained using GAN loss.

[0005] In general, while learning-based methods are often superior to traditional methods, they often neglect the importance of explicitly modeling the degradation patterns of various underexposed areas, making the enhanced image prone to deviating from the original color and detail. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a novel low-light image enhancement method based on Retinex theory. This method decouples the low-light image into a three-channel color map and a grayscale detail map to maintain consistency with the target image in color and detail representation. Furthermore, this application provides an unsupervised loss function to constrain the solution space of the Retinex decomposition, thereby improving adaptability in unknown and complex scenes. Extensive experiments demonstrate that this method outperforms state-of-the-art methods.

[0007] The technical solution adopted by this invention to solve the technical problem is:

[0008] This invention provides a low-light image enhancement method based on Retinex theory, the steps of which are as follows:

[0009] S1: Revealing the degraded content in an image using a degraded appearance restorer, denoted as P = F restorer (S;θ r )

[0010] Where P represents the degraded content, which consists of color and detail features hidden in dark areas, and F... restorer (·) indicates a degraded appearance restorer, θ r For learnable parameters, updates are achieved by minimizing the following objective function:

[0011]

[0012] In the formula L o (·) is the objective function, S is the image, S=R·L, R is the reflectance component, L is the luminance component;

[0013] The degraded appearance restorer divides appearance features into a color representation map C and a detail representation map D, performs element-wise multiplication C·D to generate the CD feature map, and sets the loss function as follows:

[0014] L CD =ω0||C·DS||2

[0015] In the formula, ω0 is the loss trade-off parameter;

[0016] S2: Retinex decomposition is performed through two branches consisting of convolution and sigmoid activation functions to estimate the reflection component and the luminance component, respectively.

[0017] S3: After obtaining the estimated reflection and luminance components, gamma correction is used to adjust the luminance component.

[0018] Furthermore, the reflection component R is set as a three-channel tensor, while the luminance component L is set as a single-channel tensor, so that the estimated reflection component R is close to the desired reflection component. set up

[0019]

[0020] Introduce the multiplication term C·D for color and detail feature maps into the above formula:

[0021]

[0022] Furthermore, give The mean of the luminance component is used to estimate the reflection component R, which is defined as the adaptive reflection loss L. A :

[0023]

[0024] In the formula, C, H, and W are the luminance components L, H, and W, respectively. chw The length, width, and height are given, and ω1 is the trade-off parameter for this loss.

[0025] Furthermore, let the reflection consistency loss L be set. RThe loss function is expressed as:

[0026] L R =||R1-R2||2.

[0027] Furthermore, the brightness loss L is set. Illu , is represented as:

[0028]

[0029] In the formula, c∈{R,G,B} refers to the RGB channels. This refers to the horizontal gradient and the vertical gradient.

[0030] Furthermore, let's define the Retinex decomposition loss:

[0031] L Retinex＝ ||R·LC·D||2+||R-(C·D) / sg(L)||2.

[0032] Furthermore, the total loss function is defined as follows:

[0033] L = L CD +L A +L R +L Illu +L Retinex .

[0034] Furthermore, the enhanced image is represented as:

[0035]

[0036] Users can adjust the brightness of the enhanced image by changing the value of γ.

[0037] The advantages and positive effects of this invention are:

[0038] 1. This invention proposes a mechanism for modeling color and detail to assist the Retinex method in learning rich information hidden in dark areas. A set of unsupervised loss functions effectively constrains the solution space of the Retinex decomposition. Simultaneously, by mutually constraining the various components of the network, its adaptability in unknown and complex scenes is improved.

[0039] 2. This invention enables end-to-end training in an unsupervised learning manner, reducing reliance on additional manual priors and effectively suppressing color deviations in enhanced images while preserving inherent details. Attached Figure Description

[0040] Figure 1 This is a model architecture diagram of the method of the present invention;

[0041] Figure 2Architecture diagram of the degraded appearance restorer model;

[0042] Figure 3 Examples of recovery of some degraded features;

[0043] Figure 4 This is a comparison chart showing the enhancement effect of the method in this application with other state-of-the-art methods. Detailed Implementation

[0044] The present invention will be further described in detail below through specific embodiments. The following embodiments are merely descriptive and not limiting, and should not be used to limit the scope of protection of the present invention.

[0045] Based on physically interpretable Retinex theory, a given image S can be decomposed into two parts:

[0046] S = R·L

[0047] In the formula, · represents element-wise multiplication, and R and L represent the corresponding reflectance and luminance components, respectively. Reflectance describes an inherent property of an image, while illumination reflects the brightness of an object and is affected by lighting conditions. Retinex-based methods are used to enhance low-light images, aiming to estimate these two components from the input image. This problem is challenging because there are no corresponding references for the reflectance and luminance components of real-world images. To address this, researchers typically introduce additional handcrafted priors or regularization terms to train their decomposition networks. This training process can be summarized as follows:

[0048]

[0049] In the formula, ||SR·L||2 guarantees the possibility of reconstructing the image using the decomposed reflection and luminance components, f(·) and g(·) are used to characterize the regularization terms of the reflection and luminance priors, respectively, and α and β represent the weights.

[0050] However, designing these priors or regularizers is a challenging task due to the diversity of natural scenes and lighting conditions. Furthermore, excessive constraints can lead to suboptimal solutions in joint optimization, balancing mismatched outcomes. Therefore, these solutions often lack adaptability and efficiency.

[0051] To find adaptive priors applicable to various complex conditions in a data-driven manner, this application uses paired low-light images to train the model, and its objective function can be expressed as:

[0052]

[0053] In the formula, S1 and S2 are paired low-light images, R1, R2, L1 and L2 are the reflection and luminance components of the paired low-light images, respectively, and ||R1-R2||2 indicates that the reflection components of the same scene are consistent.

[0054] Furthermore, Retinex theory assumes that the observed image is under ideal conditions, such as uniform illumination throughout the scene. However, in low-light conditions, various uncertain degradation modes exist. Ignoring the influence of these modes will lead to inaccurate estimates of reflectance and illuminance. Therefore, this invention proposes to recover the degradation content in low-light images before applying Retinex decomposition. This process can be expressed as:

[0055] P = F restorer (S;θ r )

[0056] Where P represents the degraded content, namely the color and detail features hidden in the dark areas. F restorer (·) indicates a degraded appearance restorer, θ r For learnable parameters, updates are achieved by minimizing the following objective function:

[0057]

[0058] In the formula L o (·) is the objective function of the low-light image enhancement solution of this application.

[0059] The overall structure diagram of the neural network of this invention is as follows: Figure 1 As shown, it consists of three stages.

[0060] Stage I: Revealing degraded content in the image through a degraded appearance restorer. This stage focuses on learning appearance features and does not enhance image brightness. These appearance features are divided into color representation maps (C) and detail representation maps (D). To maintain consistency between the features and the given input, this application performs element-wise multiplication C·D to generate CD feature maps. The loss function required for this process is defined as follows:

[0061] L CD =ω0||C·DS||2

[0062] In the formula, ω0 is the loss trade-off parameter, and in this application, ω0 = 500 is set.

[0063] Table 1 and Figure 3 The best results for each corresponding indicator are indicated in bold.

[0064]

[0065] Figure 3The paper presents some examples of degradation feature recovery, and provides corresponding quantitative comparisons in Table 1. Figure 3 The first column is the input, the second is the color representation map, and the third is the detail representation map. The gray and green boxes are the parts with the most obvious differences. It can be observed that they represent, to some extent, the physical properties and brightness information of the observed object. Although the color representation map C represents the inherent color in the dark areas, it does not depict the object's outline as clearly as the reflectance component. Conversely, the detail representation map D focuses on detail. Therefore, there are slight errors between C and D compared to the expected R and L. Ignoring these errors to perform enhancement may result in unpleasant visual effects. This phenomenon can be explained by introducing an error term:

[0066]

[0067] In the formula, ε1 and ε2 are the error terms for the estimated R and L, respectively.

[0068] Phase II: Retinex decomposition is performed through two branches consisting of convolution and a sigmoid activation function to estimate the reflectance and luminance components, respectively. Based on Retinex theory, this application sets the reflectance component R as a three-channel tensor and the luminance component L as a single-channel tensor. This principle guides the recovery of low-light images in RGB space, where the reflectance map is depicted using RGB channels, and all channels share the same grayscale illumination map. Since Retinex decomposition is a highly ill-conditioned problem, some constraints are needed to reduce the solution space of Retinex decomposition. Generally, an ideal reflectance component can effectively mitigate the luminance variations across the entire image, providing noise-free and natural image content. To make the estimated reflectance component R close to the desired reflectance component... This application defines the following optimization problem:

[0069]

[0070] However, this objective function cannot be directly used to constrain Retinex decomposition because the reflection components do not have corresponding reference images. To address this issue, this application introduces a multiplication term C·D for color and detail feature maps and rephrases the problem from the perspective of error redistribution:

[0071]

[0072] Furthermore, it can be observed that many low-light image datasets contain a wide variety of brightness characteristics, making it unreasonable to impose the same constraints on these images during training. Therefore, this application provides... The process of estimating the reflection component R by applying the mean of the luminance component can be defined as follows:

[0073]

[0074] In the formula, C, H, and W are the luminance components L and H, respectively. chw The length, width, and height are given, and ω1 is the trade-off parameter for this loss. In this application, ω1 is set to 0.1.

[0075] During training, this application uses paired low-light images as input and leverages the property that the reflection component should remain consistent under different brightness levels to fully utilize the data line delay. The loss function is expressed as:

[0076] L R =||R1-R2||2

[0077] To estimate the luminance component, this application introduces a luminance loss L. Illu This loss ensures that the estimated luminance components are smooth between pixels, and this loss can be expressed as:

[0078]

[0079] In the formula, c∈{R,G,B} refers to the RGB channels. This refers to the horizontal gradient and the vertical gradient.

[0080] To ensure that the estimated reflection and luminance components do not deviate from the input image, this application designs the following Retinex decomposition loss:

[0081] L Retinex =||R·LC·D||2+||R-(C·D) / sg(L)||2

[0082] The total loss function of this invention is defined as:

[0083] L = L CD +L A +L R +L Illu +L Retinex

[0084] Phase III: After obtaining the estimated reflection and luminance components, this application employs gamma correction to adjust the luminance component. The enhanced image can be represented as:

[0085]

[0086] Users can adjust the brightness of the enhanced image by changing the value of γ.

[0087] Figure 4This image shows a comparison of the enhancement effects of the method in this application with other state-of-the-art methods. As can be observed, RUAS struggles to restore low-light images to their natural brightness, PairLIE and RetinexNet are prone to color deviation and brightness artifacts, SNR-Aware enhancement results in blurred details, and Retinexformer exhibits significant noise. Thanks to the degradation modeling mechanism and novel unsupervised learning method proposed in this invention, the model in this application enables the enhanced image to possess the most natural brightness and the clearest details, while exhibiting almost no color deviation.

[0088] This invention can be applied to target detection, autonomous driving, and fire safety in low-light scenarios. The algorithm first enhances the low-light scene, then uses a trained detector for detection. This method effectively improves the accuracy and robustness of machine vision tasks in low-light environments.

[0089] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several modifications and improvements can be made without departing from the inventive concept, and these all fall within the protection scope of the present invention.

Claims

1. A Retinex-based low-light image enhancement method, characterized in that, The steps are as follows: S1 : revealing the degraded content in the image by the degradation appearance restorer, denoted as in This represents degraded content, including color and detail features hidden in dark areas. Indicates a degraded appearance restorer. For learnable parameters, updates are achieved by minimizing the following objective function: In the formula Let S be the objective function, and S be the image. , R For the reflection component, L For the luminance component; The described degraded appearance restorer divides appearance features into a color representation map C and a detail representation map D, and performs element-wise multiplication. Generate CD feature maps and set the loss function as follows: In the formula It is a trade-off parameter for the loss; S2: Retinex decomposition is performed through two branches consisting of convolution and sigmoid activation functions to estimate the reflection component and the luminance component, respectively. S3: After obtaining the estimated reflection and luminance components, gamma correction is used to adjust the luminance component; The reflected component Set as a three-channel tensor, with the brightness component... Set as a single-channel tensor, so that the estimated reflection components Close to the desired reflection component ,set up Introducing multiplication terms for color and detail feature maps into the above formula. : 。 2. The low-light image enhancement method based on Retinex according to claim 1, characterized in that, Give The mean of the luminance component is used to estimate the reflection component. Defined as adaptive reflection loss L A : In the formula, These are the luminance components. Length, width and height, This is the trade-off parameter for the loss.

3. The low-light image enhancement method based on Retinex according to claim 2, characterized in that, Setting reflection consistency loss L R The loss function is expressed as: 。 4. The low-light image enhancement method based on Retinex according to claim 3, characterized in that, Set brightness loss , is represented as: In the formula, This refers to the RGB channel. This refers to the horizontal gradient and the vertical gradient.

5. The low-light image enhancement method based on Retinex according to claim 4, characterized in that, Set the Retinex decomposition loss: 。 6. The low-light image enhancement method based on Retinex according to claim 5, characterized in that, The total loss function is defined as follows: 。 7. The low-light image enhancement method based on Retinex according to claim 6, characterized in that, The enhanced image is represented as follows: Users adjust The value is used to change the brightness of the enhanced image.

Images