Low-light image enhancement method, system and device

Abstract

The invention discloses a low-light image enhancement method, system and device, and relates to the technical field of low-light enhancement, and the method comprises the steps: collecting a to-be-enhanced low-light image; the spatial dimension of the low-light image is compressed step by step through down-sampling operation, the number of channels is increased, and multi-scale features are extracted; gradually recovering the spatial resolution of the low-light image by adopting up-sampling operation, and fusing the spatial resolution with the multi-scale features; adaptively adjusting the channel weight of the fused feature map through a channel attention mechanism and a CA attention mechanism, and generating a single-channel illumination component and a three-channel reflection component; multiplying the obtained illumination component by the reflection component to obtain an enhanced image; according to the method, the brightness of the image is effectively improved, and image details and color information are well recovered.

Description

Technical Field

[0001] This invention relates to the field of low-light enhancement technology, and specifically to a low-light image enhancement method, system, and apparatus. Background Technology

[0002] Many applications rely on high-quality images for good performance, such as object detection, instance segmentation, and action recognition. Unfortunately, acquired images under dim or suboptimal lighting conditions often suffer from multiple degradations, including extreme darkness, low contrast, and noise, which obscure details in the image. This undesirably impacts subsequent tasks. Significant efforts have been made over the past few decades to improve the quality of low-light images, and a wide variety of techniques have been proposed.

[0003] Traditional methods, such as histogram equalization, improve image contrast by adjusting the histogram distribution and redistributing pixel grayscale values. These methods are highly efficient at enhancing the global contrast of low-light images, but noise amplification and local over-enhancement are difficult to control effectively. Retinex-based methods, inspired by the human visual system, decompose the original image into illuminated and reflected components. The reflected component is considered to represent the essence of the object, while differences in the illuminated component affect the visual effect of the captured image. These methods can effectively simulate human color perception, but still require many prior assumptions and manually set parameters, making them lack universality. In recent years, CNN-based methods have made remarkable progress in various fields, including low-light image enhancement. These methods have overcome bottlenecks compared to traditional methods, achieving surprising results. However, these methods require low-light-normal-light image pairs as datasets to drive supervised learning of the network. However, defining what constitutes normal light is inherently an ill-posed problem. From an observer's perspective, everyone's perception of an ideal image is different, making it impossible to define a standard for finding ideal lighting conditions. Therefore, researchers have successively proposed unsupervised CNN methods. These methods get rid of the dependence on reference images, but because they lack the guidance of reference images, these methods often require some prior knowledge and manual parameter setting, resulting in poor network adaptability.

[0004] In summary, existing methods mostly rely on supervised training with "low-light-normal-light" image pairs. However, the definition of "normal light" is subjective and difficult to unify objectively, which makes these methods subject to the ill-defined labels and lack of realism in the dataset, making it difficult to generalize in real-world scenarios. Although there are unsupervised methods that attempt to get rid of the dependence on reference images, the lack of effective learning guidance often requires the introduction of complex constraints or priors, resulting in unstable network training and unbalanced enhancement effects. It is difficult to achieve a balance between brightness restoration, detail preservation, color naturalness and noise suppression, resulting in low enhancement effects for low-light images. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention proposes a low-light image enhancement method, system, and apparatus. By employing an unsupervised training framework that relies solely on paired low-light images, it eliminates the pathological dependence on "normal light" reference images, thereby resolving the problems existing in the prior art.

[0006] A low-light image enhancement method includes the following steps: Acquire low-light images to be enhanced; The low-light image to be enhanced is input into a pre-trained dual-branch enhancement network, which outputs a single-channel illumination component and a three-channel reflection component. The single-channel illumination component and the three-channel reflection component are multiplied to obtain the enhanced image. The dual-branch enhancement network is trained using low-light image pairs as the training dataset, specifically including the following steps: The L-Block module uses downsampling to progressively compress the spatial dimension of low-light image pairs and increase the number of channels to extract multi-scale features; upsampling is used to progressively restore the spatial resolution of low-light image pairs and fuse them with multi-scale features; a channel attention mechanism is used to adaptively adjust the channel weights of the fused feature map and output the illumination component of a single channel. The low-light image pairs are subjected to the same downsampling and upsampling operations as the L-Block module through the R-Block module. The channel weights of the fused feature map are adaptively adjusted through the CA attention mechanism to output the three-channel reflection components. Based on the illumination component and the reflection component, the calculation is performed using the Retinex loss and histogram. Figure 1 The total loss function is constructed by weighting the consistency loss and reconstruction loss, and the parameters of the dual-branch network are optimized iteratively through backpropagation using the total loss function to obtain the trained dual-branch augmented network.

[0007] Furthermore, the L-Block module is a three-layer U-Net structure that incorporates a channel attention mechanism. Its network structure includes three downsampling operations and three upsampling operations. Each downsampling operation is implemented through two 3×3 convolutional layers, a ReLU activation function, and a 2×2 max pooling layer. The upsampling operation is structurally symmetrical to the downsampling operation, and the result of each upsampling operation is fused with the feature map of the corresponding downsampling output. A 1×1 convolution is used to map the fused feature map to the illumination component output of one channel.

[0008] Furthermore, the R-Block module is a four-layer U-Net structure that incorporates the CA attention mechanism. Its network structure includes: four downsampling operations and four upsampling operations. Each downsampling operation is implemented through two 3×3 convolutional layers, a ReLU activation function, and a 2×2 max pooling layer. The four upsampling operations are symmetrical to the downsampling operations, and the result of each upsampling operation is fused with the feature map output by the corresponding downsampling operation. A 1×1 convolution is used to map the fused feature map to the reflection component output of three channels.

[0009] Furthermore, during the training of the dual-branch enhancement network, the Retinex loss is used to constrain the consistency of the reflection component, the consistency of the illumination component gradient changes, and the smoothness of the illumination component itself in low-light images with different exposures under the same scene; the histogram... Figure 1 The consistency loss is used to ensure that the maximum brightness channel of the enhanced low-light image is consistent with the histogram equalization result of the maximum brightness channel of the input low-light image; the reconstruction loss is used to ensure that the illumination component and the reflection component can reconstruct the original input image.

[0010] Furthermore, the joint loss function is expressed as: ; in, , and represent , and Weight values; Retinex loss Represented as: + ;in, and These represent the reflection components of two pairs of input image data. and The gradient representing the illumination component, Indicates the light component L gradient; histogram Figure 1 Induced loss is expressed as: Reconstruction losses are expressed as: ;in, Indicates the loss item. constraint.

[0011] The present invention also includes a low-light image enhancement system, comprising: The acquisition module is used to acquire low-light images to be enhanced. An enhancement module is used to input the low-light image to be enhanced into a pre-trained dual-branch enhancement network, and output a single-channel illumination component and a three-channel reflection component; the single-channel illumination component and the three-channel reflection component are multiplied to obtain the enhanced image; wherein, the dual-branch enhancement network is trained using low-light image pairs as the training dataset, specifically including: The illumination component calculation unit is used to progressively compress the spatial dimension of low-light image pairs and increase the number of channels through downsampling operations using the L-Block module to extract multi-scale features; it uses upsampling operations to progressively restore the spatial resolution of low-light image pairs and fuses them with multi-scale features; and it adaptively adjusts the channel weights of the fused feature map through a channel attention mechanism to output a single-channel illumination component. The reflection component calculation unit is used to perform the same downsampling and upsampling operations on the low-light image pairs as the L-Block module through the R-Block module, and adaptively adjust the channel weights of the fused feature map through the CA attention mechanism to output the three-channel reflection components. The optimization unit is used to calculate the Retinex loss and histogram based on the illumination and reflection components. Figure 1 The total loss function is constructed by weighting the consistency loss and reconstruction loss, and the parameters of the dual-branch network are optimized iteratively through backpropagation using the total loss function to obtain the trained dual-branch augmented network.

[0012] The present invention also includes a low-light image enhancement computer device, comprising: a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the low-light image enhancement method.

[0013] The present invention also includes a readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor, are used to perform the steps of the low-light image enhancement method.

[0014] This invention provides a low-light image enhancement method, which has the following beneficial effects: This invention designs a dual-branch UNet structure (LBlock and RBlock) to handle illumination and reflection components separately. This structure not only inherits the powerful multi-scale feature extraction and fusion capabilities of UNet, but also specifically introduces channel attention and CA attention mechanisms. This allows the network to automatically focus on key features and adaptively allocate weights to information at different scales and spatial locations, thereby achieving more accurate illumination estimation and richer detail and color restoration. Furthermore, it innovatively proposes a network framework that can be trained solely on paired low-light images and integrates three loss constraints, completely eliminating the pathological dependence of existing deep learning methods on difficult-to-define and highly subjective "normal light" reference images. This fundamentally solves the problems of difficult training data acquisition and inconsistent standards, greatly improving the practicality and generalization ability of the method in different real-world scenarios, thus enhancing network performance, effectively improving image brightness, and better restoring image details and color information. Attached Figure Description

[0015] Figure 1 This is a network architecture diagram of UALIE in an embodiment of the present invention; Figure 2 This is a comparative schematic diagram of experimental result 1 in an embodiment of the present invention; Figure 3 This is a comparative schematic diagram of experimental result 2 in the embodiment of the present invention; Figure 4 This is a schematic diagram of the ablation experiment results in an embodiment of the present invention. Detailed Implementation

[0016] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0017] Existing low-light image enhancement methods can be broadly classified into two categories: traditional methods and deep learning-based methods.

[0018] Histogram equalization methods: By readjusting the histogram distribution of an image, the distribution range is stretched to make the gray-level distribution of the image more uniform. For example, some scholars have developed a local HE algorithm that innovatively achieves a balance between preserving image brightness and enhancing contrast; some scholars have proposed a BBHE (Brightness Preservation Histogram Equalization) algorithm that preserves color information, which retains the color information of the input image while enhancing image details; and some scholars have proposed the NEIMHE (Nonlinear Exposure Intensity Modified Histogram Equalization) method, which divides the image into 354 sub-regions and uses a nonlinear weight correction strategy based on CDF (Cumulative Density Function of Histogram) to specifically achieve differentiated enhancement of underexposed / overexposed areas in non-uniform lighting scenes.

[0019] Retinex methods: The most classic Retinex method is the single-scale Retinex (SSR) method, which uses a Gaussian function in the wrap function to estimate the illumination component. Multi-scale Retinex (MSR) methods fuse different SSR outputs to obtain the final enhanced result. Subsequent low-light image enhancement methods have strengthened the adaptive operation on the illumination and reflection components. For example, some researchers adaptively calculate the weights of each single-scale Retinex enhancement result based on the characteristics of the input low-light image, and the final output is a weighted combination of the enhancement results. Some researchers have constructed a brightness filter for Retinex decomposition, enhancing image details while maintaining naturalness. These methods require many priors and constraints and have poor robustness. Some researchers have proposed a new Retinex model that incorporates a noise term and uses a new regularization term in the process of decomposing the image into illumination and reflection components, thus obtaining piecewise smooth illumination components and textured reflection components. Some researchers have proposed the MFMR method, which processes images in the HSV color space. This method obtains various feature maps through a series of processes, and then uses an adaptive multi-scale fusion strategy to unify these feature maps and reconstruct a high-quality image.

[0020] Deep learning-based methods: Many researchers have proposed low-light image enhancement methods based on deep neural networks. Most of these methods require normal-light images as supervisors. For example, LLNet (Low-Light Net) was the first to apply deep learning to low-light image enhancement. This method identifies features in low-light images by stacking autoencoders and adaptively increases image brightness. MSR-Net was the first to equate the MSR method to an end-to-end convolutional neural network. MBLLEN (Multi-Branch Low-Light Enhancement Network) decomposes the entire end-to-end network into feature extraction, enhancement, and fusion modules, effectively suppressing image noise and artifacts in low-light areas. KinD, a highly effective method, decouples the original image into illumination and reflection spaces, training the network with images under different exposure conditions; however, the enhanced images are too smooth, leading to a loss of detail and texture.

[0021] In recent years, deep learning-based low-light image enhancement methods have developed rapidly, giving rise to a series of new algorithms that combine physical models, frequency domain information, and efficient network structures. One scholar proposed URetinex-Net, which combines Retinex theory with a deep unfolding strategy. By constructing an end-to-end network, it effectively separates the illumination and reflection components of an image, significantly improving image brightness and detail recovery while maintaining color realism. Subsequently, researchers further introduced the Transformer architecture into RetinexFormer, constructing a single-stage Retinex network through an illumination-guided attention mechanism. This network performed excellently on multiple standard datasets, representing a cutting-edge direction in combining Retinex theory with visual Transformers. Meanwhile, another scholar proposed the FourLLIE method, which innovatively introduces Fourier frequency information. By adjusting the frequency domain amplitude and fusing spatial domain features, it effectively improves the structural clarity and texture preservation of images, demonstrating the potential of frequency domain modeling in low-light enhancement. Furthermore, some scholars have proposed the FLOL method, focusing on efficiency in practical applications. They have proposed a series of fast baseline models that balance enhancement quality and inference speed, suitable for mobile and real-time systems. Other scholars have used Low-Light Pair Learning, employing paired low-light images for supervised training, to propose a simple yet stable enhancer that effectively improves model transferability and generalization. In recent years, attention mechanisms have been widely introduced into image processing tasks, significantly enhancing the model's ability to focus on key features. Channel attention, in particular, effectively enhances feature representation by modeling the importance of each channel (CVPR 2018). Subsequently, the Convolutional Block Attention Module (CBAM) proposed by some scholars combines channel attention and spatial attention, enhancing the model's attention capabilities in spatial and semantic dimensions through a concatenated approach (ECCV 2018). To further improve efficiency, some scholars designed an efficient channel attention module (ECA) in ECA-Net that does not require fully connected layers, using one-dimensional convolution to capture local interactions, balancing performance and computational cost (CVPR 2020). In addition, the Coordinate Attention (CA) module proposed by some scholars integrates location information encoding into channel attention, achieving more refined spatial positioning and cross-channel dependency modeling, which is especially suitable for mobile device scenarios (CVPR 2021).These attention mechanisms provide powerful feature extraction and modeling tools for subsequent image enhancement models, demonstrating broad application prospects and potential in visual tasks such as low-light image enhancement. Due to its excellent feature extraction capabilities, the U-Net architecture is widely used in semantic segmentation, performance segmentation, and other fields.

[0022] This invention proposes a low-light image enhancement method (UALIE), which specifically includes the following steps: S1. Collect image datasets. Data used for network training and validation can be collected from existing datasets, including but not limited to the SICE (Single Image Challenge for Enhancement) dataset and the LOL (LOW-Light) dataset; after constructing paired datasets, input the paired images into the network simultaneously during training.

[0023] S2. Construct a dual-branch network structure containing L-Block and R-Block, where L-Block is used to estimate the illumination component of the image and R-Block is used to estimate the reflection component of the image.

[0024] Retinex theory states that an image is composed of both illumination and reflection components, and can be represented as: ; in, For the observed image, For the reflection component of the image, This refers to the illumination component. The color of an object is determined by its reflection component; that is, the object's color is not affected by the non-uniformity of illumination. Darker or uneven lighting will affect the displayed quality of the captured image. These are the coordinates of the pixels.

[0025] Therefore, the method proposed in this invention includes two models: L-Block and R-Block, such as Figure 1 As shown: L-Block is used to obtain the lighting part. It is a 3-layer U-Net structure that incorporates the SENET attention mechanism.

[0026] The network structure includes 3 downsampling operations and 3 upsampling operations: During the downsampling process, each layer undergoes two repeated 3x3 convolutions, a ReLU function, and a max pooling operation with a stride of 2 and a size of 2x2, which reduces the image size by half and increases the number of channels by half.

[0027] During the upsampling process, it gradually recovers the detailed information of each layer and increases the spatial dimension of the feature map.

[0028] The upsampling operation in this process is symmetrical to the downsampling operation. Finally, a 1x1 convolutional kernel is used to map the 64-dimensional feature vector to one channel, obtaining the estimated illumination portion. During the upsampling stage, the result of each upsampling step is fused with the feature map obtained from the symmetrical downsampling. Since each downsampling step reduces the image size, each downsampling stage yields feature information at different scales. Gradually fusing these feature information during the upsampling stage allows for the recovery of original image information at different scales, which is crucial for restoring image details and colors.

[0029] R-Block is used to obtain the reflection part. It is a 4-layer U-Net structure that incorporates the CA attention mechanism.

[0030] The network structure is similar to L-Block, but with a few differences. First, the final output is an RGB color image, so the last layer uses a 1*1 convolution to map to 3 channels. Second, the reflection part contains the image's details, colors, and other original information, thus requiring richer feature information to help restore it. Therefore, this network structure adds one more layer compared to L-Block to obtain richer image feature information.

[0031] Since the illumination part has only one channel, it is only affected by the grayscale values ​​of different channels in the original image. Therefore, this invention applies a channel attention mechanism in the upsampling fusion stage, enabling the network to more effectively focus on valid feature information. Because the reflection part has three channels, it requires feature information not only in the channel dimension but also in the spatial dimension; therefore, a CA attention mechanism is added in its upsampling feature fusion stage.

[0032] S3. Using a training dataset containing only low-light image pairs, the network is trained unsupervised using a combination of designed loss functions, including Retinex loss and histogram loss. Figure 1 Consistency loss and reconstruction loss. During training, the network learns all parameters autonomously, driven by the training dataset and constrained by the loss function.

[0033] According to Retinex theory, the reflective component R represents the physical essence of a target, and this essence remains unchanged regardless of variations in the illumination component L. Therefore, images captured of the same target, regardless of their visual differences, should have a consistent R. As for the illumination component L, since the target is the same object being captured, its reflective properties are constant; therefore, the distribution of L's variation should be consistent regardless of changes in exposure. Furthermore, R should contain rich texture and detail, while L should be smooth.

[0034] The network training in this invention is based on image pairs with different exposure levels, and a loss function is designed accordingly. Its definition is as follows: + ; in, and These represent the reflection components of two paired inputs. and This represents the gradient of the illumination portion. Both terms use the mean squared error to measure the difference. The first term of the formula measures the consistency of the reflection portion. The second term measures the consistency of the illumination variation. The third term... This indicates the illuminated portion. L The gradient, because L It is smooth, so its gradient should be as small as possible.

[0035] Histogram Maximum Entropy Constraint: Because the network training in this invention uses instance pairs with different exposure levels, there are no reference bright images for the network, which can easily lead to insufficient brightness in the enhanced image. Some scholars have proposed using the histogram pattern of low-light images to constrain the network, suggesting that the largest channel in the reflective part should match the largest channel in the low-light image, and its entropy should be the largest in the model. Inspired by this, this invention adds a loss function to the network, which means that the channel with the largest brightness in the reflective part should be consistent with the channel with the largest brightness in the original low-light image, and have the maximum entropy. The formula is as follows:

[0036] ; here, F (·) represents histogram equalization. This can be understood as the histogram equalization result of the largest channel in the enhanced image and the largest channel in the low-light image should be consistent; that is, the difference should be as small as possible. Other constraints for reconstruction include a brightness adjustment term.

[0037] The network design of this invention is based on Retinex, which decomposes the original image into an illumination part L and a reflection part R, and finally reconstructs the enhanced image. Therefore, this invention introduces some Retinex reconstruction constraints, as shown below:

[0038] ; To ensure that the decomposed components can reconstruct the original image, a loss term is introduced. To ensure a reasonable decomposition, among which, This represents element-wise multiplication. Once the illumination estimation is complete, the reflected portion can be obtained by dividing the original image by the illumination map pixel by pixel. Based on this, the present invention adds constraints. This is to further guide the decomposition process.

[0039] The final loss function used for network training comprises the above three parts, as shown in the following formula: ; in, , and These are the three LOSSes mentioned above. , and This represents the weight value of each Loss.

[0040] S4. After the network completes training, inference testing can be performed. Input a single image into the trained network, and obtain the corrected illumination component and reflection component through the trained R-Block and L-Block respectively. Then multiply them to obtain the enhanced image.

[0041] This invention leverages the powerful feature extraction capabilities of the U-Net network to help it fully learn various feature information. Furthermore, based on the feature information requirements of different modules, a suitable attention mechanism is incorporated to enable more accurate weight allocation. The network designed in this invention can complete effective training using only paired low-light images. The L-Block designed in this invention can accurately extract the illumination components of the original image. The R-Block designed in this invention can accurately restore the reflection components of the original image, with natural color reproduction. By enhancing network performance, this invention effectively improves image brightness and better restores image details and color information.

[0042] Experimental test: All experiments and tests in this invention are based on the following hardware system: CPU is an AMD R7-5800H processor, system RAM is 8*2G, 512G SSD hard drive, GPU is NVIDIA GEFORCE1650, and operating system is Windows 10.

[0043] The input image was randomly cropped to 128×128×3 to ensure consistency of the input data and enhance the model's generalization ability. The batch size was set to 1. The optimizer ADAM was selected to optimize the network, with an initial learning rate of 1×10⁻⁴. β 1 =0.9, β 2 =0.999 and ε =10 −8 This invention employs a dynamic learning rate decay strategy, where the learning rate is halved every 100 epochs during training. Weights of the loss function , and Set them to 1, 0.1, and 0.1 respectively.

[0044] Datasets and Metrics: The datasets primarily used were the LOL dataset and the SICE (Single Image Challenge for Enhancement) dataset. These datasets contain the same images under various exposure conditions, and image alignment has been ensured through a series of scientific techniques. Each instance selected images with different exposures, totaling 300 image pairs. Furthermore, the test set from the LOL dataset was selected as the validation set for evaluating network performance.

[0045] Testing Metrics: To comprehensively evaluate the performance of different algorithms, multiple metrics were selected to measure the comparative results of the proposed method and various networks from multiple perspectives. PSNR (Peak Signal-to-Noise Ratio) was chosen to measure pixel-level error, SSIM (Structural Similarity Index) to measure structure preservation ability, LPIPS (Perceptual Similarity Perspective) to calculate visually perceived similarity, and NIQE (Natural Image Quality Evaluator) to measure the naturalness of the image. Higher PSNR and SSIM values ​​indicate that the result is closer to the original image, while lower LPIPS and NIQE indicate a more natural visual effect.

[0046] Comparison Methods: To comprehensively analyze network performance, a substantial number of methods were selected for comparison. These methods include traditional approaches, but more predominantly deep learning methods, including MBLLEN (Multi-scale Low-light Enhancement Network), RetinexNet (Retina Network), KIND (Brightening Dark Areas), Zero-DCE (Referenceless Depth Curve Estimation), URetinex-Net (Retina-based Depth Unfolding Network), and PirLIE (Paired Low-light Image Enhancement Network). All methods utilized the official code, and parameters were configured according to recommended settings.

[0047] Quantitative indicator comparison: This invention uses the official evaluation set of the LOL dataset for experiments; Table 1 shows the comparison results of the method of this invention with 6 other methods, with the three best-performing methods highlighted in bold.

[0048] Table 1 Comparison of Quantitative Indicators for Low-Light Image Enhancement It can be observed that the method of this invention ranks among the top three in all comparative metrics. Although it did not rank first in all metrics, the comprehensive comparison demonstrates the superiority of the method of this invention. Some of the compared methods used SSIM as a constraint in their network training (KinD, UREtinex-Net, MBLLEN), while others used visual perception constraints (MBLLEN). This makes their comparisons in these metrics somewhat unfair. This further illustrates the superior competitiveness of the method of this invention.

[0049] Visual effect comparison: Figure 2 and Figure 3 This is a comparison of different low-light enhancement methods. It can be seen that all methods enhance the brightness of low-light images, but the color and detail reproduction varies. The RetinexNet method... Figure 2 A lot of noise was generated. The MBLLEN method excels in both detail and color, as can be seen from the performance comparison, but its enhancement results produce a "smearing" effect, blurring the details in the original image. The KinD method performs exceptionally well, but... Figure 3 The colors are somewhat dull. The enhancement results from the Zero-DCE method are all somewhat dark. The URetinex-Net method performs excellently, but... Figure 3 The color enhancement is slightly insufficient. While the PIRLIE method achieves good brightness and color, the enhancement result lacks naturalness. In contrast, the method of this invention offers a more balanced visual effect, demonstrating good and balanced performance in terms of brightness, contrast, color, and naturalness.

[0050] Ablation experiments: Ablation experiments use different settings to understand how each component affects network performance. Setting 1 removes the attention mechanism from the network structure. Setting 2 builds upon Setting 1 by further removing the U-Net network structure, using only a simple 6-layer convolutional neural network. These ablation experiments allow for a direct comparison of the impact of different components on network performance. The results of the ablation experiments are as follows: Figure 4 As shown, setting 1, without the attention mechanism, produces a significantly darker output with some loss of detail and color. This is because, without the attention mechanism, the network is trained using pairs of low-light instances, and the objects being learned are inherently dark, so the network's weights on feature information are not adjusted according to their importance. Setting 2 produces an unnatural visual effect because the lack of the U-Net network structure prevents the effective extraction of features from the original image. The comparison of metrics in Table 2 further confirms this conclusion.

[0051] Table 2 Quantitative Comparison Results of Ablation Studies This invention proposes a novel low-light enhancement method, UALIE. Conventional deep learning low-light enhancement methods require training with low-light-normal-light instance pairs. This invention proposes a method that can be trained using only low-light instance pairs. The network design leverages the powerful feature extraction capabilities of the U-Net network, helping it to fully learn various feature information. Furthermore, appropriate attention mechanisms are incorporated based on the feature information requirements of different modules, enabling more accurate weight allocation. Extensive experiments demonstrate that UALIE exhibits excellent enhancement performance and has further development potential.

[0052] The main contributions of this invention are as follows: The proposed UALIE does not rely on ideal images; it only requires low-light instances for training. It leverages the advantages of the attention mechanism, enabling the network to automatically allocate weights based on feature information. It fully utilizes the inherent information of low-light images for training and incorporates histogram information into the constraints. Extensive experimental results demonstrate that this method effectively improves image contrast, restores image details and colors, and requires no prior knowledge or a large number of manually manipulated parameters.

[0053] Based on the same inventive concept, this invention also proposes a low-light image enhancement system, comprising: The acquisition module is used to acquire low-light images that need to be enhanced.

[0054] The enhancement module takes the low-light image to be enhanced as input to a pre-trained dual-branch enhancement network and outputs a single-channel illumination component and a three-channel reflection component. It then multiplies the single-channel illumination component with the three-channel reflection component to obtain the enhanced image. The dual-branch enhancement network is trained using low-light image pairs as the training dataset, specifically including: The illumination component calculation unit is used to progressively compress the spatial dimension of low-light image pairs and increase the number of channels through the L-Block module using downsampling operations to extract multi-scale features; it uses upsampling operations to progressively restore the spatial resolution of low-light image pairs and fuses them with multi-scale features; and it uses a channel attention mechanism to adaptively adjust the channel weights of the fused feature map to output a single-channel illumination component.

[0055] The reflection component calculation unit is used to perform the same downsampling and upsampling operations on the low-light image pairs as the L-Block module through the R-Block module, and adaptively adjust the channel weights of the fused feature map through the CA attention mechanism to output the three-channel reflection components.

[0056] The optimization unit is used to calculate the Retinex loss and histogram based on the illumination and reflection components. Figure 1The total loss function is composed of the consistency loss and reconstruction loss weights. The parameters of the dual-branch network are optimized iteratively through backpropagation using the total loss function to obtain the trained dual-branch augmented network.

[0057] The present invention also proposes a low-light image enhancement computer device, comprising: a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the low-light image enhancement method.

[0058] The present invention also proposes a readable storage medium storing a computer program, the computer program including program instructions, which, when executed by a processor, are used to perform steps of a low-light image enhancement method.

[0059] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A low-light image enhancement method, characterized by, Includes the following steps: Acquire low-light images to be enhanced; The low-light image to be enhanced is input into a pre-trained dual-branch enhancement network, which outputs a single-channel illumination component and a three-channel reflection component. The single-channel illumination component is multiplied by the three-channel reflection component to obtain the enhanced image. The dual-branch enhancement network is trained using low-light image pairs as the training dataset, specifically including the following steps: The L-Block module uses downsampling to progressively compress the spatial dimension of low-light image pairs and increase the number of channels to extract multi-scale features; upsampling is used to progressively restore the spatial resolution of low-light image pairs and fuse them with multi-scale features; a channel attention mechanism is used to adaptively adjust the channel weights of the fused feature map and output the illumination component of a single channel. The low-light image pairs are subjected to the same downsampling and upsampling operations as the L-Block module through the R-Block module. The channel weights of the fused feature map are adaptively adjusted through the CA attention mechanism to output the three-channel reflection components. Based on the illumination and reflection components, a total loss function is calculated, which is a weighted sum of Retinex loss, histogram consistency loss, and reconstruction loss. The parameters of the dual-branch network are then optimized iteratively using backpropagation based on the total loss function to obtain the trained dual-branch augmented network.

2. The low-light image enhancement method of claim 1, wherein, The L-Block module is a three-layer U-Net structure that incorporates a channel attention mechanism. Its network structure includes three downsampling operations and three upsampling operations. Each downsampling operation is implemented through two 3×3 convolutional layers, a ReLU activation function, and a 2×2 max pooling layer. The upsampling operation is structurally symmetrical to the downsampling operation, and the result of each upsampling operation is fused with the feature map of the corresponding downsampling output. A 1×1 convolution is used to map the fused feature map to the illumination component of one channel for output.

3. The low-light image enhancement method of claim 1, wherein, The R-Block module is a four-layer U-Net structure that incorporates the CA attention mechanism. Its network structure includes: four downsampling operations and four upsampling operations. Each downsampling operation is implemented through two 3×3 convolutional layers, a ReLU activation function, and a 2×2 max pooling layer. The four upsampling operations are symmetrical to the downsampling operations, and the result of each upsampling operation is fused with the feature map output by the corresponding downsampling operation. A 1×1 convolution is used to map the fused feature map to the reflection components of three channels.

4. The low-light image enhancement method of claim 1, wherein, During the training of the dual-branch enhancement network, the Retinex loss is used to constrain the consistency of the reflection component, the consistency of the gradient change of the illumination component, and the smoothness of the illumination component itself in low-light images with different exposures under the same scene. The histogram consistency loss is used to ensure that the histogram equalization result of the maximum brightness channel of the enhanced low-light image is consistent with that of the maximum brightness channel of the input low-light image; the reconstruction loss is used to ensure that the illumination component and the reflection component can reconstruct the original input image.

5. The low-light image enhancement method of claim 1, wherein, The joint loss function is expressed as: ； in, , and represent , and Weight values; Retinex loss Represented as: + ;in, and These represent the reflection components of two pairs of input image data. and The gradient representing the illumination component, Indicates the light component L The gradient; the histogram consistency loss, expressed as: Reconstruction losses are expressed as: ;in, Indicates the loss item. constraint.

6. A low-light image enhancement system, characterized in that, include: The acquisition module is used to acquire low-light images to be enhanced. The enhancement module is used to input the low-light image to be enhanced into a pre-trained dual-branch enhancement network and output a single-channel illumination component and a three-channel reflection component. The enhanced image is obtained by multiplying the single-channel illumination component with the three-channel reflection component; wherein, the dual-branch enhancement network is trained using low-light image pairs as the training dataset, specifically including: The illumination component calculation unit is used to progressively compress the spatial dimension of low-light image pairs and increase the number of channels through downsampling operations using the L-Block module to extract multi-scale features; it uses upsampling operations to progressively restore the spatial resolution of low-light image pairs and fuses them with multi-scale features; and it adaptively adjusts the channel weights of the fused feature map through a channel attention mechanism to output a single-channel illumination component. The reflection component calculation unit is used to perform the same downsampling and upsampling operations on the low-light image pairs as the L-Block module through the R-Block module, and adaptively adjust the channel weights of the fused feature map through the CA attention mechanism to output the three-channel reflection components. The optimization unit is used to calculate the total loss function, which is a weighted sum of Retinex loss, histogram consistency loss and reconstruction loss, based on the illumination component and the reflection component. The total loss function is then used to optimize and iterate the parameters of the dual-branch network through backpropagation to obtain the trained dual-branch augmented network.

7. A low-light image enhancement computer device, characterized in that, include: A memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the low-light image enhancement method according to any one of claims 1-5.

8. A readable storage medium, characterized in that, The readable storage medium stores a computer program, the computer program including program instructions, which, when executed by a processor, are used to perform the steps of the low-light image enhancement method according to any one of claims 1-5.

Images