Light-weight low-light image enhancement method based on channel constraint

Abstract

The invention relates to the field of computer vision and low-light image enhancement, in particular to a lightweight low-light image enhancement method based on channel constraint. In the field of light-weight low-light image enhancement, if the spatial dimension is excessively emphasized and the channel information is neglected, important channel features may be lost, and the calculation complexity is significantly increased due to the excessive number of channels. In addition, a certain linear relation is introduced by directly connecting the feature map, nonlinear transformation is not combined, and the expression ability of the model is limited. Therefore, a channel regularization method is introduced to adjust the importance of each channel. In order to deal with calculation burden caused by too many channels, the invention provides a channel confidence prediction network. And while the feature extraction is enhanced, the calculation overhead is also reduced. In order to overcome the limitation of linear feature graph connection, a secondary connection unit is introduced. Nonlinear features are combined, so that the model can capture more complex data modes and relationships.

Description

Technical Field

[0001] This application relates to the fields of computer vision and low-light image enhancement. Specifically, it relates to a lightweight low-light image enhancement method based on channel constraints. Background Art

[0002] When designing lightweight networks, the main concern is how to effectively reduce the number of parameters without sacrificing the model's representational ability. Overemphasizing spatial information while neglecting channel information may lead to a decline in model performance and accuracy. Feature map channels play a crucial role in feature extraction and information representation. Efficiently extracting channel information with minimal computational overhead remains a significant challenge. In addition, the simple linear connection strategy in feature map concatenation has limitations in fully leveraging the advantages of each branch. Although lightweight convolution methods can focus on channel information, they still lack the flexibility and dynamic adjustment capabilities required for effectively extracting useful information when dealing with complex tasks. Summary of the Invention

[0003] To solve the above technical problems, we propose a channel regularization method that can adjust the importance of each channel while maintaining a lightweight structure, thereby minimizing channel information loss. To further address the information loss problem, we propose a channel confidence predictor and a quadratic connection unit. Different from lightweight convolution, our channel confidence predictor is based on lightweight convolution, simulates a channel confidence estimator, focuses on channel information through a learnable parameter vector, and dynamically adjusts the input features with minimal number of parameters.

[0004] We used two low-light image datasets: the LOL-V1 dataset and the LOL-V2 dataset. The LOL-V1 dataset contains 500 pairs of images, of which 485 pairs are used for training and 15 pairs are used for testing. The LOL-V2 dataset includes 689 pairs of training images and 100 pairs of test images. We used the peak signal-to-noise ratio (PSNR) and the structural similarity index (SSIM) to evaluate the enhancement effect, as quantitative metrics for pixel accuracy and structural fidelity respectively. In addition, we also evaluated the lightweight characteristics of the model by measuring the number of model parameters and the test time.

[0005] The CRNet network was constructed and implemented using PyTorch. During the training process, we used the Adam optimizer to update the network weights. The training configuration included a batch size of 32, a learning rate of 0.0001, a weight decay factor of 0.0001, and a total of 30,000 training iterations were performed.

[0006] Our channel confidence predictor combines lightweight convolution with a parameter vector and dynamically adjusts the input features through backpropagation. This method enhances the model's representational ability and significantly improves the learning efficiency and effect.

[0007] Initialization process of parameters: The initial value of the parameter vector is a random number drawn from a normal distribution.

[0008] p 0 = [p 1 , p 2 , …, p n , p i ~ N(0, σ 2 )

[0009] Broadcast mechanism: This mechanism extends the parameter vector to a shape that matches the input feature map, where represents the extension operation, extending p to the same height H and width W as the feature map

[0010]

[0011] Concatenation process of the extended parameter vector and the convolutional feature map:

[0012] Xcat,t = [Xconv, Pt]

[0013] Dynamic adjustment process: The parameter vector adjusts the features through backpropagation, where η is the learning rate and L is the loss function.

[0014] Description of the drawings

[0015] Figure 1 CRNet network framework

[0016] Figure 2 Channel confidence predictor architecture Specific implementation manners

[0017] Figure 1 Shows the architecture of our proposed method, mainly including two modules: the global feature extraction (GFE) module and the local network enhancement (LNE) module. The GFE module processes the V channel from the HSV color space and adjusts the input by using the average luminance value and high-order curves. The generated global features are then input into the LNE module. The LNE module integrates the low-light image, the V-channel image, and the output from the GFE module, and captures and enhances local information through multiple convolutional layers and feature fusion techniques. In the LNE module, we introduce a channel confidence predictor that combines lightweight convolution with a parameter vector. This predictor dynamically adjusts the input features through the backpropagation of the parameter vector. In addition, we also introduce a quadratic connection unit that enhances the output by combining the linear and quadratic relationships of the input. This method can capture more complex data patterns and relationships.

[0018] Figure 2 Shows the structure of the Channel Confidence Predictor (CCP). Before being processed by the CCP module, the model mainly focuses on spatial information. However, after being processed by the CCP module, channel information is effectively supplemented. Although the number of parameters of this module is small, it effectively simulates a channel confidence estimator, thereby improving the efficiency of feature extraction during the splicing process and dynamically adjusting the input features. Therefore, our CCP combines ConvRep 1×1 with the "parameter vector" to dynamically adjust the input features through backpropagation. This method enhances the expressive ability of the model and significantly improves the learning efficiency and effect.

Claims

1. A lightweight low-light image enhancement method based on channel constraints. We propose the CRNet network, a fast and lightweight low-light image enhancement network with only 0.02M model parameters. It mainly includes two modules: the global feature extraction module and the local network enhancement module. The global feature extraction module processes the V channel from the HSV color space and adjusts the input by the average brightness value and the high-order curve. The processed global features are then passed to the local network enhancement module. The local network enhancement module integrates the low-light image, the V channel image, and the output of the global feature extraction module. It uses multiple convolutional layers and feature fusion techniques to capture and enhance local information.

2. We propose a channel regularization method that adjusts the importance of each channel while maintaining a lightweight architecture, effectively reducing the loss of channel information. We evaluate our method on two low-light image datasets (LOL-V1 and LOL-V2). The LOL-V1 dataset contains 500 pairs of images, of which 485 pairs are used for training and 15 pairs are used for testing. The LOL-V2 dataset includes 689 pairs of training images and 100 pairs of test images.

3. We developed a channel confidence predictor (CCP). This method designs a convolution that simulates the channel confidence estimator and uses a learnable dynamically adjusted parameter vector to optimize the input features and focus on channel information. This improves the expressiveness of the model and improves learning efficiency and performance.

Images