Low-illumination image adaptive enhancement method and apparatus, device and storage medium

Abstract

Disclosed in the present invention are a low-illumination image adaptive enhancement method and apparatus, a device and a storage medium. The method comprises: inputting low-illumination images to be enhanced into a pre-trained low-illumination image adaptive enhancement model, so as to obtain enhanced images. A training method for the low-illumination image adaptive enhancement model comprises: acquiring a training set comprising low-illumination images and corresponding reference images; constructing a low-illumination image adaptive enhancement model by means of a Retinex algorithm, the low-illumination image adaptive enhancement model comprising a projection module, an illumination component module, a reflectance component module and an enhancement module; and using the training set to train the low-illumination image adaptive enhancement model, so as to obtain a trained low-illumination image adaptive enhancement model. The present invention enhances the brightness and contrast of images while effectively restoring color and structural information of images.

Description

A method, apparatus, device, and storage medium for adaptive enhancement of low-light images Technical Field

[0001] This application belongs to the field of digital image processing technology, and specifically relates to a method, apparatus, device and storage medium for adaptive enhancement of low-light images. Background Technology

[0002] With the development of new media technologies, communication media such as voice, images, and videos are receiving increasing attention from the public. Images are one of the most intuitive ways for people to understand the world. However, images are usually taken under suboptimal lighting conditions. Due to environmental factors such as poor lighting and improper beam angles, as well as technical limitations such as low ISO and short exposure time, insufficient photons reach the sensor, resulting in images that often have degraded features and low contrast. These images are called low-light images.

[0003] The aforementioned problems can be addressed reasonably at the camera level, such as by using a higher ISO and a longer exposure time to significantly increase image brightness. However, increasing the ISO introduces noise into the image, while longer exposures produce motion blur, resulting in a lower image quality. Another viable solution is to use image editing tools like Photoshop or Lightroom to enhance the visual appeal of low-light images. However, using these image editing tools requires specialized knowledge and skills, and typically demands a significant time investment.

[0004] Beyond improving image quality at the hardware level, research on low-light image enhancement algorithms has become a hot topic in the field of image processing. Traditional methods mainly fall into two categories: one based on histogram equalization to enhance image brightness and contrast, and the other incorporating Retinex theory to build models for image enhancement. However, these methods typically have limitations, such as overly ideal assumptions leading to loss of detail and color distortion in the enhancement results, difficulty in finding accurate priors, and complex optimization processes resulting in long runtimes. In recent years, thanks to the rapid development of deep learning, some pioneering works on low-light image enhancement have achieved remarkable results. Compared to traditional methods, deep learning-based solutions offer better accuracy, robustness, and computational speed, attracting increasing attention.

[0005] Low-light images not only degrade the perceptual quality of images but also affect the performance of a range of subsequent high-level visual tasks, including image recognition, object detection, and semantic segmentation. Therefore, enhancing low-quality images in low-light environments is of significant research and application value. However, most existing deep learning-based low-light image enhancement methods rely on supervised learning, which typically involves complex network structures and a large number of dependent parameters. A few unsupervised learning-based methods cannot simultaneously address issues such as poor brightness, color distortion, and severe noise. Achieving a lightweight and efficient deep learning-based low-light image enhancement method remains a major challenge.

[0006] Therefore, existing deep learning low-light image enhancement methods have complex network structures and rely on a large number of parameters, making it difficult to effectively handle problems such as poor brightness, color distortion, and severe noise. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a method, apparatus, device, and storage medium for adaptive enhancement of low-light images. Addressing the issue that the Retinex model decomposes images without considering noise in an ideal state, a Projection module is designed to remove inappropriate noise and features using paired low-light images. L-Net and R-Net, which contain only a few convolutional layers, residual connections, and channel attention mechanisms, decompose low-light images into illuminance and reflectance components. A lightweight adaptive adjustment curve, combined with a joint loss function consisting of a non-reference loss function and a reference loss function, is used to progressively enhance the brightness and contrast of the image while effectively restoring its color and structural information.

[0008] To achieve the above objectives, the present invention employs the following technical solution:

[0009] In a first aspect, the present invention provides a low-light image adaptive enhancement method, comprising:

[0010] The low-light image to be enhanced is input into a pre-trained low-light image adaptive enhancement model to obtain the enhanced image;

[0011] The training method for the low-light image adaptive enhancement model includes:

[0012] Obtain a training set containing low-light images and their corresponding reference images;

[0013] A low-light image adaptive enhancement model is constructed based on the Retinex algorithm; wherein, the low-light image adaptive enhancement model includes: a projection module, an illuminance component module, a reflectance component module, and an enhancement module;

[0014] The low-light image adaptive enhancement model is trained using the training set to obtain the trained low-light image adaptive enhancement model.

[0015] Furthermore, the projection module includes five 3×3 convolutional layers; wherein, the first four convolutional layers use the ReLU function as the activation function, the output of the fifth convolutional layer is residually concatenated with the input image of the projection module, and the projection module ends with the Sigmoid function;

[0016] The illumination component module includes five 3×3 convolutional layers; the first four convolutional layers use the ReLU function as the activation function, the output of the fifth convolutional layer is residually connected to the input image of the illumination component module, and the illumination component module ends with the Sigmoid function.

[0017] The reflection component module includes five 3×3 convolutional layers and a channel attention module. The first four convolutional layers use the ReLU function as the activation function. The output of the fifth convolutional layer is fed into the channel attention module and then residually connected with the input image of the reflection component module. The reflection component module ends with the Sigmoid function. The channel attention module includes global average pooling and two 1×1 convolutional layers.

[0018] The enhancement module includes seven depthwise separable convolutional layers; wherein the input of the fifth convolutional layer is the concatenation of the outputs of the third and fourth layers, the input of the sixth convolutional layer is the concatenation of the outputs of the fifth and second layers, and the input of the seventh convolutional layer is the concatenation of the outputs of the sixth and first layers; the first six layers use the ReLU function as the activation function, and the seventh layer uses the tanh function as the activation function.

[0019] Furthermore, training the low-light image adaptive enhancement model using the training set includes:

[0020] The low-light image pairs I1 and I2 are input into the projection module to obtain the projected image pairs i1 and i2;

[0021] The projected image pairs i1 and i2 are input into the illuminance component module and the reflection component module for decomposition to obtain illuminance components L1 and L2 and reflection components R1 and R2.

[0022] The illuminance component L1 is input into the enhancement network to obtain the illuminance component enhanced_L after adaptive adjustment curve;

[0023] The enhanced image E is obtained by multiplying the illuminance component enhanced_L and the reflectance component R1 element by element using the adaptive adjustment curve.

[0024] Furthermore, the low-light images and corresponding reference images are: pairs of low-light images of the same scene that are underexposed but have different exposure levels, and corresponding reference images with normal exposure, obtained from the SICE public dataset and the LOL public dataset.

[0025] Further, the step of inputting the low-light image pair I1, I2 into the projection module to obtain the projected image pair i1, i2 includes:

[0026] The low-light image pairs I1 and I2 are input into the projection module to remove noise and features that are not suitable for Retinex decomposition, and the optimized low-light images are used as the projection image pairs i1 and i2.

[0027] Among them, the projection loss function L is used. p The projection loss function L removes noise and features that are unsuitable for Retinex decomposition. p for:

[0028] In the formula, I1 represents the low-light image, and i1 represents the projected image;

[0029] The step of inputting the projected image pairs i1 and i2 into the illuminance component module and the reflection component module for decomposition to obtain illuminance components L1 and L2 and reflection components R1 and R2 includes:

[0030] Through reflection consistency loss L C The loss function L formed by the basic constraints R Decompose the projected image;

[0031] Among them, L C and L R Represented as:

[0032] Where L represents the illuminance component, R represents the reflectance component, and i represents the projected image; R1 and R2 represent the reflectance components of the low-illuminance image pair; L0 represents the initial estimate of the illuminance component, obtained by calculating the maximum values ​​of the R, G, and B channels; ▽ represents the horizontal and vertical gradients; stopgrad(L) indicates that the gradient information should not be propagated to the illuminance component when calculating this part during network training.

[0033] The loss function for this step is expressed as: L stage1 =ω0L P +ω1L C +ω2L R

[0034] Where ω0, ω1, and ω2 are weighted weights.

[0035] Further, the step of inputting the illuminance component L1 into the enhancement network to obtain the illuminance component enhanced_L after adaptive adjustment of the curve includes:

[0036] The illuminance component is iteratively enhanced using an adaptive adjustment curve; where the adaptive adjustment curve is represented as: LE n (x)=LE n-1 (x)+A n (x)LE n-1 (x)(1-LE n-1 (x))

[0037] Among them, A n (x) is a parametric graph of the same size as a given image, which can be learned by the augmentation network in the adaptive adjustment curve. n-1 (x) is the previous enhanced image in each iteration of the network;

[0038] Furthermore, the loss function for the enhanced illuminance component enhanced by the adaptive adjustment curve, denoted as `enhanced_L`, is expressed as: L stage2 =L1+L SSIM +L col +L bri +L stru

[0039] Where L1 represents the mean absolute error, L SSIM L represents the structural similarity loss between the enhanced image E and the reference image H. col L bri L stru These represent color loss, brightness loss, and structure loss, respectively.

[0040] The color loss L col Represented as:

[0041] The brightness loss L bri Represented as:

[0042] The structural loss L stru Represented as:

[0043] Among them, E (x,y) and H (x,y) Let u and v represent the pixel vectors in the x-th row and y-th column of the enhanced image and the reference image, respectively, where u and v indicate that the image has u rows and v columns. <g>represents the cosine similarity between two vectors; c represents the three color channels R, G, and B. and Represented in pixels and The pixel block centered on the image.

[0044] In a second aspect, the present invention provides a low-light image adaptive enhancement device, comprising:

[0045] The enhancement module is used to input the low-light image to be enhanced into a pre-trained low-light image adaptive enhancement model to obtain the enhanced image;

[0046] The training method for the low-light image adaptive enhancement model includes:

[0047] Obtain a training set containing low-light images and their corresponding reference images;

[0048] A low-light image adaptive enhancement model is constructed based on the Retinex algorithm; wherein, the low-light image adaptive enhancement model includes: a projection module, an illumination module, a reflection module, and an enhancement module;

[0049] The low-light image adaptive enhancement model is trained using the training set to obtain the trained low-light image adaptive enhancement model.

[0050] Thirdly, the present invention provides a computer device, including a processor and a storage medium;

[0051] The storage medium is used to store instructions;

[0052] The processor is configured to operate according to the instructions to implement the steps of the low-light image adaptive enhancement method as described in the first aspect.

[0053] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the low-light image adaptive enhancement method as described in the first aspect.

[0054] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0055] (1) In view of the case that the Retinex model does not consider noise when decomposing images under ideal conditions, the present invention designs a Projection module and removes inappropriate noise and features by using pairs of low-light images;

[0056] (2) Based on Retinex theory, this invention designs a lightweight decomposition model. Using L-Net and R-Net, which contain only a few convolutional layers, residual connections and channel attention mechanisms, low-light images are decomposed into illumination components and reflectance components. The number of network parameters is small and the complexity of the network structure is significantly reduced.

[0057] (3) This invention utilizes a lightweight adaptive adjustment curve in conjunction with a joint loss function composed of a non-reference loss function and a reference loss function to gradually enhance the brightness and contrast of the image, while effectively restoring the color and structural information of the image. It has significant advantages in objective image quality evaluation indicators such as peak signal-to-noise ratio (PSNR) and structural similarity (SSIM). Attached Figure Description

[0058] Figure 1 shows a schematic diagram of the overall process of the lightweight low-light image adaptive enhancement method based on Retinex provided in this invention.

[0059] Figure 2 shows a flowchart of the first-stage projection and decomposition network provided in an example of the present invention.

[0060] Figure 3 shows a flowchart of the second-stage enhanced network provided in the example of the present invention;

[0061] Figure 4 shows a schematic diagram of the original low-light image used for testing provided in an example of the present invention;

[0062] Figure 5 shows a schematic diagram of the experimental results obtained by the lightweight low-light image enhancement method provided in the example of the present invention. Detailed Implementation

[0063] The technical concept of this invention is as follows: For the case where the Retinex model decomposes an image without considering noise in an ideal state, a Projection module is designed to remove inappropriate noise and features using paired low-light images; L-Net and R-Net, which contain only a few convolutional layers, residual connections, and channel attention mechanisms, are used to decompose the low-light image into illumination and reflectance components; a lightweight adaptive adjustment curve is used in conjunction with a joint loss function composed of a non-reference loss function and a reference loss function to progressively enhance the brightness and contrast of the image, while effectively restoring the color and structural information of the image.

[0064] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this application or its application or use.

[0065] Example 1

[0066] This embodiment discloses an adaptive enhancement method for low-light images, including:

[0067] The low-light image is input into a pre-trained low-light image adaptive enhancement model to obtain the enhanced image;

[0068] The training method for the low-light image adaptive enhancement model is shown in Figure 1, and includes:

[0069] Step S1: Obtain two low-light image pairs I1 and I2 of the same scene that are underexposed but have different exposure levels, as well as the corresponding reference image H with normal exposure;

[0070] Step S2, the first stage, sends the low-light image pairs I1 and I2 into the Projection module, abbreviated as P-Net, to remove noise and features that are not suitable for Retinex decomposition, and generates two optimized versions of the low-light image pairs i1 and i2.

[0071] Step S3: Based on Retinex theory, the optimized low-light image pairs i1 and i2 are fed into the Illumination module (L-Net) and the Reflectance module (R-Net) for decomposition. The Illumination components L1 and L2 are obtained through L-Net decomposition, and the Reflectance components R1 and R2 are obtained through R-Net decomposition.

[0072] Step S4, the second stage, the L1 component obtained from the first stage decomposition is fed into the Enhance-Net, abbreviated as E-Net, to obtain the illuminance component enhanced_L after being enhanced by the adaptive adjustment curve;

[0073] Step S5: Multiply enhanced_L element-wise with R1 to obtain the enhanced image Enhanced_img, abbreviated as E.

[0074] In this embodiment, the underexposed but differently exposed low-light image pairs of the same scene described in step S1, along with their corresponding normally exposed reference images, are collected from the SICE and LOL public datasets. The LOL dataset is the first dataset of paired images containing low-light and normal-light images captured in real-world scenes; the SICE dataset also consists of low-light and normal-light image pairs, including multiple exposure image sequences in indoor and outdoor scenes, with each sequence containing 3 to 18 low-contrast images at different exposure levels. Normally exposed reference images and several corresponding underexposed but well-aligned low-light images from both datasets are randomly selected to construct the training set. Each normally exposed reference image in the constructed training set contains 2 to 6 underexposed low-light images. The original low-light images used for testing in this embodiment are shown in Figure 4.

[0075] In this embodiment, steps S2 and S3 refer to the flowchart of the first-stage projection and decomposition network shown in Figure 2. The P-Net described in step S2 consists of only five 3×3 convolutional layers. For the first four convolutional layers, the ReLU function is used as the activation function. The output of the fifth convolutional layer is residually connected to the input image fed into the P-Net to reduce gradient vanishing and improve network performance. The P-Net ends with a Sigmoid layer, which normalizes the output to the range [0,1]. The projection loss function L is used. p The projection loss function L guides P-Net to remove inappropriate noise and features. p for:

[0076] Where i1 represents the projected image. Inappropriate noise in the original low-light image after projection is removed.

[0077] In this embodiment, the L-Net described in step S3 is used to decompose the illuminance component of a low-light image, and mainly consists of five 3×3 convolutions. For the first four convolutional layers, the ReLU function is used as the activation function. The output of the fifth convolutional layer is residually connected to the input image fed into the L-Net to reduce gradient vanishing and improve network performance. According to Retinex theory, assuming that the three color channels have the same illuminance component, the L-Net uses a sixth convolutional layer to convert the number of channels in the output result to 1. The L-Net ends with a Sigmoid layer, normalizing the output to the range [0,1]. The R-Net is used to decompose the reflectance component of a low-light image, and mainly consists of five 3×3 convolutions and a channel attention module. For the first four convolutional layers, the ReLU function is used as the activation function. The output of the fifth convolutional layer is fed into the channel attention module and then residually connected to the input image fed into the R-Net to reduce gradient vanishing and improve network performance. The R-Net ends with a Sigmoid layer, normalizing the output to the range [0,1].

[0078] The channel attention module uses only global average pooling and two 1×1 convolutions. This module infers an attention map along the channel dimension and multiplies it by the feature map to generate a weighted feature map, which can be used for cross-channel information interaction.

[0079] The L-Net and R-Net use reflection consistency loss L C The loss function L, which is composed of some basic constraints of Retinex theory, R Decompose low-light images. C and L R It can be represented as:

[0080] Where R1 and R2 represent the reflectance classification of low-illuminance image pairs, i represents the projected image, and L0 represents the initial estimate of the illuminance components, obtained by calculating the maximum values ​​of the R, G, and B channels. This represents the horizontal and vertical gradients.

[0081] The overall loss function of the first stage described by steps S2 and S3 can be expressed as a weighted sum of weights ω0, ω1, and ω2 as: L stage1 =ω0L P +ω1L C +ω2L R

[0082] In this embodiment, the weights ω0, ω1, and ω2 are set to 100, 1, and 1, respectively.

[0083] In this embodiment, step S4 refers to the flowchart of the second-stage enhancement network in Figure 3. The adaptive adjustment curve of the iterative enhancement in step S4 can be represented as: LE n (x)=LE n-1 (x)+A n (x)LE n-1 (x)(1-LE n-1 (x))

[0084] Among them, A n (x) is a parametric graph of the same size as a given image, which can be learned by the augmentation network in the adaptive adjustment curve. n-1 (x) represents the previous augmented image in each iteration of the network. The augmentation network E-Net contains seven depthwise separable convolutional layers. For the input image, it passes through the first four convolutional layers sequentially. The input to the fifth convolutional layer is the concatenation of the outputs of the third and fourth layers. The input to the sixth convolutional layer is the concatenation of the outputs of the fifth and second layers. The input to the seventh convolutional layer is the concatenation of the outputs of the sixth and first layers. The first six layers use the ReLU function as the activation function after each convolution, while the seventh layer uses the tanh function as the activation function. The output of the seventh convolutional layer serves as the parameter for the adaptive adjustment curve (Figure A). n (x), based on the curve, adaptively enhance the illuminance component to obtain enhanced_L.

[0085] In this embodiment, step S5 obtains the final enhanced image E based on the outputs of the two stages. The weight of each loss term in the second-stage loss function is 1, which can be expressed as: L stage2 =L1+L SSIM +L col +L bri +L stru

[0086] Where L1 represents the mean absolute error, L SSIM L represents the structural similarity loss between the enhanced image E and the reference image H. col L bri L stru These represent color loss, brightness loss, and structure loss, respectively.

[0087] The color loss L col It can be represented as:

[0088] Among them, E (x,y) and H (x,y) Let u and v represent the pixel vectors in the x-th row and y-th column of the enhanced image and the reference image, respectively, where u and v indicate that the image has u rows and v columns. <g>This represents the cosine similarity between two vectors.

[0089] The brightness loss L bri It can be represented as:

[0090] Where 'c' represents the three color channels: R, G, and B. and Represented in pixels and The pixel block centered on the image.

[0091] The structural loss L stru It can be represented as:

[0092] In this embodiment, the PyTorch deep learning framework and two Nvidia Geforce GTX 1660Ti GPUs are used for network training. The low-light image pairs collected in step S1 and their corresponding reference images are sequentially fed into the first and second stage network models. In the first stage, the input low-light image pairs are randomly cropped to 128×128 pixels, the training batch size is 8, and the network is trained using an Adam optimizer with an initial learning rate of 1×10⁻⁴, with 300 training iterations. In the second stage, the input low-light image and reference image are used as image pairs for training. The low-light image is decomposed into illuminance and reflectance components by the network trained in the first stage. The illuminance component is fed into the enhancement network in the second stage for enhancement and multiplied element-wise with the reflectance component to obtain the final enhanced image. The training batch size is 8, and the network is trained using an Adam optimizer with an initial learning rate of 1×10⁻⁴, with 300 training iterations.

[0093] To compare the experimental results of this embodiment with advanced low-light image enhancement methods, the widely used evaluation metrics Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) were used to quantitatively evaluate the experimental results.

[0094] Peak signal-to-noise ratio (PSNR) can be expressed as:

[0095] Where MSE represents the mean square error of image X and Y, H and W represent the length and width of the image, respectively, and MAX... X This represents the maximum pixel value of image X. A higher PSNR value indicates better image quality.

[0096] Structural similarity (SSIM) can be represented as:

[0097] Where, μ x and μ y Let X and Y represent the means of the image, respectively. and Let σ represent the variances of images X and Y, respectively. xy denoted by covariance, c1 and c2 are non-zero constants. The value of SSIM ranges from [0,1], with values ​​closer to 1 indicating higher similarity between images X and Y.

[0098] The experimental results obtained from the lightweight low-light image enhancement method provided in this invention are shown in Figure 5. The experimental results of this embodiment are compared with those of advanced low-light image enhancement methods from the past five years on the LOL dataset test set, and the results are shown in Table 1. In the table, bold text indicates the optimal result, and underlined text indicates the second-best result. The lightweight low-light image enhancement method proposed in this embodiment achieves good evaluation metrics while having a small number of network parameters and a simple network structure.

[0099] Table 1 Evaluation metrics of different algorithms on the LOL test set

[0100] Example 2

[0101] This embodiment discloses a low-light image adaptive enhancement device, comprising:

[0102] The enhancement module is used to input the low-light image to be enhanced into a pre-trained low-light image adaptive enhancement model to obtain the enhanced image;

[0103] The training method for the low-light image adaptive enhancement model includes:

[0104] Obtain a training set containing low-light images and their corresponding reference images;

[0105] A low-light image adaptive enhancement model is constructed based on the Retinex algorithm; wherein, the low-light image adaptive enhancement model includes: a projection module, an illumination module, a reflection module, and an enhancement module;

[0106] The low-light image adaptive enhancement model is trained using the training set to obtain the trained low-light image adaptive enhancement model.

[0107] Example 3

[0108] This embodiment discloses a computer device, including a processor and a storage medium;

[0109] The storage medium is used to store instructions;

[0110] The processor is configured to operate according to the instructions to implement the steps of the low-light image adaptive enhancement method as described in the first aspect.

[0111] Example 4

[0112] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the low-light image adaptive enhancement method as described in the first aspect.

[0113] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0114] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams.

[0115] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0116] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0117] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.< / g> < / g>

Claims

1. A method for adaptive enhancement of low-light images, characterized in that, include: The low-light image to be enhanced is input into a pre-trained low-light image adaptive enhancement model to obtain the enhanced image; The training method for the low-light image adaptive enhancement model includes: Obtain a training set containing low-light images and their corresponding reference images; A low-light image adaptive enhancement model is constructed based on the Retinex algorithm; wherein, the low-light image adaptive enhancement model includes: a projection module, an illuminance component module, a reflectance component module, and an enhancement module; The low-light image adaptive enhancement model is trained using the training set to obtain the trained low-light image adaptive enhancement model.

2. The lightweight low-light image adaptive enhancement method based on Retinex according to claim 1, characterized in that, The projection module includes five 3×3 convolutional layers; the first four convolutional layers use the ReLU function as the activation function, the output of the fifth convolutional layer is residually concatenated with the input image of the projection module, and the projection module ends with the Sigmoid function. The illumination component module includes five 3×3 convolutional layers; the first four convolutional layers use the ReLU function as the activation function, the output of the fifth convolutional layer is residually connected to the input image of the illumination component module, and the illumination component module ends with the Sigmoid function. The reflection component module includes five 3×3 convolutional layers and a channel attention module. The first four convolutional layers use the ReLU function as the activation function. The output of the fifth convolutional layer is fed into the channel attention module and then residually connected with the input image of the reflection component module. The reflection component module ends with the Sigmoid function. The channel attention module includes global average pooling and two 1×1 convolutional layers. The enhancement module includes seven depthwise separable convolutional layers; wherein the input of the fifth convolutional layer is the concatenation of the outputs of the third and fourth layers, the input of the sixth convolutional layer is the concatenation of the outputs of the fifth and second layers, and the input of the seventh convolutional layer is the concatenation of the outputs of the sixth and first layers; the first six layers use the ReLU function as the activation function, and the seventh layer uses the tanh function as the activation function.

3. The lightweight low-light image adaptive enhancement method based on Retinex according to claim 1, characterized in that, The training of the low-light image adaptive enhancement model using the training set includes: The low-light image pairs I1 and I2 are input into the projection module to obtain the projected image pairs i1 and i2; The projected image pairs i1 and i2 are input into the illuminance component module and the reflection component module for decomposition to obtain illuminance components L1 and L2 and reflection components R1 and R2. The illuminance component L1 is input into the enhancement network to obtain the illuminance component enhanced_L after adaptive adjustment curve; The enhanced image E is obtained by multiplying the illuminance component enhanced_L and the reflectance component R1 element by element, which are enhanced by the adaptive adjustment curve.

4. The lightweight low-light image adaptive enhancement method based on Retinex according to claim 1, characterized in that, The low-light images and corresponding reference images are: pairs of low-light images of the same scene that are underexposed but have different levels of exposure, and corresponding reference images with normal exposure, obtained from the SICE public dataset and the LOL public dataset.

5. The lightweight low-light image adaptive enhancement method based on Retinex according to claim 3, characterized in that, The step of inputting the low-light image pair I1, I2 into the projection module to obtain the projected image pair i1, i2 includes: The low-light image pairs I1 and I2 are input into the projection module to remove noise and features that are not suitable for Retinex decomposition, and the optimized low-light images are used as the projection image pairs i1 and i2. Among them, the projection loss function L is used. p The projection loss function L removes noise and features that are unsuitable for Retinex decomposition. p for: In the formula, I1 represents the low-light image, and i1 represents the projected image; The step of inputting the projected image pairs i1 and i2 into the illuminance component module and the reflection component module for decomposition to obtain illuminance components L1 and L2 and reflection components R1 and R2 includes: Through reflection consistency loss L C The loss function L formed by the basic constraints R Decompose the projected image; Among them, L C and L R Represented as: Where L represents the illuminance component, R represents the reflectance component, and i represents the projected image; R1 and R2 represent the reflectance components of the low-illuminance image pair; L0 represents the initial estimate of the illuminance component, which is obtained by calculating the maximum values ​​of the R, G, and B channels. Represents the horizontal and vertical gradients; stopgrad(L) indicates that the gradient information should not be propagated to the illumination component during network training; the loss function for this step is expressed as: L stage1 =ω0L P +ω1L C +ω2L R Where ω0, ω1, and ω2 are weighted weights.

6. The lightweight low-light image adaptive enhancement method based on Retinex according to claim 3, characterized in that, The step of inputting the illuminance component L1 into the enhancement network to obtain the illuminance component enhanced_L after adaptive adjustment curve includes: The illuminance component is iteratively enhanced using an adaptive adjustment curve; where the adaptive adjustment curve is represented as: THE n (x)=LE n-1 (x)+A n (x)LE n-1 (x)(1-LE n-1 (x) Among them, A n (x) is a parametric graph of the same size as a given image, which can be learned by the augmentation network in the adaptive adjustment curve. n-1 (x) is the previous enhanced image in each iteration of the network.

7. The lightweight low-light image adaptive enhancement method based on Retinex according to claim 6, characterized in that, The loss function for the enhanced illuminance component enhanced by the adaptive adjustment curve, denoted as enhanced_L, is expressed as: L stage2 =L1+L SSIM +L col +L bri +L stru Where L1 represents the mean absolute error, L SSIM L represents the structural similarity loss between the enhanced image E and the reference image H. col L bri L stru These represent color loss, brightness loss, and structure loss, respectively. The color loss L col Represented as: The brightness loss L bri Represented as: The structural loss L stru Represented as: Among them, E (x,y) and H (x,y) Let u and v represent the pixel vectors in the x-th row and y-th column of the enhanced image and the reference image, respectively, where u and v indicate that the image has u rows and v columns. <g>represents the cosine similarity between two vectors; c represents the three color channels R, G, and B. < / g> and Represented in pixels and The pixel block centered on the center.

8. A low-light image adaptive enhancement device, characterized in that, include: The enhancement module is used to input the low-light image to be enhanced into a pre-trained low-light image adaptive enhancement model to obtain the enhanced image; The training method for the low-light image adaptive enhancement model includes: Obtain a training set containing low-light images and their corresponding reference images; A low-light image adaptive enhancement model is constructed based on the Retinex algorithm; wherein, the low-light image adaptive enhancement model includes: a projection module, an illuminance component module, a reflectance component module, and an enhancement module; The low-light image adaptive enhancement model is trained using the training set to obtain the trained low-light image adaptive enhancement model.

9. A computer device, characterized in that, Including processor and storage media; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to implement the steps of the low-light image adaptive enhancement method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the low-light image adaptive enhancement method as described in any one of claims 1 to 7.

Images