Low-illumination image adaptive enhancement method and device, equipment and storage medium

Through the lightweight low-illumination image adaptive enhancement method, the projection module and L-Net/R-Net decompose images, combined with the adaptive adjustment curve and joint loss function, the network complexity and large parameters in the prior art are solved. The problem has significantly improved image quality and evaluation indicators.

CN118967534APending Publication Date: 2024-11-15NANJING UNIV OF POSTS & TELECOMM
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410912479.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-09
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

The existing deep learning low-illumination image enhancement methods have complex network structure and large amounts of parameters, making it difficult to effectively deal with problems such as poor brightness, color distortion, and severe noise.

Method used

A lightweight low-illumination image adaptive enhancement method is designed, using Projection module and L-Net/R-Net to decompose low-illumination images into illumination components and reflective components, and adjust the curve and joint through lightweight adaptive The loss function gradually enhances the brightness and contrast of the image while restoring the color and structure information of the image.

Benefits of technology

The network parameters are small and structural complexity is reduced, the image quality is significantly improved, and the evaluation indicators such as peak signal-to-noise ratio PSNR and structural similarity SSIM are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118967534A_ABST
    Figure CN118967534A_ABST
Patent Text Reader

Abstract

The invention discloses a low-illumination image adaptive enhancement method, device and equipment and a storage medium, and the method comprises the steps: inputting a to-be-enhanced low-illumination image into a pre-trained low-illumination image adaptive enhancement model, and obtaining an enhanced image; the training method of the low-illumination image adaptive enhancement model comprises the following steps: acquiring a training set comprising a low-illumination image and a corresponding reference image; constructing a low-illumination image adaptive enhancement model based on a Retinex algorithm; wherein the low-illumination image adaptive enhancement model comprises a projection module, an illumination component module, a reflection component module and an enhancement module; and training the low-illumination image adaptive enhancement model by using the training set to obtain a trained low-illumination image adaptive enhancement model. The brightness and contrast of the image are enhanced, and the color and structure information of the image is effectively restored.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application belongs to the technical field of digital image processing, and in particular relates to a low-illumination image adaptive enhancement method, device, equipment and storage medium. Background Art

[0002] With the development of new media technology, communication media such as voice, images, and videos are gaining more and more attention from the public. Images are one of the most intuitive ways for people to understand the world, but images are usually taken under suboptimal lighting conditions. Due to environmental factors such as poor lighting and improper beam angles, as well as technical limitations such as low ISO and short exposure time, insufficient photons reach the sensor, resulting in images with deteriorated features and low contrast. These images are called low-light images.

[0003] The above problems can be solved in a reasonable way at the camera level, such as using higher ISO and longer exposure time to significantly improve the brightness of the image. However, increasing the ISO will introduce noise in the image, while long exposure will produce motion blur, making the image look worse. Another possible solution is to use image editing tools such as Photoshop or Lightroom to enhance the visual appeal of low-light images. However, using these image editing tools requires expertise and skills, and usually takes a lot of time.

[0004] In addition to improving image quality at the hardware level, research on low-light image enhancement algorithms has also become one of the hot topics in the field of image processing. Traditional methods are mainly divided into two categories, one is based on histogram equalization to enhance the brightness and contrast of the image, and the other introduces the retinal Retinex theory to build a model for image enhancement. However, these methods usually have their limitations, such as overly ideal assumptions leading to loss of details and color distortion in the enhancement results, difficulty in finding accurate priors, and complex optimization processes leading to long running times. In recent years, thanks to the rapid development of deep learning, some pioneering work on low-light image enhancement has achieved remarkable results, and compared with traditional methods, solutions based on deep learning have better accuracy, robustness and computing speed, attracting more and more attention.

[0005] Low-light images not only reduce the perceived quality of the image, but also affect the performance of a series of subsequent high-level visual tasks including image recognition, object detection, and semantic segmentation. Therefore, enhancing low-quality images in low-light environments has important research significance and application value. However, most of the existing deep learning-based low-light image enhancement methods are based on supervised learning. Such methods usually have complex network structures and rely on a large number of parameters; a small number of unsupervised learning-based methods cannot simultaneously solve problems such as poor brightness, color distortion, and severe noise. How to achieve a lightweight and efficient deep learning low-light image enhancement method remains a major challenge.

[0006] Therefore, the existing deep learning low-light image enhancement methods have complex network structures and rely on a large number of parameters, making it difficult to effectively deal with problems such as poor brightness, color distortion, and severe noise. Summary of the invention

[0007] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a method, device, equipment and storage medium for adaptive enhancement of low-light images. In view of the fact that the noise is not considered when the Retinex model decomposes the image under an ideal state, a Projection module is designed, and inappropriate noise and features are removed through paired low-light images; the low-light image is decomposed into illumination component and reflection component by using L-Net and R-Net containing only several convolutional layers, residual connections and channel attention mechanisms; the brightness and contrast of the image are gradually enhanced by using a lightweight adaptive adjustment curve in combination with a joint loss function composed of a non-reference loss function and a reference loss function, while effectively restoring the color and structural information of the image. .

[0008] To achieve the above object, the present invention is implemented by adopting the following technical solutions:

[0009] In a first aspect, the present invention provides a low-illumination image adaptive enhancement method, comprising:

[0010] Inputting the low-light image to be enhanced into the pre-trained low-light image adaptive enhancement model to obtain an enhanced image;

[0011] The training method of the low-light image adaptive enhancement model includes:

[0012] Obtain a training set including low-light images and corresponding reference images;

[0013] A low-illumination image adaptive enhancement model is constructed based on the Retinex algorithm; wherein the low-illumination image adaptive enhancement model includes: a projection module, an illumination component module, a reflection component module and an enhancement module;

[0014] The low-illumination image adaptive enhancement model is trained using the training set to obtain a trained low-illumination image adaptive enhancement model.

[0015] Furthermore, the projection module includes five layers of 3×3 convolutions; wherein the first four convolution layers use the ReLU function as an activation function, the output of the fifth convolution layer is residually connected with the input image of the projection module, and the projection module ends with a Sigmoid function;

[0016] The illumination component module includes five layers of 3×3 convolutions; wherein the first four convolution layers use the ReLU function as an activation function, the output of the fifth convolution layer is residually connected with the input image of the illumination component module, and the illumination component module ends with a Sigmoid function;

[0017] The reflection component module includes five layers of 3×3 convolution and a channel attention module; wherein the first four convolution layers use the ReLU function as the activation function, the output of the fifth convolution layer is sent to the channel attention module and then residually connected with the input image of the reflection component module, and the reflection component module ends with a Sigmoid function; the channel attention module includes global average pooling and two layers of 1×1 convolution;

[0018] The enhancement module includes seven layers of depth-separable convolutions; wherein the input of the fifth convolution layer is the concatenation of the outputs of the third and fourth layers, the input of the sixth convolution layer is the concatenation of the outputs of the fifth and second layers, and the input of the seventh convolution layer is the concatenation of the outputs of the sixth and first layers; the first six layers use the ReLu function as the activation function, and the seventh layer uses the tanh function as the activation function.

[0019] Furthermore, the step of using the training set to train the low-light image adaptive enhancement model includes:

[0020] Input the low illumination image pair I1 and I2 into the projection module to obtain the projection image pair i1 and i2;

[0021] Input the projection image pair i1 and i2 into the illumination component module and the reflection component module for decomposition to obtain illumination components L1 and L2 and reflection components R1 and R2;

[0022] The illumination component L1 is input into the enhancement network to obtain the illumination component enhanced_L enhanced by the adaptive adjustment curve;

[0023] The illuminance component enhanced_L enhanced by the adaptive adjustment curve and the reflection component R1 are multiplied element by element to obtain an enhanced image E.

[0024] Furthermore, the low-light image and the corresponding reference image are: a pair of low-light images of the same scene that are underexposed but have different exposure levels and a corresponding reference image with normal exposure, which are obtained from the SICE public dataset and the LOL public dataset.

[0025] Furthermore, the step of inputting the low-illumination image pair I1 and I2 into the projection module to obtain the projection image pair i1 and i2 includes:

[0026] Input the low-light image pair I1 and I2 into the projection module, remove the noise and features that are not suitable for Retinex decomposition, and obtain the optimized low-light image as the projection image pair i1 and i2;

[0027] Among them, the projection loss function is used To remove noise and features that are not suitable for Retinex decomposition, the projection loss function for:

[0028]

[0029] In the formula, Represents low-light images, represents the projected image;

[0030] The projected image pair i1, i2 is input into the illumination component module and the reflection component module for decomposition to obtain illumination components L1, L2 and reflection components R1, R2, including:

[0031] Consistency loss through reflection The loss function consisting of the basic constraints Decomposing the projected image;

[0032] in, and It is expressed as:

[0033]

[0034]

[0035] Among them, L represents the illumination component, R represents the reflection component, and i represents the projected image; and represents the reflection component of the low-light image pair; Represents the initial estimate of the illumination component, obtained by calculating the maximum value of the R, G, and B channels; Represents horizontal and vertical gradients; It means that the network training should not propagate the gradient information into the illumination component when calculating here;

[0036] The loss function of this step is expressed as:

[0037]

[0038] in, , , is the weighted weight.

[0039] Furthermore, the step of inputting the illumination component L1 into the enhancement network to obtain the illumination component enhanced_L enhanced by the adaptive adjustment curve includes:

[0040] The illuminance component is iteratively enhanced using an adaptive adjustment curve; wherein the adaptive adjustment curve is expressed as:

[0041]

[0042] in, is a parameter map of the same size as a given image in the adaptive adjustment curve that can be learned by the reinforcement network, is the last enhanced image in all iterations of the network;

[0043] Furthermore, the loss function of the illumination component enhanced_L enhanced by the adaptive adjustment curve is expressed as:

[0044]

[0045] in, represents the mean absolute error, represents the structural similarity loss between the enhanced image E and the reference image H, , , They represent color loss, brightness loss, and structure loss respectively;

[0046] The color loss It is expressed as:

[0047]

[0048] The brightness loss It is expressed as:

[0049]

[0050] The structural loss It is expressed as:

[0051]

[0052] in, and Represents the pixel vectors of the xth row and yth column of the enhanced image and the reference image respectively, u and v represent that the image has u rows and v columns, represents the cosine similarity of two vectors; c represents the three color channels of R, G, and B. and In pixels and The pixel block at the center.

[0053] In a second aspect, the present invention provides a low-illumination image adaptive enhancement device, comprising:

[0054] An enhancement module, used for inputting the low-light image to be enhanced into a pre-trained low-light image adaptive enhancement model to obtain an enhanced image;

[0055] The training method of the low-light image adaptive enhancement model includes:

[0056] Obtain a training set including low-light images and corresponding reference images;

[0057] A low-illumination image adaptive enhancement model is constructed based on the Retinex algorithm; wherein the low-illumination image adaptive enhancement model includes: a Projection module, an Illumination module, a Reflectance module and an enhancement module;

[0058] The low-illumination image adaptive enhancement model is trained using the training set to obtain a trained low-illumination image adaptive enhancement model.

[0059] In a third aspect, the present invention provides a computer device, including a processor and a storage medium;

[0060] The storage medium is used to store instructions;

[0061] The processor is used to operate according to the instruction to implement the steps of the low-illumination image adaptive enhancement method as described in the first aspect.

[0062] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the low-illumination image adaptive enhancement method as described in the first aspect.

[0063] Compared with the prior art, the present invention has the following beneficial effects:

[0064] (1) In view of the fact that the Retinex model does not consider noise when decomposing images under ideal conditions, this paper designs a Projection module and removes inappropriate noise and features through paired low-light images;

[0065] (2) The present invention designs a lightweight decomposition model based on the Retinex theory, and decomposes the low-light image into illumination component and reflection component by using L-Net and R-Net which only contain several convolutional layers, residual connections and channel attention mechanism. The network parameters are small and the complexity of the network structure is significantly reduced.

[0066] (3) The present invention utilizes a lightweight adaptive adjustment curve in combination with a joint loss function consisting of a non-reference loss function and a reference loss function to gradually enhance the brightness and contrast of the image, while effectively restoring the color and structural information of the image. It has significant advantages in objective image quality evaluation indicators such as peak signal-to-noise ratio (PSNR) and structural similarity (SSIM). BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 The figure shows the overall process diagram of the lightweight low-light image adaptive enhancement method based on Retinex provided by an example of the present invention;

[0068] Figure 2 The figure shows a schematic diagram of the process of the first stage projection and decomposition network provided by an example of the present invention;

[0069] Figure 3 The figure is a schematic diagram of the process of the second stage enhanced network provided by an example of the present invention;

[0070] Figure 4 Shown is a schematic diagram of an original low-light image for testing provided by an example of the present invention;

[0071] Figure 5 Shown is a schematic diagram of experimental results obtained according to a lightweight low-illumination image enhancement method provided by an example of the present invention. DETAILED DESCRIPTION

[0072] The technical concept of the present invention is as follows: in view of the situation that the noise is not considered when the Retinex model decomposes the image under ideal conditions, a Projection module is designed, and inappropriate noise and features are removed through paired low-light images; the low-light image is decomposed into an illumination component and a reflection component by using L-Net and R-Net which only contain several convolutional layers, residual connections and channel attention mechanisms; a lightweight adaptive adjustment curve is used in combination with a joint loss function composed of a non-reference loss function and a reference loss function to gradually enhance the brightness and contrast of the image, while effectively restoring the color and structural information of the image.

[0073] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present application and its application or use.

[0074] Example 1

[0075] This embodiment discloses a low-illumination image adaptive enhancement method, including:

[0076] Input the low-light image into the pre-trained low-light image adaptive enhancement model to obtain an enhanced image;

[0077] The training method of the low-light image adaptive enhancement model is as follows: Figure 1 As shown, including:

[0078] Step S1, obtaining two low-light image pairs I1 and I2 of the same scene that are underexposed but have different exposure levels, and a corresponding reference image H of normal exposure;

[0079] Step S2, in the first stage, the low-light image pair I1 and I2 are sent to the Projection module, abbreviated as P-Net, to remove noise and features that are not suitable for Retinex decomposition, and generate two optimized versions of the low-light image pair i1 and i2;

[0080] Step S3, based on the Retinex theory, the optimized low-light image pair i1 and i2 are sent to the Illumination module, abbreviated as L-Net, and the Reflectance module, abbreviated as R-Net, for decomposition. The illumination components L1 and L2 are obtained by L-Net decomposition, and the reflection components R1 and R2 are obtained by R-Net decomposition;

[0081] Step S4, the second stage, the L1 component decomposed in the first stage is sent to the enhancement network Enhance-Net, abbreviated as E-Net, to obtain the illumination component enhanced_L enhanced by the adaptive adjustment curve;

[0082] Step S5, multiply enhanced_L and R1 element by element, and finally obtain an enhanced image Enhanced_img, abbreviated as E.

[0083] In this embodiment, the underexposed but different exposure levels of low-light image pairs of the same scene in step S1 and the corresponding normally exposed reference images are collected from the SICE public dataset and the LOL public dataset. The LOL dataset is the first paired image dataset that includes low-light and normal-light images taken in real scenes; the SICE dataset is also composed of low-light and normal-light image pairs, including multiple exposure image sequences in indoor and outdoor scenes, and each sequence contains 3 to 18 low-contrast images with different exposures. Normally exposed reference images and corresponding underexposed and well-aligned low-light images from the two datasets are randomly selected to construct a training set. Each normally exposed reference image in the constructed training set contains 2 to 6 underexposed low-light images. The original low-light image reference used for testing in this embodiment Figure 4 .

[0084] In this embodiment, steps S2 and S3 refer to Figure 2 The flowchart of the first stage projection and decomposition network shown in step S2, the P-Net consists of only five 3×3 convolutional layers. For the first four convolutional layers, the ReLU function is used as the activation function, and the output of the fifth convolutional layer is residually connected with the input image fed into the P-Net to reduce gradient dissipation and improve network performance. The P-Net ends with a Sigmoid layer to normalize the output to between [0,1]. The projection loss function is used To guide P-Net to remove inappropriate noise and features, the projection loss function for:

[0085]

[0086] in, Represents the projected image. After projection, the inappropriate noise in the original low-light image is removed.

[0087] In this embodiment, the L-Net described in step S3 is used to decompose the illumination component of the low-light image, and is mainly composed of five 3×3 convolutions. For the first four convolutional layers, the ReLU function is used as the activation function, and the output of the fifth convolutional layer is residually connected with the input image sent to L-Net to reduce gradient dissipation and improve network performance. According to the Retinex theory, it is assumed that the three color channels have the same illumination component, so L-Net uses the sixth convolutional layer to convert the number of channels of the output result to 1. L-Net ends with a Sigmoid layer to standardize the output to [0,1]. R-Net is used to decompose the reflection component of the low-light image, and is mainly composed of five 3×3 convolutions and a channel attention module. For the first four convolutional layers, the ReLU function is used as the activation function, and the output of the fifth convolutional layer is sent to the channel attention module and then residually connected with the input image sent to R-Net to reduce gradient dissipation and improve network performance. R-Net ends with a Sigmoid layer to standardize the output to [0,1].

[0088] The channel attention module only uses global average pooling and two 1×1 convolutions. This module infers the attention map along the channel dimension and multiplies it by the feature map to generate a weighted feature map, which can be used for cross-channel information interaction.

[0089] The L-Net and R-Net pass the reflection consistency loss The loss function consists of some basic constraints of Retinex theory Decomposition of low-light images. and It can be expressed as:

[0090]

[0091]

[0092] in, and represents the reflectance classification of low-light image pairs, represents the projected image, Represents the initial estimate of the illumination component, obtained by calculating the maximum value of the R, G, and B channels. Represents horizontal and vertical gradients.

[0093] The overall loss function of the first stage described by steps S2 and S3 can be weighted , , The weighted expression is:

[0094]

[0095] Among them, the weight , , In this embodiment, the values ​​are 100, 1, and 1 respectively.

[0096] In this embodiment, step S4 refers to Figure 3 The flowchart of the second stage enhanced network in step S4 is shown in FIG. The adaptive adjustment curve of the iterative enhancement in step S4 can be expressed as:

[0097]

[0098] in, is a parameter map of the same size as a given image in the adaptive adjustment curve that can be learned by the reinforcement network, It is the last enhanced image in all iterations of the network. The enhanced network E-Net contains seven layers of depth-separable convolution. For the input image, it passes through the first four layers of convolution in sequence. The input of the fifth layer of convolution is the concatenation of the outputs of the third and fourth layers, the input of the sixth layer of convolution is the concatenation of the outputs of the fifth and second layers, and the input of the seventh layer of convolution is the concatenation of the outputs of the sixth and first layers. The first six layers use the ReLu function as the activation function after each convolution, and the seventh layer uses the tanh function as the activation function after convolution. The output of the seventh layer of convolution is used as the parameter diagram of the adaptive adjustment curve , according to the curve, the illumination component is adaptively enhanced to obtain enhanced_L.

[0099] In this embodiment, step S5 can obtain the final enhanced image E according to the outputs of the two stages. The weights of each loss term in the loss function of the second stage are all 1, which can be expressed as:

[0100]

[0101] in, represents the mean absolute error, represents the structural similarity loss between the enhanced image E and the reference image H, , , They represent color loss, brightness loss, and structure loss respectively.

[0102] The color loss It can be expressed as:

[0103]

[0104] in, and Represents the pixel vectors of the xth row and yth column of the enhanced image and the reference image respectively, u and v represent that the image has u rows and v columns, Represents the cosine similarity of two vectors.

[0105] The brightness loss It can be expressed as:

[0106]

[0107] Among them, c represents the three color channels of R, G, and B. and In pixels and The pixel block at the center.

[0108] The structural loss It can be expressed as:

[0109]

[0110] In this embodiment, the Pytorch deep learning framework and two Nvidia Geforce GTX 1660Ti GPUs are used for network training. The low-light image pairs collected in step S1 and the corresponding reference images are sequentially sent to the network models of the first and second stages. In the first stage, the input low-light image pairs are randomly cropped to a size of 128×128 pixels, the training batch size is 8, and the network is trained using the Adam optimizer with an initial learning rate of 1×10-4, and the number of training iterations is set to 300. In the second stage, the input low-light image and the reference image are used as image pairs for training. The low-light image is decomposed into an illumination component and a reflection component through the network trained in the first stage. The illumination component is sent to the enhancement network of the second stage for enhancement and multiplied element-by-element with the reflection component to obtain the final enhanced image. The training batch size is 8, and the network is trained using the Adam optimizer with an initial learning rate of 1×10-4, and the number of training iterations is set to 300.

[0111] In order to compare the experimental results of this embodiment with the advanced low-light image enhancement methods in recent stages, the widely used evaluation indicators Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) are used to quantitatively evaluate the experimental results.

[0112] The peak signal-to-noise ratio PSNR can be expressed as:

[0113] MSE = 1 H · W ∑ x = 0 H − 1 ∑ y = 0 W − 1 [ X ( x , y ) − Y ( x , y ) ] 2

[0114]

[0115] in, represents the mean square error of image X and Y, H and W represent the length and width of the image respectively, Indicates the maximum pixel value of image X. The larger the PSNR value, the better the image quality.

[0116] The structural similarity SSIM can be expressed as:

[0117]

[0118] in, and Represent the mean of images X and Y respectively, and Represent the variance of images X and Y respectively, represents the covariance, and is a non-zero constant. The value range of SSIM is [0,1]. The closer it is to 1, the higher the similarity between images X and Y.

[0119] The experimental results obtained according to the lightweight low-light image enhancement method provided by the example of the present invention are as follows: Figure 5 The experimental results of this embodiment are compared with the advanced low-light image enhancement methods in the past five years on the test set of the LOL dataset, and the results are shown in Table 1. The bold fonts in the table represent the best results, and the underlined fonts represent the suboptimal results. The lightweight low-light image enhancement method proposed in this embodiment has achieved good evaluation indicators while having a small number of network parameters and a simple network structure.

[0120] algorithm PSNR SSIM LIME-2017 16.76 0.445 EnlightenGAN-2019 17.48 0.651 Zero-DCE-2020 14.86 0.562 DRBN-2020 16.29 0.551 RRDNet-2020 11.39 0.468 RUAS-2021 18.03 0.697 SCI-2022 14.78 0.522 PairLIE-2023 18.74 0.716 Self-DACE-2023 19.15 0.724 FLW-Net-2023 22.84 0.83 Ours <![CDATA[ 19.98 ]]> <![CDATA[ 0.763 ]]>

[0121] Table 1 Evaluation indicators of different algorithms on the LOL test set

[0122] Example 2

[0123] This embodiment discloses a low-illumination image adaptive enhancement device, including:

[0124] An enhancement module, used for inputting the low-light image to be enhanced into a pre-trained low-light image adaptive enhancement model to obtain an enhanced image;

[0125] The training method of the low-light image adaptive enhancement model includes:

[0126] Obtain a training set including low-light images and corresponding reference images;

[0127] A low-illumination image adaptive enhancement model is constructed based on the Retinex algorithm; wherein the low-illumination image adaptive enhancement model includes: a Projection module, an Illumination module, a Reflectance module and an enhancement module;

[0128] The low-illumination image adaptive enhancement model is trained using the training set to obtain a trained low-illumination image adaptive enhancement model.

[0129] Example 3

[0130] This embodiment discloses a computer device, including a processor and a storage medium;

[0131] The storage medium is used to store instructions;

[0132] The processor is used to operate according to the instruction to implement the steps of the low-illumination image adaptive enhancement method as described in the first aspect.

[0133] Example 4

[0134] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the low-illumination image adaptive enhancement method as described in the first aspect.

[0135] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0136] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0137] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0138] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0139] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A low-light image adaptive enhancement method, It is characterized in that include: Inputting the low-light image to be enhanced into the pre-trained low-light image adaptive enhancement model to obtain an enhanced image; The training method of the low-light image adaptive enhancement model includes: Obtain a training set including low-light images and corresponding reference images; A low-illumination image adaptive enhancement model is constructed based on the Retinex algorithm; wherein the low-illumination image adaptive enhancement model includes: a projection module, an illumination component module, a reflection component module and an enhancement module; The low-illumination image adaptive enhancement model is trained using the training set to obtain a trained low-illumination image adaptive enhancement model.

2. According to claim 1, the lightweight low-light image adaptive enhancement method based on Retinex, It is characterized in that The projection module includes five layers of 3×3 convolutions; the first four convolution layers use the ReLU function as an activation function, the output of the fifth convolution layer is residually connected with the input image of the projection module, and the projection module ends with a Sigmoid function; The illumination component module includes five layers of 3×3 convolutions; wherein the first four convolution layers use the ReLU function as an activation function, the output of the fifth convolution layer is residually connected with the input image of the illumination component module, and the illumination component module ends with a Sigmoid function; The reflection component module includes five layers of 3×3 convolution and a channel attention module; wherein the first four convolution layers use the ReLU function as the activation function, the output of the fifth convolution layer is sent to the channel attention module and then residually connected with the input image of the reflection component module, and the reflection component module ends with a Sigmoid function; the channel attention module includes global average pooling and two layers of 1×1 convolution; The enhancement module includes seven layers of depth-separable convolutions; wherein the input of the fifth convolution layer is the concatenation of the outputs of the third and fourth layers, the input of the sixth convolution layer is the concatenation of the outputs of the fifth and second layers, and the input of the seventh convolution layer is the concatenation of the outputs of the sixth and first layers; the first six layers use the ReLu function as the activation function, and the seventh layer uses the tanh function as the activation function.

3. The lightweight low-light image adaptive enhancement method based on Retinex according to claim 1, It is characterized in that The method of using the training set to train the low-light image adaptive enhancement model includes: Input the low illumination image pair I1 and I2 into the projection module to obtain the projection image pair i1 and i2; Input the projection image pair i1, i2 into the illumination component module and the reflection component module for decomposition to obtain illumination components L1, L2 and reflection components R1, R2; The illumination component L1 is input into the enhancement network to obtain the illumination component enhanced_L enhanced by the adaptive adjustment curve; The illuminance component enhanced_L enhanced by the adaptive adjustment curve and the reflection component R1 are multiplied element by element to obtain an enhanced image E.

4. The lightweight low-light image adaptive enhancement method based on Retinex according to claim 1, It is characterized in that The low-light images and the corresponding reference images are: a pair of low-light images of the same scene that are underexposed but have different exposure levels and a corresponding reference image with normal exposure, which are obtained from the SICE public dataset and the LOL public dataset.

5. The lightweight low-light image adaptive enhancement method based on Retinex according to claim 3, It is characterized in that The step of inputting the low-illumination image pair I1 and I2 into the projection module to obtain the projection image pair i1 and i2 includes: Input the low-light image pair I1 and I2 into the projection module, remove the noise and features that are not suitable for Retinex decomposition, and obtain the optimized low-light image as the projection image pair i1 and i2; Among them, the projection loss function is used To remove noise and features that are not suitable for Retinex decomposition, the projection loss function for: In the formula, Represents low-light images, represents the projected image; The projected image pair i1, i2 is input into the illumination component module and the reflection component module for decomposition to obtain illumination components L1, L2 and reflection components R1, R2, including: Consistency loss through reflection The loss function consisting of the basic constraints Decomposing the projected image; in, and It is expressed as: Among them, L represents the illumination component, R represents the reflection component, and i represents the projected image; and represents the reflection component of the low-light image pair; Represents the initial estimate of the illumination component, obtained by calculating the maximum value of the R, G, and B channels; Represents horizontal and vertical gradients; It means that the network training should not propagate the gradient information into the illumination component when calculating here; The loss function of this step is expressed as: in, , , is the weighted weight.

6. The lightweight low-light image adaptive enhancement method based on Retinex according to claim 3, It is characterized in that The step of inputting the illumination component L1 into the enhancement network to obtain the illumination component enhanced_L enhanced by the adaptive adjustment curve includes: The illuminance component is iteratively enhanced using an adaptive adjustment curve; wherein the adaptive adjustment curve is expressed as: in, is a parameter map of the same size as a given image in the adaptive adjustment curve that can be learned by the reinforcement network, It is the previous enhanced image in all iterations of the network.

7. The lightweight low-light image adaptive enhancement method based on Retinex according to claim 6, It is characterized in that The illumination component enhanced_L enhanced by the adaptive adjustment curve, the loss function of this step is expressed as: in, represents the mean absolute error, represents the structural similarity loss between the enhanced image E and the reference image H, , , They represent color loss, brightness loss, and structure loss respectively; The color loss It is expressed as: The brightness loss It is expressed as: The structural loss It is expressed as: in, and Represents the pixel vectors of the xth row and yth column of the enhanced image and the reference image respectively, u and v represent that the image has u rows and v columns, represents the cosine similarity of two vectors; c represents the three color channels of R, G, and B. and In pixels and The pixel block at the center.

8. A low-light image adaptive enhancement device, It is characterized in that include: An enhancement module, used for inputting the low-light image to be enhanced into a pre-trained low-light image adaptive enhancement model to obtain an enhanced image; The training method of the low-light image adaptive enhancement model includes: Obtain a training set including low-light images and corresponding reference images; A low-illumination image adaptive enhancement model is constructed based on the Retinex algorithm; wherein the low-illumination image adaptive enhancement model includes: a projection module, an illumination component module, a reflection component module and an enhancement module; The low-illumination image adaptive enhancement model is trained using the training set to obtain a trained low-illumination image adaptive enhancement model.

9. A computer device, It is characterized in that including processor and storage medium; The storage medium is used to store instructions; The processor is used to operate according to the instruction to implement the steps of the low-illumination image adaptive enhancement method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the low-illumination image adaptive enhancement method according to any one of claims 1 to 7 is implemented.

Citation Information

Cited By

  • Low-illumination image enhancement method and device, storage medium and computer equipment

    CN120259154A