Low-light adjustable brightness enhancement method based on reference brightness index

Abstract

This invention relates to a low-light adjustable brightness enhancement method guided by a reference brightness index, belonging to the field of computer vision image technology. The first step involves extracting features from the input low-light image and the reference brightness index using methods such as convolution, downsampling, and global average pooling to obtain a feature vector. The second step decomposes the feature vector into brightness and content feature components, combining the content component of the low-light image with the brightness component of the reference brightness index to achieve feature recombination. The third step reconstructs the recombinated feature vector using methods such as transposed convolution, upsampling, and skip connections. This invention is rationally designed, fully considering the different lighting needs of various application scenarios or users. It efficiently utilizes and preserves the brightness and content information of the image. Furthermore, the network complexity is low, achieving good brightness enhancement while maintaining a fast running speed, resulting in good overall performance in adjustable brightness enhancement of low-light images.

Description

Technical Field

[0001] This invention belongs to the field of computer vision image technology, and in particular to a low-light adjustable brightness enhancement method based on a reference brightness index. Background Technology

[0002] With the advancement of science and technology and the development and widespread use of mobile devices, people have found it increasingly convenient to acquire images. However, during image acquisition, image quality often degrades due to limitations of the camera equipment and uncontrolled shooting environments and lighting conditions. For example, smartphone cameras have small apertures and sensors, resulting in limited dynamic range; insufficient ambient light leads to noise and color distortion in the image. These not only affect the visual effect of the image, making the information presentation incomplete, but also limit subsequent applications. To improve image quality, post-processing is often required for image enhancement. Since the perception of image visual quality is a cognitive task related to factors such as application scenarios, user personality, and aesthetics, the image enhancement process should be application-scenario- or user-specific and adjustable. Although existing professional software provides tools for manually adjusting images, these tools are either complex to operate, requiring professional knowledge, or simple to operate but with poor enhancement effects. Therefore, how to efficiently and adjustablely enhance the brightness of low-quality, low-light images according to different application scenarios or user needs is an important but challenging research topic in the field of computer vision.

[0003] Low-light image enhancement addresses the problems of low brightness, low contrast, and noise in poorly lit images, restoring image brightness and improving visual quality. Low-light image enhancement is a subjective process; different application scenarios or users have different definitions of normal brightness levels, resulting in varying required brightness levels. Therefore, adjustable brightness enhancement networks greatly enhance practicality. Low-light image enhancement provides crucial technical support for computers to better observe, analyze, and process images, and has significant application value in many fields, including medical imaging and surveillance systems.

[0004] Traditional low-light image enhancement methods primarily focus on increasing image contrast. Histogram equalization (HE) and its variants expand the limited dynamic range of an image by adjusting its histogram; Retinex Theory views an image as a product of a reflective layer and an illumination layer, enhancing image brightness by estimating and adjusting the illumination layer. However, these methods cannot handle the complex mapping from dark to normal brightness in low-light conditions, and therefore cannot guarantee effective image enhancement.

[0005] Benefiting from the development of deep learning, an increasing number of researchers are applying neural networks to nonlinear low-light enhancement problems. Examples include low-light enhancement techniques such as LLNet (training convolutional neural networks with data simulated using random gamma correction), EnlightenGAN (training generative adversarial networks with unpaired data), Zero-DCE (learning the mapping from image to curve using convolutional neural networks and applying the curve to the original image to adjust pixel values), and DeepUPE (estimating the mapping from image to illumination map using convolutional neural networks). However, these methods can only achieve fixed brightness enhancement, i.e., a one-to-one mapping, without considering the different brightness requirements of the application scenario or users. KinD (decomposing an image into a reflection layer and an illumination layer using convolutional neural networks, performing denoising in the reflection layer, and adjusting brightness in the illumination layer using a scaling factor) can adjust the brightness level within a limited range, but the brightness level of the image observed by the human eye is not linearly related to the scaling factor. PieNet, a low-light enhancement technique that uses a convolutional network to learn a preference vector representing user preferences to guide the brightness enhancement process, can generate a user-specific enhanced image from multiple preferred images selected by the user. However, the preparation and operation are relatively complex, and in addition to brightness information, the preference vector also learns color information, which leads to color distortion in the enhanced image. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art and propose a low-illuminance adjustable brightness enhancement method that is reasonably designed, fully considers subjective factors, and is relatively lightweight, based on a reference brightness index.

[0007] The technical problem solved by this invention is achieved through the following technical solution:

[0008] A low-light adjustable brightness enhancement method based on a reference brightness index includes the following steps:

[0009] Step 1: In the image input stage, image enhancement techniques are randomly applied to transform the sample content;

[0010] Step 2: Input the low-light image and reference brightness index into the network and extract features respectively. This stage includes operations such as convolution, downsampling and global average pooling, and outputs feature vectors of the low-light image and reference brightness index.

[0011] Step 3: Decompose the two feature vectors obtained in Step 2 into brightness feature components and content feature components respectively. Through cutting and splicing operations, the feature vectors are recombined, and the output is a feature vector composed of the brightness component of the reference index and the content component of the dark image.

[0012] Step 4: Reconstruct the recombined feature vector obtained in Step 3. This stage includes operations such as transposed convolution, upsampling, and skip connections, and outputs a feature map with enhanced brightness to a level close to the reference level.

[0013] The feature maps obtained in steps 5 and 4 are processed by convolution and mapped to the output feature dimension. The number of output channels of the convolution kernel is the same as the number of channels of the input image. The final output is a normal brightness image with brightness similar to the reference index.

[0014] Step 6: Re-feed the output image from Step 5 into the network. Extract features through convolution, downsampling, and global average pooling, and then crop to obtain two feature components. In the latent space, calculate the loss function using the feature components obtained in this step and Step 3 to optimize the network.

[0015] Furthermore, the specific method for data augmentation in step 1 includes the following steps:

[0016] (1) If it is in the testing phase, proceed directly to step 2; if it is in the training phase, perform the following operations.

[0017] (2) For the input dark light image and the reference brightness image, determine their flipping method with a probability of 1 / 4 each, including no flipping, horizontal flipping, vertical flipping and horizontal and vertical flipping. The two images have the same flipping method.

[0018] (3) In the flipped input image, randomly locate an image block with a size of 100×100, and the position corresponds to that in the dark image and the reference brightness image;

[0019] (4) For the located image patch, this part of the dark image is replaced with the content of the reference brightness image patch at the corresponding location for supervision. That is, the network learns the identity mapping of these pixels, which plays a regularization role in the network learning. It restricts the enhancement processing to only where brightness enhancement is needed, avoiding overexposure (over-enhancement) in the processed image. It forces the network not only to learn how to enhance brightness, but also to learn where to enhance brightness.

[0020] Furthermore, the specific details of step 2 include the following:

[0021] (1) If it is in the testing phase, the reference brightness index is any normal brightness index; if it is in the training phase, the reference brightness index is the supervised normal brightness image corresponding to the dark image.

[0022] (2) The feature extraction stage is downsampled step by step, which can be regarded as an encoder. There are four downsampling layers and one global average pooling layer.

[0023] (3) Each downsampling layer first performs two 3×3 convolutions to further extract features. The convolution does not change the feature map size, but changes the number of output channels to 32, 64, 128 and 256 respectively. Then, max pooling is used to reduce the feature map size to 1 / 2 of the original size, compressing and fusing spatial features, preserving texture content, expanding the receptive field of the convolutional network, and extracting more semantic information.

[0024] (4) Before global average pooling, the feature map after four layers of downsampling is first convolved once with a 3×3 to change its number of channels to 512. Then global average pooling is used to transform the feature map into a 512-dimensional feature vector, which is used as the output of this stage.

[0025] Furthermore, the specific details of step (1) include the following:

[0026] ① The reference brightness index used in the testing phase can be any normal brightness image, a content-free image, and the average brightness value of the image;

[0027] ② If the reference brightness index is a normal brightness image, then directly input it into the network to extract brightness features;

[0028] ③ If the reference brightness index is a blank image, such as a white image, then directly input it into the network to extract brightness features;

[0029] ③ If the reference brightness index is the average brightness value of the image, then the value is first expanded and replicated to become an image with the same size and number of channels as the dark image, and then input into the network to extract brightness features.

[0030] Furthermore, the specific details of step 3 include the following:

[0031] (1) Decompose the feature vector and restrict its first 96 dimensions to represent image brightness information and its 97-512 dimensions to represent image content information;

[0032] (2) By cropping, the brightness feature components and content feature components of the dark image and the reference brightness index are obtained respectively;

[0033] (3) The brightness component of the reference brightness index and the content component of the dark image are spliced ​​together to obtain a new 512-dimensional feature vector. The recombined vector contains target brightness information and content information that is invariant before and after brightness enhancement.

[0034] Furthermore, the specific details of step 4 include the following:

[0035] (1) The image reconstruction stage is upsampled step by step, which can be regarded as a decoder. There are four upsampling layers, which correspond one-to-one with the downsampling in step 2.

[0036] (2) Before upsampling, the 512-dimensional feature vector is first expanded into a target feature map by dimensional copying. This feature map has the same size and number of channels as the feature map before the fourth downsampling in the feature extraction stage.

[0037] (3) Each layer upsampling first uses transposed convolution to upsample the feature map by 2 times, and concatenates it with the output of the corresponding layer with the same size in the channel dimension to timely supplement the spatial information lost due to downsampling during feature extraction, and at the same time merge deep semantic feature information.

[0038] (4) Perform two 3×3 convolutions on the concatenated feature map to further reconstruct the image information. Repeat this process four times and use the feature map after the fourth layer upsampling as the output of this stage.

[0039] Furthermore, the specific details of step (2) include the following:

[0040] ① For the input 512-dimensional feature vector, the number of channels is changed to 256 by passing it through a fully connected layer;

[0041] ② Expand the 256-dimensional feature vector to a 1×1×256 feature map;

[0042] ③ The feature map obtained in ② is copied dimensionally to obtain a target feature map with size H×W×256, where H and W are the height and width of the feature map before the fourth downsampling in the feature extraction stage, respectively.

[0043] Furthermore, the specific details of step 6 include the following:

[0044] (1) If in the testing phase, ignore this step and step 5 is the final step; if in the training phase, perform the following operations.

[0045] (2) Perform the same operation as step 2 on the normal brightness image sent to the feature extraction stage to obtain a 512-dimensional feature vector.

[0046] (3) Perform the same operation on the feature vector as in step (2) in step 3 to obtain the brightness component and content component representing different information of the image;

[0047] (4) Calculate the mean square error loss using the content components in (3) and the content components of the dark light image in step (2) of step 3;

[0048] (5) Calculate the triplet loss using the luminance component from (3) and the luminance components from the dark image and the reference luminance image from step (2) of step 3.

[0049] The advantages and positive effects of this invention are:

[0050] 1. The present invention is reasonably designed and fully considers that low-light image enhancement is a subjective process. Since different application scenarios or users usually have different definitions of normal lighting and different required brightness levels, a reference brightness index is introduced as a standard for brightness enhancement in the process of low-light image enhancement. This allows the network to efficiently utilize the brightness information of the reference index while preserving the content information of the original dark image to the greatest extent.

[0051] 2. This invention divides the low-light adjustable enhancement process guided by a reference brightness index into three stages. The first stage extracts features from the low-light image and the reference brightness index. The second stage reconstructs the extracted feature vectors, preserving the content information of the original low-light image and incorporating the brightness information of the reference index. The third stage reconstructs the reconstructed feature vectors, ultimately outputting a normal brightness image similar to the reference index brightness. The network can change the enhancement effect of the low-light image by inputting reference indices of different brightness levels to meet the needs of different application scenarios or users, achieving both brightness enhancement and adjustable brightness. Furthermore, due to the low network complexity, the algorithm's running speed is considered while ensuring the enhancement effect. Attached Figure Description

[0052] Figure 1 This is a diagram of the backbone framework of the low-light adjustable brightness enhancement network of the present invention;

[0053] Figure 2 This is a framework diagram of the feature extraction and image reconstruction module of the present invention; Detailed Implementation

[0054] The embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0055] A low-light adjustable brightness enhancement method guided by a reference brightness index, such as Figures 1 to 2 As shown, it includes the following steps:

[0056] Step S1: In the image input stage, image enhancement technology is randomly applied to transform the sample content;

[0057] Step S2: Input the low-light image and the reference brightness index into the network and extract features respectively. This stage includes operations such as convolution, downsampling and global average pooling, and outputs feature vectors of the low-light image and the reference brightness index.

[0058] Step S3: Decompose the two feature vectors obtained in step S2 into brightness feature components and content feature components respectively. Through cutting and splicing operations, the feature vectors are recombined, and the output is a feature vector composed of the brightness component of the reference index and the content component of the dark image.

[0059] Step S4: Reconstruct the recombined feature vector obtained in step S3. This stage includes operations such as transposed convolution, upsampling, and skip connections, and outputs a feature map with enhanced brightness to a level close to the reference level.

[0060] The feature maps obtained in steps S5 and S4 are processed by convolution and mapped to the output feature dimension. The number of output channels of the convolution kernel is the same as the number of channels of the input image. Finally, the output is a normal brightness image with brightness similar to the reference index.

[0061] Step S6: The output image from step S5 is re-input into the network. Features are extracted through convolution, downsampling, and global average pooling, and two feature components are obtained by cropping. In the latent space, the loss function is calculated using the feature components obtained in this step and step S3 to optimize the network.

[0062] The specific implementation method of step S2 is as follows:

[0063] Step S2.1: If in the testing phase, the reference brightness index is any normal brightness index; if in the training phase, the reference brightness index is the supervised normal brightness image corresponding to the dark image.

[0064] Step S2.2: This feature extraction stage involves progressive downsampling, which can be considered as an encoder. It consists of four downsampling layers and one global average pooling layer.

[0065] Step S2.3: For each downsampling layer, two 3×3 convolutions are performed first to further extract features. The convolution does not change the feature map size, but changes the number of output channels to 32, 64, 128 and 256 respectively. Then, max pooling is used to reduce the feature map size to 1 / 2 of the original size to compress and fuse spatial features, preserve texture content, expand the receptive field of the convolutional network, and extract more semantic information.

[0066] Step S2.4: Before global average pooling, first perform a 3×3 convolution on the feature map after four layers of downsampling to change its channel number to 512, and then use global average pooling to transform the feature map into a 512-dimensional feature vector, which is used as the output of this stage.

[0067] The specific implementation method of step S2.1 is as follows:

[0068] Step S2.1.1: The reference brightness index used in the testing phase can be any normal brightness image, a content-free image, and the average brightness value of the image;

[0069] Step S2.1.2: If the reference brightness index is a normal brightness image, then directly input it into the network to extract brightness features;

[0070] Step S2.1.3: If the reference brightness index is a blank image, such as a white image, then directly input it into the network to extract brightness features;

[0071] Step S2.1.4: If the reference brightness index is the average brightness value of the image, then first perform dimensional expansion and dimension replication operations on this value to transform it into an image with the same size and number of channels as the dark light image, and then input it into the network to extract brightness features.

[0072] The specific implementation method of step S3 is as follows:

[0073] Step S3.1: Decompose the feature vector, restricting its first 96 dimensions to represent image brightness information, and dimensions 97-512 to represent image content information;

[0074] Step S3.2: By cropping, the luminance feature components and content feature components of the dark image and the reference luminance index are obtained respectively;

[0075] Step S3.3: The luminance component of the reference luminance index and the content component of the dark image are concatenated to obtain a new 512-dimensional feature vector. The recombined vector contains target luminance information and content information that is invariant before and after luminance enhancement.

[0076] The specific implementation method of step S4 is as follows:

[0077] Step S4.1: This image reconstruction stage involves progressive upsampling, which can be considered as a decoder. There are four upsampling layers, which correspond one-to-one with the downsampling in step S2.

[0078] Step S4.2: Before upsampling, the 512-dimensional feature vector is first expanded into a target feature map by dimensional copying. This feature map has the same size and number of channels as the feature map before the fourth downsampling in the feature extraction stage.

[0079] Step S4.3: For each layer upsampling, transposed convolution is first used to upsample the feature map by 2 times, and concatenate it with the output of the corresponding layer with the same size in the channel dimension to timely supplement the spatial information lost due to downsampling during feature extraction, and at the same time merge deep semantic feature information.

[0080] Step S4.4: Perform two 3×3 convolutions on the concatenated feature map to further reconstruct the image information. Repeat this process four times and use the feature map after the fourth layer upsampling as the output of this stage.

[0081] The specific implementation method of step S4.2 is as follows:

[0082] Step S4.2.1: For the input 512-dimensional feature vector, pass it through a fully connected layer to change the number of channels to 256;

[0083] Step S4.2.2: Expand the 256-dimensional feature vector to a 1×1×256 feature map;

[0084] Step S4.2.3: Copy the feature map obtained in step S4.2.2 by dimension to obtain a target feature map with size H×W×256, where H and W are the height and width of the feature map before the fourth downsampling in the feature extraction stage, respectively.

[0085] The specific implementation method of step S6 is as follows:

[0086] Step S6.1: If in the testing phase, ignore this step; step S5 is the final step. If in the training phase, perform the following operations.

[0087] Step S6.2: Perform the same operation as step S2 on the normal brightness image sent to the feature extraction stage to obtain a 512-dimensional feature vector;

[0088] Step S6.3: Perform the same operation as in step S3.2 on the feature vector to obtain the brightness component and content component representing different information of the image;

[0089] Step S6.4: Calculate the mean square error loss using the content components from step S6.3 and the content components of the dark light image from step S3.2;

[0090] Step S6.5: Calculate the triplet loss using the luminance components from step S6.3 and the luminance components from the dark image and the reference luminance image from step S3.2.

[0091] By following the steps above, you can obtain a normal brightness image that is similar in brightness to the reference image.

[0092] Finally, we train the network with the goal of minimizing the absolute error loss function (L1 loss function), mean squared error loss function (L2 loss function), triplet loss function, and cosine loss function. We quantitatively evaluate the network performance using PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity Index), and qualitatively evaluate the network performance through visual results. The method is as follows:

[0093] Test environment: Python 3.7; TensorFlow framework; Ubuntu 16.04 system; NVIDIA GTX 2080ti GPU

[0094] Test sequences: The selected datasets are the Low-Light dataset (LOL) for low-light enhancement, which contains 500 pairs of low-light-normal-light image pairs; and the high-quality dataset (MIT Adobe 5K) for image enhancement, which contains 5 sets (A / B / C / D / E) of 5000 pairs of low-light-normal-light image pairs.

[0095] Testing method: 15 image pairs were selected from LOL and 500 image pairs from group C in MIT Adobe 5K were selected as the test set to quantitatively and qualitatively evaluate the network performance.

[0096] Test metrics: This invention uses PSNR and SSIM metrics for evaluation, calculates metric data for different popular algorithms and then compares the results. In addition, the visual effects of the same image enhanced by different methods are compared, proving that this invention can achieve good results in the field of low-light enhancement, and can output enhancement results of different brightness levels based on the reference image.

[0097] The test results are as follows:

[0098] Table 1. Performance comparison of the present invention with the SOTA method on the LoL dataset.

[0099] Method CRM Dong LIME MF Retinex MSR NPE GLAD KinD MIRNet Ours PSNR 17.20 16.72 16.76 18.79 16.77 13.17 16.97 19.72 20.87 24.14 27.90 SSIM 0.64 0.58 0.56 0.64 0.56 0.48 0.59 0.70 0.80 0.83 0.86

[0100] Table 2. Performance comparison of the present invention with the SOTA method on the LoL dataset.

[0101] Method HDRNet W-Box DR DPE DeepUPE MIRNet Ours PSNR 21.96 18.57 20.97 22.15 23.04 23.73 28.93 SSIM 0.866 0.701 0.841 0.850 0.893 0.925 0.952

[0102] Any aspects not covered in this invention are applicable to existing technologies.

[0103] It should be emphasized that the embodiments described in this invention are illustrative rather than limiting. Therefore, this invention includes, but is not limited to, the embodiments described in the specific implementation. Any other implementations derived by those skilled in the art based on the technical solutions of this invention are also within the scope of protection of this invention.

Claims

1. A low-illuminance adjustable brightness enhancement method guided by a reference brightness index, characterized in that... Includes the following steps: Step 1: In the image input stage, image enhancement techniques are randomly applied to transform the sample content; Step 2: Input the low-light image and reference brightness index into the network and extract features respectively. This stage includes operations such as convolution, downsampling and global average pooling, and outputs feature vectors of the low-light image and reference brightness index. Step 3: Decompose the two feature vectors obtained in Step 2 into brightness feature components and content feature components respectively. Through cutting and splicing operations, the feature vectors are recombined, and the output is a feature vector composed of the brightness component of the reference index and the content component of the dark image. Step 4: Reconstruct the recombined feature vector obtained in Step 3. This stage includes transposed convolution, upsampling, and skip connection operations, and outputs a feature map with enhanced brightness to the same level as the reference. The feature maps obtained in steps 5 and 4 are processed by convolution and mapped to the output feature dimension. The number of output channels of the convolution kernel is the same as the number of channels of the input image. The final output is a normal brightness image with brightness similar to the reference index. Step 6: Re-feed the output image from Step 5 into the network. Extract features from it through convolution, downsampling, and global average pooling operations. Obtain two feature components by cropping. In the latent space, calculate the loss function using the feature components obtained in this step and Step 3 to optimize the network.

2. The low-light adjustable brightness enhancement method based on a reference brightness index according to claim 1, characterized in that, The specific method for data augmentation in step 1 includes the following steps: (1) If it is in the testing phase, proceed directly to step 2; if it is in the training phase, perform the following operations; (2) For the input dark light image and the reference brightness image, determine their flipping method with a probability of 1 / 4 each, including no flipping, horizontal flipping, vertical flipping and horizontal and vertical flipping. The two images have the same flipping method. (3) In the flipped input image, randomly locate an image block with a size of 100×100, and the position corresponds to that in the dark image and the reference brightness image; (4) For the located image block, this part of the dark image is replaced with the content of the reference brightness image block at the corresponding location for supervision. That is, the network learns the identity mapping of this part of the pixels, which plays a regularization role in the network learning, restricting it to only perform enhancement processing where brightness needs to be enhanced, avoiding overexposure of the processed image, and forcing the network not only to learn how to enhance brightness, but also to learn where to enhance brightness.

3. The low-light adjustable brightness enhancement method based on a reference brightness index according to claim 1, characterized in that, The specific details of step 2 include the following: (1) If it is in the testing phase, the reference brightness index is any normal brightness index; if it is in the training phase, the reference brightness index is the supervised normal brightness image corresponding to the dark image. (2) The feature extraction stage is downsampled step by step, which can be regarded as an encoder. There are four downsampling layers and one global average pooling layer. (3) Each downsampling layer first performs two 3×3 convolutions to further extract features. The convolution does not change the feature map size, but changes the number of output channels to 32, 64, 128 and 256 respectively. Then, max pooling is used to reduce the feature map size to 1 / 2 of the original size, compressing and fusing spatial features, preserving texture content, expanding the receptive field of the convolutional network, and extracting more semantic information. (4) Before global average pooling, the feature map after four layers of downsampling is first convolved once with a 3×3 convolution to change its number of channels to 512. Then, global average pooling is used to transform the feature map into a 512-dimensional feature vector, which is used as the output of this stage.

4. The low-light adjustable brightness enhancement method based on a reference brightness index according to claim 3, characterized in that, The specific details of step (1) include the following: ① The reference brightness index used in the testing phase can be any normal brightness image, a content-free image, and the average brightness value of the image; ② If the reference brightness index is a normal brightness image, then directly input it into the network to extract brightness features; ③ If the reference brightness index is an image without content, then directly input it into the network to extract brightness features; ④ If the reference brightness index is the average brightness value of the image, then the value is first expanded and replicated to become an image with the same size and number of channels as the dark image, and then input into the network to extract brightness features.

5. The low-light adjustable brightness enhancement method based on a reference brightness index according to claim 1, characterized in that, The specific details of step 3 include the following: (1) Decompose the feature vector and restrict its first 96 dimensions to represent image brightness information and its 97-512 dimensions to represent image content information; (2) By cropping, the brightness feature components and content feature components of the dark image and the reference brightness index are obtained respectively; (3) The brightness component of the reference brightness index and the content component of the dark image are spliced ​​together to obtain a new 512-dimensional feature vector. The recombined vector contains target brightness information and content information that is invariant before and after brightness enhancement.

6. The low-light adjustable brightness enhancement method based on a reference brightness index according to claim 1, characterized in that, The specific details of step 4 include the following: (1) The image reconstruction stage is upsampled step by step, which can be regarded as a decoder. There are four upsampling layers, which correspond one-to-one with the downsampling in step 2. (2) Before upsampling, the 512-dimensional feature vector is first expanded into a target feature map by dimensional copying. This feature map has the same size and number of channels as the feature map before the fourth downsampling in the feature extraction stage. (3) Each layer upsampling first uses transposed convolution to upsample the feature map by 2 times, and concatenates it with the output of the corresponding layer with the same size in the channel dimension to timely supplement the spatial information lost due to downsampling during feature extraction, and at the same time merge deep semantic feature information. (4) Perform two 3×3 convolutions on the concatenated feature map to further reconstruct the image information. Repeat this process four times and use the feature map after upsampling in the fourth layer as the output of this stage.

7. The low-light adjustable brightness enhancement method based on a reference brightness index according to claim 6, characterized in that, The specific details of step (2) include the following: ① For the input 512-dimensional feature vector, the number of channels is changed to 256 by passing it through a fully connected layer; ② Expand the 256-dimensional feature vector to a 1×1×256 feature map; ③ The feature map obtained in ② is copied dimensionally to obtain a target feature map with size H×W×256, where H and W are the height and width of the feature map before the fourth downsampling in the feature extraction stage, respectively.

8. The low-light adjustable brightness enhancement method based on a reference brightness index according to claim 1, characterized in that, The specific details of step 6 include the following: (1) If in the testing phase, ignore this step and step 5 is the final step; if in the training phase, perform the following operations. (2) Perform the same operation as step 2 on the normal brightness image sent to the feature extraction stage to obtain a 512-dimensional feature vector; (3) Perform the same operation as described in claim 4 on the feature vector to obtain the brightness component and content component representing different information of the image; (4) Calculate the mean square error loss using the content components in (3) and the content components of the dark light image in step (2) of claim 5; (5) Calculate the triplet loss using the luminance component from (3) and the luminance components from the dark image and the reference luminance image in step (2) of claim 5.

Images