Low-illumination image enhancement method without normal illumination reference based on HSV color space and Retinex theory

Abstract

The present invention designs a low-light image enhancement method without a normal exposure image reference based on the HSV color space and Retinex theory, which belongs to the field of computer vision image technology. For the low-light enhancement task without a normal exposure image reference, the present invention decomposes it into a color preservation task and a brightness enhancement task. For the color preservation task, the present invention separates the color information through the HSV color space and retains it on the enhanced image. For the brightness enhancement task, the present invention obtains an image pair of different illumination levels of the same scene by perturbing the separated brightness information, obtains the reflection component through the reflection consistency constraint in the Retinex theory, and uses the reflection component as the result of brightness enhancement. Experiments show that the present invention can effectively enhance low-light images and has strong generalization capabilities in various data sets and low-light images with different illumination levels.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision images, and in particular to a low-illumination image enhancement method without normal illumination reference based on HSV color space and Retinex theory. Background Art

[0002] As human material culture becomes richer, people are more inclined to use photography to record scenes in their lives. However, due to limitations in the scene or equipment, many photos are not taken under optimal lighting conditions. One of the main problems is that the photos are taken in a dim environment, resulting in low-light images. In low-light conditions, the aesthetic quality of the photos is reduced, and the information that humans can capture through photos is also limited. Therefore, how to efficiently perform brightness enhancement operations on low-light images, thereby improving image quality, is an important research topic in the field of computer vision.

[0003] Low-light image enhancement technology enhances the brightness of images captured in low-light conditions or underexposed images to produce properly exposed images. It is an important technical solution for brightness enhancement from a software perspective. Enhanced low-light images provide crucial technical support for better computer observation and analysis, and have significant application value in areas such as surveillance systems.

[0004] There are two main types of traditional low-light image enhancement methods: (1) Methods based on histogram equalization. Histogram equalization can expand the dynamic range of the image, thereby improving the brightness. Common methods include global histogram equalization and local histogram equalization. (2) Methods based on Retinex theory. Retinex theory means that an image can be decomposed into reflection component and illumination component, and the reflection component is considered to be consistent under different lighting conditions. Common methods include single-scale Retinex (SSR), which applies Gaussian smoothing to the illumination component; multi-scale Retinex (MSRCR), which adds multi-scale Gaussian smoothing and color restoration to SSR; a method that uses a weighted variational model to simultaneously estimate the illumination component and the reflection component (SRIE); a method that uses structural prior knowledge to only estimate the illumination component and uses the reflection component as the final result (LIME). Although these methods can also achieve good results, they are limited by the model capacity and are difficult to apply to a wider range of scenarios. In addition, the entire process requires people to carefully modify various parameters.

[0005] Convolutional neural networks are a type of neural network specifically designed to process data with a grid-like structure (for example, images can be considered as two-dimensional pixel grids). They have been successful in a large number of different types of computer vision processing tasks (such as image classification, target detection, etc.). Unlike traditional methods, people train convolutional neural networks with a large amount of data to allow the network to learn a complex mapping function between input and output. According to the data type of the training neural network, low-light image enhancement methods based on convolutional neural networks can be divided into: (1) supervised learning methods. Supervised methods refer to the need for paired low-light images and normal-light images in the process of training convolutional neural networks, calculating the difference between the output of the low-light image after passing through the network and the normal-light image, and further optimizing the neural network. For example, the low-light image and the normal-light image are first decomposed into illumination components and reflection components through the neural network, and the reflection components of the two are constrained to be consistent. Then, the illumination component of the low-light image is brightness-enhanced through the neural network, and recombined with the reflection component to obtain a brightness-enhanced image, and the brightness of the brightness-enhanced image is constrained to be consistent with the normal-light image (Retinex-net). Although supervised methods can achieve good results, obtaining paired low-light images and normal-light images is a difficult problem. One approach is to manually adjust the normal-light exposure images to obtain low-light images, but this type of dataset still has a certain gap with the real data. Another approach is to shoot a real paired dataset by adjusting the sensitivity, exposure time, etc., but there will be problems such as camera shake and scene changes, and the captured data needs to be carefully screened. (2) Learning methods based on generative adversarial networks (GANs). The advantage of generative adversarial networks is that training can be completed without paired images. The low-light image is first obtained by the generator to obtain a brightness-enhanced image, and then the loss is calculated by the discriminator and the real normal-light exposure image. Since it is not a paired image, the discriminator often measures the difference between the distributions of the two images. However, although the method based on generative adversarial networks can effectively avoid the need for paired datasets, the unpaired dataset also needs to be carefully selected, and the training process is relatively cumbersome, requiring iterative training of the generator and discriminator. (3) No-reference learning methods. Reference-free learning methods use only low-light images when training a neural network, placing constraints on the output images. These methods are also known as learning methods without reference to normal-light images. These methods can use constraints such as brightness control, spatial structure, color, and smoothness (Zero-DCE), or compare the results with histogram equalization. For example, a low-light image can be decomposed into an illumination component and a reflection component. By constraining the maximum channel of the reflection component (taking the maximum value of each pixel's RGB channel value to form a new channel) and performing histogram equalization on the maximum channel of the input low-light image, the final output is a brightness-enhanced reflection component.Since normal exposure images are not required for training, the number of low-light images that can be used as training sets can be greatly increased, making this training method a recent research hotspot. However, many theories used in supervised learning methods or those based on generative adversarial networks have not yet been applied to no-reference learning methods, and no-reference learning still has many different goals from supervised learning methods or those based on generative adversarial networks.

[0006] For the no-reference learning method, all information comes from the low-light image, so the color information should be retained as much as possible in the low-light image, rather than adjusting it to the color corresponding to the normal exposure image like the supervised learning method. Therefore, the present invention decomposes the low-light image enhancement task into a color retention task and a brightness enhancement task. For the color retention task, the color information is decoupled from the brightness information by converting the image to the HSV color space, and the color information is retained. For the brightness information, the present invention obtains a set of image pairs of the same scene with different lighting by perturbing it, and realizes brightness enhancement by using the reflection consistency in the Retinex theory and combining it with other losses. Summary of the Invention

[0007] The purpose of the present invention is to overcome the shortcomings of the existing technology and propose a rationally designed low-illumination image enhancement method without a normal illumination reference image based on the HSV color space and Retinex theory.

[0008] The present invention solves the technical problem by adopting the following technical solutions:

[0009] A low-light image enhancement method without normal illumination reference based on HSV color space and Retinex theory includes the following steps:

[0010] Step 1: At the image input stage, the low-light image is normalized from the pixel value range [0, 255] to [0, 1];

[0011] Step 2: Convert the image obtained in step 1 from RGB color space to HSV color space, and separate the three channels to obtain hue component (H), saturation component (S), and value component (V);

[0012] Step 3: Calculate the average value of the luminance component (V) obtained in step 2. For images with an average value ≤ 0.5, generate a power function with an exponent in the interval [0, 1]. For images with an average value > 0.5, generate a power function with an exponent in the interval [1, 5]. Adjust the luminance component (V) obtained in step 2 using the generated power function to obtain the perturbed luminance component (Vd).

[0013] Step 4: The luminance component (V) obtained in step 2 and the disturbed luminance component (Vd) obtained in step 3 are calculated using the Retinex theory to obtain the reflection component (R) of the luminance component and the reflection component (Rd) of the disturbed luminance component;

[0014] Step 5: Recombining the hue component (H) and saturation component (S) obtained in step 2 with the reflectance component (R) of the lightness component obtained in step 4 to obtain a three-channel image in the HSV color space;

[0015] Step 6: Convert the three-channel image in the HSV color space obtained in step 5 to the RGB color space, and adjust its pixel value range from [0, 1] to [0, 255] to obtain the enhanced low-light image.

[0016] Furthermore, the specific details of converting the low-light image from the RGB color space to the HSV color space in step 2 include the following:

[0017] (1) Calculate the maximum channel (c max ), minimum channel (c min ) and contrast (Δ): c max =max

[0018] (R,G,B), c min =min(R,G,B),Δ=c max -c min (where R, G, and B represent the three channels in the RGB color space, the same below);

[0019] (2) Calculate hue (H) according to the formula:

[0020]

[0021] (3) Calculate saturation (S) according to the formula:

[0022]

[0023] (4) Calculate the brightness (V) according to the formula:

[0024] V=c max .

[0025] Furthermore, the specific details of obtaining the reflection component by using the Retinex theory in step 4 include the following:

[0026] (1) The luminance component (V) and the perturbed luminance component (Vd) are fed into a convolutional neural network to obtain the inverse form of the illumination component of the luminance component (L) and the inverse form of the illumination component of the perturbed luminance component (Ld);

[0027] (2) The reflection component (R) of the luminance component and the reflection component (Rd) of the disturbed luminance component are obtained by the formula R = V·L;

[0028] (3) Optimize the network by calculating reflection consistency and other loss constraints R, Rd, L, Ld.

[0029] Furthermore, the specific details of the convolutional neural network in step (1) are:

[0030] ① Input the luminance component into the input module of the convolutional neural network. The input module consists of two 3×3×32 convolutional layers, performs preliminary feature extraction on the luminance component of the low-light image, and outputs the original feature map;

[0031] ② The output of step ① is sequentially input into four cascaded encoding modules, which consist of downsampling and two convolutional layers. The encoding module extracts high-level features from the input original feature map through multiple convolutional layers and downsampling layers, and finally outputs a feature map with a smaller spatial size and a higher semantic level.

[0032] ③ The high-level feature maps output from ② are input into the decoding network, which consists of four decoding modules symmetrically connected to the encoding network, each consisting of bilinear interpolation upsampling and two convolutional layers. This module uses a skip connection structure to combine the rich spatial information of the previous layer with the high-level feature information output by the encoding module, ultimately obtaining a feature map that takes into account both spatial and semantic information.

[0033] ④ The output feature map of the decoding network is convolved by the output module and mapped to the output feature dimension, and the final enhanced brightness component is output. The number of output channels of the convolution kernel is consistent with the number of channels of the original image input by the input module.

[0034] Furthermore, the specific method for the encoding module in step ② to process the input features includes the following steps:

[0035] a. At the beginning of each encoding module, 2×2 maximum pooling is used to reduce the size of the input feature map to 1 / 2 of its original size. This performs spatial feature fusion, improves the receptive field of the convolutional network, and extracts more semantic information.

[0036] b. Use two cascaded convolutional layers to extract the reduced-size feature information. Each convolutional layer consists of a convolution operation and a Reluctant Unit (ReLU) activation function. For the convolution operation, the convolution kernel sizes used in the four cascaded encoding modules are 3×3×64, 3×3×128, 3×3×256, and 3×3×256, respectively. Each encoding module uses a convolution kernel with the same number of channels.

[0037] Furthermore, the specific method for the decoding module in step ③ to process the input features includes the following steps:

[0038] a. At the beginning of each decoding module, the input feature map is upsampled to twice its original size using bilinear interpolation, with the number of channels unchanged, to gradually restore the size of the original input image;

[0039] b. Take the feature outputs with the same spatial size of the bilinear interpolation output in the input module and the encoding module and the current decoding module (i.e., encoding module 3 and decoding module 1, encoding module 2 and decoding module 2, encoding module 1 and decoding module 3, input module and decoding module 3), and splice them with the output of the current bilinear interpolation in the channel dimension. The number of channels of the input module and the output of the first three cascaded encoding modules are 32, 64, 128, and 256, respectively. The number of channels of the input of the four symmetrical cascaded decoding modules are 256, 128, 64, and 32, respectively. After splicing, the channel dimension becomes 512, 256, 128, and 64, which are used as the input of the subsequent convolutional layers of the decoding network, respectively, and the semantic information of the deep feature map and the spatial information of the shallow feature map are comprehensively utilized;

[0040] c. After splicing, the four feature maps with the number of channels of 512, 256, 128, and 64 are input into the decoding convolution layers of the four cascaded decoding modules for feature fusion. Each convolution layer consists of a cascade of one convolution operation and a ReLU activation function. The decoding convolution layer fuses the spatial information of the shallow feature map with the semantic information of the deep feature map. The decoding convolution layer is composed of two convolution layers with different convolution kernels. The convolution kernels used in the four cascaded decoding modules are 3×3×256, 3×3×128, 3×3×128, 3×3×64, 3×3×64, 3×3×32, 3×3×32, and 3×3×32, respectively. Therefore, the number of channels output by the four cascaded decoding modules are 128, 64, 32, and 32, respectively.

[0041] Furthermore, the specific details of the calculation of reflection consistency and other constraints in step (3) are as follows:

[0042] ① Calculate the consistency constraint of the reflection component: According to the Retinex theory, the reflection component of the luminance component (R) and the reflection component of the perturbed luminance component (Rd) should be consistent;

[0043] ②Calculate the brightness constraint of the reflection component: the average brightness of the reflection component (R) of the brightness component is maintained at 0.6;

[0044] ③ Calculate the consistency constraint of the reflectance component structure: the brightness component (V) and its reflectance component (R) should have similar spatial structures;

[0045] ④ Calculate the smoothness constraint of the lighting component: According to Retinex theory, the lighting component should be smooth.

[0046] The advantages and positive effects of the present invention are:

[0047] 1. The present invention decouples brightness and color by converting low-light images from RGB color space to HSV color space, thereby reducing the loss of color information during brightness enhancement.

[0048] 2. The present invention restores the image under standard illumination from a single low-illumination image by perturbing the brightness component of the low-illumination image, completing the unsupervised low-illumination enhancement process;

[0049] 3. The present invention achieves the generalization capability for low-illumination images with different degrees of illumination by restoring images under standard illumination. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is the overall network framework of the present invention, where weight sharing means that the two convolutional neural networks use the same parameters, and consistency means that the two reflection components R and Rd remain consistent.

[0051] Figure 2 This is a specific structural diagram of the brightness enhancement module of the present invention. The square in the figure is a feature map, and the number above or below it represents its number of channels. The black hollow advancing arrows represent 3×3 and ReLU activation functions, the black solid arrows represent 2×2 maximum pooling, the black solid hollow arrows represent bilinear interpolation upsampling, and the dotted arrows represent skip connection structures, that is, the feature map on the left end of the arrow is directly applied to the right end of the arrow. DETAILED DESCRIPTION

[0052] The embodiments of the present invention are further described below in conjunction with the accompanying drawings.

[0053] A low-light image enhancement method without normal lighting reference based on HSV color space and Retinex theory, such as Figures 1 to 2 As shown, the following steps are included:

[0054] Step S1: At the image input stage, the low-light image is normalized from the pixel value range [0, 255] to [0, 1];

[0055] Step S2: convert the image obtained in step S1 from the RGB color space to the HSV color space, and separate the three channels to obtain the hue component (H), saturation component (S), and lightness component (V);

[0056] The specific implementation method of step S2 is as follows:

[0057] S2.1 Calculate the maximum channel (c max ), minimum channel (c mi) and contrast (Δ): c max =max

[0058] (R,G,B), c min =min(R,G,B),Δ=c max -c mib (R, G, B represent RGB color space

[0059] The three channels in the middle, the same below);

[0060] S2.2 Calculate the hue (H) according to the formula:

[0061]

[0062] S2.3 Calculate saturation (S) according to the formula:

[0063]

[0064] S2.4 Calculate the brightness (V) according to the formula:

[0065] V=c max .

[0066] Step S3: Calculate the average value of the luminance component (V) obtained in step 2. For images with an average value ≤ 0.5, generate a power function with an exponent in the interval [0, 1]. For images with an average value > 0.5, generate a power function with an exponent in the interval [1, 5]. Adjust the luminance component (V) obtained in step S2 using the generated power function to obtain the disturbed luminance component (Vd).

[0067] Step S4: The luminance component (V) obtained in step S2 and the disturbed luminance component (Vd) obtained in step S3 are used to calculate the reflection component (R) of the luminance component and the reflection component (Rd) of the disturbed luminance component using the Retinex theory.

[0068] The specific implementation method of step S4 is as follows:

[0069] S4.1 sends the luminance component (V) and the disturbed luminance component (Vd) into the convolutional neural network respectively. Figure 1 In the figure, two convolutional neural networks with shared weights are used to obtain the inverse form of the illumination component of the luminance component (L) and the inverse form of the illumination component of the perturbed luminance component (Ld).

[0070] The specific implementation method of step S4.1 is as follows:

[0071] S4.1.1 Input the luminance component into the input module of the convolutional neural network. The input module consists of two 3×3×32 convolutional layers, performs preliminary feature extraction on the luminance component of the low-light image, and outputs the original feature map.

[0072] S4.1.2 inputs the output of S4.1.1 into four cascaded encoding modules, which consist of downsampling and two convolutional layers. The encoding module extracts high-level features from the input original feature map through multiple convolutional and downsampling layers, and finally outputs a feature map with a smaller spatial size and a higher semantic level.

[0073] The specific implementation method of step S4.1.2 is as follows:

[0074] S4.1.2.1 At the beginning of each encoding module, use 2×2 maximum pooling to reduce the size of the input feature map to 1 / 2 of the original size, as shown in Figure 2 The black solid arrows in the figure perform spatial feature fusion to improve the receptive field of the convolutional network and extract more semantic information.

[0075] S4.1.2.2 uses two cascaded convolutional layers to extract feature information after the size is reduced. Each convolutional layer consists of a convolution operation and a ReLU activation function, such as Figure 2 For the convolution operation, in the cascaded four encoding modules, the convolution kernel sizes used are 3×3×64, 3×3×128, 3×3×256, and 3×3×256, respectively. Each encoding module uses the same convolution kernel with the same number of channels. The number of feature map channels is marked in Figure 2 above or below each feature map in .

[0076] S4.1.3: The high-level feature maps output by S4.1.2 are fed into the decoding network, which consists of four decoding modules symmetrically connected to the encoding network, each consisting of bilinear interpolation upsampling and two convolutional layers. This module uses a skip connection structure to combine the rich spatial information of the previous layer with the high-level feature information output by the encoding module, ultimately obtaining a feature map that combines both spatial and semantic information.

[0077] The specific implementation method of step S4.1.3 is as follows:

[0078] S4.1.3.1 At the beginning of each decoding module, the input feature map is upsampled to twice its original size using bilinear interpolation, with the number of channels unchanged, as shown in Figure 2 The black solid hollow arrows in are used to gradually restore the size of the input original image;

[0079] S4.1.3.2 Take the feature outputs of the input module and the encoding module with the same bilinear interpolation output space size as the current decoding module (i.e., encoding module 3 and decoding module 1, encoding module 2 and decoding module 2, encoding module 1 and decoding module 3, input module and decoding module 3), as Figure 2 The dotted line skip connection structure in the figure is spliced ​​with the output of the current bilinear interpolation in the channel dimension. The number of channels output by the input module and the first three cascaded encoding modules are 32, 64, 128, and 256 respectively. The number of channels input by the four symmetrical cascaded decoding modules are 256, 128, 64, and 32 respectively. After splicing, the channel dimension becomes 512, 256, 128, and 64, which are used as the input of the subsequent convolutional layers of the decoding network respectively, comprehensively utilizing the semantic information of the deep feature map and the spatial information of the shallow feature map;

[0080] S4.1.3.3 After concatenation, the four feature maps with the number of channels of 512, 256, 128, and 64 are input into the decoding convolution layers of the four cascaded decoding modules for feature fusion. Each convolution layer consists of a convolution operation and a ReLU activation function cascade, as shown in the following example: Figure 2 The black hollow advancing arrow in the figure indicates that the decoding convolution layer fuses the spatial information of the shallow feature map with the semantic information of the deep feature map. The decoding convolution layer consists of two convolution layers with different convolution kernels. The convolution kernels used in the four cascaded decoding modules are 3×3×256, 3×3×128, 3×3×128, 3×3×64, 3×3×64, 3×3×32, 3×3×32, and 3×3×32, respectively. Therefore, the number of channels output by the four cascaded decoding modules is 128, 64, 32, and 32, respectively. The number of feature map channels is marked in Figure 2 above or below each feature map in .

[0081] S4.1.4 convolutionally processes the output feature map of the decoding network to map it to the output feature dimension, and outputs the final enhanced brightness component. The number of output channels of the convolution kernel is consistent with the number of channels of the original image input to the input module.

[0082] S4.2 Obtain the reflected component (R) of the luminance component and the reflected component (Rd) of the disturbed luminance component using the formula R = V·L;

[0083] S4.3 optimizes the network by calculating reflection consistency and other loss constraints R, Rd, L, Ld.

[0084] The specific implementation method of step S4.3 is as follows:

[0085] S4.3.1 Consistency constraint for calculating reflection component: According to the Retinex theory, the reflection component of the luminance component (R) and the reflection component of the perturbed luminance component (Rd) should be consistent. Figure 1 In the expression, it is the consistency of R and Rd;

[0086] S4.3.2 Calculate the brightness constraint of the reflected component: The average brightness of the reflected component (R) of the luminance component is maintained at 0.6;

[0087] S4.3.3 Calculate the reflectance component structure consistency constraint: The luminance component (V) and its reflectance component (R) should have similar spatial structures;

[0088] S4.3.4 Calculate the structural smoothness constraint of the illumination component: According to the Retinex theory, the illumination component should be smooth.

[0089] Step S5: Recombining the hue component (H) and saturation component (S) obtained in step S2 with the reflectance component (R) of the lightness component obtained in step S4 to obtain a three-channel image in the HSV color space;

[0090] Step S6: Convert the three-channel image in the HSV color space obtained in step S5 to the RGB color space, and adjust its pixel value range from [0, 1] to [0, 255] to obtain an enhanced low-light image.

[0091] Through the above steps, the low-light enhanced image can be obtained.

[0092] Finally, we train the network by minimizing the consistency of the reflectance component (R) of the luminance component and the reflectance component (Rd) of the perturbed luminance component, the brightness and structural consistency of the reflectance component (R) of the luminance component, and the loss of structural smoothness of the inverse form of the illumination component (L) of the luminance component, and use PSNR (Peak Signal to Noise Ratio) and SSIM (structural similarity index) to evaluate the network performance.

[0093] Test environment: Python 3.6; Pytorch deep learning framework; Ubuntu 16.04 system; NVIDIA GTX1080ti GPU.

[0094] Test sequence: The selected dataset is a low-light image randomly taken from the first third of each image sequence in the SICE Part 2 dataset, resulting in a total of 229 low-light images and their true value images.

[0095] Testing method: The low-light image enhancement model is obtained through training, tested using the above dataset, and the evaluation index data is calculated.

[0096] Test metrics: This paper uses metrics such as PSNR and SSIM for evaluation. These metrics are calculated for different popular algorithms and the results are compared, demonstrating that this paper can achieve superior results in the field of low-light image enhancement.

[0097] Test results:

[0098] Table 1. Performance comparison of the proposed method with other algorithms on the SICE dataset (PSNR / SSIM)

[0099] algorithm PSNR SSIM SRIE 14.41 0.54 LIME 16.17 0.57 Li et al. 15.19 0.54 RetinexNet 15.99 0.53 Wang et al. 13.52 0.49 EnlightenGAN 16.21 0.59 Zero-DCE 16.60 0.62 The present invention 18.11 0.63

[0100] Any matters not described in the present invention are applicable to the prior art.

[0101] It should be emphasized that the embodiments described in the present invention are illustrative rather than restrictive. Therefore, the present invention includes but is not limited to the embodiments described in the specific embodiments. Any other embodiments derived by those skilled in the art based on the technical solutions of the present invention also fall within the scope of protection of the present invention.

Claims

1. A low-light image enhancement method without normal illumination reference based on HSV color space and Retinex theory, characterized by The following steps are involved: Step 1: At the image input stage, the low-light image is normalized from the pixel value range [0, 255] to [0, 1]; Step 2: Convert the image obtained in step 1 from RGB color space to HSV color space, and separate the three channels to obtain hue component (H), saturation component (S), and value component (V); Step 3: Calculate the average value of the luminance component (V) obtained in step 2, generate a power function with an exponent in the interval [0, 1] for images with an average value ≤ 0.5, and generate a power function with an exponent in the interval [1, 5] for images with an average value > 0.5; adjust the luminance component (V) obtained in step 2 by the generated power function to obtain the perturbed luminance component (Vd); Step 4: Based on the Retinex theory, the brightness (V) is composed of the illumination component (I) and the reflection component (R), and the formula is V = I·R. For images of different brightness of the same scene, the reflection component should remain consistent. The convolutional neural network is used to establish a functional relationship between the brightness component (V) and the reciprocal form of the illumination component (L), and the formula is L = f(V). The reflection component (R) of the brightness component and the reflection component (Rd) of the perturbed brightness component are calculated by the formula R = V·L. By calculating the reflection consistency loss, reflection component brightness loss, reflection component structural consistency loss, illumination component smoothness loss and total loss, the network is optimized by reducing the total loss. Step 5: Recombining the hue component (H) and saturation component (S) obtained in step 2 with the reflectance component (R) of the lightness component obtained in step 4 to obtain a three-channel image in the HSV color space; Step 6: Convert the three-channel image in the HSV color space obtained in step 5 to the RGB color space, and adjust its pixel value range from [0, 1] to [0, 255] to obtain the enhanced low-light image.

2. The low-light image enhancement method without normal illumination reference based on HSV color space and Retinex theory according to claim 1, characterized in that: The specific method of converting from RGB to HSV color space in step 2 includes the following steps: (1) Calculate the maximum channel (c max ), minimum channel (c min ) and contrast (Δ): c max =max(R,G,B),c min =min(R,G,B),Δ=c max -c min (where R, G, and B represent the three channels in the RGB color space, the same below); (2) Calculate hue (H) according to the formula: (3) Calculate saturation (S) according to the formula: (4) Calculate the brightness (V) according to the formula: V=c max 。 3. The low-light image enhancement method without normal illumination reference based on HSV color space and Retinex theory according to claim 1, characterized in that: The convolutional neural network in step 4 specifically includes the following steps: (1) The luminance component is input into the input module of the convolutional neural network. The input module consists of two 3×3×32 convolutional layers, which performs preliminary feature extraction on the luminance component of the low-light image and outputs the original feature map; (2) The output of (1) is sequentially input into four cascaded encoding modules, which are composed of downsampling and two convolutional layers. The encoding module extracts high-level features from the input original feature map through multiple convolutional layers and downsampling layers, and finally outputs a feature map with a smaller spatial size and a higher semantic level. (3) The high-level feature map output from (2) is input into the decoding network, which is symmetrical with the encoding network and consists of four decoding modules consisting of bilinear interpolation upsampling and two convolutional layers. This module uses a skip connection structure to combine the rich spatial information of the previous layer with the high-level feature information output by the encoding module, and finally obtains a feature map that takes into account both spatial information and semantic information. (4) The output feature map of the decoding network is convolved by the output module and mapped to the output feature dimension, and the final enhanced brightness component is output. The number of output channels of the convolution kernel is consistent with the number of channels of the original image input by the input module.

4. The convolutional neural network according to claim 3, wherein The encoding module of step (2) specifically includes the following steps: ① At the beginning of each encoding module, 2×2 maximum pooling is used to reduce the size of the input feature map to 1 / 2 of the original size, perform spatial feature fusion, improve the receptive field of the convolutional network, and extract more semantic information; ② Use two cascaded convolutional layers to extract feature information after size reduction; each convolutional layer consists of one convolution operation and one ReLU activation function; for the convolution operation, in the four cascaded encoding modules, the convolution kernel sizes used are 3×3×64, 3×3×128, 3×3×256, and 3×3×256, respectively, and each encoding module uses a convolution kernel with the same number of channels.

5. The convolutional neural network according to claim 3, wherein The decoding module of step (3) specifically includes the following steps: ① At the beginning of each decoding module, the input feature map is upsampled to twice its original size using bilinear interpolation, with the number of channels unchanged, to gradually restore the size of the original input image; ② Take the feature output of the input module and the encoding module with the same spatial size as the bilinear interpolation output in the current decoding module, that is, encoding module 3 and decoding module 1, encoding module 2 and decoding module 2, encoding module 1 and decoding module 3, input module and decoding module 3, and the output of the current bilinear interpolation are spliced ​​in the channel dimension. The number of channels of the input module and the output of the first three cascaded encoding modules are 32, 64, 128, and 256 respectively. The number of channels of the input of the four symmetrical cascaded decoding modules are 256, 128, 64, and 32 respectively. After splicing, the channel dimension becomes 512, 256, 128, and 64, which are used as the input of the subsequent convolutional layer of the decoding network respectively, and the semantic information of the deep feature map and the spatial information of the shallow feature map are comprehensively utilized; ③ After splicing, the four feature maps with the number of channels of 512, 256, 128, and 64 are input into the decoding convolution layers of the four cascaded decoding modules for feature fusion. Each convolution layer consists of a cascade of one convolution operation and a ReLU activation function. The decoding convolution layer fuses the spatial information of the shallow feature map with the semantic information of the deep feature map. The decoding convolution layer is composed of two convolution layers with different convolution kernels. The convolution kernels used in the four cascaded decoding modules are 3×3×256, 3×3×128, 3×3×128, 3×3×64, 3×3×64, 3×3×32, 3×3×32, and 3×3×32, respectively. Therefore, the number of channels output by the four cascaded decoding modules are 128, 64, 32, and 32, respectively.

6. The low-light image enhancement method without normal illumination reference based on HSV color space and Retinex theory according to claim 1, characterized in that: The reflection consistency loss, reflection component brightness loss, reflection component structure consistency loss, illumination component smoothness loss and total loss in step 4 specifically include the following steps: (1) Calculate the consistency loss of the reflection component: According to the Retinex theory, the reflection component of the luminance component (R) and the reflection component of the perturbed luminance component (R d ) should be consistent, the formula is , where ‖·‖2 represents the mean square error, the same below; (2) Calculation of the brightness loss of the reflection component: The average brightness of the reflection component (R) of the brightness component is maintained at 0.7, and the formula is , R n Represents the reflection component after 16×16 average pooling, E represents the reflection component after 16×16 average pooling, and R n Matrices of the same size with all values ​​equal to 0.7; (3) Calculate the structural consistency loss of the reflection component: the brightness component (V) and its reflection component (R) should have similar spatial structures. The formula is: , R m and V m Represent the reflection component and input brightness after 4×4 average pooling, Represents the first-order differential in the horizontal and vertical directions of the image; (4) Calculation of the smoothness loss of the illumination component: According to the Retinex theory, the illumination component (I) and its reciprocal form (L) should be smooth, and the formula is (5) Calculate the total loss: The formula is L = L rc +L ec +L ss +0.1L is .

Images