Underwater low-illumination color image enhancement method

Abstract

In order to solve the problems of low brightness, poor contrast and color distortion of an underwater acquired image, the invention provides a low-illumination underwater color image enhancement method with relatively high color reduction degree, which is mainly characterized by comprising the following steps of: (1) decomposing a low-illumination image into an illumination image and a reflection image through a constructed initialization network, and taking the illumination image as the input of the network, the reflection map is used as an attention map; (2) constructing a multi-condition guide enhancement network model and a double-discriminator model which take the reflection map as an attention mechanism, so as to realize the enhancement of the underwater low-illumination color image; and (3) when the neural network is trained, a multivariate loss function is constructed to realize that an output image is closer to a real image, and an experimental result shows that compared with a mainstream enhancement algorithm, the method not only can improve the definition of the underwater low-illumination color image, but also is more effective in color restoration.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and in particular relates to a low-light underwater color deep learning image enhancement method with high color fidelity. Background Technology

[0002] Water has the ability to absorb and scatter natural light, and light decays exponentially with distance and depth. Therefore, underwater images often appear blue-green and hazy, leading to color distortion and low contrast. Low-light image enhancement, as a method to improve image quality, has attracted widespread attention in the field of computer vision. Directly enlarging low-light images is usually the most intuitive and simplest way to adjust the visibility of dark areas. However, this operation also brings another problem: excessive brightening can destroy the color effect of the image. Therefore, low-light color image enhancement methods are often relatively easy to implement in improving image sharpness, but they cannot achieve satisfactory results in restoring the actual image color, which poses a more severe challenge to further color image-related processing and recognition tasks. With the development of deep learning, many researchers have begun to try to use deep learning algorithms to solve the image enhancement problem. However, these networks are limited by model complexity and the locality of simple local difference convolution, and the problem of image color distortion still exists. This invention addresses this problem by studying high-quality enhancement of low-light color images, using low-light underwater images as the research object. Summary of the Invention

[0003] To address the issues of low brightness, poor contrast, and color distortion in underwater images, this invention proposes a low-light underwater color image enhancement method with high color fidelity. The main features are: (1) a constructed initialization network decomposes the low-light image into an illuminance map and a reflectance map, using the illuminance map as the network input and the reflectance map as the attention map; (2) a multi-condition guided enhancement network model and a dual-discriminator model using the reflectance map as the attention mechanism are constructed to enhance the underwater low-light color image; (3) a multivariate loss function is constructed during neural network training to make the output image closer to the real image. Experimental results show that, compared with mainstream enhancement algorithms, this method not only improves the clarity of underwater low-light color images but is also more effective in color fidelity.

[0004] The present invention aims to provide an underwater low-light image enhancement method through the following technical solution, comprising the following steps:

[0005] Step 1: Construct an initialization network that decomposes the low-light image into an illumination map and a reflectance map. Use the illumination map as the input to the network and the reflectance map as the attention map.

[0006] Step 2: Construct a multi-condition guided enhancement network model and a dual discriminator model that use reflection maps as the attention mechanism;

[0007] Step 3: When training the neural network, construct a multivariate loss function to make the output image closer to the real image;

[0008] Step 4: Conduct image enhancement simulation comparison experiments using low-light images to verify the effectiveness and superiority of the model.

[0009] Specifically, the proposed model is a low-light enhancement network based on the idea of ​​generative adversarial networks.

[0010] The entire model can be mainly divided into three parts: ① an initialization network used to divide the illuminance map and reflectance map, such as... Figure 1 The initialization module is shown; ② The enhancement network for low-light image reconstruction, as shown. Figure 2 The enhanced network architecture is shown; ③ The discriminant network used to determine the enhancement quality, such as... Figure 3 The discriminator network architecture is shown. The workflow of this model is as follows: ① The acquired low-light image is input into the network's initialization module to generate the input image's illumination map and reflectance map, followed by image enhancement generation; ② The image is input into the constructed fully convolutional neural network structure to complete the low-light image enhancement process; ③ The enhanced image is then input into a network structure with dual discriminators to guide the generation of a high-quality enhanced image.

[0011] The low contrast and color distortion in underwater color images are primarily due to the varying attenuation of light at different wavelengths as it propagates through water, causing the same object to often appear different colors under different lighting conditions. Therefore, based on this characteristic, Retinex theory can be used for initial image decomposition to enable targeted enhancement processing. Retinex theory can achieve a balance between dynamic range compression, edge enhancement, and color constancy, allowing for adaptive enhancement of various image types.

[0012] Retinex's theoretical foundation relies on a core assumption: a color image can be decomposed into two main components: the illuminance component and the reflectance component, expressed as formula (1):

[0013] S = I·R, (1)

[0014] Here, S is the input image, I is the illuminance component, and R is the reflectance component. The reflectance component represents features in the image, such as the material and color of objects, that are independent of the surrounding environment. The illuminance component represents the brightness and darkness information resulting from the interaction between the lighting environment and the geometry of the object. In low-light images, the illuminance component is usually dark and unevenly distributed; therefore, low-light color underwater image enhancement primarily targets the illuminance component.

[0015] According to Retinex theory, an image can be decomposed into a reflectance component R and an illuminance component I. Therefore, an initialization module can be trained to automatically decompose an image into a reflectance map and an illuminance map.

[0016] During the training phase of the initialization module, pairs of low-light and normal images from the dataset need to be input into the decomposition network to obtain two reflectance maps and two illumination maps respectively, while preserving their reflectance... Figure 1 The illumination component of the normal light image is used as a constraint to enable the decomposition network to learn to decompose the image in a data-driven manner. The decomposition process is as follows: First, features are extracted using a three-layer convolutional layer consisting of a 3x3 convolutional kernel and a rectified linear unit activation function; then, the features are converted into reflectance and illuminance components using a 3x3 convolutional kernel and a rectified linear unit activation function. This module is trained separately using low-illuminance and normal light images. When a low-illuminance image is input, a loss function is used as shown in Equation (2):

[0017]

[0018] Where μ is the smoothing coefficient. It is a loss of lighting smoothness. For reconstruction loss, a loss function is used when inputting a normally lit image, as shown in Equation (3):

[0019]

[0020] in, Represents the gradient. It relaxes the restrictions on smoothness imposed by image structure and lighting discontinuities.

[0021] Low-light image enhancement based on deep learning networks is typically a pixel-level reconstruction process based on image features, which does not consider the color distribution of normally lit images, resulting in distorted generated images. To avoid the loss of information in the bottleneck layer of the generator network during low-light image enhancement and to better ensure that colors are not distorted during image generation, this paper uses the U-Net structure to construct the generator of this network model.

[0022] Using the reflectance map as the attention map allows the network to learn the graphic characteristics of the image and utilize environment-independent features such as object material and color in the reflectance map as conditional distributions. This incorporates image color information into the latent space, improving color saturation in low-light conditions. The illumination map is used as input, and the size of the reflectance map is adjusted to fit each feature map. This is then multiplied by all intermediate feature maps and the output image. In the generator, except for the output layer which uses an sigmoid growth curve activation function, the remaining layers use a linear rectified activation function.

[0023] Set G0(x) as the input to the generator network, and set G... i If (x) is set as the output of the convolution unit, then the following formula (4) holds:

[0024] skip(x) = f i，2 [f i，1 (G i (x))] (0<i ≤ 3), (4)

[0025] Here, skip(x) denotes skipping feature maps, and f represents the non-linear mapping structure in the generated network. The decoding part of the generated network amplifies the feature maps and concatenates them with the skipped feature maps before feature extraction, ultimately outputting an enhanced image.

[0026] In low-light images under uneven lighting conditions, the ideal is to enhance dark areas rather than bright areas, so that the output image is neither overexposed nor underexposed, and the image remains unchanged. Given the illuminance component of a low-light image, the reflectance component features are used as interest information to reveal detail and color features.

[0027] The discriminator network consists of a global discriminator and local discriminators. The input to the global discriminator is the complete image, while the input to the local discriminator is random slices of the image. The local discriminator, drawing inspiration from Markov discriminators, focuses on and enhances local texture details of the image by judging random local slices of the image region.

[0028] The discriminator network uses a modified linear unit activation function with α = 0.2. This nonlinear mapping structure improves the network's ability to extract image features, while the addition of normalization layers reduces redundant information, speeds up training, and enhances network stability. Convolutional layers with a stride of 2 are used instead of pooling layers, and batch normalization layers are not used in the first nonlinear mapping. Although the discriminator network consists of two discriminators, they are not trained simultaneously; one is used to determine whether an image is real or fake, and the other is used to judge the similarity of local textures in the image. Let df(x) be df0(x) = GCI. low The nonlinear mapping elements of ) are as shown in formulas (5) and (6):

[0029] df i (x)=max(aH(x),H(x)), a=0.2, (5)

[0030] Among them, a i ,β i These are the reconstruction parameters for Batch Normalization (BN). For example, D... j(x) is the output of the convolutional unit in the discriminator network. If D0(x) = 0, then it is as shown in formula (7):

[0031] D j (x)=df j，2 [dF j，1 (D j-1 (x))],0<j≤5 (7)

[0032] Finally, a 1×1 convolution is used for D. j (x) Perform channel compression, using two fully connected layers to map the result to a single value, and use a sigmoid growth curve activation function to map the output.

[0033] When training a neural network, a loss function is used to achieve the goal of making the output image increasingly closer to the real image. To qualitatively and quantitatively improve image quality, commonly used error metrics cannot reflect the degree of optimization in various aspects. By integrating various aspects, the discriminator in this paper uses three loss functions: structural similarity loss (SSIM) and mean squared error (MSE), as shown in Equation (8):

[0034] L loss =ɑLoss SSIM +βLoss MSE (8)

[0035] Here, α and β are three positive balance parameters. SSIM refers to Overall Visual Effect, which measures the difference between the real image and the image enhanced using the algorithm in this paper, aiming to improve the visual quality of the output image.

[0036]

[0037] In formula (9), N represents the total number of pixels in the image, μ x and μ y The average values ​​of x and y are respectively. σx 2 and σ y 2 These are the variances of x and y, respectively, and σ xy Let C1 be the covariance of x and y, where C1 = (k1L). 2 And C2=(k2L) 2 These are constants used to ensure the stability of the loss function and prevent the denominator from being zero. Where k1 = 0.01 and k2 = 0.03.

[0038] Mean squared error is the average of the sum of squares of the differences between the target variable and the predicted value. The mean squared error loss function amplifies the gap between the maximum and minimum errors, and it is highly sensitive to outliers, as shown in formula (10):

[0039]

[0040] Where N represents the total number of pixels in the image, p represents the total number of pixels in the slice, and Y(p) These represent the pixel values ​​p in the ordinary light image and the network output light-enhanced image, respectively.

[0041] The mean squared error loss function can effectively measure the difference between pixels in two images and performs well in image content enhancement, making the generated image visually closer to the target image.

[0042] Experiments have demonstrated that the invention has good applicability and can effectively enhance images in different scenarios. Furthermore, the enhanced images exhibit an overall improved subjective appearance, higher brightness, sharpness, and deeper texture details, with an overall color composition that more closely approximates subjective perception. Attached Figure Description

[0043] Figure 1 It is an initialization module;

[0044] Figure 2 It is about enhancing the network architecture;

[0045] Figure 3 It is a discriminator network architecture;

[0046] Figure 4 This is a comparison diagram of the ablation experiment results;

[0047] Figure 5 It compares the results of experiments with different algorithms. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] Combination Figures 1-5 This invention proposes a low-light image enhancement method, comprising the following steps:

[0050] Step 1: Construct an initialization network that decomposes the low-light image into an illumination map and a reflectance map. Use the illumination map as the input to the network and the reflectance map as the attention map.

[0051] Step 2: Construct a multi-condition guided enhancement network model and a dual discriminator model that use reflection maps as the attention mechanism;

[0052] Step 3: When training the neural network, construct a multivariate loss function to make the output image closer to the real image;

[0053] Step 4: Conduct image enhancement simulation comparison experiments using low-light images to verify the effectiveness and superiority of the model.

[0054] Specifically, the proposed model is a low-light enhancement network based on the idea of ​​generative adversarial networks.

[0055] The entire model can be mainly divided into three parts: ① an initialization network for dividing the illumination map and reflectance map; ② an enhancement network for low-light image reconstruction; and ③ a discriminant network for judging the enhancement quality. The model's workflow is as follows: ① The acquired low-light image is input into the network's initialization module to generate the input image's illumination map and reflectance map, followed by image enhancement generation; ② The image is input into the constructed fully convolutional neural network structure to complete the low-light image enhancement process; ③ The enhanced image is then input into a network structure with dual discriminators to guide the generation of a high-quality enhanced image.

[0056] The low contrast and color distortion in underwater color images are primarily due to the varying attenuation of light at different wavelengths as it propagates through water, causing the same object to often appear different colors under different lighting conditions. Therefore, based on this characteristic, Retinex theory can be used for initial image decomposition to enable targeted enhancement processing. Retinex theory can achieve a balance between dynamic range compression, edge enhancement, and color constancy, allowing for adaptive enhancement of various image types.

[0057] Retinex's theoretical foundation relies on a core assumption: a color image can be decomposed into two main components: the illuminance component and the reflectance component, expressed as formula (1):

[0058] S = I·R, (1)

[0059] Here, S is the input image, I is the illuminance component, and R is the reflectance component. The reflectance component represents features in the image, such as the material and color of objects, that are independent of the surrounding environment. The illuminance component represents the brightness and darkness information resulting from the interaction between the lighting environment and the geometry of the object. In low-light images, the illuminance component is usually dark and unevenly distributed; therefore, low-light color underwater image enhancement primarily targets the illuminance component.

[0060] According to Retinex theory, an image can be decomposed into a reflectance component R and an illuminance component I. Therefore, an initialization module can be trained to automatically decompose an image into a reflectance map and an illuminance map.

[0061] During the training phase of the initialization module, pairs of low-light and normal images from the dataset need to be input into the decomposition network to obtain two reflectance maps and two illumination maps respectively, while preserving their reflectance... Figure 1 The illumination component of the normal light image is used as a constraint to enable the decomposition network to learn to decompose the image in a data-driven manner. The decomposition process is as follows: First, features are extracted using a three-layer convolutional layer consisting of a 3x3 convolutional kernel and a rectified linear unit activation function; then, the features are converted into reflectance and illuminance components using a 3x3 convolutional kernel and a rectified linear unit activation function. This module is trained separately using low-illuminance and normal light images. When a low-illuminance image is input, a loss function is used as shown in Equation (2):

[0062]

[0063] Where μ is the smoothing coefficient. It is a loss of lighting smoothness. For reconstruction loss, a loss function is used when inputting a normally lit image, as shown in Equation (3):

[0064]

[0065] in, Represents the gradient. It relaxes the restrictions on smoothness imposed by image structure and lighting discontinuities.

[0066] Low-light image enhancement based on deep learning networks is typically a pixel-level reconstruction process based on image features, which does not consider the color distribution of normally lit images, resulting in distorted generated images. To avoid the loss of information in the bottleneck layer of the generator network during low-light image enhancement and to better ensure that colors are not distorted during image generation, this paper uses the U-Net structure to construct the generator of this network model.

[0067] Using the reflectance map as the attention map allows the network to learn the graphic characteristics of the image and utilize environment-independent features such as object material and color in the reflectance map as conditional distributions. This incorporates image color information into the latent space, improving color saturation in low-light conditions. The illumination map is used as input, and the size of the reflectance map is adjusted to fit each feature map. This is then multiplied by all intermediate feature maps and the output image. In the generator, except for the output layer which uses an sigmoid growth curve activation function, the remaining layers use a linear rectified activation function.

[0068] Set G0(x) as the input to the generator network, and set G... i If (x) is set as the output of the convolution unit, then the following formula (4) holds:

[0069] skip(x) = f i，2 [f i，1 (G i (x))] (0<i ≤ 3), (4)

[0070] Here, skip(x) denotes skipping feature maps, and f represents the non-linear mapping structure in the generated network. The decoding part of the generated network amplifies the feature maps and concatenates them with the skipped feature maps before feature extraction, ultimately outputting an enhanced image.

[0071] In low-light images under uneven lighting conditions, the ideal is to enhance dark areas rather than bright areas, so that the output image is neither overexposed nor underexposed, and the image remains unchanged. Given the illuminance component of a low-light image, the reflectance component features are used as interest information to reveal detail and color features.

[0072] The discriminator network consists of a global discriminator and local discriminators. The input to the global discriminator is the complete image, while the input to the local discriminator is random slices of the image. The local discriminator, drawing inspiration from Markov discriminators, focuses on and enhances local texture details of the image by judging random local slices of the image region.

[0073] The discriminator network uses a modified linear unit activation function with α = 0.2. This nonlinear mapping structure improves the network's ability to extract image features, while the addition of a normalization layer reduces redundant information, speeds up training, and enhances network stability. Convolutional layers with a stride of 2 are used instead of pooling layers, and batch normalization layers are not used in the first nonlinear mapping. Although the discriminator network consists of two discriminators, they are not trained simultaneously; one is used to determine whether an image is real or fake, and the other is used to judge the similarity of local textures in the image. Let df(x) be df0(x) = G(I). low The nonlinear mapping elements of ) are as shown in formulas (5) and (6):

[0074] df i (x)=max(aH(x),H(x)), a=0.2, (5)

[0075] Among them, a i ,β i These are the reconstruction parameters for Batch Normalization (BN). For example, D... j (x) is the output of the convolutional unit in the discriminator network. If D0(x) = 0, then it is as shown in formula (7):

[0076] D j (x)=df j，2 [dF j，1 (D j-1(x))],0<j≤5 (7)

[0077] Finally, a 1×1 convolution is used for D. j (x) Perform channel compression, using two fully connected layers to map the result to a single value, and use a sigmoid growth curve activation function to map the output.

[0078] When training a neural network, a loss function is used to achieve the goal of making the output image increasingly closer to the real image. To qualitatively and quantitatively improve image quality, commonly used error metrics cannot reflect the degree of optimization in various aspects. By integrating various aspects, the discriminator in this paper uses three loss functions: structural similarity loss (SSIM) and mean squared error (MSE), as shown in Equation (8):

[0079] L loss =αLoss SSIM +βLoss MSE (8)

[0080] Here, α and β are three positive balance parameters. SSIM refers to Overall Visual Effect, which measures the difference between the real image and the image enhanced using the algorithm in this paper, aiming to improve the visual quality of the output image.

[0081]

[0082] In formula (9), N represents the total number of pixels in the image, μ x and μ y σ are the average values ​​of x and y, respectively. x 2 and σ y 2 These are the variances of x and y, respectively, and σ xy Let C1 be the covariance of x and y, where C1 = (k1L). 2 And C2=(k2L) 2 These are constants used to ensure the stability of the loss function and prevent the denominator from being zero. Where k1 = 0.01 and k2 = 0.03.

[0083] Mean squared error is the average of the sum of squares of the differences between the target variable and the predicted value. The mean squared error loss function amplifies the gap between the maximum and minimum errors, and it is highly sensitive to outliers, as shown in formula (10):

[0084]

[0085] Where N represents the total number of pixels in the image, p represents the total number of pixels in the slice, and Y(p) These represent the pixel values ​​p in the ordinary light image and the network output light-enhanced image, respectively.

[0086] The mean squared error loss function can effectively measure the difference between pixels in two images and performs well in image content enhancement, making the generated image visually closer to the target image.

[0087] Experiments have demonstrated that the invention has good applicability and can effectively enhance images in different scenarios. Furthermore, the enhanced images exhibit an overall improved subjective appearance, higher brightness, sharpness, and deeper texture details, with an overall color composition that more closely approximates subjective perception.

[0088] Dataset and Implementation:

[0089] The training dataset consisted of 1000 images constructed using datasets such as EUVP and UIEBD. All images were converted to PNG format and resized to 224*224 pixels. Test images were selected from real-world low-light scene images collected by the network. The Adam optimizer was used with a batch size of 16 and a learning rate of 1e-4. The learning rate was then linearly decayed to 0 after 100 training iterations.

[0090] Ablation experiments verified:

[0091] To demonstrate the effectiveness of our proposed Low Light EnhancementNet (LLEN) model, we conducted ablation experiments. Two ablation experiments were designed, removing the local discriminator and the attention component, respectively. The first column shows low-light images, the second column shows normal-light images, the third column shows the enhancement effect of removing the attention mechanism, the fourth column shows the enhancement effect of retaining only the global discriminator, and the last column shows the complete enhancement effect of our proposed network.

[0092] (1) Subjective evaluation: Based on the ablation comparison experiment, such as Figure 4 The comparison images from the ablation experiments show that, compared to the second column of normally lit images, the enhancement results without the attention mechanism contain severely distorted colors or underexposed local areas, such as the water textures, reefs, and halo areas in the images. Removing the local discriminator results in poor detail processing in the enhancement results of the fourth column, as shown in the texture areas of fish, water, and coral reefs. Conversely, the full-structure enhancement results from the proposed model contain more realistic color representation, making the images more visually appealing and validating the effectiveness of the dual discriminator design and the attention mechanism.

[0093] (2) Quantitative evaluation: Subjective evaluation is easily affected by the subjective factors of the observer and therefore does not have good reliability. In this paper, Peak Signal to Noise Ratio (PSNR)

[25] and Structural Similarity Index (SSIM) are used to evaluate the quality of the enhanced image. Peak Signal to Noise Ratio is the peak signal to noise ratio. It can evaluate the image quality based on the error between pixels, evaluate the image at the pixel level, and show the effectiveness of the enhancement. The larger the value, the better the image quality. Structural Similarity represents structural similarity. Consistent with human perception, it measures the similarity of images from brightness, contrast and structure. The value range of Structural Similarity is [0, 1]. The larger the value, the smaller the image distortion and the better the effect. The experimental results are shown in Table 1.

[0094] Table 1 Objective evaluation results of the images

[0095]

[0096] Table 1 shows a quantitative comparison between our proposed method and other methods. Our proposed method achieves higher peak signal-to-noise ratio (PSNR) and structural similarity values, indicating that it generates results closer to the fundamental facts. Our algorithm produces higher PNR and structural similarity values, demonstrating better resolution in detail recovery. Other algorithms are visually effective, but their image textures are sometimes blurred.

[0097] Comparison of experiments with mainstream models:

[0098] In this section, we compare the image enhancement performance of the proposed network model with several mainstream enhancement models, such as CycleGAN, RetinexNet, and LIME, through a series of experiments, and compare their performance through subjective and objective evaluations.

[0099] (1) Subjective evaluation: such as Figure 5 The comparison of different algorithm experiments is shown in the figure. The first column displays the original low-light image, and the second to fifth columns display the enhanced image. As shown, the LIME network easily leads to overexposure and distortion. The CycleGAN and RetinexNet networks have average visual effects in terms of brightness and naturalness. Through comparison, the proposed network model performs well in handling dark texture details and achieves satisfactory color fidelity.

[0100] (2) Quantitative evaluation

[0101] In terms of objective image quality assessment, in addition to peak signal-to-noise ratio and structural similarity, researchers have discovered a more effective evaluation metric—the Natural Image Quality Evaluator (NIQE). This evaluation method is more suitable for assessing the quality of images generated by deep learning networks.

[0102] The natural image quality assessment method is represented as the distance between a multivariate Gaussian model of the statistical features of the natural scene extracted from the test image and a multivariate Gaussian model of the quality-perceived features extracted from the natural image corpus, as shown in Equation (11).

[0103]

[0104] Where v1, v2, ∑1, and ∑2 represent the mean vector and covariance matrix of the natural multivariate Gaussian model and the distorted image multivariate Gaussian model, respectively. The larger the parameter values, the worse the image quality. Experimental results are shown in Table 2.

[0105] Table 2 Image NIQE Results

[0106]

[0107] Table 2 shows a quantitative comparison between the proposed method and other methods. Evaluation results for the two methods demonstrate that the proposed algorithm has good applicability and can effectively enhance images in different scenarios. Furthermore, the enhanced images exhibit an overall improved subjective appearance, higher brightness, sharpness, and deeper texture details, with an overall color composition closer to subjective perception.

[0108] The low-light image enhancement method provided by the present invention has been described in detail above. Specific examples have been used to illustrate the principle and implementation of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation and application scope based on the idea of ​​the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A low-light image enhancement method, characterized in that, Includes the following steps: Step 1: Construct an initialization network that decomposes the low-light image into an illumination map and a reflectance map. Use the illumination map as the input to the network and the reflectance map as the attention map. Step 2: Construct a multi-condition guided enhancement network model and a dual discriminator model that use reflection maps as the attention mechanism; Step 3: When training the neural network, construct a multivariate loss function to make the output image closer to the real image; Step 4: Conduct image enhancement simulation comparison experiments using low-light images to verify the effectiveness and superiority of the model.

2. According to claim 1, the main reason for the low contrast and color distortion of underwater color images is that light of different wavelengths will produce different attenuations when it propagates in water. The same object often shows different colors under different lighting conditions. Therefore, based on this feature, Retinex theory can be used for initial image decomposition in order to perform targeted enhancement processing. Retinex theory can achieve a balance between dynamic range compression, edge enhancement and color constancy, allowing for adaptive enhancement of various types of images. Retinex's theoretical foundation relies on a core assumption: a color image can be decomposed into two main components: the illuminance component and the reflectance component, expressed as formula (1): S = I·K, (1) Where S is the input image, I is the illuminance component, and R is the reflectance component. The reflectance component represents the features of the object in the image, such as material and color, which are independent of the surrounding environment. The illuminance component represents the brightness information after the interaction between the lighting environment and the geometric structure of the object. In low-illuminance images, the illuminance component is usually dark and unevenly distributed. Therefore, low-illuminance color underwater image enhancement takes the illuminance component as the main enhancement object. According to Retinex theory, an image can be decomposed into a reflectance component R and an illuminance component I. Therefore, an initialization module can be trained to automatically decompose the image into a reflectance map and an illuminance map. During the training phase of the initialization module, pairs of low-light images and normal images from the dataset need to be input into the decomposition network to obtain two reflectance maps and two illumination maps respectively. The reflectance maps are kept consistent, and the illumination component of the normal light image is used as a constraint so that the decomposition network can learn to decompose the image in a data-driven manner. The decomposition process is as follows: First, a three-layer convolutional layer consisting of a 3x3 convolutional kernel and a rectified linear unit activation function is used to extract features; then, the features are converted into reflectance components and illumination components using a 3x3 convolutional kernel and a rectified linear unit activation function. This module is trained separately using low-light images and normal light images. When a low-light image is input, a loss function is used, as shown in Equation (2): Where μ is the smoothing coefficient. It is a loss of lighting smoothness. To determine the reconstruction loss, a loss function is used when a normally lit image is input, as shown in Equation (3): in, Represents the gradient. It relaxes the restrictions on smoothness imposed by image structure and lighting discontinuities.

3. As described in claim 2, characterized in that, Low-light image enhancement based on deep learning networks is usually a pixel-level reconstruction process based on image features, which does not consider the color distribution of normal lighting images, resulting in distorted generated images. In order to avoid the loss of information in the bottleneck layer of the generation network during low-light image enhancement and to better ensure that the colors are not distorted during image generation, this paper uses the U-Net structure to build the generator of this network model. Using the reflectance map as the attention map allows the network to learn the graphic characteristics of the image and utilize environmentally independent features, such as object material and color, as conditional distributions. This incorporates image color information into the latent space, enhancing color saturation in low-light conditions. The illumination map is used as input, and the reflectance map size is adjusted to fit each feature map. This is then multiplied by all intermediate feature maps and the output image. In the generator, except for the output layer which uses an S-shaped growth curve activation function, the remaining layers use a linear rectified function activation function. G0(x) is set as the input to the generator network, and G... i If (x) is set as the output of the convolution unit, then the following formula (4) holds: skip(x)=f i，2 [f i，1 (G i (x))](0<i≤3), (4) Where skip(x) means skipping the feature map, f represents the non-linear mapping structure in the generated network, the decoding part of the generated network amplifies the feature map and connects it with the skipped feature map before feature extraction, and finally outputs the enhanced image; In low-illuminance images under uneven lighting conditions, the ideal is to enhance dark areas rather than bright areas, so that the output image is neither overexposed nor underexposed, and the image remains unchanged. Given the illuminance component of the low-illuminance image, the reflectance component features are used as attention information to present detail and color features. The discriminator network includes a global discriminator and a local discriminator. The input of the global discriminator is the complete image, and the input of the local discriminator is a random slice of the image. The local discriminator borrows the idea of ​​the Markov discriminator and can focus on and enhance the local texture details of the image by judging the random local slice image regions. The discriminator network uses a modified linear unit activation function with α = 0.

2. This nonlinear mapping structure improves the network's ability to extract image features. The addition of a normalization layer reduces redundant information, speeds up training, and enhances network stability. Convolutional layers with a stride of 2 are used instead of pooling layers, and batch normalization layers are not used in the first nonlinear mapping. Although the discriminator network consists of two discriminators, they are not trained simultaneously. One is used to determine whether an image is real or fake, and the other is used to judge the similarity of local textures in the image. Let df(x) be df0(x) = G(I). low The nonlinear mapping elements of ) are as shown in formulas (5) and (6): df i (x)=max(aH(x),H(x)),a=0.2, (5) Among them, a i ,β i These are the reconstruction parameters of BN, such as D. j (x) is the output of the convolutional unit in the discriminator network. If D0(x) = 0, then it is as shown in formula (7): D j (x)=df j，2 [dF j，1 (D j-1 (x))],0<j≤5 (7) Finally, a 1×1 convolution is used for D. j (x) Perform channel compression, using two fully connected layers to map the result to a single value, and use a sigmoid growth curve activation function to map the output.

4. According to claim 3, when training the neural network, a loss function is used to achieve the goal of making the output image closer and closer to the real image. In order to improve the image quality qualitatively and quantitatively, the commonly used error metric cannot reflect the degree of optimization of the image in various aspects. Through the integration of various aspects, the discriminator in this paper uses three loss functions, namely structural similarity loss (SSIM) and mean squared error (MSE), as shown in formula (8): L loss =αLoss SSIM +βLoss MSE , (8) in, α and β are three positive balance parameters, and SSIM refers to the overall visual effect, used to measure the difference between the real image and the image enhanced using the algorithm in this paper, aiming to improve the visual quality of the output image. In formula (9), N represents the total number of pixels in the image, μ x and μ y σ are the average values ​​of x and y, respectively. x 2 and σ y 2 These are the variances of x and y, respectively, and σ xy Let C1 be the covariance of x and y, where C1 = (k1L). 2 And C2=(k2L) 2 is a constant used to ensure the stability of the loss function and prevent the denominator from being 0, where k1 = 0.01, k2 = 0.03, and mean squared error is the average of the sum of squares of the differences between the target variable and the predicted value. The mean squared error loss function amplifies the gap between the maximum and minimum errors, and the mean squared error loss function is very sensitive to outliers, as shown in formula (10): Where N represents the total number of pixels in the image, p represents the total number of pixels in the slice, and Y(p) These represent the pixel values ​​p in the ordinary light image and the light-enhanced image output by the network, respectively. The mean squared error loss function can effectively measure the difference between pixels in two images and performs well in image content enhancement, making the generated image visually closer to the target image. Experiments have shown that the invention has good applicability and can effectively enhance images in different scenarios. In addition, the enhanced image has an overall improved subjective appearance, high brightness, sharpness and deep texture details, and its overall color composition is closer to subjective perception.

Images