Self-supervised low-illumination image enhancement method and device based on Retinex decomposition
Through the self-supervised method based on Retinex decomposition, combined with the unsupervised loss function and the consistency loss function, self-supervised training of reflection and lighting components, the problems of decomposition accuracy and noise suppression in low-light image enhancement are solved, and high-quality low Light image enhancement.
Patent Information
- Application Number
- CN202410968173.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-18
- Publication Date
- 2024-11-12
AI Technical Summary
The existing low-light image enhancement methods have pathological problems of reflection and light component decomposition, which is difficult to adapt to complex and diverse low-light environments. In addition, deep learning methods rely on a large amount of training data, are easy to overfit, and noise is difficult to suppress during the enhancement process.
Using a self-supervised method based on Retinex decomposition, the reflection component estimation network and the lighting component estimation network combine unsupervised loss function and consistency loss function to perform self-supervised training to generate reflection and lighting components of low-light images, and use neighborhood sampling. Strategy to reduce noise.
The combination of interpretability in traditional methods and performance advantages of deep learning methods is achieved, reducing training costs, reducing overfitting risks, improving the visual quality of low-light images, and effectively suppressing noise.
Smart Images

Figure CN118941484A_ABST
Abstract
Description
Technical Field
[0001] This document relates to the field of computer technology, and in particular to a self-supervised low-light image enhancement method and device based on Retinex decomposition. Background Art
[0002] Images captured in low-light environments are usually affected by multiple degradation factors, such as low contrast, low visibility, and sensor noise. These distortions significantly reduce the image quality, which in turn affects subsequent image analysis and processing. Low-light image enhancement algorithms aim to improve the visual quality of images in low-light environments and are a basic and important task in computer vision. Existing low-light image enhancement methods are mainly divided into two categories: traditional methods and deep learning methods.
[0003] Traditional methods, such as histogram equalization and gamma correction, adjust the contrast of an image by expanding the local or global dynamic range. Such methods are simple to implement and have high applicability, but they are prone to over-enhancement or under-enhancement effects, and noise will be amplified while enhancing the image. Another classic method is based on the Retinex theory. The Retinex theory decomposes a given image into two components: reflection and illumination. The theory assumes that illumination describes the light intensity of an object, and the illumination component should be piecewise continuous and textureless, and assumes that the three color channels of the image have the same illumination. The reflection component represents the physical properties of the object and should contain texture and details. Retinex-based methods attempt to use the reflection component as an enhancement result, or to recombine the enhanced illumination component with the reflection component to obtain an enhanced image. Using the color constancy characteristics of the scene, traditional methods based on Retinex decomposition can achieve a good balance in contrast adjustment, detail enhancement, and color constancy. However, Retinex decomposition is a highly ill-posed problem. In order to solve the reflection and illumination components, various image priors are introduced into the decomposition process. However, these manually designed priors are difficult to adapt to complex and diverse low-light environments, resulting in low accuracy of decomposition results and poor generalization ability.
[0004] Low-light image enhancement schemes based on deep learning learn the mapping path from low-light images to normal-light images by training neural networks. Thanks to the powerful feature representation and nonlinear fitting capabilities of deep neural networks, such methods are generally superior to traditional methods. However, low-light image enhancement methods based on deep learning are usually resource-intensive and depend on the quantity and quality of the collected paired training data. For example, the LOL dataset records clean images in low-light environments by changing the exposure time and sensitivity. Complex post-processing is required to eliminate potential pixel shifts between image pairs. In addition, compared with high-level visual tasks such as object classification and detection, existing low-light datasets are relatively small, so the enhancement networks trained in these datasets are at greater risk of overfitting. Summary of the invention
[0005] The purpose of the present invention is to provide a self-supervised low-light image enhancement method and device based on Retinex decomposition, aiming to solve the above-mentioned problems in the prior art.
[0006] The present invention provides a self-supervised low-light image enhancement method based on Retinex decomposition, comprising:
[0007] Randomly initialize the reflection component estimation network and obtain training samples, use a neighborhood sampler to perform low-light image sampling on the training samples, and generate two first sub-images;
[0008] Using a reflection component estimation network and an illumination component estimation network to estimate the reflection component and the illumination component of a first sub-image, and evaluating the illumination component estimation results and the first reflection component estimation results based on an unsupervised loss function of the Retinex theory;
[0009] Using the reflection component estimation network to estimate the reflection component of another first sub-image, to obtain a second reflection component estimation result;
[0010] Using a reflection component estimation network to estimate the reflection component of the original low-light image in the training sample, using the neighborhood sampler to sample the reflection component of the training sample to generate two second sub-images, and obtaining a third reflection component estimation result and a fourth reflection component estimation result corresponding to the two second sub-images;
[0011] Calculating a consistency loss function based on the first reflection component estimation result, the second reflection component estimation result, the third reflection component estimation result, and the fourth reflection component estimation result, and training the reflection component estimation network to obtain a trained reflection component estimation network;
[0012] The test sample is input into the trained reflection component estimation network to obtain the reflection component estimation of the test sample.
[0013] The present invention provides a self-supervised low-light image enhancement device based on Retinex decomposition, comprising:
[0014] A preprocessing module, used for randomly initializing a reflection component estimation network and obtaining training samples, and performing low-light image sampling on the training samples using a neighborhood sampler to generate two first sub-images;
[0015] An estimation module is used to use a reflection component estimation network and an illumination component estimation network to perform reflection component estimation and illumination component estimation on a first sub-image, and evaluate the illumination component estimation result and the first reflection component estimation result based on an unsupervised loss function of the Retinex theory; use the reflection component estimation network to perform reflection component estimation on another first sub-image to obtain a second reflection component estimation result; use the reflection component estimation network to perform reflection component estimation on an original low-light image in the training sample, use the neighborhood sampler to sample the reflection component of the training sample, generate two second sub-images, and obtain a third reflection component estimation result and a fourth reflection component estimation result corresponding to the two second sub-images;
[0016] A training module is used to calculate a consistency loss function based on the first reflection component estimation result, the second reflection component estimation result, the third reflection component estimation result and the fourth reflection component estimation result, and to train the reflection component estimation network to obtain a trained reflection component estimation network.
[0017] An embodiment of the present invention also provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of the above-mentioned self-supervised low-light image enhancement method based on Retinex decomposition are implemented.
[0018] An embodiment of the present invention also provides a computer-readable storage medium, on which a program for implementing information transmission is stored. When the program is executed by a processor, the steps of the above-mentioned self-supervised low-light image enhancement method based on Retinex decomposition are implemented.
[0019] By adopting the embodiments of the present invention, it is possible to have both the interpretability of traditional methods and the performance advantages of deep learning methods, get rid of the dependence on paired training data, reduce training costs, and effectively improve the quality of low-light image enhancement. Through the self-supervised training method, the risk of overfitting of the model is greatly reduced. In addition, low-light image noise suppression is achieved through the neighborhood sampling strategy and consistency loss. The consistency loss function enables the network to learn the reflection component more accurately, thereby effectively removing the sensor noise amplified during the enhancement process and improving the overall visual quality of the enhanced image. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] In order to more clearly illustrate one or more embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0021] Figure 1 is a flow chart of a self-supervised low-light image enhancement method based on Retinex decomposition according to an embodiment of the present invention;
[0022] Figure 2 is a flowchart of detailed processing of a self-supervised low-light image enhancement method based on Retinex decomposition according to an embodiment of the present invention;
[0023] Figure 3 is a schematic diagram of the overall framework of a self-supervised low-light image enhancement method based on Retinex decomposition according to an embodiment of the present invention;
[0024] Figure 4 is a schematic diagram of a neighborhood sampling strategy according to an embodiment of the present invention;
[0025] Figure 5 is a schematic diagram of the structure of a reflection component estimation network and an illumination component estimation network according to an embodiment of the present invention;
[0026] Figure 6 is a schematic diagram of a comparison of subjective enhancement results according to an embodiment of the present invention;
[0027] Figure 7 is a schematic diagram of a self-supervised low-light image enhancement device based on Retinex decomposition according to an embodiment of the present invention;
[0028] Figure 8 is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0029] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the following will be combined with the drawings in one or more embodiments of this specification to clearly and completely describe the technical solutions in one or more embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this document.
[0030] Method Embodiment
[0031] According to an embodiment of the present invention, a self-supervised low-light image enhancement method based on Retinex decomposition is provided. Figure 1 is a flow chart of a self-supervised low-light image enhancement method based on Retinex decomposition according to an embodiment of the present invention. Figure 1 As shown, the self-supervised low-light image enhancement method based on Retinex decomposition according to an embodiment of the present invention specifically includes:
[0032] Step S101, randomly initializing a reflection component estimation network and obtaining training samples, using a neighborhood sampler to perform low-light image sampling on the training samples to generate two first sub-images;
[0033] The step of using a neighborhood sampler to perform low-light image sampling on the training sample to generate two first sub-images specifically includes:
[0034] Step 1, dividing the original image of the training sample into non-overlapping units of size 2×2;
[0035] Step 2, for each cell, randomly sample a pixel and an adjacent pixel;
[0036] Step 3, repeat step 2 for all cells to obtain two first sub-images with a width of W / 2 and a height of H / 2.
[0037] Step S102, using a reflection component estimation network and an illumination component estimation network to perform reflection component estimation and illumination component estimation on a first sub-image, and evaluating the illumination component estimation result and the first reflection component estimation result based on an unsupervised loss function of the Retinex theory;
[0038] The reflection component estimation network specifically includes: a first convolution layer, a first ReLU layer, a second convolution layer, a second ReLU layer, a third convolution layer, a third ReLU layer, a first channel attention layer, a second channel attention layer, a third channel attention layer, a fourth channel attention layer, a fourth convolution layer, a fourth ReLU layer, a fifth convolution layer, a fifth ReLU layer, a sixth convolution layer, a Sigmoid layer, and 3 output channels connected in sequence;
[0039] The illumination component estimation network specifically includes: a first convolutional layer, a first ReLU layer, a second convolutional layer, a second ReLU layer, a third convolutional layer, a third ReLU layer, a first channel attention layer, a second channel attention layer, a third channel attention layer, a fourth channel attention layer, a fourth convolutional layer, a fourth ReLU layer, a fifth convolutional layer, a fifth ReLU layer, a sixth convolutional layer, a Sigmoid layer, and 1 output channel, which are connected in sequence.
[0040] The unsupervised loss function based on Retinex theory is specifically:
[0041]
[0042] Among them, θ 1 represents the unsupervised loss function, L represents the illumination component, R represents the reflection component, I represents the low-light image, and L 0 represents the initialized illumination component, k and t are the prior terms of illumination component and reflection component respectively, α 0 , α 1 , α 3 represents weight, R, G, B represent the three channels of illumination, x, y represent the horizontal and vertical directions respectively, is the derivative operator, w x and w y is the weight, G represents Gaussian filtering, * represents convolution, I g I represents the grayscale image, N represents the number of 16×16 non-overlapping regions, Y represents the mean of the 16×16 non-overlapping regions, and v is the exposure.
[0043] Step S103, using a reflection component estimation network to estimate the reflection component of another first sub-image to obtain a second reflection component estimation result;
[0044] Step S104, using the reflection component estimation network to estimate the reflection component of the original low-light image in the training sample, using the neighborhood sampler to sample the reflection component of the training sample to generate two second sub-images, and obtaining a third reflection component estimation result and a fourth reflection component estimation result corresponding to the two second sub-images;
[0045] The step of sampling the reflection component of the training sample using the neighborhood sampler to generate two second sub-images specifically includes:
[0046] Step 1, dividing the reflection component of the training sample into non-overlapping units of size 2×2;
[0047] Step 2, for each cell, randomly sample a pixel and an adjacent pixel;
[0048] Step 3, repeat step 2 for all cells to obtain two second sub-images with a width of W / 2 and a height of H / 2.
[0049] Step S105, calculating a consistency loss function based on the first reflection component estimation result, the second reflection component estimation result, the third reflection component estimation result and the fourth reflection component estimation result, training the reflection component estimation network to obtain a trained reflection component estimation network;
[0050] The consistency loss function specifically includes:
[0051]
[0052] Among them, θ 2 represents the consistency loss function, r 1 With r 2 represents the estimation result of the first reflection component and the second reflection component of the sub-image, R 1 With R 2 The third reflection component estimation result and the fourth reflection component estimation result of the original low-light image are represented.
[0053] The step of training the reflection component estimation network to obtain a trained reflection component estimation network specifically includes:
[0054] The reflection component estimation network is trained to determine whether the reflection component estimation network converges. If not, the network parameters are updated and the training is performed again. If convergence is determined, the training ends to obtain a trained reflection component estimation network.
[0055] Step S106: input the test sample into the trained reflection component estimation network to obtain the reflection component estimation of the test sample.
[0056] By adopting the embodiment of the present invention, the advantages of Retinex theory and deep learning technology are combined to decompose low-light images into reflection and illumination components in a self-supervised manner. Based on the Retinex theory, a set of unsupervised loss functions are designed to guide the convolutional neural network to predict the illumination component and reflection component of the low-light image. These loss functions include reconstruction loss for reconstructing the original image, initialization loss for calculating the initial estimate, and prior loss for ensuring accurate decomposition. The embodiment of the present invention can combine the interpretability of traditional methods with the performance advantages of deep learning methods, can get rid of the dependence on paired training data, reduce training costs, and effectively improve the quality of low-light image enhancement. Through the self-supervised training method, the risk of overfitting of the model is greatly reduced. In addition, low-light image noise suppression is achieved through neighborhood sampling strategy and consistency loss. A neighborhood sampling strategy is provided to randomly sample the original low-light image to generate two conditionally independent and content-similar sub-images. Based on this strategy, a consistency loss function is further provided, so that the network can learn the reflection component more accurately, thereby effectively removing the sensor noise amplified during the enhancement process and improving the overall visual quality of the enhanced image.
[0057] The embodiment of the present invention proposes a self-supervised low-light image enhancement method based on Retinex decomposition, which successfully achieves image contrast enhancement and noise removal in low-light environments. Figure 2 As shown, specifically, the present invention is divided into two processes: a training phase and a testing phase. The specific steps of the training phase are as follows:
[0058] Step (1): Randomly initialize the network;
[0059] Step (2): Randomly select training samples from the training set;
[0060] Step (3): Use a neighborhood sampler (such as Figure 4 As shown in the figure), the low-light image is sampled to generate two random sub-images;
[0061] Step (4): Use the network to estimate the illumination and reflection components of the two sub-images, and use the same network to estimate the reflection component of the original image;
[0062] Step (5): Use the same neighborhood sampler to sample the reflection component of the original image to generate two sub-images;
[0063] Step (6): Calculate the loss function based on the estimated illumination and reflection components;
[0064] Step (7): Determine whether the network has converged. If not, update the network parameters and jump to step (2); if converged, end the training.
[0065] In the testing phase, the reflection component (enhanced image) of the test image is obtained based on the trained network.
[0066] The overall framework of the embodiment of the present invention is as follows Figure 3 As shown. In the training stage, the sub-image is used as the input of the illumination component estimation network and the reflection component estimation network, and the unsupervised loss based on Retinex theory is used to evaluate the current decomposition and guide the network to generate more accurate results. Specifically, the present invention proposes the following unsupervised loss for optimization:
[0067]
[0068] Among them, L represents the illumination component, R represents the reflection component, I represents the low-light image, and L 0 represents the initialized illumination component, k and t are the prior terms of illumination component and reflection component respectively, α 0 , α 1 , α3 represents the weight. The present invention uses the maximum value of the R, G, and B channels as the initial estimation of the illumination:
[0069]
[0070] Considering that the illumination of the surface of a natural image should be piecewise smooth, the present invention designs a gradient-based prior constraint k to ensure that the estimated illumination is piecewise smooth:
[0071]
[0072] Among them, x and y represent the horizontal and vertical directions respectively. is the derivative operator, wx and wy are weights. wx and wy are calculated as follows:
[0073]
[0074] Wherein, G represents Gaussian filtering, * represents convolution, and Ig represents the grayscale image of I. In order to ensure that the enhanced image has a reasonable exposure level, the present invention designs an exposure control prior t:
[0075]
[0076] Where N is the number of 16×16 non-overlapping regions, Y is the mean of 16×16 non-overlapping regions, and v is the exposure. In the experiment, v is set to 0.5.
[0077] Unsupervised loss based on Retinex theory can guide image decomposition well. However, while improving the contrast of low-light images, the noise hidden in the dark area will also be amplified at the same time. In order to solve this problem, the present invention introduces consistency constraints. Most existing methods remove noise by adding smoothing terms or post-processing. The former easily leads to excessive smoothing of image details, while the latter increases additional computational costs. The present invention combines the neighborhood sampling strategy, performs Retinex decomposition on the sub-image, and makes full use of two specific sub-images to remove noise. Specifically, the present invention proposes the following consistency loss:
[0078]
[0079] Where r1 and r2 represent the reflectance components of the sub-image, and R1 and R2 represent the reflectance components of the original low-light image. Formula 5 uses independent noisy images of the same scene to train the denoising network. The second term in Formula 6 is used to reduce over-smoothing because r1 and r2 are not exactly the same.
[0080] Finally, the proposed method combines the unsupervised loss based on Retinex theory and the consistency loss based on the neighborhood sampling strategy to train the network. In the test phase, the trained reflectance component estimation network is used to directly predict the reflectance component of the low-light image to obtain the enhanced image.
[0081] The neighborhood sampling strategy of the embodiment of the present invention is as follows Figure 4 Given a low-light image with width W and height H, the proposed neighborhood sampling strategy generates two sub-images according to the following steps:
[0082] Step (1): Divide the original image into 2×2 non-overlapping units;
[0083] Step (2): For each cell, randomly sample a pixel and a neighboring pixel (each pixel has three neighboring pixels);
[0084] Step (3): Repeat step (2) for all cells to obtain two sub-images with a width of W / 2 and a height of H / 2.
[0085] The neighborhood sampling strategy proposed in the embodiment of the present invention can generate two sub-images that are similar in content but independent in conditions. This is because the pixels at corresponding positions of the sub-images are neighbors in the original image.
[0086] The reflection component estimation network and the illumination component estimation network are as follows Figure 5As shown in the figure. The reflection component estimation network and the illumination component estimation network are roughly the same, except that the number of output channels of the reflection component estimation network is 3, while the number of output channels of the illumination component estimation network is 1. This is because the Retinex theory assumes that the three color channels of the image have the same illumination. Both networks end with a Sigmoid layer to ensure that the pixel values are in the range [0, 1]. Both networks consist of a convolutional layer, a ReLU activation function layer, and a channel attention layer. The network structure does not contain a downsampling layer to better preserve the texture details of the image. Figure 5 The numbers in indicate the number of channels.
[0087] The embodiments of the present invention have achieved good results on both benchmark datasets. It can be seen from the objective quality evaluation indicators and subjective enhancement results that the proposed method can significantly improve the visual quality of low-light images. The experimental results are shown in Tables 1 and Figure 6 As shown, Table 1 shows the performance comparison of LOL and SICE datasets.
[0088] Table 1
[0089]
[0090]
[0091] In summary, the embodiments of the present invention provide a hybrid image enhancement architecture (such as Figure 2 and Figure 3 As shown in the figure, the interpretability of traditional methods and the performance advantages of deep learning methods are fully utilized to achieve high-quality, self-supervised quality enhancement of low-light images. In view of the noise amplification problem in the process of low-light image enhancement, the embodiment of the present invention also provides a neighborhood sampling strategy for low-light image enhancement (as shown in the figure). Figure 4 In addition, the embodiment of the present invention also provides a convolutional neural network structure based on a channel attention mechanism and non-downsampling operation (as shown in Figure 5 It can extract rich low-light image feature representations while better preserving the image’s texture detail information.
[0092] The technical solution of the embodiment of the present invention has the following beneficial effects:
[0093] (1) Improve scene visibility. Through innovative self-supervised learning methods and hybrid architecture design, the present invention combines the advantages of traditional methods and deep learning, so that it has a higher enhancement effect and interpretability, and can effectively enhance scene visibility.
[0094] (2) Efficient denoising. Most existing methods ignore the noise problem of low-light images, or use simple post-processing to remove noise, which is difficult to achieve ideal results. This paper designs a neighborhood sampling strategy and consistency loss function that can effectively remove amplified sensor noise and preserve the texture details of the image.
[0095] (3) Strong model adaptability: The method proposed in the present invention does not require paired training data, which makes the present invention more flexible and adaptable in practical applications and can handle a variety of low-light scenes.
[0096] (4) Improve the performance of downstream visual tasks. While enhancing low-light images to natural-light images, the present invention is expected to significantly improve the performance of downstream computer vision tasks such as object recognition, detection, and segmentation in low-light scenes, thereby providing strong support for applications such as autonomous driving, safety monitoring, and image analysis.
[0097] Device Example 1
[0098] According to an embodiment of the present invention, a self-supervised low-light image enhancement device based on Retinex decomposition is provided. Figure 7 is a schematic diagram of a self-supervised low-light image enhancement device based on Retinex decomposition according to an embodiment of the present invention. Figure 7 As shown, the self-supervised low-light image enhancement device based on Retinex decomposition according to an embodiment of the present invention specifically includes:
[0099] A preprocessing module 70 is used to randomly initialize the reflection component estimation network and obtain training samples, and use a neighborhood sampler to perform low-light image sampling on the training samples to generate two first sub-images;
[0100] An estimation module 72 is used to use a reflection component estimation network and an illumination component estimation network to perform reflection component estimation and illumination component estimation on a first sub-image, and evaluate the illumination component estimation result and the first reflection component estimation result based on an unsupervised loss function of the Retinex theory; use the reflection component estimation network to perform reflection component estimation on another first sub-image to obtain a second reflection component estimation result; use the reflection component estimation network to perform reflection component estimation on the original low-light image in the training sample, use the neighborhood sampler to sample the reflection component of the training sample, generate two second sub-images, and obtain a third reflection component estimation result and a fourth reflection component estimation result corresponding to the two second sub-images;
[0101] The training module 74 is used to calculate the consistency loss function based on the first reflection component estimation result, the second reflection component estimation result, the third reflection component estimation result and the fourth reflection component estimation result, and train the reflection component estimation network to obtain a trained reflection component estimation network.
[0102] The embodiment of the present invention is a device embodiment corresponding to the above method embodiment. The specific operations of each module can be understood by referring to the description of the method embodiment, which will not be repeated here.
[0103] Device Example 2
[0104] An embodiment of the present invention provides an electronic device, such as Figure 8 As shown, it includes: a memory 80, a processor 82, and a computer program stored in the memory 80 and executable on the processor 82. When the computer program is executed by the processor 82, the steps described in the method embodiment are implemented.
[0105] Device Example 3
[0106] An embodiment of the present invention provides a computer-readable storage medium, on which a program for implementing information transmission is stored. When the program is executed by the processor 82, the steps described in the method embodiment are implemented.
[0107] The computer-readable storage medium in this embodiment includes, but is not limited to, ROM, RAM, magnetic disk or optical disk, etc.
[0108] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A self-supervised low-light image enhancement method based on Retinex decomposition, characterized in that: include: Randomly initialize the reflection component estimation network and obtain training samples, use a neighborhood sampler to perform low-light image sampling on the training samples, and generate two first sub-images; Using a reflection component estimation network and an illumination component estimation network to estimate the reflection component and the illumination component of a first sub-image, and evaluating the illumination component estimation results and the first reflection component estimation results based on an unsupervised loss function of the Retinex theory; Using the reflection component estimation network to estimate the reflection component of another first sub-image, to obtain a second reflection component estimation result; Using a reflection component estimation network to estimate the reflection component of the original low-light image in the training sample, using the neighborhood sampler to sample the reflection component of the training sample to generate two second sub-images, and obtaining a third reflection component estimation result and a fourth reflection component estimation result corresponding to the two second sub-images; Calculating a consistency loss function based on the first reflection component estimation result, the second reflection component estimation result, the third reflection component estimation result, and the fourth reflection component estimation result, and training the reflection component estimation network to obtain a trained reflection component estimation network; The test sample is input into the trained reflection component estimation network to obtain the reflection component estimation of the test sample.
2. The method according to claim 1, characterized in that The reflection component estimation network specifically includes: a first convolution layer, a first ReLU layer, a second convolution layer, a second ReLU layer, a third convolution layer, a third ReLU layer, a first channel attention layer, a second channel attention layer, a third channel attention layer, a fourth channel attention layer, a fourth convolution layer, a fourth ReLU layer, a fifth convolution layer, a fifth ReLU layer, a sixth convolution layer, a Sigmoid layer, and 3 output channels connected in sequence; The illumination component estimation network specifically includes: a first convolutional layer, a first ReLU layer, a second convolutional layer, a second ReLU layer, a third convolutional layer, a third ReLU layer, a first channel attention layer, a second channel attention layer, a third channel attention layer, a fourth channel attention layer, a fourth convolutional layer, a fourth ReLU layer, a fifth convolutional layer, a fifth ReLU layer, a sixth convolutional layer, a Sigmoid layer, and 1 output channel, which are connected in sequence.
3. The method according to claim 1, characterized in that The unsupervised loss function based on Retinex theory is specifically: Among them, θ1 represents the unsupervised loss function, L represents the illumination component, R represents the reflection component, I represents the low-light image, L0 represents the initialized illumination component, k and t are the prior terms of the illumination component and the reflection component respectively, α0, α1, α3 represent weights, R, G, B represent the three channels of illumination, x, y represent the horizontal and vertical directions respectively, is the derivative operator, w x and w y is the weight, G represents Gaussian filtering, * represents convolution, I g I represents the grayscale image, N represents the number of 16×16 non-overlapping regions, Y represents the mean of the 16×16 non-overlapping regions, and v is the exposure.
4. The method according to claim 1, characterized in that: The consistency loss function specifically includes: Wherein, θ2 represents the consistency loss function, r1 and r2 represent the estimation results of the first reflection component and the second reflection component of the sub-image, and R1 and R2 represent the estimation results of the third reflection component and the fourth reflection component of the original low-light image.
5. The method according to claim 1, characterized in that: Using a neighborhood sampler to perform low-light image sampling on the training sample to generate two first sub-images specifically includes: Step 1, dividing the original image of the training sample into non-overlapping units of size 2×2; Step 2, for each cell, randomly sample a pixel and an adjacent pixel; Step 3, repeat step 2 for all cells to obtain two first sub-images with a width of W / 2 and a height of H / 2.
6. The method according to claim 1, characterized in that Sampling the reflection component of the training sample using the neighborhood sampler to generate two second sub-images specifically includes: Step 1, dividing the reflection component of the training sample into non-overlapping units of size 2×2; Step 2, for each cell, randomly sample a pixel and an adjacent pixel; Step 3, repeat step 2 for all cells to obtain two second sub-images with a width of W / 2 and a height of H / 2.
7. The method according to claim 1, characterized in that Training the reflection component estimation network to obtain a trained reflection component estimation network specifically includes: The reflection component estimation network is trained to determine whether the reflection component estimation network converges. If not, the network parameters are updated and the training is performed again. If convergence is determined, the training ends to obtain a trained reflection component estimation network.
8. A self-supervised low-light image enhancement device based on Retinex decomposition, characterized in that: include: A preprocessing module, used for randomly initializing a reflection component estimation network and obtaining training samples, and performing low-light image sampling on the training samples using a neighborhood sampler to generate two first sub-images; An estimation module, used for performing reflection component estimation and illumination component estimation on a first sub-image using a reflection component estimation network and an illumination component estimation network, and evaluating the illumination component estimation result and the first reflection component estimation result based on an unsupervised loss function of Retinex theory; Using the reflection component estimation network to estimate the reflection component of another first sub-image, to obtain a second reflection component estimation result; Using a reflection component estimation network to estimate the reflection component of the original low-light image in the training sample, using the neighborhood sampler to sample the reflection component of the training sample to generate two second sub-images, and obtaining a third reflection component estimation result and a fourth reflection component estimation result corresponding to the two second sub-images; A training module is used to calculate a consistency loss function based on the first reflection component estimation result, the second reflection component estimation result, the third reflection component estimation result and the fourth reflection component estimation result, and to train the reflection component estimation network to obtain a trained reflection component estimation network.
9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of the self-supervised low-light image enhancement method based on Retinex decomposition as described in any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an implementation program for information transmission, and when the program is executed by a processor, the steps of the self-supervised low-light image enhancement method based on Retinex decomposition as described in any one of claims 1 to 7 are implemented.
Citation Information
Cited By
Low-illumination image enhancement modeling method, enhancement method, equipment and storage medium
CN121073850A
Low-illumination image enhancement modeling method, enhancement method, device and storage medium
CN121073850B
