Unsupervised Single Image Enhancement Method and System Based on Retinex Fractional Variational Network

Abstract

The present invention proposes an unsupervised single image enhancement method and system based on a Retinex fractional-order variational network, which belongs to the field of image processing. It includes: obtaining a single observation image and preprocessing the observation image; building a BP neural network, constructing a Retinex fractional-order variational framework as the objective function of the BP neural network, and obtaining a Retinex fractional-order variational network; training the Retinex fractional-order variational network to obtain a trained Retinex fractional-order variational network; inputting the preprocessed image into the trained Retinex fractional-order variational network to obtain an estimated illumination image; based on the illumination image, calculating the reflection image, and correcting the reflection image to obtain a single enhanced image. The present invention does not require the collection of a large amount of target data, realizes the single image enhancement function, and does not require the production of samples and corresponding label data. It is an unsupervised optimization method that can not only enhance the contrast of the target image, but also solve the problem that the texture detail features of the low-light area are difficult to restore during the enhancement process.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and in particular relates to an unsupervised single image enhancement method and system based on a Retinex fractional-order variational network. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] High-quality image signals are essential for efficient computer vision systems. During image capture, low contrast, poor visibility, and blurred details (such as edges and textures) can occur due to low lighting conditions, uneven illumination, and the quality of the image capture equipment. These issues can complicate subsequent object recognition, tracking, foreground and background segmentation, and ultimately, image understanding and predictive analysis.

[0004] To better utilize these severely degraded images, techniques such as image enhancement are inevitably needed. Image enhancement intentionally emphasizes the overall or local characteristics of an image, sharpening previously unclear images and magnifying the differences between features of different objects within the image. This improves image quality, enhances image recognition, and satisfies specific analytical needs. Image enhancement methods primarily fall into the following categories: histogram equalization, homomorphic filtering, wavelet transform, image enhancement algorithms based on physical dehazing models, image enhancement algorithms based on deep learning, and image enhancement algorithms based on Retinex. Methods based on Retinex theory are currently the most popular image enhancement algorithms.

[0005] With the in-depth study of various image enhancement algorithms, many problems have also emerged in the image processing process, such as sensitivity to noise, loss of detail information, over-enhancement in certain result areas, or relatively low contrast enhancement. Therefore, choosing an appropriate variational model is particularly important. With the widespread promotion of deep learning, neural network-based methods have been successfully applied to solving complex differential equations due to their powerful expressive power. Compared with some classic numerical iterative solutions, neural network-based methods significantly shorten training time while ensuring accuracy. Retinex-based variational methods are mostly solved using classic optimization algorithms such as the Split Bregman method. However, there is relatively little research on using deep learning algorithms to solve variational models.

[0006] The inventors found that some researchers have combined deep learning algorithms for image enhancement, but most of them use large labeled data sets to train the network. In fact, the number of low-light image data sets is limited, and there is still a difference between the synthesized low-light images and the actual images. Summary of the Invention

[0007] To overcome the shortcomings of the above-mentioned prior art, the present invention provides an unsupervised single image enhancement method and system based on the Retinex fractional-order variational network. This method does not require the collection of a large amount of target data, realizes the single image enhancement function, and does not require the production of samples and corresponding label data. It is an unsupervised optimization method that can not only enhance the contrast of the target image, but also solve the problem that the texture detail features of low-light areas are difficult to restore during the enhancement process.

[0008] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0009] The first aspect of the present invention provides an unsupervised single image enhancement method based on the Retinex fractional-order variational network.

[0010] The unsupervised single image enhancement method based on Retinex fractional-order variational network includes the following steps:

[0011] Acquire a single observation image and preprocess the observation image;

[0012] Build a BP neural network, establish the objective function of the BP neural network based on Retinex theory and fractional-order variation method, and obtain the Retinex fractional-order variation network;

[0013] The Retinex fractional-order variational network is trained using a single preprocessed observation image to obtain a trained Retinex fractional-order variational network.

[0014] The preprocessed image is input into the trained Retinex fractional-order variational network to obtain the estimated illumination image;

[0015] Based on the illumination image, the reflected image is calculated and corrected to obtain a single enhanced image.

[0016] The second aspect of the present invention provides an unsupervised single image enhancement system based on the Retinex fractional-order variational network.

[0017] An unsupervised single image enhancement system based on the Retinex fractional-order variational network, including:

[0018] The image preprocessing module is configured to: obtain a single observation image and preprocess the observation image;

[0019] The Retinex fractional-order variational network building module is configured to: build a BP neural network, establish the objective function of the BP neural network based on the Retinex theory and the fractional-order variational method, and obtain the Retinex fractional-order variational network;

[0020] The training module is configured to: train the Retinex fractional-order variational network using a preprocessed single observation image to obtain a trained Retinex fractional-order variational network;

[0021] The illumination image acquisition module is configured to: input the preprocessed image into the trained Retinex fractional-order variational network to obtain an estimated illumination image;

[0022] The enhanced image acquisition module is configured to: calculate the reflected image based on the illumination image, and correct the reflected image to obtain a single enhanced image.

[0023] The third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of the unsupervised single image enhancement method based on the Retinex fractional-order variational network as described in the first aspect of the present invention.

[0024] The fourth aspect of the present invention provides an electronic device, comprising a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps of the unsupervised single image enhancement method based on the Retinex fractional-order variational network as described in the first aspect of the present invention are implemented.

[0025] One or more of the above technical solutions have the following beneficial effects:

[0026] 1. The present invention constructs an objective function based on the Retinex fractional-order variational model, and uses fractional-order differentials and fractional-order powers to control the degree of enhancement of image detail information, thereby achieving the effect of retaining image detail information.

[0027] 2. The present invention uses a neural network to nonlinearly represent the mapping relationship between the coordinates of pixel points in the image and the color values, adopts the Retinex fractional-order variational model as the objective function to constrain the training of the neural network, constructs a Retinex-based fractional-order variational network, and estimates the illumination image from the observed image. The entire algorithm realizes the true single image enhancement function, that is, a single image input corresponds to a single image output, without the need to collect a large number of target images, and without the need to create label images corresponding to samples and target images. It is an unsupervised image enhancement algorithm that saves a lot of manpower and material resources.

[0028] 3. The present invention can not only enhance the contrast of the target image, but also solve the problem that the texture detail features in the low-light area are difficult to restore during the enhancement process.

[0029] Advantages of additional aspects of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0031] Figure 1 This is a flowchart of the unsupervised single image enhancement method based on the Retinex fractional-order variational network of the present invention.

[0032] Figure 2 This is the Retinex fractional-order variational network architecture diagram of the present invention.

[0033] Figure 3(a) is a low-light target image.

[0034] Figure 3(b) is the result image after the MF algorithm processes Figure 3(a).

[0035] Figure 3(c) is the result image after the HE algorithm processes Figure 3(a).

[0036] Figure 3(d) is the result image after the BPDHE algorithm processes Figure 3(a).

[0037] Figure 3(e) is the result image after the PM_SIRE algorithm processes Figure 3(a).

[0038] FIG3( f ) is the result image after FIG3( a ) is processed by the method of the present invention. DETAILED DESCRIPTION

[0039] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0040] It should be noted that the terms used herein are for describing particular embodiments only and are not intended to limit the exemplary embodiments according to the present invention.

[0041] In the absence of conflict, the embodiments of the present invention and the features thereof may be combined with each other.

[0042] The overall idea proposed by the present invention is:

[0043] This paper proposes an unsupervised single-image enhancement method based on a Retinex fractional-order variational network. Based on the Gray Word hypothesis and the assumption of spatial smoothness of the illumination image and the reflection image, an objective function for unsupervised single-image enhancement is constructed. A neural network is used to represent the nonlinear mapping relationship between pixel positions and pixel values in the illumination image, and a Retinex fractional-order variational network is established to estimate the illumination image from a single observed image. The reflection image is calculated and corrected using the Retinex theoretical formula to obtain the enhanced low-light image. This method achieves single-image enhancement without the need to collect large amounts of target data. It also eliminates the need to generate samples and corresponding label data. This unsupervised optimization method not only enhances the contrast of the target image but also addresses the difficulty in restoring texture detail features in low-light areas during the enhancement process.

[0044] Example 1

[0045] This embodiment discloses an unsupervised single image enhancement method based on the Retinex fractional-order variational network.

[0046] The unsupervised single image enhancement method based on Retinex fractional-order variational network includes the following steps:

[0047] Acquire a single observation image and preprocess the observation image;

[0048] Build a BP neural network, establish the objective function of the BP neural network based on Retinex theory and fractional-order variation method, and obtain the Retinex fractional-order variation network;

[0049] The Retinex fractional-order variational network is trained using a single preprocessed observation image to obtain a trained Retinex fractional-order variational network.

[0050] The preprocessed image is input into the trained Retinex fractional-order variational network to obtain the estimated illumination image;

[0051] Based on the illumination image, the reflected image is calculated and corrected to obtain a single enhanced image.

[0052] In this embodiment, if Figure 1-Figure 2 As shown in the figure, the unsupervised single image enhancement method based on the Retinex fractional-order variational network includes the following steps:

[0053] Step 1: Obtain a single observation image and preprocess the observation image. The specific steps are as follows:

[0054] Step 1.1, read the observation image I(x,y), and convert I(x,y) from RGB color space to HSV color space;

[0055] Step 1.2, separate the three channels of the observed image in the HSV color space: hue (H), saturation (S), and value (V);

[0056] Step 1.3: Normalize the V component of the observed image to the range of [0,1] and set it to V i (x,y);

[0057] Step 1.4, use the formula i(x,y)=log(V i (x,y)) maps the V component of the observed image to the logarithmic domain to obtain i(x,y).

[0058] Step 2: Build a BP neural network. Based on the Retinex theory and fractional-order variation method, establish the objective function of the BP neural network and obtain the Retinex fractional-order variation network. The specific steps are as follows:

[0059] Step 2.1: Based on the BP neural network, construct a three-layer neural network structure, such as Figure 2 As shown in the figure, the number of input layer nodes is 2, representing the coordinates x and y of the pixel points of the observed image in the image, the number of output layer nodes is 1, representing the pixel value l(x,y) of the illuminated image at the (x,y) position, the number of hidden layer nodes is n, and the activation function uses the Sigmoid function.

[0060] The position X = (x, y) of each pixel in the observed image is used as the input of the Retinex fractional-order variation network, and the pixel value l(x, y) at the corresponding position (x, y) of the illumination image to be estimated is used as the output. The relationship between the input and output of the Retinex fractional-order variation network is expressed as:

[0061] l=f(Uf(WX))

[0062] Among them, W and U represent the weights of the hidden layer and the output layer respectively, f is the Sigmoid function, and l is the output of the Retinex fractional-order variational network.

[0063] In step 2.2, the illumination image is used as the target variable. Based on the Retinex theory and Gray Word hypothesis, the objective function of the BP neural network is established based on the Retinex theory and the fractional variation method. The specific objective function is as follows:

[0064]

[0065] Where Ω∈R 2 is the target image area, i is the observed image, v1 and v2 are both positive real numbers, represents the v1-order differential operator, α1, α2, and α3 are all constant coefficients, and l is the output of the Retinex fractional-order variational network.

[0066] in:

[0067] Item 1 The role of is to constrain the spatial smoothness of the illumination image;

[0068] Item 2 It is the fidelity term, which ensures that the gap between the illumination image l and the observation image i is not too large;

[0069] The third term ReLU(il) is a regular term added to satisfy the condition l≥i;

[0070] Item 4 It is a gradient penalty term used to ensure the clarity and visual effect of the reflectivity r (r = il);

[0071] Item 5 The Gray Word hypothesis is used as the theoretical basis, and its purpose is to remove the influence of ambient light on actual objects.

[0072] The Gray Word hypothesis states that for an image with a large number of color changes, the average values of the three components tend to the same gray value G, which is approximately "gray", that is, 1 / 2 (the color value is in the range of 0 to 1). It can be expressed as (R-1 / 2)→0, or as (e r -1 / 2)→0.

[0073] Furthermore, the Retinex fractional-order variational network is trained using the preprocessed single observation image to obtain a trained Retinex fractional-order variational network, which specifically includes:

[0074] Step 2.3, initialize the Retinex fractional-order variational network structure, set the maximum number of iterations to 30, the learning rate to 0.5, the number of hidden layer nodes to 16, and randomly initialize the network weights W and U using a normal distribution.

[0075] Step 2.4, the specific steps for training the Retinex fractional-order variational network model are as follows:

[0076] Step 2.4.1, forward propagation process.

[0077] The pixel positions X = (x, y) are input in sequence. After calculation by the neurons in each layer of the Retinex fractional-order variational network, the network output is expressed by the following formula, that is, the pixel value l(x, y) at the corresponding position in the illumination image:

[0078] l=f(Uf(WX))

[0079] Substituting the network output into the objective function, we can get:

[0080]

[0081] Step 2.4.2, back propagation process.

[0082] Calculate the derivative of the objective function with respect to the connection weights of each layer respectively, and use the gradient descent method to update the weights. The update formula of the weight matrix is:

[0083]

[0084]

[0085] Among them, W(k+1) is the weight W after the K+1th iteration, W(k) is the weight W after the Kth iteration, U(k+1) is the weight U after the K+1th iteration, U(k) is the weight U after the Kth iteration, and η is the learning rate.

[0086] The network is trained iteratively through the above iterative formula until the maximum number of iterations is reached and the training is stopped.

[0087] Furthermore, the preprocessed image is input into the trained Retinex fractional-order variational network to obtain the estimated illumination image and output the result.

[0088] Furthermore, based on the illumination image, the reflected image is calculated and corrected to obtain a single enhanced image, specifically:

[0089] Step 3.1: According to the Retinex theory formula, derive the mapping r(x,y) of the V component of the reflected image in the logarithmic domain, specifically:

[0090] r(x,y)=i(x,y)-l(x,y)

[0091] Step 3.2: Convert r(x,y) through exponential transformation to obtain the V component R of the reflected image V (x,y):

[0092] R V (x,y)=exp(r(x,y))

[0093] Step 3.3: Use Gamma correction and CLAHE algorithm to correct the V component R of the reflected image V (x,y) is adjusted;

[0094] Step 3.4: The obtained R V(x, y) is combined with the H and S components of the observation image I(x, y) to obtain the pixel value of the reflected image in the HSV color space, and then it is converted from the HSV color space back to the RGB color space;

[0095] Step 3.5: Expand the image pixel value range to [0, 255] to obtain the enhanced image R(x, y). Use the Peak Signal to Noise Ratio (PSNR) and structural similarity index (SSIM) to evaluate the quality of the enhanced image.

[0096] In order to obtain more accurate comparison results, an experiment was conducted in this embodiment.

[0097] In this experiment, a low-light image (Figure 3(a)) is used as the target image. An unsupervised single image enhancement method based on the Retinex fractional-order variational network is used to enhance the image. The results are compared with those of other image enhancement algorithms to evaluate the performance of the algorithm.

[0098] Other image enhancement algorithms include histogram equalization (HE), brightness preservation-based histogram equalization (BPDHE), fusion-based low-light image enhancement method MF (Multi-deviation Fusion), and image enhancement probabilistic method PM-SIRE that simultaneously estimates illumination map and reflectance.

[0099] Figure 3(b)-(f) shows the visual comparison of the processing results of Figure 3(a) by different image enhancement algorithms.

[0100] Figure 3(b)-(e) are the results of other methods, and Figure 3(f) is the result after processing by the method proposed in the present invention.

[0101] Table 1 shows the comparison of evaluation index results of different image enhancement algorithms.

[0102] Table 1: Quantitative evaluation of different image enhancement algorithms on Figure 3(a)

[0103]

[0104] As can be seen from Figure 3, the image color distortion after enhancement by the HE and BPDHE methods is more serious, and the contrast is lower than that of other algorithms; the texture, text and edges of the image processed by the MF and PM-SIRE methods are not prominent; the method proposed in the present invention not only enhances the contrast effect of the low-light image as a whole, but also makes the edge, texture and text features obvious.

[0105] From the two image quality evaluation indicators of peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) in Table 1, the image index results after being processed by the method proposed in the present invention are the best, indicating that the enhancement effect of the present invention is better and the image distortion is smaller.

[0106] Example 2

[0107] This embodiment discloses an unsupervised single image enhancement system based on the Retinex fractional-order variational network.

[0108] An unsupervised single image enhancement system based on the Retinex fractional-order variational network, including:

[0109] The image preprocessing module is configured to: obtain a single observation image and preprocess the observation image;

[0110] The Retinex fractional-order variational network building module is configured to: build a BP neural network, establish the objective function of the BP neural network based on the Retinex theory and the fractional-order variational method, and obtain the Retinex fractional-order variational network;

[0111] The training module is configured to: train the Retinex fractional-order variational network using a preprocessed single observation image to obtain a trained Retinex fractional-order variational network;

[0112] The illumination image acquisition module is configured to: input the preprocessed image into the trained Retinex fractional-order variational network to obtain an estimated illumination image;

[0113] The enhanced image acquisition module is configured to: calculate the reflected image based on the illumination image, and correct the reflected image to obtain a single enhanced image.

[0114] Example 3

[0115] The purpose of this embodiment is to provide a computer-readable storage medium.

[0116] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the unsupervised single image enhancement method based on the Retinex fractional-order variational network as described in Example 1 of the present disclosure.

[0117] Example 4

[0118] The purpose of this embodiment is to provide an electronic device.

[0119] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps of the unsupervised single image enhancement method based on the Retinex fractional-order variational network as described in Example 1 of the present disclosure are implemented.

[0120] The steps involved in the apparatuses of Examples 2, 3, and 4 above correspond to those of Method Example 1. For detailed implementations, please refer to the relevant description of Example 1. The term "computer-readable storage medium" should be understood to mean a single medium or multiple media containing one or more instruction sets; it should also be understood to include any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and causing the processor to perform any method of the present invention.

[0121] Those skilled in the art will appreciate that the modules or steps of the present invention described above can be implemented using a general-purpose computer device. Alternatively, they can be implemented using program code executable by a computing device, which can then be stored in a storage device and executed by the computing device. Alternatively, they can be fabricated into separate integrated circuit modules, or multiple modules or steps can be fabricated into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0122] Although the above describes the specific embodiments of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without any creative work are still within the scope of protection of the present invention.

Claims

1. An unsupervised single image enhancement method based on Retinex fractional-order variational network, characterized by: The following steps are involved: Acquire a single observation image and preprocess the observation image; Build a BP neural network, establish the objective function of the BP neural network based on Retinex theory and fractional-order variation method, and obtain the Retinex fractional-order variation network; The objective function of the BP neural network is expressed as: Where Ω∈R 2 is the observed image area, i is the mapping of the V component of the observed image on the logarithmic domain, v1 and v2 are both positive real numbers, represents the v1-order differential operator, α1, α2, and α3 are all constant coefficients, and l is the output of the Retinex fractional-order variational network; The Retinex fractional-order variational network is trained using a single preprocessed observation image to obtain a trained Retinex fractional-order variational network. The preprocessed image is input into the trained Retinex fractional-order variational network to obtain the estimated illumination image; Based on the illumination image, the reflection image is calculated and corrected to obtain a single enhanced image; The correction is as follows: According to the Retinex theoretical formula, the mapping r(x,y) of the reflected image V component in the logarithmic domain is derived, and r(x,y) is converted into R through exponential transformation. V (x, y), Gamma correction and CLAHE algorithm are used to correct the V component R of the reflected image V (x,y) is adjusted.

2. The unsupervised single image enhancement method based on Retinex fractional-order variational network according to claim 1, characterized in that: Preprocess the observed image, specifically: The observed image is converted from the RGB color space to the HSV color space, the V component of the observed image is normalized to the range of [0, 1], and the normalized V component is mapped to the logarithmic domain to obtain the brightness image i in the logarithmic domain.

3. The unsupervised single image enhancement method based on Retinex fractional-order variational network according to claim 1, characterized in that: Build a BP neural network structure, specifically: based on the BP neural network, build a neural network structure including an input layer, a hidden layer and an output layer.

4. The unsupervised single image enhancement method based on Retinex fractional-order variational network according to claim 1, characterized in that: The position X = (x, y) of each pixel in the observed image is used as the input of the Retinex fractional-order variation network, and the pixel value l(x, y) at the corresponding position (x, y) of the illumination image to be estimated is used as the output. The relationship between the input and output of the Retinex fractional-order variation network is expressed as: l=f(Uf(WX)) Where W and U represent the weights of the hidden layer and the output layer respectively, f is the Sigmoid function, and l is the output of the Retinex fractional-order variational network.

5. The unsupervised single image enhancement method based on Retinex fractional-order variational network according to claim 4, characterized in that: The Retinex fractional-order variational network is trained as follows: The output of the Retinex fractional-order variational network is substituted into the objective function, the derivative of the objective function with respect to the hidden layer weights and the output layer weights is calculated, and the weights are updated using the gradient descent method until the set number of iterations is reached.

6. An unsupervised single image enhancement system based on Retinex fractional-order variational networks, characterized by: include: The image preprocessing module is configured to: obtain a single observation image and preprocess the observation image; The Retinex fractional-order variational network building module is configured to: build a BP neural network, establish the objective function of the BP neural network based on the Retinex theory and the fractional-order variational method, and obtain the Retinex fractional-order variational network; The objective function of the BP neural network is expressed as: Where Ω∈R 2 is the observed image area, i is the mapping of the V component of the observed image on the logarithmic domain, v1 and v2 are both positive real numbers, represents the v1-order differential operator, α1, α2, and α3 are all constant coefficients, and l is the output of the Retinex fractional-order variational network; The training module is configured to: train the Retinex fractional-order variational network using a preprocessed single observation image to obtain a trained Retinex fractional-order variational network; The illumination image acquisition module is configured to: input the preprocessed image into the trained Retinex fractional-order variational network to obtain an estimated illumination image; The enhanced image acquisition module is configured to: calculate a reflected image based on the illumination image, and correct the reflected image to obtain a single enhanced image; The correction is as follows: According to the Retinex theoretical formula, the mapping r(x,y) of the reflected image V component in the logarithmic domain is derived, and r(x,y) is converted into R through exponential transformation. V (x, y), Gamma correction and CLAHE algorithm are used to correct the V component R of the reflected image V (x,y) is adjusted.

7. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the steps of the unsupervised single image enhancement method based on the Retinex fractional-order variational network are implemented.

8. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the unsupervised single image enhancement method based on the Retinex fractional-order variational network are implemented as described in any one of claims 1 to 5.

Images