Non-uniform illumination image enhancement methods, systems, storage media and devices

Abstract

This invention relates to a method, system, storage medium, and device for enhancing images under non-uniform illumination, comprising the following steps: acquiring and preprocessing an original image; normalizing the luminance component V to a pixel within a set range, representing it as X(i,j), as the original variable; using a differential neural network, with the luminance image Y(i,j) as the target variable, and the difference between X(i,j) and the enhanced image Xen(i,j) and the target variable, using the entropy maximization principle, Bézier curve enhancement, and total variational denoising algorithm as loss functions of different dimensions of the differential neural network; the model formed after training the differential neural network combines the hue component H and saturation component S corresponding to Y(i,j) and X(i,j) to obtain the pixel values ​​of the image in the HSV color space, and converting them back to the RGB color space; the image pixel values ​​are then mapped and amplified to obtain the enhanced image. Different dimensional loss functions are used as optimization targets to control the degree of enhancement of image detail information, thereby preserving image detail information.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to methods, systems, storage media, and devices for enhancing images under non-uniform illumination. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Due to limitations in the environment and camera equipment, images captured under uneven lighting conditions are prone to being too dark, too bright, overexposed, or underexposed. These images are called non-uniformly lit images. On the one hand, they affect the aesthetic appeal of the images, and on the other hand, they pose difficulties for subsequent target recognition, target tracking, foreground and background segmentation, final image understanding, and predictive analysis.

[0004] To address the aforementioned issues, traditional image processing curves, such as the gamma correction curve (power-law curve), offer excellent contrast adjustment capabilities. However, a drawback is that once the exponent is determined, both dark pixels (x) and bright pixels (x') will be enhanced or weakened, making it difficult to enhance the overall or local features of the image according to actual needs. Consequently, it fails to reflect the differences between features of different objects in the image. While some deep learning methods can achieve image enhancement, they often encounter numerous problems during image processing, such as noise sensitivity, loss of detail, over-enhancement of certain result areas, or relatively low contrast enhancement. Furthermore, deep learning relies on large labeled datasets to train the network, but the number of relevant image datasets is limited. The images in these datasets are often synthetic images, which differ from the actual generated images, leading to unsatisfactory image enhancement results. Summary of the Invention

[0005] To address the technical problems mentioned above, this invention provides a method, system, storage medium, and device for image enhancement under non-uniform illumination. It utilizes a differential neural network model and objective functions including entropy maximization loss, Bézier curve enhancement loss, and total variational denoising loss to control the degree of enhancement of image detail information with optimization objectives of different dimensions, achieving the effect of preserving image detail information. The mapping relationship between input and output in the image is non-linearly represented using a neural network. A differential neural network structure model and loss functions of different dimensions are used as objectives to constrain the training of the neural network. The entire algorithm achieves true single-image enhancement (single image input corresponds to single image output), eliminating the need to collect a large number of target images or create corresponding label images. It is an unsupervised image enhancement algorithm, saving significant human and material resources.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] The first aspect of the present invention provides a method for enhancing images under non-uniform illumination, comprising the following steps:

[0008] Step 1: Acquire the original image and preprocess it. Normalize the brightness component V to the pixels within a set range and represent them as X(i,j), which will be used as the original variable.

[0009] Step 2: Using a differential neural network, with the brightness image Y(i,j) as the target variable, and the difference between X(i,j) and the enhanced image Xen(i,j) and the target variable, based on the entropy maximization principle, Bézier curve enhancement, and total variation denoising algorithm, as the loss functions of different dimensions of the differential neural network;

[0010] Step 3: The model formed after training the differential neural network combines the hue component H and saturation component S corresponding to Y(i,j) and X(i,j) to obtain the pixel value of the image in the HSV color space, and converts it back to the RGB color space. The image pixel value is then mapped and amplified to obtain the enhanced image.

[0011] The preprocessing steps in step one are as follows:

[0012] Step 1.1: Obtain the image to be processed, Xinput, and convert Xinput from the RGB color space to the HSV color space;

[0013] Step 1.2: Separate the image into three channels in the HSV color space: Hue (H), Saturation (S), and Lightness (V);

[0014] Step 1.3: Normalize the brightness component V of the image to a set range, denoted as X(i,j).

[0015] The differential neural network in step two is based on the BP neural network. The input layer, hidden layer and output layer form a two-way subtraction neuron structure, and the activation functions are logarithmic function and exponential function.

[0016] In step two, the image entropy loss function is constructed based on the entropy maximization principle framework:

[0017]

[0018] Where H and W represent the height and width of the image, respectively, Xen represents the image enhanced using the entropy maximization principle, and Y = X d1 The value of d1 is: M is a set value, u m These are the statistical values ​​representing the probability distribution of pixel intensity.

[0019] In step two, the constructed Bézier curve loss function is:

[0020]

[0021] Where H and W represent the height and width of the image, respectively, and the intensity values ​​of the dark and bright regions are used as control functions to generate X after image enhancement. Bz The difference between the generated image Y and the image Y is used as the variable in the loss function.

[0022] In step two, the loss function constructed based on the total variation denoising algorithm is as follows:

[0023]

[0024] Where H and W represent the height and width of the image, respectively.

[0025] In step two, the training process of the differential neural network is as follows:

[0026] Forward propagation process: Input image X(i,j), after calculation by neurons in each layer of the neural network, the network output is obtained, i.e., the image output value:

[0027] Backpropagation process: Obtain the derivative of the objective function with respect to the connection weights of each layer, update the weights using gradient descent, and iteratively train the network using the weights until the maximum number of iterations is reached, then stop training and output the results. All output results constitute the estimated brightness image.

[0028] A second aspect of the present invention provides a system for implementing the above-described method, comprising:

[0029] The preprocessing module is configured to: acquire the original image and preprocess it, normalize the brightness component V to the pixels within a set range and represent them as X(i,j), as the original variable;

[0030] The image enhancement module is configured to: use a differential neural network, with the brightness image Y(i,j) as the target variable, and the difference between X(i,j) and the enhanced image Xen(i,j) and the target variable, and use entropy maximization, Bézier curve enhancement and total variation denoising algorithm as loss functions of different dimensions of the differential neural network;

[0031] The output module is configured as follows: the model formed after training the differential neural network combines the hue component H and saturation component S corresponding to Y(i,j) and X(i,j) to obtain the pixel value of the image in the HSV color space, and converts it back to the RGB color space. The image pixel value is then mapped and amplified to obtain the enhanced image.

[0032] A third aspect of the present invention provides a computer-readable storage medium.

[0033] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the non-uniform illumination image enhancement method described above.

[0034] A fourth aspect of the present invention provides a computer device.

[0035] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the non-uniform illumination image enhancement method described above.

[0036] Compared with existing technologies, one or more of the above technical solutions have the following beneficial effects:

[0037] 1. By utilizing a differential neural network model and objective functions including entropy maximization loss function, Bézier curve enhancement loss function, and total variation denoising loss function, the degree of enhancement of image detail information is controlled by optimization objectives of different dimensions, so as to achieve the effect of preserving image detail information; the mapping relationship between input and output in the image is represented non-linearly by a neural network, and the training of the neural network is constrained by a model with a differential neural network structure and loss functions of different dimensions.

[0038] 2. The entire algorithm achieves true single-image enhancement (single image input corresponds to single image output), without the need to collect a large number of target images or create corresponding label images for the target images. It is an unsupervised image enhancement algorithm that saves a lot of human and material resources. Attached Figure Description

[0039] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0040] Figure 1 This is a flowchart of a non-uniform illumination image enhancement method provided in one or more embodiments of the present invention;

[0041] Figure 2 This is a schematic diagram of the network architecture in the image enhancement process provided by one or more embodiments of the present invention;

[0042] Figures 3-6 These are all image enhancement effect diagrams provided by one or more embodiments of the present invention. Detailed Implementation

[0043] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0044] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0045] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0046] As described in the background section, non-uniformly illuminated images not only affect their aesthetic appeal but also pose challenges for subsequent tasks such as target recognition, target tracking, foreground / background segmentation, final image understanding, and predictive analysis. Existing technologies have only achieved unsatisfactory enhancements and fail to meet practical needs.

[0047] Therefore, the following embodiments provide a method, system, storage medium, and device for image enhancement under non-uniform illumination. A differential neural network model is constructed, along with objective functions including entropy maximization loss, Bézier curve enhancement loss, and total variational denoising loss. Different optimization objectives are used to control the degree of enhancement of image detail information, achieving the effect of preserving image detail. The mapping relationship between input and output in the image is non-linearly represented using a neural network. A differential neural network model and loss functions of different dimensions are used as objectives to constrain the training of the neural network. The entire algorithm achieves true single-image enhancement (single image input corresponds to single image output), without the need to collect a large number of target images or create corresponding label images. It is an unsupervised image enhancement algorithm, saving significant human and material resources.

[0048] Example 1:

[0049] like Figure 1-6 As shown, the non-uniform illumination image enhancement method includes the following steps:

[0050] Step 1: Image preprocessing. Read in the image to be processed, Xinput, and convert Xinput from the RGB color space to the HSV color space. Take the lightness V component and normalize it to the pixel at position (i,j) in the range [0,1], which is represented as X(i,j).

[0051] Step 2: Image processing is performed using a differential neural network. The brightness image Y(i,j) is used as the target variable, and the differences between X(i,j) and the enhanced image Xen(i,j) and the target variable are used as enhancement cost functions of different dimensions; these include the entropy maximization loss function, the Bézier curve enhancement loss function, and the total variation denoising loss function.

[0052] Step 3: After training, the model combines the H and S components corresponding to Y(i,j) and X(i,j) to obtain the pixel value of the image in the HSV color space, and converts it back from the HSV color space to the RGB color space, mapping the range of image pixel values ​​to integers between [0,255] to obtain the enhanced image.

[0053] Specifically:

[0054] Step 1: Perform preprocessing operations on the image to be processed. The specific steps are as follows:

[0055] Step 1.1: Read in the image to be processed, Xinput, and convert Xinput from the RGB color space to the HSV color space;

[0056] Step 1.2: Separate the three channels of the observed image in the HSV color space: Hue (H), Saturation (S), and Value (V);

[0057] Step 1.3: Normalize the V component of the observed image to the range [0,1], and set it as X(i,j).

[0058] Step 2: Image processing is performed using a differential neural network. The specific steps are as follows:

[0059] Step 2.1: Network Structure Construction: Based on the BP neural network, construct a three-layer neural network structure, such as... Figure 2 As shown, the input layer, hidden layer, and output layer form a two-way subtraction neuron structure; the activation functions used are the log(·) logarithmic function and the exp(·) exponential function.

[0060] Step 2.2, construct the objective function of the unsupervised single-image enhancement method: using the brightness image X(i,j) as the original variable and Y(i,j) as the target variable, the objective function of the neural network is based on the framework of entropy maximization, Bézier curve enhancement, and total variation denoising algorithm (unlabeled). The specific objective function is shown below:

[0061] Image entropy loss function (unlabeled) established within the framework of the entropy maximization principle:

[0062]

[0063] Where HW represents the height and width of the image, and Xen represents the image enhanced using the entropy maximization principle, with the formula Y = X. d1 The value of d1 is: M = 256,u m These are the statistical values ​​representing the probability distribution of pixel intensity.

[0064] The constructed Bézier curve loss function is:

[0065]

[0066] The formula for a second-order Bézier curve is as follows: P(t) = (1-t) 2 P0+2(1-t)tP1+t 2 P2

[0067] Using the intensity values ​​of dark and bright areas as control functions, X is generated after image enhancement. Bz The difference between the generated image Y and the image Y is used as the variable in the loss function.

[0068] The formula for the loss function of the Total Variation (TV) denoising algorithm is as follows:

[0069]

[0070] Step 2.3, Initialize the neural network structure: Set the number of hidden layer nodes, the maximum number of training iterations, and the learning rate, and randomly initialize the network weights with a normal distribution; in this embodiment, the maximum number of iterations is set to 100, the learning rate is 0.005, the number of hidden layer nodes is 40, and the network weights W and U are randomly initialized.

[0071] Step 2.4, the specific steps for training the neural network model are as follows:

[0072] Step 2.4.1, Forward Propagation Process. The input image X(i,j) is processed by neurons in each layer of the neural network. The network output, i.e., the image output value, can be represented by the following formula:

[0073] Y(i,j)=C1(X(i,j))-C2(X(i,j))

[0074] The expressions for the two branch networks (n = 1, 2) are as follows:

[0075]

[0076] W and U represent the weights of the hidden layer and the output layer, respectively, and b is the offset.

[0077] K is the number of convolution kernels φ. Activation function φ(x) = T*log(x), where T is a settable parameter.

[0078] Step 2.4.2, Backpropagation Process. Calculate the derivative of the objective function with respect to the connection weights of each layer, and update the weights using gradient descent. The update formula for the weight matrix is:

[0079]

[0080]

[0081] The network is trained iteratively using weights until the maximum number of iterations is reached, at which point training stops, and the results are output. All output results constitute the estimated brightness image.

[0082] Step 3: Calculation of the output image, the specific steps are as follows:

[0083] Step 3.1: Output the image Y(i,j) = C1(X(i,j)) - C2(X(i,j)) according to the network model;

[0084] Step 3.2: Combine Y(i,j) with the H and S components of Xinpu to obtain the pixel value of the reflected image in the HSV color space, and convert it back from the HSV color space to the RGB color space.

[0085] Step 3.3: Map the image pixel value range to integers between [0, 255] to obtain the enhanced image Youtput.

[0086] In this embodiment, Figures 3-6 Sample images for processing.

[0087] The above method utilizes a differential neural network model and objective functions including entropy maximization loss function, Bézier curve enhancement loss function, and total variation denoising loss function to control the degree of enhancement of image detail information with optimization objectives of different dimensions, thereby achieving the effect of preserving image detail information. The mapping relationship between input and output in the image is represented non-linearly by a neural network, and the model with a differential neural network structure and loss functions of different dimensions are used as objectives to constrain the training of the neural network.

[0088] The entire algorithm achieves true single-image enhancement (single image input corresponds to single image output), without the need to collect a large number of target images or create corresponding label images for the target images. It is an unsupervised image enhancement algorithm that saves a lot of human and material resources.

[0089] Example 2:

[0090] A system for implementing the above method includes:

[0091] The preprocessing module is configured to: acquire the original image and preprocess it, normalize the brightness component V to the pixels within a set range and represent them as X(i,j), as the original variable;

[0092] The image enhancement module is configured to: use a differential neural network, with the brightness image Y(i,j) as the target variable, and the difference between X(i,j) and the enhanced image Xen(i,j) and the target variable, and use entropy maximization, Bézier curve enhancement and total variation denoising algorithm as loss functions of different dimensions of the differential neural network;

[0093] The output module is configured as follows: the model formed after training the differential neural network combines the hue component H and saturation component S corresponding to Y(i,j) and X(i,j) to obtain the pixel value of the image in the HSV color space, and converts it back to the RGB color space. The image pixel value is then mapped and amplified to obtain the enhanced image.

[0094] The system utilizes a differential neural network model and objective functions including entropy maximization loss, Bézier curve enhancement loss, and total variational denoising loss to control the degree of image detail enhancement with optimization objectives of different dimensions, achieving the effect of preserving image details. The system non-linearly represents the mapping relationship between input and output in an image using a neural network, employing a differential neural network structure and loss functions of different dimensions as objectives to constrain the training of the neural network. It achieves true single-image enhancement (single image input corresponds to single image output), eliminating the need to collect a large number of target images or create corresponding label images. This unsupervised image enhancement algorithm saves significant human and material resources.

[0095] Example 3:

[0096] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the non-uniform illumination image enhancement method described in Embodiment 1 above.

[0097] This non-uniform illumination image enhancement method system utilizes a differential neural network model and objective functions including entropy maximization loss, Bézier curve enhancement loss, and total variational denoising loss to control the degree of enhancement of image details with optimization objectives of different dimensions, achieving the effect of preserving image details. The system non-linearly represents the mapping relationship between input and output in the image using a neural network, employing a differential neural network structure and loss functions of different dimensions as objectives to constrain the training of the neural network. It achieves true single-image enhancement (single image input corresponds to single image output), eliminating the need to collect a large number of target images or create corresponding label images. This unsupervised image enhancement algorithm saves significant human and material resources.

[0098] Example 4:

[0099] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the non-uniform illumination image enhancement method described in Embodiment 1 above.

[0100] This non-uniform illumination image enhancement method system utilizes a differential neural network model and objective functions including entropy maximization loss, Bézier curve enhancement loss, and total variational denoising loss to control the degree of enhancement of image details with optimization objectives of different dimensions, achieving the effect of preserving image details. The system non-linearly represents the mapping relationship between input and output in the image using a neural network, employing a differential neural network structure and loss functions of different dimensions as objectives to constrain the training of the neural network. It achieves true single-image enhancement (single image input corresponds to single image output), eliminating the need to collect a large number of target images or create corresponding label images. This unsupervised image enhancement algorithm saves significant human and material resources.

[0101] The steps or modules involved in Embodiments 2 to 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0102] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for image enhancement under non-uniform illumination, characterized in that: Includes the following steps: Step 1: Acquire the original image and preprocess it, normalizing the luminance component V to the pixels within a set range. X(i, j)， As the original variable; Step 2: Utilize a differential neural network to obtain the brightness image. Y(i,j) Let X(i,j) be the target variable, and calculate the image X obtained by enhancement. Bz The difference between (i,j) and Y (i,j), and the image X obtained by enhancing X (i,j). en The difference between (i,j) and Y (i,j) is used as the loss function for different dimensions of the differential neural network based on the entropy maximization principle, Bézier curve enhancement, and total variation denoising algorithm. The image entropy loss function is constructed based on the entropy maximization principle framework: Where H and W represent the height and width of the image, respectively. X en This represents the image after X(i,j) has been enhanced using the entropy maximization principle. en =X d1 The value of d1 is: M is the set value, u m These are the statistical values ​​representing the probability distribution of pixel intensity. The constructed Bézier curve loss function is: Where H and W represent the height and width of the image, respectively, and the intensity values ​​of the dark and bright regions of the original variable X(i,j) are used as control functions to generate X after enhancing the image of X(i,j). Bz (i,j), X Bz The difference between (i,j) and the generated image Y(i,j) is used as the variable of the loss function; The loss function constructed based on the total variation denoising algorithm is as follows: Where H and W represent the height and width of the image, respectively; Step 3: The model formed after training the differential neural network will... Y(i,j) and X(i,j) The corresponding hue component H and saturation component S are combined to obtain the pixel value of the image in the HSV color space, and then converted back to the RGB color space. The image pixel value is then mapped and amplified to obtain the enhanced image.

2. The non-uniform illumination image enhancement method as described in claim 1, characterized in that: The preprocessing process in step one is as follows: Step 1.1: Obtain the image to be processed, Xinput, and convert Xinput from the RGB color space to the HSV color space; Step 1.2: Separate the image into three channels in the HSV color space: Hue (H), Saturation (S), and Lightness (V); Step 1.3: Normalize the image's luminance component V to a set range, set to... X ( i , j ).

3. The non-uniform illumination image enhancement method as described in claim 1, characterized in that: The differential neural network in step two is based on the BP neural network. The input layer, hidden layer and output layer form a two-way subtraction neuron structure, and the activation functions are logarithmic function and exponential function.

4. The non-uniform illumination image enhancement method as described in claim 1, characterized in that: In step two, the training process of the differential neural network is as follows: Forward propagation process: Input image X ( i , j The network output, i.e., the image output value, is obtained through calculations by the neurons in each layer of the neural network. Backpropagation process: Obtain the derivative of the objective function with respect to the connection weights of each layer, update the weights using gradient descent, and iteratively train the network using the weights until the maximum number of iterations is reached, then stop training and output the results. All output results constitute the estimated brightness image.

5. A non-uniform illumination image enhancement system, characterized in that: include: The preprocessing module is configured to: acquire the original image and preprocess it, normalizing the luminance component V to the pixel representation within a set range. X(i,j), As the original variable; The image enhancement module is configured to use a differential neural network to enhance the brightness of the image. Y(i,j) Let X(i,j) be the target variable, and calculate the image X obtained by enhancement. Bz The difference between (i,j) and Y (i,j), and the image X obtained by enhancing X (i,j). en The difference between (i,j) and Y (i,j) is used as the loss function for different dimensions of the differential neural network based on entropy maximization, Bézier curve enhancement, and total variation denoising algorithm. The image entropy loss function is constructed based on the entropy maximization principle framework: Where H and W represent the height and width of the image, respectively. X en This represents the image after X(i,j) has been enhanced using the entropy maximization principle. en =X d1 The value of d1 is: M is the set value, u m These are the statistical values ​​representing the probability distribution of pixel intensity. The constructed Bézier curve loss function is: Where H and W represent the height and width of the image, respectively, and the intensity values ​​of the dark and bright regions of the original variable X(i,j) are used as control functions to generate X after enhancing the image of X(i,j). Bz (i,j), X Bz The difference between (i,j) and the generated image Y(i,j) is used as the variable of the loss function; The loss function constructed based on the total variation denoising algorithm is as follows: Where H and W represent the height and width of the image, respectively; The output module is configured to: generate the model after training the differential neural network, Y(i,j) and X(i,j) The corresponding hue component H and saturation component S are combined to obtain the pixel value of the image in the HSV color space, and then converted back to the RGB color space. The image pixel value is then mapped and amplified to obtain the enhanced image.

6. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the non-uniform illumination image enhancement method as described in any one of claims 1-4.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the non-uniform illumination image enhancement method as described in any one of claims 1-4.

Images