A low-light image enhancement method based on the principle of camera imaging

Abstract

The present invention belongs to the technical field of image processing, and particularly relates to a low-light image enhancement method based on the camera imaging principle, which is characterized in that it includes the following steps: Step 1, convert the low-light image to be enhanced into the HSV color space and extract the V component image for the first light estimation to obtain the V component image 1; Step 2, invert the low-light image to be enhanced and then convert it into the HSV color space to extract the V component image for the second light estimation to obtain the V component image 2; Step 3, use the camera response principle for the V component image 1 in Step 1 to obtain a series of virtual high-exposure images and then select an optimal high-exposure image from them; Step 4, use the camera response principle for the V component image 2 in Step 2 to obtain a series of virtual low-exposure images and then select an optimal low-exposure image from them. The present invention ensures that image details will not be lost while enhancing the image, and obtains an enhanced result that is more in line with human vision.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and specifically to a low-light image enhancement method based on the camera imaging principle. Background Art

[0002] In real life, with the increasing use of more and more electronic devices, low-light environments are relatively common shooting environments. Especially for images taken under lighting conditions such as rainy days, foggy days, and nights, the overall quality seriously deteriorates; low-light images generally exhibit characteristics such as low image visibility, color degradation, and greatly reduced visual effects, often making it difficult to identify important information in the images.

[0003] Due to various reasons, how to enhance the image and identify important information in low-light images without losing image details has become an urgent problem to be solved. Summary of the Invention

[0004] (1) Technical Problem to be Solved

[0005] Aiming at the deficiencies of the prior art, the present invention provides a low-light image enhancement method based on the camera imaging principle, and solves the problems raised in the above background art.

[0006] (2) Technical Solution

[0007] The present invention specifically adopts the following technical solutions to achieve the above objectives:

[0008] A low-light image enhancement method based on the camera imaging principle includes the following steps,

[0009] Step 1: Convert the low-light image to be enhanced into the HSV color space and extract the V component image for the first light estimation to obtain the V component image 1;

[0010] Step 2: Invert the low-light image to be enhanced and then convert it into the HSV color space to extract the V component image for the second light estimation to obtain the V component image 2;

[0011] Step 3: Use the camera response principle for the V component image 1 in Step 1 to obtain a series of virtual high-exposure images and then select the best high-exposure image from them;

[0012] Step 4: Use the camera response principle for the V component image 2 in Step 2 to obtain a series of virtual low-exposure images and then select the best low-exposure image from them;

[0013] Step 5: Enhance the best high-exposure image selected in Step 3 to obtain the enhanced high-exposure image;

[0014] Step 6: Enhance the best low-exposure image selected in Step 4 to obtain an enhanced low-exposure image;

[0015] Step 7: Combine the enhanced high-exposure image obtained in Step 5 with the H and S components in the HSV color space in Step 1 to obtain Enhanced Image 1;

[0016] Step 8: Combine the enhanced low-exposure image obtained in Step 6 with the H and S components in the HSV color space in Step 2 to obtain Enhanced Image 2;

[0017] Step 9: Fuse Enhanced Image 1 and Enhanced Image 2 obtained in Step 7 and Step 8 to obtain the final fused image.

[0018] Further, Step 3 is specifically as follows

[0019] 1. Utilize the proportional relationship of the aperture in the camera response principle to obtain a series of high-exposure images of the V-component image 1 through the geometric ratio relationship;

[0020] 2. Determine the best high-exposure image through brightness comparison and information entropy.

[0021] Further, Step 4 is specifically as follows:

[0022] 1. Repeat Step 3(1) to obtain a series of low-exposure images of the V-component image 2;

[0023] 2. Repeat Step 3(2) to determine the best low-exposure image.

[0024] Further, Step 5 is specifically as follows:

[0025] Enhance the high-exposure image H selected in Step 3 I to obtain an enhanced high-exposure image H0:

[0026] 1. First, for the best high-exposure image H selected in Step 3 I adopt the Contrast Limited Adaptive Histogram Equalization (CLAHE) method to enhance the V-component of the image to improve the local contrast of the image and restore more detailed information. Divide the input image into non-overlapping sub-blocks, as shown in formula (1):

[0027]

[0028] In formula (1), nx and ny represent the number of pixels of the pixel block in the x and y directions respectively, K represents the gray level, and c represents the clipping coefficient;

[0029] 2. Crop the histogram and reallocate the positions of the pixels in the image. Crop each pixel block h(x) with T in formula (1), and then the cropped pixels are evenly distributed to each gray level in the image. Assume the total number of pixels exceeding the amplitude T is S, and evenly distribute it to each gray level. Let the number of distributed pixels be A; as shown in formulas (2) and (3),

[0030]

[0031]

[0032] Define the reallocated histogram as h′(x):

[0033]

[0034] Finally, use the bilinear interpolation method to combine adjacent blocks. The obtained high-exposure enhanced image H0 is then merged with the H and S components in the HSV color space in step 1 to obtain the enhanced image 1.

[0035] Further, step 6 is specifically as follows,

[0036] Perform low-exposure enhancement on the best low-exposure image LI selected in step 4 through non-linear enhancement to obtain the enhanced low-exposure image L0;

[0037] The obtained enhanced low-exposure image L0 is then merged with the H and S components in the HSV color space in step 2 to obtain the enhanced image 2.

[0038] Further, step 9 is specifically as follows:

[0039] 1. Decompose the two images to be processed, the enhanced image 1 obtained in step 7 and the enhanced image 2 obtained in step 8, into N image blocks of the same size in sequence through formula (5), and decompose each image block into three components: signal intensity, signal structure, and average intensity:

[0040]

[0041] x in formula (5) n (n = 1, 2, 3) represents the image block; represents the column vector composed of the mean values of all pixels in the image block; represents the image block after subtracting the pixel mean; C n represents the image block x n 's signal intensity; S n represents the image block x n 's signal structure; l n represents the image block x n 's average intensity component ||·|| 2 represents the l2 norm;

[0042] 2. Select the highest signal intensity from two image blocks to be processed using the maximum value of information entropy to determine the specific operation of the fused image block as shown in formula (6):

[0043]

[0044] In formula (6) represents the signal intensity of the fused image block;

[0045] 3. Use formula (7) to preserve the structural integrity of the fused image, so that the fused image block contains all the structures of the three image blocks:

[0046]

[0047] In formula (7) represents the signal structure of the fused image block; s1 is obtained by formula (8):

[0048]

[0049] In formula (8), S(·) represents the weighting function, which is used to determine the contribution of each image block when fusing the signal structure. The stronger the signal intensity of the image block, the greater its contribution. Therefore, the weight value of the contribution signal structure of the image block is determined by formula (9):

[0050]

[0051] In formula (9), p represents the exponential parameter. The larger p is, the stronger the signal intensity and the greater its contribution;

[0052] 4. Adopt the processing method of signal structure to realize the processing of the average intensity of the image block:

[0053]

[0054] In formula (10) represents the average intensity of the fused image block; μ n represents the pixel mean of image block x n ; G(·) represents the weighting function;

[0055] The weighting function G(·):

[0056]

[0057] In formula (11), use the pixel mean μ n of the image to be processed and the average intensity l n of the current image block as inputs, which can be used to quantify image block x nExposure level; where μ c represents the pixel median value of the image to be processed; l c represents the pixel median value of the image block; σ1, σ2 represent the expansion of the control function profile along μ n and l n width;

[0058] 5. The fused image block can be obtained through formula (12):

[0059]

[0060] After obtaining the fused image block, all the image blocks are aggregated together to complete all the fusion processes and obtain the final enhanced image.

[0061] (III) Advantageous Effects

[0062] Compared with the prior art, the present invention provides a low-light image enhancement method based on the camera imaging principle, having the following advantageous effects:

[0063] In the present invention, while enhancing the image, it is ensured that the image details will not be lost, and a more visually appealing enhancement result is obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 is the low-light image enhancement method of the present invention.

[0065] Figure 2 is the result diagram of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0066] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0067] Embodiment

[0068] As Figure 1-2 shown, a low-light image enhancement method based on the camera imaging principle proposed in an embodiment of the present invention includes the following steps:

[0069] Step 1. Convert the low-light image to be enhanced to the HSV color space and extract the V-component image for the first light estimation to obtain the V-component image 1;

[0070] Step 2. Invert the low-light image to be enhanced and then convert it to the HSV color space to extract the V-component image for the second light estimation to obtain the V-component image 2;

[0071] Step 3: Based on the camera response principle, a series of virtual high-exposure images are obtained from the V-component image 1 in Step 1, and then an optimal high-exposure image is selected from them;

[0072] Step 4: Based on the camera response principle, a series of virtual low-exposure images are obtained from the V-component image 2 in Step 2, and then an optimal low-exposure image is selected from them;

[0073] In this step, the optimal low-exposure image L is selected I ;

[0074] Step 5: Enhance the optimal high-exposure image selected in Step 3 to obtain an enhanced high-exposure image;

[0075] Step 6: Enhance the optimal low-exposure image selected in Step 4 to obtain an enhanced low-exposure image;

[0076] Step 7: Combine the enhanced high-exposure image obtained in Step 5 with the H and S components in the HSV color space in Step 1 to obtain an enhanced image 1;

[0077] Step 8: Combine the enhanced low-exposure image obtained in Step 6 with the H and S components in the HSV color space in Step 2 to obtain an enhanced image 2;

[0078] Step 9: Fuse the enhanced image 1 and the enhanced image 2 obtained in Step 7 and Step 8 to obtain the final fused image.

[0079] As Figure 1 shown, in some embodiments, Step 3 is specifically as follows

[0080] 1. Based on the geometric ratio relationship of the aperture in the camera response principle, a series of high-exposure images of the V-component image 1 are obtained; by finding out a series of high-exposure images of the V-component image 1;

[0081] 2. Determine the optimal high-exposure image through brightness comparison and information entropy.

[0082] As Figure 1 shown, in some embodiments, Step 4 is specifically as follows:

[0083] 1. Repeat Step 3(1) to find out a series of low-exposure images of the V-component image 2;

[0084] 2. Repeat Step 3(2) to determine the optimal low-exposure image.

[0085] As Figure 1 shown, in some embodiments, Step 5 is specifically as follows:

[0086] Enhance the high-exposure image H selected in Step 3 I to obtain an enhanced high-exposure image H0:

[0087] 1. First, for the best high-exposure image H selected in step 3 I Use the Contrast Limited Adaptive Histogram Equalization (CLAHE) method to enhance the V component of the image to improve the local contrast of the image, so as to restore more detailed information. The input image is divided into non-overlapping sub-blocks, as shown in formula (1):

[0088]

[0089] In formula (1), nx and ny represent the number of pixels of the pixel block in the x and y directions respectively, K represents the gray level, and c represents the clipping coefficient;

[0090] 2. Clip the histogram and re-distribute the positions of the pixel points in the image. Each pixel block h(x) is clipped by T in formula (1), and then the clipped pixels are evenly distributed to each gray level in the image. Assume that the total number of pixels exceeding the amplitude T is S, and it is evenly distributed to each gray level. Let the number of distributed pixels be A; as shown in formulas (2) and (3),

[0091]

[0092]

[0093] Define the re-distributed histogram as h′(x):

[0094]

[0095] Finally, use the bilinear interpolation method to combine adjacent blocks. The obtained high-exposure enhanced image H0 is then merged with the H and S components in the HSV color space in step 1 to obtain the enhanced image 1.

[0096] As Figure 1 shown, in some embodiments, step 6 is specifically:

[0097] Perform low-exposure enhancement on the best low-exposure image LI selected in step 4 through non-linear enhancement to obtain the enhanced low-exposure image L0;

[0098] The obtained enhanced low-exposure image L0 is then merged with the H and S components in the HSV color space in step 2 to obtain the enhanced image 2.

[0099] As Figure 1 shown, in some embodiments, step 9 is specifically:

[0100] 1. Decompose the enhanced image 1 obtained in step 7 and the enhanced image 2 obtained in step 8 into N image patches of the same size in sequence through formula (5), and decompose each image patch into three components: signal intensity, signal structure, and average intensity:

[0101]

[0102] x in formula (5) n (n = 1, 2, 3) represents an image patch; represents the column vector composed of the mean values of all pixels of the image patch; represents the image patch after subtracting the pixel mean; C n represents the signal intensity of the image patch x n ; S n represents the signal structure of the image patch x n ; l n represents the image patch x n 's average intensity component ||·||2 represents the l2 norm;

[0103] 2. Select the highest signal intensity among the two image patches to be processed to determine the fused image patch by using the maximum value of information entropy. The specific operation is as formula (6):

[0104]

[0105] In formula (6) represents the signal intensity of the fused image patch;

[0106] 3. Use formula (7) to preserve the structural integrity of the fused image, so that the fused image patch contains all the structures of the three image patches:

[0107]

[0108] In formula (7) represents the signal structure of the fused image patch; s1 is obtained by formula (8):

[0109]

[0110] In formula (8), S(·) represents a weighting function, which is used to determine the contribution of each image patch when fusing the signal structure. The stronger the signal intensity of the image patch, the greater its contribution; therefore, the weight value of the contribution signal structure of the image patch is determined by formula (9):

[0111]

[0112] In formula (9), p represents an exponential parameter. The larger p is, the stronger the signal intensity and the greater its contribution;

[0113] 4. Implement the processing of the average intensity of image blocks by using a signal structure processing method:

[0114]

[0115] In formula (10) represents the average intensity of the fused image block; μ n represents the pixel mean of image block x n ; G(·) represents the weighting function;

[0116] The weighting function G(·):

[0117]

[0118] In formula (11), the pixel mean μ n of the image to be processed and the average intensity l n of the current image block are used as inputs, which can be used to quantify the exposure degree of image block x n ; where μ c represents the pixel median of the image to be processed; l c represents the pixel median of the image block; σ1, σ2 represent the expansion of the control function profile along the widths of μ n and l n ;

[0119] 5. The fused image block can be obtained through formula (12):

[0120]

[0121] After obtaining the fused image block, all the image blocks are aggregated together to complete all the fusion processes and obtain the final enhanced image.

[0122] Finally, it should be noted that the above are only the preferred embodiments of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions recorded in the foregoing embodiments or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A low-light image enhancement method based on the camera imaging principle, characterized in that: It includes the following steps: Step 1: Convert the low-light image to be enhanced to the HSV color space, extract the V-component image, and perform the first light estimation to obtain the V-component image 1. Step 2: Invert the low-light image to be enhanced, then convert it to the HSV color space, extract the V-component image, and perform the second light estimation to obtain the V-component image 2. Step 3: Based on the camera response principle, obtain a series of virtual high-exposure images from the V-component image 1 in Step 1, and then select an optimal high-exposure image from them. Step 4: Based on the camera response principle, obtain a series of virtual low-exposure images from the V-component image 2 in Step 2, and then select an optimal low-exposure image from them. Step 5: Enhance the optimal high-exposure image selected in Step 3 to obtain the enhanced high-exposure image. Step 5 specifically is: Enhance the high-exposure image H selected in step 3 I to obtain an enhanced high-exposure image H0:

1. First, for the best high-exposure image H selected in step 3 I Use the Contrast Limited Adaptive Histogram Equalization (CLAHE) method to enhance the V component of the image to improve the local contrast of the image, so as to restore more detailed information. Divide the input image into non-overlapping sub-blocks, as shown in formula (1): In formula (1), n x and n y represent the number of pixels of the pixel block in the x and y directions respectively, K represents the gray level, and c represents the clipping coefficient; 2. Crop the histogram, reassign the positions of the pixels in the image. Crop each pixel block h(x) by T in formula (1), and then the cropped pixels are evenly distributed to each gray level in the image. Assume the total number of pixels exceeding the amplitude T is S, and evenly distribute it to each gray level. Let the number of distributed pixels be A. As shown in formulas (2) and (3), Define the reallocated histogram as h′(x): Finally, use the bilinear interpolation method to combine adjacent blocks to obtain the high-exposure enhanced image H0, and then merge it with the H and S components in the HSV color space in Step 1 to obtain the enhanced image 1. Step 6: Enhance the optimal low-exposure image selected in Step 4 to obtain the enhanced low-exposure image. Step 7: Merge the enhanced high-exposure image obtained in Step 5 with the H and S components in the HSV color space in Step 1 to obtain the enhanced image 1. Step 8: Merge the enhanced low-exposure image obtained in Step 6 with the H and S components in the HSV color space in Step 2 to obtain the enhanced image 2. Step 9: Fuse the enhanced image 1 and the enhanced image 2 obtained in Step 7 and Step 8 to obtain the final fused image.

2. The low-light image enhancement method based on the camera imaging principle according to claim 1, wherein: Step 9 specifically is:

1. Decompose the enhanced image 1 obtained in Step 7 and the enhanced image 2 obtained in Step 8 into N image blocks of the same size in turn through formula (5), and decompose each image block into three components: signal intensity, signal structure, and average intensity. x in formula (5) n (n = 1, 2, 3) represents an image block; represents the column vector composed of the mean values of all pixels of the image block; represents the image block after subtracting the pixel mean; C n represents the image block x n 's signal strength; S n represents the image block x n 's signal structure; l n represents the average intensity component of the image block x n ; ||·||2 represents the l2 norm; 2. Use the maximum information entropy to select the highest signal intensity in the two image blocks to be processed to determine the signal intensity of the fused image block. The specific operation is as formula (6): In formula (6) represents the signal intensity of the fused image block; 3. Use formula (7) to retain the structural integrity of the fused image, so that the fused image block contains all the structures of the three image blocks. In formula (7) represents the signal structure of the fused image block; s1 is obtained from formula (8): In formula (8), S(·) represents the weighting function, which is used to determine the contribution of each image block when fusing the signal structure. The stronger the signal intensity of the image block, the greater its contribution. Therefore, determine the weight value of the contribution signal structure of the image block by formula (9): In formula (9), p represents the exponential parameter. The larger p is, the stronger the signal intensity and the greater its contribution.

4. Implement the processing of the average intensity of the image block in the same way as the processing of the signal structure. In formula (10) represents the average intensity of the fused image block; μ n represents the pixel mean of image block x n ; G(·) represents the weighting function Weighting function G(·): In formula (11), the pixel mean μ of the image to be processed is used n and the average intensity l of the current image block n as inputs, which can be used to quantify the exposure degree of the image block x n ; where μ c represents the pixel median of the image to be processed; l c represents the pixel median of the image block; σ1 and σ2 represent the expansion of the control function profile along the widths of μ n and l n ; 5. The fused image block can be obtained through formula (12): After obtaining the fused image patches, all the image patches are aggregated together to complete all the fusion processes and obtain the final enhanced image.

3. A low-light image enhancement method based on the camera imaging principle according to claim 1, wherein: Step 3 is specifically as follows.

1. Use the proportional relationship of the aperture based on the camera response principle to obtain a series of high-exposure images of the V-component image 1; 2. Determine the optimal high-exposure image through brightness contrast and information entropy.

4. A low-light image enhancement method based on the principle of camera imaging according to claim 1, characterized in that: Step 4 is specifically as follows:

1. Repeat step 3(1) to obtain a series of low-exposure images of the V-component image 2.

2. Repeat step 3(2) to determine the optimal low-exposure image.

5. A low-light image enhancement method based on the principle of camera imaging according to claim 1, characterized in that: Step 6 is specifically as follows. Enhance the best low-exposure image L selected in step 4 through non-linear enhancement I Perform low-exposure enhancement to obtain an enhanced low-exposure image L0; The obtained enhanced low-exposure image L0 is merged with the H and S components in the HSV color space in step 2 to obtain the enhanced image 2.

Images