A method for improving the clarity of wide dynamic range images

Abstract

The present invention provides a method for improving the clarity of wide dynamic range images. The method includes the following steps: S1. Divide the image into M*N blocks, and statistically calculate the histogram and mean of each block, as well as the histogram and mean of the entire image; S2. Use a histogram equalization algorithm based on logarithmic function mapping, and adaptively adjust the degree of contrast enhancement and brightness improvement through the mean to obtain the brightness mapping function of each block and the global mapping function of the image; S3. Adjust the mapping function of each block according to the mean and the global mapping function, and update the block mapping function; S4. Interpolate to obtain the pixel values after mapping for each point according to the Gaussian weight. This method ensures that while increasing the image brightness, it enhances the image details and contrast, thereby achieving the purpose of improving the image clarity.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to a method for improving the clarity of wide dynamic range images. Background Art

[0002] Currently in the ISP field, wide dynamic range images are mainly generated by fusing and compressing two images with different exposures. On the one hand, due to the limitations of the exposure ratio and exposure time of the two images, the obtained wide dynamic images are darker than normal dynamic range images; on the other hand, image details and contrast are lost during fusion and compression, resulting in blurred images.

[0003] Histogram equalization, as a commonly used method for improving image clarity, has the characteristics of simple calculation and real-time effectiveness, and is widely used. There are also many improved histogram equalization algorithms with their own characteristics. Among them, the histogram equalization algorithm based on logarithmic function mapping uses the logarithmic function as the cumulative distribution function in the histogram equalization algorithm, making the image effect more in line with human perception. However, due to the characteristics of wide dynamic range images, such as larger dynamic range and stronger image layering compared to normal dynamic range images, the traditional histogram equalization algorithm based on logarithmic function mapping cannot achieve good results, and there are problems such as insufficient brightness improvement in dark areas, excessive brightness improvement in bright areas, and insufficient improvement of image details.

[0004] In addition, the following are common terms in the prior art:

[0005] WDR: Wide Dynamic Range.

[0006] ISP: image signal processer, an image processor, a module that performs algorithm processing on the raw images input by the CMOS sensor, including WDR, interpolation, noise reduction, sharpening, etc.

[0007] PDF: probability density function, probability density function.

[0008] CDF: cumulative distribution function, cumulative distribution function.

[0009] YUV is a color encoding method. It is often used in various video processing components. When encoding photos or videos, YUV takes into account the human perception ability and allows reducing the bandwidth of chrominance. In the YUV representation method, the physical meaning of the Y component is brightness, and the U and V components represent color differences. Summary of the Invention

[0010] To solve the above problems, the purpose of this method is to propose a method for improving the clarity of wide dynamic range images, which ensures that while increasing the image brightness, it enhances the image details and contrast, and thus achieves the purpose of improving the image clarity.

[0011] Specifically, the present invention provides a method for improving the clarity of wide dynamic range images, and the method includes the following steps:

[0012] S1. Divide the image into M*N blocks, and statistically calculate the histogram and mean value of each block as well as the histogram and mean value of the entire image;

[0013] S2. Use the histogram equalization algorithm based on logarithmic function mapping, and adaptively adjust the degree of contrast enhancement and the degree of brightness improvement through the mean value to obtain the brightness mapping function of each block and the global mapping function of the image;

[0014] S3. Adjust the mapping function of each block according to the mean value and the global mapping function, and update the block mapping function;

[0015] S4. Interpolate to obtain the pixel value after mapping of each point according to the Gaussian weight.

[0016] Before step S1 of the method, it may further include:

[0017] S0. Convert the image from the RGB space to the YUV space, and only process the Y component subsequently.

[0018] In step S1, the following information is statistically calculated:

[0019] (1) The histogram hist of each block i and the probability density function P i , where i ∈ [1,..., M·N];

[0020] (2) The brightness mean value mean of each block i , where i ∈ [1,..., M·N];

[0021] (3) The global histogram hist_g of the image and the probability density function P_g i ;

[0022] (4) The global brightness mean value mean_g of the image.

[0023] In step S2, calculating the global and local brightness mapping functions further includes:

[0024] S2.1. Adjust the gradient of the logarithmic function to control the degree of contrast enhancement:

[0025]

[0026] Among them, λ controls the shape of the logarithmic curve; k represents the pixel value size, and k ∈ [0, 255]; α and β control the gradient of the mapping function, and α + β·(L - 1) = 1, where α ∈ [0, 1];

[0027] These parameters can all be set, and among them, α can be adaptively adjusted according to the average brightness:

[0028]

[0029] Among them, x_α = [0, 63, 127, 191, 255], y_α = [0.5, 0.4, 0.3, 0.2, 0], and the specific number of segments and parameter values can be adjusted according to requirements;

[0030] S2.2. According to the gradient of the adjusted mapping function, that is, the probability density function PDF, restrict the PDF of the original image, and the method is as follows:

[0031]

[0032] Among them, P src (k) is the probability density function of the original image, and P dst (k) represents the target probability density function, and P over (k) = P log (k) + 2·P log (255), and P under = P log (k) - P log (255) / 2;

[0033] S2.3. According to the PDF, the cumulative distribution function CDF can be obtained, and then the mapping function can be obtained:

[0034]

[0035]

[0036] S2.4. In order to reduce the degree of brightness improvement in the highlight area and retain more details in the bright area, some adjustments can be made to the mapping function of the blocks with a larger image mean:

[0037] f i (k) = min(thr i , f i (k))

[0038] Among them, thr i is calculated according to the mean mean i of each block. The larger the average brightness, the smaller the stretching degree of the mapping curve.

[0039]

[0040] Among them, x_thr and y_thr can be adjusted;

[0041] S2.5. According to the above calculations, M*N local f can be obtained respectively i , where i ∈ [1, M·N], and the global mapping function f_g.

[0042] In the step S2.4, x_thr = [0, 80, 100, 150, 220] and y_thr = [80, 130, 160, 200, 220] are used.

[0043] The step S3 of adjusting the mapping function of each block further includes: determining the fusion weight of each block mapping curve and the global mapping curve according to the relationship between the global image mean and the mean of each block image, so as to adjust the mapping curve of each block and avoid the situation that the contrast is increased too high or too low. Obtain the final mapping function g of each block i .

[0044]

[0045] g i = w_g i ·f_g+(1 - w_g i )·f i

[0046] where diff i represents the absolute value of the difference between the mean of the i-th image block and the global mean. According to this value, the fusion weight w_g of the global mapping curve can be calculated i , and then the global and local block curves are weighted and fused according to the weight to obtain the mapping curve of each block.

[0047] In the step S3, x_w = [0, 10, 20, 30, 50] and y_w = [0, 0.1, 0.2, 0.3, 0.4] are used.

[0048] In the step S4, it further includes:

[0049] Gaussian weight fusion, performing weighted fusion on the results obtained from each mapping curve, calculating the distance from the current point

[0050] to the center point of the remaining blocks of the image, and determining the fusion weight according to the distance:

[0051] S4.1. Calculate the distance d from the current point p(x0, y0) to the center point of the remaining image blocks i , where i ∈ [1, M·N - 1];

[0052] S4.2. According to d iDetermine the Gaussian weight value:

[0053]

[0054] Among them, both a and c are adjustable parameters. The smaller c is, the more concentrated the weight distribution is, and the more obvious the regional difference is. The larger c is, the more dispersed the weight distribution is, and the smaller the regional difference is;

[0055] S4.3. Substitute the pixel value of the current point into each block mapping curve to calculate the corresponding mapped pixel value p′ i ;

[0056] S4.4. Perform weighted fusion to obtain the result after mapping for each pixel point:

[0057]

[0058] In S4.2, the tested image is 1080p, c = 300, and a = 1.

[0059] After step S4 of the method, it may further include:

[0060] S5. Combine the obtained Y after mapping with the UV of the input image, and convert it to the RGB space to obtain a wide dynamic range image with improved clarity.

[0061] Thus, the advantages of this application are: This method

[0062] (1) Combine local information with global information, fully enhance the brightness of the dark area, and make the details in the dark area more sufficient;

[0063] (2) Effectively suppress the improvement degree of the bright area through local statistical information, and enhance the clarity of the bright area;

[0064] (3) Use Gaussian weight to fuse the local mapping results to eliminate the block effect. Description of the Drawings

[0065] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not limit the present invention.

[0066] Figure 1 is a schematic flowchart of the method of this application.

[0067] Figure 2 is a schematic flowchart of the steps of a specific embodiment of the method of this application. Detailed Description of the Invention

[0068] In order to be able to more clearly understand the technical content and advantages of the present invention, the present invention will now be further described in detail with reference to the drawings.

[0069] AsFigure 1 As shown in the figure, a method for improving the clarity of wide dynamic range images according to the present invention, the method includes the following steps:

[0070] S1. Divide the image into M*N blocks, and statistically calculate the histogram and mean value of each block, as well as the histogram and mean value of the entire image;

[0071] S2. Use the histogram equalization algorithm based on logarithmic function mapping, and adaptively adjust the degree of contrast enhancement and brightness improvement through the mean value to obtain the brightness mapping function of each block and the global mapping function of the image;

[0072] S3. Adjust the mapping function of each block according to the mean value and the global mapping function, and update the block mapping function;

[0073] S4. Interpolate to obtain the pixel value after mapping of each point according to the Gaussian weight.

[0074] As Figure 2 shown, the method can be specifically further described as follows:

[0075] 1. Convert the image from the RGB space to the YUV space, and only process the Y component subsequently

[0076] 2. Divide the image into M*N blocks, and statistically calculate the following information:

[0077] (1) The histogram hist i and probability density function P i of each block, where i ∈ [1,..., M·N];

[0078] (2) The brightness mean value mean i of each block, where i ∈ [1,..., M·N];

[0079] (3) The global histogram hist_g and probability density function P_g i of the image;

[0080] (4) The global brightness mean value mean_g of the image;

[0081] 3. Calculate the global and local brightness mapping functions:

[0082] First, adjust the gradient of the logarithmic function to control the degree of contrast enhancement:

[0083]

[0084] Among them, λ controls the shape of the logarithmic curve; k represents the pixel value, where k ∈ [0, 255]; α and β control the gradient of the mapping function, and α + β·(L - 1) = 1, with α ∈ [0, 1]. These parameters can all be set, and among them, α can be adaptively adjusted according to the brightness mean:

[0085]

[0086] Among them, x_α = [0, 63, 127, 191, 255], y_α = [0.5, 0.4, 0.3, 0.2, 0], and the specific number of segments and parameter values can be adjusted according to requirements.

[0087] Then, according to the gradient of the adjusted mapping function, that is, the PDF, the PDF of the original image is restricted as follows:

[0088]

[0089] Among them, P src (k) is the probability density function of the original image, and P dst (k) represents the target probability density function, and P over (k) = P log (k) + 2·P log (255), and P under = P log (k) - P log (255) / 2.

[0090] According to the PDF, the CDF can be obtained, and then the mapping function can be obtained:

[0091]

[0092]

[0093] In addition, in order to reduce the degree of brightness improvement in the highlighted area and retain more details in the bright area, the mapping function of the block with a larger image mean can be adjusted as follows:

[0094] f i (k) = min(thr i , f i (k))

[0095] Among them, thr i is calculated according to the mean mean i of each block. The larger the brightness mean, the smaller the stretching degree of the mapping curve.

[0096]

[0097] Among them, x_thr and y_thr can be adjusted. In the present invention, x_thr = [0, 80, 100, 150, 220] and y_thr = [80, 130, 160, 200, 220] are used.

[0098] According to the above calculations, M * N local f values can be obtained respectively. i , where i ∈ [1, M·N], and the global mapping function f_g.

[0099] 4. According to the relationship between the global image mean and the mean of each image block, determine the fusion weight of each block mapping curve and the global mapping curve, so as to adjust the mapping curve of each block and avoid the situation of excessive or too low contrast improvement. Obtain the final mapping function g of each block. i .

[0100]

[0101] g i = w_g i ·f_g + (1 - w_g i )·f i

[0102] where diff i represents the absolute value of the difference between the mean of the i-th image block and the global mean. According to this value, the fusion weight w_g of the global mapping curve can be calculated. i , and then the global and local block curves are weighted and fused according to the weight to obtain the mapping curve of each block. Among them, x_w = [0, 10, 20, 30, 50] and y_w = [0, 0.1, 0.2, 0.3, 0.4].

[0103] 5. In order to reduce the difference between mapped image blocks, it is necessary to fuse the results of each mapping curve. Since high-dynamic-range images have richer image content and the gap between blocks is large, obvious block effects will still exist when using the traditional bilinear interpolation method. In order to achieve a better smoothing effect, the present invention adopts the Gaussian weight fusion method to perform weighted fusion on the results obtained from each mapping curve. Calculate the distance from the current point to the center point of the remaining image blocks, and determine the fusion weight according to the distance:

[0104] (1) Calculate the distance d from the current point p(x0, y0) to the center point of the remaining image blocks. i , where i ∈ [1, M·N - 1].

[0105] (2) Determine the Gaussian weight value according to d i :

[0106]

[0107] Among them, both a and c are adjustable parameters. The smaller c is, the more concentrated the weight distribution is, and the more obvious the regional difference is. The larger c is, the more dispersed the weight distribution is, and the smaller the regional difference is. The image tested in the present invention is 1080p, c = 300, and a = 1.

[0108] (3) Substitute the pixel value of the current point into each block mapping curve to calculate the corresponding mapped pixel value p'. i .

[0109] (4) Perform weighted fusion to obtain the result after mapping for each pixel point:

[0110]

[0111] 6. Combine the obtained Y after mapping with the UV of the input image and convert it to the RGB space to obtain the wide dynamic range image with improved clarity.

[0112] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, various changes and modifications can be made to the embodiments of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for improving the clarity of wide dynamic range images, characterized in that, The method includes the following steps: S1. Divide the image into M*N blocks, and calculate the histogram and mean of each block as well as the histogram and mean of the entire image; S2. Use the histogram equalization algorithm based on logarithmic function mapping, and make adaptive adjustments to the degree of contrast enhancement and brightness improvement through the mean value to obtain the brightness mapping function of each block and the global mapping function of the image; In step S2, calculating the global and local brightness mapping functions further includes: S2.

1. Adjust the gradient of the logarithmic function to control the degree of contrast enhancement: Where, λ controls the shape of the logarithmic curve; k represents the pixel value size, k ∈ [0, 255]; α and β control the gradient of the mapping function, and α + β·(L - 1) = 1, α ∈ [0, 1]; These parameters can all be set, and among them, α can be adaptively adjusted according to the brightness mean: Where, x_α = [0, 63, 127, 191, 255], y_α = [0.5, 0.4, 0.3, 0.2, 0], and the specific number of segments and parameter values can be adjusted according to requirements; S2.

2. According to the gradient of the adjusted mapping function, that is, the probability density function PDF, limit the PDF of the original image, and the method is as follows: Among them, P src (k) is the probability density function of the original image, and P dst (k) represents the target probability density function, where P over (k) = P log (k) + 2·P log (255), and P under = P log (k) - P log (255) / 2; S2.

3. According to the PDF, the cumulative distribution function CDF can be obtained, and then the mapping function can be obtained: S2.

4. In order to reduce the degree of brightness improvement in the highlight area and retain more details in the bright area, make some adjustments to the mapping function of the block with a larger image mean: f i (k) = min(thr i , f i (k)) Among them, thr i is calculated based on the mean mean i of each block. The greater the mean brightness, the smaller the stretching degree of the mapping curve Where x_thr and y_thr can be adjusted; S2.

5. M*N local f's can be obtained respectively according to the above calculations i , where i ∈ [1, M·N], and the global mapping function f_g; S3. Determine the fusion weights of each block mapping curve and the global mapping curve according to the relationship between the global image mean and the mean of each block image, so as to adjust the mapping curve of each block and obtain the final mapping function g of each block i ; S4. According to the Gaussian weight, perform weighted fusion on the results obtained from each mapping curve, and interpolate to obtain the pixel value after mapping for each point.

2. The method for improving the clarity of a wide dynamic range image according to claim 1, wherein Before step S1 of the method, it may further include: S0. Convert the image from the RGB space to the YUV space, and only process the Y component subsequently.

3. A method for improving the clarity of wide dynamic range images according to claim 1, characterized in that, In step S1, the following information is calculated: (1) Histogram hist of each block i and probability density function P i , where i ∈ [1,..., M·N]; (2) Probability density function \(P_g\) of the global histogram hist_g of the image i .

4. A method for improving the clarity of wide dynamic range images according to claim 1, characterized in that, In step S2.4, x_thr = [0, 80, 100, 150, 220] and y_thr = [80, 130, 160, 200, 220] are used.

5. A method for improving the clarity of wide dynamic range images according to claim 1, characterized in that, In the step S3: the obtained mapping function g of each block i : diff i = |mean_g - mean i | g i = w_g i ·f_g + (1 - w_g i )·f i Among them, the global brightness mean of the image is mean_g, diff i represents the absolute value of the difference between the mean of the i-th image block and the global mean. Based on this value, the fusion weight w_g of the global mapping curve can be calculated i , and then the global and local block curves are weighted and fused according to the weight to obtain the mapping curve of each block.

6. A method for improving the clarity of wide dynamic range images according to claim 5, characterized in that, In step S3, x_w = [0, 10, 20, 30, 50] is used, y_w = [0, 0.1, 0.2, 0.3, 0.4].

7. A method for improving the clarity of wide dynamic range images according to claim 1, characterized in that, In step S4, it further includes: Calculate the distance from the current point to the center point of the remaining blocks in the image, and determine the fusion weight according to the distance: S4.

1. Calculate the distance d between the current point p(x0, y0) and the center point of the remaining image block i , where i ∈ [1, M·N - 1]; S4.

2. Determine according to d i Determine the Gaussian weight value: Where, both a and c are adjustable parameters. The smaller c is, the more concentrated the weight distribution is, and the more obvious the regional difference is. The larger c is, the more dispersed the weight distribution is, and the smaller the regional difference is; S4.

3. Substitute the current pixel value into each block mapping curve to calculate the corresponding mapped pixel value p′ i ; S4.

4. Perform weighted fusion to obtain the result after mapping for each pixel point:

8. A method for improving the clarity of wide dynamic range images according to claim 7, characterized in that In S4.2, the tested image is 1080p, c = 300, and a = 1.

9. A method for improving the clarity of a wide dynamic range image according to claim 1, characterized in that, After step S4 of the method, it may further include: S5. Combine the mapped Y with the UV of the input image, and convert it to the RGB space to obtain a high - definition wide - dynamic - range image.

Images