A multi-scale WLS filtering fusion-based tone mapping method

Abstract

The application discloses a tone mapping algorithm based on multi-scale WLS filtering fusion, and belongs to the field of image enhancement. High dynamic range images (HDR) cannot be displayed on commonly used low dynamic range (LDR) display devices due to higher dynamic range and bit depth, and thus need to be subjected to tone mapping. The tone mapping algorithm based on multi-scale WLS filtering fusion comprises the following steps: S1, inputting a high dynamic range image, calculating the luminance graph of the input image, and converting to a logarithmic domain; S2, using WLS filters of different scales to filter the luminance graph to obtain a base layer, using the base layer to realize multi-scale detail layer extraction, then weighting and fusing the details of different scales with the base layer to realize detail enhancement; S3, according to a human visual sensitivity model, adjusting the dynamic range of the fused image; and S4, color reconstruction, to obtain a final LDR image.

Description

Technical Field

[0001] This invention relates to the field of image enhancement technology, and in particular to a high dynamic range image tone mapping method based on multi-scale WLS filtering. Background Technology

[0002] Dynamic range refers to the ratio between the brightness values ​​of the brightest and darkest points in an image or scene. In the real world, the brightness range of a scene is typically 10. -6 Up to 10 9 cd / m 2 The dynamic range can reach 10. 15 Standard dynamic range (LDR) images are typically 24-bit RGB images, with 8 bits per color channel. Their dynamic range is only 0 to 255. In high dynamic range scenes, these images may suffer from overexposure in bright areas, underexposure in dark areas, and loss of detail.

[0003] Currently, high dynamic range (HDR) imaging technology is attracting increasing attention, with huge application prospects in fields such as digital television, medical care, and security monitoring. HDR images are typically generated by synthesizing photographs of the same scene at different exposures. They can store the full brightness values ​​of a real scene proportionally, with each color channel being 12 bits or higher, thus possessing a higher dynamic range and better reproducing the colors and details of a real scene. However, commonly used low dynamic range (LDR) display devices have 8 bits per color channel, with a dynamic range of only 10. 2 The dynamic range is on the order of magnitude smaller than that of an LDR image, so HDR images cannot be directly displayed on LDR devices. In order to reproduce the rich colors and details of a high dynamic range image on a conventional display device, the dynamic range needs to be appropriately compressed; this process is called tone mapping.

[0004] Currently, tone mapping techniques are generally divided into two categories: global tone mapping algorithms and local tone mapping algorithms. Global tone mapping algorithms transform all pixels of an image using a spatially invariant mapping formula, thus offering advantages such as high computational speed, simple implementation, and no halo effect. However, because the same formula is applied to all pixels, the mapped image has low contrast and loses some detail information, presenting limitations. Local tone mapping algorithms utilize the statistical distribution of brightness values ​​within a certain neighborhood of a pixel to perform compression to varying degrees, offering greater flexibility and helping to preserve image detail information and improve contrast. However, compared to global mapping algorithms, they are computationally more complex and prone to halos and artifacts. In recent years, most research has focused on improving the performance of local tone mapping algorithms, reducing halos and artifacts, and increasing computational speed. Global tone mapping algorithms have received less attention due to the lower contrast of the mapped image, but compared to local mapping, global mapping is computationally efficient and does not produce artifacts.

[0005] In addition, most tone mapping algorithms need to adjust parameters when mapping different high dynamic range images to obtain high-quality LDR images. However, in practical applications, it is difficult to manually adjust parameters one by one for different high dynamic range scenes. Therefore, an efficient tone mapping algorithm must be robust to different high dynamic range scenes. SUMMARY

[0006] The present application aims to solve the shortcomings of the existing global tone mapping algorithm, which has poor contrast and is prone to lose detail information, and the problem of poor robustness of the algorithm to the mapping results of different high dynamic range scenes, and proposes a tone mapping algorithm based on multi-scale WLS filtering fusion, which improves the overall contrast of the image, retains rich color and detail information, and has the advantages of no parameter adjustment, strong adaptability and good robustness, facilitating application on current low dynamic range display devices.

[0007] To achieve the above purpose, the present application adopts the following technical solutions:

[0008] It should be noted that:

[0009] The WLS filter used in the present application is an edge-preserving filter that can not only preserve edges but also achieve multi-scale detail extraction when smoothing images, and can also avoid halo effect and gradient reversal. The WLS filter is based on a weighted least squares framework, which needs to achieve two contradictory objectives. For an input image I, the filtered image B should be as close to I as possible, and I should be as smooth as possible except for places where the edge gradient changes greatly. The formula is as follows:

[0010]

[0011] Where subscript p is the position of the pixel; the first term achieves the purpose of preserving edges by minimizing the difference between the filtered image B and the input image I, and the second term achieves the purpose of smoothing the image by reducing large gradient points in the filtered image B; λ is called a smoothing coefficient, and increasing λ can improve the smoothing degree of the image.a x,p (I) and a y,p (I) is a smoothing weight coefficient, and the formula is as follows:

[0012]

[0013]

[0014] Where the exponential parameter α controls the gradient sensitivity of the image I, and ε is a small number to make the denominator not equal to 0. Based on the above content, the present application proposes:

[0015] A tone mapping algorithm based on multi-scale WLS filtering fusion, specifically comprising the following steps:

[0016] S1, input a high dynamic range image, calculate the luminance map of the input image, and convert to the logarithmic domain;

[0017] S2, use WLS filters of different scales to filter the luminance map to obtain a base layer, use the base layer to realize multi-scale detail layer extraction, then weight and fuse the details of different scales with the base layer to realize detail enhancement;

[0018] S3, according to the human visual sensitivity model, adjust the dynamic range of the fused image;

[0019] S4, color reconstruction, color reconstruction is performed on the mapped luminance map L1 by calculating the ratio of the color channel to the initial luminance map L0, and the final LDR image is obtained, the calculation formula is:

[0020]

[0021] Wherein, C1 is the low dynamic range image after tone mapping, C0 is the input high dynamic range image, C=R, G, B, representing three color channels.

[0022] Preferably, the S1 specifically comprises the following contents:

[0023] S101, input a high dynamic range image, which can be selected from several existing high-quality public HDR data sets;

[0024] S102, according to the pixel values of R, G, B three channels of the input image, calculate the luminance value L0 and convert to the logarithmic domain L wl , the calculation formula is:

[0025] L0=0.299*R+0.587*G+0.114*B

[0026] Wherein, R, G, B represents the three channel pixel values of the input image.

[0027] Preferably, the S2 specifically comprises the following contents:

[0028] S201, use three WLS filters of different scales to filter and decompose L wl , sequentially obtain the base layers u1, u2, u3, wherein u3 is the smoothest image of the largest scale, and take L wl as the base layer u0;

[0029] S202, subtract the base layers of adjacent scales to obtain the detail layers, realize multi-scale detail information extraction, and the calculation formula is:

[0030] d i =u i-1 -u i i = 1, 2, 3

[0031] S203, d detail layers of different scales i Weighted fusion is performed with the base layer u3, by adjusting the weights w. i To achieve enhanced detail, the calculation formula is as follows:

[0032]

[0033] Preferably, S3 specifically includes the following:

[0034] S301. Based on the dynamic range of L1 and the display's dynamic range, calculate the linear compression operator K1; since the human visual system's response to brightness should follow a logarithmic property, the calculation is performed in the logarithmic domain, and the formula is as follows:

[0035]

[0036] S302. Since human visual sensitivity is concentrated in the logarithmic mean brightness and follows a Gaussian distribution, the Gaussian weight ω(L) is multiplied by the linear compression factor K1 so that the weight is maximized when the brightness value of L1 equals the logarithmic mean μ. The Gaussian weighting yields the nonlinear compression factor K2, calculated using the following formula:

[0037] K2(L)=(1-K1)ω(L)+K1

[0038]

[0039] Where ω(L) is the Gaussian weight, satisfying K1≤K2(L)≤1.0; μ is the average logarithmic brightness of the input brightness; d0 is the range of the input brightness; c1 is the user parameter, controlling the brightness and local contrast of the mapped image; σ is the ratio of the two.

[0040] S303. Substituting the values ​​into the final nonlinear compression formula, we obtain the mapped brightness value. The calculation formula is as follows:

[0041] L1 = exp(c2k2(L1-μ) + μ)

[0042] Where c2 is the compression factor, which controls the dynamic range after mapping;

[0043] S304. In order to optimize the dynamic range of the display device, the mapped brightness values ​​are sorted, limited to 1% to 99%, and normalized.

[0044] The present application filters high dynamic range images by using WLS filters of different scales, so as to obtain corresponding smooth images and detail information, then fuses the multi-scale detail information with the smooth images to obtain a brightness graph with enhanced details.

[0045] Compared with the prior art, the present application provides a tone mapping algorithm based on multi-scale WLS filter fusion, which has the following beneficial effects:

[0046] (1) Compared with the traditional global tone mapping method, the image obtained by the method provided by the present application can maintain high overall contrast and local contrast, and after detail enhancement, the edge characteristics are well maintained, and rich detail information is preserved in high brightness and low brightness places.

[0047] (2) The method provided by the present application can have good processing results for all high dynamic range scene images, without the need to adjust parameters, and is more efficient and convenient for application on common LDR display devices.

[0048] (3) The image obtained by the method provided by the present application will not appear halo and gradient reversal. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 The present application provides a tone mapping method based on multi-scale WLS filter fusion, and the algorithm flowchart is shown in the figure;

[0050] Figure 2 The effect graph of the algorithm for tone mapping in the present application embodiment 1 is shown in the figure; Figure 2 (a) is a schematic diagram of the original brightness value display result of the high dynamic range image; Figure 2 (b) is a schematic diagram of the result of the algorithm for dynamic range compression;

[0051] Figure 3 The filtering result schematic diagram of the WLS filter of different scales in the present application embodiment 1 is shown in the figure, from left to right, u0, u1, u2, u3 respectively;

[0052] Figure 4 The detail extraction result schematic diagram of different scales in the present application embodiment 1 is shown in the figure, from left to right, d0, d1, d2 respectively;

[0053] Figure 5 、 Figure 6The figures below are comparisons of the effects of the method proposed in this invention in Example 1 with other tone mapping methods; where, figure a is the mapping result using Drago's adaptive logarithmic algorithm; figure b is the mapping result using Reinhard's photographic algorithm; figure c is the result using Min H. Kim's global tone mapping algorithm; and figure d is the mapping result of the method proposed in this invention. Detailed Implementation

[0054] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0055] Example 1:

[0056] Please see Figures 1-5 This example uses a 768*512 high dynamic range church image as input and processes it using the method of this invention, specifically including the following steps:

[0057] S1. Input a high dynamic range image, calculate the brightness map of the input image, and convert it to the logarithmic domain. Specifically, this includes the following:

[0058] S101. Read a high dynamic range image in .hdr format, such as... Figure 2 As shown in (a);

[0059] S102. Calculate the luminance value L0 based on the pixel values ​​of the R, G, and B channels of the input image and convert it to the logarithmic domain L. wl The calculation formula is:

[0060] L0 = 0.299*R + 0.587*G + 0.114*B

[0061] Where R, G, and B are the three-channel pixel values ​​of the input image;

[0062] S2. The brightness map is filtered using WLS filters of different scales to obtain the base layer. The base layer is then used to extract details at multiple scales. Finally, the details at different scales are weighted and fused with the base layer to achieve detail enhancement. This includes the following:

[0063] S201, Using three WLS filters of different scales for L wl Filtering decomposition is performed, where the smoothing coefficient λ is set to 0.125, 1.0, and 8.0 respectively, and the gradient sensitivity α is set to 1.2, resulting in smoothed images u1, u2, and u3, where u3 is the smoothed image at the largest scale. L is then... wl As the base layer u0. Smooth images u0, u1, u2, u3 as follows Figure 3 As shown.

[0064] S202. Subtract the base layers of adjacent scales to obtain detail layers d0, d1, d2, as follows: Figure 4 As shown:

[0065] d i =u i-1 -u i i = 1, 2, 3

[0066] S203, d detail layers of different scales i The enhanced luminance map L1 is obtained by weighted fusion with the maximum-scale smoothed image u3, where w0, w1, w2, and w3 are taken as 1.0, 1.0, 1.6, and 0.4, respectively; the calculation formula is as follows:

[0067]

[0068] S3. Based on the human visual sensitivity model, adjust the dynamic range of the fused image, specifically including the following:

[0069] S301. Calculate the linear compression operator K1 based on the dynamic range of L1 and the display's dynamic range. Since the human visual system's response to brightness follows a logarithmic property, the calculation is performed in the logarithmic domain. The calculation formula is:

[0070]

[0071] S302. Since human visual sensitivity is concentrated in the logarithmic mean brightness and follows a Gaussian distribution, the Gaussian weight ω(L) is multiplied by the linear compression factor K1 so that the weight is maximized when the brightness value of L1 equals the logarithmic mean μ. The Gaussian weighting yields the nonlinear compression factor K2, calculated using the following formula:

[0072] K2(L)=(1-K1)ω(L)+K1

[0073]

[0074] Where ω(L) is the Gaussian weight, satisfying K1≤K2(L)≤1.0; μ is the average logarithmic brightness of the input brightness; d0 is the range of the input brightness; c1 is the user parameter, controlling the brightness and local contrast of the mapped image; σ is the ratio of the two. Extensive experiments have shown that c1 values ​​between 3.0 and 5.0 result in better overall brightness and contrast in the mapped image; here, 3.0 is chosen.

[0075] S303. Substituting the values ​​into the final nonlinear compression formula, we obtain the mapped brightness value. The calculation formula is as follows:

[0076] L1 = exp(c2k2(L1-μ) + μ)

[0077] wherein c2 is a compression factor, controlling the mapped dynamic range, which is taken as 0.3 here.

[0078] S304, in order to optimize the dynamic range of the display device, the mapped luminance value is sorted, limited to 1%~99%, and normalized.

[0079] S4, color reconstruction, the mapped luminance map L1 is color reconstructed by calculating the ratio of the color channel and the initial luminance map L0, to obtain the final LDR image, and the calculation formula is:

[0080]

[0081] wherein C1 is the low dynamic range image after tone mapping, C0 is the input high dynamic range image, C=R, G, B, representing three color channels.

[0082] The final image after tone mapping is shown in Figure 2 (b).

[0083] Figure 3 The figure is a schematic diagram of the filtering results of the WLS filter with different scales used in the application, from left to right, u0, u1, u2, u3 respectively;

[0084] Figure 4 The figure is a schematic diagram of the detail extraction results of different scales, from left to right, d0, d1, d2 respectively;

[0085] Figure 5 、 Figure 6 The figure is a comparison diagram of the effect of the method proposed in the application and other tone mapping methods; wherein a figure is the mapping result diagram of the adaptive logarithmic algorithm of Drago; b figure is the mapping result diagram of the photographic algorithm of Reinhard; c figure is the result diagram of the global tone mapping algorithm of Min H. Kim; d figure is the mapping result diagram of the method proposed in the application.

[0086] The above is only the preferred specific embodiment of the application, but the protection scope of the application is not limited to this, any skilled person in the art, according to the technical solution and the inventive concept of the application, makes equivalent replacement or change within the technical range disclosed by the application, which should be covered in the protection scope of the application.

Claims

1. A multi-scale WLS filter fusion based tone mapping method, characterized in that, The image with enhanced details is obtained by using multi-scale WLS filtering, specifically including the following steps: S1, input a high dynamic range image, calculate the luminance of the input image, and convert to the logarithmic domain; S2, use WLS filters of different scales to filter the luminance to obtain a base layer, use the base layer to extract a multi-scale detail layer, then weight and fuse the details of different scales with the base layer to achieve detail enhancement; S3, adjust the dynamic range of the fused image according to the human visual sensitivity model; specifically including the following contents: S301、According to L 1. The dynamic range of the display, the linear compression operator is calculated K 1; Since the response of the human visual system to brightness obeys the logarithmic property, the calculation is carried out in the logarithmic domain, and the calculation formula is: S302、Since human visual sensitivity is concentrated on the logarithmic average brightness, and obeys Gaussian distribution, multiply the linear compression factor by Gaussian weight ω ( L ) with Gaussian weight K 1, so that L 1 the brightness value is equal to the logarithmic average value μ When the weight is maximum; the nonlinear compression factor after Gaussian weighting is obtained K 2, the calculation formula is: K 2（ L ）=（1- K 1） ω （ L ）+ K 1 wherein, ω L ) is a Gaussian weight satisfying K 1≤ K 2( L )≤1.0; μ is the average log luminance of the input luminance; d 0 is the range of the input luminance; c 1 is a user parameter controlling the luminance and local contrast of the mapped image; σ is the ratio of the two;​ S303, input the final nonlinear compression formula to obtain the mapped luminance value, the calculation formula is: L 1 = exp( - 1 c 2 k 2( - 1 L 1- μ ) + 1 μ ) wherein, c 2 is a compression factor, controlling the mapped dynamic range; S304, sort the mapped luminance value, limit it to 1%~99%, and normalize it; S4, color reconstruction, by computing the color channels from the initial luminance map L 0 the ratio of the mapped luminance map L 1 color reconstruction, to obtain the final LDR image, the formula is: wherein, C 1 is a low dynamic range image after tone mapping, C 0 is an input high dynamic range image, C = R, G, B, representing three color channels.

2. The multi-scale WLS filter fusion based tone mapping method of claim 1, wherein, The S1 specifically includes the following contents: S101, input a high dynamic range image, select from several existing high-quality public HDR data sets; S102、According to the pixel value of R, G, B three channels of the input image, the luminance value is calculated L 0 and converted to the logarithmic domain L wl The calculation formula is: L 0= 0.299 * R + 0.587 * G + 0.114 * B Wherein, R, G, B represents the three channel pixel values of the input image.

3. The multi-scale WLS filtering fusion based tone mapping method of claim 1, wherein, The S2 specifically includes the following contents: S201, Using three different scale WLS filters L wl Perform filtering decomposition to obtain the base layer sequentially. u 1, u 2, u 3, of which u 3 is the smoothed image at the largest scale, and it... L wl As the base layer u 0; S202, subtract the base layers of adjacent scales to obtain the detail layers, and realize multi-scale detail information extraction, the calculation formula is: d i = u i-1 - u i ， i =1,2,3 S203, Layering details at different scales d i With the base layer u 3. Perform weighted fusion by adjusting the weights. w i To achieve enhanced detail, the calculation formula is as follows: 。

Images