Image dimming method and device with high dynamic range

Abstract

The invention provides an image dimming method and device with a high dynamic range. The method comprises the following steps: acquiring a histogram statistical value of a current image; traversing the statistical value of the image histogram to obtain an initial platform threshold value; on the basis of the initial platform threshold value, traversing and modifying the histogram statistical value to obtain an optimal platform threshold value of the background image and the target image; and through the optimal threshold value of the platform, the quality and the overall brightness of the image are ensured, and the adaptive problem of the scene is solved. Obtaining a mapping relation of the background image and a mapping relation of the target image based on the platform optimal threshold; looking up the table by taking the current image gray as an index to obtain a background image and a target image; based on a background image and a target image, a pyramid fusion algorithm is adopted to decompose the target image and the background image into image layers of different scales, and the image layers of the maximum scale are fused in sequence to obtain a fusion image which gives consideration to the background and the target at the same time, so that the problem of detail loss is solved.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a method and device for dimming an image with a high dynamic range. Background Art

[0002] The output image of the infrared detector is usually 14-bit or 16-bit, and in general scenes, the effective grayscale only occupies a small part of the total grayscale range, while the display device can generally only display 8-bit or 10-bit images. Therefore, it is necessary to analyze the effective grayscale from the image and increase it to the display grayscale range. This process is called dimming and is a dynamic range compression algorithm.

[0003] Commonly used dynamic range compression algorithms mainly include the following aspects:

[0004] A: Information statistics. Perform histogram statistics on the input image, starting from the first grayscale, accumulate the number of pixels on the current grayscale. When the accumulated sum is greater than or equal to the preset THR1, the grayscale at this time is the starting point G1 of the valid grayscale. When the accumulated sum is greater than or equal to the preset THR2, the current grayscale is the ending point G2 of the valid grayscale.

[0005] B: Mapping relationship calculation. There are two commonly used methods for mapping relationship calculation: linear mapping and nonlinear mapping. Linear mapping is to map G1 to the starting point of the target grayscale range, map G2 to the end point of the target grayscale, and obtain the intermediate grayscale by linear interpolation. Nonlinear mapping usually uses histogram equalization or its improved algorithm. The basic principle is to perform grayscale mapping based on the probability distribution of the grayscale of the image. The contrast of grayscales with a large proportion is stretched, and the contrast of grayscales with a small proportion is compressed.

[0006] Conventional dynamic range compression algorithms have the following defects:

[0007] A: The background or target details are seriously lost. When the dynamic range is large or the invalid grayscale accounts for a large proportion, the linear mapping will cause serious loss of image details and the image will not be transparent. When the target is small and the grayscale is small, non-linear mapping will cause serious detail loss. When the background accounts for too large a proportion, problems such as over-stretching, grayscale faults, and unnatural brightness will occur.

[0008] B: Scene adaptability problem. The judgment threshold of linear mapping and the optimal value of the platform histogram threshold under nonlinear mapping change with the scene. If the threshold is unified, the algorithm will perform better in some scenes, but perform poorly in other scenes, or even cause image abnormalities. Summary of the invention

[0009] In order to overcome the deficiencies of the prior art, the present invention provides a high-dynamic-range image dimming method and device. By processing the original image, two sub-images are obtained: one sub-image focuses on retaining background details, and the other sub-image focuses on target details. These two sub-images are fused to generate an image that retains both the background and highlights the target, thus effectively solving the problems of image detail loss and scene adaptability.

[0010] To achieve the above object, a high-dynamic-range image dimming method of the present invention specifically includes the following steps:

[0011] Obtain the histogram statistical value of the current image;

[0012] Traverse the histogram statistical value of the image to obtain the initial platform threshold; based on the initial platform threshold, traverse and modify the histogram statistical value to obtain the optimal platform thresholds of the background image and the target image;

[0013] Based on the optimal platform thresholds, obtain the mapping relationships of the background image and the target image; look up the table with the current image gray level as the index to obtain the background image and the target image;

[0014] Based on the background image and the target image, use the pyramid fusion algorithm to decompose the target image and the background image into image layers of different scales, and fuse them sequentially starting from the largest-scale image layer to obtain the final fused image.

[0015] Preferably, when traversing the histogram statistical value of the image, start from the statistical value of 0 and go up, traverse the gray level at each statistical value, and count the number of non-zero gray levels; stop when the cumulative sum of the statistical values is greater than the set threshold, and use the current statistical value as the initial platform threshold.

[0016] Preferably, the steps for obtaining the optimal platform threshold include:

[0017] Traverse the histogram statistical value starting from gray level 0. If the histogram statistical value is greater than the initial platform threshold, modify the current statistical value to the initial platform threshold; if the histogram statistical value is less than or equal to the initial platform threshold, keep the original value unchanged to obtain the first statistical value;

[0018] Traverse the first statistical value and calculate the cumulative sum of the first statistical value;

[0019] Optimize the cumulative sum of the first statistical value using a preset optimization coefficient to obtain the first platform threshold;

[0020] Traverse the first statistical value of the histogram starting from gray level 0. If the first statistical value is greater than the first platform threshold, modify the current statistical value to the first platform threshold; if the first statistical value is less than or equal to the first platform threshold, keep the original value unchanged to obtain the second statistical value;

[0021] Traverse the second statistical value, calculate the cumulative sum of the second statistical value, and obtain the optimal platform threshold.

[0022] Preferably, the steps of the pyramid fusion algorithm for decomposing the target image and the background image include:

[0023] Perform downsampling on the target image and the background image respectively to obtain downsampled image information;

[0024] Upsample the downsampled background image to obtain the low-frequency map of the first-layer background image;

[0025] Use the current background image information and the downsampled target image information as the input image, and iterate the image downsampling and the background image upsampling operations to the upper level to obtain multi-scale image information of different layers.

[0026] Preferably, after obtaining the background image information of each layer, perform a difference operation with the background image before sampling to obtain the background difference map information.

[0027] Preferably, use a low-pass filter to filter the target image information in the image information of each layer to remove the low-pass image in the target image; subtract the low-pass image from the target image before filtering to obtain the high-frequency image.

[0028] Preferably, when fusing the image information within a layer, the steps include:

[0029] Calculate the variance map and the mean map of the target image and the background image respectively under the window radius r;

[0030] Correct the variance map;

[0031] Calculate the fusion coefficient;

[0032] Fuse the target image information and the background image information according to the fusion coefficient to obtain the fused image.

[0033] Preferably, the correction formula for the variance map is:

[0034] img_bg_l4_vm[i] = { 0 img_bg_l4_m[i] = 0 img_bg_l4_v[i] * (img_fg_l4_v[i] / img_bg_l4_m[i]) 2 else } ,

[0035] The calculation formula for the fusion coefficient coef[i] is:

[0036] sum_v[i] = img_fg_l4_v[i] + img_bg_l4_vm[i] ,

[0037] coef[i] = { 0 sum_v[i] = 0 img_fg_l4_v[i] * coef_set / sum_v[i] else } ,

[0038] The formula for fusing the target image and the background image is:

[0039] img_fus_l4[i] = img_bg_l4[i] + coef[i] * img_fg_l4_hf[i] ;

[0040] Among them, img_bg_l4_vm[i] is the corrected background variance array, img_bg_l4_v[i] is the background variance array of the current layer, and img_fg_l4_v[i] is the target variance array of the current layer; img_bg_l4_m[i] is the background mean array of the current layer, and img_fg_l4_m[i] is the target mean array of the current layer; sum_v[i] is the fusion array of the background image and the corrected background image of the current layer, and coef_set is the parameter set by the user; img_fus_l4[i] is the fusion array of the current layer, and img_fg_l4_hf[i] is the background difference array during the decomposition of the current layer.

[0041] Preferably, when fusing the inter-layer image information, the steps include:

[0042] Start fusing from the largest-scale image layer, and obtain the fused image of the current layer through the intra-layer image information fusion operation;

[0043] Combine the fused image of the current layer with the background difference map information for upsampling reconstruction to obtain the reconstructed image;

[0044] Use the reconstructed image of the current layer and the background image of the next layer as the input images, and iterate the inter-layer fusion process to obtain the reconstructed map;

[0045] Use the reconstructed map, the background image, and the high-frequency image of the target image as the input, and iterate the intra-layer image fusion process to obtain the final fused image.

[0046] The present invention also provides a high-dynamic-range image dimming device, which includes:

[0047] A histogram statistics module that obtains the histogram statistics value of the current image;

[0048] A calibration module that traverses the histogram statistics value of the image to obtain the initial platform threshold; based on the initial platform threshold, traverses and modifies the histogram statistics value to obtain the optimal platform thresholds of the background image and the target image; based on the optimal platform thresholds, obtains the mapping relationships of the background image and the target image; looks up the table with the current image grayscale as the index to obtain the background image and the target image;

[0049] A fusion module that, based on the background image and the target image, uses the pyramid fusion algorithm to decompose the target image and the background image into image layers of different scales, and starts fusing from the largest-scale image layer in sequence to obtain the final fused image.

[0050] A high-dynamic-range image dimming method and device provided by the present invention, compared with the prior art, has the beneficial effects that:

[0051] 1. The present invention performs multiple checks and comparisons on the image histogram statistical values and the platform preset values, and then forms an adaptive optimal platform threshold. This optimization process can not only effectively ensure the quality of the background image, but also ensure that the overall brightness of the background is within a suitable range; in addition, the histogram check method proposed by the present invention can effectively prevent the background from being over-stretched. In image processing, over-stretching of the background will cause abnormal image brightness and then lead to the phenomenon of brightness stratification; this method avoids over-stretching of the background and fundamentally avoids the image stratification problem caused by this problem.

[0052] 2. Based on the image histogram statistical values, the present invention divides the original image into a background image and a target image, and calculates and checks the optimal platform threshold for each respectively. Through this processing, this method can generate images that respectively emphasize the background and the target, so as to present the background and the target completely. On the basis of obtaining the background image and the target image, by fusing the two, it can ensure that the details of both the background and the target image are completely retained. At the same time, the fused image can take into account the advantages of both the background and the target and show their respective advantages. Compared with the prior art, this method can be perfectly compatible with backgrounds and targets with good dynamic ranges and has strong adaptability to various scenarios.

[0053] 3. In the prior art, the pyramid fusion algorithm has certain limitations. Its fusion result cannot effectively distinguish between valid information and invalid information, resulting in a large deviation between the fused visual presentation and the expectation. In the process of image fusion of the present invention, the pyramid fusion algorithm is used to perform multi-level decomposition processing on the background image and the target image, generating image layers of different sizes (i.e., different scales). These image layers are stacked in sequence according to the pyramid structure, showing the characteristic that the size gradually increases from bottom to top. When performing each level of decomposition on the image, the background information and target information covered in the current layer image can be comprehensively obtained, specifically including the fusion processing of the intra-layer image and the image fusion operation between layers. Specifically, first, the background information and target information of each layer are respectively and finely reconstructed, and then the inter-layer image reconstruction is performed. After the above reconstruction operations on multiple layers of images, the generated final background information and target information will be further subjected to comprehensive image reconstruction processing to obtain a high-quality final fused image. Therefore, the present invention dynamically determines the incorporation ratio of the target image at the current position by deeply analyzing the distribution of valid information on the background image. When the content of valid information on the background image is low, the incorporation ratio of the target image increases accordingly; conversely, when the valid information on the background image is relatively rich, the incorporation ratio of the target image decreases accordingly. Compared with the existing mature fusion algorithms, the fusion method adopted by the present invention has significant advantages such as a simple calculation process, strong algorithm adaptability, and a natural and realistic fusion effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a flowchart of a high-dynamic-range image dimming method provided by the present invention.

[0055] Figure 2 It is a flowchart for calculating the background image in a high-dynamic-range image dimming method provided by the present invention;

[0056] Figure 3 It is a schematic diagram of the initial platform threshold thr_p1 value for calculating the histogram of the target image in a high-dynamic-range image dimming method provided by the present invention.

[0057] Figure 4 It is a flowchart for calculating the target image in a high-dynamic-range image dimming method provided by the present invention.

[0058] Figure 5 It is a schematic diagram of the initial platform threshold thr_mm for calculating the histogram of the background image in a high-dynamic-range image dimming method provided by the present invention.

[0059] Figure 6 It is a flowchart of image fusion in a high-dynamic-range image dimming method provided by the present invention.

[0060] Figure 7 In the high-dynamic-range image dimming method provided by the present invention, the flowchart of pyramid decomposition adopted in image fusion.

[0061] Figure 8 In the high-dynamic-range image dimming method provided by the present invention, the flowchart of downsampling the background image during image decomposition.

[0062] Figure 9 In the high-dynamic-range image dimming method provided by the present invention, the flowchart of downsampling the target image during image decomposition.

[0063] Figure 10 In the high-dynamic-range image dimming method provided by the present invention, the flowchart of image reconstruction.

[0064] Figure 11 In the high-dynamic-range image dimming method provided by the present invention, the flowchart of upsampling during image reconstruction.

[0065] Figure 12 Schematic diagram of a high-dynamic-range image dimming device provided by the present invention.

[0066] Figure 13 Original image for dimming.

[0067] Figure 14 Image effect obtained by the dimming method in the prior art.

[0068] Figure 15 Effect diagram fused by the pyramid fusion algorithm in the prior art.

[0069] Figure 16 Background image obtained by the high-dynamic-range image dimming method provided by the present invention.

[0070] Figure 17 Target image obtained by the high-dynamic-range image dimming method provided by the present invention.

[0071] Figure 18 Effect diagram after dimming by the high-dynamic-range image dimming method provided by the present invention. Detailed implementation manners

[0072] The following describes the implementation manners of the present invention through specific specific examples. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific implementation manners. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0073] As Figure 1 shown, a high dynamic range image dimming method provided by the present invention specifically includes the following steps:

[0074] Obtain the histogram statistical value of the current image;

[0075] Traverse the histogram statistical value of the image to obtain the initial platform threshold; based on the initial platform threshold, traverse and modify the histogram statistical value to obtain the optimal platform thresholds of the background image and the target image;

[0076] Based on the optimal platform thresholds, obtain the mapping relationships of the background image and the target image; look up the table with the current image gray level as the index to obtain the background image and the target image;

[0077] Based on the background image and the target image, use the pyramid fusion algorithm to decompose the target image and the background image into image layers of different scales, and fuse them in sequence starting from the image layer of the largest scale to obtain the final fused image.

[0078] Specifically, the present invention conducts multiple verifications and comparisons on the histogram statistical value of the image and the platform preset value, and then forms an adaptive optimal platform threshold, so as to ensure that the quality and overall brightness of the background image are within a suitable range and solve the image layering problem. Based on the histogram statistical value of the image, the original image is divided into a background image and a target image, and the optimal platform thresholds are calculated and verified respectively. Through this processing, this method can generate images that respectively emphasize the background and the target, so as to present the background and the target completely. Based on the obtained images, the pyramid fusion algorithm is used to perform multi-level decomposition processing on the background image and the target image to generate image layers of different sizes with background information and target information. Through the fusion processing of the images within the layer and the image fusion operation between the layers, a fused image that can present the advantages of the target and the background is finally obtained. Among them, the optimal platform threshold refers to the dimming threshold that can adapt to the current application scenario environment.

[0079] The way to obtain the histogram statistical value is to count the number of pixels of each gray level on the image. The general platform histogram threshold is set by the user. However, in different application scenarios, its adaptability is poor. For example, when the background occupies a relatively large proportion, the image is prone to be hazy, and the dark area is relatively bright. AsFigure 14 As shown. In view of this, the present invention calculates the optimal platform threshold for the histogram statistical value, and then derives an appropriate histogram mapping relationship to ensure that the background has appropriate brightness and good contrast.

[0080] Based on the histogram statistical value, a background image tending to background details and a target image tending to target details are obtained through an adaptive optimal platform threshold. It should be noted that in the present invention, the method for obtaining the optimal platform threshold for the background image and the target image is the same. In order to more clearly and detailedly describe the obtaining methods of these two images, they will be described separately below.

[0081] As Figure 2 shown, for the processing of the background image:

[0082] (a) Traverse upward from the statistical value of 0. At each statistical value, traverse the number of non-zero grayscales from left to right for the grayscale, as Figure 3 shown. Stop when the cumulative sum is greater than the set platform threshold thr_h1, and the current statistical value is used as the subsequent initial platform threshold thr_p1. That is, search from bottom to top. When the sum of the blue area between the red line and 0 is greater than the condition, the value on the r-axis is the required initial platform threshold thr_p1. Through this step, a suitable histogram platform threshold for the current image can be found to adapt to the background brightness of the current image.

[0083] (b) Traverse the histogram statistical value starting from the grayscale of 0. When the histogram statistical value is greater than thr_p1, modify the statistical value to thr_p1, otherwise keep the original value unchanged to obtain a new statistical value hist_mb; that is: traverse the histogram statistical value starting from the grayscale of 0. If the histogram statistical value is greater than the initial platform threshold thr_p1, modify the current statistical value to the initial platform threshold thr_p1. If the histogram statistical value is less than or equal to the initial platform threshold thr_p1, keep the original value unchanged to obtain the first statistical value hist_mb. By further correcting the initial platform threshold, the contrast adaptability of the background image is better, avoiding being too blurred or having too strong a contrast. Its expression is:

[0084] hist _ m b [ i ] = { hist [ i ] hist [ i ] <= thr_p1 thr_p1 hist [ i ] > thr_p1 } , where hist[i] is the current statistical value.

[0085] (c) Traverse the first statistical value hist_mb and calculate the cumulative sum sum_all_mb of the first statistical value. Its expression is:

[0086] sum_all_mb = ∑ hist_mb[i] .

[0087] (d) Optimize the accumulated sum of the first statistical value sum_all_mb using the preset optimization coefficient coef_mb to obtain the first platform threshold thr_mb1, that is:

[0088] thr_mb1 = sum_all_mb * coef_mb.

[0089] (e) Traverse the first statistical value hist_mb of the histogram starting from gray level 0. If the first statistical value hist_mb is greater than the first platform threshold thr_mb1, modify the current statistical value to the first platform threshold thr_mb1. If the first statistical value is less than or equal to the first platform threshold, keep the original value unchanged to obtain the second statistical value hist_bg. That is:

[0090] hist _ bg [ i ] = { hist _ mb [ i ] hist _ mb [ i ] <= thr_mb1 thr_mb1 hist _ mb [ i ] > thr_mb1 } 。

[0091] (f) Traverse the second statistical value hist_bg, calculate the accumulated sum of the second statistical value sum_all_bg, and obtain the optimal platform threshold. That is:

[0092] sum_all_bg = ∑ hist_bg[i] 。

[0093] Automatically adjust the platform threshold to adapt to the current background image through steps (b) to (f), so that the background image can display its details well.

[0094] (g) Calculate the mapping relationship of the background image, that is:

[0095] tab _ bg [ i ] = hist _ bg [ i ]* 255 / sum _ all _ bg 。

[0096] (h) Look up the table tab_bg with the image gray level as the index to obtain the background image img_bg, that is:

[0097] img_bg[i]=tab_bg[img_in[i]], where img_in is the input image.

[0098] As Figure 4 shown, the calculation steps of the target image include:

[0099] (a) Traverse the histogram statistical value starting from gray level 0. If the histogram statistical value is greater than the initial platform threshold thr_m1, modify the current statistical value to the initial platform threshold thr_m1. If the histogram statistical value is less than or equal to the initial platform threshold thr_m1, keep the original value unchanged to obtain the first statistical value hist_mm. That is:

[0100] hist _ mm [ i ] = { hist [ i ] / thr_m1 hist [ i ] <= thr_m1 thr_m1 / thr_m1 hist [ i ] > thr_m1 } 。

[0101] (b) Traverse the first statistical value hist_mm and calculate the cumulative sum sum_all_mm of the first statistical value. That is:

[0102] sum_all_mm = ∑ hist_mm[i] .

[0103] (c) Optimize the cumulative sum sum_all_mm of the first statistical value using a preset optimization coefficient coef_mm to obtain the first platform threshold thr_mm. That is:

[0104] thr_mm = w*h*coef_mm / sum_all_mm,

[0105] where w and h are the width and height of the image respectively. As Figure 5 shown, the product of the image width and height divided by the sum of the areas of the blue parts before the red line to the 0 line is the first platform threshold thr_mm.

[0106] (d) Traverse the first statistical value hist_mm of the histogram starting from a gray level of 0. If the first statistical value hist_mm is greater than the first platform threshold thr_mm, then modify the current statistical value to the first platform threshold thr_mm. If the first statistical value hist_mm is less than or equal to the first platform threshold thr_mm, then keep the original value unchanged to obtain the second statistical value hist_fg;

[0107] hist _ fg [ i ] = { hist _ mm [ i ] hist _ mm [ i ] <= thr_mm thr_mm hist _ mm [ i ] > thr_mm } .

[0108] (e) Traverse the second statistical value hist_fg and calculate the cumulative sum sum_all_fg of the second statistical value to obtain the optimal platform threshold.

[0109] sum_all_fg = ∑ hist_fg[i] .

[0110] (f) Calculate the mapping relationship of the target image. That is:

[0111] tab _ f g [ i ] = hist _ f g [ i ]* 255 / sum _ all _ f g .

[0112] (g) Look up the table tab_fg with the image gray level as the index to obtain the target image img_fg.

[0113] img_fg[i]=tab_fg[img_in[i]], where img_in is the input image.

[0114] As Figures 6 - 9As shown, for the background image and the target image obtained based on the histogram statistical values, the pyramid fusion algorithm is used to fuse the images highlighting the background and the target. In view of the overall brightness characteristics of the background image and the target image respectively, the present invention adopts the method of using the background image as the base layer and then superimposing and fusing the target image into the background image. If the target image is selected as the base layer, it will cause the overall fused image to be darker, unable to meet the actual requirements of relevant application scenarios, so the scheme of using the target image as the base layer is not adopted.

[0115] Based on the two obtained images, the pyramid fusion algorithm is used to decompose the images into four images of different scales, namely the images corresponding to the first layer l1, the second layer l2, the third layer l3, and the fourth layer l4. Starting from the image of the largest scale for fusion processing, after the fusion is completed, it is restored to the previous level for reconstruction; then the fusion operation is performed again and restored to the previous reconstruction again, and so on until the image is restored to the original size. Taking the image of the first layer as an example, its specific steps include:

[0116] (a) Respectively perform 2X2 downsampling on the target image img_fg and the background image img_bg to obtain the downsampled image information, that is, sample the first-layer background image img_bg_l1 and the first-layer target image img_fg_l1. The sampling method is preferably the mean value or other methods.

[0117] (b) Upsample the first-layer downsampled background image img_bg_l1 to obtain the first-layer background image low-frequency map img_bg_l1_sm. Among them, the upsampling method can use bilinear or other methods.

[0118] Among them, after obtaining the information of each layer of the background image, it is subtracted from the background image before sampling to obtain the background difference map information. In view of the inevitable detail loss during the decomposition process, the background difference map information is obtained to accurately record the loss situation, so as to provide a basis for effectively supplementing the loss during the subsequent reconstruction process. The 0th-layer background difference map img_bg_l0_hf = the 0th-layer background image img_bg - the first-layer background image low-frequency map img_bg_l1_sm.

[0119] (c) Using the current background image information img_bg_l1 (the first-layer background image information) and the downsampled target image information img_fg_l1 as the input images, iteratively perform image downsampling and background image upsampling operations to a higher level to obtain image information at multiple different scales. That is: using the first-layer background image img_bg_l1 and the first-layer downsampled target image img_fg_l1 as the input to perform the next-layer decomposition to obtain the second-layer background image img_bg_l2, the second-layer downsampled target image img_fg_l2, and the first-layer background difference image img_bg_l1_hf. And so on, to obtain the decomposition information of the third and fourth layers respectively, that is: the third-layer background image img_bg_l3, the third-layer downsampled target image img_fg_l3, the second-layer background difference image img_bg_l2_hf, the fourth-layer background image img_bg_l4, the fourth-layer downsampled target image img_fg_l4, and the third-layer background difference image img_bg_l3_hf.

[0120] In the decomposition process of the present invention, steps (a) and (b) are the decomposition process of one layer. After the first-layer decomposition is completed, using the first-layer background image information img_bg_l1 and the first-layer downsampled target image information img_fg_l1 as the input to perform the next-layer decomposition, that is, repeating the process of steps (a) and (b) again, and so on to obtain the decomposition information of the third and fourth layers, as Figure 8 shown. Among them, the specific number of layers in the present invention is only used as an example, and the present invention does not make any limitations.

[0121] (d) Use a low-pass filter to filter the target image information in each layer of image information to remove the low-pass image in the target image; subtract the low-pass image before filtering from the target image to obtain the high-frequency image.

[0122] As Figure 9 shown, the target image information of each layer is respectively: the target image img_fg of the 0th layer, the target image img_fg_l1 after the first-layer downsampling, the target image img_fg_l2 after the second-layer downsampling, the target image img_fg_l3 after the third-layer downsampling, the target image img_fg_l4 after the fourth-layer downsampling; the low-pass images after filtering are respectively: the low-pass image img_fg_lf corresponding to the target image of the 0th layer, the first-layer low-pass image img_fg_l1_lf, the second-layer low-pass image img_fg_l2_lf, the third-layer low-pass image img_fg_l3_lf, the fourth-layer low-pass image img_fg_l4_lf.

[0123] Subtract the target image before filtering from the low-pass image to obtain the high-frequency image, namely: the high-frequency image img_fg_hf of the 0th layer, the high-frequency image img_fg_l1_hf of the first layer, the high-frequency image img_fg_l2_hf of the second layer, the high-frequency image img_fg_l3_hf of the third layer, and the high-frequency image img_fg_l4_hf of the fourth layer. Among them, the low-pass filter can be a Gaussian, guided, bilateral, or other filters.

[0124] As Figure 10 shown, perform image reconstruction operations based on the decomposed multi-layer image information. The steps include: intra-layer reconstruction and inter-layer reconstruction. During the reconstruction process, first perform the reconstruction operation of the intra-layer image information, and then perform the inter-layer image information reconstruction after completion, aiming to fuse the information of each layer to generate an image information that combines the advantages of the background image and the target image. Based on the target image information and the background image information, implement information fusion through intra-layer reconstruction operations, and finally obtain a fused image with excellent performance in both background details and target details.

[0125] For convenience of description, take the 4th layer as an example. Among them, the steps of intra-layer image reconstruction include:

[0126] (a)Calculate the variance map and mean map of the target image and the background image under the window radius r respectively. That is: calculate the variance map img_bg_l4_v and the mean map img_bg_l4_m of the 4th layer background image img_bg_l4 under the window radius r, calculate the variance map img_fg_l4_v and the mean map mg_fg_l4_m of the 4th layer target image img_fg_l4 under the window radius r, and r can be 1, 2 or other values.

[0127] (b)Correct the variance map, and its correction formula is:

[0128] img_bg_l4_vm[i] = { 0 img_bg_l4_m[i] = 0 img_bg_l4_v[i] * (img_fg_l4_v[i] / img_bg_l4_m[i]) 2 else } ,

[0129] (c)Calculate the fusion coefficient; that is:

[0130] sum_v[i] = img_fg_l4_v[i] + img_bg_l4_vm[i] ,

[0131] coef[i] = { 0 sum_v[i] = 0 img_fg_l4_v[i] * coef_set / sum_v[i] else } 。

[0132] (d)Fuse the target image information and the background image information according to the fusion coefficient to obtain the fused image. That is:

[0133] img_fus_l4[i] = img_bg_l4[i] + coef[i] * img_fg_l4_hf[i] ;

[0134] Among them, img_bg_l4_vm[i] is the corrected background variance array, which can better reflect the local detail features of the image. img_bg_l4_v[i] is the background variance array of the current layer, and img_fg_l4_v[i] is the target variance array of the current layer; img_bg_l4_m[i] is the background mean array of the current layer, and img_fg_l4_m[i] is the target mean array of the current layer; sum_v[i] is the fusion array of the background image and the corrected background image of the current layer, and coef_set is the parameter set by the user; the calculation formula "img_fg_l4_v [i] / sum_v [i]" is used to obtain the foreground and background fusion coefficient. In order to flexibly adjust the fusion parameters, thereby affecting the final fusion effect and making it more inclined to the background or foreground, the present invention introduces the user-set parameter coef_set. Usually, if the value of the user-set parameter coef_set is 1.0, it means no adjustment is made, and the coefficient calculated by the original algorithm is directly used for fusion processing. img_fus_l4[i] is the fusion array of the current layer, and img_fg_l4_hf[i] is the background difference array during the decomposition of the current layer.

[0135] In the intra-layer fusion operation of the present invention, compared with the traditional pyramid fusion algorithm, the fusion result cannot distinguish between effective information and invalid information. The present invention determines the fusion ratio (i.e., the fusion coefficient) of the target image at the current position by analyzing the situation of the effective information on the background image. When there is less effective information on the background image, the fusion ratio of the target image is larger, and vice versa. The fusion method of the present invention is simple in calculation, strong in adaptability, and natural in effect.

[0136] As Figure 11 shown, the steps of inter-layer image information reconstruction include:

[0137] (a) Start fusing from the largest scale image layer (i.e., the fourth layer), and obtain the fused image of the current layer through intra-layer image information fusion.

[0138] (b) Combine the fused image of the current layer with the background difference map information for upsampling reconstruction to obtain the reconstructed image. That is: upsample the fused image img_fus_l4 of the fourth layer to the image size of the third layer through 2X2 upsampling. The upsampling method can use bilinear interpolation or other methods to obtain the upsampled image img_fus_l3_up of the third layer; the upsampled image img_fus_l3_up of the third layer is added to the high-frequency image img_bg_l3_hf of the third layer to obtain the reconstructed image img_fus_l3_re of the third layer. In this step, the purpose of fusing the fused image of the fourth layer with the background difference image is to compensate for the detail loss during image decomposition, so that the image retains the original pixels, thereby improving the clarity of the image.

[0139] (c) Use the reconstructed image of the current layer and the background image of the next layer as the input images, and iterate the inter-layer fusion process to obtain the reconstructed image; that is: use the reconstructed image of the third layer img_fus_l3_re and the background image of the third layer img_bg_l3 as the input, repeat the operations in steps (a) - (b) to obtain the fused image of the third layer img_fus_l3; upsample the fused image of the third layer img_fus_l3 to obtain the upsampled image of the third layer img_fus_l2_up, add the high-frequency image of the second layer img_bg_l2_hf to obtain the reconstructed image of the second layer img_fus_l2_re, and so on to obtain the reconstructed image of the 0th layer img_fus_l0_re.

[0140] (d) Use the reconstructed image, the background image, and the high-frequency image of the target image as the input, and iterate the intra-layer image fusion process to obtain the final fused image. That is: use the reconstructed image of the 0th layer img_fus_l0_re, the background image of the 0th layer img_bg, and the high-frequency image of the 0th layer img_fg_hf as the input, and through the intra-layer fusion operation, calculate the final fused image img_fus_out.

[0141] Based on the above high-dynamic-range image dimming method, the present invention also provides a high-dynamic-range image dimming device. The above high-dynamic-range image dimming method is used in the high-dynamic-range image dimming device.

[0142] As Figure 12 shown, a high-dynamic-range image dimming device provided by the present invention includes: a histogram statistics module, a calibration module, and a fusion module. The histogram statistics module is used to obtain the histogram statistics value of the current image. The calibration module is used to traverse the histogram statistics value of the image to obtain the initial platform threshold; based on the initial platform threshold, traverse and modify the histogram statistics value to obtain the optimal platform thresholds of the background image and the target image; based on the optimal platform thresholds, obtain the mapping relationships of the background image and the target image; look up the table with the current image grayscale as the index to obtain the background image and the target image. The fusion module is used to decompose the target image and the background image into image layers of different scales based on the background image and the target image, and start fusing from the image layer with the largest scale in turn to obtain the final fused image.

[0143] To verify the technical solution of the present invention, the present invention uses Figure 13 as the original image for dimming processing. Based on this original image, use the existing technology to dim it and the present invention to perform dimming processing, and compare the obtained pictures for explanation. Among them, the existing technologies for dimming can be platform histogram, histogram equalization HE, adaptive histogram equalization AHE, contrast-limited adaptive histogram equalization (CLAHE), etc. The image effects obtained by the existing dimming methods are as Figure 14as shown Figure 15 is the image effect fused by the traditional pyramid fusion algorithm

[0144] Figure 16 is the background image obtained during the dimming process of the present invention Figure 17 shows the target image obtained by the present invention. From Figure 16 and Figure 17 it can be clearly observed that the two images respectively highlight the corresponding advantageous features Figure 18 shows the effect diagram obtained by the dimming method of the present invention. Comparing the dimming effect of the present invention with the prior art, the dimming effect of the present invention presents a more natural visual effect and is clearer in detail presentation. In addition, the effect diagram shown in FIG. 16 can show that the advantages of the target image and the background image can both be presented in the final effect image, and no detail loss occurs during the image fusion process

[0145] Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all embodiments. Based on the embodiments in the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention

Claims

1. A high dynamic range image dimming method, characterized in that: The specific steps include: Get the histogram statistics of the current image; Traverse the image histogram statistics to obtain the initial platform threshold; based on the initial platform threshold, traverse and modify the histogram statistics to obtain the platform optimal threshold of the background image and the target image; Based on the platform's optimal threshold, the mapping relationship of the background image and the mapping relationship of the target image are obtained; the background image and the target image are obtained by looking up the table with the current image grayscale as the index; Based on the background image and the target image, the pyramid fusion algorithm is used to decompose the target image and the background image into image layers of different scales, and then they are fused in sequence from the image layer with the largest scale to obtain the final fused image.

2. The high dynamic range image dimming method according to claim 1, characterized in that: When traversing the image histogram statistics, start from the statistical value of 0 and go up, traverse the grayscale of each statistical value, and count the number of grayscales that are not 0; When the accumulated statistical value is greater than the set threshold, the process stops and the current statistical value is used as the initial platform threshold.

3. The high dynamic range image dimming method according to claim 1, characterized in that: The platform optimal threshold acquisition step includes: Traverse the histogram statistical value starting from grayscale 0. If the histogram statistical value is greater than the initial platform threshold, modify the current statistical value to the initial platform threshold. If the histogram statistical value is less than or equal to the initial platform threshold, keep the original value unchanged to obtain the first statistical value. Traverse the first statistical value and calculate the cumulative sum of the first statistical value; The first statistical value cumulative sum is optimized using a preset optimization coefficient to obtain a first platform threshold; Traverse the first statistical value of the histogram starting from grayscale 0. If the first statistical value is greater than the first platform threshold, modify the current statistical value to the first platform threshold. If the first statistical value is less than or equal to the first platform threshold, keep the original value unchanged to obtain the second statistical value. The second statistical values ​​are traversed, and the cumulative sum of the second statistical values ​​is calculated to obtain the optimal threshold of the platform.

4. The high dynamic range image dimming method according to claim 1, characterized in that: The step of decomposing the target image and the background image by the pyramid fusion algorithm comprises: Down-sample the target image and the background image respectively to obtain down-sampled image information; Up-sampling the down-sampled background image to obtain a low-frequency image of the first layer background image; Taking the current background image information and the downsampled target image information as the input images, the image downsampling and background image upsampling operations are iterated one level up to obtain multiple layers of image information of different scales.

5. The high dynamic range image dimming method according to claim 4, characterized in that: After obtaining the background image information of each layer, it is subtracted from the background image before sampling to obtain the background difference map information.

6. The high dynamic range image dimming method according to claim 4, characterized in that: A low-pass filter is used to filter the target image information in each layer of image information to remove the low-pass image in the target image; the target image before filtering is subtracted from the low-pass image to obtain a high-frequency image.

7. The high dynamic range image dimming method according to claim 1, characterized in that: When the image information within a layer is fused, the steps include: Calculate the variance map and mean map of the target image and background image under the window radius r respectively; Correct the variance map; Calculate the fusion coefficient; The target image information and the background image information are fused according to the fusion coefficient to obtain a fused image.

8. The high dynamic range image dimming method according to claim 7, characterized in that: The correction formula of the variance map is: ， The calculation formula of the fusion coefficient coef[i] is: ， ， The formula for fusing the target image and the background image is: ； Among them, img_bg_l4_vm[i] is the corrected background variance array, img_bg_l4_v[i] is the background variance array of the current layer, and img_fg_l4_v[i] is the target variance array of the current layer; img_bg_l4_m[i] is the background mean array of the current layer, and img_fg_l4_m[i] is the target mean array of the current layer; sum_v[i] is the fusion array of the current layer background image and the corrected background image, and coef_set is the user-set parameter; img_fus_l4[i] is the current layer fusion array, and img_fg_l4_hf[i] is the background difference array when the current layer is decomposed.

9. The high dynamic range image dimming method according to claim 1, characterized in that: When inter-layer image information is fused, the steps include: Start fusion from the largest scale image layer, and obtain the fused image of the current layer through the image information fusion operation within the layer; The fused image of the current layer is combined with the background difference image information for upsampling and reconstruction to obtain a reconstructed image; The reconstructed image of the current layer and the background image of the next layer are used as input images, and the inter-layer fusion process is iterated to obtain the reconstructed image; Taking the reconstructed image, the background image and the high-frequency image of the target image as input, the intra-layer image fusion process is iterated to obtain the final fused image.

10. A high dynamic range image dimming device, characterized in that: include: Histogram statistics module, obtains the histogram statistics of the current image; The calibration module traverses the image histogram statistics to obtain the initial platform threshold; Based on the initial platform threshold, the histogram statistics are traversed and modified to obtain the platform optimal threshold of the background image and the target image; based on the platform optimal threshold, the mapping relationship of the background image and the mapping relationship of the target image are obtained; the current image grayscale is used as an index to look up the table to obtain the background image and the target image; The fusion module, based on the background image and the target image, uses a pyramid fusion algorithm to decompose the target image and the background image into image layers of different scales, and fuses them sequentially starting from the image layer of the largest scale to obtain the final fused image.

Images