A high dynamic range image dimming method and device

Abstract

The present invention provides a high dynamic range image dimming method and device, which includes: obtaining histogram statistics of the current image; traversing the image histogram statistics to obtain an initial platform threshold; traversing and modifying the histogram statistics based on the initial platform threshold to obtain the platform optimal threshold of the background image and the target image; using the platform optimal threshold, the image quality and overall brightness are guaranteed, solving the problem of scene adaptation. Based on the platform optimal threshold, the mapping relationship of the background image and the mapping relationship of the target image are obtained; using the current image grayscale as an index to look up the table to obtain the background image and the target image; based on the background image and the target image, using a pyramid fusion algorithm to decompose the target image and the background image into image layers of different scales, and fusing them sequentially starting from the largest scale image layer to obtain a fused image that takes into account both the background and the target, thereby solving the problem of detail loss.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a method and device for dimming an image with a high dynamic range. Background Art

[0002] The output image of the infrared detector is usually 14-bit or 16-bit, and in general scenes, the effective grayscale only occupies a small part of the total grayscale range. The display device can generally only display 8-bit or 10-bit images. Therefore, it is necessary to analyze the effective grayscale from the image and bring it up to the display grayscale range. This process is called dimming and is a dynamic range compression algorithm.

[0003] Commonly used dynamic range compression algorithms mainly include the following aspects:

[0004] A: Information statistics. Perform histogram statistics on the input image, starting from the first grayscale and accumulating the number of pixels at the current grayscale. When the accumulated sum is greater than or equal to the preset THR1, the grayscale at that time is the starting point G1 of the valid grayscale. When the accumulated sum is greater than or equal to the preset THR2, the current grayscale is the ending point G2 of the valid grayscale.

[0005] B: Mapping relationship calculation. There are two common methods for mapping relationship calculation: linear mapping and nonlinear mapping. Linear mapping maps G1 to the starting point of the target grayscale range, G2 to the end point of the target grayscale, and the intermediate grayscales are obtained by linear interpolation. Nonlinear mapping usually uses histogram equalization or its improved algorithm. The basic principle is to perform grayscale mapping based on the probability distribution of image grayscale levels. The contrast of grayscales with a large proportion is stretched, while the contrast of grayscales with a small proportion is compressed.

[0006] Conventional dynamic range compression algorithms have the following defects:

[0007] A: Background or object details are severely lost. Linear mapping results in severe image detail loss and opacity when the dynamic range is large or when a large proportion of ineffective grayscale is used. Nonlinear mapping can lead to severe detail loss when the object is small and occupies a small grayscale. When the background is too large, problems such as overstretching, grayscale discontinuity, and unnatural brightness can occur.

[0008] B: Scene adaptability. The optimal threshold for linear mapping and the platform histogram threshold for nonlinear mapping vary with the scene. Using a unified threshold can lead to algorithms performing well in some scenarios but poorly in others, or even causing image anomalies. Summary of the Invention

[0009] In order to overcome the shortcomings of the existing technology, the present invention provides a high dynamic range image dimming method and device. The method processes the original image to obtain two sub-images: one sub-image focuses on retaining background details, and the other sub-image focuses on target details. The two sub-images are fused to generate an image that retains the background and highlights the target, thereby effectively solving the problems of image detail loss and scene adaptation.

[0010] To achieve the above-mentioned object, a high dynamic range image dimming method of the present invention comprises the following steps:

[0011] Get the histogram statistics of the current image;

[0012] Traverse the image histogram statistics to obtain the initial platform threshold; based on the initial platform threshold, traverse and modify the histogram statistics to obtain the platform optimal threshold of the background image and the target image;

[0013] Based on the platform's optimal threshold, the mapping relationship of the background image and the mapping relationship of the target image are obtained; the background image and the target image are obtained by looking up the table with the current image grayscale as the index;

[0014] Based on the background image and the target image, the pyramid fusion algorithm is used to decompose the target image and the background image into image layers of different scales, and then fuse them in sequence starting from the image layer with the largest scale to obtain the final fused image.

[0015] Preferably, when traversing the image histogram statistical values, start from the statistical value of 0 and go up, traverse the grayscale of each statistical value, and count the number of grayscales that are not 0; stop when the cumulative sum of the statistical values ​​is greater than the set threshold, and use the current statistical value as the initial platform threshold.

[0016] Preferably, the platform optimal threshold acquisition step includes:

[0017] Starting from grayscale 0, the histogram statistics are traversed. If the histogram statistics is greater than the initial platform threshold, the current statistics are modified to the initial platform threshold. If the histogram statistics is less than or equal to the initial platform threshold, the original value is kept unchanged to obtain the first statistics.

[0018] Traverse the first statistical value and calculate the cumulative sum of the first statistical value;

[0019] Optimizing the cumulative sum of the first statistical values ​​using a preset optimization coefficient to obtain a first platform threshold;

[0020] Starting from grayscale 0, traverse the first statistical value of the histogram. If the first statistical value is greater than the first platform threshold, modify the current statistical value to the first platform threshold. If the first statistical value is less than or equal to the first platform threshold, keep the original value unchanged to obtain the second statistical value.

[0021] The second statistical values ​​are traversed, and the cumulative sum of the second statistical values ​​is calculated to obtain the optimal threshold of the platform.

[0022] Preferably, the step of decomposing the target image and the background image using the pyramid fusion algorithm includes:

[0023] Down-sample the target image and the background image respectively to obtain down-sampled image information;

[0024] Upsample the downsampled background image to obtain the low-frequency image of the first layer background image;

[0025] Taking the current background image information and the downsampled target image information as the input images, the image downsampling and background image upsampling operations are iterated one level up to obtain multiple layers of image information of different scales.

[0026] Preferably, after obtaining the background image information of each layer, a difference process is performed between the background image and the background image before sampling to obtain background difference image information.

[0027] Preferably, a low-pass filter is used to filter the target image information in each layer of image information to remove the low-pass image in the target image; and the target image before filtering is subtracted from the low-pass image to obtain a high-frequency image.

[0028] Preferably, when the image information within a layer is fused, the steps include:

[0029] Calculate the variance map and mean map of the target image and background image under the window radius r respectively;

[0030] Correct the variance map;

[0031] Calculate the fusion coefficient;

[0032] The target image information and the background image information are fused according to the fusion coefficient to obtain a fused image.

[0033] Preferably, the correction formula of the variance map is:

[0034] img_bg_l4_vm[i] = { 0 img_bg_l4_m[i] = 0 img_bg_l4_v[i]*(img_fg_l4_v[i] / img_bg_l4_m[i]) 2 else } ,

[0035] The calculation formula of the fusion coefficient coef[i] is:

[0036] sum_v[i] = img_fg_l4_v[i] + img_bg_l4_vm[i] ,

[0037] coef[i] = { 0 sum_v[i] = 0 img_fg_l4_v[i]*coef_set / sum_v[i] else } ,

[0038] The formula for fusing the target image and the background image is:

[0039] img_fus_l4[i] = img_bg_l4[i] + coef[i]*img_fg_l4_hf[i] ;

[0040] Among them, img_bg_l4_vm[i] is the corrected background variance array, img_bg_l4_v[i] is the background variance array of the current layer, and img_fg_l4_v[i] is the target variance array of the current layer; img_bg_l4_m[i] is the background mean array of the current layer, and img_fg_l4_m[i] is the target mean array of the current layer; sum_v[i] is the fusion array of the current layer background image and the corrected background image, and coef_set is the user-set parameter; img_fus_l4[i] is the current layer fusion array, and img_fg_l4_hf[i] is the background difference array when the current layer is decomposed.

[0041] Preferably, when inter-layer image information is fused, the steps include:

[0042] Start fusion from the largest scale image layer and obtain the fused image of the current layer through the image information fusion operation within the layer;

[0043] Combine the fused image of the current layer with the background difference image information for upsampling and reconstruction to obtain a reconstructed image;

[0044] The reconstructed image of the current layer and the background image of the next layer are used as input images, and the inter-layer fusion process is iterated to obtain the reconstructed image;

[0045] Taking the reconstructed image, background image and high-frequency image of the target image as input, the intra-layer image fusion process is iterated to obtain the final fused image.

[0046] The present invention also provides a high dynamic range image dimming device, comprising:

[0047] Histogram statistics module, obtains the histogram statistics of the current image;

[0048] The calibration module traverses the image histogram statistics to obtain the initial platform threshold; based on the initial platform threshold, the histogram statistics are traversed and modified to obtain the platform optimal threshold of the background image and the target image; based on the platform optimal threshold, the mapping relationship of the background image and the mapping relationship of the target image are obtained; the current image grayscale is used as the index to look up the table to obtain the background image and the target image;

[0049] The fusion module uses a pyramid fusion algorithm to decompose the target image and the background image into image layers of different scales based on the background image and the target image, and fuses them sequentially starting from the image layer with the largest scale to obtain the final fused image.

[0050] The present invention provides a high dynamic range image dimming method and device, which, compared with the prior art, has the following advantages:

[0051] 1. The present invention forms an adaptive platform optimal threshold by repeatedly verifying and comparing image histogram statistics with platform preset values. This optimization process not only effectively guarantees the quality of the background image but also ensures that the overall background brightness is within an appropriate range. Furthermore, the histogram verification method proposed in this invention effectively prevents excessive background stretching. In image processing, excessive background stretching can cause abnormal image brightness and lead to brightness stratification. By avoiding excessive background stretching, this method fundamentally avoids the image stratification problem caused by this problem.

[0052] 2. The present invention divides the original image into a background image and a target image based on the statistical value of the image histogram, and performs platform optimal threshold calculation and verification respectively. Through this processing, the present method can generate images that focus on the background and the target respectively, thereby presenting the background and the target in their entirety. On the basis of obtaining the background image and the target image, by fusing the two, it can be ensured that the details of the background and target images are completely preserved. At the same time, the fused image can take into account the advantages of both the background and the target, and show their respective advantages. Compared with the existing technology, the present method can perfectly be compatible with backgrounds and targets with a good dynamic range, and has strong adaptability to various scenarios.

[0053] 3. Pyramid fusion algorithms in existing technologies have limitations. Their fusion results cannot effectively distinguish valid from invalid information, resulting in significant deviations from the intended visual presentation after fusion. During the image fusion process, the present invention applies a pyramid fusion algorithm to perform multi-level decomposition of the background and target images, generating image layers of varying sizes (i.e., scales). These image layers are stacked sequentially in a pyramid structure, exhibiting a characteristic of gradually increasing size from bottom to top. At each level of image decomposition, the background and target information contained in the current layer is fully captured, specifically encompassing both intra-layer image fusion and inter-layer image fusion. Specifically, the background and target information of each layer is first finely reconstructed, followed by inter-layer image reconstruction. After these multi-layer image reconstruction operations, the resulting background and target information is further subjected to comprehensive image reconstruction to produce a high-quality final fused image. Therefore, the present invention dynamically determines the integration ratio of the target image at the current location by deeply analyzing the distribution of valid information within the background image. When the background image has less effective information, the target image's integration ratio increases accordingly; conversely, when the background image has more effective information, the target image's integration ratio decreases accordingly. Compared to existing mature fusion algorithms, the fusion method used in this invention has significant advantages, such as a simple calculation process, strong algorithm adaptability, and natural and realistic fusion effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 This is a flow chart of a high dynamic range image dimming method provided by the present invention.

[0055] Figure 2 A flowchart of calculating a background image in a high dynamic range image dimming method provided by the present invention;

[0056] Figure 3 This is a schematic diagram of calculating the initial platform threshold thr_p1 value of the target image histogram in a high dynamic range image dimming method provided by the present invention.

[0057] Figure 4 This is a flow chart for calculating a target image in a high dynamic range image dimming method provided by the present invention.

[0058] Figure 5 This is a schematic diagram of calculating the initial platform threshold thr_mm of the background image histogram in a high dynamic range image dimming method provided by the present invention.

[0059] Figure 6 This is a flow chart of image fusion in a high dynamic range image dimming method provided by the present invention.

[0060] Figure 7 This is a flowchart of pyramid decomposition used in image fusion in a high dynamic range image dimming method provided by the present invention.

[0061] Figure 8 This is a flow chart of downsampling a background image during image decomposition in a high dynamic range image dimming method provided by the present invention.

[0062] Figure 9 This is a flow chart of downsampling a target image during image decomposition in a high dynamic range image dimming method provided by the present invention.

[0063] Figure 10 This is a flow chart of image reconstruction in a high dynamic range image dimming method provided by the present invention.

[0064] Figure 11 This is a flow chart of upsampling during image reconstruction in a high dynamic range image dimming method provided by the present invention.

[0065] Figure 12 This is a schematic diagram of a high dynamic range image dimming device provided by the present invention.

[0066] Figure 13 This is the original image for dimming.

[0067] Figure 14 This is the image effect obtained by the dimming method in the prior art.

[0068] Figure 15 This is the fusion effect diagram of the pyramid fusion algorithm in the existing technology.

[0069] Figure 16 This is a background image obtained by the high dynamic range image dimming method provided by the present invention.

[0070] Figure 17 This is a target image obtained by the high dynamic range image dimming method provided by the present invention.

[0071] Figure 18 This is a diagram showing the dimming effect of a high dynamic range image dimming method provided by the present invention. DETAILED DESCRIPTION

[0072] The following describes the embodiments of the present invention through specific examples. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0073] like Figure 1 As shown, the present invention provides a high dynamic range image dimming method, the specific steps of which include:

[0074] Get the histogram statistics of the current image;

[0075] Traverse the image histogram statistics to obtain the initial platform threshold; based on the initial platform threshold, traverse and modify the histogram statistics to obtain the platform optimal threshold of the background image and the target image;

[0076] Based on the platform's optimal threshold, the mapping relationship of the background image and the mapping relationship of the target image are obtained; the background image and the target image are obtained by looking up the table with the current image grayscale as the index;

[0077] Based on the background image and the target image, the pyramid fusion algorithm is used to decompose the target image and the background image into image layers of different scales, and then fuse them in sequence starting from the image layer with the largest scale to obtain the final fused image.

[0078] Specifically, the present invention forms an adaptive platform optimal threshold by performing multiple checks and comparisons on the image histogram statistics and the platform preset values, thereby ensuring that the quality and overall brightness of the background image are within an appropriate range and solving the image stratification problem. Based on the image histogram statistics, the original image is divided into a background image and a target image, and the platform optimal threshold is calculated and checked separately. Through this process, the method can generate images that focus on the background and the target respectively, thereby presenting the background and the target completely. Based on the obtained image, a pyramid fusion algorithm is used to perform multi-level decomposition processing on the background image and the target image to generate image layers of different sizes with background information and target information. Through the fusion processing of the image within the layer and the image fusion operation between layers, a fused image that can present the advantages of the target and background is finally obtained. Among them, the platform optimal threshold refers to a dimming threshold that can adapt to the current application scene environment.

[0079] The way to obtain histogram statistics is to count the number of pixels of each grayscale value in the image. The general platform histogram threshold is set by the user, but its adaptability is poor in different application scenarios. For example, when the background accounts for a large proportion, the image is prone to blurring, and the dark areas are brighter, such as Figure 14 In view of this, the present invention calculates the platform optimal threshold value based on the histogram statistics, and then derives the appropriate histogram mapping relationship, so as to ensure that the background brightness is appropriate and the contrast is good.

[0080] Based on histogram statistics, an adaptive platform-optimal threshold is used to obtain a background image that favors background details and a target image that favors target details. It should be noted that in the present invention, the platform-optimal threshold is obtained in the same manner for both the background image and the target image. To provide a clearer and more detailed explanation of the methods for obtaining these two images, each is described separately.

[0081] like Figure 2 As shown, for the processing of background image:

[0082] (a) Start from the statistical value of 0 and traverse upwards. At each statistical value, the grayscale is traversed from left to right to count the number of grayscales that are not 0, such as Figure 3 As shown, the process stops when the accumulated sum exceeds the set plateau threshold thr_h1, and the current statistical value serves as the initial plateau threshold thr_p1. In other words, searching from bottom to top, the r-axis value where the sum of the blue area between the red line and 0 exceeds the condition is the required initial plateau threshold thr_p1. This step can be used to find the appropriate histogram plateau threshold for the current image, adapting to the current image background brightness.

[0083] (b) Traverse the histogram statistics starting from grayscale 0. When the histogram statistics value is greater than thr_p1, the statistics value is modified to thr_p1, otherwise the original value remains unchanged to obtain a new statistics value hist_mb; that is: traverse the histogram statistics value starting from grayscale 0. If the histogram statistics value is greater than the initial platform threshold thr_p1, the current statistics value is modified to the initial platform threshold thr_p1. If the histogram statistics value is less than or equal to the initial platform threshold thr_p1, the original value remains unchanged to obtain the first statistics value hist_mb. By further correcting the initial platform threshold, the contrast adaptability of the background image is better, avoiding blurring or excessive contrast. Its expression is:

[0084] hist _ m b [ i ] = { hist [ i ] hist [ i ] <= thr_p1 thr_p1 hist [ i ] > thr_p1 } , where hist[i] is the current statistical value.

[0085] (c) Traverse the first statistical value hist_mb and calculate the cumulative sum of the first statistical value sum_all_mb, which is expressed as:

[0086] sum_all_mb = ∑ hist_mb[i] .

[0087] (d) Optimize the first statistical value accumulation sum sum_all_mb using the preset optimization coefficient coef_mb to obtain the first platform threshold thr_mb1, that is:

[0088] thr_mb1 = sum_all_mb * coef_mb.

[0089] (e) Starting from grayscale 0, traverse the first statistical value hist_mb of the histogram. If the first statistical value hist_mb is greater than the first platform threshold thr_mb1, the current statistical value is modified to the first platform threshold thr_mb1. If the first statistical value is less than or equal to the first platform threshold, the original value is kept unchanged to obtain the second statistical value hist_bg. That is:

[0090] hist _ bg [ i ] = { hist _ mb [ i ] hist _ mb [ i ] <= thr_mb1 thr_mb1 hist _ mb [ i ] > thr_mb1 } .

[0091] (f) Traverse the second statistical value hist_bg, calculate the cumulative sum of the second statistical value sum_all_bg, and obtain the optimal threshold of the platform. That is:

[0092] sum_all_bg = ∑ hist_bg[i] .

[0093] Through steps (b) to (f), the platform threshold is automatically adjusted to adapt to the current background image, so that the background image can show its details well.

[0094] (g) Calculate the mapping relationship of the background image, that is:

[0095] tab _ bg [ i ] = hist _ bg [ i ]* 255 / sum _ all _ bg .

[0096] (h) Use the image grayscale as the index to look up the table tab_bg and obtain the background image img_bg, that is:

[0097] img_bg[i]=tab_bg[img_in[i]], where img_in is the input image.

[0098] like Figure 4 As shown, the calculation steps of the target image include:

[0099] (a) Start traversing the histogram statistics from grayscale 0. If the histogram statistics is greater than the initial platform threshold thr_m1, modify the current statistics to the initial platform threshold thr_m1. If the histogram statistics is less than or equal to the initial platform threshold thr_m1, keep the original value unchanged to obtain the first statistics hist_mm. That is:

[0100] hist _ mm [ i ] = { hist [ i ] / thr_m1 hist [ i ] <= thr_m1 thr_m1 / thr_m1 hist [ i ] > thr_m1 } .

[0101] (b) Traverse the first statistical value hist_mm and calculate the cumulative sum of the first statistical value sum_all_mm. That is:

[0102] sum_all_mm = ∑ hist_mm[i] .

[0103] (c) Optimize the first statistical value accumulation sum_all_mm using the preset optimization coefficient coef_mm to obtain the first platform threshold thr_mm. That is:

[0104] thr_mm = w*h*coef_mm / sum_all_mm,

[0105] Where w and h are the width and height of the image respectively. Figure 5 As shown, the product of the image width and height divided by the sum of the areas of the blue part from the red line to the 0 line is the first platform threshold thr_mm.

[0106] (d) Starting from grayscale 0, traverse the first statistical value hist_mm of the histogram. If the first statistical value hist_mm is greater than the first platform threshold thr_mm, modify the current statistical value to the first platform threshold thr_mm. If the first statistical value hist_mm is less than or equal to the first platform threshold thr_mm, keep the original value unchanged to obtain the second statistical value hist_fg;

[0107] hist _ fg [ i ] = { hist _ mm [ i ] hist _ mm [ i ] <= thr_mm thr_mm hist _ mm [ i ] > thr_mm } .

[0108] (e) Traverse the second statistical value hist_fg, calculate the cumulative sum of the second statistical values ​​sum_all_fg, and obtain the platform optimal threshold.

[0109] sum_all_fg = ∑ hist_fg[i] .

[0110] (f) Calculate the mapping relationship of the target image. That is:

[0111] tab _ f g [ i ] = hist _ f g [ i ]* 255 / sum _ all _ f g .

[0112] (g) Use the image grayscale as the index to look up the table tab_fg and obtain the target image img_fg.

[0113] img_fg[i]=tab_fg[img_in[i]], where img_in is the input image.

[0114] like Figures 6-9As shown, a pyramid fusion algorithm is used to fuse the background and target images, obtained based on histogram statistics. Given the overall brightness characteristics of the background and target images, the present invention uses the background image as a base layer and then superimposes the target image onto the background image. If the target image were used as the base layer, the fused image would appear dark overall, failing to meet the practical needs of the relevant application scenario. Therefore, using the target image as the base layer is not adopted.

[0115] Based on the two acquired images, a pyramid fusion algorithm is used to decompose the images into four images of different scales: the first layer l1, the second layer l2, the third layer l3, and the fourth layer l4. The fusion process begins with the image with the largest scale. After fusion is complete, it is restored to the previous level for reconstruction. The fusion operation is then repeated, and the image is restored to the previous level for reconstruction. This cycle continues until the image is restored to its original size. Taking the first layer image as an example, the specific steps include:

[0116] (a) Perform 2x2 downsampling on the target image img_fg and the background image img_bg to obtain downsampled image information. Specifically, sample the first-layer background image img_bg_l1 and the first-layer target image img_fg_l1. The sampling method is preferably mean or other methods.

[0117] (b) Upsample the downsampled background image img_bg_l1 of the first layer to obtain the low-frequency image img_bg_l1_sm of the first layer background image. The upsampling method can use bilinear or other methods.

[0118] After obtaining each layer's background image information, it is subtracted from the pre-sampling background image to obtain a background difference map. Given the inevitable loss of detail during the decomposition process, obtaining this background difference map accurately records this loss, providing a basis for effectively compensating for this loss during the subsequent reconstruction process. The layer 0 background difference map img_bg_l0_hf = layer 0 background image img_bg – layer 1 background image low-frequency map img_bg_l1_sm.

[0119] (c) Using the current background image img_bg_l1 (the first-layer background image) and the downsampled target image img_fg_l1 as input, iteratively downsamples the image and upsamples the background image one level higher to obtain multiple layers of image information at different scales. Specifically, using the first-layer background image img_bg_l1 and the first-layer downsampled target image img_fg_l1 as input, the next-layer decomposition yields the second-layer background image img_bg_l2, the second-layer downsampled target image img_fg_l2, and the first-layer background difference image img_bg_l1_hf. Similarly, the third and fourth layer decomposition information are obtained respectively, namely: the third layer background image img_bg_l3, the third layer downsampled target image img_fg_l3, the second layer background difference image img_bg_l2_hf, the fourth layer background image img_bg_l4, the fourth layer downsampled target image img_fg_l4 and the third layer background difference image img_bg_l3_hf.

[0120] In the decomposition process of the present invention, steps (a) and (b) are the decomposition process of one layer. After the first layer decomposition is completed, the first layer background image information img_bg_l1 and the first layer downsampled target image information img_fg_l1 are used as input to perform the next layer decomposition, that is, the process of steps (a) and (b) is repeated again, and the decomposition information of the third and fourth layers is obtained by analogy. Figure 8 The specific number of layers in the present invention is only an example and is not limited in the present invention.

[0121] (d) A low-pass filter is used to filter the target image information in each layer of image information to remove the low-pass image in the target image; the target image before filtering is subtracted from the low-pass image to obtain a high-frequency image.

[0122] like Figure 9 As shown, the target image information of each layer is: the target image img_fg of the 0th layer, the target image img_fg_l1 after downsampling of the first layer, the target image img_fg_l2 after downsampling of the second layer, the target image img_fg_l3 after downsampling of the third layer, and the target image img_fg_l4 after downsampling of the fourth layer; the low-pass images after filtering are: the low-pass image img_fg_lf corresponding to the target image of the 0th layer, the low-pass image img_fg_l1_lf of the first layer, the low-pass image img_fg_l2_lf of the second layer, the low-pass image img_fg_l3_lf of the third layer, and the low-pass image img_fg_l4_lf of the fourth layer.

[0123] Subtract the unfiltered target image from the low-pass image to obtain high-frequency images: img_fg_hf, the first high-frequency image img_fg_l1_hf, the second high-frequency image img_fg_l2_hf, the third high-frequency image img_fg_l3_hf, and the fourth high-frequency image img_fg_l4_hf. The low-pass filter can be a Gaussian, guided, or bilateral filter.

[0124] like Figure 10 As shown in the figure, image reconstruction is performed based on the decomposed multi-layer image information. This involves intra-layer reconstruction and inter-layer reconstruction. During the reconstruction process, intra-layer image reconstruction is performed first, followed by inter-layer reconstruction. The goal is to fuse the information from each layer to generate an image that combines the advantages of both the background and target images. Based on the target and background image information, information fusion is performed using intra-layer reconstruction, ultimately resulting in a fused image that exhibits excellent background and target details.

[0125] For ease of description, the fourth layer is used as an example. The steps of image reconstruction within the layer include:

[0126] (a) Compute the variance map and mean map of the target image and background image, respectively, within a window radius r. That is, compute the variance map img_bg_l4_v and mean map img_bg_l4_m of the fourth-layer background image img_bg_l4, and compute the variance map img_fg_l4_v and mean map mg_fg_l4_m of the fourth-layer target image img_fg_l4, within a window radius r. r can be 1, 2, or any other value.

[0127] (b) Correct the variance map. The correction formula is:

[0128] img_bg_l4_vm[i] = { 0 img_bg_l4_m[i] = 0 img_bg_l4_v[i]*(img_fg_l4_v[i] / img_bg_l4_m[i]) 2 else } ,

[0129] (c) Calculate the fusion coefficient; that is:

[0130] sum_v[i] = img_fg_l4_v[i] + img_bg_l4_vm[i] ,

[0131] coef[i] = { 0 sum_v[i] = 0 img_fg_l4_v[i]*coef_set / sum_v[i] else } .

[0132] (d) Fuse the target image information and the background image information according to the fusion coefficient to obtain a fused image. That is:

[0133] img_fus_l4[i] = img_bg_l4[i] + coef[i]*img_fg_l4_hf[i] ;

[0134] Among them, img_bg_l4_vm[i] is the corrected background variance array, which makes it more able to reflect the local detail features of the image, img_bg_l4_v[i] is the background variance array of the current layer, and img_fg_l4_v[i] is the target variance array of the current layer; img_bg_l4_m[i] is the background mean array of the current layer, and img_fg_l4_m[i] is the target mean array of the current layer; sum_v[i] is the fusion array of the current layer background image and the corrected background image, and coef_set is a user-set parameter; the calculation formula "img_fg_l4_v [i] / sum_v [i]" is used to obtain the front and back background fusion coefficients. In order to be able to flexibly adjust the fusion parameters and thus affect the final fusion effect, making it more inclined to the background or foreground, the present invention introduces the user-set parameter coef_set. Under normal circumstances, if the user-set parameter coef_set is set to 1.0, it means that no adjustment is made, and the coefficients calculated by the original algorithm are directly used for fusion processing. img_fus_l4[i] is the current layer fusion array, and img_fg_l4_hf[i] is the background difference array when the current layer is decomposed.

[0135] Compared to traditional pyramid fusion algorithms, which fail to distinguish valid from invalid information in their fusion results, the present invention's intra-layer fusion operation analyzes the valid information in the background image to determine the integration ratio (i.e., the fusion coefficient) of the target image at the current location. The less valid information in the background image, the greater the integration ratio of the target image, and vice versa. This fusion method is computationally simple, highly adaptable, and produces natural results.

[0136] like Figure 11 As shown, the steps of reconstructing inter-layer image information include:

[0137] (a) Fusion starts from the largest scale image layer (i.e., the fourth layer), and the fused image of the current layer is obtained by fusing the image information within the layer.

[0138] (b) The fused image of the current layer is combined with the background difference image information for upsampling and reconstruction to obtain a reconstructed image. Specifically, the fused image img_fus_l4 of the fourth layer is upsampled 2x2 to the size of the third layer image. The upsampling method can use bilinear interpolation or other methods to obtain the third layer upsampled image img_fus_l3_up. The third layer upsampled image img_fus_l3_up is added to the third layer high-frequency image img_bg_l3_hf to obtain the third layer reconstructed image img_fus_l3_re. In this step, the purpose of fusing the fused image of the fourth layer with the background difference image is to compensate for the detail loss during image decomposition, so that the image retains the original pixels and thus improves image clarity.

[0139] (c) Take the reconstructed image of the current layer and the background image of the next layer as input images, iterate the inter-layer fusion process, and obtain the reconstructed image; that is: take the third-layer reconstructed image img_fus_l3_re and the third-layer background image img_bg_l3 as input, repeat steps (a) to (b) to obtain the third-layer fused image img_fus_l3; upsample the third-layer fused image img_fus_l3 to obtain the third-layer upsampling image img_fus_l2_up, and add the second-layer high-frequency image img_bg_l2_hf to obtain the second-layer reconstructed image img_fus_l2_re, and so on to obtain the 0th-layer reconstructed image img_fus_l0_re.

[0140] (d) Taking the reconstructed image, background image, and high-frequency image of the target image as input, the intra-layer image fusion process is iterated to obtain the final fused image. That is, taking the 0th layer reconstructed image img_fus_l0_re, the 0th layer background image img_bg, and the 0th layer high-frequency image img_fg_hf as input, the final fused image img_fus_out is calculated through the intra-layer fusion operation.

[0141] Based on the above-mentioned high dynamic range image dimming method, the present invention further provides a high dynamic range image dimming device. The above-mentioned high dynamic range image dimming method is used in the high dynamic range image dimming device.

[0142] like Figure 12 As shown, the present invention provides a high dynamic range image dimming device, which includes: a histogram statistics module, a calibration module and a fusion module. The histogram statistics module is used to obtain the histogram statistics of the current image. The calibration module is used to traverse the image histogram statistics to obtain the initial platform threshold; based on the initial platform threshold, the histogram statistics are traversed and modified to obtain the platform optimal threshold of the background image and the target image; based on the platform optimal threshold, the mapping relationship of the background image and the mapping relationship of the target image are obtained; the grayscale of the current image is used as an index to look up the table to obtain the background image and the target image. The fusion module is used to decompose the target image and the background image into image layers of different scales based on the background image and the target image using a pyramid fusion algorithm, and fuse them in sequence starting from the image layer with the largest scale to obtain the final fused image.

[0143] In order to verify the technical solution of the present invention, the present invention adopts Figure 13 As the original image for dimming processing, based on the original image, the existing technology is used to dim it and the dimming processing of the present invention is performed, and the obtained images are compared and explained. Among them, the existing technology for dimming can be platform histogram, histogram equalization HE, adaptive histogram equalization AHE, limited contrast adaptive histogram equalization (CLAHE), etc. The image effect obtained by the existing dimming method is as follows Figure 14shown. Figure 15 This is the image effect of the traditional pyramid fusion algorithm.

[0144] Figure 16 is the background image obtained during the dimming process of the present invention, Figure 17 What is presented is the target image obtained by the present invention. Figure 16 and Figure 17 It can be clearly observed that the two images each highlight the corresponding advantageous features. Figure 18 This figure illustrates the effects achieved by the dimming method of the present invention. Comparing the dimming effects of the present invention with those of the prior art, the present invention exhibits a more natural visual effect and clearer detail. Furthermore, the image shown in Figure 16 demonstrates that the advantages of both the target image and the background image are fully realized in the final image, with no detail lost during the image fusion process.

[0145] Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

Claims

1. A high dynamic range image dimming method, characterized in that: The specific steps include: Obtain the histogram statistics of the current image; wherein the method of obtaining the histogram statistics is to count the number of pixels of each gray value on the image; Traverse the image histogram statistics to obtain the initial platform threshold; based on the initial platform threshold, traverse and modify the histogram statistics to obtain the platform optimal threshold of the background image and the target image; The initial platform threshold is obtained by traversing the image histogram statistics, starting from the statistical value of 0 and going upwards, traversing the grayscale of each statistical value, and counting the number of grayscales that are not 0; when the cumulative sum of the statistical values ​​is greater than the set threshold, stop and use the current statistical value as the initial platform threshold; When obtaining the platform optimal threshold value of the background image and the target image, the steps include: Starting from grayscale 0, the histogram statistics are traversed. If the histogram statistics is greater than the initial platform threshold, the current statistics are modified to the initial platform threshold. If the histogram statistics is less than or equal to the initial platform threshold, the original value is kept unchanged to obtain the first statistics. Traverse the first statistical value and calculate the cumulative sum of the first statistical value; Optimizing the cumulative sum of the first statistical values ​​using a preset optimization coefficient to obtain a first platform threshold; Starting from grayscale 0, traverse the first statistical value of the histogram. If the first statistical value is greater than the first platform threshold, modify the current statistical value to the first platform threshold. If the first statistical value is less than or equal to the first platform threshold, keep the original value unchanged to obtain the second statistical value. Traversing the second statistical values, calculating the cumulative sum of the second statistical values, and obtaining the optimal threshold of the platform; Based on the platform's optimal threshold, the mapping relationship of the background image and the mapping relationship of the target image are obtained; the background image and the target image are obtained by looking up the table with the current image grayscale as the index; Based on the background image and the target image, the pyramid fusion algorithm is used to decompose the target image and the background image into image layers of different scales, and the image layers are fused in sequence starting from the largest scale to obtain the final fused image. The steps of decomposing the target image and the background image by the pyramid fusion algorithm include: Down-sample the target image and the background image respectively to obtain down-sampled image information; Upsample the downsampled background image to obtain the low-frequency image of the first layer background image; Taking the current background image information and the downsampled target image information as input images, iteratively downsample the image and upsample the background image one level to obtain multiple layers of image information at different scales. When the image information within a layer is fused, the steps include: Calculate the variance map and mean map of the target image and background image under the window radius r respectively; Correct the variance map. The correction formula for the variance map is: ， The calculation formula of the fusion coefficient coef[i] is: ， ， The formula for fusing the target image and the background image is: ； Among them, img_bg_l4_vm[i] is the corrected background variance array, img_bg_l4_v[i] is the background variance array of the current layer, and img_fg_l4_v[i] is the target variance array of the current layer; img_bg_l4_m[i] is the background mean array of the current layer, and img_fg_l4_m[i] is the target mean array of the current layer; sum_v[i] is the fusion array of the current layer background image and the corrected background image, and coef_set is the user-set parameter; img_fus_l4[i] is the current layer fusion array, and img_fg_l4_hf[i] is the background difference array when the current layer is decomposed; Calculate the fusion coefficient; The target image information and the background image information are fused according to the fusion coefficient to obtain a fused image; When inter-layer image information is fused, the steps include: Start fusion from the largest scale image layer and obtain the fused image of the current layer through the image information fusion operation within the layer; Combine the fused image of the current layer with the background difference image information for upsampling and reconstruction to obtain a reconstructed image; The reconstructed image of the current layer and the background image of the next layer are used as input images, and the inter-layer fusion process is iterated to obtain the reconstructed image; Taking the reconstructed image, background image and high-frequency image of the target image as input, the intra-layer image fusion process is iterated to obtain the final fused image.

2. The high dynamic range image dimming method according to claim 1, wherein: After obtaining the background image information of each layer, it is subtracted from the background image before sampling to obtain background difference map information.

3. A high dynamic range image dimming device, characterized in that: include: A histogram statistics module obtains the histogram statistics of the current image; wherein the histogram statistics are obtained by counting the number of pixels of each grayscale value on the image; The calibration module traverses the image histogram statistics to obtain the initial platform threshold; based on the initial platform threshold, the histogram statistics are traversed and modified to obtain the platform optimal threshold of the background image and the target image; based on the platform optimal threshold, the mapping relationship of the background image and the mapping relationship of the target image are obtained; the current image grayscale is used as the index to look up the table to obtain the background image and the target image; The initial platform threshold is obtained by traversing the image histogram statistics, starting from the statistical value of 0 and going upwards, traversing the grayscale of each statistical value, and counting the number of grayscales that are not 0; when the cumulative sum of the statistical values ​​is greater than the set threshold, stop and use the current statistical value as the initial platform threshold; When obtaining the platform optimal threshold value of the background image and the target image, the steps include: Starting from grayscale 0, the histogram statistics are traversed. If the histogram statistics is greater than the initial platform threshold, the current statistics are modified to the initial platform threshold. If the histogram statistics is less than or equal to the initial platform threshold, the original value is kept unchanged to obtain the first statistics. Traverse the first statistical value and calculate the cumulative sum of the first statistical value; Optimizing the cumulative sum of the first statistical values ​​using a preset optimization coefficient to obtain a first platform threshold; Starting from grayscale 0, traverse the first statistical value of the histogram. If the first statistical value is greater than the first platform threshold, modify the current statistical value to the first platform threshold. If the first statistical value is less than or equal to the first platform threshold, keep the original value unchanged to obtain the second statistical value. Traversing the second statistical values, calculating the cumulative sum of the second statistical values, and obtaining the optimal threshold of the platform; The fusion module uses a pyramid fusion algorithm to decompose the target image and the background image into image layers of different scales based on the background image and the target image, and fuses them sequentially starting from the image layer with the largest scale to obtain the final fused image; The steps of decomposing the target image and the background image by the pyramid fusion algorithm include: Down-sample the target image and the background image respectively to obtain down-sampled image information; Upsample the downsampled background image to obtain the low-frequency image of the first layer background image; Taking the current background image information and the downsampled target image information as input images, iteratively downsample the image and upsample the background image one level to obtain multiple layers of image information at different scales. When the image information within a layer is fused, the steps include: Calculate the variance map and mean map of the target image and background image under the window radius r respectively; Correct the variance map. The correction formula for the variance map is: ， The calculation formula of the fusion coefficient coef[i] is: ， ， The formula for fusing the target image and the background image is: ； Among them, img_bg_l4_vm[i] is the corrected background variance array, img_bg_l4_v[i] is the background variance array of the current layer, and img_fg_l4_v[i] is the target variance array of the current layer; img_bg_l4_m[i] is the background mean array of the current layer, and img_fg_l4_m[i] is the target mean array of the current layer; sum_v[i] is the fusion array of the current layer background image and the corrected background image, and coef_set is the user-set parameter; img_fus_l4[i] is the current layer fusion array, and img_fg_l4_hf[i] is the background difference array when the current layer is decomposed; Calculate the fusion coefficient; The target image information and the background image information are fused according to the fusion coefficient to obtain a fused image; When inter-layer image information is fused, the steps include: Start fusion from the largest scale image layer and obtain the fused image of the current layer through the image information fusion operation within the layer; Combine the fused image of the current layer with the background difference image information for upsampling and reconstruction to obtain a reconstructed image; The reconstructed image of the current layer and the background image of the next layer are used as input images, and the inter-layer fusion process is iterated to obtain the reconstructed image; Taking the reconstructed image, background image and high-frequency image of the target image as input, the intra-layer image fusion process is iterated to obtain the final fused image.

Images