A real-time tone mapping method based on cumulative histogram

Abstract

The invention discloses a real-time tone mapping method based on cumulative histogram, and the method comprises the following steps: histogram smoothing is performed to obtain a smoothed histogram; a brightness offset coefficient and a contrast factor are set to adjust the smoothed histogram; a cumulative histogram is calculated according to the adjusted smoothed histogram; a global mapping curve is obtained by transforming the cumulative histogram; a unique histogram smoothing algorithm is used to effectively process the histogram in various dimensions, and the horizontal direction and the vertical direction of the histogram are protected to be processed separately, so as to achieve a wide dynamic effect without relying on a single frame of a multi-frame image, and the overall calculation amount is extremely low, and the real-time performance is achieved while ensuring the wide dynamic effect, and the smoothing algorithm does not use a complex calculation process and too many parameters, which greatly saves resources and improves the efficiency of calculation.

Description

Technical Field

[0001] The invention relates to the technical field of image processing, and in particular to a real-time tone mapping method based on cumulative histogram. Background Art

[0002] The tone mapping function is an image enhancement technology that attempts to represent a high dynamic range input image with a low dynamic range output image. The goal of tone mapping is to retain the local contrast and details that can be observed in the high dynamic range input image, while maintaining the overall balance and structure of the picture in the output image. Tone mapping is a computer graphics and imaging technology that approximates the display of high dynamic range images on a limited dynamic range medium. Printers, monitors, and projectors all have limited dynamic ranges. Essentially, the problem that tone mapping solves is to perform a large contrast attenuation to transform the scene brightness to a displayable range while maintaining image details and colors, which are very important for representing the original scene. Depending on the application, the goal of tone mapping can be expressed differently. In some cases, generating a "good-looking" image is the main purpose, while in other cases, the emphasis may be on generating as many details as possible or maximizing image contrast.

[0003] For example, for an image, the dynamic range of the image refers to the ratio of the maximum brightness value to the minimum brightness value of the image; and a high dynamic range image refers to an image with a large dynamic range; for a high dynamic range image, its bit number is often higher than 8 bits (the bit number of an ordinary grayscale image is generally 8 bits), and the grayscale of the display is only 8 bits, so the color of the high dynamic range image must be transformed before it can be displayed; in addition, the grayscale value of the high dynamic range image is very unevenly distributed, and only a few pixels are brighter, so if the image is directly linearly normalized (the maximum grayscale value is mapped to 255, and the minimum grayscale value is mapped to 0) and then displayed, the image will be black; tone mapping is born to solve this problem.

[0004] Tone mapping has three main functions:

[0005] 1 Enhance contrast, color saturation, sharpness, enhance image effect, and achieve wide dynamic effect;

[0006] 2Simulate glare effects and virtual scene simulation for computer graphics;

[0007] 3. Compress and clip color range and dynamic range, display of HDR images.

[0008] Tone mapping algorithms generally fall into two categories: global color mapping and local tone mapping.

[0009] 1. Global tone mapping. Global tone mapping processes each brightness component value of each frame through a mapping curve. The transfer curve specifies the output value of a series of points. The points between the two (points that do not exist in the table) are derived using linear interpolation. Most global tone mapping algorithms have nonlinear mapping functions, and the same tone mapping curve is applied to each pixel of the image. The entire algorithm is simple and efficient. It cannot make adaptive adjustments based on scene information, resulting in the algorithm being unable to adapt to most scenes.

[0010] 2 Local tone mapping. The local tone mapping process includes applying a set of global mapping curves to each local tone mapped pixel value in the frame with weights; the mapping transfer curve is specified as a set of point values, and the points between the two are derived using linear interpolation; the transfer curve of each local segment is obtained by analyzing each local segment of the pixel after the global mapping process; local tone mapping is not applied in bypass mode; the local tone mapping algorithm is implemented by analyzing the overall pixels of the source image as well as the local pixels, so it has strong adaptability; but the algorithm process is complex and cannot achieve real-time performance.

[0011] At present, traditional tone mapping algorithms can be roughly divided into two categories: global methods and local methods;

[0012] Global method, global tone mapping method, refers to the method that uses the same mapping function for the entire image; for example, the gamma correction mentioned above is a global tone mapping method; the characteristics of the global method are:

[0013] 1. Any pixels of the same color will still be the same color after mapping;

[0014] 2. Global algorithms are generally simpler and faster;

[0015] 3. The performance of global algorithms is generally inferior to that of local methods.

[0016] Common global methods include: Gamma correction, logarithmic correction, histogram normalization, segmented grayscale transformation, etc.

[0017] A brief mention of logarithmic correction and histogram normalization is in order here:

[0018] Logarithmic correction is a tone mapping method based on the Weber-Fechner law. For human vision, within a mild range, the brightness perceived by the human eye is logarithmically related to the actual light intensity. In the unprocessed original image, the grayscale value of the pixel is linearly scaled by the actual light intensity, so a logarithmic transformation will make people feel more comfortable.

[0019] Histogram normalization is a tone mapping method that seeks to maximize the information entropy of an image. Information entropy is a quantity derived from thermodynamics that represents the amount of information in the data. The information entropy is calculated as -∑p·ln(p), where p is the probability of a certain grayscale level appearing in the image. For an image, if its grayscale histogram is uniformly distributed, then it has the maximum information entropy. Therefore, histogram normalization is a tone mapping method that makes the histogram as close to uniform distribution as possible after the tone is transformed.

[0020] Local method, local tone mapping method, refers to the method that the grayscale value after mapping may be different when the pixel is located at different positions; the mapping result of the pixel point is affected by other factors; the characteristics of the local method are:

[0021] 1. Pixels with the same color before mapping may have different colors after mapping;

[0022] 2. Local algorithms are generally more complex and relatively slower than global methods;

[0023] 3. Local algorithms generally perform better than global methods;

[0024] 4. Halo and other phenomena may appear.

[0025] Nowadays, the research hotspots of tone mapping methods are mainly concentrated on the innovation of local methods, and there are no tone mapping methods based on global method improvements. Summary of the invention

[0026] In view of the problem that there is no tone mapping method based on global method improvement in the current image processing technology field, the present invention provides a real-time tone mapping method based on cumulative histogram, which can achieve wide dynamic range (WDR) effect through global tone mapping.

[0027] To achieve the above object, the embodiments of the present invention adopt the following technical solutions:

[0028] A real-time tone mapping method based on cumulative histogram, the method comprising the following steps:

[0029] The histogram is smoothed to obtain a smoothed histogram;

[0030] Set the brightness offset coefficient and contrast factor to adjust the smoothed histogram;

[0031] Calculate the cumulative histogram based on the adjusted smoothed histogram;

[0032] The global mapping curve is obtained by transforming the cumulative histogram.

[0033] According to one aspect of the present invention, the histogram smoothing process to obtain a smoothed histogram comprises the following steps:

[0034] Get the histogram index values ​​corresponding to the overly bright area and the overly dark area;

[0035] Eliminate the histogram values ​​of overly bright and dark areas according to the index value;

[0036] Calculate the mean of the histogram values;

[0037] The histogram is obtained after limiting the maximum value of the histogram and the average value of the histogram is recalculated;

[0038] Calculate the maximum and minimum values ​​of the histogram segment by segment;

[0039] The maximum and minimum values ​​of the histogram are determined segmentally, so that the value of the segmented histogram is between the maximum and minimum values;

[0040] Perform a histogram smoothing operation to obtain a smoothed histogram.

[0041] According to one aspect of the present invention, the histogram smoothing process to obtain a smoothed histogram further includes the following steps: inputting a frame of image, obtaining the original histogram of the current image, normalizing the histogram, and setting the brightness index range of the histogram.

[0042] According to one aspect of the present invention, the histogram smoothing process to obtain a smoothed histogram also includes the following steps: obtaining four parameters: maximum brightness area suppression percentage, maximum dark area suppression percentage, maximum bright area suppression cumulative value percentage and maximum dark area suppression cumulative value percentage.

[0043] According to one aspect of the present invention, the obtaining of histogram index values ​​corresponding to the over-bright area and the over-dark area includes:

[0044] Calculate the index values ​​of the bright area histogram and the dark area histogram according to the maximum brightness area suppression percentage, the maximum dark area suppression percentage and the total number of items in the histogram;

[0045] Perform an index scan of dark pixels from the minimum value of the brightness index range of the histogram to the index value area of ​​the dark area histogram, and accumulate the value of the histogram;

[0046] When the accumulated value reaches the maximum dark area suppression accumulated value percentage, the corresponding index is the over-dark area histogram index value;

[0047] Perform an index scan of bright pixels from the maximum value of the brightness index range of the histogram to the index value area of ​​the bright area histogram, and accumulate the value of the histogram;

[0048] When the accumulated value reaches the maximum bright area suppression accumulated value percentage, the corresponding index is the over-bright area histogram index value.

[0049] According to one aspect of the present invention, the method of eliminating the histogram values ​​of overly bright and overly dark areas according to the index values ​​includes: setting the histogram values ​​corresponding to the histogram index values ​​from 0 to the overly dark area and the histogram index values ​​from the overly bright area to index 64 to 0, and obtaining a new histogram after the calculation is completed.

[0050] According to one aspect of the present invention, the histogram smoothing process to obtain a smoothed histogram includes the following steps: obtaining the ratio of the maximum value and the minimum value of each segment of the segmented histogram to the average value, and storing them as Celling and Floor respectively; calculating the start index and the end index of each histogram sub-area.

[0051] According to one aspect of the present invention, the segmented determination of the maximum and minimum values ​​of the histogram so that the value of the segmented histogram is between the maximum and minimum values ​​includes: determining the histogram maximum value of each segment of the histogram through Celling, determining the minimum value of each segment of the histogram through Floor, and then applying the segment histogram values ​​to be between the minimum and maximum values, defining the value below the minimum value to be equal to the minimum value, and defining the value above the maximum value to be equal to the maximum value.

[0052] According to one aspect of the present invention, the cumulative histogram is calculated according to the adjusted smoothed histogram, including: defining the smoothed histogram as Hist(i), the brightness offset coefficient as Wb, the contrast factor as Wc, and the smoothed histogram as Hist(i)=Hist(i)*Wc+Wb; calculating the cumulative histogram AccHist, the calculation method is as follows:

[0053] AccHist(0) = Hist(0);

[0054] AccHist(i)=AccHist(i-1)+Hist(i);

[0055] The other 0th item is the histogram 0 item, and the subsequent cumulative histogram i item is equal to the cumulative histogram i-1 item plus the histogram i item.

[0056] According to one aspect of the present invention, the transformation through the cumulative histogram to obtain the global mapping curve includes: calculating the global mapping curve MappingCurve, α is the mixing ratio; transforming through the cumulative histogram AccHist to obtain the global mapping curve:

[0057] mappingCurve i =(1-α)*mappingCurve i-1 +α*AccHist.

[0058] The advantages of the implementation of the present invention are as follows: histogram smoothing and brightness contrast factor are introduced, a unique histogram smoothing algorithm is used, the histogram is effectively processed in various dimensions, the horizontal direction and the vertical direction of the histogram are protected and processed respectively, a wide dynamic effect is achieved without relying on a single frame of a multi-frame image, and the overall calculation amount is extremely low. While ensuring the wide dynamic effect, real-time performance is achieved. The smoothing algorithm does not use complex calculation processes and excessive parameters, which greatly saves resources and improves calculation efficiency; it is the first to achieve fine-tuning of image brightness and contrast through linear changes in the histogram; it is aimed at low power consumption, high real-time, simple and reliable implementation, no complex calculations, and very few resources, and the algorithm is more conducive to integrated circuit design and development; the effect is significant; the cumulative histogram is used to transform the histogram, and the adjustment of the mixing factor is added to generate a global mapping curve. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0060] Figure 1 is the original data histogram described in the present invention;

[0061] Figure 2 The histogram after smoothing processing described in the present invention;

[0062] Figure 3 is a schematic diagram of the global mapping curve of the present invention;

[0063] Figure 4 This is a flowchart of the histogram smoothing process described in the present invention;

[0064] Figure 5 This is an overall flow chart of the real-time tone mapping method based on cumulative histogram described in the present invention. DETAILED DESCRIPTION

[0065] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0066] like Figure 1 , Figure 2 , Figure 3 , Figure 4 and Figure 5 As shown, a real-time tone mapping method based on cumulative histogram, the method comprises the following steps:

[0067] Step S1: histogram smoothing to obtain a smoothed histogram;

[0068] Histogram smoothing, the core processing of global wide dynamic range (WDR) is histogram smoothing, which includes the following steps:

[0069] Step S11: obtaining histogram index values ​​corresponding to the over-bright area and the over-dark area;

[0070] First, input a frame of image and obtain the original histogram hist_orig of the current image (such as Figure 1 As shown), the histogram is normalized and the brightness index range of the histogram is set to 0 to HIST_N (for example, the 8bi pixel value is HIST_N is 255).

[0071] Calculate the over-bright area index bright_Idx and the over-dark area index dark_Idx of the histogram; the purpose is to limit the width of the horizontal processing of the histogram to avoid the overall impact of over-saturation and under-saturation points;

[0072] The calculation process requires the following four parameters:

[0073] MaxBrightSuppress: Maximum bright area suppression percentage;

[0074] MaxDarkSuppress: Maximum dark area suppression percentage;

[0075] BrightSuppressRatio: The percentage of the maximum bright area suppression cumulative value;

[0076] DarkSuppressRatio: Maximum dark area suppression cumulative value percentage;

[0077] Step S111: Calculate the index values ​​of the bright and dark area histograms according to the maximum brightness, the dark area suppression percentage and the total number of histogram items HIST_N:

[0078] Dark_Idx=MaxDarkSuppress*HIST_N;

[0079] Bright_Idx=HIST_N-(MaxBrightSuppress*HIST_N)-1.

[0080] Step S112: According to the result of step S111, scan the area from 0 to darkIdx and the area from HIST_N to BrightIdx; scan the index of dark pixels and accumulate the value of the histogram (that is, accumulate the value of the current histogram). When the accumulated value reaches the specified ratio DarkSupressRatio, the corresponding index is DarkIdx; scan the index of bright pixels and accumulate the value of the histogram. When the accumulated value reaches the specified ratio brightSupressRatio, the corresponding index is brightIdx.

[0081] Step S12: Eliminate the histogram values ​​of overly bright and overly dark areas according to the index value;

[0082] Eliminate the histogram values ​​corresponding to the overly bright and dark areas. The specific operation is to set the histogram values ​​corresponding to the indexes from 0 to darkIdx and brightidx to 64 to 0; the new histogram Hist is obtained after the calculation.

[0083] Step S13: Calculate the average value of the histogram;

[0084] According to the results obtained in step S11 and step S12, the average value dmean of the histogram value is calculated. The specific calculation process is: firstly perform the cumulative sum HistSum of the Hist values, and calculate the mean value of the histogram in the valid area; and ignore the invalid area;

[0085] Dmean=HistSum / (brightidx-darkIdx+1).

[0086] Step S14: obtaining a histogram after limiting the maximum value of the histogram and recalculating the average value of the histogram value;

[0087] Limit the maximum value of the histogram, set the maximum value of the histogram to HistMax, where OverShotThreshold is the maximum value threshold ratio; the calculation method is HistMax = dmean * OverShotThreshold; when the value of the histogram exceeds the maximum value, it is equal to HistMax, and after this process of calculation, the histogram Hist is obtained, and the mean dmean of the histogram is recalculated according to step S13;

[0088] Step S15: Calculate the maximum and minimum values ​​of the histogram segment by segment;

[0089] The processed histogram Hist is divided into N segments, and the maximum and minimum values ​​of the histogram of each segment are calculated. Two important parameters, Celling and Floor, are used. The parameters store the ratio of the maximum and minimum values ​​of each segment to the mean.

[0090] Step S16: determining the maximum and minimum values ​​of the histogram in segments, so that the value of the segmented histogram is between the maximum and minimum values;

[0091] Assume that the starting index ss and the ending index ee of the ii segment histogram sub-region, where ii ranges from 0 to N; calculate the starting index and the ending index of each sub-region according to the calculation formula;

[0092]

[0093]

[0094] Step S17: performing a histogram smoothing operation to obtain a smoothed histogram.

[0095] The maximum value segMin of the histogram of segment ii is determined by item ii of Celling, and the minimum value segMax of the histogram of segment ii is determined by item ii of Floor. Then the histogram value Hist(i) of this segment is applied. All values ​​below segMin are equal to segMin, and values ​​above segMax are equal to segMax.

[0096] segMin = Ceiling[ii]*dmean;

[0097] segMax = Floor[ii]*dmean;

[0098] Hist(i)=min(Hist(i), segMin);

[0099] Hist(i)=max(Hist(i), segMax);

[0100] i∈[ss,ee]

[0101] After the N-segment histogram processing is completed, a histogram smoothing operation is performed, and the smoothing processing is performed through a one-dimensional smoothing kernel K. For example, the smoothing kernel K is [0.05, 0.10, 0.20, 0.30, 0.20, 0.10, 0.05]; and the algorithm method is as follows:

[0102] Hist_smooth[i-3:i+3]=Hist[i-3:i+3]*K[i-3:i+3]; i belongs to [0, N-1]

[0103] After smoothing, the histogram Hist_smooth is obtained as follows Figure 2 shown.

[0104] Step S2: Setting the brightness offset coefficient and contrast factor to adjust the smoothed histogram;

[0105] The brightness offset coefficient Wb and the contrast factor Wc are set in the cumulative histogram, which effectively provides the overall brightness and contrast of the picture; according to step S1, a smoothed histogram is obtained, Hist(i)=Hist(i)*Wc+Wb, and the smoothed histogram is adjusted in the vertical direction, which further effectively enhances the wide dynamic effect.

[0106] Step S3: Calculating a cumulative histogram based on the adjusted smoothed histogram;

[0107] Calculate the cumulative histogram, obtain the smoothed histogram according to step S17, and calculate the cumulative histogram AccHist. The calculation method is as follows:

[0108] AccHist(0) = Hist(0);

[0109] AccHist(i)=AccHist(i-1)+Hist(i);

[0110] The other 0th item is the histogram 0 item, and the subsequent accumulated histogram item i is equal to the accumulated histogram item i-1 plus the histogram item i; step S2 obtains the accumulated histogram AccHist.

[0111] Step S4: Transform the cumulative histogram to obtain a global mapping curve.

[0112] Calculate the global mapping curve MappingCurve, AccHist is the cumulative histogram after step 3 of the histogram Hist, and α is the mixing ratio; this step transforms the cumulative histogram to obtain the global mapping curve, such as Figure 3 As shown:

[0113] mappingCurve i =(1-α)*mappingCurve i-1 +α*AccHist.

[0114] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with the art within the technical scope disclosed in the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A real-time tone mapping method based on cumulative histogram, characterized in that: The method comprises the following steps: The histogram is smoothed to obtain a smoothed histogram; Set the brightness offset coefficient and contrast factor to adjust the smoothed histogram; Calculate the cumulative histogram based on the adjusted smoothed histogram; The global mapping curve is obtained by transforming the cumulative histogram; The histogram smoothing process to obtain a smoothed histogram includes the following steps: Get the histogram index values ​​corresponding to the overly bright area and the overly dark area; Eliminate the histogram values ​​of overly bright and dark areas according to the index value; Calculate the mean of the histogram values; The histogram is obtained after limiting the maximum value of the histogram and the average value of the histogram is recalculated; Calculate the ratio of the maximum and minimum values ​​of each section of the histogram to the average value; The maximum and minimum values ​​of the histogram are determined segmentally, so that the value of the segmented histogram is between the maximum and minimum values; Perform a histogram smoothing operation to obtain a smoothed histogram; The histogram smoothing process to obtain a smoothed histogram further includes the following steps: Get four parameters: maximum brightness area suppression percentage, maximum dark area suppression percentage, maximum bright area suppression cumulative value percentage, and maximum dark area suppression cumulative value percentage; The obtaining of the histogram index values ​​corresponding to the over-bright area and the over-dark area includes: Calculate the index values ​​of the bright area histogram and the dark area histogram according to the maximum brightness area suppression percentage, the maximum dark area suppression percentage and the total number of items in the histogram; Perform index scanning of dark pixels from the minimum value of the brightness index range of the histogram to the index value area of ​​the dark area histogram, and accumulate the value of the histogram; When the accumulated value reaches the maximum dark area suppression accumulated value percentage, the corresponding index is the over-dark area histogram index value; Perform an index scan of bright pixels from the maximum value of the brightness index range of the histogram to the index value area of ​​the bright area histogram, and accumulate the value of the histogram; When the accumulated value reaches the maximum bright area suppression accumulated value percentage, the corresponding index is the over-bright area histogram index value.

2. The real-time tone mapping method based on cumulative histogram according to claim 1, characterized in that: The histogram smoothing process to obtain a smoothed histogram also includes the following steps: inputting a frame of image, obtaining the original histogram of the current image, normalizing the histogram, and setting the brightness index range of the histogram.

3. The real-time tone mapping method based on cumulative histogram according to claim 1, characterized in that: The method of eliminating the histogram values ​​of overly bright and overly dark areas according to the index values ​​includes: setting the values ​​of the histogram corresponding to the histogram index values ​​from 0 to the overly dark area and the histogram index values ​​from the overly bright area to the index 64 to 0, and obtaining a new histogram after the calculation is completed.

4. The real-time tone mapping method based on cumulative histogram according to claim 3, characterized in that: The histogram smoothing process to obtain a smoothed histogram includes the following steps: obtaining the ratio of the maximum value and the minimum value of each segment of the segmented histogram to the average value, and storing them as Celling and Floor respectively; calculating the start index and the end index of each histogram sub-area.

5. The real-time tone mapping method based on cumulative histogram according to claim 4, characterized in that: The segmented determination of the maximum and minimum values ​​of the histogram so that the value of the segmented histogram is between the maximum and the minimum includes: determining the maximum value of the histogram of each segment of the histogram by Celling, determining the minimum value of each segment of the histogram by Floor, and then applying the segmented histogram values ​​to be between the minimum and the maximum, defining the value below the minimum value to be equal to the minimum value, and defining the value above the maximum value to be equal to the maximum value; The maximum value segMin of the histogram of segment ii is determined by item ii of Celling, and the minimum value segMax of the histogram of segment ii is determined by item ii of Floor. The calculation method is as follows: Among them, the starting index of the histogram sub-region of segment ii is ss, and the ending index is ee; dmean is the average value of the histogram.

6. The real-time tone mapping method based on cumulative histogram according to any one of claims 1 to 5, characterized in that: The step of calculating the cumulative histogram according to the adjusted smoothed histogram includes: defining the smoothed histogram as Hist(i), the brightness offset coefficient as Wb, the contrast factor as Wc, and the smoothed histogram as Hist(i) = Hist(i)*Wc + Wb; and calculating the cumulative histogram AccHist, the calculation method is as follows: The other 0th item is the histogram 0 item, and the subsequent cumulative histogram i item is equal to the cumulative histogram i-1 item plus the histogram i item.

7. The real-time tone mapping method based on cumulative histogram according to claim 6, characterized in that: The transformation by the cumulative histogram to obtain the global mapping curve includes: calculating the global mapping curve MappingCurve, is the mixing ratio; the global mapping curve is obtained by transforming the cumulative histogram AccHist: 。

Images