Histogram adjustment and image processing method, readable storage medium and electronic device

Abstract

A histogram adjustment and image processing method, a readable storage medium and an electronic device, wherein the histogram adjustment method comprises: a first adjustment step. The first adjustment step comprises: deleting the number of pixels in each bin that exceeds its own limit value in the histogram to obtain a first intermediate histogram, and calculating the total number of deleted pixels to obtain a first deleted total number; calculating the total number of pixels in each bin that does not reach its own limit value in the first intermediate histogram to obtain a first remaining total number; in the first intermediate histogram, supplementing the first deleted total number to each bin that does not reach its own limit value according to a distribution ratio to obtain an adjusted histogram; wherein the distribution ratio is: the ratio of the number of pixels corresponding to each bin that does not reach its own limit value to the first remaining total number.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a histogram adjustment and image processing method, a readable storage medium and an electronic device. Background Art

[0002] In recent years, more and more cameras, camcorders, and mobile phone cameras can capture high dynamic range images. High dynamic range images can simultaneously record the contents of bright and dark areas in a scene. For example, when shooting a person in front of a window indoors on a sunny day, a high dynamic range image can simultaneously record the dim portrait in front of the window and the bright scenery outside the window. However, most display devices are still low dynamic range (LDR) devices with a dynamic range of 255, and cannot directly display high dynamic range (HDR) images. It is necessary to first perform dynamic range compression on high dynamic range images through image processing, usually using tone mapping to convert them into low dynamic range images.

[0003] Tone mapping based on histogram adjustment can improve the contrast of the image, make the image more in line with the observation habits of the human eye, and improve the image quality. However, it is technically difficult to implement histogram adjustment for high-dynamic images because the number of vertical bars (bins) required for statistical histograms is much larger than that for low-dynamic images. For example, for a 20-bit high-dynamic image, if 1 bin is assigned to each possible DN value (pixel value), then 2 bins will be required. 20 ≈ 1 million bins are used for histogram statistics. Such a large number of bins will result in slow calculation speed and even be difficult to implement in hardware such as FPGA and DSP.

[0004] There are many methods for adjusting histograms in the prior art, for example, Figure 1 As shown, a limit height curve limit_curve_1 is set for the original histogram hist1, and the bins in the original histogram hist1 that are larger than the limit height curve limit_curve_1 are cut off, and then the total number of cut pixels is added to each bin on average to obtain the adjusted histogram hist1-1. The limit height curve limit_curve_1 can be a horizontal line or a curve, and those skilled in the art can set it according to experiments or experience, in order to prevent the contrast of the image from being excessively enhanced. For more technical details, please refer to the Chinese patent application with publication number CN113691739A.

[0005] The above-mentioned prior art can achieve a relatively good histogram adjustment effect for low dynamic range images (LDR), but the adjustment effect for high dynamic range images is poor. Figure 2 In the 20-bit high dynamic range image shown, the bright part of the image is a transmissive resolution target, and the dark part is a background such as a reflective resolution target.

[0006] The Chinese patent application with publication number CN113691739A proposes a method of using segmented histogram adjustment to make detailed histogram bin divisions for dark areas and coarse histogram bin divisions for bright areas. Thus, while compressing the dynamic range of high-dynamic images, the compressed images have a higher contrast, and this process can be completed with a smaller number of bins, making it easy to implement in hardware. However, the technology in this patent application has a more detailed contrast stretching effect on dark areas in high-dynamic images, but the contrast stretching effect on the details of bright areas is not sufficient.

[0007] Figure 3 for Figure 2 The histogram of the image shown in FIG. 1 shows that most of the pixels are concentrated in the dark area, and the number of pixels in the bright area is relatively small. The histogram is adjusted using the existing technology: the bins truncated by the limit curve are evenly distributed to each bin, such as Figure 4 As shown in the figure. Since there are more bins to be assigned, the number of pixels assigned to the bins on the right side representing the bright area of ​​the image is small. After the histogram is adjusted, the individual bins in the right pixel set are still low, which will cause the calculated mapping curve to be unable to fully stretch the details of the bright area. Summary of the invention

[0008] The problem solved by the present invention is that the adjustment of the histogram in the prior art may result in insufficient stretching of the contrast.

[0009] In order to solve the above problem, the present invention provides a histogram adjustment method, comprising: a first adjustment step.

[0010] The first adjustment step comprises:

[0011] In the histogram, the number of pixels in each bin exceeding its own limit value is deleted to obtain a first intermediate histogram, and the total number of deleted pixels is calculated to obtain a first total number of deleted pixels;

[0012] Calculate the total number of pixels of each bin in the first intermediate histogram that does not reach its own limit value to obtain a first remaining total number;

[0013] In the first intermediate histogram, the first total amount of deletions is added to each bin that has not reached its own limit value according to the distribution ratio to obtain an adjusted histogram;

[0014] The allocation ratio is: the ratio of the number of pixels corresponding to each bin that has not reached its own limit value to the first remaining total amount.

[0015] Optionally, the histogram adjustment method may further include: when the adjusted histogram has a bin whose number of pixels exceeds the self-limited value, repeatedly executing the first adjustment step until the number of executions of the first adjustment step reaches a threshold number of times.

[0016] Optionally, the histogram adjustment method may further include: when the number of executions of the first adjustment step reaches the number threshold, executing the second adjustment step.

[0017] The second adjustment step comprises:

[0018] In the histogram, the number of pixels in each bin exceeding its own limit value is deleted to obtain a second intermediate histogram, and the total number of deleted pixels is calculated to obtain a second total number of deleted pixels;

[0019] In the second intermediate histogram, the second total amount of deletions is evenly distributed to each bin to obtain an adjusted histogram.

[0020] The present invention also provides an image processing method, comprising:

[0021] Get the first layer histogram corresponding to the original image;

[0022] Executing the above histogram adjustment method on the first layer histogram to obtain an adjusted first layer histogram;

[0023] generating a first layer mapping curve according to the adjusted first layer histogram;

[0024] When the discreteness of the first layer histogram does not exceed a preset threshold, mapping the original image according to the first layer mapping curve to obtain a target image;

[0025] When the discreteness of the first layer histogram exceeds the preset threshold, a plurality of bins are selected from the first layer histogram, and a second layer histogram is formed according to the selected bins and their corresponding pixels;

[0026] Performing a histogram adjustment on the second layer histogram to obtain an adjusted second layer histogram;

[0027] generating a second layer mapping curve according to the first layer mapping curve, the adjusted second layer histogram and the selected bin;

[0028] The original image is mapped according to the first layer mapping curve and the second layer mapping curve to obtain a target image.

[0029] Optionally, the step of generating a first-layer mapping curve according to the adjusted first-layer histogram includes:

[0030] Calculate the cumulative histogram using the adjusted first-layer histogram to obtain the first-layer cumulative histogram;

[0031] Normalizing the first-layer cumulative histogram to obtain a normalized first-layer cumulative histogram;

[0032] The first layer mapping curve is generated according to the normalized first layer cumulative histogram.

[0033] Optionally, the formula for generating the first layer mapping curve is:

[0034] map_curve1 = Lout * cdf1

[0035] Among them, cdf1 is the normalized first-layer cumulative histogram, map_curve1 represents the first-layer mapping curve, and Lout is the maximum available pixel value of the target image.

[0036] Optionally, mapping the input pixel value range of the selected bin according to the first layer mapping curve to obtain the output pixel value range of the selected bin;

[0037] The formula for generating the second layer mapping curve is:

[0038] map_curve2=y_min+(y_max-y_min)*cdf2

[0039] Among them, map_curve2 represents the second layer mapping curve, y_min is the minimum value in the output pixel value range of the selected bin, and y_max is the maximum value in the output pixel value range of the selected bin.

[0040] Optionally, the second layer histogram includes: a first second layer histogram and a second second layer histogram;

[0041] Based on the first layer histogram, among the bins whose brightness values ​​are less than the brightness threshold, at least the bin with the highest number of pixels is selected as the first bin, and histogram statistics are performed on the first bin to form the first second layer histogram;

[0042] Based on the first layer histogram, among the bins whose brightness values ​​are greater than or equal to the brightness threshold, at least the bin with the highest number of pixels is selected as the second bin, and histogram statistics are performed on the second bin to form the second second layer histogram;

[0043] A first second-layer mapping curve corresponding to the first second-layer histogram and a second second-layer mapping curve corresponding to the second second-layer histogram are generated.

[0044] Optionally, the step of selecting at least the bin with the highest number of pixels as the first bin includes:

[0045] The bin with the highest number of pixels and the bin with the higher number of pixels in the two adjacent bins are selected and merged to form the first bin.

[0046] Optionally, the step of selecting at least the bin with the highest number of pixels as the second bin includes:

[0047] The bin with the highest number of pixels and the bin with the higher number of pixels in the two adjacent bins are selected and merged as the second bin.

[0048] Optionally, mapping the original image according to the first layer mapping curve and the second layer mapping curve to obtain a target image includes:

[0049] When the brightness value of a pixel in the original image falls within the brightness range corresponding to the first bin, mapping is performed using the first second-layer mapping curve;

[0050] When the brightness value of a pixel in the original image falls within the brightness range corresponding to the second bin, mapping is performed using the second second-layer mapping curve;

[0051] When the brightness value of a pixel in the original image does not fall within the brightness range corresponding to the first bin and the second bin, mapping is performed using the first layer mapping curve.

[0052] Optionally, the calculation formula of the discreteness is:

[0053]

[0054] Among them, D represents the discreteness, bin(i) represents the number of pixels corresponding to the i-th bin, and k represents the total number of bins divided by the first layer histogram.

[0055] Optionally, the step of mapping the original image according to the first layer mapping curve to obtain a target image includes:

[0056] A first mapping is performed on the brightness value of the original image according to the first layer mapping curve to obtain the brightness value of each pixel of the target image.

[0057] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the steps of the above method.

[0058] The present invention also provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and wherein the processor executes the steps of the above method when running the computer program.

[0059] Compared with the prior art, the technical solution of the present invention has the following advantages:

[0060] Using the histogram adjustment method provided by the present invention, the histogram can be adjusted according to the ratio of the number of pixels corresponding to the bin that has not reached its own limit value to the total number of pixels remaining after deletion. Since the height of the bin is increased in proportion, the height of the bin with a higher height is also increased, and the bin with a relatively concentrated number of pixels can obtain better contrast stretching. Moreover, by making full use of grayscale, the contrast of bins with an excessively concentrated number of pixels (such as large uniform areas) can be prevented from being over-stretched, and bins with a relatively concentrated number of pixels can obtain better contrast stretching, and reduce computing resources, which is easy to implement in hardware.

[0061] The image processing method provided by the present invention can adaptively determine the pixel value range of most pixels in the high-dynamic image, and perform fine histogram bin division on the pixel value range in a targeted manner, while performing a rough histogram bin division on the pixel value range with fewer pixels. It is suitable for high-dynamic images with various brightness distribution characteristics and has high flexibility. Using a smaller number of histogram bins can achieve dynamic range compression of high-dynamic images and improve image contrast. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 It is a schematic diagram of an existing histogram adjustment;

[0063] Figure 2 It is a high dynamic image schematic diagram;

[0064] Figure 3 yes Figure 2 Histogram of the image shown;

[0065] Figure 4 Based on existing technology Figure 3 The adjusted histogram;

[0066] Figure 5 is a schematic diagram of a first adjustment step flow chart provided by an embodiment of the present invention;

[0067] Figure 6 is a schematic diagram of an adjusted histogram obtained using this embodiment of the present invention;

[0068] Figure 7 is a flowchart of an image processing method provided by an embodiment of the present invention;

[0069] Figure 8 It is a schematic diagram of the first layer mapping curve;

[0070] Fig. 9 This is the first schematic diagram of the second layer histogram;

[0071] Fig.10 It is a schematic diagram of the first second layer mapping curve;

[0072] Fig.11 This is a schematic diagram of the second second layer histogram;

[0073] Fig.12 is a schematic diagram of the second layer 2 mapping curve;

[0074] Fig.13 It is an LDR image obtained using the technical solution of the present invention;

[0075] Fig.14 It is an LDR image obtained using existing technology. DETAILED DESCRIPTION

[0076] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0077] An embodiment of the present invention provides a histogram adjustment method, including: a first adjustment step.

[0078] like Figure 5 As shown, the first adjustment step includes:

[0079] Step S11, deleting the number of pixels in each bin exceeding its own limit value in the histogram to obtain a first intermediate histogram, and calculating the total number of deleted pixels to obtain a first total number of deleted pixels;

[0080] Step S12, calculating the total number of pixels of each bin in the first intermediate histogram that does not reach its own limit value to obtain a first remaining total number;

[0081] Step S13, in the first intermediate histogram, the first deleted total amount is added to each bin that has not reached its own limit value according to the distribution ratio to obtain an adjusted histogram; wherein the distribution ratio is: the ratio of the number of pixels corresponding to each bin that has not reached its own limit value to the first remaining total amount.

[0082] by Figure 3 Taking the histogram hist1 as an example, in the figure, the limit height curve limitcurve represents the number of pixels of the limit value of each bin. It can be understood that the number of pixels of the limit value of each bin can also be different, that is, the limit height curve limitcurve is not a horizontal line.

[0083] Execute step S11, in the histogram hist1, cut off the part higher than the limit height curve limitcurve, that is, delete the number of pixels in each bin that exceeds its own limit value, to obtain a first intermediate histogram (not shown). The height value of each bin represents the number of pixels in the image within the pixel value range represented by the bin.

[0084] Calculate the total number of deleted pixels, that is, add together the part of each bin in the histogram hist1 that is above the limit curve to obtain the first total number of deleted pixels, trunc_sum:

[0085]

[0086] k represents the number of bins divided by the histogram, bin(i) represents the number of pixels in the i-th bin, limit_curve_1(i) represents the limit value corresponding to the limit height curve in the i-th bin, and max(a,b) represents the larger value of a and b. That is to say, if bin(i) is smaller than limit_curve_1(i), it will not be included in the first total deletion trunc sum.

[0087] The existing technology is to divide the sum of the truncated parts of each bin equally into each bin, raise each bin, and obtain an adjusted histogram. The characteristics of high dynamic images are: a large number of bins have very small or 0 pixels. Therefore, a large part of the sum of the truncated parts of each bin is allocated to bins with very small or 0 pixels, while bins with concentrated pixels are not allocated more raised values, resulting in insufficient contrast stretching.

[0088] In this embodiment, through step S12 and step S13, the first deletion total trunc_sum is added to each bin (i) that has not reached its own limit value according to the distribution ratio. 未截断 ). Therefore, each bin (i 未截断 )Multiply the original number of pixels by a coefficient coef, coef>=1, and increase the height of the bin proportionally.

[0089] The sum of the number of pixels in the raised part of each bin is equal to the first total deletion trunc_sum, so:

[0090] trunc_sum=∑i 未截断 (coef-1)*bin(i 未截断 ) Formula 2

[0091] thereby:

[0092]

[0093] In formulas 2 and 3, i 未截断 Indicates the number of the bin that has not reached its own limit value, bin(i 未截断 ) represents the number of pixels in the bin that do not reach their own limit value.

[0094] Each untruncated bin (i 未截断 ) is multiplied by the coefficient coef, the number of pixels corresponding to the truncated bin is equal to limit_curve_1(i), thus, the number of pixels bin_adjust(i) of each bin(i) of the adjusted histogram is obtained:

[0095]

[0096] In formula 4, i 未截断 Indicates the number of the bin that has not reached its own limit value, i 截断 Indicates the number of the bin that reaches its limit value.

[0097] like Figure 6 As shown, in the adjusted histogram hist1-2 obtained by using this embodiment, the height of the higher bin itself is also increased to a higher level, so that the bin with a relatively concentrated number of pixels can obtain better contrast stretching. In addition, by making full use of grayscale, the contrast of the bin with an excessively concentrated number of pixels (such as a large uniform area) can be prevented from being over-stretched, and the bin with a relatively concentrated number of pixels can obtain better contrast stretching, and the computing resources are reduced, which is easy to implement in hardware.

[0098] The histogram adjustment method described in this embodiment may further include: when the adjusted histogram has a bin whose number of pixels exceeds the self-limited value, repeatedly executing the first adjustment step until the number of executions of the first adjustment step reaches a number threshold.

[0099] Specifically, after multiplying the coefficient coef in Formula 4, the height of some bins may exceed limit_curve_1 again. In this case, the steps of Formula 1 to Formula 4 can be iteratively repeated, and the iteration is stopped when the number of iterations reaches the number threshold.

[0100] Update the value of the first deletion total trunc_sum in each iteration:

[0101] At the beginning of each iteration, let bin(i) = bin_adjust(i), and:

[0102]

[0103] When the number of iterations reaches a threshold number, a second adjustment step may be performed.

[0104] The second adjustment step comprises:

[0105] In the histogram, the number of pixels in each bin exceeding its own limit value is deleted to obtain a second intermediate histogram, and the total number of deleted pixels is calculated to obtain a second total number of deleted pixels;

[0106] In the second intermediate histogram, the second total amount of deletions is evenly distributed to each bin to obtain an adjusted histogram.

[0107] The second adjustment step is actually an existing histogram adjustment method. Combining it with the first adjustment step can achieve full distribution of the truncated histogram, pull apart the grayscale between adjacent pixel value concentration areas, and increase the image layering. Therefore, the iteration number threshold can also be set to a smaller value, such as 1 or 2, so as to leave a larger second deletion total amount to perform the second adjustment step for average distribution.

[0108] The advantage of this embodiment is that by multiplying the bin pixel size by a coefficient, the contrast of the bin with a relatively concentrated pixel number can be fully stretched; at the same time, by limiting the height curve limit curve, it is ensured that the contrast is not over-stretched; finally, the second total deletion amount is evenly distributed to open up the grayscale between adjacent pixel value concentrated areas, thereby increasing the image layering.

[0109] like Figure 7 As shown, the embodiment of the present invention also provides an image processing method, including:

[0110] Step S21, obtaining the first layer histogram corresponding to the original image;

[0111] Step S22, performing histogram adjustment on the first layer histogram to obtain an adjusted first layer histogram;

[0112] Step S23, generating a first layer mapping curve according to the adjusted first layer histogram;

[0113] Step S24, determining whether the dispersion of the first layer histogram exceeds a preset threshold, if not, executing step S25, if exceeding, executing steps S26 to S29;

[0114] Step S25, mapping the original image according to the first layer mapping curve to obtain a target image;

[0115] Step S26, selecting a number of bins in the first layer histogram, and forming a second layer histogram according to the selected bins and their corresponding pixels;

[0116] Step S27, adjusting the second layer histogram to obtain an adjusted second layer histogram;

[0117] Step S28: generating a second layer mapping curve according to the first layer mapping curve, the adjusted second layer histogram and the selected bin.

[0118] Step S29: Map the original image according to the first layer mapping curve and the second layer mapping curve to obtain a target image.

[0119] The above steps are described in detail below by giving examples.

[0120] In step S21, the original image is an HDR image, and a histogram corresponding to the HDR image can be obtained through existing technology.

[0121] In step S22, the histogram adjustment method provided in the previous embodiment may be used to adjust the histogram of the first layer. For detailed description, please refer to the description of the previous embodiment.

[0122] The step S23 may include steps S231 to S233:

[0123] Step S231, using the adjusted first layer histogram to calculate the cumulative histogram to obtain the first layer cumulative histogram;

[0124] Step S232, normalizing the first-layer cumulative histogram to obtain a normalized first-layer cumulative histogram;

[0125] Step S233: Generate the first layer mapping curve according to the normalized first layer cumulative histogram.

[0126] The normalized cumulative histogram cdf is:

[0127]

[0128] bin(i) represents the number of pixels corresponding to the i-th bin of the histogram, i=0,1,...,k-1, and k represents the number of bins into which the histogram is divided.

[0129] The method for generating a mapping curve map_curve from a normalized cumulative histogram cdf is:

[0130] map_curve=ymin+(ymax-ymin)*cdf Formula 7

[0131] In Formula 7, ymin and ymax are respectively the minimum value and the maximum value of the pixel value range to be mapped.

[0132] For the first layer mapping curve map_curve1, it is usually hoped that the range it maps to is the largest, that is, it maps to all available pixel values, so ymin = 0, ymax = Lout, and substitute into formula 7:

[0133] map_curve1(x)=Lout*cdf1(x) Formula 8

[0134] 0≤x≤Lin

[0135] In Formula 8, cdf1 is a normalized first-layer cumulative histogram, Lout is a maximum available pixel value of a target image after mapping, and Lin is a maximum available pixel value of an HDR image before mapping, and the target image is an LDR image.

[0136] In step S24, the discreteness of the first layer histogram needs to be calculated, and the calculation formula of the discreteness is:

[0137]

[0138] In Formula 9, D represents the discreteness, bin(i) represents the number of pixels corresponding to the i-th bin, and k represents the total number of bins divided by the first layer histogram.

[0139] When the calculated discreteness of the first layer histogram does not exceed the preset threshold, step S25 is executed to obtain the target image. For example, the brightness value of the original image is first mapped according to the first layer mapping curve to obtain the brightness value of each pixel of the target image. The specific implementation method can be implemented by using the existing technology, which will not be repeated here.

[0140] The key point of this embodiment is that when the calculated discreteness of the first layer histogram exceeds a preset threshold, the second layer histogram needs to be adjusted, and the target image is obtained in combination with the adjustment result of the second layer histogram.

[0141] Specifically, step S26 is first performed to select a number of bins in the first layer histogram, and a second layer histogram is formed according to the selected bins and their corresponding pixels. The number of the selected bins may be two or four.

[0142] If two bins are selected from the first layer histogram: then the selection method is:

[0143] Set the brightness threshold bin1_idx_th on the horizontal axis (bin number) of the first layer histogram, and select the bin with the largest number of pixels as bin1_max1 from the bin on the left side of the brightness threshold bin1_idx_th of the first layer histogram (bin1_idx≤bin1_idx_th); select the bin with the largest number of pixels as bin1_max2 from the bin on the right side of the brightness threshold bin1_idx_th of the first layer histogram (bin1_idx>bin1_idx_th).

[0144] If four bins are selected from the first layer histogram, the selection method is:

[0145] From all the bins on the left side of the brightness threshold bin1_idx_th of the first layer histogram (bin1_idx≤bin1_idx_th), select the bin with the largest number of pixels, that is, bin1_max1, and then select the higher bin from the two bins on both sides of bin1_max1 and merge it with bin1_max1; from all the bins on the right side of the brightness threshold bin1_idx_th of the first layer histogram (bin1_idx>bin1_idx_th), select the bin with the largest number of pixels, that is, bin1_max2, and then select the higher bin from the two bins on both sides of bin1_max2 and merge it with bin1_max2.

[0146] The advantage of merging the four bins into two bins is that the pixel values ​​in the area with uniform brightness in the image are sometimes located near the boundary between two adjacent bins. At this time, if the two adjacent bins are merged together and then processed, it can not only save computing resources but also make the transition of the processed image more coherent and natural.

[0147] The reason why two bins are selected from the first layer histogram is that HDR images usually contain both dark areas and bright areas. Selecting two bins can take into account the statistical accuracy of the two areas.

[0148] Since even if four bins are selected, they are merged into two bins for processing, the following description will be given using two bins as an example (ie, bin1_max1 and bin1_max2).

[0149] bin1_idx represents the number of each bin in the first layer histogram. For example, if the first layer histogram is divided into 128 bins, the value range of bin1_idx is 0 to 127. The brightness threshold bin1_idx_th is a preset parameter that can be determined by experiment or experience. The brightness threshold bin1_idx_th is set to distinguish between dark and bright areas in the HDR image. The area where bin1_idx≤bin1_idx_th belongs to the dark area of ​​the HDR image, and the area where bin1_idx>bin1_idx_th belongs to the bright area of ​​the HDR image.

[0150] In the first layer histogram, within the pixel value range [x1_min, x1_max] represented by bin1_max1, the second layer histogram statistics are performed to obtain the first second layer histogram hist2_1; similarly, within the pixel value range [x2_min, x2_max] represented by bin1_max2, the second layer histogram statistics are performed to obtain the second second layer histogram hist2_2.

[0151] In this way, it is possible to perform fine histogram bin division and statistics (i.e., second-level histogram statistics) on pixel value ranges with dense pixel numbers to improve accuracy; and perform relatively coarse histogram bin division and statistics on pixel value ranges with fewer pixels to reduce the amount of calculation and save storage and computing resources.

[0152] Next, step S27 is executed to adjust the first second-layer histogram hist2_1 and the second second-layer histogram hist2_2 to obtain adjusted second-layer histograms.

[0153] Since the second-layer histogram adjustment is no longer aimed at the pixel value range of the entire HDR image, but at a specific bin in the first-layer histogram, the variation range of the pixel values ​​involved is much smaller than that of the HDR image. Therefore, the first adjustment step for the HDR image in the previous embodiment is not used to adjust the second-layer histogram, but the second adjustment step is directly used to adjust the second-layer histogram.

[0154] After step S27, step S28 is performed:

[0155] First, the input pixel value range of the selected bin is mapped according to the first layer mapping curve to obtain the output pixel value range of the selected bin.

[0156] Substitute the input pixel value range [x1_min, x1_max] represented by bin1_max1 into the mapping result of the first layer mapping curve map_curve1 to obtain the corresponding output pixel value range [y1_min, y1_max]. The same method can be used to obtain the input pixel value range [x2_min, x2_max] represented by bin1_max2 and the corresponding output pixel value range [y2_min, y2_max].

[0157] After the second-layer histogram is adjusted to obtain the adjusted histogram, its normalized cumulative histogram cdf2_1 is calculated, and then the first second-layer mapping curve map_curve2_1 is calculated by formula 7:

[0158] map_curve2_1=y1_min+(y1_max-y1_min)*cdf2_1 formula 10

[0159] Using the same method, the second layer mapping curve map_curve2_2 can be calculated:

[0160] map_curve2_2=y2_min+(y2_max-y2_min)*cdf2_2 formula 11

[0161] After the above steps, three mapping curves have been obtained, namely: the first layer mapping curve map_curve1, the first second layer mapping curve map_curve2_1 and the second second layer mapping curve map_curve2_2.

[0162] Finally, S29 is executed to map the HDR image into an LDR image through the three mapping curves.

[0163] For any pixel in the HDR image, the mapping method of its pixel value x to the pixel value y of the pixel at the same coordinate in the LDR image is:

[0164] If x1_min≤x≤x1_max, then x is mapped to y via map_curve2_1;

[0165] If x2_min≤x≤x2_max, then x is mapped to y via map_curve2_2;

[0166] Otherwise: x is mapped to y via map_curve1.

[0167] Easy to know:

[0168] When x=x1_min, map_curve2_1(x)=map_curve1(x)=y1_min;

[0169] When x=x1_max, map_curve2_1(x)=map_curve1(x)=y1_max;

[0170] When x=x2_min, map_curve2_2(x)=map_curve1(x)=y2_min;

[0171] When x=x2_max, map_curve2_2(x)=map_curve1(x)=y2_max;

[0172] Therefore, at the two points x1_min and x1_max, map_curve2_1(x) is continuous with map_curve1(x); at the two points x2_min and x2_max, map_curve2_2(x) is continuous with map_curve1(x); that is, in the range of x∈[0,Lin] in map_curve1, the segment [x1_min,x1_max] is replaced by the more accurate map_curve2_1(x), and the segment [x2_min,x2_max] is replaced by the more accurate map_curve2_2(x).

[0173] It should be noted that the three mapping curves map_curve1, map_curve2_1 and map_curve2_2 are obtained through cumulative histograms, and the interval of their input independent variables (horizontal coordinates) is equal to the width of their respective histogram bins. Therefore, the mapping from x to y needs to be obtained through interpolation, such as linear interpolation.

[0174] The following is Figure 2 The images shown are processed for schematic illustration.

[0175] Figure 2 The provided image is a 20-bit high dynamic HDR image, and the goal is to map it to a 10-bit low dynamic LDR image. The first-level histogram of the HDR image is counted using 128 bins to obtain the first-level histogram hist1, as shown in Figure 3 shown.

[0176] Perform histogram analysis on the first layer histogram hist1 and set the preset threshold D1_th = 500. According to the simulation experiment, when D1>D1_th, the discrete degree of the first layer histogram hist1 is very high. The discrete degree D1 = 970 calculated by formula 9 is much larger than the threshold D1_th, so the second layer histogram statistics is required. Figure 3It can also be seen that the first layer histogram is highly discrete, and most of the pixels are concentrated in a few bins.

[0177] The brightness threshold bin1_idx_th=20 is set on the horizontal axis (bin number) of the first layer histogram hist1. The setting of the brightness threshold is related to the number of bins in the first layer histogram hist1. The number of bins can be determined through experiments and experience.

[0178] The brightness threshold bin1_idx_th divides the first layer histogram hist1 into two parts: dark area and bright area, and selects one largest bin from each of the dark area and bright area of ​​the first layer histogram hist1: bin1_max1 and bin1_max2. The bin number of bin1_max1 in the first layer histogram is: bin1_max1_idx=0; the bin number of bin1_max2 in the first layer histogram is: bin1_max2_idx=31.

[0179] Since the pixel value range is 0 to 2 20 -1, divided into 128 bins for histogram statistics, and then bin width = 2 20 / 128=8192, it is easy to know that the pixel value range represented by bin1_max1 is [x1_min,x1_max]=[bin1_max1_idx*8192,(bin1_max1_idx+1)*8192]=[0,8192]; the pixel value range represented by bin1_max2 is [x2_min,x2_max]=[253952,262144]. The second-layer histogram statistics are performed in the range of [x1_min,x1_max] and [x2_min,x2_max] respectively, and the first second-layer histogram hist2_1 and the second second-layer histogram hist2_2 are obtained, wherein the first second-layer histogram hist2_1 and the second second-layer histogram hist2_2 are each divided into 128 bins.

[0180] In this example, by adopting the technical solution of this embodiment, only 128*3 bins are needed to achieve relatively fine histogram statistics for most pixels, while the traditional technology needs 128*128=2 to achieve the same accuracy. 14 bin, which requires more than 40 times more storage and computing resources.

[0181] The first layer histogram hist1 is adjusted, and the adjusted first layer histogram hist1-2 is as follows Figure 6 shown.

[0182] Then, the adjusted first layer histogram hist1-2 is used to calculate the cumulative histogram and normalize it to obtain the normalized first layer cumulative histogram cdf1. Then, the first layer mapping curve map_curve1 is calculated by formula 8. In this example, the 20-bit HDR image is mapped to a 10-bit LDR image, so Lout = 2 10 -1=1023,Lin=20 20 -1=1048575. Lout is the maximum available pixel value of the low dynamic LDR image after mapping, and Lin is the maximum available pixel value of the high dynamic HDR image before mapping. For the first layer mapping curve map_curve1, we usually hope that it maps to the largest range, that is, to map to all available pixel values. Therefore, we can know that in formula 7, ymin=0, ymax=Lout=1023. The first layer mapping curve map_curve1 is as follows Figure 8 shown.

[0183] According to the input pixel value range [x1_min,x1_max]=[0,8192] corresponding to bin1_max1, we can substitute it into map_curve1 to get the output pixel value range [y1_min,y1_max]=[0,702] after mapping; similarly, we can get the output pixel value range [y2_min,y2_max]=[814,869] corresponding to [x2_min,x2_max]=[253952,262144].

[0184] Then bin1_max1 and bin1_max2 are evenly divided into 128 bins, and the second-level histogram statistics are performed on them respectively, and the obtained Fig. 9 The first and second layer histograms hist2_1 and Fig.11 The second second layer histogram hist2_2 is shown, and then the second layer histogram adjustment is performed respectively.

[0185] The first second layer histogram hist2_1 is subjected to histogram adjustment. Since the pixel value range represented by the first second layer histogram hist2_1 is no longer highly dynamic, the second adjustment step can be used to perform histogram adjustment.

[0186] The normalized cumulative histogram cdf2_1 is calculated from the adjusted first second-layer histogram. Then the mapping curve map_curve2_1 is calculated, and y1_min, y1_max, and cdf2_1 are substituted into Formula 7:

[0187] map_curve2_1=y1_min+(y1_max-y1_min)*cdf2_1

[0188] The obtained mapping curve map_curve2_1 is as follows Fig.10 The same method can be used to obtain the second layer mapping curve map_curve2_2 corresponding to bin1_max2, as shown in Fig.12 shown.

[0189] After the above steps, three mapping curves are obtained: map_curve1( Figure 8 ), map_curve2_1(such as Fig.10 ), map_curve2_2( Fig.12 ), by mapping the pixel values ​​of the high dynamic HDR image, a low dynamic LDR image can be obtained. For any pixel in the HDR image, the mapping method of its pixel value x to the pixel value y of the pixel at the same coordinate in the LDR image is:

[0190] If x1_min≤x≤x1_max, then x is mapped to y via map_curve2_1;

[0191] If x2_min≤x≤x2_max, then x is mapped to y via map_curve2_2;

[0192] Otherwise: x is mapped to y via map_curve1.

[0193] contrast Fig.13 and Fig.14 The scene is set in a dark room, all light sources except the transmissive light box are turned off, and a transmissive resolution target is inserted into the uniform light exit window of the transmissive light box. The dynamic range of the scene is as high as 116dB. The LDR image obtained by using the technical solution of this embodiment is as follows: Fig.13 As shown, the LDR image obtained by using the prior art is as follows Fig.14 As shown. By comparing the two pictures, it can be seen that the LDR image obtained by adopting the technical solution of this embodiment can not only clearly see most of the content in the bright and dark areas, but also retain a good contrast in the bright area, which lays a good foundation for further processing of the subsequent ISP pipeline. The existing technology uses 512 bins, which exceeds the number of bins in this solution = 128*3, but the dark area is still darker, while the bright area is close to saturation. This solution can be implemented using fewer hardware resources and is easy to output in real time.

[0194] It is worth noting that the technical solution of the present invention is not only applicable to black and white images, but also to color images.

[0195] The method described in the above scheme only takes black and white images as an example, so the expression "pixel value" is always used. However, the technical scheme of the present invention is also applicable to color images, and the method is: extract the brightness component Y of the high dynamic color image RGB (existing technology, such as Y in the YUV color space, L in the CIELAB color space, etc.); apply this scheme to the brightness component Y to obtain the brightness component Y' after dynamic range compression (the expression "brightness value" should be used instead of "pixel value"); according to Y, Y' and the high dynamic color image RGB, obtain the color image RGB' after dynamic range compression (existing technology, such as adjusting the proportional relationship between RGB' and RGB according to the proportional relationship between Y' and Y, which will not be repeated here).

[0196] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the scope defined by the claims.

Claims

1. A histogram adjustment method, characterized in that: include: First adjustment step: The first adjustment step comprises: In the histogram, the number of pixels in each bin exceeding its own limit value is deleted to obtain a first intermediate histogram, and the total number of deleted pixels is calculated to obtain a first total number of deleted pixels; Calculate the total number of pixels of each bin in the first intermediate histogram that does not reach its own limit value to obtain a first remaining total number; In the first intermediate histogram, the first total amount of deletions is added to each bin that has not reached its own limit value according to the distribution ratio to obtain an adjusted histogram; The allocation ratio is: the ratio of the number of pixels corresponding to each bin that has not reached its own limit value to the first remaining total amount.

2. The histogram adjustment method according to claim 1, characterized in that: Also includes: When the adjusted histogram has a bin whose number of pixels exceeds the self-limited value, the first adjustment step is repeatedly performed until the number of executions of the first adjustment step reaches a threshold number of times.

3. The histogram adjustment method according to claim 2, characterized in that: Also includes: When the execution times of the first adjustment step reaches the times threshold, executing the second adjustment step; The second adjustment step comprises: In the histogram, the number of pixels in each bin exceeding its own limit value is deleted to obtain a second intermediate histogram, and the total number of deleted pixels is calculated to obtain a second total number of deleted pixels; In the second intermediate histogram, the second total amount of deletions is evenly distributed to each bin to obtain an adjusted histogram.

4. An image processing method, characterized in that: include: Get the first layer histogram corresponding to the original image; Performing the histogram adjustment method described in any one of claims 1 to 3 on the first layer histogram to obtain an adjusted first layer histogram; generating a first layer mapping curve according to the adjusted first layer histogram; When the discreteness of the first layer histogram does not exceed a preset threshold, mapping the original image according to the first layer mapping curve to obtain a target image; When the discreteness of the first layer histogram exceeds the preset threshold, a plurality of bins are selected from the first layer histogram, and a second layer histogram is formed according to the selected bins and their corresponding pixels; Performing a histogram adjustment on the second layer histogram to obtain an adjusted second layer histogram; generating a second layer mapping curve according to the first layer mapping curve, the adjusted second layer histogram and the selected bin; The original image is mapped according to the first layer mapping curve and the second layer mapping curve to obtain a target image.

5. The image processing method according to claim 4, characterized in that: The step of generating a first layer mapping curve according to the adjusted first layer histogram includes: Calculate the cumulative histogram using the adjusted first-layer histogram to obtain the first-layer cumulative histogram; Normalizing the first-layer cumulative histogram to obtain a normalized first-layer cumulative histogram; The first layer mapping curve is generated according to the normalized first layer cumulative histogram.

6. The image processing method according to claim 5, characterized in that: The formula for generating the first layer mapping curve is: map_curve1 = Lout * cdf1 Among them, cdf1 is the normalized first-layer cumulative histogram, map_curve1 represents the first-layer mapping curve, and Lout is the maximum available pixel value of the target image.

7. The image processing method according to claim 6, characterized in that: Mapping the input pixel value range of the selected bin according to the first layer mapping curve to obtain the output pixel value range of the selected bin; The formula for generating the second layer mapping curve is: map_curve2=y_min+(y_max-y_min)*cdf2 Among them, map_curve2 represents the second layer mapping curve, y_min is the minimum value in the output pixel value range of the selected bin, and y_max is the maximum value in the output pixel value range of the selected bin.

8. The image processing method according to claim 7, characterized in that: The second layer histogram includes: a first second layer histogram and a second second layer histogram; Based on the first layer histogram, among the bins whose brightness values ​​are less than the brightness threshold, at least the bin with the highest number of pixels is selected as the first bin, and histogram statistics are performed on the first bin to form the first second layer histogram; Based on the first layer histogram, among the bins whose brightness values ​​are greater than or equal to the brightness threshold, at least the bin with the highest number of pixels is selected as the second bin, and histogram statistics are performed on the second bin to form the second second layer histogram; A first second-layer mapping curve corresponding to the first second-layer histogram and a second second-layer mapping curve corresponding to the second second-layer histogram are generated.

9. The image processing method according to claim 8, characterized in that: The step of selecting at least the bin with the highest number of pixels as the first bin comprises: The bin with the highest number of pixels and the bin with the higher number of pixels in the two adjacent bins are selected and merged to form the first bin.

10. The image processing method according to claim 8, characterized in that: The step of selecting at least the bin with the highest number of pixels as the second bin comprises: The bin with the highest number of pixels and the bin with the higher number of pixels in the two adjacent bins are selected and merged as the second bin.

11. The image processing method according to claim 8, characterized in that: Mapping the original image according to the first layer mapping curve and the second layer mapping curve to obtain a target image comprises: When the brightness value of a pixel in the original image falls within the brightness range corresponding to the first bin, mapping is performed using the first second-layer mapping curve; When the brightness value of a pixel in the original image falls within the brightness range corresponding to the second bin, mapping is performed using the second second-layer mapping curve; When the brightness value of a pixel in the original image does not fall within the brightness range corresponding to the first bin and the second bin, mapping is performed using the first layer mapping curve.

12. The image processing method according to claim 4, characterized in that: The calculation formula of the discreteness is: Among them, D represents the discreteness, bin(i) represents the number of pixels corresponding to the i-th bin, and k represents the total number of bins divided by the first layer histogram.

13. The image processing method according to claim 4, characterized in that: The step of mapping the original image according to the first layer mapping curve to obtain a target image comprises: A first mapping is performed on the brightness value of the original image according to the first layer mapping curve to obtain the brightness value of each pixel of the target image.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program is executed by a processor to implement the steps of the method according to any one of claims 1 to 3.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program is executed by a processor to implement the steps of the method according to any one of claims 4 to 13.

16. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, wherein: When the processor runs the computer program, the steps of the method according to any one of claims 1 to 3 are performed.

17. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, wherein: When the processor runs the computer program, the steps of the method according to any one of claims 4 to 13 are performed.

Images