Image processing method, chip, electronic device and computer readable storage medium

Abstract

Embodiments of the present application relate to the technical field of image processing, and disclose an image processing method, a chip, an electronic device and a computer readable storage medium. The image processing method comprises: acquiring a tone mapping curve corresponding to a high dynamic range (HDR) image to be processed; selecting a control point in a coordinate system in which the tone mapping curve is located according to a dynamic range of the HDR image; determining a first ordinate corresponding to the abscissa of the control point on the tone mapping curve; if the first ordinate is not in a preset range, adjusting the tone mapping curve so that a second ordinate corresponding to the abscissa of the control point on the adjusted tone mapping curve is in the preset range; and obtaining a low dynamic range (LDR) image corresponding to the HDR image according to the adjusted tone mapping curve, so that the distortion of the LDR image obtained after compression can be avoided while the HDR image is compressed into the LDR image, and thus the LDR image obtained after compression meets the perception of the human eye.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image processing method, a chip, an electronic device, and a computer-readable storage medium. Background Technology

[0002] Due to limitations in physical hardware and economic costs, most display devices currently cannot display or process High Dynamic Range (HDR) images at high speeds. Therefore, tone mapping techniques are typically used to compress HDR images into Low Dynamic Range (LDR) images to suit the display and processing capabilities of most devices. However, while current tone mapping techniques can compress HDR images into LDR images, they often result in distortion of the compressed LDR image, which does not conform to human visual perception. Summary of the Invention

[0003] The purpose of this application is to provide an image processing method, chip, electronic device, and computer-readable storage medium that can compress an HDR image into an LDR image while avoiding distortion of the LDR image obtained after compression, so that the LDR image obtained after compression conforms to the perception of the human eye.

[0004] In a first aspect, embodiments of this application provide an image processing method, comprising: acquiring a tone mapping curve corresponding to a high dynamic range (HDR) image to be processed; selecting a control point in a coordinate system where the tone mapping curve is located according to the dynamic range of the HDR image; determining a first ordinate on the tone mapping curve corresponding to the abscissa of the control point; if the first ordinate is not within a preset range, adjusting the tone mapping curve so that a second ordinate on the adjusted tone mapping curve corresponding to the abscissa of the control point is within the preset range; wherein the preset range is a range of pixel values ​​that conforms to human visual perception; and obtaining a low dynamic range (LDR) image corresponding to the HDR image based on the adjusted tone mapping curve.

[0005] As one possible implementation, the tone mapping curve is generated based on the target luminance histogram of the HDR image. Adjusting the tone mapping curve includes: determining the target frequency to be cropped based on the first ordinate and the preset range; determining the region to be cropped and the region to be allocated in the target luminance histogram; cropping the region to be cropped based on the target frequency and allocating the region to be allocated based on the frequency, obtaining an updated luminance histogram; calculating a cumulative probability distribution curve based on the updated luminance histogram, and using the cumulative probability distribution curve as the adjusted tone mapping curve.

[0006] As one possible implementation, determining the target frequency to be clipped based on the first ordinate and the preset range includes: if the first ordinate is greater than the upper limit of the preset range, then the target frequency to be clipped is calculated using the following formula: num_clip = (y_orig - y_up) * N; if the first ordinate is less than the lower limit of the preset range, then the target frequency to be clipped is calculated using the following formula: num_clip = (y_down - y_orig) * N; where num_clip is the target frequency, y_orig is the first ordinate, y_up is the upper limit, y_down is the lower limit, and N is the total frequency of the target brightness histogram.

[0007] As one possible implementation, the step of frequency clipping the region to be clipped based on the target frequency and frequency allocation of the region to be allocated to obtain an updated luminance histogram includes: determining a clipping height based on the target frequency; clipping the frequency of each Bin interval in the region to be clipped based on the clipping height, and frequency allocation of each Bin interval in the region to be allocated to obtain an updated luminance histogram; the clipping height and the number of Bin intervals in the region to be clipped satisfy the following relationship:

[0008] (K-1)*n≤K*n≤(K+1)*n

[0009] Where K is the cutting height, and n is the number of Bin intervals in the area to be cut.

[0010] As one possible implementation, the decrease in the histogram height corresponding to each interval Bin in the region to be clipped after frequency clipping is equal to the clipping height; the sum of the increases in the histogram height corresponding to each interval Bin in the region to be allocated after frequency allocation is equal to the target frequency.

[0011] As one possible implementation, the control points include m control points arranged in ascending order of their abscissas. Determining the region to be clipped and the region to be allocated in the target brightness histogram includes: when the first ordinate of the i-th control point on the tone mapping curve is greater than the upper limit of the preset range, then the abscissa range of the region to be clipped in the target brightness histogram is determined to be (a, b), and the abscissa range of the region to be allocated in the target brightness histogram is determined to be (c, d); where b is the abscissa x of the i-th control point. i c is x iAdd 1; when i = 1, a is 0, and d is the x-coordinate of the second control point (x2); when i = m, a is the x-coordinate of the (m-1)th control point (x). m-1 Add 1, where d is the maximum x-coordinate of the target brightness histogram. end When 1 < i < m, a is the x-coordinate of the (i-1)th control point. i-1 Add 1, where d is the x-coordinate of the (i+1)th control point. i+1 Alternatively, if the first ordinate of the i-th control point on the tone mapping curve is less than the lower limit of the preset range, then the abscissa range of the region to be clipped in the target brightness histogram is determined to be (e, f), and the abscissa range of the region to be allocated in the target brightness histogram is determined to be (g, h); where e is the abscissa x of the i-th control point. i Add 1, h is x i When i = 1, f is the x-coordinate of the second control point (x2), and g is 0; when i = m, f is the maximum x-coordinate of the target brightness histogram. end g is the x-coordinate of the (m-1)th control point. m-1 Add 1; when 1 < i < m, f is the x-coordinate of the (i+1)th control point. i+1 g is the x-coordinate of the (i-1)th control point. i-1 Add 1.

[0012] As one possible implementation, the frequency allocation for each interval Bin in the region to be allocated includes: calculating the average frequency allocated to each interval Bin in the region to be allocated using the following formula: in, The average frequency is given, where num_clip is the target frequency, and M is the total number of intervals Bin in the region to be allocated; the average frequency is added to the frequency corresponding to each interval Bin in the region to be allocated.

[0013] As one possible implementation, the frequency allocation for each interval Bin in the region to be allocated includes: calculating the frequency k allocated to the u-th interval Bin in the region to be allocated using the following formula. u :

[0014]

[0015] Where num_clip is the target frequency, M is the total number of intervals Bin in the region to be allocated, and d u Let d be the width of the u-th interval Bin in the region to be allocated. jThe width of the j-th interval Bin in the region to be allocated, 1≤u≤M, 1≤j≤M; the frequency corresponding to the u-th interval Bin in the region to be allocated is increased by k. u .

[0016] As one possible implementation, the HDR image is synthesized based on R LDR images with an exposure ratio of Ex, wherein the dynamic range of the LDR images with an exposure ratio of Ex is D. L The dynamic range of the HDR image is based on Ex, R, D. L The step of selecting control points in the coordinate system where the tone mapping curve is located based on the dynamic range of the HDR image includes: selecting m control points in the coordinate system where the tone mapping curve is located based on the dynamic range of the HDR image; where m = R-1, and when the m control points are arranged in ascending order of their abscissas, the abscissa of the first control point is D. L The x-coordinate of the i-th control point is D. L *Ex i , 1 < i ≤ m.

[0017] As one possible implementation, the preset range is determined based on a target dynamic range, which is the specified dynamic range to which the HDR image needs to be compressed.

[0018] As one possible implementation, the target luminance histogram of the HDR image is obtained as follows: based on the luminance value of each pixel in the HDR image, the original luminance histogram of the HDR image is obtained; the original luminance histogram is low-pass filtered to obtain a filtered luminance histogram; and based on the filtered luminance histogram, the target luminance histogram of the HDR image is obtained.

[0019] As one possible implementation, obtaining the target luminance histogram of the HDR image based on the filtered luminance histogram includes: obtaining the target luminance histogram of the HDR image using the following formula:

[0020] Hist_new=Hist_LPF*α+Hist*(1-α);

[0021] Wherein, Hist_new is the target brightness histogram, Hist_LPF is the filtered brightness histogram, Hist is the original brightness histogram, and α is a preset filtering intensity coefficient.

[0022] As one possible implementation, obtaining the original brightness histogram of the HDR image based on the brightness value of each pixel in the HDR image includes: obtaining the original brightness histogram of the HDR image based on the brightness value of each pixel in the HDR image and a preset interval Bin of unequal width; wherein, the interval Bin, which represents a larger brightness value, has a larger width.

[0023] As one possible implementation, the tone mapping curve includes: a global tone mapping curve and / or a local tone mapping curve for each pixel block in the HDR image; the first ordinate includes: the ordinate corresponding to the abscissa of the control point on the global tone mapping curve and / or the ordinate corresponding to the abscissa of the control point on the local tone mapping curve; the adjusted tone mapping curve includes: an adjusted global tone mapping curve and / or an adjusted local tone mapping curve.

[0024] As one possible implementation, when the adjusted tone mapping curve includes an adjusted global tone mapping curve and an adjusted local tone mapping curve, obtaining the low dynamic range (LDR) image corresponding to the HDR image based on the adjusted tone mapping curve includes: obtaining a global brightness image based on the adjusted global tone mapping curve; obtaining a local brightness image based on the adjusted local tone mapping curve; and fusing the global brightness image and the local brightness image to obtain the low dynamic range (LDR) image corresponding to the HDR image.

[0025] As one possible implementation, obtaining a local brightness image based on the adjusted local tone mapping curve includes: for the Lth pixel block in the HDR image, determining the pixel blocks in the neighborhood centered on the Lth pixel block; where 1≤L≤T, and T is the total number of pixel blocks in the HDR image; determining the weights corresponding to the pixel blocks in the neighborhood based on the brightness difference between the pixel blocks in the neighborhood and the Lth pixel block; smoothing the adjusted local tone mapping curve of the Lth pixel block based on the weights corresponding to the pixel blocks in the neighborhood to obtain a smoothed local tone mapping curve of the Lth pixel block; and obtaining a local brightness image based on the smoothed local tone mapping curves of T pixel blocks in the HDR image.

[0026] As one possible implementation, the step of smoothing the adjusted local tone mapping curve of the Lth pixel block according to the weights corresponding to the pixel blocks in the neighborhood, to obtain the smoothed local tone mapping curve of the Lth pixel block, includes: obtaining the smoothed local tone mapping curve of the Lth pixel block using the following formula:

[0027]

[0028] Among them, LUT new Let LUT be the smoothed local tone mapping curve of the Lth pixel block. q For the q-th pixel block in the neighborhood, w is the adjusted local tone mapping curve. q Let be the weight of the q-th pixel block, and Q be the total number of pixel blocks in the neighborhood.

[0029] As one possible implementation, after obtaining the low dynamic range (LDR) image corresponding to the HDR image based on the adjusted tone mapping curve, the method further includes: performing contrast enhancement on the LDR image corresponding to the HDR image based on a preset contrast enhancement curve to obtain a contrast-enhanced LDR image.

[0030] As one possible implementation, the step of enhancing the contrast of the LDR image corresponding to the HDR image according to a preset contrast enhancement curve to obtain a contrast-enhanced LDR image includes: normalizing the brightness values ​​of each pixel in the LDR image; mapping the normalized brightness values ​​of each pixel in the LDR image according to the preset contrast enhancement curve to output a brightness mapping image; and restoring the dynamic range of the brightness mapping image to the dynamic range before normalization to obtain the contrast-enhanced LDR image.

[0031] As one possible implementation, the contrast enhancement curve is a Gamma curve, and the parameter γ of the Gamma curve is determined as follows:

[0032]

[0033] Wherein, avgL is the average brightness value of the LDR image, TH1_AVGL is the first preset threshold, TH2_AVGL is the second preset threshold, and TH2>TH1≥1.

[0034] As one possible implementation, the slope of the contrast enhancement curve is less than 1 at positions where the brightness value is less than a preset brightness value.

[0035] Secondly, embodiments of this application also provide a chip located within an electronic device and connected to a memory within the electronic device. The memory stores instructions executable by the chip, which are then executed by the chip to enable the chip to perform the image processing method described above.

[0036] Thirdly, embodiments of this application also provide an electronic device, including: the chip described above, and a memory connected to the chip.

[0037] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the image processing method described above.

[0038] In this embodiment, by adjusting the tone mapping curve corresponding to the HDR image, it is ensured that the ordinate of the control point on the adjusted tone mapping curve is within a preset range. Since the preset range is the range of pixel values ​​that conform to human visual perception, the LDR image corresponding to the HDR image obtained based on the adjusted tone mapping curve can conform to human visual perception, ensuring that the LDR image observed by the human eye presents a better effect. Therefore, while compressing the HDR image into an LDR image, distortion of the LDR image obtained after compression can be avoided, thereby making the LDR image obtained after compression conform to human visual perception. Attached Figure Description

[0039] One or more embodiments are illustrated by way of example with reference to the accompanying drawings, and these illustrative descriptions do not constitute a limitation on the embodiments.

[0040] Figure 1 This is a flowchart of an image processing method mentioned in this embodiment of the application;

[0041] Figure 2 This is a flowchart illustrating a method for obtaining the target brightness histogram of an HDR image as mentioned in this embodiment of the application;

[0042] Figure 3 This is a schematic diagram of the original brightness histogram obtained based on the equal-width interval Bin mentioned in this embodiment of the application;

[0043] Figure 4 This is a schematic diagram of the original brightness histogram obtained based on the Bin intervals of unequal widths mentioned in this embodiment of the application;

[0044] Figure 5 This is a schematic diagram comparing the original luminance histogram and the filtered luminance histogram mentioned in this embodiment of the application;

[0045] Figure 6 This is a schematic diagram of the tone mapping curve generated based on the target brightness histogram mentioned in this embodiment of the application;

[0046] Figure 7 This is a schematic diagram of the adjusted tone mapping curve generated based on the updated luminance histogram mentioned in this embodiment of the application;

[0047] Figure 8 This is a flowchart illustrating one implementation method for adjusting the tone mapping curve mentioned in this embodiment of the application;

[0048] Figure 9 This is a schematic diagram of curve adjustment based on control point cp1 mentioned in this embodiment of the application;

[0049] Figure 10 This is a schematic diagram of curve adjustment based on control point cp2 mentioned in this embodiment of the application;

[0050] Figure 11 This is a flowchart of one implementation of step 105 mentioned in this embodiment of the application;

[0051] Figure 12 This is a flowchart of one implementation of step 1052 mentioned in this embodiment of the application;

[0052] Figure 13 This is a schematic diagram of the pixel block mentioned in this embodiment of the application;

[0053] Figure 14 This is a schematic diagram of a relationship curve between the brightness difference and the weight mentioned in this embodiment of the application;

[0054] Figure 15 This is a schematic diagram of the Gamma curve mentioned in this embodiment of the application;

[0055] Figure 16 This is a schematic diagram of the spline curve mentioned in this embodiment of the application;

[0056] Figure 17 This is a schematic diagram of the sigmoid curve mentioned in this embodiment of the application;

[0057] Figure 18 This is a flowchart of the process described in this embodiment of the application, which describes how to enhance the contrast of an LDR image according to a preset contrast enhancement curve to obtain an LDR image with enhanced contrast.

[0058] Figure 19 This is a schematic diagram of the image processing method mentioned in this embodiment of the application;

[0059] Figure 20 This is a schematic diagram of the structure of the electronic device mentioned in this embodiment of the application. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the various embodiments of this application will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been presented in the various embodiments of this application to enable readers to better understand this application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in this application can be implemented. The division of the various embodiments below is for the convenience of description and should not constitute any limitation on the specific implementation of this application. The various embodiments can be combined with and referenced by each other without contradiction.

[0061] This application relates to an image processing method applied to an electronic device, which can be an image processing device. The image processing in this embodiment mainly refers to using tone mapping technology to compress an HDR image into an LDR image, so as to preserve the details in the HDR image while avoiding distortion in the LDR image during compression. The pixel value range of an HDR image is higher than the pixel value range that a specific device can process; the pixel value range of an HDR image is generally 16 bits (0-65535), 14 bits (0-16383), 12 bits (0-4095), etc. The pixel value range of an LDR image is lower than the pixel value range of an HDR image; the pixel value range of an LDR image is generally 10 bits (0-1023), 8 bits (0-255), etc.

[0062] The flowchart of the image processing method in this embodiment can be found in [reference]. Figure 1 ,include:

[0063] Step 101: Obtain the tone mapping curve corresponding to the HDR image to be processed.

[0064] Step 102: Select control points in the coordinate system where the tone mapping curve is located, based on the dynamic range of the HDR image.

[0065] Step 103: Determine the first ordinate of the control point's x-coordinate on the tone mapping curve.

[0066] Step 104: If the first ordinate is not within the preset range, adjust the tone mapping curve so that the second ordinate corresponding to the x-coordinate of the control point on the adjusted tone mapping curve is within the preset range.

[0067] Step 105: Obtain the LDR image corresponding to the HDR image based on the adjusted tone mapping curve.

[0068] In this embodiment, by adjusting the tone mapping curve corresponding to the HDR image, it is ensured that the horizontal coordinate of the control point and the vertical coordinate corresponding to the adjusted tone mapping curve are within a preset range. Since the preset range is the range of pixel values ​​that conform to human visual perception, the LDR image corresponding to the HDR image obtained based on the adjusted tone mapping curve can conform to human visual perception, ensuring that the LDR image observed by the human eye presents a better effect. Therefore, while compressing the HDR image into an LDR image, distortion of the LDR image obtained after compression can be avoided, thereby making the LDR image obtained after compression conform to human visual perception.

[0069] The following is a detailed description of the implementation details of the image processing method in this embodiment. The following content is only for the convenience of understanding and is not necessary for implementing this solution.

[0070] In step 101, the HDR image to be processed acquired by the electronic device can be a Raw Bayer image or an RGB image. The electronic device can first acquire the target brightness histogram of the HDR image, and then generate a tone mapping curve based on the target brightness histogram. Specifically, the tone mapping curve can be obtained based on the histogram equalization principle.

[0071] The target luminance histogram is used to characterize the luminance value distribution of each pixel in an HDR image. Specifically, electronic devices can statistically analyze pixels within a preset luminance value range based on the luminance value of each pixel in the HDR image to obtain luminance value range-frequency information, and then draw the original luminance histogram based on this luminance value range-frequency information. A luminance value range can be understood as an interval (Bin), which is the luminance value interval.

[0072] In this embodiment, a target brightness histogram can be obtained based on the original brightness histogram. The target brightness histogram can be the original brightness histogram, or it can be a brightness histogram obtained after further processing of the original brightness histogram.

[0073] In some embodiments, the method for obtaining the target brightness histogram of an HDR image can be found in [reference needed]. Figure 2 The flowchart in the document includes:

[0074] Step 201: Obtain the original brightness histogram of the HDR image based on the brightness value of each pixel in the HDR image.

[0075] Step 202: Perform a low-pass filter on the original brightness histogram to obtain the filtered brightness histogram.

[0076] Step 203: Obtain the target brightness histogram of the HDR image based on the filtered brightness histogram.

[0077] In step 201, the electronic device can statistically analyze the pixels within a preset brightness value range based on the brightness value of each pixel in the HDR image, obtain brightness value range-frequency information, and draw an original brightness histogram based on this brightness value range-frequency information. The preset brightness value range can be set according to actual needs, aiming to ensure that the original brightness histogram reflects the number of pixels in different brightness value ranges in the HDR image. The preset brightness value range can also be understood as a preset brightness level.

[0078] In some embodiments, step 201 can be implemented by: obtaining the original luminance histogram of the HDR image based on the luminance value of each pixel in the HDR image and a preset equal-width interval Bin. For example, a schematic diagram of the original luminance histogram obtained based on the equal-width interval Bin can be found in [reference needed]. Figure 3 The horizontal axis of the original brightness histogram represents the brightness value, and the vertical axis represents the frequency. Figure 3 It reflects the number of pixels distributed across different brightness value ranges, and the width of each brightness value range is equal.

[0079] In some embodiments, step 201 can be implemented as follows: obtaining the original brightness histogram of the HDR image based on the brightness value of each pixel in the HDR image and a preset interval Bin of unequal width; wherein the interval Bin representing the larger the brightness value has a larger width. For example, a schematic diagram of the original brightness histogram obtained based on the interval Bin of unequal width can be found in [reference needed]. Figure 4 The widths of different brightness value ranges are not equal, and the width of the brightness value range increases as the brightness value increases. For example, the widths of the Bin ranges with unequal widths can be as follows: 2 0 ,2 1 ,2 2 ,...,2 n However, this embodiment does not specifically limit the width of the Bin interval with unequal width, and other unequal widths can also be used.

[0080] This embodiment takes into account that, according to Weber's law, the human eye's response to changes in brightness is non-linear. Traditional methods for obtaining brightness histograms involve transforming image grayscale using a non-linear function (e.g., a logarithmic function) and then performing histogram statistics on the transformed image. However, this embodiment proposes a histogram statistics method based on intervals of unequal width (Bin) to replace the non-linear transformation of image grayscale, thus avoiding the time-consuming non-linear transformation operation. Therefore, this embodiment's method of obtaining the original brightness histogram based on intervals of unequal width (Bin) can effectively enhance the image while also shortening image processing time.

[0081] In step 202, the low-pass filtering (LPF) of the original luminance histogram by the electronic device can be understood as histogram smoothing. The filtered luminance histogram can be understood as the smoothed luminance histogram. Histogram smoothing is achieved by filtering the original luminance histogram using the LPF. Specifically, the filtered luminance histogram is obtained by weighted averaging the frequencies corresponding to each bin and the frequencies corresponding to the bins in its surrounding bins. The weight of each bin can be determined by the LPF kernel. For example, a comparison diagram of the original luminance histogram and the filtered luminance histogram can be found in [reference needed]. Figure 5 , Figure 5 The weights of the five Bin intervals are 1, 2, 2, 2, and 1, respectively.

[0082] In step 203, the electronic device can directly use the filtered luminance histogram as the target luminance histogram of the HDR image. Alternatively, the filtered luminance histogram can be further processed to obtain the target luminance histogram. For example, the filtered luminance histogram can be processed based on the original luminance histogram and a preset filtering intensity coefficient to obtain the target luminance histogram.

[0083] In this embodiment, by performing low-pass filtering on the original luminance histogram, i.e., histogram smoothing on the original luminance histogram, it is beneficial to avoid the problem of image distortion caused by excessively large slope of the tone mapping curve generated based on the original luminance histogram.

[0084] In some embodiments, step 203 can be implemented by obtaining the target brightness histogram of the HDR image using the following formula:

[0085] Hist_new=Hist_LPF*α+Hist*(1_α);

[0086] Here, Hist_new is the target luminance histogram, Hist_LPF is the filtered luminance histogram, Hist is the original luminance histogram, and α is the preset filtering intensity coefficient. α belongs to the interval [0,1]. The specific value of α can be set according to actual needs. The larger α is, the higher the LPF intensity, and the lower the contrast will be. Conversely, the lower the α is, the higher the contrast will be, but the distortion problem will be more obvious.

[0087] In this embodiment, the intensity of LPF can be controlled by the above formula, thereby taking into account both image contrast and image distortion, so that the target brightness histogram can maintain a certain contrast without significant distortion.

[0088] In step 102, the electronic device selects control points in the coordinate system containing the tone mapping curve based on the dynamic range of the HDR image. These selected control points are used to determine whether the tone mapping curve needs adjustment. The horizontal coordinate of the control points can be determined based on the dynamic range of the HDR image, while the vertical coordinate can be manually set by those skilled in the art to conform to human visual perception.

[0089] In some embodiments, the HDR image is synthesized based on R LDR images with an exposure ratio of Ex, and the dynamic range of the LDR images with an exposure ratio of Ex is D. L The dynamic range of HDR images is based on Ex, R, D. L Determined, step 102 above can be implemented as follows: Based on the dynamic range of the HDR image, select m control points in the coordinate system where the tone mapping curve is located; where m = R-1, and when the m control points are arranged in ascending order of their abscissas, the abscissa of the first control point is D. L The x-coordinate of the i-th control point is D. L *Ex i , 1 < i ≤ m.

[0090] In practice, the image sensor sets exposure parameters based on the brightness of the scene being photographed, resulting in R low dynamic range (LDR) images. The exposure ratio of these R LDR images is Ex, which refers to the ratio of exposure times for different LDR images. The sensor then uses the exposure ratio to generate R images with a dynamic range of D. L A high dynamic range image is synthesized from low dynamic range (LDR) images. Therefore, Ex, R, D L The dynamic range of a high dynamic range (HDR) image is determined. The purpose of this embodiment is to compress a synthesized HDR image into a low dynamic range (LDR) image. When the above Ex, R, D... L Once the three parameters are determined, control points can be selected using these parameters. The specific selection method is as follows: the number of control points equals R-1. Assuming R=3, then the number of control points is 2. The x-coordinate of the control points is determined based on Ex and D. L The calculated range of the ordinate is manually set. For example, the x-coordinate of the first control point is x1 = D. L The x-coordinate of the second control point is x2 = D L *Ex 2 The x-coordinate of the m-th control point is x m =D L *Ex m .

[0091] Assuming the ordinate ranges of the first and second control points mentioned above are represented as [y1_down, y1_up] and [y2_down, y2_up] respectively, the ordinate ranges of different control points can be the same or different. Specifically, they can be manually set by those skilled in the art according to actual needs. The manually set ordinate range can be a range of pixel values ​​that conforms to human visual perception. For example, [y1_down, y1_up] can be [0.4*D, 0.7*D], and [y2_down, y2_up] can be [0.8*D, 0.9*D], where D is the specified dynamic range to which the HDR image needs to be compressed. The above manually set ordinate ranges can be obtained through prior data experiments. The determination of prior data can be achieved by adjusting the ordinates of the control points, specifically by adjusting the coefficients before D in the above ordinate ranges, thereby ensuring the best effect observed by the human eye, so that the image seen by the human eye is not distorted.

[0092] For example, you can refer to Figure 6 ,pass Figure 6 It can be seen that the tone mapping curve generated based on the target brightness histogram uses cp1 and cp2 as control points in the coordinate system of the tone mapping curve. It should be noted that... Figure 6 The shape of the target luminance histogram is shown in the coordinate system of the tone mapping curve only for the purpose of illustrating the relationship between the target luminance histogram and the tone mapping curve. It can be understood that the horizontal axis of the target luminance histogram and the tone mapping curve have the same meaning; the vertical axis of the target luminance histogram represents the frequency, and the vertical axis of the tone mapping curve represents the pixel value of the mapped LDR.

[0093] In step 103, the electronic device can calculate the first ordinate of the control point's abscissa on the tone mapping curve. For example, the tone mapping curve can be represented as a tone mapping function. The value of the control point's abscissa can be substituted into the tone mapping function to calculate the mapping value, which is the first ordinate.

[0094] In step 104, the electronic device can determine whether the first ordinate is within a preset range, where the preset range is a range of pixel values ​​that conform to human visual perception. The preset range can be set by those skilled in the art based on actual needs, provided that the pixel value range conforms to human visual perception. The preset range can be understood as a preset range of ordinates for a control point; the preset ranges set for different control points can be the same or different.

[0095] In some embodiments, the preset range is determined based on a target dynamic range, which is the specified dynamic range to which the HDR image needs to be compressed. (See reference...) Figure 6Let the ordinate ranges of control points cp1 and cp2 be represented as [y1_down, y1_up] and [y2_down, y2_up], respectively. Optionally, [y1_down, y1_up] can be [0.4*D, 0.7*D], and [y2_down, y2_up] can be [0.8*D, 0.9*D], where D is the specified dynamic range to which the HDR image needs to be compressed. The above-mentioned artificially set ordinate range can be obtained through prior data experiments. The determination of the prior data can be achieved by adjusting the ordinates of the control points, specifically by adjusting the coefficients before D in the above ordinate range, thereby ensuring the best effect observed by the human eye and preventing image distortion.

[0096] If the first ordinate is determined to be outside the preset range, the tone mapping curve is adjusted so that the second ordinate corresponding to the control point's abscissa on the adjusted tone mapping curve falls within the preset range. The adjusted tone mapping curve can be represented as an adjusted tone mapping function. Substituting the control point's abscissa into the adjusted tone mapping function, the mapped value, i.e., the second ordinate, is calculated and falls within the preset range.

[0097] This embodiment obtains an updated luminance histogram by frequency cropping and frequency allocation of the target luminance histogram, and then generates an adjusted tone mapping curve based on the updated luminance histogram. For example, a schematic diagram of the adjusted tone mapping curve generated based on the updated luminance histogram can be found in [reference needed]. Figure 7 The dashed line represents the tone mapping curve before adjustment, and the solid line represents the tone mapping curve after adjustment. Figure 7 After adjusting the tone mapping curves of control points cp1 and cp2, it can be assumed that the second ordinate of the x-coordinates of cp1 and cp2 on the adjusted tone mapping curves falls within a preset range. Combined with... Figure 6 and Figure 7 It can also be seen that, Figure 7 The updated brightness histogram in the image is compared to... Figure 6 In the target brightness histogram, the frequencies corresponding to the first 7 Bin intervals are all reduced, while the frequencies corresponding to the 8th to 12th Bin intervals are increased. The increased portion is shown in a dashed box.

[0098] In some embodiments, the implementation of adjusting the tone mapping curve can be found in [reference needed]. Figure 8 ,include:

[0099] Step 1041: Determine the target frequency to be clipped based on the first vertical coordinate and the preset range.

[0100] Step 1042: Determine the region to be clipped and the region to be allocated in the target brightness histogram.

[0101] Step 1043: Based on the target frequency, perform frequency clipping on the region to be clipped and frequency allocation on the region to be allocated to obtain the updated brightness histogram.

[0102] Step 1044: Calculate the cumulative probability distribution curve based on the updated luminance histogram, and use the cumulative probability distribution curve as the adjusted tone mapping curve.

[0103] In step 1041, the preset range has an upper limit and a lower limit. When the first ordinate is greater than the upper limit, the target frequency to be clipped can be calculated based on the difference between the first ordinate and the upper limit. When the first ordinate is less than the lower limit, the target frequency to be clipped can be calculated based on the difference between the first ordinate and the lower limit.

[0104] In some embodiments, if the first ordinate is greater than the upper limit of a preset range, the target frequency to be clipped can be calculated using a formula:

[0105] num_clip = (y_orig - y_up) * N;

[0106] If the first ordinate is less than the lower limit of the preset range, the target frequency to be clipped can be calculated using the following formula:

[0107] num_clip=(y_down-y_orig)*N;

[0108] Where num_clip is the target frequency, y_orig is the first ordinate, y_up is the upper limit value, y_down is the lower limit value, and N is the total frequency of the target brightness histogram.

[0109] In step 1042, the electronic device can determine the region to be clipped and the region to be allocated in the target brightness histogram based on the abscissa of the control point. The region to be clipped can be understood as the area in the target brightness histogram whose height needs to be reduced; reducing height can also be understood as reducing frequency. The region to be allocated can be understood as the area in the target brightness histogram whose height needs to be increased; increasing height can also be understood as increasing frequency.

[0110] In some embodiments, the control points include m control points arranged in ascending order of their x-coordinates. Step 1042 can be implemented as follows:

[0111] When the first ordinate of the i-th control point on the tone mapping curve is greater than the upper limit of the preset range, the x-coordinate range of the region to be clipped in the target luminance histogram is determined to be (a, b), and the x-coordinate range of the region to be allocated in the target luminance histogram is determined to be (c, d). Here, b is the x-coordinate of the i-th control point. i c is xi Add 1.

[0112] When i = 1, a is 0, and d is the x-coordinate of the second control point (x2).

[0113] When i = m, a is the x-coordinate of the (m-1)th control point. m-1 Add 1, where d is the maximum x-coordinate of the target brightness histogram. end .

[0114] When 1 < i < m, a is the x-coordinate of the (i-1)th control point. i-1 Add 1, where d is the x-coordinate of the (i+1)th control point. i+1 .

[0115] or,

[0116] When the first ordinate of the i-th control point on the tone mapping curve is less than the lower limit of a preset range, the x-coordinate range of the region to be clipped in the target luminance histogram is determined to be (e, f), and the x-coordinate range of the region to be allocated in the target luminance histogram is determined to be (g, h). Here, e is the x-coordinate of the i-th control point. i Add 1, h is x i .

[0117] When i = 1, f is the x-coordinate of the second control point, x2, and g is 0;

[0118] When i = m, f is the maximum x-coordinate of the target brightness histogram. end g is the x-coordinate of the (m-1)th control point. m-1 Add 1;

[0119] When 1 < i < m, f is the x-coordinate of the (i+1)th control point. i+1 g is the x-coordinate of the (i-1)th control point. i-1 Add 1.

[0120] To facilitate understanding of the above embodiments, a specific example is given below:

[0121] Assuming m = 2, the two control points arranged in ascending order of their x-coordinates are cp1 and cp2. The x-coordinate of cp1 is x1, and the y-coordinate range is set to [y1_down, y1_up]. The x-coordinate of cp2 is x2, and the y-coordinate range is set to [y2_down, y2_up].

[0122] refer to Figure 9 ,for Figure 9For control point cp1, if the x-coordinate of cp1 on the tone mapping curve, specifically the first y-coordinate y1_orig, is greater than y1_up, then the x-coordinate range of the region to be clipped is (0, x1), the x-coordinate range of the region to be allocated is (x1+1, x2), and the target frequency num_clip = (y2_orig - y2_up) * N. If the x-coordinate of cp1 on the tone mapping curve, specifically the first y-coordinate y1_orig, is less than y1_down, then the x-coordinate range of the region to be clipped is (x1+1, x2), the x-coordinate range of the region to be allocated is (0, x1), and the target frequency num_clip = (y1_down - y1_orig) * N. Here, x1 can represent the upper limit of the current interval Bin where the x-coordinate of cp1 is located, and x1+1 represents the next interval Bin.

[0123] refer to Figure 10 ,for Figure 10 If the x-coordinate of control point cp2 in the tone mapping curve is less than y2_down, then the x-coordinate range of the region to be cropped is (x2+1, x...). end The x-coordinate range of the region to be allocated is (x1+1, x2), and the target frequency num_clip = (y2_down - y2_orig) * N. If the x-coordinate of cp2 on the tone mapping curve, specifically the first y-coordinate y2_orig, is greater than y2_up, then the x-coordinate range of the region to be clipped is (x1+1, x2), and the x-coordinate range of the region to be allocated is (x2+1, x...). end The target frequency num_clip = (y2_orig - y2_up) * N. Here, x2 can represent the upper limit of the current interval Bin where the x-coordinate of cp2 is located, and x2+1 represents the next interval Bin.

[0124] In step 1043, frequency clipping can be understood as follows: after the region to be clipped is clipped, the sum of the frequencies corresponding to each interval Bin in the region to be clipped is the target frequency. Frequency allocation can be understood as follows: the sum of the frequencies corresponding to each interval Bin in the region to be allocated is the target frequency.

[0125] In some embodiments, step 1043 can be implemented as follows: Determine the clipping height based on the target frequency. Based on the clipping height, clip the frequency of each Bin interval in the region to be clipped, and allocate the frequency of each Bin interval in the region to be allocated, to obtain an updated luminance histogram. The clipping height and the number of Bin intervals in the region to be clipped satisfy the following relationship:

[0126] (K-1)*n≤K*n≤(K+1)*n

[0127] Where K is the clipping height and n is the number of Bin intervals in the region to be clipped.

[0128] In some embodiments, the clipping height K for each Bin interval in the region to be clipped can be determined based on the target frequency and a binary search algorithm. The clipping height is the frequency at which each Bin interval in the region to be clipped is clipped; that is, the clipping height K can be understood as the frequency to be clipped. The sum of the clipping heights of all Bin intervals in the region to be clipped may be equal to or not exactly equal to the target frequency. For example, the sum of the clipping heights of all Bin intervals in the region to be clipped may be close to but not exactly equal to the target frequency. In this case, if the determined K satisfies the aforementioned inequality, frequency clipping and allocation can still be performed based on the K value, ensuring that even if the sum of the clipping heights of all Bin intervals in the region to be clipped is not equal to the calculated target frequency, the required frequency clipping and allocation can still be completed.

[0129] In some embodiments, the decrease in the histogram height corresponding to each interval Bin in the cropped region after frequency clipping is equal to the clipping height. The sum of the increases in the histogram height corresponding to each interval Bin in the assigned region after frequency allocation is equal to the target frequency.

[0130] In some embodiments, step 1043, which involves frequency allocation for each interval Bin in the region to be allocated, includes calculating the average frequency allocated to each interval Bin in the region to be allocated using the following formula: in, The average frequency is calculated as follows: num_clip is the target frequency, and M is the total number of intervals (Bin) in the region to be allocated. The average frequency is added to the frequency corresponding to each interval (Bin) in the region to be allocated. In other words, in this embodiment, the target frequency is used as the total frequency to be allocated, and this total frequency is evenly distributed to each interval (Bin) in the region to be allocated.

[0131] For example, refer to Figure 9 or Figure 10 The Bin intervals in the region to be allocated include the 8th to 12th Bin intervals, and the frequencies corresponding to the 8th to 12th Bin intervals have all increased. Increased frequencies are indicated by dashed boxes. (Reference) Figure 9 The Bin interval in the area to be clipped includes the 1st to 7th Bin intervals, and the frequencies corresponding to the 1st to 7th Bin intervals are all reduced by K / 7.

[0132] In some embodiments, step 1043, which involves frequency allocation for each interval Bin in the region to be allocated, includes calculating the frequency k allocated to the u-th interval Bin in the region to be allocated using the following formula. u :

[0133]

[0134] Where num_clip is the target frequency, M is the total number of intervals Bin in the region to be allocated, and d u Let d be the width of the u-th interval Bin in the region to be assigned. j The width of the j-th interval Bin in the region to be allocated, 1≤u≤M, 1≤j≤M; the frequency corresponding to the u-th interval Bin in the region to be allocated is increased by k. u In other words, in this embodiment, the target frequency is used as the total frequency to be allocated, and this total frequency is weighted and allocated to each interval Bin in the region to be allocated.

[0135] In step 1044, the updated luminance histogram is the luminance histogram obtained after frequency clipping and frequency allocation. Based on the updated luminance histogram, the cumulative probability distribution curve can be calculated, and finally, the cumulative probability distribution curve is used as the adjusted tone mapping curve.

[0136] In some embodiments, multiple control points are selected, and the tone mapping curve can be adjusted sequentially based on each control point, ultimately ensuring that the ordinates of the x-coordinates of the multiple control points on the adjusted tone mapping curve are all within a preset range. (Reference) Figure 9 and Figure 10 , Figure 9 First, based on control point cp1, the tone mapping curve (denoted as curve 1) is adjusted to obtain the adjusted tone mapping curve (denoted as curve 2) passing through control point cp1. Then, refer to... Figure 10 Using curve 2 as a reference, if the first ordinate of the x-coordinate of cp2 on curve 2 is not within the preset range, then curve 2 is adjusted to obtain the tone mapping curve after cp2 (denoted as curve 3). Curve 3 is the final tone mapping curve obtained after adjustment. The second ordinate of the x-coordinates of control points cp1 and cp2 on curve 3 are both within the preset range, that is, control points cp1 and cp2 are both located on curve 3.

[0137] In step 105, the electronic device performs tone mapping on the HDR image according to the adjusted tone mapping curve to compress the HDR image and obtain the corresponding LDR image. Through tone mapping technology, image details in the HDR image can be preserved while compressing the image. By adjusting the tone mapping curve, the horizontal coordinate of the control point is made to fall within a preset range on the corresponding vertical coordinate of the adjusted tone mapping curve, which helps to avoid distortion in the compressed LDR image and ensures that the compressed LDR image conforms to human visual perception.

[0138] In some embodiments, the tone mapping curve includes: a global tone mapping curve and / or local tone mapping curves for each pixel block in the HDR image. The first ordinate includes: the ordinate corresponding to the x-coordinate of the control point on the global tone mapping curve and / or the ordinate corresponding to the x-coordinate of the control point on the local tone mapping curve; the adjusted tone mapping curve includes: the adjusted global tone mapping curve and / or the adjusted local tone mapping curve.

[0139] In some embodiments, the tone mapping curve obtained in step 101 is a global tone mapping curve, which is obtained based on the global target luminance histogram of the HDR image. The global target luminance histogram can be obtained based on the global original luminance histogram. Steps 102 to 105 are all processes based on the global tone mapping curve. In step 105, global tone mapping is performed on the HDR image according to the adjusted global tone mapping curve to obtain the low dynamic range (LDR) image corresponding to the HDR image. Global tone mapping refers to using a spatially invariant mapping function to map all pixels in the image. Since each pixel in the HDR image uses the same mapping function, it is a one-to-one mapping relationship, the algorithm is simple, runs fast, and helps save image processing time.

[0140] In some embodiments, the tone mapping curve obtained in step 101 is a local tone mapping curve for each pixel block in the HDR image. Each pixel block may include several pixels, and the local tone mapping curve is obtained based on the local target brightness histogram of each pixel block in the HDR image. The local target brightness histogram can be obtained based on the local original brightness histogram. Steps 102 to 105 are all processes based on the local tone mapping curve. In step 105, the low dynamic range (LDR) image corresponding to the HDR image is obtained according to the adjusted local tone mapping curve. That is, the HDR image is locally tone mapped according to the adjusted local tone mapping curve to obtain the corresponding LDR image. Local tone mapping refers to adjusting the mapping function based on local pixel statistics and local pixel context relationships. Local tone mapping performs different transformations for different regions of each pixel in the image. Therefore, it is possible that two pixels with the same brightness value in the same image, due to different positions, will obtain different new values ​​after mapping, or two pixels with different brightness values ​​will obtain the same new value after mapping. Local tone mapping is beneficial for obtaining a better image.

[0141] In some embodiments, the tone mapping curves obtained in step 101 include: a global tone mapping curve and local tone mapping curves for each pixel block in the HDR image. In step 104, methods such as... Figure 8 The adjustment methods shown adjust the local tone mapping curve and the global tone mapping curve respectively, resulting in the adjusted local tone mapping curve and the adjusted global tone mapping curve. The implementation method of step 105 can be found in [reference needed]. Figure 11 ,include:

[0142] Step 1051: Obtain the global brightness image based on the adjusted global tone mapping curve.

[0143] Step 1052: Obtain the local brightness image based on the adjusted local tone mapping curve.

[0144] Step 1053: Fuse the global brightness image and the local brightness image to obtain the LDR image corresponding to the HDR image.

[0145] In step 1051, the electronic device can perform global tone mapping on the HDR image according to the adjusted global tone mapping curve to obtain the mapped global brightness image.

[0146] In step 1052, the electronic device can perform local tone mapping on each pixel block according to the adjusted local tone mapping curve corresponding to each pixel block in the HDR image, and obtain the mapped local brightness image corresponding to each pixel block.

[0147] In step 1053, the electronic device can fuse the global brightness image and the local brightness image according to a preset fusion ratio to obtain a fused brightness image, and use the fused brightness image as the LDR image corresponding to the obtained HDR image.

[0148] In some embodiments, the fused brightness image Pic_new can be obtained using the following formula:

[0149] Pic_new=Pic_1*ε+Pic_2*(1-ε);

[0150] Wherein, Pic_1 is the global brightness image, Pic_2 is the local brightness image, and ε is the preset fusion ratio. The size of the fusion ratio can be set according to actual needs, and this embodiment does not impose a specific limitation on it.

[0151] In this embodiment, global tone mapping and local tone mapping are performed on the HDR image to be processed, and the global brightness image obtained from global tone mapping and the local brightness image obtained from local tone mapping are fused. This combines the advantages of global tone mapping and local tone mapping to obtain a better LDR image. While compressing the HDR image into an LDR image, distortion of the LDR image obtained after compression is greatly avoided, making the compressed LDR image more consistent with human visual perception.

[0152] In some embodiments, the implementation of step 1052 can be found in [reference needed]. Figure 12 ,include:

[0153] Step 301: For the Lth pixel block in the HDR image, determine the pixel blocks in the neighborhood centered on the Lth pixel block.

[0154] Step 302: Determine the weights of the pixel blocks in the neighborhood based on the brightness difference between the pixel blocks in the neighborhood and the Lth pixel block.

[0155] Step 303: Based on the weights corresponding to the pixel blocks in the neighborhood, smooth the adjusted local tone mapping curve of the Lth pixel block to obtain the smoothed local tone mapping curve of the Lth pixel block.

[0156] Step 304: Obtain the local brightness image based on the smoothed local tone mapping curves of T pixel blocks in the HDR image.

[0157] In step 301, 1 ≤ L ≤ T, where T is the total number of pixel blocks in the HDR image. For example, referring to Figure 13, the Lth pixel block is... Figure 13 The pixel block marked L, and the pixel blocks in the neighborhood centered on the Lth pixel block, are called the Lth pixel blocks. Figure 13 The nine pixel blocks centered on the Lth pixel block are: the pixel block marked L and the pixel blocks marked L1 to L8. Figure 13 Each of these nine pixel blocks also has its local tone mapping curve marked.

[0158] In step 302, the brightness value of each pixel block within the neighborhood can be calculated. This brightness value can be the average brightness of multiple pixels within each pixel block. Then, the brightness difference between the average brightness value of each pixel block and the average brightness value of the Lth pixel block is calculated. The brightness difference and weights can have a preset correspondence, so that the electronic device can obtain the weights corresponding to the pixel blocks within the neighborhood based on this correspondence.

[0159] In some embodiments, the correspondence between brightness difference and weight can be a curve as shown in Figure 14, which the electronic device can refer to. Figure 14 The relationship curve shown determines the weights of the pixel blocks within the neighborhood. It should be noted that... Figure 14 This only provides an exemplary relationship curve between brightness difference and weight. In actual implementation, the exemplary relationship curve between brightness difference and weight can be set according to actual needs, and is not limited to this. Figure 14 The relationship curve shown is the limit.

[0160] In step 303, the higher the weight of a pixel block, the smoother it is. This smoothing process can be understood as follows: based on the weights of the neighboring pixel blocks, the local tone mapping curve of the Lth pixel block is weighted and filtered to obtain the smoothed local tone mapping curve of the Lth pixel block. In other words, the tone mapping curve of the Lth pixel block is weighted and mixed with the tone mapping curves of other neighboring pixel blocks according to certain weights to obtain the smoothed local tone mapping curve of the Lth pixel block.

[0161] In some embodiments, step 303 can be implemented by obtaining the smoothed local tone mapping curve of the Lth pixel block using the following formula:

[0162]

[0163] Among them, LUT new Let LUT be the smoothed local tone mapping curve of the Lth pixel block. q Let w be the adjusted local tone mapping curve for the q-th pixel block in the neighborhood. qLet be the weight of the q-th pixel block, where Q is the total number of pixel blocks in the neighborhood. The local tone mapping curve of each pixel block can be stored in a look-up table (LUT), which can specifically store the coordinate information of several points on the tone mapping curve.

[0164] See Figure 13 The weights corresponding to the pixel block labeled L and the pixel blocks labeled L1 to L8 are 1, w1, w2, ..., w8, respectively. The local tone mapping curves of the pixel block labeled L and the pixel blocks labeled L1 to L8 can be represented as LUTs. L LUT L1 LUT L2 ....LUT L8 Then the expression for the smoothed local tone mapping curve of the Lth pixel block can be as follows:

[0165]

[0166] Following the above method, the smoothed local tone mapping curve of each pixel block in the HDR image can be obtained, and finally the smoothed local tone mapping curves of all pixel blocks, i.e., T pixel blocks, in the HDR image can be obtained.

[0167] In step 304, the local brightness images corresponding to the T pixel blocks are obtained based on the smoothed local tone mapping curves of the T pixel blocks in the HDR image.

[0168] In this embodiment, smoothing each pixel block in the HDR image helps to suppress artifacts in the final low dynamic range (LDR) image and can also suppress the brightness of halos in the LDR image.

[0169] In some embodiments, after step 105, a process of contrast enhancement for the LDR image is further included. In this embodiment, the contrast of the LDR image corresponding to the HDR image can be enhanced according to a preset contrast enhancement curve to obtain a contrast-enhanced LDR image. The input to the contrast enhancement curve can be the LDR image after the HDR image has been compressed by a tone mapping curve, and the output can be the contrast-enhanced LDR image.

[0170] In some embodiments, the contrast enhancement curve can be selected according to actual needs, for example, a curve such as... Figure 15 The Gamma curve shown is as follows: Figure 16 The spline curve shown, such as Figure 17 The sigmoid curve shown is an example.

[0171] In some embodiments, the parameters of the contrast enhancement curve can be set manually according to actual needs. For example, when the contrast enhancement curve is a Gamma curve, the parameter γ of the Gamma curve can be greater than or equal to 1 in order to compress the brightness of the dark areas of the LDR image.

[0172] In some embodiments, the parameters of the contrast enhancement curve can also be determined based on the average brightness of the LDR image. For example, the contrast enhancement curve is a Gamma curve, and the parameter γ of the Gamma curve is determined as follows:

[0173]

[0174] Where avgL is the average brightness value of the LDR image, TH1_AVGL is the first preset threshold, and TH2_AVGL is the second preset threshold, where TH2 > TH1 ≥ 1. The first and second preset thresholds can be selected from 0 to 1 according to actual needs. For example, TH1 = 1, TH2 = 2, TH1_AVGL = 0.2, and TH2_AVGL = 0.5; however, this embodiment does not impose specific limitations on this.

[0175] In some embodiments, the contrast enhancement curve can be selected as follows: Figure 16 The spline curve shown, for example, is obtained through piecewise cubic Hermite interpolation. Four reference points are selected: (0,0), c1(x1,y1), c2(x2,y2), and c3(1,1). The contrast enhancement curve is obtained by interpolating the curve using these reference points. The LDR image is then subjected to contrast enhancement processing based on this interpolated contrast enhancement curve. (x1,y1) and (x2,y2) can be understood as parameters of the spline curve, whose values ​​can be determined by the average brightness of the LDR image. In one possible implementation, x1 = 0.3, x2 = 0.7, y1 = 0.1, and y2 = 0.8.

[0176] In some embodiments, the LDR image is contrast-enhanced according to a preset contrast enhancement curve to obtain the contrast-enhanced LDR image. A flowchart illustrating this process can be found in [the relevant documentation / reference]. Figure 18 ,include:

[0177] Step 401: Normalize the brightness values ​​of each pixel in the LDR image.

[0178] Step 402: Based on the preset contrast enhancement curve, map the normalized brightness values ​​of each pixel in the LDR image to output a brightness mapping image.

[0179] Step 403: Restore the dynamic range of the luminance-mapped image to the dynamic range before normalization to obtain the contrast-enhanced LDR image.

[0180] In step 401, normalization can be understood as normalizing the brightness value of each pixel in the LDR image from the current dynamic range [0, D] of the LDR image to [0, 1] to obtain the normalized brightness image.

[0181] In step 402, each pixel in the normalized luminance image can be mapped using the selected contrast enhancement curve to output a luminance mapping image. The contrast enhancement curve can be represented by a contrast enhancement function, the normalized luminance image can be used as the input to the contrast enhancement function, and the luminance mapping image can be used as the output of the contrast enhancement function.

[0182] In step 403, the current dynamic range [0, 1] of the brightness mapping image can be restored to the dynamic range [0, D] before normalization to obtain the contrast-enhanced LDR image.

[0183] In this embodiment, considering that the LDR image obtained by tone mapping curves may enhance the brightness of dark areas, resulting in an overall higher image brightness and reduced contrast (contrast refers to the range of difference between dark and bright areas in an image), this embodiment uses a preset contrast enhancement curve to enhance the contrast of the LDR image. This helps to suppress the brightness of dark areas, avoid an overall higher image brightness, and enhance the contrast of the LDR image. In other words, the image processing method in this embodiment helps to avoid distortion in the compressed LDR image, thereby ensuring that the compressed LDR image conforms to human visual perception while enhancing the contrast of the LDR image.

[0184] In some embodiments, the slope of the contrast enhancement curve is less than 1 at positions where the brightness value is less than a preset brightness value. The preset brightness value can be set according to actual needs, indicating that the brightness value is within a small range. Controlling the slope of the contrast enhancement curve to be less than 1 at positions where the brightness value is less than the preset brightness value helps to suppress noise.

[0185] In some embodiments, a schematic diagram of the image processing method may be referred to. Figure 19 The input image is the HDR image to be processed, which can be a Raw Bayer image or an RGB image. The image processing mainly includes:

[0186] S1, Luminance Generation, which generates a luminance image based on a Raw Bayer image or an RGB image.

[0187] S2. Histogram statistics: Based on the brightness image, pixels within a preset brightness range are statistically analyzed to obtain brightness value range-frequency information. Specifically, this can generate a global raw brightness histogram and a local raw brightness histogram.

[0188] S3. Histogram smoothing filtering: This involves applying low-pass filtering to both the global and local raw brightness histograms. Specifically, the brightness information remains unchanged, while the frequency information is smoothed according to the weights specified by the LPFkernel.

[0189] S4. Generation of tone mapping curves: This involves generating a global tone mapping curve based on the global target brightness histogram obtained after smoothing filtering. Local tone mapping curves are also generated based on the local target brightness histogram obtained after smoothing filtering.

[0190] S4, Tone Mapping Curve Adjustment, adjusts the local tone mapping curve and the global tone mapping curve according to the specified control points.

[0191] S5. Local curve smoothing: Based on the brightness information of each pixel block and its neighboring pixel blocks, the adjusted local tone mapping curve of each pixel block is smoothed.

[0192] S6. Local and global brightness image fusion, which involves mixing the local brightness image and the global brightness image in a certain proportion.

[0193] S7. Contrast Enhancement: This step further enhances the contrast of the output LDR image according to a preset contrast enhancement curve, resulting in a contrast-enhanced LDR image.

[0194] The image processing method in this embodiment can compress a high dynamic range (HDR) image into a low dynamic range (LDR) image, while preserving as much detail as possible in the HDR image and avoiding distortion in the compressed LDR image, thus making the compressed LDR image conform to human visual perception. It can also suppress artifacts and halos in the LDR image, enhance its contrast, and suppress noise.

[0195] It should be noted that the examples described above in this embodiment are merely illustrative for ease of understanding and do not constitute a limitation on the technical solution of this application.

[0196] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this patent.

[0197] This application also relates to a chip, as shown in the embodiments below. Figure 20Chip 501 is connected to memory 505, which stores instructions that can be executed by the chip. The instructions are executed by the chip to enable the chip to perform the image processing method described in the above embodiments.

[0198] The memory 505 and chip 501 are connected via a bus. This bus can include any number of interconnecting buses and bridges, connecting various circuits of one or more chips 501 and the memory 505 together. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by chip 501 is transmitted over a wireless medium via an antenna, which further receives data and transmits it to chip 501.

[0199] Chip 501 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory 505 can be used to store data used by chip 501 during operation.

[0200] This application also relates to an electronic device, as shown in the embodiments below. Figure 20 It includes: the aforementioned chip 501, and a memory 502 connected to the chip 501.

[0201] This application also relates to a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the above-described method embodiments.

[0202] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0203] Those skilled in the art will understand that the above embodiments are specific implementations of this application, and in practical applications, various changes can be made in form and detail without departing from the spirit and scope of this application.

Claims

1. An image processing method, characterized in that, include: Obtain the tone mapping curve corresponding to the high dynamic range (HDR) image to be processed; Based on the dynamic range of the HDR image, control points are selected in the coordinate system where the tone mapping curve is located; Determine the first ordinate on the tone mapping curve corresponding to the x-coordinate of the control point; If the first ordinate is not within the preset range, the tone mapping curve is adjusted so that the second ordinate corresponding to the x-coordinate of the control point on the adjusted tone mapping curve is within the preset range; wherein, the preset range is a range of pixel values ​​that conforms to human visual perception. Based on the adjusted tone mapping curve, the low dynamic range (LDR) image corresponding to the HDR image is obtained; The tone mapping curve is generated based on the target brightness histogram of the HDR image, and adjusting the tone mapping curve includes: Based on the first vertical coordinate and the preset range, determine the target frequency that needs to be clipped; Determine the region to be clipped and the region to be allocated in the target brightness histogram; Based on the target frequency, the region to be clipped is clipped by frequency, and the region to be allocated is allocated by frequency, resulting in an updated brightness histogram. Based on the updated luminance histogram, a cumulative probability distribution curve is calculated, and the cumulative probability distribution curve is used as the adjusted tone mapping curve.

2. The image processing method according to claim 1, characterized in that, The step of determining the target frequency to be clipped based on the first vertical coordinate and the preset range includes: If the first ordinate is greater than the upper limit of the preset range, the target frequency to be clipped is calculated using the following formula: num_clip=(y_orig y_up)*N; If the first ordinate is less than the lower limit of the preset range, the target frequency to be clipped is calculated using the following formula: num_clip=(y_down y_orig)*N; Wherein, num_clip is the target frequency, y_orig is the first ordinate, y_up is the upper limit value, y_down is the lower limit value, and N is the total frequency of the target brightness histogram.

3. The image processing method according to claim 1, characterized in that, The step of cropping the region to be cropped based on the target frequency and allocating the region to be allocated based on the frequency to obtain an updated luminance histogram includes: Determine the cutting height based on the target frequency; Based on the clipping height, frequency clipping is performed on each Bin interval in the region to be clipped, and frequency allocation is performed on each Bin interval in the region to be allocated, to obtain an updated brightness histogram. The cutting height and the number of Bin intervals in the region to be cut satisfy the following relationship: Where K is the cutting height, and n is the number of Bin intervals in the area to be cut.

4. The image processing method according to claim 3, characterized in that, The decrease in the histogram height corresponding to each interval Bin in the region to be clipped after frequency clipping is equal to the clipping height; the sum of the increases in the histogram height corresponding to each interval Bin in the region to be allocated after frequency allocation is equal to the target frequency.

5. The image processing method according to claim 1, characterized in that, The control points include m control points arranged in ascending order of their horizontal coordinates. Determining the region to be clipped and the region to be allocated in the target brightness histogram includes: When the first longitudinal coordinate corresponding to the i-th control point on the tone mapping curve is greater than the upper limit value of the preset range, it is determined that the horizontal coordinate range of the region to be cropped in the target brightness histogram is (a, b), and the horizontal coordinate range of the region to be allocated in the target brightness histogram is (c, d); wherein b is the horizontal coordinate x i of the i-th control point i , c is x i +1 When i=1, a is 0, and d is the x-coordinate of the second control point (x2). When i=m, a is the x-coordinate of the (m-1)th control point. m-1 Add 1, where d is the maximum x-coordinate of the target brightness histogram. end ; When 1 < i < m, a is the x-coordinate of the (i-1)th control point. i-1 Add 1, where d is the x-coordinate of the (i+1)th control point. i+1 ; or, When the first ordinate of the i-th control point on the tone mapping curve is less than the lower limit of the preset range, the x-coordinate range of the region to be clipped in the target brightness histogram is determined to be (e, f), and the x-coordinate range of the region to be allocated in the target brightness histogram is determined to be (g, h); where e is the x-coordinate of the i-th control point. i Add 1, h is x i ; When i=1, f is the x-coordinate of the second control point, x2, and g is 0; When i=m, f is the maximum x-coordinate of the target brightness histogram. end g is the x-coordinate of the (m-1)th control point. m-1 Add 1; When 1 < i < m, f is the x-coordinate of the (i+1)th control point. i+1 g is the x-coordinate of the (i-1)th control point. i-1 Add 1.

6. The image processing method according to claim 1, characterized in that, Frequency allocation is performed on each Bin interval within the region to be allocated, including: The frequency of the average allocation of each Bin interval in the region to be allocated is calculated using the following formula: ;in, The average frequency is defined as num_clip, the target frequency is defined as num_clip, and M is the total number of intervals Bin in the region to be allocated. The average frequency is increased by the frequency corresponding to each interval Bin in the region to be allocated.

7. The image processing method according to claim 1, characterized in that, Frequency allocation is performed on each Bin interval within the region to be allocated, including: The frequency of Bin allocation to the u-th interval in the region to be allocated is calculated using the following formula. : ； Where num_clip is the target frequency, and M is the total number of intervals Bin in the region to be allocated. Let be the width of the u-th interval Bin in the region to be allocated. Let be the width of the j-th interval Bin in the region to be allocated, 1≤u≤M, 1≤j≤M; The frequency corresponding to the u-th interval Bin in the region to be allocated is increased by the frequency of the interval Bin. .

8. The image processing method according to any one of claims 1 to 7, characterized in that, The HDR image is synthesized based on R LDR images with an exposure ratio of Ex, and the dynamic range of the LDR images with an exposure ratio of Ex is D. L The dynamic range of the HDR image is based on Ex, R, D. L Determining that selecting control points in the coordinate system containing the tone mapping curve based on the dynamic range of the HDR image includes: Based on the dynamic range of the HDR image, m control points are selected in the coordinate system containing the tone mapping curve; where m = R-1, and when the m control points are arranged in ascending order of their horizontal coordinates, the horizontal coordinate of the first control point is D. L The x-coordinate of the i-th control point is D. L *Ex i , 1 < i ≤ m, Ex i Let be the exposure ratio of the i-th LDR image.

9. The image processing method according to any one of claims 1 to 7, characterized in that, The preset range is determined based on the target dynamic range, which is the specified dynamic range to which the HDR image needs to be compressed.

10. The image processing method according to any one of claims 1 to 7, characterized in that, The target brightness histogram of the HDR image is obtained in the following way: Based on the brightness value of each pixel in the HDR image, obtain the original brightness histogram of the HDR image; The original brightness histogram is low-pass filtered to obtain the filtered brightness histogram. Based on the filtered luminance histogram, the target luminance histogram of the HDR image is obtained.

11. The image processing method according to claim 10, characterized in that, The step of obtaining the target luminance histogram of the HDR image based on the filtered luminance histogram includes: The target brightness histogram of the HDR image is obtained using the following formula: Hist_new=Hist_LPF*α+Hist*(1 α); Wherein, Hist_new is the target brightness histogram, Hist_LPF is the filtered brightness histogram, Hist is the original brightness histogram, and α is a preset filtering intensity coefficient.

12. The image processing method according to claim 10, characterized in that, The step of obtaining the original luminance histogram of the HDR image based on the luminance value of each pixel in the HDR image includes: Based on the brightness value of each pixel in the HDR image and a preset Bin interval of unequal width, the original brightness histogram of the HDR image is obtained; wherein, the wider the Bin interval representing the larger the brightness value, the larger the width.

13. The image processing method according to any one of claims 1 to 7, characterized in that, The tone mapping curve includes: a global tone mapping curve and / or a local tone mapping curve for each pixel block in the HDR image; the first ordinate includes: the ordinate corresponding to the x-coordinate of the control point on the global tone mapping curve and / or the ordinate corresponding to the x-coordinate of the control point on the local tone mapping curve; the adjusted tone mapping curve includes: the adjusted global tone mapping curve and / or the adjusted local tone mapping curve.

14. The image processing method according to claim 13, characterized in that, When the adjusted tone mapping curve includes an adjusted global tone mapping curve and an adjusted local tone mapping curve, obtaining the low dynamic range (LDR) image corresponding to the HDR image based on the adjusted tone mapping curve includes: The global brightness image is obtained based on the adjusted global tone mapping curve; Based on the adjusted local tone mapping curve, a local brightness image is obtained; The global brightness image and the local brightness image are fused to obtain the low dynamic range (LDR) image corresponding to the HDR image.

15. The image processing method according to claim 14, characterized in that, The step of obtaining a local brightness image based on the adjusted local tone mapping curve includes: For the Lth pixel block in the HDR image, determine the pixel blocks in the neighborhood centered on the Lth pixel block; where 1≤L≤T, and T is the total number of pixel blocks in the HDR image; The weights corresponding to the pixel blocks in the neighborhood are determined based on the brightness difference between the pixel blocks in the neighborhood and the Lth pixel block. Based on the weights corresponding to the pixel blocks in the neighborhood, the adjusted local tone mapping curve of the Lth pixel block is smoothed to obtain the smoothed local tone mapping curve of the Lth pixel block. A local brightness image is obtained based on the smoothed local tone mapping curves of T pixel blocks in the HDR image.

16. The image processing method according to claim 15, characterized in that, The step of smoothing the adjusted local tone mapping curve of the Lth pixel block according to the weights corresponding to the pixel blocks in the neighborhood, to obtain the smoothed local tone mapping curve of the Lth pixel block, includes: The smoothed local tone mapping curve of the Lth pixel block is obtained using the following formula: in, The local tone mapping curve of the Lth pixel block after smoothing is given. The adjusted local tone mapping curve for the q-th pixel block within the neighborhood. Let be the weight of the q-th pixel block, and Q be the total number of pixel blocks in the neighborhood.

17. The image processing method according to any one of claims 1 to 7, characterized in that, After obtaining the low dynamic range (LDR) image corresponding to the HDR image based on the adjusted tone mapping curve, the method further includes: According to the preset contrast enhancement curve, the LDR image corresponding to the HDR image is contrast enhanced to obtain the contrast-enhanced LDR image.

18. The image processing method according to claim 17, characterized in that, The step of enhancing the contrast of the LDR image corresponding to the HDR image according to a preset contrast enhancement curve to obtain the contrast-enhanced LDR image includes: Normalize the brightness values ​​of each pixel in the LDR image; Based on a preset contrast enhancement curve, the normalized luminance values ​​of each pixel in the LDR image are mapped to output a luminance mapping image. The dynamic range of the luminance-mapped image is restored to the dynamic range before normalization to obtain the contrast-enhanced LDR image.

19. The image processing method according to claim 17, characterized in that, The contrast enhancement curve is a Gamma curve, and the parameter γ of the Gamma curve is determined in the following way: in, The average brightness value of the LDR image. The first preset threshold, The second preset threshold is TH2 > TH1 ≥ 1.

20. The image processing method according to claim 17, characterized in that, The slope of the contrast enhancement curve is less than 1 at positions where the brightness value is less than the preset brightness value.

21. A chip, characterized in that, The chip is located within an electronic device and connected to a memory within the electronic device. The memory stores instructions that can be executed by the chip, which are then executed by the chip to enable the chip to perform the image processing method as described in any one of claims 1 to 20.

22. An electronic device, characterized in that, include: The chip as claimed in claim 21, and the memory connected to the chip.

23. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the image processing method according to any one of claims 1 to 20.

Images