Dynamic range mapping method and device

Abstract

The present application provides a method and apparatus for dynamic range mapping, which helps avoid the abnormal phenomenon that the brightness of the pixels of a mapped terminal device is brighter than the original image when the maximum display brightness of an image is close to the maximum display brightness of a display device. The dynamic range mapping method includes: obtaining display parameters of a terminal device; obtaining characteristic information of image data; obtaining first parameters of a first tone mapping curve for the image data; when a preset condition is met, obtaining second parameters of a second tone mapping curve based on the first parameters, the display parameters of the terminal device, and the characteristic information of the image data, wherein the output brightness at a first point on the second tone mapping curve is not higher than the input brightness at the first point on the second tone mapping curve; and performing dynamic range mapping on the image data according to the second parameters of the second tone mapping curve.

Description

[0001] This application is a divisional application of the Chinese application with application number 202010365696.3, application date April 30, 2020, and invention name “Method and device for dynamic range mapping”. Technical Field

[0002] The present application relates to the field of display technology, and more particularly, to a method and apparatus for dynamic range mapping. Background Art

[0003] Dynamic range (DR) is used in many fields to represent the ratio of the maximum and minimum values ​​of a variable. In digital images, dynamic range represents the ratio between the maximum brightness and the minimum brightness within the displayable range of the image, that is, the number of grayscale levels from "brightest" to "darkest" in the image. Its unit is candela per square meter (cd / m2), which can also be expressed as nits. The larger the dynamic range of an image, the richer the brightness levels it can represent, and the more realistic the visual effect of the image. Since the dynamic range of natural scenes in the real world is between 10 -3 to 10 6 The dynamic range is very large, so it is called high dynamic range (HDR). Compared with high dynamic range images, the dynamic range of ordinary images is standard dynamic range (SDR) or low dynamic range (SDR).

[0004] At present, display devices with a dynamic range of less than 0.1 to 400 nits are generally called SDR display devices; those with a dynamic range of more than 0.01 to 540 nits are called HDR display devices. Different high dynamic range display devices also have different display dynamic ranges, such as 0.01 to 540 nits HDR display devices, 0.005 to 1000 nits HDR display devices, etc. The dynamic range mapping method is mainly used in the adaptation process of the front-end HDR signal and the back-end HDR display device, including the high-to-low tone mapping process and the low-to-high tone mapping process. For example, the front end collects a 4000 nit light signal, while the HDR display capability of the back-end display device is only 500 nit. Therefore, mapping the 4000 nit light signal to a 500 nit display device is a high-to-low mapping process. For example, the front end collects a 100-nit SDR light signal, while the HDR display capability of the back-end display device is 2000 nit. Therefore, mapping the 100-nit light signal to the 2000-nit display device is a mapping process from low to high.

[0005] In existing technology, when the maximum brightness of an image is lower than the maximum display brightness of the display device, a dynamic range mapping algorithm based on an "S" curve can be used to adjust the high dynamic range image to the dynamic range that the display device can display. However, if the maximum brightness of the image is close to the maximum display brightness of the display device, if this solution is still used, the brightness of the pixels on the display device after mapping will be abnormally brighter than the original image, affecting the user experience. Summary of the Invention

[0006] The present application provides a method and apparatus for dynamic range mapping, which helps to avoid the abnormal phenomenon that the brightness of the pixels of the mapped terminal device is brighter than the original image when the maximum display brightness of the image is close to the maximum display brightness of the display device.

[0007] In a first aspect, a method for dynamic range mapping is provided, comprising:

[0008] Get the display parameters of the terminal device;

[0009] Obtaining feature information of image data;

[0010] Obtaining a first parameter of a first tone mapping curve of the image data;

[0011] When a preset condition is met, obtaining a second parameter of a second tone mapping curve according to the first parameter, the display parameter of the terminal device, and the characteristic information of the image data, wherein the output brightness at the first point on the second tone mapping curve is not higher than the input brightness of the first point on the second tone mapping curve, and the second parameter includes a first linear spline curve parameter, and the first linear spline curve parameter includes a slope MB[0][0] of a first linear spline in the second tone mapping curve or a maximum value TH3[0] of brightness values ​​of pixel points in an interval of the first linear spline;

[0012] Dynamic range mapping is performed on the image data according to second parameters of the second tone mapping curve.

[0013] Therefore, the embodiment of the present application further adjusts the parameters of the first tone mapping curve so that the output brightness of a point on the tone mapping curve (i.e., the second tone mapping curve) corresponding to the adjusted curve parameters (i.e., the second parameters) is not higher than the input brightness corresponding to the point, thereby helping to avoid the abnormal phenomenon that the brightness of the pixels of the terminal device after mapping is brighter than the original image when the maximum display brightness of the image is close to the maximum display brightness of the display device.

[0014] In an embodiment of the present application, when the image data is dynamically mapped according to the second parameter, the straight line portion (i.e., the first first-order spline) can be used to perform tone mapping in the dark area of ​​the image data. This can control the brightness gain and more conveniently control the second parameter to gradually change from a straight line to a straight line of y=x, where the straight line of y=x is equivalent to the output brightness of any point on the tone mapping curve being equal to the input brightness. Therefore, this embodiment itself is not prone to causing flickering for content with gradually changing brightness.

[0015] The embodiments of the present application can be applied to a terminal device, such as a display device. The display device can be in the form of a set-top box, a television display device, a mobile phone display device, or an electronic device such as a conversion device for live streaming or video applications. As an example, on a set-top box, a television display device, or a mobile phone display device, the solution provided by the embodiments of the present application can be implemented in the form of a hardware chip. On a live streaming or video playback device, the solution provided by the embodiments of the present application is mainly implemented in the form of software program code, but the embodiments of the present application are not limited thereto.

[0016] Exemplarily, the image data may be, for example, an HDR source or an SDR source, for example, pixel data in an image, such as brightness and color data of each pixel.

[0017] For example, the characteristic information of the image data can be obtained from the metadata M of the image data, and the metadata M can include, for example, the curve parameters M corresponding to the image data. curve 、The target system displays the actual peak brightness M TPL (targeted system display actual peak luminance), the maximum value MaxSource of the brightness of the content of the image data (the maximum of the Y component of all pixels, or the maximum of the maximum values ​​of the RGB components of all pixels), the minimum value MinSource (the minimum of the Y component of all pixels, or the minimum of the maximum values ​​of the RGB components of all pixels), the average value (the average value of the Y component of all pixels, or the average value of the maximum values ​​of the RGB components of all pixels), the range of change of the displayed content, etc., are not limited in the embodiments of the present application.

[0018] In some embodiments, feature information of the image data may be obtained from pixel information of the image data V; or a feature information value of the image data with a preset value may be used, which is not limited in the embodiments of the present application.

[0019] For example, the display parameter M of the terminal device TPL It may include the maximum display brightness MaxDisplay and / or the minimum display brightness MinDisplay of the terminal device, or other parameters, which are not limited in the embodiments of the present application.

[0020] In conjunction with the first aspect, in certain implementations of the first aspect, when any one of the following conditions is met, the preset condition is met:

[0021] When tone mapping is performed on the image data according to the first parameter, the output brightness at the second point on the first tone mapping curve is higher than the input brightness at the second point on the first tone mapping curve; or

[0022] The parameter p in the first parameter P1 Greater than a first value Tp, wherein the first value Tp is based on a in the first parameter P1 , and the preset a P1 With p P1 The corresponding relationship is obtained, where Tp represents the threshold of the curve parameter p. P1 When Tp is exceeded, it is possible that the output brightness at a certain point on the second tone mapping curve is higher than the input brightness; or

[0023] Parameter a in the first parameter P1 is greater than a second value Ta, wherein the second value Ta is based on p in the first parameter P1 , and the preset a P1 With p P1 Where Ta represents the threshold of the curve parameter a. When the first parameter a P1 When Ta is exceeded, it is possible that the output brightness at a certain point on the second tone mapping curve is higher than the input brightness; or

[0024] Parameter a in the first parameter P1 With parameter p P1 The product of is greater than the third value Tap, wherein the third value Tap is a preset rational number. When the parameter a in the first parameter is P1 With parameter p P1 When the product of exceeds Tap, it is possible that the output brightness at a certain point on the second tone mapping curve is higher than the input brightness. Exemplarily, the third value Tap can be a rational number between 3 and 4, such as 3.2 or 3.4, which is not limited in this embodiment of the present application.

[0025] Therefore, the embodiment of the present application can execute the process of generating the second parameters of the above-mentioned second tone mapping curve when the above-mentioned preset conditions are met, that is, when tone mapping of the image data according to the first tone mapping curve will cause the output brightness of a certain point on the first tone mapping curve to be higher than the input brightness of the point on the first tone mapping curve.

[0026] In combination with the first aspect, in some implementations of the first aspect, the first parameter includes a second linear spline curve parameter, the second linear spline curve parameter includes a slope MB_mid[0][0] of the second linear spline in the first tone mapping curve and a maximum luminance value TH3_mid[0] of a pixel point in an interval of the second linear spline, the display parameter includes a maximum display brightness MaxDisplay of the terminal device, and the characteristic information includes a maximum luminance correction value max_lum of the image data;

[0027] The acquiring, according to the first parameter, the display parameter, and the characteristic information, a second parameter of the second tone mapping curve includes:

[0028] According to the maximum display brightness MaxDisplay and the maximum brightness correction value max_lum, the curve parameters MB_mid[0][0] and TH3_mid[0] are adjusted to obtain the curve parameters MB[0][0] and TH3[0].

[0029] Therefore, the embodiment of the present application can obtain the slope MB[0][0] of the first linear spline of the second tone mapping curve and the maximum value TH3[0] of the brightness value of the interval pixel points of the first linear spline based on the slope MB_mid[0][0] of the second linear spline in the first tone mapping curve and the maximum value TH3_mid[0] of the brightness value of the interval pixel points of the second linear spline, as well as the maximum display brightness MaxDisplay of the terminal device and the maximum brightness correction value max_lum of the image data.

[0030] In combination with the first aspect, in some implementations of the first aspect, the curve parameters MB_mid[0][0] and TH3_mid[0], as well as the curve parameters MB[0][0] and TH3[0], satisfy the following formula:

[0031]

[0032]

[0033] in,

[0034] or,

[0035]

[0036] in

[0037] or

[0038]

[0039] Where L is the input signal, G(L) is the inverse function of the tone mapping curve function H(L), m_a, m_b, m_m, m_n, k1, k2, k3 are curve parameters, G(L,m_a_T) means that when the parameter M_a of G(L) is m_a_T, the G(L) value N1 and N2 corresponding to the input variable L are rational numbers, max(a, b) means finding the larger value of a and b, min(a, b) means finding the smaller value of a and b, H(L) is

[0040]

[0041] or,

[0042] In combination with the first aspect, in certain implementations of the first aspect, the second parameter includes a cubic spline curve parameter, and the cubic spline curve parameter includes the interpolation point values ​​TH1[1], TH2[1], and TH3[1] of the cubic spline on the second tone mapping curve, wherein TH1[1] represents the minimum brightness value of the pixel points in the first interval of the cubic spline, TH2[1] represents the maximum brightness value of the pixel points in the first interval of the cubic spline and the minimum brightness value of the pixel points in the second interval of the cubic spline, and TH3[1] represents the maximum brightness value of the pixel points in the second interval of the cubic spline.

[0043] In combination with the first aspect, in certain implementations of the first aspect, the interpolation point values ​​TH1[1], TH2[1], and TH3[1] of the cubic spline are obtained based on preset offset values ​​of calculated correlation values ​​of the second linear spline curve parameter TH3[0] and the interpolation point values ​​TH1[1], TH2[1], and TH3[1] of the cubic spline in the first parameter, as shown below:

[0044] TH1[1]=TH3[0];

[0045] TH2[1]=TH1[1]+B;

[0046] TH3[1]=TH2[1]+C*TH2[1]-D*TH1[1];

[0047] Among them, B, C and D are preset values ​​for calculating the correlation values ​​of the interpolation point values ​​TH1[1], TH2[1], TH3[1] of the cubic spline, B is the preset offset value corresponding to the brightness value of the pixel point in the dark area transition area, and C and D are the preset weighting coefficients corresponding to the brightness value of the pixel point in the bright area.

[0048] Therefore, the embodiment of the present application can obtain the interpolation point values ​​TH1[1], TH2[1], TH3[1] of the cubic spline in the second parameter based on the preset offset value of the calculated correlation value of the second linear spline curve parameter and the interpolation point values ​​TH1[1], TH2[1], TH3[1] of the cubic spline in the first parameter.

[0049] In combination with the first aspect, in certain implementations of the first aspect, the interpolation point values ​​TH1[1], TH2[1], and TH3[1] of the cubic spline are obtained based on the calculated correlation values ​​of the second linear spline curve parameter TH3[0] and the interpolation point values ​​TH1[1], TH2[1], and TH3[1] of the cubic spline in the first parameter, as shown below:

[0050] TH1[1]=3Spline_TH[i][0][w];

[0051] TH2[1]=3Spline_TH[i][0][w]+3Spline_TH_Delta1[i][1][w];

[0052] TH3[1]=3Spline_TH[i][0][w]+3Spline_TH_Delta1[i][1][w]+3Spline_TH_Delta1[i][2][w];

[0053] Among them, 3Spline_TH[i][0][w], 3Spline_TH_Delta1[i][1][w], and 3Spline_TH_Delta1[i][2][w] are the relevant values ​​calculated for the interpolation point values ​​TH1[1], TH2[1], and TH3[1] of the cubic spline extracted from the metadata.

[0054] Therefore, the embodiment of the present application can obtain the interpolation point values ​​TH1[1], TH2[1], TH3[1] of the cubic spline in the second parameter based on the calculated correlation values ​​of the second linear spline curve parameters in the first parameter and the interpolation point values ​​TH1[1], TH2[1], TH3[1] of the cubic spline extracted from the metadata.

[0055] In combination with the first aspect, in certain implementations of the first aspect, the Y coordinate of the linear spline in the second tone mapping curve at TH3[0] is the same as the Y coordinate of the cubic spline in the second tone mapping curve at TH1[1], and the first-order derivative of the linear spline at TH3[0] is the same as the first-order derivative of the cubic spline at TH1[1].

[0056] In this way, the linear spline curve in the second tone mapping curve and the cubic spline curve in the second tone mapping curve can be continuous at TH[1].

[0057] In combination with the first aspect, in certain implementations of the first aspect, the Y coordinate of the first cubic spline in the second tone mapping curve at TH2[1] is the same as the Y coordinate of the second cubic spline in the second tone mapping curve at TH2[1], and the first-order derivative of the first cubic spline at TH2[1] is the same as the first-order derivative of the second cubic spline at TH2[1].

[0058] In this way, the first cubic spline curve and the second cubic spline curve in the second tone mapping curve can be continuous at TH[2].

[0059] In combination with the first aspect, in certain implementations of the first aspect, the Y coordinate of the second cubic spline in the second tone mapping curve at TH3[1] is the same as the Y coordinate of the third tone mapping function in the second tone mapping curve at TH3[1], and the first-order derivative of the second cubic spline at TH3[1] is the same as the first-order derivative of the third tone mapping function at TH3[1].

[0060] In this way, the second cubic spline curve in the second tone mapping curve and the curve of the third tone mapping function can be continuous at TH[3].

[0061] In combination with the first aspect, in some implementations of the first aspect, obtaining a first parameter of a first tone mapping curve of the image data includes:

[0062] Acquire metadata of the image data;

[0063] A first parameter of the first tone mapping curve is determined according to the metadata and the display parameter.

[0064] For example, the display terminal device may calculate the average value average_maxrgb of the brightness of the content of the image data V in the metadata M, and / or the maximum value MaxSource of the brightness, and / or the minimum value MinSource of the brightness, and / or the maximum display brightness MaxDisplay of the display terminal device, and / or the minimum display brightness MinDisplay of the display terminal device, and / or the curve parameter M curve ( p1 , p2, ...), and / or other data to obtain a first parameter of the first tone mapping curve, which can be expressed as P1 curve (X, p1, p2, ...). Where X is the input brightness value, and p1, p2, ... are the curve parameter values.

[0065] In combination with the first aspect, in certain implementations of the first aspect, the second parameter also includes a linear spline curve parameter, and the linear spline curve parameter includes the maximum value TH3C of the brightness value of the interval pixel point of the first linear spline in the second tone mapping curve, and the slope Dark of the first linear spline.

[0066] In an embodiment of the present application, when the image data is dynamically mapped according to the second parameter, the straight line portion (i.e., the first first-order spline) can be used to perform tone mapping in the dark area of ​​the image data. This can control the brightness gain and more conveniently control the second parameter to gradually change from a straight line to a straight line of y=x, where the straight line of y=x is equivalent to the output brightness of any point on the tone mapping curve being equal to the input brightness. Therefore, this embodiment itself is not prone to causing flickering for content with gradually changing brightness.

[0067] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes:

[0068] Obtain the maximum value TH3C0 of the brightness value of the pixel points in the initial interval of the first linear spline;

[0069] Obtain the initial slope Dark0 of the first linear spline;

[0070] Determine the maximum value TH3C of the brightness value of the pixel points in the interval according to the maximum value TH3C0 of the brightness value of the pixel points in the initial interval;

[0071] The slope Dark is determined according to the initial slope Dark0.

[0072] In combination with the first aspect, in some implementations of the first aspect, obtaining the maximum value TH3C0 of the brightness values ​​of the pixels in the initial interval of the first linear spline includes:

[0073] Determine a maximum value TH3C0 of the brightness values ​​of the pixels in the initial interval according to a first parameter, wherein the first parameter includes a maximum value TH3[0] of the brightness values ​​of the pixels in the interval of the second first-order spline in the first tone mapping curve; or

[0074] Determine the maximum value TH3C0 of the brightness value of the pixel point in the initial interval according to a preset value, such as the decomposition of dark vision and photopic vision, that is, the brightness at which the strength of cone cells and rod cells of the human eye changes accordingly, such as 1 nit; or

[0075] The maximum value TH3C0 of the brightness value of the pixel points in the initial interval is determined according to the metadata of the image data, wherein the metadata includes characteristic data of the number of pixels in the dark area of ​​the histogram.

[0076] In combination with the first aspect, in some implementations of the first aspect, obtaining the initial slope Dark0 of the first linear spline includes:

[0077] Determining the initial slope Dark0 according to a first parameter, wherein the first parameter includes the slope MB[0][0] of the second first-order spline in the first tone mapping curve; or

[0078] determining the initial slope Dark0 according to a ratio of a fourth value to a maximum value TH3C of the luminance values ​​of the pixels in the interval, wherein the fourth value is an output value of the first tone mapping curve at the maximum value TH3C of the luminance values ​​of the pixels in the interval; or

[0079] The initial slope Dark0 is determined according to a slope value of a preset input value of the first tone mapping curve between 0 and a maximum value TH3C of the brightness value of the pixel points in the interval.

[0080] In combination with the first aspect, in certain implementations of the first aspect, the maximum brightness value TH3C0 of the initial interval pixel points, the maximum brightness value TH3C of the interval pixel points, the initial slope Dark0, and the slope Dark satisfy the following formula:

[0081] TH3C=TH3C0+(MaxSource-TH3C0)*(WA) N2 ,

[0082] Dark=Dark0+(1-Dark0)*(WA) N1 ,

[0083]

[0084] or

[0085]

[0086] Wherein, TH3C is greater than TH3C0 and less than MaxSource, TH3C0 is less than MaxSource, N1 and N2 are rational numbers greater than 0, H(L) is a tone mapping curve, and G(L) is an inverse function of H(L).

[0087] In combination with the first aspect, in certain implementations of the first aspect, the maximum brightness value TH3C0 of the initial interval pixel points, the maximum brightness value TH3C of the interval pixel points, the initial slope Dark0, and the slope Dark satisfy the following formula:

[0088] TH3C=TH3C0+(MaxLum-TH3C0)*(WA) N2 ,

[0089] DARK=DARK0+(1-DARK0)*(WA) N1 ,

[0090]

[0091] or

[0092]

[0093] Wherein, MaxLum is the adjustment value of the maximum brightness of the image data, TH3C is greater than TH3C0 and less than MaxSource, TH3C0 is less than MaxSource, N1 and N2 are rational numbers greater than 0, H(L) is the tone mapping curve function, and G(L) is the inverse function of H(L).

[0094] In combination with the first aspect, in some implementations of the first aspect, the second parameter further includes a cubic spline curve parameter, and the cubic spline curve parameter includes a minimum value TH1D of brightness values ​​of pixels in a first interval of a first cubic spline of the second tone mapping curve;

[0095] The method further comprises:

[0096] The minimum value TH1D of the brightness value of the pixel points in the first interval is determined according to the maximum value TH3C of the brightness value of the pixel points in the interval of the first linear spline in the second tone mapping curve.

[0097] In combination with the first aspect, in some implementations of the first aspect, the second parameter further includes a cubic spline curve parameter, and the cubic spline curve parameter includes a maximum value TH2D of brightness values ​​of pixels in a first interval of a first cubic spline of the second tone mapping curve;

[0098] The method further comprises:

[0099] A maximum value TH2D of the brightness values ​​of the pixels in the first interval is determined according to the minimum value TH1D of the brightness values ​​of the pixels in the first interval.

[0100] In combination with the first aspect, in some implementations of the first aspect, determining the maximum TH2D of the brightness values ​​of the pixels in the first interval based on the minimum TH1D of the brightness values ​​of the pixels in the first interval includes:

[0101] determining a maximum value TH2D of the brightness values ​​of the pixels in the first interval according to the minimum value TH1D of the brightness values ​​of the pixels in the first interval and the first parameter; or

[0102] Determine a maximum value TH2D of the brightness values ​​of the pixels in the first interval according to a minimum value TH1D of the brightness values ​​of the pixels in the first interval and a preset rational value; or

[0103] A maximum value TH2D of the brightness values ​​of the pixels in the first interval is determined according to a minimum value TH1D of the brightness values ​​of the pixels in the first interval and metadata of the image data.

[0104] In combination with the first aspect, in certain implementations of the first aspect, the minimum value TH1D of the brightness value of the pixel point in the first interval of the first cubic spline of the second tone mapping curve is the same as the maximum value TH3C of the brightness value of the pixel point in the interval of the first linear spline, the output value of the linear spline in the second tone mapping curve and the first cubic spline at TH1D are the same, and the first-order derivatives of the linear spline in the second tone curve and the first cubic spline at TH1D are the same.

[0105] In this way, the linear spline curve in the second tone mapping curve and the cubic spline curve in the second tone mapping curve can be continuous at TH1D.

[0106] In combination with the first aspect, in some implementations of the first aspect, the cubic spline curve parameter further includes a maximum value TH3D of brightness values ​​of pixels in a second interval of a second cubic spline of the second tone mapping curve;

[0107] The method further comprises:

[0108] The maximum value TH3D of the brightness value of the pixel points in the second interval is determined according to the minimum value TH1D of the brightness value of the pixel points in the first interval and the maximum value TH2D of the brightness value of the pixel points in the first interval.

[0109] In combination with the first aspect, in some implementations of the first aspect, determining the maximum luminance value TH3D of the pixel points in the second interval based on the minimum luminance value TH1D of the pixel points in the first interval and the maximum luminance value TH2D of the pixel points in the first interval includes:

[0110] determining the third maximum input brightness TH3D according to the minimum brightness value TH1D of the pixel points in the first interval, the maximum brightness value TH2D of the pixel points in the first interval, and the first parameter; or

[0111] determining the third maximum input brightness TH3D according to a minimum value TH1D of the brightness values ​​of the pixels in the first interval, a maximum value TH2D of the brightness values ​​of the pixels in the first interval, and a preset rational number; or

[0112] The third maximum input brightness TH3D is determined according to a minimum value TH1D of the brightness values ​​of the pixels in the first interval, a maximum value TH2D of the brightness values ​​of the pixels in the first interval, and metadata of the image data.

[0113] In combination with the first aspect, in certain implementations of the first aspect, the minimum value of the brightness value of the pixel points in the second interval of the second cubic spline is the same as the maximum value TH2D of the brightness value of the pixel points in the first interval of the first cubic spline, the output value of the first cubic spline and the second cubic spline at the maximum value TH2D of the brightness value of the pixel points in the interval is the same, and the first-order derivatives of the first cubic spline and the second cubic spline at the maximum value TH2D of the brightness value of the pixel points in the interval are the same.

[0114] In this way, the first cubic spline curve and the second cubic spline curve in the second tone mapping curve can be made continuous at TH2D.

[0115] In combination with the first aspect, in certain implementations of the first aspect, the second parameter also includes a curve parameter of a tone mapping subfunction of the second tone mapping curve, the minimum value of the brightness value of the pixel points in the third interval of the tone mapping subfunction is the same as the maximum value TH3D of the brightness value of the pixel points in the second interval, the output value of the second cubic spline and the tone mapping subfunction at the maximum value TH3D of the brightness value of the pixel points in the second interval is the same, and the first-order derivative of the second cubic spline and the tone mapping subfunction at the maximum value TH3D of the brightness value of the pixel points in the second interval is the same.

[0116] In this way, the second cubic spline curve in the second tone mapping curve and the curve of the tone mapping sub-function can be made continuous at TH3D.

[0117] In conjunction with the first aspect, in some implementations of the first aspect, the first parameter includes a P1 、p P1 , obtaining a second parameter of a second tone mapping curve according to the first parameter, the display parameter, and the characteristic information, includes:

[0118] According to a P1 And the preset a P1 With p P1 The corresponding relationship is obtained to obtain the first value Tp;

[0119] If p P1 If it is greater than Tp, then the p in the first parameter P1 Replace with Tp;

[0120] The first parameter after replacement is used as the second parameter.

[0121] Therefore, by adding p in the first parameter P1 Replacing the first parameter with Tp and using the replaced first parameter as the second parameter can help ensure that the output brightness at the first point on the second tone mapping curve is not higher than the input brightness of the first point on the second tone mapping curve.

[0122] In conjunction with the first aspect, in some implementations of the first aspect, the first parameter includes a P1 、p P1 , obtaining a second parameter of a second tone mapping curve according to the first parameter, the display parameter, and the characteristic information, includes:

[0123] According to p P1 And the preset a P1 With p P1 The corresponding relationship obtains the second value Ta;

[0124] If a P1 If it is greater than Ta, then the first parameter a P1 Replaced by Ta;

[0125] The first parameter after replacement is used as the second parameter.

[0126] Therefore, by adding a P1 Replacing the first parameter with Ta and using the replaced first parameter as the second parameter can help ensure that the output brightness at the first point on the second tone mapping curve is not higher than the input brightness of the first point on the second tone mapping curve.

[0127] In conjunction with the first aspect, in some implementations of the first aspect, the first parameter includes a P1 、p P1 , obtaining a second parameter of a second tone mapping curve according to the first parameter, the display parameter, and the characteristic information, includes:

[0128] If a P1 *p P1 is greater than the third value Tap, then the p in the first parameter is P1 Replaced with Tap / a P1 , or change the first parameter a P1 Replaced with Tap / p P1 ;

[0129] The first parameter after replacement is used as the second parameter.

[0130] Therefore, by adding p in the first parameter P1 Replaced with Tap / a P1 , or change the a in the first parameterP1 Replaced with Tap / p P1 , and using the replaced first parameter as the second parameter can help ensure that the output brightness at the first point on the second tone mapping curve is not higher than the input brightness of the first point on the second tone mapping curve.

[0131] In a second aspect, a dynamic range mapping device is provided, comprising an acquisition unit, a processing unit and a mapping unit.

[0132] An acquisition unit, configured to acquire display parameters of a terminal device;

[0133] The acquisition unit is further configured to acquire feature information of the image data;

[0134] The acquisition unit is further configured to acquire a first parameter of a first tone mapping curve of the image data;

[0135] a processing unit, configured to obtain, when a preset condition is met, second parameters of a second tone mapping curve based on the first parameters, the display parameters of the terminal device, and the characteristic information of the image data, wherein the output brightness at the first point on the second tone mapping curve is not higher than the input brightness of the first point on the second tone mapping curve, and the second parameters include first linear spline curve parameters, and the first linear spline curve parameters include a slope MB[0][0] of a first linear spline in the second tone mapping curve or a maximum brightness value TH3[0] of pixel points in an interval of the first linear spline;

[0136] A mapping unit is configured to perform dynamic range mapping on the image data according to a second parameter of the second tone mapping curve.

[0137] In conjunction with the second aspect, in certain implementations of the second aspect, when any one of the following conditions is met, the preset condition is met:

[0138] When tone mapping is performed on the image data according to the first parameter, the output brightness at the second point on the first tone mapping curve is higher than the input brightness at the second point on the first tone mapping curve; or

[0139] The parameter p in the first parameter P1 Greater than a first value Tp, wherein the first value Tp is based on a in the first parameter P1 , and the preset a P1 With p P1 or

[0140] Parameter a in the first parameter P1 is greater than a second value Ta, wherein the second value Ta is based on p in the first parameterP1 , and the preset a P1 With p P1 or

[0141] Parameter a in the first parameter P1 With parameter p P1 The product of is greater than a third value Tap, wherein the third value Tap is a preset rational number.

[0142] In combination with the second aspect, in some implementations of the second aspect, the first parameter includes a second linear spline curve parameter, the second linear spline curve parameter includes a slope MB_mid[0][0] of the second linear spline in the first tone mapping curve and a maximum brightness value TH3_mid[0] of a pixel point in an interval of the second linear spline, the display parameter includes a maximum display brightness MaxDisplay of the terminal device, and the characteristic information includes a maximum brightness correction value max_lum of the image data;

[0143] The processing unit is specifically used for:

[0144] According to the maximum display brightness MaxDisplay and the maximum brightness correction value max_lum, the curve parameters MB_mid[0][0] and TH3_mid[0] are adjusted to obtain the curve parameters MB[0][0] and TH3[0].

[0145] In conjunction with the second aspect, in certain implementations of the second aspect, the curve parameters MB_mid[0][0] and TH3_mid[0], as well as the curve parameters MB[0][0] and TH3[0], satisfy the following formula:

[0146]

[0147]

[0148] in,

[0149] or

[0150]

[0151]

[0152] or

[0153]

[0154] Where L is the input signal, G(L) is the inverse function of the tone mapping curve function H(L), m_a, m_b, m_m, m_n, k1, k2, k3 are curve parameters, G(L,m_a_T) means that when the parameter M_a of G(L) is m_a_T, the G(L) value N1 and N2 corresponding to the input variable L are rational numbers, max(a, b) means finding the larger value of a and b, min(a, b) means finding the smaller value of a and b, H(L) is

[0155]

[0156] or,

[0157] In combination with the second aspect, in certain implementations of the second aspect, the second parameter includes a cubic spline curve parameter, and the cubic spline curve parameter includes the interpolation point values ​​TH1[1], TH2[1], and TH3[1] of the cubic spline on the second tone mapping curve, wherein TH1[1] represents the minimum brightness value of the pixel points in the first interval of the cubic spline, TH2[1] represents the maximum brightness value of the pixel points in the first interval of the cubic spline and the minimum brightness value of the pixel points in the second interval of the cubic spline, and TH3[1] represents the maximum brightness value of the pixel points in the second interval of the cubic spline.

[0158] In combination with the second aspect, in certain implementations of the second aspect, the interpolation point values ​​TH1[1], TH2[1], and TH3[1] of the cubic spline are obtained based on the preset offset values ​​of the second linear spline curve parameter TH3[0] and the interpolation point values ​​TH1[1], TH2[1], and TH3[1] in the first parameter, as shown below:

[0159] TH1[1]=TH3[0];

[0160] TH2[1]=TH1[1]+B;

[0161] TH3[1]=TH2[1]+C*TH2[1]-D*TH1[1];

[0162] Among them, B, C and D are preset values ​​for calculating the correlation values ​​of the interpolation point values ​​TH1[1], TH2[1], TH3[1] of the cubic spline, B is the preset offset value corresponding to the brightness value of the pixel point in the dark area transition area, and C and D are the preset weighting coefficients corresponding to the brightness value of the pixel point in the bright area.

[0163] In conjunction with the second aspect, in certain implementations of the second aspect, the interpolation point values ​​TH1[1], TH2[1], and TH3[1] of the cubic spline are calculated based on the calculated correlation values ​​of the second linear spline curve parameter TH3[0] and the interpolation point values ​​TH1[1], TH2[1], and TH3[1] in the first parameter, as shown below:

[0164] TH1[1]=3Spline_TH[i][0][w];

[0165] TH2[1]=3Spline_TH[i][0][w]+3Spline_TH_Delta1[i][1][w];

[0166] TH3[1]=3Spline_TH[i][0][w]+3Spline_TH_Delta1[i][1][w]+3Spline_TH_Delta1[i][2][w];

[0167] Among them, 3Spline_TH[i][0][w], 3Spline_TH_Delta1[i][1][w], and 3Spline_TH_Delta1[i][2][w] are calculated related values ​​of the interpolation point values ​​TH1[1], TH2[1], and TH3[1] extracted from metadata.

[0168] In combination with the second aspect, in certain implementations of the second aspect, the Y coordinate of the linear spline in the second tone mapping curve at TH3[0] is the same as the Y coordinate of the cubic spline in the second tone mapping curve at TH1[1], and the first-order derivative of the linear spline at TH3[0] is the same as the first-order derivative of the cubic spline at TH1[1].

[0169] In combination with the second aspect, in certain implementations of the second aspect, the Y coordinate of the first cubic spline in the second tone mapping curve at TH2[1] is the same as the Y coordinate of the second cubic spline in the second tone mapping curve at TH2[1], and the first-order derivative of the first cubic spline at TH2[1] is the same as the first-order derivative of the second cubic spline at TH2[1].

[0170] In combination with the second aspect, in certain implementations of the second aspect, the Y coordinate of the second cubic spline in the second tone mapping curve at TH3[1] is the same as the Y coordinate of the third tone mapping function in the second tone mapping curve at TH3[1], and the first-order derivative of the second cubic spline at TH3[1] is the same as the first-order derivative of the third tone mapping function at TH3[1].

[0171] In conjunction with the second aspect, in some implementations of the second aspect, the acquiring unit is specifically configured to:

[0172] Acquire metadata of the image data;

[0173] A first parameter of the first tone mapping curve is determined according to the metadata and the display parameter.

[0174] In a third aspect, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer, the computer executes the method described in the first aspect.

[0175] In a fourth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute the method described in the first aspect.

[0176] In a fifth aspect, an electronic device is provided, comprising the apparatus for processing media data as described in the second aspect.

[0177] It should be understood that the beneficial effects achieved by the second to fifth aspects of the present application and the corresponding implementation methods can be referred to the beneficial effects achieved by the first aspect of the present application and the corresponding implementation methods, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0178] Figure 1 is the image of the PQ photoelectric transfer function.

[0179] Figure 2 This is the image of the HLG photoelectric transfer function.

[0180] Figure 3 is the image of the SLF photoelectric transfer function.

[0181] Figure 4 A schematic diagram of a dynamic range adjustment curve for a high dynamic range image provided in an embodiment of the present application.

[0182] Figure 5 A schematic diagram of a sigmoid curve is shown.

[0183] Figure 6 A schematic diagram of a Bezier curve is shown.

[0184] Figure 7 This is an example of a mapping curve when the maximum brightness of the image is the same as the maximum display brightness of the display device.

[0185] Figure 8 A schematic diagram of the system architecture provided by an embodiment of the present application is shown.

[0186] Figure 9A schematic flow chart of a dynamic range mapping method provided in an embodiment of the present application is shown.

[0187] Figure 10 A schematic block diagram of a dynamic range mapping device provided in an embodiment of the present application is shown.

[0188] Figure 11 A schematic block diagram of another dynamic range mapping device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0189] The technical solution in this application will be described below with reference to the accompanying drawings.

[0190] First, a brief introduction is given to the relevant concepts and technologies involved in the embodiments of this application.

[0191] 1. Dynamic range is used in many fields to express the ratio of the maximum to minimum values ​​of a variable. In digital images, dynamic range represents the ratio between the maximum and minimum brightness values ​​within the image's displayable range. The dynamic range in nature is very large. For example, the brightness of a night scene under a starry sky is approximately 0.001 cd / m². 2 The sun itself has a brightness of 1,000,000,000 cd / m 2 Among them, cd / m 2 (candela per square meter) is the SI derived unit of measurement for brightness. Thus, the dynamic range of nature reaches 1000,000,000 / 0.001=10 13 order of magnitude.

[0192] However, in real scenes in nature, the brightness of the sun and the brightness of the stars are not obtained at the same time. -3 to 10 6 Because this is a very large dynamic range, we often refer to it as high dynamic range (HDR). In contrast to high dynamic range, the dynamic range of ordinary images is called low dynamic range (LDR) or standard dynamic range (SDR). From this, we can understand that the imaging process of a digital camera is actually a mapping from the high dynamic range of the real world to the low dynamic range of a photograph. Figure 1 An example of mapping from the high dynamic range of the real world to the low dynamic range of a display device is shown.

[0193] The greater the dynamic range of an image, the more scene detail it displays, the richer the brightness levels, and the more realistic the visual effect. Traditional digital images typically use one byte, or 8 bits, to store a pixel value. High dynamic range, however, uses multiple bytes of floating-point numbers to store a pixel value, thus enabling the representation of a high dynamic range in natural scenes.

[0194] The process of optical digital imaging (for example, the imaging process of a digital camera) converts the light radiation of a real scene into electrical signals through an image sensor and stores them as digital images. The purpose of image display, on the other hand, is to reproduce the real scene depicted by a digital image through a display device. The ultimate goal of both is to provide users with the same visual perception as if they were directly observing the real scene.

[0195] The brightness levels of real scenes displayed by optical radiation (light signals) are almost linear, so optical signals are also called linear signals. However, in the process of converting optical signals into electrical signals in optical digital imaging, not every optical signal corresponds to an electrical signal. The converted electrical signals are nonlinear, so they are also called nonlinear signals.

[0196] 2. The optical electro transfer function (OETF) represents the conversion relationship between the linear signal and the nonlinear signal of the image pixel.

[0197] Before the advent of cameras capable of capturing HDR images, traditional cameras could only record captured light information within a certain range by controlling the exposure value. Because the maximum illumination information of a display device cannot match the brightness information of the real world, and we view images through a display device, a photoelectric transfer function is required. Early display devices were cathode ray tube (CRT) displays, whose photoelectric transfer function was the gamma function. This photoelectric transfer function, based on the "gamma" function, is defined in the International Telecommunications Union-Radio Communications Sector (ITU-R) recommendation BT.1886, as shown in the following formula (1).

[0198]

[0199] The image quantized to 8 bits through the above conversion is the traditional SDR image. The SDR image and the transfer function in the above formula (1) are in the traditional display device (illuminance is 100cd / m 2 Good performance on the left and right.

[0200] With the continuous upgrading of display devices, the dynamic range that can be displayed by current display devices is increasing compared to traditional display devices. Existing consumer-grade HDR displays can reach 600cd / m 2 , high-end HDR displays can reach 2000cd / m 2 , far exceeding the display range of traditional SDR displays. Therefore, the optoelectronic transfer function (OTF) specified in the ITU-R BT.1886 standard, which is compatible with traditional SDR displays, no longer adequately represents the performance of current HDR displays. Therefore, improvements to the OTF are needed to accommodate the upgrade of HDR displays.

[0201] There are three main types of HDR OETFs: perceptual quantizer (PQ), hybrid log-gamma (HLG), and scene luminance fidelity (SLF). These three OETFs are specified by the audio and video coding standard (AVS).

[0202] The PQ photoelectric transfer function is a perceptual quantization photoelectric transfer function proposed based on the brightness perception model of the human eye. Figure 2 , Figure 2 is the image of the PQ photoelectric transfer function.

[0203] The PQ photoelectric transfer function represents the conversion relationship between the linear signal value of the image pixel and the nonlinear signal value in the PQ domain. The PQ photoelectric transfer function can be expressed as formula (2):

[0204]

[0205]

[0206] The calculation of each parameter in formula (2) is as follows:

[0207]

[0208] Wherein, L represents the linear signal value, and its value is normalized to [0, 1]; L' represents the nonlinear signal value, and its value range is [0, 1]; m1, m2, c1, c2, and c3 are PQ photoelectric transfer coefficients.

[0209] The HLG photoelectric transfer function is improved based on the traditional Gamma curve. Figure 3 , Figure 3 This is the image of the HLG photoelectric transfer function.

[0210] The HLG photoelectric transfer function applies the traditional Gamma curve in the low range and supplements the log curve in the high range. The HLG photoelectric transfer function represents the conversion relationship between the linear signal value of the image pixel and the nonlinear signal value in the HLG domain. The HLG photoelectric transfer function can be expressed as formula (3):

[0211]

[0212] Wherein, L represents the linear signal value, and its value range is [0, 12]; L represents the nonlinear signal value, and its value range is [0, 1]; a, b, and c are HLG photoelectric transfer coefficients, a = 0.17883277, b = 0.28466892, and c = 0.55991073.

[0213] The SLF photoelectric transfer function is the optimal curve obtained based on the brightness distribution of the HDR scene while meeting the optical characteristics of the human eye. Figure 4 , Figure 4 is the image of the SLF photoelectric transfer function.

[0214] The SLF photoelectric transfer curve represents the conversion relationship between the linear signal value of an image pixel and the nonlinear signal value in the SLF domain. The conversion relationship between the linear signal value of an image pixel and the nonlinear signal value in the SLF domain is shown in formula (4):

[0215]

[0216] Among them, the SLF photoelectric transfer function can be expressed as formula (5):

[0217]

[0218] Where L represents the linear signal value, which is normalized to [0, 1]. L' represents the nonlinear signal value, which ranges from [0, 1]. p, m, a, and b are the SLF photoelectric transfer coefficients, where p = 2.3, m = 0.14, a = 1.12762, and b = -0.12762.

[0219] 3. The dynamic range mapping method is mainly used in the adaptation process of the front-end HDR signal and the back-end HDR display device, including the tone mapping process from high to low, and the tone mapping process from low to high. For example, the front end collects a 4000nit light signal, and the HDR display capability of the back-end display device (such as TV series, tablet computers, etc.) is only 500nit. Therefore, mapping the 4000nit light signal to the 500nit display device is a high-to-low tone-mapping process. For another example, the front end collects a 100nit SDR light signal, and the HDR display capability of the back-end display device is 2000nit. Therefore, mapping the 100nit light signal to the 2000nit display device is a low-to-high tone-mapping process.

[0220] Dynamic range mapping methods can be categorized as static dynamic range mapping and dynamic dynamic range mapping. Static dynamic range mapping uses a single piece of data for the entire tone mapping process, based on the same video content or disk content. This means the processing curve for the same video content or disk content is typically the same. This method has the advantage of carrying less information and a simpler processing flow. However, its disadvantage is that the same curve is used for tone mapping in every scene, which can lead to information loss in some scenes. For example, if the curve prioritizes bright areas, details in extremely dark scenes may be lost or rendered invisible, impacting the user experience.

[0221] Dynamic mapping methods dynamically adjust the image quality based on specific regions, scenes, or frame content. This method offers the advantage of applying different curves to specific regions, scenes, or frames, resulting in better image display results. However, its disadvantage is that each frame or scene must carry relevant scene information, resulting in a large amount of information.

[0222] Currently, there are five main tone-mapping technologies. These five tone-mapping technologies are described below.

[0223] Technology 1 is a tone-mapping process based on the sigmoid curve mainly proposed by Dolby. Figure 5 A schematic diagram of the sigmoid curve is shown. Figure 5, the horizontal axis represents the input brightness, that is, the brightness of the HDR image before dynamic range adjustment, and the vertical axis represents the output brightness, that is, the brightness of the image after dynamic range adjustment. The shape of the sigmoid curve is "S" type, and the slope of the curve first rises and then falls. For example, taking the adjustment point on the sigmoid curve as an example, the brightness can be adjusted to about 300cd / m 2 The source adaptation level is mapped to a brightness of approximately 30 cd / m 2 target adaptation level.

[0224] The second technique is the tone-mapping process based on Bezier curves. Figure 6 A schematic diagram of a Bezier curve is shown. Figure 6 The horizontal axis represents the input brightness, that is, the brightness of the HDR image before dynamic range adjustment, and the vertical axis represents the output brightness, that is, the brightness of the image after dynamic range adjustment. s The range is a linear mapping process, the input brightness is K s The range from 0 to 1 is an "S" curve, and the slope of the curve first rises and then falls.

[0225] The third technique is a tone-mapping process based on the S-shaped curve of human eye perception. The form of this curve is shown in the following formula (6):

[0226]

[0227] Where L and L′ represent normalized electrical or optical signals, a ranges from 0.0 to 1.0, b ranges from 0.0 to 1.0, p, n, and m range from 0.1 to N, where N is a rational number greater than 0.1, L′ ranges from 0.0 to 1.0, L ranges from 0.0 to 1.0, and k1, k2, and k3 are rational numbers.

[0228] Technique 4 is a tone-mapping process that combines cubic splines and straight lines with S-shaped curves. Part of the curve is in the form of the following formula (7):

[0229]

[0230] Where L and L' are normalized electrical or optical signals. The value range of a is 0.0 to 1.0, the value range of b is 0.0 to 1.0, and the values ​​of p, n, and m are 0.1 to N, where N is a rational number greater than 0.1. L' is a rational number in the range of 0.0 to 1.0, L is a rational number in the range of 0.0 to 1.0, k1, k2, and k3 are rational numbers, k1 and k2 are not simultaneously 0, and K3 is not 0. TH1[i], TH2[i], and TH3[i] are rational numbers in the range of 0.0 to 1.0.

[0231] Technique 5 is another tone-mapping process that combines cubic splines and straight lines with S-shaped curves. Part of the curve is in the form of the following formula (8):

[0232]

[0233] Where L and L' are normalized electrical or optical signals. The value range of a is 0.0 to 1.0, the value range of b is 0.0 to 1.0, and the value range of p, n, and m is 0.1 to N, where N is a rational number greater than 0.1. L' is a rational number in the range of 0.0 to 1.0, L is a rational number in the range of 0.0 to 1.0, and k1, k2, and k3 are rational numbers. LT is a preset rational number in the range of 0.0 to 1.0. TH1[i], TH2[i], and TH3[i] are rational numbers in the range of 0.0 to 1.0.

[0234] Metadata related to the curve parameters is sent in the dynamic metadata.

[0235] For technology 1, the dynamic metadata definition related to Dolby Digital Standard (St2094-10) not only transmits statistical values ​​such as maximum (maximum PQ-encoded maxRGB), minimum (minimum PQ-encoded maxRGB), and average (averagePQ-encoded maxRGB), but also transmits parameters related to the sigmoid curve, such as tone mapping offset, tone mapping gain, and tone mapping gamma.

[0236] For technology 2, the dynamic metadata definition related to St2094-40 includes histogram information (distribution MaxRGB) and Bezier curve parameters (Bezier curve anchors) for directly generating curves.

[0237] Moreover, in the St2094 series of standards, the metadata includes the targeted system display actual peak luminance.

[0238] For techniques 3, 4, and 5, the metadata can transmit information such as maximum value, minimum value, and average value, as well as curve parameters such as p, m, a, b, n, K1, K2, and K3.

[0239] If the maximum brightness of an image is lower than the maximum brightness of the display device, the dynamic range mapping algorithms described in Techniques 1 through 5 above can be used to adjust the high dynamic range image to the dynamic range that the display device can display. However, if the maximum brightness of the image is close to the maximum brightness of the display device, continuing to use the dynamic range mapping algorithms described in Techniques 1 through 5 above can result in the abnormal phenomenon that the brightness of the pixels on the display device after mapping is brighter than the original image.

[0240] Figure 7 The maximum brightness of the image is the same as the maximum display brightness of the display device (for example, both are 500 cd / m 2 ) when the tone-mapping curve is used. Figure 7 , where the line y=x (with endpoints A(500,500) and B(0,0)) corresponds to the tone-mapping curve that sets the input brightness to the output brightness. According to this line y=x, the brightness of the pixels on the display device is the same as the brightness of the original image. For example, for point D on y=x, the input brightness and output brightness are both 450cd / m 2 .

[0241] Continue to refer Figure 7 For an S-shaped curve (with two endpoints A(500,500) and B(0,0), for example, a sigmoid curve), when the input brightness is close to 500 cd / m 2 In the case of the same input brightness, the output brightness of tone-mapping by "S" curve is higher than the output brightness of tone-mapping by y=x. For example, for point E on the "S" curve with the same input brightness as point D, its output brightness is 480cd / m 2 .

[0242] In view of this, the present application provides a method for dynamic range mapping. When the maximum brightness of an image is close to the maximum display brightness of a display device, the parameters of the original tone mapping curve are adjusted so that the output brightness of the tone mapping curve corresponding to the adjusted parameters is no higher than its input brightness, thereby helping to avoid the abnormal phenomenon that the brightness of the pixels of the display device after mapping is brighter than the original image. Here, the original tone mapping curve can be a fixed curve adjusted according to the actual peak brightness of the target system display in the metadata of the image data, such as the tone mapping curves of techniques 1 to 5 above.

[0243] Figure 8 A schematic diagram showing the system architecture of the dynamic range mapping method provided by an embodiment of the present application is shown. Figure 8 The front end can acquire HDR content through acquisition and production, and send the HDR content and its metadata to the display end via the transport layer. The display end can include an HDR display device or an SDR display device. As an example, when the display end includes an HDR display device, HDR content can be mapped to the HDR display device. When the display end includes an SDR display device, HDR content can be mapped to the SDR display device.

[0244] For example, the product form of the display end can be a set-top box, a television display device, a mobile phone display device, as well as electronic devices such as a conversion device for online live broadcasting and video applications.

[0245] As an example, on a set-top box, a television display device, or a mobile phone display device, the solution provided by the embodiments of the present application can be implemented in the form of a hardware chip. On a live broadcast or video playback device, the solution provided by the embodiments of the present application is mainly implemented in the form of software program code, but the embodiments of the present application are not limited to this.

[0246] It should be noted that the embodiments of this application are only based on Figure 7 The application scenario in the embodiment of the present application is described as an example, but the system architecture applied to the embodiment of the present application is not limited to this. For example, the front end can also obtain SDR content. At this time, when the display end includes an HDR display device, the SDR content can be mapped to the HDR display device.

[0247] Figure 9 A schematic flow chart of a method 900 for dynamic range mapping provided in an embodiment of the present application. The method 900 is applicable to Figure 8 The application scenarios provided in Figure 8 The display side execution shown in . Figure 9 , method 900 includes the following steps 910 to 940.

[0248] 910. Obtain characteristic information of the image data and display parameters of the local display device. Here, the image data (which may be represented as V) may be HDR image data or SDR image data, which is not limited in this embodiment of the present application.

[0249] Exemplarily, the display end device may receive a video source from the front end, which mainly includes image data V, such as pixel data, etc. As a specific example, for a 4K video source, it may include brightness and color data of 3840*2160 pixels.

[0250] It should be noted that the embodiments of the present application do not limit the format of the image data V. For example, in terms of the color space of the pixel data, the image data V can be image data in the Y (luminance) and UV (chrominance) space, or image data in the RGB pixel space. For another example, in terms of the bit width of the pixel data, the image data V can be 8 bits, 10 bits, or 12 bits.

[0251] In some embodiments, when acquiring the image data V, characteristic information of the image data may also be acquired, for example, from metadata M. The metadata M of the image data V is used to represent the data characteristics of the image data, for example, it may include the format of the image data, or the curve parameters M corresponding to the image data V. curve 、The target system displays the actual peak brightness M TPL (targeted system display actual peak luminance), the maximum value MaxSource of the brightness of the content of the image data (the maximum of the Y component of all pixels, or the maximum of the maximum values ​​of the RGB components of all pixels), the minimum value MinSource (the minimum of the Y component of all pixels, or the minimum of the maximum values ​​of the RGB components of all pixels), the average value (the average value of the Y component of all pixels, or the average value of the maximum values ​​of the RGB components of all pixels), the range of change of the displayed content, etc., are not limited in the embodiments of the present application.

[0252] In some embodiments, feature information of the image data may be obtained from pixel information of the image data V; or a feature information value of the image data with a preset value may be used, which is not limited in the embodiments of the present application.

[0253] It should be noted that when the metadata M includes the curve parameter M curve When the embodiment of the present application does not change the curve parameter M curve For example, for the third technique above, the curve parameter M included in the metadata is curveIt can be p, m, a, b, n, K1, K2, K3, etc.

[0254] In some embodiments, metadata includes dynamic metadata and static metadata, as described in ST2094-1 dynamic metadata for color volume transform or static metadata standards. For example, metadata can be packaged along with the image, such as SEI packets containing different file formats and coding standards, including some packet structures related to hardware HDMI.

[0255] In some embodiments, the display terminal device may also obtain the display parameter M of the display terminal device (ie, the actual terminal device P, or the local display device). TPL (It can also be called display brightness parameter). For example, the display parameter M TPL It may include the maximum display brightness MaxDisplay of the display terminal device and the minimum display brightness MinDisplay of the display terminal device, or other parameters, which are not limited in the embodiments of the present application.

[0256] 920. Obtain first parameters of a first tone mapping curve of the image data.

[0257] For example, the display terminal device may display the image data V based on the metadata M and the display parameter M of the display terminal device. TPL , obtain the first parameter of the first tone mapping curve of the image data V. For example, according to the average value average_maxrgb of the brightness of the content of the image data V in the metadata M, and / or the maximum value MaxSource of the brightness, and / or the minimum value MinSource of the brightness, and / or the maximum display brightness MaxDisplay of the display end device, and / or the minimum display brightness MinDisplay of the display end device, and / or the curve parameter M curve (p1, p2, ...), and / or other data, to obtain the first parameter of the first tone mapping curve, which can be expressed as P1 curve (X, p1, p2, ...). Where X is the input brightness value, and p1, p2, ... are the curve parameter values.

[0258] It should be noted that, in the embodiment of the present application, the first parameter P1 of the first tone mapping curve curve In addition, the embodiment of the present application generates the first parameter P1 curve The data used when generating the first parameter P1 curve For example, the first parameter P1 is generated curveThe data may be metadata, and / or display parameters of a display device, or other preset data.

[0259] As a specific example, for the above technique 5, the curve parameter M curve For example, including parameter values ​​(p, m, a, b, n, K1, K2, K3) and (TH1[i], TH2[i], TH3[i], MB0), the curve parameters M curve Get the first parameter P1 of the first tone mapping curve curve , for example (p P1 、m P1 、a P1 、b P1 、n P1 、K1 P1 、K2 P1 、K3 P1 , TH1[i], TH2[i], TH3[i], MD1[i], MC1[i], MB1[i], MA1[i], MD2[i], MC2[i], MB2[i], MA2[i], MB3).

[0260] As another specific example, for the fourth technique mentioned above, the curve parameter M curve For example, including parameter values ​​(p, m, a, b, n, K1, K2, K3) and (TH1[i], TH2[i], TH3[i], MB0), the curve parameters M curve Get the first parameter P1 of the first tone mapping curve curve , for example (p P1 、m P1 、a P1 、b P1 、n P1 、K1 P1 、K2 P1 、K3 P1 , TH1[i], TH2[i], TH3[i], MD1[i], MC1[i], MB1[i], MA1[i], MD2[i], MC2[i], MB2[i], MA2[i]).

[0261] It should be noted that the first tone mapping curve in the embodiment of the present application is an example of the above-mentioned original tone mapping curve, including but not limited to the tone mapping curves used in Techniques 1, 2, 3, 4, and 5. The first parameters of the first mapping curve in the present application include but are not limited to parameters related to the tone mapping curves used in Techniques 1, 2, 3, 4, and 5.

[0262] 930. When a preset condition is met, obtain a second parameter of a second tone mapping curve based on the characteristic information, the display parameter, and the first parameter. The output brightness at the first point on the second tone mapping curve is not higher than the input brightness at the first point on the second tone mapping curve. That is, within the input brightness range of the second tone mapping curve, the output brightness obtained by mapping any input brightness according to the second tone mapping curve is not higher than the input brightness. The second parameter is used to perform dynamic range mapping on the image data and can be expressed as R curve .

[0263] As an example, the input brightness of the tone mapping curve can be linear light, a nonlinear value, or a normalized value of the linear relationship (for example, 10000 as 1, or the maximum brightness of the content as 1). This embodiment of the present application is not limited to this.

[0264] For example, the first parameter P1 curve , and the average value average_maxrgb of the brightness of the content of the image data V, and / or the maximum value MaxSource of the brightness, and / or the minimum value MinSource of the brightness, and / or the maximum display brightness MaxDisplay of the display end device, and / or the minimum display brightness MinDisplay of the display end device, and / or other data, to obtain the second parameter R of the above-mentioned second tone mapping curve curve .

[0265] For example, the second parameter R curve It can have the form shown in the following formula (9):

[0266]

[0267] Wherein, L and L′ are normalized electrical signals or optical signals, and Dark, TH3C, TH2D, TH3D, MD1D, MC1D, MB1D, MA1D, MD2D, MC2D, MB2D, and MA2D are rational numbers.

[0268] In some embodiments, the preset condition, such as tone mapping the image data according to the first tone mapping curve, may cause the output brightness of a point on the first tone mapping curve to be higher than the input brightness of the point on the first tone mapping curve.

[0269] It should be noted that in the embodiments of the present application, when the difference between the output luminance and the input luminance of the tone mapping curve is within a first range, the output luminance and the input luminance can be considered to be substantially the same. In other words, when the portion of the tone mapping curve where the output luminance is higher than the input luminance is within the first range, the two can be considered to be substantially the same. Conversely, when the portion of the tone mapping curve where the output luminance is higher than the input luminance is outside the first range, the output luminance can be considered to be higher than the input luminance.

[0270] When the preset condition is met, that is, tone mapping the image data according to the first parameter causes the output brightness of the first tone mapping curve to be higher than the input brightness of the first tone mapping curve, the second parameter R of the second tone mapping curve is generated. curve process.

[0271] As a possible implementation method, the above preset condition can be the parameter p in the first parameter P1 Greater than a first value Tp, wherein the first value Tp is based on a in the first parameter P1 , and the preset a P1 With p P1 Wherein, Tp represents the threshold value of the curve parameter p in the third, fourth or fifth technique. When the first parameter p P1 When Tp is exceeded, it is possible that the output brightness at a certain point on the second tone mapping curve is higher than the input brightness.

[0272] As a specific example, for the above-mentioned technique 4 or technique 5, the first parameter P1 curve Including a P1 、p P1 Parameters such as P1 As Ta, the lookup table Tpa (Tp, Ta) obtains the corresponding first value Tp. Here, the table Tpa (Tp, Ta) is the preset a P1 With p P1 An example of the corresponding relationship of . Wherein, Ta represents the threshold value of the curve parameter a in the third technique, the fourth technique, or the fifth technique.

[0273] If p P1 If it is greater than Tp, the preset condition is met. Optionally, the first parameter P1 can be curve Medium P1 Replaced with the first value Tp obtained by looking up the table. In this way, the first parameter P1 after replacement curve That is, the second parameter R curve .

[0274] If p P1is less than or equal to Tp, then there is no need to generate the second parameter R of the second tone mapping curve curve process.

[0275] As another possible implementation, the above-mentioned preset condition can be parameter a in the first parameter. P1 is greater than a second value Ta, wherein the second value Ta is based on p in the first parameter P1 , and the preset a P1 With p P1 When the first parameter a P1 When Ta is exceeded, it is possible that the output brightness at a certain point on the second tone mapping curve is higher than the input brightness.

[0276] As a specific example, for the above-mentioned technique 4 or technique 5, the first parameter P1 curve Including a P1 、p P1 At this time, you can use p P1 As Tp, the lookup table Tpa (Tp, Ta) obtains the corresponding second value Ta. Here, the table Tpa (Tp, Ta) is the preset a P1 With p P1 An example of the corresponding relationship.

[0277] If a P1 If it is greater than Ta, the preset condition is met. Optionally, the first parameter P1 can be curve a in P1 Replace it with the second value Ta obtained by looking up the table. In this way, the first parameter after replacement can be the second parameter R curve .

[0278] If a P1 is less than or equal to Ta, then there is no need to generate the second parameter R of the second tone mapping curve curve process.

[0279] In the above example, table Tpa(Tp, Ta) is a preset rational number combination, such as (3.5, 0.879), (4.5, 0.777), etc. It should be noted that for values ​​not appearing in the table, the value can be generated using linear differences, adjacent values, or weighted averages of adjacent values. In addition, the embodiment of the present application does not limit the specific form of table Tpa(Tp, Ta). For example, table Tpa(Tp, Ta) can also be expressed as a functional relationship between Tp and Ta.

[0280] As another possible implementation method, the above preset condition is parameter a in the first parameter P1 With parameter p P1The product of is greater than a third value Tap, wherein the third value Tap is a preset rational number. Exemplarily, the third value Tap can be a rational number between 3 and 4, such as 3.2 or 3.4, which is not limited in the present embodiment.

[0281] As a specific example, for the above-mentioned technique 4 or technique 5, the first parameter P1 curve Including a P1 、p P1 At this point, we can determine the parameter a P1 With parameter p P1 The product of a P1 *p P1 Is it greater than the preset value Tap?

[0282] If a P1 *p P1 If it is greater than Tap, the preset condition is met. Optionally, the first parameter P1 curve p in P1 Replaced with Tap / a P1 , or change the first parameter a P1 Replaced with Tap / p P1 In this way, the first parameter after replacement can be the second parameter R curve .

[0283] If a P1 *p P1 If it is less than or equal to Tap, then there is no need to generate the second parameter R of the second tone mapping curve. curve process.

[0284] In some other embodiments, the first parameter P1 corresponding to the first tone mapping curve may also be curve Convert to an absolute brightness space, such as a linear space, or a nonlinear space such as PQ or HLG. It is necessary to ensure that values ​​of the same magnitude as y and x have the same brightness. Then, the presence of a portion of the first tone mapping curve above y=x can be determined based on whether the first tone mapping curve intersects the y=x line. For example, if the first tone mapping curve intersects the y=x line, it can be determined that the first tone mapping curve has a portion above y=x. If the first tone mapping curve does not intersect y=x, it can be determined that the first tone mapping curve does not have a portion above y=x.

[0285] Therefore, in the embodiments of the present application, by further adjusting the parameters of the first tone mapping curve, the output brightness of a certain point on the tone mapping curve (i.e., the second tone mapping curve) corresponding to the adjusted curve parameters (i.e., the second parameters) is not higher than the input brightness corresponding to this point, which helps to avoid the abnormal phenomenon that the brightness of the pixels of the terminal device after mapping is brighter than the original image when the maximum display brightness of the image is close to the maximum display brightness of the display device.

[0286] In some optional embodiments, the above-mentioned second parameters further include linear spline curve parameters, and the linear spline curve parameters include the maximum value TH3C (which can also be referred to as the first maximum input brightness TH3C) of the brightness values of the interval pixel points of the linear spline (which can be denoted as the first linear spline) in the second tone mapping curve, and the slope Dark of the first linear spline. Exemplarily, the first linear spline is, for example, the tone mapping curve with an input brightness less than TH3C in the above formula (9), that is, Dark×L, L < TH3C. Where L < TH3 is the interval pixel point of the first linear spline.

[0287] Optionally, the display device can obtain the maximum value TH3C0 (which can also be referred to as the initial first maximum input brightness TH3C0) of the brightness values of the initial interval pixel points of the first linear spline, and the initial slope Dark0, and then determine the above-mentioned first maximum input brightness TH3C according to the initial first maximum input brightness TH3C0, and determine the slope Dark according to the initial slope Dark0.

[0288] The following are three methods provided by the embodiments of the present application for obtaining the initial first maximum input brightness TH3C0.

[0289] Method 1

[0290] The display device can determine the above-mentioned initial first maximum input brightness TH3C0 according to the first parameter P1 curve . Exemplarily, when there is a linear spline (which can be denoted as the second linear spline, such as in the above-mentioned Technique 2, Technique 4 or Technique 5) in the first tone mapping curve, the initial first maximum input brightness TH3C0 can be determined as the maximum value of the brightness values of the interval pixel points of the second linear spline.

[0291] Method 2

[0292] The display device determines the initial first maximum input brightness TH3C0 according to a preset value. Exemplarily, the preset value can be the decomposition of scotopic vision and photopic vision, that is, the brightness at which the relative strengths of the human eye's cone cells and rod cells change, such as 1 nit.

[0293] Method 3

[0294] The display terminal device determines the initial first maximum input brightness TH3C0 based on the metadata M of the image data V. The metadata M includes characteristic data of the number of pixels in the dark area of ​​the histogram, such as the characteristic brightness position of the number of pixels in the dark area of ​​the histogram, or the brightness of the dark area pixels that changes significantly from dark to bright pixels / accumulated number of pixels, or the number of pixels from 0 to the characteristic brightness that accounts for more than a preset proportion of the total pixels.

[0295] The following are three methods for obtaining the initial slope Dark0 provided in the embodiments of the present application.

[0296] Method 1

[0297] The display terminal device can use the first parameter P1 curve , determine the initial slope Dark0. Exemplarily, when the first tone mapping curve has a first-order spline (eg, a second first-order spline), the initial slope Dark0 may be determined to be the slope of the second first-order spline, such as MB0 in technique four or technique five.

[0298] Method 2

[0299] The display end device can determine the initial slope Dark0 according to the ratio of the fourth value and the first maximum input brightness TH3C, wherein the fourth value is the output value of the first tone mapping curve at the first maximum input brightness TH3C. For example, the fourth value can be expressed as Vdark=P1 curve (TH3C), at this time the initial slope Dark0 can be expressed as (Vdark / TH3C).

[0300] Method 3

[0301] The display device may determine the initial slope Dark0 based on the slope value of the first tone mapping curve for a preset input value between 0 and the first maximum input brightness TH3C. For example, the initial slope Dark0 may be an average value, a maximum value, or an intermediate value of the slope values ​​between 0 and the first maximum input brightness TH3C, which is not limited in this application.

[0302] It should be noted that the above method of obtaining the initial maximum input brightness TH3C0 or the initial slope Dark0 is only an example and does not limit the embodiments of the present application. For example, the initial maximum input brightness TH3C0 or the initial slope Dark0 obtained by a method similar to the above method or by replacing the above method with conventional means is also within the protection scope of the embodiments of the present application.

[0303] The following are two methods provided in embodiments of the present application for obtaining the first maximum input brightness TH3C and the slope Dark of the second target tone mapping curve according to the initial first maximum input brightness TH3C0 and the initial slope Dark0.

[0304] Method 1

[0305] The first maximum input brightness TH3C and the slope Dark can be determined according to the following formulas (10) and (11), that is, the first initial maximum input brightness TH3C0, the first maximum input brightness TH3C, the initial slope Dark0 and the slope Dark satisfy the following formulas (10) and (11).

[0306] TH3C=TH3C0+(MaxSource-TH3C0)*(WA) N2 (10)

[0307] Dark=Dark0+(1-Dark0)*(WA) N1 (11)

[0308] in,

[0309]

[0310] or

[0311]

[0312] Wherein, N1 and N2 are rational numbers greater than 0, H(L) is a tone mapping curve, and G(L) is an inverse function of H(L).

[0313] Method 2

[0314] The first maximum input brightness TH3C and the slope Dark can be determined according to the following formulas (12) and (13), that is, the first initial maximum input brightness TH3C0, the first maximum input brightness TH3C, the initial slope Dark0 and the slope Dark satisfy the following formulas (12) and (13).

[0315] TH3C=TH3C0+(MaxLum-TH3C0)*(WA) N2 (12)

[0316] Dark=Dark0+(1-Dark0)*(WA) N1 (13)

[0317] in,

[0318]

[0319] or

[0320]

[0321] Wherein, MaxLum is the adjustment value of the maximum brightness MaxSource of the image data, H(L) is the tone mapping curve function, and G(L) is the inverse function of H(L). It should be noted that the adjustment method from MaxSource to MaxLum is not limited in this embodiment of the application.

[0322] For example, for the tone mapping curve in the following formula (14-1), its inverse function G(L) is shown in formula (15-1).

[0323]

[0324]

[0325] For example, for the tone mapping curve in the following formula (14-2), its inverse function G(L) is shown in formula (15-2).

[0326]

[0327]

[0328] In an embodiment of the present application, when dynamic range mapping is performed on image data according to the second parameter, the straight line portion can be used for tone mapping in the dark area of ​​the image data, so that the brightness gain can be controlled, and at the same time, it is more convenient to control the second parameter to gradually change from a straight line to a straight line y=x, where the straight line y=x is equivalent to the output brightness of any point on the tone mapping curve being equal to the input brightness. Therefore, this embodiment itself is not prone to causing flickering for content with gradually changing brightness.

[0329] In some optional embodiments, the second tone mapping curve further includes a cubic spline curve, and the second parameter R curve The maximum value of the brightness value of the pixel points in the first interval of the first cubic spline of the second tone mapping curve is also included. At this time, the display end device can determine the maximum value TH1D of the brightness value of the pixel points in the first interval of the first cubic spline based on the maximum value TH3C of the brightness value of the pixel points in the interval of the first cubic spline in the second tone mapping curve, that is, the first maximum input brightness TH3C. Exemplarily, the maximum value TH1D of the brightness value of the pixel points in the first interval of the first cubic spline can be equal to the first maximum input brightness TH3C, that is, TH1D = TH3C.

[0330] For example, the tone mapping curve corresponding to the first cubic spline can be the tone mapping curve in the above formula (9) with an input brightness range greater than or equal to TH3C and less than TH2D, that is, MD1D×(L-TH1D)3 +MC1D×(L-TH1D) 2 +MB1D×(L−TH1D)+MA1D, TH1D≤L<TH2D, wherein TH1D≤L<TH2D is the pixel point in the first interval.

[0331] Among them, the second parameter R curve The maximum value TH2D of the brightness value of the pixel point in the first interval of the first cubic spline is also included, which can also be called the second maximum input brightness TH2D. Exemplarily, the second maximum input brightness TH2D can be determined according to the maximum value TH1D of the brightness value of the pixel point in the first interval above.

[0332] In some optional embodiments, the second parameter R curve The maximum value TH3D of the brightness value of the pixel point in the second interval of the second cubic spline of the second tone mapping curve is also included, which can also be called the third maximum input brightness TH3D. Optionally, the minimum value of the brightness value of the pixel point in the second interval can be the above-mentioned TH2D. Exemplarily, the tone mapping curve corresponding to the second cubic spline can be a tone mapping curve with an input brightness range greater than or equal to TH2D and less than TH3D in the above formula (9), that is, MD2D×(L-TH2D) 3 +MC2D×(L-TH2D) 2 +MB2D×(L-TH2D)+MA2D, TH2D≤L≤TH3D, where TH2D≤L≤TH3D is the pixel point in the second interval.

[0333] Exemplarily, the display terminal device may determine the third maximum input brightness TH3D according to the maximum value TH1D of the brightness values ​​of the pixels in the first interval and the second maximum input brightness TH2D.

[0334] The following are three methods for determining the second maximum input brightness TH2D and the third maximum input brightness TH3D provided by embodiments of the present application.

[0335] Method 1

[0336] The display device may determine the second maximum input brightness TH2D based on the maximum brightness value TH1D of the pixel points in the first interval and TH1[0], TH2[0], TH3[0] in the first parameter (or the parameters TH1[0], TH2[0], TH3[0] included in the metadata M). For example, TH2D and TH3D may satisfy the following formulas (16-1) and (17-1), respectively:

[0337] TH2D=TH1D+TH2[0]-TH1[0] (16-1)

[0338] TH3D=TH2D+TH3[0]-TH2[0] (17-1)

[0339] Method 2

[0340] The display device may determine the second maximum input brightness TH2D based on the maximum brightness value TH1D of the pixel points in the first interval and deltaTH2[0] and deltaTH3[0] in the first parameter (or the parameters deltaTH2[0] and deltaTH3[0] included in the metadata M). For example, TH2D and TH3D may satisfy the following formulas (16-2) and (17-2), respectively:

[0341] TH2D=TH1D+deltaTH2[0] (16-2)

[0342] TH3D=TH2D+deltaTH3[0] (17-2)

[0343] Method 3

[0344] The display terminal device can determine TH2D and TH3D based on TH1D and a preset value. For example, TH2D and TH3D can satisfy the following formulas (18) and (19), respectively:

[0345] TH2D=TH1D+B (18)

[0346] TH3D=TH2D+C*TH2D-D*TH1D (19)

[0347] Among them, B is a rational number greater than 0, for example, it can be the offset value corresponding to the brightness value of the pixel point in the dark area transition area, and the default value can be 0.15; C and D are rational numbers greater than 0, for example, they are the weighting coefficients corresponding to the brightness value of the pixel point in the bright area, and the default value can be 0.5.

[0348] Optionally, a second parameter R can be determined curve Parameters such as MD1D, MC1D, MB1D, MA1D, MD2D, MC2D, MB2D, and MA2D in (i.e., in formula (9)) can be determined, for example, according to the following formulas (14) to (19).

[0349] In some embodiments, the minimum value TH1D of the brightness of the first interval pixel points of the first cubic spline of the second tone mapping curve is the same as the first maximum input brightness TH3C (i.e., the maximum value TH3C of the brightness values ​​of the interval pixel points of the first linear spline). In this way, the interval pixel points of the linear spline of the second tone mapping curve are continuous with the first interval pixel points of the first cubic spline. In addition, the output value of the first linear spline and the first cubic spline in the second tone mapping curve at TH1D is the same, and the first-order derivative of the first linear spline and the first cubic spline in the second tone mapping curve at TH1D is the same, that is, the second tone mapping curve is continuous at TH1D.

[0350] For example, for R in formula (9) curve In terms of parameters, when the second tone mapping curve is continuous at TH1D, MA1D, Dark, TH3C, and MB1D satisfy the following formulas (20) and (21):

[0351] Dark×TH3C=MA1D (20)

[0352] Dark=MB1D (21)

[0353] In some embodiments, the maximum luminance value of the pixel points in the second interval of the second cubic spline is the same as the maximum luminance value TH2D of the pixel points in the first interval of the first cubic spline, i.e., the second maximum input luminance TH2D. The output values ​​of the second cubic spline and the first cubic spline at TH2D are the same. This ensures that the pixel points in the first interval of the first cubic spline and the pixel points in the second interval of the second cubic spline are continuous. Furthermore, the first-order derivatives of the second cubic spline and the first cubic spline at TH2D are the same, i.e., the second tone mapping curve is continuous at TH2D.

[0354] For example, for R in formula (9) curve In terms of parameters, when the second tone mapping curve is continuous at TH2D, MD1D, TH3C, MC1D, MB1D, MA1D, MD2D, TH2D, MB2D, and MA2D satisfy the following formulas (22) and (23):

[0355] MD1D×(TH2D-TH3C) 3 +MC1D×(TH2D-TH3C) 2 +MB1D×(TH2D-TH3C)+MA1D=MA2D (twenty two)

[0357] 3×MD1D×(TH2D-TH3C) 3+2×MC1D×(TH2D-TH3C)+MB1D=MB2D (23)

[0358] In some embodiments, the second parameters also include curve parameters of a tone mapping sub-function of the second tone mapping curve, wherein the minimum luminance value of the third interval pixel points of the tone mapping sub-function is the same as the maximum luminance value TH3D of the second interval pixel points of the second cubic spline, i.e., the third maximum input luminance TH3D. In this way, the second interval pixel points of the second cubic spline of the second tone mapping curve and the second interval pixel points corresponding to the tone mapping sub-function are continuous. Furthermore, the second cubic spline and the tone mapping sub-function have the same output value at the TH3D, and the first-order derivatives of the second cubic spline and the tone mapping sub-function at the TH3D are the same, i.e., the second tone mapping curve is continuous at TH3D.

[0359] For example, for R in formula (9) curve In terms of parameters, when the second tone mapping curve is continuous at TH3D, MD2D, TH3D, TH2D, MC2D, MB2D, MA2D, MD2D, MC2D, and MB2D satisfy the following formulas (24) and (25):

[0360] MD2D×(TH3D-TH2D) 3 +MC2D×(TH3D-TH2D) 2 +MB2D×(TH3D-TH2D)+MA2D=P curve (TH3D) (24)

[0361]

[0362] In addition, in some embodiments, the values ​​of the two cubic spline segments (i.e., the first cubic spline and the second cubic spline) at TH2D can be obtained according to a preset strategy. For example, the values ​​of the two cubic spline segments at TH2D can be the value of the midpoint of the line connecting the two points with input luminances TH1D and TH3D on the second tone mapping curve.

[0363] In the embodiment of the present application, the cubic spline curve in the second tone mapping curve can, on the one hand, smoothly connect the linear spline curve and the base curve, and on the other hand, facilitate control of the gain of the portion adjacent to the straight line portion.

[0364] It should be noted that, in the embodiment of the present application, only the process of obtaining the linear spline parameters in the second parameters may be included, and there is no need to involve the process of obtaining parameters in the second parameters other than the linear spline parameters.

[0365] 940. Perform dynamic range mapping on the image data according to the second parameter of the second tone mapping curve. Exemplarily, after obtaining the second parameter, a second tone mapping curve may be obtained for performing dynamic range mapping on the image data.

[0366] Exemplarily, according to the above formula (9), a mapping relationship between normalized HDR / SDR source data and normalized HDR / SDR display data can be obtained. For example, the mapping value L′ can be reverse normalized to between the maximum display capability and the minimum display capability of the display terminal device based on the maximum display capability and the minimum hour capability (for example, 0) of the display terminal device. It should be noted that the above reverse normalization calculation can be a nonlinear space of PQ or a linear space normalized from 0 to 1. In addition, the reverse normalization can be 0-10000nit, or 0.0001-100000nit, etc. The embodiment of the present application does not limit the reverse normalization range and process of the HDR / SDR mapping data L′.

[0367] It should be noted that after obtaining the second tone mapping curve, the subsequent display adaptation processing not only includes tone mapping, but can also be further adjusted before display to adjust its saturation processing, and / or color gamut conversion processing, and / or denoising processing, and / or sharpening processing, etc., which is not limited to the embodiments of the present application.

[0368] It should also be noted that the maximum display capability of the display terminal device can be obtained based on the device parameters or manufacturer's information. The minimum display capability of the display terminal device is usually 0 nit or 1 nit, which is not limited in the present embodiment.

[0369] Therefore, the embodiments of the present application further adjust the parameters of the tone mapping curve so that the output brightness of the tone mapping curve corresponding to the adjusted curve parameters is no higher than the input brightness of the tone mapping curve. This helps to avoid the abnormal phenomenon that the brightness of the pixels of the display device after mapping is brighter than the original image when the maximum display brightness of the image is close to the maximum display brightness of the display device. Therefore, the embodiments of the present application can provide greater flexibility for terminal display devices with different brightness, thereby achieving better presentation effects when the parameters are adjusted reasonably.

[0370] In some optional embodiments of the present application, the linear spline curve parameters in the second parameters (which can be recorded as first linear spline curve parameters) may include the slope of the first linear spline in the second tone mapping curve (for example, it can be expressed as MB[0][0]) and the maximum value of the brightness value of the pixel points in the interval of the first linear spline (for example, it can be expressed as TH3[0]).

[0371] In this embodiment, the linear spline curve parameters included in the above-mentioned first parameters (which can be recorded as second linear spline curve parameters) may include the slope of the second linear spline in the first tone mapping curve (for example, it can be expressed as MB_mid[0][0]) and the maximum value of the brightness value of the interval pixel points of the second linear spline (for example, it can be expressed as TH3_mid[0]).

[0372] At this time, when the preset conditions are met, one implementation method for obtaining the second parameters of the second tone mapping curve according to the first parameters, the display parameters and the characteristic information can be: according to the maximum display brightness MaxDisplay and the maximum brightness correction value max_lum, the curve parameters MB_mid[0][0] and TH3_mid[0] are adjusted to obtain the curve parameters MB[0][0] and TH3[0].

[0373] For example, if the parameter m_a (i.e., the tone mapping curve parameter a) is greater than Tm_ap(m_p), the process of generating the second parameter is executed, that is, adjusting MB[0][0] and TH3[0] according to max_lum / MaxDisplay. Tm_ap can be obtained based on a preset mapping relationship between m_p_T and m_a_T, for example, by looking up a table (m_p_T, m_a_T), where m_p corresponds to the tone mapping curve parameter p, and the preset value Tm_ap(m_p) of m_a obtained based on m_p is set to m_a_T.

[0374] For example, when adjusting the curve parameters MB_mid[0][0] and TH3_mid[0], the input can be the highest display brightness MaxDisplay (the value in the PQ domain) of the display brightness range of the display device, the maximum brightness correction value max_lum of the frame to be processed, the targeted_system_display_maximum_luminance in the metadata (if targeted_system_display_maximum_luminance does not exist in the metadata, targeted_system_display_maximum_luminance is equal to MaxDisplay); the original linear spline curve (i.e., the linear spline curve in the first tone mapping curve) parameters, MB[0][0], TH3[0]; the color signal mapping curve parameters Ptone_mapping, including m_p, m_m, m_n, m_a, m_b, k1, k2, k3. The output can be the linear spline curve (i.e., the linear spline of the second tone mapping curve) parameters, MB[0][0], TH3[0].

[0375] As a possible implementation, the curve parameters MB_mid[0][0] and TH3_mid[0], as well as the curve parameters MB[0][0] and TH3[0], satisfy the following formulas (26) and (27):

[0376] TH3[0]=TH3_mid[0]+(MaxSource-TH3_mid[0])*(WA) N2 (26)

[0377] MB[0][0]=MB_mid[0][0]+(1-MB_mid[0][0])*(WA) N1 (27)

[0378] in,

[0379]

[0380] or

[0381]

[0382] Where N1 and N2 are rational numbers greater than 0, G(L) is the mapping curve parameter T curve The inverse function of .

[0383] As another possible implementation, the curve parameters MB_mid[0][0] and TH3_mid[0], as well as the curve parameters MB[0][0] and TH3[0], satisfy the following formulas (28) and (29):

[0384] TH3[0]=TH3_mid[0]+(MaxLum-TH3_mid[0])*(WA) N2 (28)

[0385] MB[0][0]=MB_mid[0][0]+(1-MB_mid[0][0])*(WA) N1 (29)

[0386] in,

[0387]

[0388] or

[0389]

[0390] Where MaxLum is the maximum brightness correction value (the adjustment value of MaxSource), G(L) is the mapping curve parameter T curve The inverse function of .

[0391] As another possible implementation, the curve parameters MB_mid[0][0] and TH3_mid[0], as well as the curve parameters MB[0][0] and TH3[0], satisfy the following formulas (30) and (31):

[0392]

[0393]

[0394] in,

[0395] or

[0396]

[0397] in

[0398]

[0399] or

[0400]

[0401] Where L is the input signal, G(L) is the inverse function of the tone mapping curve function H(L), m_a, m_b, m_m, m_n, k1, k2, and k3 are curve parameters. G(L,m_a_T) represents the value of G(L) corresponding to the input variable L when the parameter M_a of G(L) is m_a_T. The same applies to H(L,m_a_T). N1 and N2 are rational numbers; for example, the default values ​​of N1 and N2 can be 0. max(a, b) returns the larger of a and b; min(a, b) returns the smaller of a and b.

[0402] In some embodiments, k1 and k2 are not 0 at the same time, and K3 is not 0.

[0403] For example, H(L) is as follows:

[0404]

[0405] or

[0406]

[0407] It should be noted that, in the embodiment of the present application, only the process of obtaining the first-order spline parameters in the second parameters may be included, and there is no need to involve the process of obtaining parameters other than the first-order spline parameters in the second parameters, that is, there is no need to involve the following processing process.

[0408] In some optional embodiments, the second parameter includes a cubic spline curve parameter, and the cubic spline curve parameter includes the interpolation point values ​​TH1[1], TH2[1], and TH3[1] of the cubic spline on the second tone mapping curve, wherein TH1[1] represents the minimum value of the brightness value of the pixel points in the first interval of the cubic spline, TH2[1] represents the maximum value of the brightness value of the pixel points in the first interval of the cubic spline and the minimum value of the brightness value of the pixel points in the second interval of the cubic spline, and TH3[1] represents the maximum value of the brightness value of the pixel points in the second interval of the cubic spline. Exemplarily, TH1[1] can be an example of TH1D, TH2[1] can be an example of TH2D, and TH3[1] can be an example of TH3D.

[0409] As a possible implementation, the interpolation point values ​​TH1[1], TH2[1], and TH3[1] of the cubic spline can be calculated based on a preset offset value of the calculated correlation value between the second linear spline curve parameter TH3[0] and the interpolation point values ​​TH1[1], TH2[1], and TH3[1] of the cubic spline in the first parameter. For example, TH1[1], TH2[1], and TH3[1] satisfy the following formulas (32) to (34):

[0410] TH1[1]=TH3[0] (32)

[0411] TH2[1]=TH1[1]+B (33)

[0412] TH3[1]=TH2[1]+C*TH2[1]-D*TH1[1] (34)

[0413] Wherein, B, C, and D are preset values ​​for calculating correlation values ​​of the interpolation point values ​​TH1[1], TH2[1], and TH3[1] of the cubic spline, B is a preset offset value corresponding to the brightness value of the pixel point in the dark area transition area, and C and D are preset weighting coefficients corresponding to the brightness value of the pixel point in the bright area. For example, the default value of B can be 0.15, and the default values ​​of C and D can be 0.5.

[0414] As another possible implementation, the interpolation point values ​​TH1[1], TH2[1], and TH3[1] of the cubic spline can be calculated based on the calculated correlation values ​​of the second linear spline curve parameter TH3[0] in the first parameter and the interpolation point values ​​TH1[1], TH2[1], and TH3[1] of the cubic spline. For example, TH1[1], TH2[1], and TH3[1] satisfy the following formulas (35) to (37):

[0415] TH1[1]=3Spline_TH[i][0][w] (35)

[0416] TH2[1]=3Spline_TH[i][0][w]+3Spline_TH_Delta1[i][1][w] (36)

[0417] TH3[1]=3Spline_TH[i][0][w]+3Spline_TH_Delta1[i][1][w]+3Spline_TH_Delta1[i][2][w] (37)

[0418] Among them, 3Spline_TH[i][0][w], 3Spline_TH_Delta1[i][1][w], and 3Spline_TH_Delta1[i][2][w] are the relevant values ​​calculated for the interpolation point values ​​TH1[1], TH2[1], and TH3[1] of the cubic spline extracted from the metadata.

[0419] In some optional embodiments, coordinates (e.g., Y coordinates) corresponding to TH1[1], TH2[1], and TH3[1] in the second tone mapping curve may be obtained, for example, and may be expressed as VA1, VA2, and VA3, respectively. The Y coordinate of the linear spline at TH3[0] in the second tone mapping curve is the same as the Y coordinate of the cubic spline at TH1[1] in the second tone mapping curve, and the first-order derivative of the linear spline at TH3[0] is the same as the first-order derivative of the cubic spline at TH1[1].

[0420] In some optional embodiments, the Y coordinate of the first cubic spline in the second tone mapping curve at TH2[1] is the same as the Y coordinate of the second cubic spline in the second tone mapping curve at TH2[1], and the first-order derivative of the first cubic spline at TH2[1] is the same as the first-order derivative of the second cubic spline at TH2[1].

[0421] In some optional embodiments, it is characterized in that the Y coordinate of the second cubic spline in the second tone mapping curve at TH3[1] is the same as the Y coordinate of the third tone mapping function in the second tone mapping curve at TH3[1], and the first-order derivative of the second cubic spline at TH3[1] is the same as the first-order derivative of the third tone mapping function at TH3[1].

[0422] For example, the formula (38) can be obtained based on the first-order spline in the second tone mapping curve:

[0423] F(L)=MB[0][0]×L (38)

[0424] Then, L is set to TH[1], and the coordinate VA1 of TH[1] is calculated to satisfy the following formula (39):

[0425] VA1=MB[0][0]×TH[1] (39)

[0426] Next, according to the first cubic spline in the second tone mapping curve, MA[0][1] and MA[1][1] are obtained, satisfying the following equations (40) and (41):

[0427] MA[0][1]=VA1 (40)

[0428] MA[1][1]=VA2 (41)

[0429] Then, the first-order derivative GD1 of the first cubic spline in the second tone mapping curve is calculated so that MB[0][1]=GD1, and the first-order derivative GD3 of the second cubic spline in the second tone mapping curve at TH3[1] is calculated to satisfy the following equations (42) and (43):

[0430] MB[0][1]=GD1=MB[0][0] (42)

[0431] GD3=m_a×m_m×m_p×K3×m_n×TH3[1] m_m-1 ×DGD3(L) (43)

[0432] in,

[0433] Afterwards, the value VA2[0] of the first cubic spline (i.e., the first cubic spline) curve in the second tone mapping curve at TH2[1] is calculated, and the value VA3[0] of the second cubic spline (i.e., the second cubic spline) curve at TH3[1] is calculated, and VA3[0]=VA3.

[0434] Calculate the derivative GD3[0] of the second cubic spline curve at TH3[1] so that GD3[0]=GD3.

[0435] Calculate the first-order derivatives GD2[0] and GD2[1] of the first cubic spline curve and the second cubic spline curve at TH2[1] respectively, so that GD2[0]=GD2[1].

[0436] Calculate the second-order derivatives GGD2[0] and GGD2[1] of the two cubic spline curves at TH2[1] so that GGD2[0]=GGD2[1].

[0437] In summary, we can get the following formula (44):

[0438]

[0439] Among them, DTH2 = (TH2[1] - TH1[1]), DTH3 = (TH3[1] - TH2[1]).

[0440] Through the above calculation, combined with other conditions, the parameters such as MC[0][1], MD[0][1], MB[1][1], MC[1][1], MD[1][1] in the second parameter can be obtained.

[0441] Therefore, the embodiment of the present application further adjusts the parameters of the first tone mapping curve so that the output brightness of a point on the tone mapping curve (i.e., the second tone mapping curve) corresponding to the adjusted curve parameters (i.e., the second parameters) is not higher than the input brightness corresponding to the point, thereby helping to avoid the abnormal phenomenon that the brightness of the pixels of the terminal device after mapping is brighter than the original image when the maximum display brightness of the image is close to the maximum display brightness of the display device.

[0442] Combined with the above Figure 9 The method of dynamic range mapping in the embodiment of the present application is described in detail. Figure 10 and Figure 11 The dynamic range mapping device of the embodiment of the present application is described, and it should be understood that Figure 10 and Figure 11 The described dynamic range mapping apparatus is capable of performing Figure 9 The various steps of the dynamic range mapping method shown above Figure 9 The restrictions on each step in the same way apply to Figure 10 and Figure 11 The device shown, therefore, is described below Figure 10 and Figure 11 For the sake of brevity, repeated descriptions are omitted.

[0443] Figure 10 FIG1 is a schematic block diagram of a dynamic range mapping apparatus 1000 according to an embodiment of the present application. The apparatus 1000 includes an acquisition unit 1010 , a processing unit 1020 , and a mapping unit 1030 .

[0444] The acquiring unit 1010 is configured to acquire display parameters of the terminal device.

[0445] The acquisition unit 1010 is further configured to acquire feature information of the image data.

[0446] The acquiring unit 1010 is further configured to acquire a first parameter of a first tone mapping curve of the image data.

[0447] The processing unit 1020 is used to obtain a second parameter of a second tone mapping curve based on the first parameter, the display parameter of the terminal device and the characteristic information of the image data when a preset condition is met, wherein the output brightness at the first point on the second tone mapping curve is not higher than the input brightness of the first point on the second tone mapping curve.

[0448] The mapping unit 1030 is configured to perform dynamic range mapping on the image data according to the second parameter of the second tone mapping curve.

[0449] In certain implementations of the present application, the preset condition is met when any one of the following conditions is met:

[0450] When tone mapping is performed on the image data according to the first parameter, the output brightness at the second point on the first tone mapping curve is higher than the input brightness at the second point on the first tone mapping curve; or

[0451] The parameter pP1 in the first parameters is greater than a first value Tp, wherein the first value Tp is obtained according to aP1 in the first parameters and a preset corresponding relationship between aP1 and pP1; or

[0452] Parameter aP1 in the first parameters is greater than a second value Ta, wherein the second value Ta is obtained according to pP1 in the first parameters and a preset corresponding relationship between aP1 and pP1; or

[0453] The product of the parameter aP1 and the parameter pP1 in the first parameter is greater than a third value Tap, wherein the third value Tap is a preset rational number.

[0454] In certain implementations of the present application, the second parameter includes a first linear spline curve parameter, and the first linear spline curve parameter includes the slope MB[0][0] of the first linear spline in the second tone mapping curve or the maximum value TH3[0] of the brightness value of the interval pixel point of the first linear spline.

[0455] In some implementations of the present application, the first parameter includes a second linear spline curve parameter, the second linear spline curve parameter includes a slope MB_mid[0][0] of the second linear spline in the first tone mapping curve and a maximum luminance value TH3_mid[0] of a pixel point in an interval of the second linear spline, the display parameter includes a maximum display brightness MaxDisplay of the terminal device, and the characteristic information includes a maximum luminance correction value max_lum of the image data;

[0456] The processing unit 1020 is specifically configured to:

[0457] According to the maximum display brightness MaxDisplay and the maximum brightness correction value max_lum, the curve parameters MB_mid[0][0] and TH3_mid[0] are adjusted to obtain the curve parameters MB[0][0] and TH3[0].

[0458] In some implementations of the present application, the curve parameters MB_mid[0][0] and TH3_mid[0], as well as the curve parameters MB[0][0] and TH3[0], satisfy the following formula:

[0459]

[0460]

[0461] in,

[0462] or

[0463]

[0464] in

[0465]

[0466] or

[0467]

[0468] Where L is the input signal, G(L) is the inverse function of the tone mapping curve corresponding to the function H(L), m_a, m_b, m_m, m_n, k1, k2, k3 are curve parameters, G(L,m_a_T) represents the G(L) value corresponding to the input variable L when the parameter M_a of G(L) is m_a_T, similarly H(L,m_a_T), N1 and N2 are rational numbers, max(a, b) means finding the larger value of a and b, min(a, b) means finding the smaller value of a and b, H(L) is

[0469]

[0470] or,

[0471] In certain implementations of the present application, the second parameter includes a cubic spline curve parameter, and the cubic spline curve parameter includes the interpolation point values ​​TH1[1], TH2[1], and TH3[1] of the cubic spline on the second tone mapping curve, wherein TH1[1] represents the minimum brightness value of the pixel points in the first interval of the cubic spline, TH2[1] represents the maximum brightness value of the pixel points in the first interval of the cubic spline and the minimum brightness value of the pixel points in the second interval of the cubic spline, and TH3[1] represents the maximum brightness value of the pixel points in the second interval of the cubic spline.

[0472] In certain implementations of the present application, the interpolation point values ​​TH1[1], TH2[1], and TH3[1] of the cubic spline are obtained based on the preset offset values ​​of the second linear spline curve parameter TH3[0] and the interpolation point values ​​TH1[1], TH2[1], and TH3[1] in the first parameter, as shown below:

[0473] TH1[1]=TH3[0];

[0474] TH2[1]=TH1[1]+B;

[0475] TH3[1]=TH2[1]+C*TH2[1]-D*TH1[1];

[0476] Among them, B, C and D are preset values ​​for calculating the correlation values ​​of the interpolation point values ​​TH1[1], TH2[1], TH3[1] of the cubic spline, B is the preset offset value corresponding to the brightness value of the pixel point in the dark area transition area, and C and D are the preset weighting coefficients corresponding to the brightness value of the pixel point in the bright area.

[0477] In certain implementations of the present application, the interpolation point values ​​TH1[1], TH2[1], and TH3[1] of the cubic spline are calculated based on the calculated correlation values ​​of the second linear spline curve parameter TH3[0] in the first parameter and the interpolation point values ​​TH1[1], TH2[1], and TH3[1], as shown below:

[0478] TH1[1]=3Spline_TH[i][0][w];

[0479] TH2[1]=3Spline_TH[i][0][w]+3Spline_TH_Delta1[i][1][w];

[0480] TH3[1]=3Spline_TH[i][0][w]+3Spline_TH_Delta1[i][1][w]+3Spline_TH_Delta1[i][2][w];

[0481] Among them, 3Spline_TH[i][0][w], 3Spline_TH_Delta1[i][1][w], and 3Spline_TH_Delta1[i][2][w] are calculated related values ​​of the interpolation point values ​​TH1[1], TH2[1], and TH3[1] extracted from metadata.

[0482] In certain implementations of the present application, the Y coordinate of the linear spline in the second tone mapping curve at TH3[0] is the same as the Y coordinate of the cubic spline in the second tone mapping curve at TH1[1], and the first-order derivative of the linear spline at TH3[0] is the same as the first-order derivative of the cubic spline at TH1[1].

[0483] In certain implementations of the present application, the Y coordinate of the first cubic spline in the second tone mapping curve at TH2[1] is the same as the Y coordinate of the second cubic spline in the second tone mapping curve at TH2[1], and the first-order derivative of the first cubic spline at TH2[1] is the same as the first-order derivative of the second cubic spline at TH2[1].

[0484] In certain implementations of the present application, the Y coordinate of the second cubic spline in the second tone mapping curve at TH3[1] is the same as the Y coordinate of the third tone mapping function in the second tone mapping curve at TH3[1], and the first-order derivative of the second cubic spline at TH3[1] is the same as the first-order derivative of the third tone mapping function at TH3[1].

[0485] In some implementations of the present application, the acquiring unit 1010 is specifically configured to:

[0486] Obtaining metadata of the image data;

[0487] A first parameter of the first tone mapping curve is determined according to the metadata and the display parameter.

[0488] Figure 11 1 is a schematic diagram of the hardware structure of the dynamic range mapping device 1100 according to an embodiment of the present application. Figure 11 The apparatus 1100 shown can be regarded as a computer device. The apparatus 1100 can be used as an implementation of the apparatus for dynamic range mapping according to an embodiment of the present application, or as an implementation of the method for dynamic range mapping according to an embodiment of the present application. The apparatus 1100 includes a processor 1101, a memory 1102, an input / output interface 1103, and a bus 1105. It may also include a communication interface 1104. The processor 1101, the memory 1102, the input / output interface 1103, and the communication interface 1104 are communicatively connected to each other via the bus 1105.

[0489] The processor 1101 can be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the functions required to be executed by the modules in the device for processing media data in the embodiment of the present application, or to execute the method for processing media data in the method embodiment of the present application. The processor 1101 may be an integrated circuit chip with signal processing capabilities. During implementation, the steps of the above method can be completed by hardware integrated logic circuits in the processor 701 or instructions in the form of software. The above-mentioned processor 1101 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 1102, and the processor 1101 reads the information in the memory 1102 and, in combination with its hardware, completes the functions required to be executed by the modules included in the apparatus for processing media data in the embodiments of the present application, or executes the method for processing media data in the embodiments of the present application.

[0490] The memory 1102 may be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1102 may store an operating system and other application programs. When software or firmware is used to implement the functions required to be performed by the modules included in the apparatus for processing media data in the embodiment of the present application, or when the method for processing media data in the embodiment of the present application is executed, the program code for implementing the technical solution provided in the embodiment of the present application is stored in the memory 1102, and the processor 1101 executes the operations required to be performed by the modules included in the apparatus for processing media data, or executes the method for processing media data provided in the embodiment of the present application.

[0491] The input / output interface 1103 is used to receive input data and information, and output data such as operation results.

[0492] The communication interface 1104 uses a transceiver device such as, but not limited to, a transceiver to implement communication between the device 1100 and other devices or communication networks, and can serve as an acquisition module or a sending module in a processing device.

[0493] The bus 1105 may include a path for transmitting information between the various components of the device 1100 , such as the processor 1101 , the memory 1102 , the input / output interface 1103 , and the communication interface 1104 .

[0494] It should be noted that although Figure 11 The device 1100 shown only shows a processor 1101, a memory 1102, an input / output interface 1103, a communication interface 1104, and a bus 1105. However, in the specific implementation process, those skilled in the art should understand that the device 1100 also includes other devices necessary for normal operation, for example, it may also include a display for displaying the video data to be played. At the same time, according to specific needs, those skilled in the art should understand that the device 1100 may also include hardware devices for implementing other additional functions. In addition, those skilled in the art should understand that the device 1100 may also include only the devices necessary to implement the embodiments of the present application, and does not necessarily include Figure 11 All devices shown in .

[0495] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0496] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the above-mentioned dynamic range mapping method.

[0497] An embodiment of the present application further provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to perform the above-mentioned dynamic range mapping method.

[0498] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0499] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0500] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0501] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0502] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0503] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0504] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for dynamic range mapping, characterized in that: include: Get the display parameters of the terminal device; Obtaining feature information of image data; Obtaining a first parameter of a first tone mapping curve of the image data; When a preset condition is met, obtaining a second parameter of a second tone mapping curve according to the first parameter, the display parameter of the terminal device, and the characteristic information of the image data, wherein the output brightness at the first point on the second tone mapping curve is not higher than the input brightness of the first point on the second tone mapping curve, and the second parameter includes a first linear spline curve parameter, and the first linear spline curve parameter includes a slope MB[0][0] of a first linear spline in the second tone mapping curve or a maximum value TH3[0] of brightness values ​​of pixel points in an interval of the first linear spline; Dynamic range mapping is performed on the image data according to second parameters of the second tone mapping curve.

2. The method according to claim 1, characterized in that The pre-condition is met when any of the following conditions are met: When tone mapping is performed on the image data according to the first parameter, the output brightness at the second point on the first tone mapping curve is higher than the input brightness at the second point on the first tone mapping curve; or The parameter p in the first parameter P1 Greater than a first value Tp, wherein the first value Tp is based on a in the first parameter P1 , and the preset a P1 With p P1 or Parameter a in the first parameter P1 is greater than a second value Ta, wherein the second value Ta is based on p in the first parameter P1 , and the preset a P1 With p P1 or Parameter a in the first parameter P1 With parameter p P1 The product of is greater than a third value Tap, wherein the third value Tap is a preset rational number; The first parameter of the first tone mapping curve includes p P1 and a P1 .

3. The method according to claim 1, characterized in that The first parameter includes a second linear spline curve parameter, the second linear spline curve parameter includes a slope MB_mid[0][0] of the second linear spline in the first tone mapping curve and a maximum luminance value TH3_mid[0] of a pixel point in an interval of the second linear spline, the display parameter includes a maximum display brightness MaxDisplay of the terminal device, and the characteristic information includes a maximum luminance correction value max_lum of the image data; The acquiring, according to the first parameter, the display parameter, and the characteristic information, a second parameter of the second tone mapping curve includes: According to the maximum display brightness MaxDisplay and the maximum brightness correction value max_lum, the curve parameters MB_mid[0][0] and TH3_mid[0] are adjusted to obtain the curve parameters MB[0][0] and TH3[0].

4. The method according to claim 3, characterized in that The curve parameters MB_mid[0][0] and TH3_mid[0], as well as the curve parameters MB[0][0] and TH3[0], satisfy the following formula: in, or Where L is the input signal, G(L, m_a) is the inverse function of the tone mapping curve function H(L), m_p, m_a, m_b, m_m, m_n, K1, K2, K3 are curve parameters, G(MaxDisplay, m_a_T) means that when the parameter m_a of G(L, m_a) is m_a_T, the input variable L is the value of G(L, m_a) corresponding to MaxDisplay, N1 and N2 are rational numbers, max(a, b) means finding the larger value of a and b, min(a, b) means finding the smaller value of a and b, H(L) is or, 5. The method according to any one of claims 1 to 4, characterized in that: The second parameters also include cubic spline curve parameters, which include the interpolation point values ​​TH1[1], TH2[1], and TH3[1] of the cubic spline on the second tone mapping curve, wherein TH1[1] represents the minimum brightness value of the pixel points in the first interval of the cubic spline, TH2[1] represents the maximum brightness value of the pixel points in the first interval of the cubic spline and the minimum brightness value of the pixel points in the second interval of the cubic spline, and TH3[1] represents the maximum brightness value of the pixel points in the second interval of the cubic spline.

6. The method according to claim 5, characterized in that The interpolation point values ​​TH1[1], TH2[1], TH3[1] of the cubic spline are calculated based on the second linear spline curve parameter TH3[0] in the first parameter. p1 , the preset offset values ​​of the interpolation point values ​​TH1[1], TH2[1], and TH3[1] are obtained as follows: TH1[1]=TH3[0] p1 ; TH2[1]=TH1[1]+B; TH3[1]=TH2[1]+C*TH2[1]-D*TH1[1]; Among them, B, C and D are preset values ​​for calculating the correlation values ​​of the interpolation point values ​​TH1[1], TH2[1], TH3[1] of the cubic spline, B is the preset offset value corresponding to the brightness value of the pixel point in the dark area transition area, and C and D are the preset weighting coefficients corresponding to the brightness value of the pixel point in the bright area.

7. The method according to claim 5, characterized in that The interpolation point values ​​TH1[1], TH2[1], and TH3[1] of the cubic spline are calculated based on the calculated correlation values ​​of the interpolation point values ​​TH1[1], TH2[1], and TH3[1], as shown below: TH1[1]=3Spline_TH[i][0][w]; TH2[1]=3Spline_TH[i][0][w]+3Spline_TH_Delta1[i][1][w]; TH3[1]=3Spline_TH[i][0][w]+3Spline_TH_Delta1[i][1][w]+3Spline_TH_Delta1[i][2][w]; Among them, 3Spline_TH[i][0][w], 3Spline_TH_Delta1[i][1][w], 3Spline_TH_Delta1[i][2][w] are the calculated related values ​​of the interpolation point values ​​TH1[1], TH2[1], TH3[1] extracted from the metadata of the image data.

8. The method according to claim 7, characterized in that The Y coordinate of the linear spline in the second tone mapping curve at TH3[0] is the same as the Y coordinate of the cubic spline in the second tone mapping curve at TH1[1], and the first-order derivative of the linear spline at TH3[0] is the same as the first-order derivative of the cubic spline at TH1[1].

9. The method according to claim 7 or 8, characterized in that The Y coordinate of the first cubic spline in the second tone mapping curve at TH2[1] is the same as the Y coordinate of the second cubic spline in the second tone mapping curve at TH2[1], and the first-order derivative of the first cubic spline at TH2[1] is the same as the first-order derivative of the second cubic spline at TH2[1].

10. The method according to claim 7 or 8, characterized in that The Y coordinate of the second cubic spline in the second tone mapping curve at TH3[1] is the same as the Y coordinate of the third tone mapping function in the second tone mapping curve at TH3[1], and the first-order derivative of the second cubic spline at TH3[1] is the same as the first-order derivative of the third tone mapping function at TH3[1].

11. The method according to any one of claims 1-4, 6-8, characterized in that: The obtaining of a first parameter of a first tone mapping curve of the image data includes: Obtaining metadata of the image data; A first parameter of the first tone mapping curve is determined according to the metadata and the display parameter.

12. A device for dynamic range mapping, characterized in that: include: An acquisition unit, configured to acquire display parameters of a terminal device; The acquisition unit is further configured to acquire feature information of the image data; The acquisition unit is further configured to acquire a first parameter of a first tone mapping curve of the image data; a processing unit, configured to obtain, when a preset condition is met, second parameters of a second tone mapping curve based on the first parameters, the display parameters of the terminal device, and the characteristic information of the image data, wherein the output brightness at the first point on the second tone mapping curve is not higher than the input brightness of the first point on the second tone mapping curve, and the second parameters include first linear spline curve parameters, and the first linear spline curve parameters include a slope MB[0][0] of a first linear spline in the second tone mapping curve or a maximum brightness value TH3[0] of pixel points in an interval of the first linear spline; A mapping unit is configured to perform dynamic range mapping on the image data according to a second parameter of the second tone mapping curve.

13. The device according to claim 12, characterized in that The pre-condition is met when any of the following conditions are met: When tone mapping is performed on the image data according to the first parameter, the output brightness at the second point on the first tone mapping curve is higher than the input brightness at the second point on the first tone mapping curve; or The parameter p in the first parameter P1 Greater than a first value Tp, wherein the first value Tp is based on a in the first parameter P1 , and the preset a P1 With p P1 or Parameter a in the first parameter P1 is greater than a second value Ta, wherein the second value Ta is based on p in the first parameter P1 , and the preset a P1 With p P1 or Parameter a in the first parameter P1 With parameter p P1 The product of is greater than a third value Tap, wherein the third value Tap is a preset rational number; The first parameter of the first tone mapping curve includes p P1 and a P1 .

14. The device according to claim 12, characterized in that The first parameter includes a second linear spline curve parameter, the second linear spline curve parameter includes a slope MB_mid[0][0] of the second linear spline in the first tone mapping curve and a maximum luminance value TH3_mid[0] of a pixel point in an interval of the second linear spline, the display parameter includes a maximum display brightness MaxDisplay of the terminal device, and the characteristic information includes a maximum luminance correction value max_lum of the image data; The processing unit is specifically used for: According to the maximum display brightness MaxDisplay and the maximum brightness correction value max_lum, the curve parameters MB_mid[0][0] and TH3_mid[0] are adjusted to obtain the curve parameters MB[0][0] and TH3[0].

15. The device according to claim 14, characterized in that The curve parameters MB_mid[0][0] and TH3_mid[0], as well as the curve parameters MB[0][0] and TH3[0], satisfy the following formula: in, or Where L is the input signal, G(L, m_a) is the inverse function of the tone mapping curve corresponding to the function H(L), m_p, m_a, m_b, m_m, m_n, K1, K2, K3 are curve parameters, G(MaxDispla, m_a_T) means that when the parameter m_a of G(L, m_a) is m_a_T, the input variable L is the value of G(L, m_a) corresponding to MaxDisplay, max(a, b) means finding the larger value of a and b, N1 and N2 are rational numbers, min(a, b) means finding the smaller value of a and b, H(L) is or, 16. The device according to any one of claims 12 to 15, characterized in that: The second parameters also include cubic spline curve parameters, which include the interpolation point values ​​TH1[1], TH2[1], and TH3[1] of the cubic spline on the second tone mapping curve, wherein TH1[1] represents the minimum brightness value of the pixel points in the first interval of the cubic spline, TH2[1] represents the maximum brightness value of the pixel points in the first interval of the cubic spline and the minimum brightness value of the pixel points in the second interval of the cubic spline, and TH3[1] represents the maximum brightness value of the pixel points in the second interval of the cubic spline.

17. The device according to claim 16, characterized in that The interpolation point values ​​TH1[1], TH2[1], TH3[1] of the cubic spline are calculated based on the second linear spline curve parameter TH3[0] in the first parameter. p1 , the preset offset values ​​of the interpolation point values ​​TH1[1], TH2[1], and TH3[1] are obtained as follows: TH1[1]=TH3[0] p1 ; TH2[1]=TH1[1]+B; TH3[1]=TH2[1]+C*TH2[1]-D*TH1[1]; Among them, B, C and D are preset values ​​for calculating the correlation values ​​of the interpolation point values ​​TH1[1], TH2[1], TH3[1] of the cubic spline, B is the preset offset value corresponding to the brightness value of the pixel point in the dark area transition area, and C and D are the preset weighting coefficients corresponding to the brightness value of the pixel point in the bright area.

18. The device according to claim 16, characterized in that The interpolation point values ​​TH1[1], TH2[1], and TH3[1] of the cubic spline are calculated based on the calculated correlation values ​​of the interpolation point values ​​TH1[1], TH2[1], and TH3[1], as shown below: TH1[1]=3Spline_TH[i][0][w]; TH2[1]=3Spline_TH[i][0][w]+3Spline_TH_Delta1[i][1][w]; TH3[1]=3Spline_TH[i][0][w]+3Spline_TH_Delta1[i][1][w]+3Spline_TH_Delta1[i][2][w]; Among them, 3Spline_TH[i][0][w], 3Spline_TH_Delta1[i][1][w], 3Spline_TH_Delta1[i][2][w] are the calculated related values ​​of the interpolation point values ​​TH1[1], TH2[1], TH3[1] extracted from the metadata of the image data.

19. The device according to claim 18, characterized in that The Y coordinate of the linear spline in the second tone mapping curve at TH3[0] is the same as the Y coordinate of the cubic spline in the second tone mapping curve at TH1[1], and the first-order derivative of the linear spline at TH3[0] is the same as the first-order derivative of the cubic spline at TH1[1].

20. The device according to claim 18 or 19, characterized in that The Y coordinate of the first cubic spline in the second tone mapping curve at TH2[1] is the same as the Y coordinate of the second cubic spline in the second tone mapping curve at TH2[1], and the first-order derivative of the first cubic spline at TH2[1] is the same as the first-order derivative of the second cubic spline at TH2[1].

21. The device according to claim 18 or 19, characterized in that The Y coordinate of the second cubic spline in the second tone mapping curve at TH3[1] is the same as the Y coordinate of the third tone mapping function in the second tone mapping curve at TH3[1], and the first-order derivative of the second cubic spline at TH3[1] is the same as the first-order derivative of the third tone mapping function at TH3[1].

22. The device according to any one of claims 12-15, 17-19, characterized in that: The acquisition unit is specifically configured to: Obtaining metadata of the image data; A first parameter of the first tone mapping curve is determined according to the metadata and the display parameter.

Images