Encoders, decoders, systems and methods for determining tone mapping curve parameters

Abstract

The present invention relates generally to the field of video processing, and more specifically to high dynamic range (HDR) video and image processing. In particular, the present invention relates to determining one or more curve parameters of a tone mapping curve. For example, a device (encoder or decoder) can obtain a high dynamic range (HDR) video frame and metadata associated with the HDR video frame. The device can also obtain a pair of anchor points from the HDR video frame and the metadata. The pair of anchor points includes a first anchor point and a second anchor point of the tone mapping curve. Furthermore, the device can generate the one or more curve parameters of the tone mapping curve from the pair of anchor points.

Description

Technical Field

[0001] This invention generally relates to the field of video processing, and more specifically to high dynamic range (HDR) video and image processing. To this end, the invention discloses a method for determining one or more curve parameters of a tone mapping curve, an encoder for encoding HDR video frames, a decoder for decoding HDR video frames, and a system including the encoder and the decoder. In particular, one or more curve parameters of the tone mapping curve can be generated based on a previously obtained pair of anchor points. Background Technology

[0002] In digital imaging, dynamic range typically refers to the range of brightness in the scene being photographed, the limit of the range of brightness that a digital camera or film can capture, or the range of brightness that a display can show.

[0003] The dynamic range of a typical real-world scene is usually around 10. -3 NEET and 10 6 Between nits. In contrast, the dynamic range of consumer displays is typically much smaller. If you want to display a realistic scene on a display, you usually need to reduce the high dynamic range of the real scene to the lower dynamic range of the display: this process is called tone mapping. Tone mapping is usually a non-linear mapping process.

[0004] In HDR image and video processing, perception quantization (PQ) curves are often used to convert nits or cd / m². 2 The optical signal is converted into an electrical signal between 0 and 1. The equation for a typical PQ curve is shown below:

[0005]

[0006] Where L is the luminance value in the linear domain, ranging from 0 nits to 10000 nits, etc. L can be, for example, R value (luminance value of the red component), G value (luminance value of the green component), or B value (luminance value of the blue component) or luminance component Y.

[0007] L' represents the electrical signal in the PQ domain and includes values ​​in or within the range [0, 1], which is commonly referred to as the PQ value or the value in the PQ domain.

[0008]

[0009]

[0010]

[0011]

[0012]

[0013] The input to the PQ transfer function is an optical signal in the linear domain, and the output is an electrical signal in the PQ domain. Because of the one-to-one mapping, the input and output values ​​are practically equivalent if quantization is not applied. The only difference is that they reside in two different domains: the linear domain and the PQ domain.

[0014] Furthermore, the PQ electro-optical transfer function (OETF) is frequently used for quantization. HDR images in the linear domain are first transferred to the PQ domain and then quantized to 10 or 12 bits. Images in the PQ domain can be compressed by a codec for storage or transmission. Quantization in the PQ domain is more uniform for the human visual system because the human visual system is non-linear. If quantization were performed in the linear domain, perceptual distortion would be much greater.

[0015] Some traditional methods for tone mapping are based on using multiple points to determine the tone mapping curve to use. Furthermore, some traditional methods are based on using a first curve (e.g., a linear curve) in the low-light region, a second curve (e.g., a parabola) in the mid-tone range, and a third curve (e.g., a linear curve) in the highlight region.

[0016] However, one problem with traditional methods is that they do not allow for the free selection of point values ​​or different parts of various tone mapping curves themselves, which greatly limits the flexibility of the final shape of the tone mapping curve.

[0017] Another problem with traditional methods is that choosing the curve parameters for tone mapping curves is quite difficult.

[0018] Therefore, an improved tone mapping method is needed. Summary of the Invention

[0019] In view of the above-mentioned problems and disadvantages, embodiments of the present invention aim to improve conventional methods, encoders, decoders, and systems used for tone mapping. The object is to provide a method for determining one or more curve parameters of a tone mapping curve to obtain an improved tone mapping curve suitable for HDR video frames. In particular, the embodiments allow for more flexibility in obtaining the final shape of the tone mapping curve. Furthermore, the embodiments improve efficiency and conserve resources.

[0020] The objective is achieved by the embodiments of the invention described in the appended independent claims. Advantageous implementations of the embodiments of the invention are further defined in the dependent claims.

[0021] In particular, embodiments of the present invention may include: acquiring a pair of anchor points, fine-tuning the anchor points, and using the fine-tuned anchor points to generate curve parameters for a tone mapping curve.

[0022] A first aspect of the present invention provides a method for determining one or more curve parameters of a tone mapping curve. The method includes: acquiring an HDR video frame and metadata associated with the HDR video frame; acquiring a pair of anchor points based on the HDR video frame and the metadata, wherein the pair of anchor points includes a first anchor point and a second anchor point of the tone mapping curve; and generating the one or more curve parameters of the tone mapping curve based on the pair of anchor points.

[0023] The method can be performed (e.g., in whole or in part) by electronic devices such as encoders, decoders, systems including encoders and decoders, HDR systems, HDR televisions (TV), HDR color grading software, and HDR video transcoders.

[0024] The method can be used to determine one or more curve parameters of the tone mapping curve. The tone mapping curve can be an adaptive HDR tone mapping curve. For example, the tone mapping curve (e.g., the parameters of the tone mapping curve) can be adaptively determined for different HDR video frames. The tone mapping curve can represent a mapping from input luminance values ​​(x-coordinate) to output luminance values ​​(y-coordinate values) (e.g., performed by a decoder).

[0025] An example of the tone mapping curve of this invention can be the basic curve of the China Ultra-HD Video Industrial Alliance (CUVA) HDR standard.

[0026] Furthermore, the HDR video frame and the metadata associated with it can be obtained. For example, the HDR video frame may be in the Perceptual Quantizer (PQ) domain, or may be converted to the PQ domain. Additionally, the metadata associated with the HDR video frame can be derived or extracted from it. The metadata may be, or may include, statistical data defining the luminance characteristics of the HDR frame, for example, derived or extracted from HDR frames of the same scene and / or other HDR frames.

[0027] For example, the mapping curve may be the basic curve of the CUVA HDR standard, and the metadata may include the “MinSource” and “MaxSource” values ​​defined in the CUVA HDR standard.

[0028] In some embodiments, the method further includes obtaining the pair of anchor points in the PQ domain.

[0029] In general, the method of the first aspect provides an efficient and resource-saving way to obtain one or more curve parameters of a tone mapping curve, wherein the curve parameters enable the decoder to obtain a tone mapping curve that can be optimized for tone mapping of the HDR video frame. Therefore, an improved tone mapping curve for tone mapping of the HDR video frame can be obtained.

[0030] In one implementation of the first aspect, obtaining a pair of anchor points includes: obtaining multiple pairs (e.g., at least two, three, or four pairs) of anchor points; generating a piecewise linear curve for each pair of anchor points; performing tone mapping on the HDR video frame according to each piecewise linear curve to obtain multiple tone-mapped HDR video frames; and selecting, from the multiple tone-mapped HDR video frames, the pair of anchor points used to generate the one or more curve parameters.

[0031] For example, in some embodiments, multiple pairs of anchor points can be obtained (e.g., at least two pairs of anchor points). Furthermore, a piecewise linear curve can be generated based on each pair of anchor points, i.e., at least two piecewise linear curves can be generated. Additionally, for each piecewise linear curve, tone mapping can be performed on the HDR video frame, thereby obtaining at least two tone-mapped HDR video frames. Based on these, for example, a pair of anchor points for generating the tone-mapped curve can be selected according to a metric representing or indicating the quality of each HDR frame in the tone-mapped HDR frames obtained using the piecewise linear curve (e.g., the local contrast, color distortion model, or perceptual quality model of the frame, particularly the HDR frame).

[0032] In one implementation of the first aspect, obtaining a pair of anchor points includes: obtaining multiple pairs (e.g., at least two, three, or four pairs) of anchor points;

[0033] Construct a piecewise linear curve for each of the plurality of anchor points; estimate the local contrast variation of the HDR video frame for each of the plurality of piecewise linear curves; select the pair of anchor points that produces the highest local contrast.

[0034] In one implementation of the first aspect, acquiring a pair of anchor points includes: acquiring multiple pairs (e.g., at least two, three, or four pairs) of anchor points, including a pair of initial anchor points; constructing a piecewise linear curve for each of the multiple pairs of anchor points; estimating the change in local contrast of an HDR video frame tone-mapped using the piecewise linear curve constructed using the pair of initial anchor points compared to the local contrast of an HDR video frame for each of the multiple piecewise linear curves; and selecting a pair of anchor points that produces the highest local contrast.

[0035] Selecting a pair of anchor points from the plurality of anchor points to generate the curve parameters helps improve the tone mapping curve. For example, it helps determine one or more curve parameters of the tone mapping curve, improving the perceived quality of the HDR video frame after tone mapping, such as contrast.

[0036] In another implementation of the first aspect, the piecewise linear curve of each pair of anchor points connects a predetermined minimum anchor point to a corresponding first anchor point, connects the corresponding first anchor point to a corresponding second anchor point, and connects the corresponding second anchor point to a predetermined maximum anchor point.

[0037] In another implementation of the first aspect, selecting a pair of anchor points for generating the one or more curve parameters includes: determining local contrast for the plurality of tone-mapped HDR video frames to obtain a plurality of local contrasts; and selecting the pair of anchor points for generating the one or more curve parameters from the plurality of anchor points based on the plurality of local contrasts.

[0038] For example, the multiple local contrast ratios (i.e., the local contrast ratio of each tone-mapped HDR video frame) can be obtained for the multiple tone-mapped HDR video frames. Furthermore, a pair of anchor points that produce the maximum or highest local contrast ratio can be selected (e.g., the selected pair of anchor points is a pair of anchor points used to obtain the tone-mapped HDR video frame with the maximum local contrast ratio).

[0039] An improved tone mapping curve can be produced by selecting, from the plurality of pairs of anchor points, the pair of anchor points used to generate the one or more curve parameters based on the local contrast.

[0040] In another implementation of the first aspect, obtaining multiple pairs of anchor points includes: obtaining a pair of initial anchor points; and obtaining multiple other pairs of anchor points based on the pair of initial anchor points.

[0041] In some embodiments, the x and y coordinate values ​​of any pair of second anchor points are always greater than the x and y coordinate values ​​of the corresponding first anchor points.

[0042] In another implementation of the first aspect, the x-coordinate value of the first anchor point of the pair of other anchor points is the same as the x-coordinate value of the first anchor point of the pair of initial anchor points; and / or the x-coordinate value of the second anchor point of the pair of other anchor points is the same as the x-coordinate value of the second anchor point of the pair of initial anchor points.

[0043] In another implementation of the first aspect, the y-coordinate value of the first anchor point of the pair of other anchor points is different from the y-coordinate value of the first anchor point of the pair of initial anchor points; and / or the y-coordinate value of the second anchor point of the pair of other anchor points is the same as the y-coordinate value of the second anchor point of the pair of initial anchor points.

[0044] In another implementation of the first aspect, the x-coordinate value of the first anchor point of the pair of initial anchor points is selected within a range between a predetermined minimum threshold and a predetermined center threshold; the x-coordinate value of the second anchor point of the pair of initial anchor points is selected within a range between the predetermined center threshold and a predetermined maximum threshold.

[0045] In some embodiments, the predetermined minimum threshold may be, for example, a brightness value at which human cone cells can still perceive color, hereinafter also referred to as "minCone", and may in particular have a value of 0.15.

[0046] Furthermore, the predetermined center threshold can be, for example, the minimum human skin tone brightness value of an HDR video frame (hereinafter also referred to as "midLight"), and specifically can have a value of 0.35. Additionally, the predetermined maximum threshold can be, for example, the brightness value of diffuse white (hereinafter also referred to as "defusingLight") as an upper threshold.

[0047] By selecting the initial anchor point as described above, an improved tone mapping curve can be obtained.

[0048] In another implementation of the first aspect, the y-coordinate value of the first anchor point of the pair of initial anchor points is equal to the x-coordinate value of the first anchor point of the pair of initial anchor points.

[0049] In another implementation of the first aspect, the y-coordinate value of the second anchor point of the pair of initial anchor points is calculated based on the luminance histogram of the HDR video frame.

[0050] For example, in some embodiments, the brightness value histogram of the HDR video frame may be a histogram of pixels between “minCone” and “maxSource”.

[0051] In another implementation of the first aspect, the method further includes: calculating the number of pixels whose brightness values ​​are between the predetermined minimum threshold and the predetermined maximum threshold; comparing the values ​​of the histogram elements of the brightness value histogram with the calculated number of pixels; if the values ​​of one or more histogram elements are greater than the calculated number, then cropping pixel brightness values ​​greater than the predetermined maximum display brightness value from the one or more histogram elements, and setting the y-coordinate value of the second anchor point to the average brightness values ​​of all pixels belonging to the one or more histogram elements, wherein the values ​​of the one or more histogram elements are greater than the calculated number of pixels; and if the values ​​of the one or more histogram elements are not greater than the calculated number, then cropping pixel brightness values ​​of pixels between the predetermined center threshold and the predetermined maximum threshold, and setting the y-coordinate value of the second anchor point to the average brightness values ​​of all pixels, wherein the average value is between the predetermined center threshold and the predetermined maximum threshold of the HDR video frame.

[0052] In another implementation of the first aspect, the method further includes: generating the tone mapping curve based on the one or more curve parameters.

[0053] In another implementation of the first aspect, the mapping curve is given by the following equation:

[0054]

[0055] Where L is the brightness of the input pixel of the HDR video frame, m_n is a first value, specifically, m_n = 1, m_m is a second value, specifically, m_m = 2.4, m_b is a predetermined PQ value, m_p is a brightness control factor, m_a is a scaling factor that defines the maximum brightness of the output pixel, and the one or more curve parameters include m_p and m_a.

[0056] In another implementation of the first aspect, the method further includes: receiving the metadata and the HDR video frame.

[0057] In another implementation of the first aspect, the method also transmits the one or more curve parameters as other metadata.

[0058] In another implementation of the first aspect, the method is performed by an encoder and / or a decoder.

[0059] A second aspect of the invention provides an encoder for encoding HDR video frames, wherein the encoder is configured to perform the method according to the first aspect and / or any implementation thereof.

[0060] The encoder of the second aspect implements all the advantages and effects described for the method of the first aspect.

[0061] A third aspect of the invention provides a decoder for decoding HDR video frames, wherein the decoder is configured to perform the method according to the first aspect and / or any implementation thereof.

[0062] The decoder of the third aspect implements all the advantages and effects described for the method of the first aspect.

[0063] A fourth aspect of the invention provides a system for generating tone mapping curves, wherein the system includes an encoder according to the second aspect and / or any implementation thereof and a decoder according to the third aspect and / or any implementation thereof.

[0064] A fifth aspect of the present invention provides a computer program, wherein the computer program includes program code for performing the method according to the first aspect or any implementation thereof.

[0065] A sixth aspect of the present invention provides a non-transitory storage medium for storing executable program code that, when executed by a processor, performs the method described according to the first aspect and / or any implementation thereof.

[0066] It should be noted that all devices, elements, units, and modules described in this application can be implemented in software or hardware elements or any combination thereof. The steps performed by the various entities described in this application, and the functions to be performed by the various entities, are intended to refer to the various entities performing the respective steps and functions. Even in the description of the following specific embodiments, if a particular function or step to be performed by an external entity is not reflected in the description of the specific detailed elements of the entity performing that particular step or function, it should be clear to those skilled in the art that these methods and functions can be implemented in the corresponding software or hardware elements, or in any combination of such elements. Attached Figure Description

[0067] The following description of specific embodiments, in conjunction with the accompanying drawings, will illustrate the above aspects and their implementation, wherein:

[0068] Figure 1 A schematic diagram of an apparatus (particularly an encoder for encoding HDR video frames or a decoder for decoding HDR video frames) provided according to an embodiment of the present invention;

[0069] Figures 2A to 2C A diagram of an example tone mapping curve;

[0070] Figure 3A and Figure 3B A diagram of exemplary metadata;

[0071] Figure 4 A flowchart illustrating a method for generating one or more curve parameters for a tone mapping curve for an encoder and transmitting the generated curve parameters to a decoder;

[0072] Figure 5 A flowchart of a method for generating one or more curve parameters for a tone mapping curve for a decoder;

[0073] Figure 6 A flowchart of a method for selecting a pair of anchor points for one or more curve parameters used to generate a tone mapping curve;

[0074] Figure 7 A diagram illustrating an example of obtaining the first initial anchor point;

[0075] Figure 8 A diagram for obtaining an example of a second initial anchor point (e.g., a pair of initial anchor points);

[0076] Figure 9 A diagram illustrating an example of a piecewise linear curve generated for a pair of anchor points;

[0077] Figure 10 A diagram showing an example of a tone mapping curve generated from curve parameters based on a piecewise linear curve (shown in comparison);

[0078] Figure 11 A flowchart illustrating a method for determining one or more curve parameters according to an embodiment of the present invention;

[0079] Figure 12 An example of the pipeline for the HDR dynamic tone mapping process is shown. Detailed Implementation

[0080] Figure 1 This is a schematic diagram of a device 100 provided according to an embodiment of the present invention. Device 100 may be an encoder for encoding HDR video frames. Alternatively, device 100 may be a decoder for decoding HDR video frames. A system may also be formed including at least one such encoder and one such decoder. Device 100 may be used to perform a method 1100 for generating one or more curve parameters 131, 132 of a tone mapping curve 130 (see also...). Figure 11 (See the schematic diagram shown). For example, curve parameters 131 and 132 can be parameters m_a and m_p as described above, respectively. The tone mapping curve 130 can be used for tone mapping of HDR video frames. It should be noted that embodiments of the present invention can... Figure 12The method 1100 is implemented in block 1201 or 1204 of the pipe 1200 shown. In particular, the method 1100 can be performed in block 1201 or 1204 of the pipe 1200.

[0081] Device 100 is used to acquire HDR video frame 111 and metadata 112 associated with HDR video frame 111. For example, device 100 can receive metadata 112 and HDR video frame 111 separately. However, device 100 can also extract metadata 112 from HDR video frame 111. It should be noted that metadata 112 can be dynamic metadata, that is, metadata 112 can change from one HDR video frame 111 to another HDR video frame 111, and / or change from one scene of the HDR video to another scene.

[0082] The device 100 is also used to acquire a pair of anchor points 120. The pair of anchor points 120 includes a first anchor point 121 and a second anchor point 122 of a tone mapping curve 130. Specifically, the device 100 can acquire a pair of anchor points 120 based on acquired HDR video frames 111 and / or acquired metadata 112. The device 100 can also be used to select a pair of anchor points 120 from a plurality of anchor points 120, for example, selecting a pair of anchor points 120 from multiple candidate pairs of anchor points 120. If a pair of anchor points 120 is used to approximate a tone mapping curve, and a tone mapping operation is performed based on the approximate tone mapping curve, selection can be based on a pair of anchor points 120 that provides the highest local contrast.

[0083] Furthermore, the device 100 is used to generate one or more curve parameters 131, 132 (e.g., curve parameters m_a and m_p) of a tone mapping curve 130 based on a pair of anchor points 120. The one or more curve parameters 131, 132 can thus define the tone mapping curve 130.

[0084] If device 100 is a decoder, it can also generate a tone mapping curve 130 based on one or more curve parameters 131, 132 (e.g., curve parameters m_a and m_p). Furthermore, device 100 can then use the generated tone mapping curve 130 to perform tone mapping on HDR video frames 111. The decoder or decoding device may include, or be connected to, a display and can output tone-mapped HDR video frames to the display.

[0085] If device 100 is an encoder, it can transmit one or more curve parameters 131, 132 (e.g., curve parameters m_a and m_p) as additional metadata 322b (see examples in Figures 3 and 4; additional metadata may also be referred to as artistic metadata). For this purpose, device 100 can use curve parameters 131, 132 as additional metadata 322b and can include the additional metadata 322b in the acquired metadata 112 to obtain enhanced metadata, which is then transmitted. For example, it can be transmitted to a decoder, which can then extract one or more curve parameters 131, 132 from the additional metadata 322b and generate a tone mapping curve 130 based on these curve parameters. Typically, tone mapping curves are only needed when displaying HDR video frames. Therefore, the encoder can also generate tone mapping curves, for example, where the encoder or encoding device includes or is connected to a display and outputs tone-mapped HDR video frames to the display, but it can also determine only one or more curve parameters of the tone mapping curve (e.g., for transmission or storage) without generating the tone mapping curve or the corresponding tone-mapped HDR video.

[0086] It should be noted that tone mapping curve 130 can be the "phoenix" tone mapping curve described below.

[0087] Device 100 (encoder or decoder) may include processing circuitry. Figure 1 (Not shown in the image), the processing circuitry is used to perform, conduct, or initiate various operations of the device 100 described herein. The processing circuitry may include hardware and software. The hardware may include analog or digital circuitry, or both. The digital circuitry may include components such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), or multi-purpose processors. In one embodiment, the processing circuitry includes one or more processors and non-transitory memory connected to the one or more processors. The non-transitory memory may carry executable program code that, when executed by the one or more processors, causes the device 100 to perform, conduct, or initiate the operations or methods described herein.

[0088] Specifically, the device 100 may include a processor for executing a computer program, which includes program code for executing method 1100, i.e., for controlling the device 100 to perform the above steps.

[0089] refer to Figure 2A , Figure 2B and Figure 2C Examples of tone mapping curves 130 that can be generated by the decoder are shown, with particular examples of tone mapping curves 130A, 130B and 130C in the PQ domain (x-axis or coordinate: input luminance in the PQ domain, y-axis or coordinate: output / tone mapping luminance in the PQ domain).

[0090] The obtained tone mapping curve 130 can be called the "Phoenix curve" and can be given by the following formula:

[0091]

[0092] As described above in the Summary of the Invention section of this invention, tone mapping curve 130 can be used to tone map HDR video frames 111 via a decoder.

[0093] For the "Phoenix curve", the parameter "m_b" can be fixed as the minimum display brightness (the minimum display brightness value in the PQ domain, or in other words, the PQ value of the minimum display brightness), the parameter m_n can be 1, and the parameter m_m can be 2.4. Furthermore, the two remaining variables are the parameters m_a and m_p, which can be one or more of the curve parameters 131 and 132 discussed above, or can be included therein. That is, device 100 can be used to generate the parameters m_a and m_p of the tone mapping curve 130.

[0094] It should be noted that the curve parameter m_p represents the brightness control factor (in particular, a larger value of m_p indicates a brighter mapped hue). Furthermore, the curve parameter m_a is a scaling factor that controls the maximum output brightness of the output pixel (obtained through hue mapping using hue mapping curve 130).

[0095] Other embodiments using the "Phoenix curve" can use other parameters, for example, m_m can be in the range of 1 to 5, and m_n can be in the range of 0.5 to 2.

[0096] The implementation may use other nonlinear tone mapping curves (in addition to the "Phoenix curve") and approximate the nonlinear tone mapping curves with piecewise linear curves having two adaptive anchor points to determine a pair of optimal anchor points for these other nonlinear tone mapping curves (e.g., regarding perceived quality).

[0097] The tone mapping curve 130 can be generated in the PQ domain. In other words, both the input L and the output of the tone mapping curve 130 can reference PQ values. The input L can range from 0 to 1, where a PQ value of 0 is 0 nits in the linear domain and a PQ value of 1 is 10000 nits in the linear domain. Furthermore, the output value can range from 0 to a PQ value equal to or less than the maximum display brightness in the PQ domain. The minimum and maximum display brightness depend on the actual display and may vary from display to display. Embodiments of the encoder and / or decoder know or at least assume that the minimum and maximum display brightness of the display for which one or more parameters of the tone mapping curve are determined.

[0098] Specifically, exemplary tone mapping curves 130A, 130B, and 130C (in...) can be generated by the decoder. Figures 2A to 2C (As shown in the diagram) The tone mapping curve 130A is generated based on different maximum input brightness (e.g., related to HDR video frame 111) and maximum display brightness (e.g., the display to which HDR video frame 111 will be tone-mapped). Furthermore, for example, m_p = 5.0. The tone mapping curve 130A can be generated by the decoder based on a maximum input brightness of 10,000 nits and a maximum display brightness of 500 nits. Furthermore, the tone mapping curve 130B can be generated by the decoder based on a maximum input brightness of 10,000 nits and a maximum display brightness of 1,000 nits. Furthermore, the tone mapping curve 130C can be generated by the decoder based on a maximum input brightness of 4,000 nits and a maximum display brightness of 1,000 nits.

[0099] Now for reference Figure 3A and Figure 3B A diagram illustrating exemplary metadata 112 is shown. Specifically, metadata 112 for the first mode is as follows: Figure 3A As shown, the metadata 112 used for the second mode is as follows: Figure 3BAs shown. The method 1100 according to an embodiment of the present invention can be executed in a first mode or a second mode. Therefore, the encoder and decoder can operate in these different modes. For example, referring to the CUVA HDR standard, the first mode is referred to below as "automatic mode" and the second mode may be referred to below as "art mode". In both modes, one or more curve parameters 131, 132 (e.g., curve parameters m_a and m_p) can be generated based on HDR video frame 111 and / or metadata 112, as described above. It should be noted that "art mode" does not mean that the generation of curve parameters 131 and 132 is done manually. Traditionally, in art mode, one or more curve parameters can be manually designed and inserted into metadata 112. However, according to an embodiment of the present invention, curve parameters 131, 132 (e.g., curve parameters m_a and m_p) can also be automatically generated by device 100 and / or method 1100 in art mode. Therefore, the expression "art mode" in the present invention does not mean that a human artist or colorist is involved in generating one or more curve parameters 131, 132.

[0100] • Automatic mode: mode flag tone_mapping_mode = 0.

[0101] In the first mode, the acquired metadata 112 may include, for example, reference to the CUVA HDR standard. Figure 3A The “basic metadata” 312 is shown in the figure. Furthermore, one or more curve parameters 131, 132 (e.g., curve parameters m_a and m_p) can be calculated based on the basic metadata 312 (included in metadata 112), particularly in the decoder acting as device 100. The basic metadata 312 may include typical image statistics, such as the minimum luminance value, maximum luminance value, average luminance value, and / or variance of luminance values ​​for HDR video frame 111 (e.g., HDR frame 111). The basic metadata 312 may include a minimum set of parameters sufficient to calculate curve parameters 131 and 132 (e.g., curve parameters m_a and m_p). For example, the basic metadata 312 may include the following four parameters (refer to the CUVA HDR standard):

[0102] • minimum_maxrgb_pq: The minimum maxrgb value of all pixels in the frame. The value is in the PQ field.

[0103] •average_maxrgb_pq: The average value of the maxrgb values ​​of all pixels in the frame.

[0104] •variance_maxrgb_pq: The difference between the 90th percentile of the maxrgb values ​​of all pixels in the frame and the 10th percentile of the maxrgb values ​​of all pixels in the frame.

[0105] • maximum_maxrgb_pq: The maximum value of maxrgb for all pixels in the frame. The value is in the PQ field.

[0106] The maxrgb value of a pixel is the maximum of its R, G, and B values. This value resides in the PQ field. All four parameters given above are values ​​in the PQ field (hence each value's name ends with _pq).

[0107] These parameters of the basic metadata 312 can be sent by the encoder and received by the decoder (e.g., when both the encoder and decoder are operating in automatic mode), and can be used by the decoder as the basis for generating one or more curve parameters 131, 132, that is, firstly, a pair of anchor points 120 are obtained according to the basic metadata 312, and then, as described above, curve parameters 131, 132 are generated according to the pair of anchor points 120.

[0108] In addition, metadata 112 may optionally include color metadata 322a (e.g., color adjustment or weighting factor). In the first mode, one or more curve parameters 131, 132 (e.g., curve parameters m_a and m_p) of tone mapping curve 130 (e.g., “phoenix” curve) can be calculated based on the basic metadata 312 of metadata 112 and optionally based on color metadata 322a.

[0109] • Artistic mode (mode flag tone_mapping_mode = 1):

[0110] In the second mode, one or more curve parameters 131, 132 (e.g., curve parameters m_a and m_p) can be determined at the encoder and then added to metadata 112 (specifically, to basic metadata 312, and optionally also to color metadata 322a), particularly as additional metadata 322b. The generated curve parameters 131, 132 (e.g., curve parameters m_a and m_p) can be embedded within the additional metadata 322b included in metadata 112. Metadata 112, and therefore, the additional metadata 322b, can then be provided to the decoder. The decoder (in art mode) can directly obtain the additional metadata (e.g., one or more curve parameters 131, 132, such as curve parameters m_a and m_p) from the bitstream (e.g., by directly parsing art mode metadata from the bitstream), thus eliminating the need to determine whether one or more curve parameters 131, 132 originate from the basic metadata and HDR video frames. This helps reduce the complexity and / or processing power on the decoder side.

[0111] Specifically, other metadata 322b can be referred to as art mode metadata 322b (refer to the CUVA HDR standard). In addition to the generated tone mapping curve parameters 131, 132 (e.g., the calculated "Phoenix" curve parameters m_a, m_p), other metadata 322b may also include cubic spline parameters, such as "TH1, TH2, TH3, TH_strength". TH1 and TH3 are the x-values ​​of the start and end points of the cubic spline curve. The y-values ​​are the same as the corresponding points on the Phoenix curve. Furthermore, TH2 is the x-value between the two. Additionally, TH_strength is the y-value of the cubic spline curve at TH2, which controls the height of the cubic spline curve. Within the range of TH1 and TH3, a cubic spline curve can be used, which can replace the basic tone mapping curve (Phoenix curve). Furthermore, one or more cubic spline curves within non-overlapping ranges can be supported.

[0112] The decoder can generate tone mapping curve 130 based on other metadata 322b. Specifically, the decoder can generate tone mapping curve 130 using one or more curve parameters 131, 132 (e.g., curve parameters m_a and m_p) included in the other metadata 322b. For example, the decoder can extract one or more curve parameters 131, 132 (e.g., curve parameters m_a and m_p) from the other metadata 322b. It should be noted that the decoder can also discard these curve parameters 131, 132 included in the other metadata 322b, and can calculate one or more new curve parameters 131, 132 based on the basic metadata 312 (e.g., in cases where the decoder does not support "Art Mode").

[0113] In summary, the difference between the first mode (e.g., automatic mode) and the second mode (e.g., artistic mode) is that in the first mode, the basic metadata 312 (of metadata 112) is transmitted to the decoder, and the decoder generates one or more curve parameters 131, 132 accordingly. In the second mode, one or more curve parameters 131, 132 are calculated on the encoder side and embedded as other metadata 322b in the metadata 112, and then sent to the decoder.

[0114] Figure 4 An embodiment of the invention is illustrated, wherein method 1100 is executed on the encoder side in a second mode (e.g., art mode). Specifically, Figure 4 A flowchart of method 400, performed by a system of encoder and decoder (which may be a conventional decoder), is shown, wherein the encoder generates one or more curve parameters 131, 132 and transmits the generated one or more curve parameters 131, 132 as other metadata 322b to the decoder.

[0115] In the following discussion, the system is illustrated by way of method 400, in which some steps (i.e., steps S401 to S404) are performed by the encoder and some steps (i.e., steps S405 and S406) are performed by the decoder, without limiting the present disclosure.

[0116] In step S401, the encoder acquires metadata 112, including basic metadata 312. For example, encoder 100 can acquire HDR frame 111 (of an HDR video source) and extract metadata 112, specifically basic metadata 312, from HDR video frame 111.

[0117] For example, the encoder can thus obtain one or more of the following parameters: maximum display brightness MaxDisplay (PQ value); minimum display brightness MinDisplay (PQ value); and the RGB field pixel buffer f[Nframe][3] of the current HDR video frame 111. f[Nframe][3] can be a two-dimensional array, where Nframe is the number of pixels in the current HDR video frame 111, and 3 represents 3 color channels, namely R, G, and B. For example, f

[11] [0] can be the R value of the 12th pixel in the raster scan order, f

[11] [1] is G, and f

[11] [2] is B.

[0118] In step S402, the encoder acquires a pair of anchor points 120 as described above (a more detailed description of how to acquire / select a pair of anchor points 120 is given below). For example, the encoder may calculate two anchor points including the first anchor point 121 and the second anchor point 122 described above. As explained below, the encoder may select a pair of anchor points 120, and the pair of anchor points 120 may be fine-tuned (e.g., according to the tone map that produces the highest local contrast).

[0119] In step S403, the encoder generates one or more curve parameters 131, 132 (e.g., curve parameters m_a and m_p) for the tone mapping curve 130 based on a pair of anchor points 120. For example, the encoder can calculate the curve parameters m_a and m_p for the aforementioned Phoenix curve based on a pair of anchor points 120.

[0120] In another example, the encoder can generate a curve parameter set P. tone_mapping This includes one or more of the following parameters (as described above): m_p, m_m, m_n, m_b, K1, K2, K3. However, m_p and m_a can be important, while the other parameters can be preset (m_m: 2.4; m_n: 1; K1, K2, and K3 are all 1).

[0121] In step S404, the encoder acquires / generates enhanced metadata 112 including other metadata 322b. For example, the encoder can embed the other metadata 322b (here exemplarily referred to as art mode metadata) into metadata 112, that is, add it to the basic metadata 312. The other metadata 322b includes one or more curve parameters 131, 132 (e.g., curve parameters m_a and m_p). Furthermore, the encoder can transmit the enhanced metadata 112, including the basic metadata 312 and the other metadata 322b, to the decoder.

[0122] In step S405, the decoder acquires curve parameters 131 and 132. For example, the decoder can extract one or more curve parameters 131 and 132 (e.g., parameters m_a and m_p) from the acquired enhanced metadata 112, and in particular from other metadata 322b.

[0123] In step S406, the decoder generates tone mapping curve 130. For example, the decoder can generate tone mapping curve 130 based on curve parameters 131 and 132 (obtained in step S405). For example, the decoder can generate a Phoenix curve as one or more curve parameters 131 and 132 based on m_a and m_p.

[0124] Figure 5Another embodiment of the invention is shown, wherein method 1100 is executed on the decoder side in a first mode (e.g., automatic mode). Specifically, Figure 5 A flowchart of method 500 performed by a system of encoders (which may be conventional encoders) and decoders is shown, wherein the decoder generates one or more curve parameters 131, 132 (e.g., curve parameters m_a and m_p).

[0125] In the following discussion, the system is illustrated by way of method 500, wherein step S501 is performed by the encoder and steps S502 to S504 are performed by the decoder, without limiting the present disclosure.

[0126] In step S501, the encoder acquires metadata 112. For example, the encoder can acquire HDR frame 111 (from an HDR video source) and extract metadata 112, specifically basic metadata 312, from the HDR video frame 111. The encoder can then provide the metadata 112 and the HDR video frame 111 to the decoder. That is, the decoder can receive the metadata 112 and the HDR video frame 111.

[0127] In step S502, the decoder acquires a pair of anchor points 120. For example, the decoder can acquire a pair of anchor points 120 based on the received HDR video frame 111 and / or basic metadata 312. The pair of anchor points 120 may include a first anchor point 121 and a second anchor point 122. As explained below, the encoder can select a pair of anchor points 120, and the pair of anchor points 120 can be fine-tuned (e.g., based on the tone map that produces the highest local contrast).

[0128] In step S503, the decoder generates one or more curve parameters 131, 132 (e.g., curve parameters m_a and m_p) of the tone mapping curve 130 based on a pair of anchor points 120. For example, the decoder 200 can calculate the curve parameters m_a and m_p of a Phoenix curve based on a pair of anchor points 120.

[0129] In step S504, the decoder can generate tone mapping curve 130. For example, the decoder can generate tone mapping curve 130 based on the acquired curve parameters m_a and m_p, that is, the decoder can generate the Phoenix curve as described above.

[0130] about Figure 4 and Figure 5In contrast, for example, due to video compression, the HDR video frame 111 on the decoder side may differ from the HDR video frame 111 on the encoder side. On the encoder side, method 1100 can be executed before encoding, thus processing the raw HDR video frame 111. Executing method 1100 on the encoder side reduces complexity (it is executed only once on the encoder side, rather than once on each decoder side, and it also provides better parameters, resulting in better quality, as it uses the raw HDR video frame to determine one or more curve parameters). However, on the decoder side, both the encoded and compressed HDR video frame 111 can be decoded and processed using method 1100.

[0131] Now for reference Figure 6 A flowchart of an exemplary method 600 for determining a pair of anchor points 120 for generating one or more curve parameters 131, 132 (e.g., curve parameters m_a and m_p), as described above, is shown. Specifically, method 600 describes an example of selecting a pair of anchor points 120 and fine-tuning the pair of anchor points 120 (e.g., according to a tone map that produces the highest local contrast), as described above. The pair of anchor points 120 for generating one or more curve parameters 131, 132 (e.g., curve parameters m_a and m_p) is selected from multiple pairs of anchor points 120. Generally, method 600 includes acquiring multiple pairs of anchor points 120 (via device 100); generating a piecewise linear curve 901 for each pair of anchor points 120; tone mapping HDR video frames 111 according to each piecewise linear curve 901 to acquire multiple tone-mapped HDR video frames; and selecting a pair of anchor points 120 from the multiple pairs of anchor points 120 for generating one or more curve parameters 131, 132 (e.g., curve parameters m_a and m_p) according to the multiple tone-mapped HDR video frames.

[0132] Method 600 can be executed by an encoder or a decoder. In the following discussion, method 600 is exemplarily described as a method typically executed by device 100 (e.g., Figure 1 (as shown in the illustration), without limiting this disclosure.

[0133] In steps S601a and S601b, device 100 acquires a pair of initial anchor points 120, specifically initial anchor point 1 (i.e., first initial anchor point 121) and initial anchor point 2 (i.e., second initial anchor point 122).

[0134] Figure 7 A diagram illustrating an example of obtaining the first initial anchor point 121.

[0135] For example, the y-coordinate value of the first initial anchor point 121 can be equal to the x-coordinate value of the first initial anchor point 121. The x-coordinate value of the first initial anchor point 121 can be selected within a range between a predetermined minimum threshold and a predetermined center threshold.

[0136] Specifically, the first initial anchor point 121 can be represented as (L3, F3N), where L3 is the x-coordinate value and F3N is the y-coordinate value of the first initial anchor point. Therefore, in one embodiment, L3 can be equal to the Average_dark value (PQ value) in the metadata 112, and F3N can be equal to the Perceptual_dark value (PQ value) of the target display. In another embodiment, L3 and F3N values ​​are equal and can be calculated using the following formula:

[0137] For all f[i] in (minCone, midLight),

[0138] Here, f[i] is the maximum value among f[i][0], f[i][1], and f[i][2], in other words, the maximum value among the R, G, and B values ​​of the i-th pixel or the maximum RGB value of the i-th pixel. Only the f[i] value between minCone (i.e., the minimum threshold) and midLight (i.e., the center threshold) is considered, and Nframe refers to the number of pixels with a maximum RGB value between minCone and midLight. Furthermore, the value of minCone can be 0.15, which is the PQ value of the lower threshold of the brightness range, where cone cells in the human visual system can perceive color. Additionally, the value of midLight can be 0.35, which is the lower limit of the skin tone brightness range recommended in the BT2408 standard.

[0139] The embodiments can be used to employ predetermined thresholds that represent or take into account features of the human visual system or human visual perception. For example, a minimum threshold can be set to a threshold brightness level below which the cone cells of the human visual system no longer perceive more color (e.g., minCone), thus providing less information. Therefore, a first anchor point is selected that is greater than the minimum threshold, such as greater than minCone. A center threshold and a maximum threshold can be selected that are related to color perception. For example, the center threshold can be set to a value representing or used for human skin color, such as a lower threshold for human skin color (e.g., midLight), and the maximum threshold can be set to a value representing white (e.g., defusingLight). Therefore, a second anchor point is selected between these two values ​​so that skin color and white are better controlled by the second anchor point. In one embodiment, the minimum threshold can be set to minCone, the center threshold can be set to midLight, and the maximum threshold can be set to defusingLight. Other embodiments may use only the thresholds described above and / or one or more of other thresholds.

[0140] Figure 8A diagram illustrating an example of a pair of initial anchor points 120 is shown, the pair including a first initial anchor point 121 and a second initial anchor point 122. The x-coordinate value of the second initial anchor point 122 can be selected within a range between a predetermined center threshold and a predetermined maximum threshold.

[0141] Specifically, the second initial anchor point 122 can be represented as [M1, N1N], where M1 is the x-coordinate value and N1N is the y-coordinate value of the second initial anchor point. M1 can be equal to the average_midLight (PQ value) in the metadata 112, and can be calculated by averaging all maxRGB values ​​within the ranges of midLight (i.e., the center threshold) and defusingLight (i.e., the maximum threshold). Furthermore, defusingLight = midLight + (MaxSource – midLight) * ratio, where the ratio can be preset to 4 / 6. Additionally, MaxSource can be the maximum value of the maxRGB values ​​in the source image (HDR video frame 111). MinSource can be the minimum value of the maxRGB values ​​in the source image (HDR video frame 111). The value of defusingLight can be calculated based on a lower threshold of the white light dissipation range, for example, as suggested in the BT2048 standard.

[0142] In addition, the y-coordinate value of the second initial anchor point 122 can be calculated based on the luminance value histogram of the HDR video frame 111.

[0143] Specifically, N1N can be the value of Perceptual_midLight (PQ value), and can be calculated based on the histogram of all maxRGB values ​​in the range of (midLight, defusingLight), as follows.

[0144] First, device 100 can calculate a histogram of maxRGB values ​​within the range of (minCone, MaxSource). The size of the histogram element can be set to (MaxSource – minCone) * V / U, where U and V are positive integers, U is recommended to be 6, and V is less than or equal to 3. Furthermore, device 100 can calculate the number of pixels with maxRGB values ​​within the range of (midLight, defusingLight) and name it Half_Num. Additionally, if the number of samples for a histogram element is greater than Half_Num, the element is called an HISA element.

[0145] Here, the HISA element represents the peak portion of the histogram. In other words, the HISA element comprises a larger proportion of pixels compared to other elements. Therefore, if the HISA element exists, it is more important than other elements because it includes significantly more pixels. Thus, anchor point calculation is based on the pixels within the HISA element. If the HISA element does not exist, it means the histogram is flat, all elements comprise a similar number of pixels, and no element is more important than others. In this case, anchor point 2 is not calculated based on a specific histogram element, but rather on all pixels between midLight and defusingLight.

[0146] Secondly, device 100 can calculate N1N based on the HISA elements. If one or more HISA elements exist, it is as follows:

[0147] For all f[i] in (midLight, defusingLight, and HISA elements),

[0148]

[0149] However, if the HISA element is not present, System 300 calculates N1N as follows:

[0150] For all f[i] in (midLight, defusingLightH),

[0151]

[0152] Where, difusingLightH = midLight + (MaxSource - midLight) * ratioH, it is recommended that ratioH be 5 / 6.

[0153] Typically, in one embodiment, device 100 can calculate the number of pixels whose brightness values ​​fall between a predetermined minimum threshold and a predetermined maximum threshold. Furthermore, device 100 can compare the values ​​of the histogram elements of a brightness value histogram with the calculated number of pixels. Then, if the values ​​of one or more histogram elements are greater than the calculated number, pixel brightness values ​​greater than a predetermined maximum display brightness value are clipped from the one or more histogram elements, and the y-coordinate value of a second anchor point is set to the average brightness value of all pixels belonging to the one or more histogram elements, wherein the values ​​of the one or more histogram elements are greater than the calculated number of pixels; and if the values ​​of one or more histogram elements are not greater than the calculated number, pixel brightness values ​​are clipped between a predetermined center threshold and a predetermined maximum threshold, and the y-coordinate value of the second anchor point is set to the average brightness value of all pixels, wherein the average value is between the predetermined center threshold and the predetermined maximum threshold of the HDR video frame 111.

[0154] In step S602, device 100 calculates one or more other candidate anchor points for the first anchor point 121 and the second anchor point 122, that is, one or more pairs of other anchor points 120. In particular, device 100 can obtain one or more pairs of other anchor points 120 based on a pair of initial anchor points 120.

[0155] For example, device 100 can obtain other candidate anchor points for a second anchor point 122 represented as (M1, N1N-M1*MaxDisplay*E / (MaxSource*10)), and can obtain other candidate anchor points for a first anchor point 121 represented as (L3, F3N*MaxDisplay*F / (MaxSource*10)). Device 100 can select different values ​​of E and F to obtain more pairs of candidate anchor points 120. The value of E can be in the range [1, 20], and the value of F can be in the range [1, 10].

[0156] When other anchor points 121, 122 are selected (by device 100), the x-coordinate value of the first anchor point 121 of a pair of other anchor points 120 can be the same as the x-coordinate value of the first initial anchor point 121, and / or the x-coordinate value of the second anchor point 122 of a pair of other anchor points 120 can be the same as the x-coordinate value of the second initial anchor point 122 of a pair of initial anchor points.

[0157] Furthermore, the y-coordinate value of the first anchor point 121 of a pair of other anchor points 120 may be different from the y-coordinate value of the first initial anchor point 121, and / or the y-coordinate value of the second anchor point 122 of a pair of anchor points 120 may be the same as the y-coordinate value of the second initial anchor point 122.

[0158] In step S603, device 100 generates a piecewise linear curve 901 for each pair of anchor points 120, that is, for each pair of anchor points 120, there is a pair of initial anchor points 120 and a pair or more other anchor points 120. Figure 9 A diagram illustrating an example of a piecewise linear curve 901 generated from a pair of anchor points 120.

[0159] Therefore, the piecewise linear curve 901 of each pair of anchor points 120 connects the predetermined minimum anchor point 902 with the corresponding first anchor point 121, connects the corresponding first anchor point 121 with the corresponding second anchor point 122, and connects the corresponding second anchor point 122 with the predetermined maximum anchor point 902.

[0160] For example, device 100 can select a pair of candidate anchor points 121, 122 together with (minSource, minDisplay) (i.e., minimum anchor point 902) and (maxSource, maxDisplay) (i.e., maximum anchor point 902) to generate a piecewise linear curve 901 for the pair of anchor points 120. The piecewise linear curve 901 can be used as a candidate curve for approximating the final tone mapping curve 130.

[0161] In step S604, device 100 performs tone mapping using piecewise linear curve 901 and calculates local contrast. Generally, device 100 can perform tone mapping on HDR video frames 111 according to each piecewise linear curve 901 to obtain multiple tone-mapped HDR video frames. Device 100 can also determine local contrast for each frame in the multiple tone-mapped HDR video frames to obtain multiple local contrast ratios.

[0162] For example, different values ​​of E and F can produce different pairs of anchor points 120, resulting in different piecewise linear curves 901. Furthermore, each piecewise linear curve 901 can be used to tone map the current HDR video frame 111 and calculate local contrast. Local contrast can be calculated for tone-mapped HDR video frames as follows:

[0163] (1) Device 100 can divide tone-mapped HDR video frames into smaller patches. The patch size can be 8×8, 16×16, 32×32 or 64×64.

[0164] (2) Device 100 can calculate the maximum maxRGB value of all pixels and the minimum maxRGB value of all pixels for each image block, and the local contrast is the difference between the maximum and minimum values.

[0165] (3) The device 100 can average the local contrast of all blocks and can select the average value as the local contrast value of the tone-mapped HDR video frame.

[0166] In step S605, device 100 then selects a pair of anchor points 120 from multiple pairs of anchor points 120 to generate one or more curve parameters 131, 132 (e.g., curve parameters m_a and m_p) based on multiple tone-mapped HDR video frames (specifically, based on multiple local contrast ratios). For example, device 100 may select a pair of anchor points 120 that produces the highest local contrast ratio.

[0167] For example, device 100 can select the values ​​E and F of the maximum local contrast of the HDR video frame that generates tone mapping, respectively, and calculate the final anchor points 121 and 122, namely [L3, F3] and [M1, N1].

[0168] In step S606, device 100 can acquire some preset parameters of tone mapping curve 130.

[0169] In step S607, device 100 calculates curve parameters 131 and 132. For example, device 100 can calculate parameters m_p and m_a of tone mapping curve 130.

[0170] For example, device 100 can use a selected pair of anchor points [L3, F3] and [M1, N1], and the following formula to calculate m_a and m_p:

[0171]

[0172]

[0173] Where m_m can be preset to 2.4.

[0174] In addition, device 100 can generate tone mapping curve 130. Tone mapping curve 130 is generated based on one or more curve parameters 131, 132. Optionally, device 100 can also send preset parameters for tone mapping curve 130.

[0175] For comparison, Figure 10 A tone mapping curve 130 generated based on one or more generated curve parameters 131, 132 is shown, and a piecewise linear curve 901 generated based on a selected pair of anchor points 121 is shown, from which one or more curve parameters 131, 132 are generated.

[0176] Figure 11 A flowchart of method 1100 for determining one or more curve parameters 131, 132, as provided in an embodiment of the present invention, is shown. Method 1100 can be performed by device 100 (i.e., the encoder or decoder described above).

[0177] Method 1100 includes step S1101: obtaining HDR video frame 111 and metadata 112 associated with HDR video frame 111.

[0178] Method 1100 further includes step S1102: obtaining a pair of anchor points 120 based on HDR video frame 111 and metadata 112, wherein the pair of anchor points 120 includes a first anchor point 121 and a second anchor point 122 of tone mapping curve 130.

[0179] Method 1100 further includes step S1103: generating one or more curve parameters 131, 132 of tone mapping curve 130 based on the pair of anchor points 120. For example, one or more curve parameters 131, 132 may be curve parameters m_a and m_p, respectively.

[0180] Figure 12 An example of a signal processing pipeline 1200 for an HDR dynamic tone mapping process is shown. The system input is HDR video, for example, HDR video frames of an HDR video. Generally, this HDR video can be the output of the post-production stage, where colorists edit the video using a color grading system to achieve better quality or certain artistic intentions. HDR videos have higher peak brightness, typically 1000 or 2000 nits, and in the near future 4000 or 10000 nits. Furthermore, the pixel values ​​of the video are in the PQ domain.

[0181] In HDR preprocessing block 1201, the HDR video remains identical to the input. However, the metadata is computable. Furthermore, in HDR video encoding block 1202, the HDR video is compressed, for example, by a video codec, such as one based on H.265 or any other video standard (national, international, or proprietary). Additionally, metadata is embedded in the header of the video stream sent from the encoder to the decoder (or stored on a storage medium for later retrieval by the decoder). In HDR video decoding block 1203, the decoder receives the HDR video stream, decodes the compressed video, and extracts the metadata from the header.

[0182] In addition, tone mapping is performed in the HDR dynamic tone mapping block 1204 to adapt the HDR video to the display capacity.

[0183] The invention has been described in conjunction with various embodiments and implementations as examples. However, based on a study of the drawings, the invention, and the independent claims, those skilled in the art will be able to understand and implement other variations in practicing the claimed invention. In the claims and the description, the word "comprising" does not exclude other elements or steps, and "a" does not exclude multiple elements or steps. A single element or other unit may fulfill the function of several entities or items described in the claims. The mere fact that certain measures are described in mutually different dependent claims does not mean that a combination of these measures cannot be used effectively.

Claims

1. A method for determining one or more curve parameters of a tone mapping curve, characterized in that, The method includes: Acquire high dynamic range (HDR) video frames and metadata associated with the HDR video frames; Based on the HDR video frame and the metadata, a pair of anchor points is obtained, wherein the pair of anchor points includes a first anchor point and a second anchor point of the tone mapping curve; Based on the pair of anchor points, generate the one or more curve parameters of the tone mapping curve; The tone mapping curve is given by the following equation: ， Where L is the brightness of the input pixel of the HDR video frame, m_n is a first value, specifically m_n=1, m_m is a second value, specifically m_m=2.4, m_b is a predetermined perceptual quantization (PQ) value, m_p is a brightness control factor, m_a is a scaling factor that defines the maximum brightness of the output pixel, and the one or more curve parameters include m_p and m_a.

2. The method according to claim 1, characterized in that, The process of obtaining a pair of anchor points includes: Obtain multiple pairs of anchor points; Generate a piecewise linear curve for each of the plurality of anchor points; The HDR video frames are tone-mapped according to each piecewise linear curve to obtain multiple tone-mapped HDR video frames. Based on the multiple tone-mapped HDR video frames, select one pair of anchor points from the multiple pairs of anchor points to generate the one or more curve parameters.

3. The method according to claim 2, characterized in that, The piecewise linear curve for each pair of anchor points connects a predetermined minimum anchor point to a corresponding first anchor point, the corresponding first anchor point to a corresponding second anchor point, and the corresponding second anchor point to a predetermined maximum anchor point.

4. The method according to claim 2 or 3, characterized in that, The selection of the pair of anchor points used to generate the one or more curve parameters includes: For each of the multiple tone-mapped HDR video frames, determine the local contrast to obtain multiple local contrast ratios; Based on the plurality of local contrasts, select one pair of anchor points from the plurality of anchor points to generate the one or more curve parameters.

5. The method according to claim 2 or 3, characterized in that, The acquisition of multiple pairs of anchor points includes: Obtain a pair of initial anchor points; Based on the initial pair of anchor points, obtain multiple other pairs of anchor points.

6. The method according to claim 5, characterized in that, The x-coordinate value of the first anchor point of a pair of other anchor points is the same as the x-coordinate value of the first anchor point of the pair of initial anchor points; and / or The x-coordinate value of the second anchor point of the other pair of anchor points is the same as the x-coordinate value of the second anchor point of the initial pair of anchor points.

7. The method according to claim 6, characterized in that, The y-coordinate value of the first anchor point of the other pair of anchor points is different from the y-coordinate value of the first anchor point of the initial pair of anchor points; and / or The y-coordinate value of the second anchor point of the other pair of anchor points is the same as the y-coordinate value of the second anchor point of the initial pair of anchor points.

8. The method according to claim 5, characterized in that, The x-coordinate value of the first anchor point of the pair of initial anchor points is selected within a range between a predetermined minimum threshold and a predetermined center threshold; and / or The x-coordinate value of the second anchor point of the pair of initial anchor points is selected within the range between the predetermined center threshold and the predetermined maximum threshold.

9. The method according to claim 8, characterized in that, The y-coordinate value of the first anchor point of the pair of initial anchor points is equal to the x-coordinate value of the first anchor point of the pair of initial anchor points.

10. The method according to claim 8 or 9, characterized in that, The y-coordinate value of the second anchor point of the pair of initial anchor points is calculated based on the luminance histogram of the HDR video frame.

11. The method according to claim 8 or 9, characterized in that, The method includes: Calculate the number of pixels whose brightness values ​​fall between the predetermined minimum threshold and the predetermined maximum threshold; The values ​​of the histogram elements of the brightness value histogram are compared with the calculated number of pixels; If the value of one or more histogram elements is greater than the calculated number of pixels, the brightness values ​​of pixels that are greater than the predetermined maximum display brightness value are clipped from the one or more histogram elements, and the y-coordinate value of the second anchor point is set to the average of the brightness values ​​of all pixels belonging to the one or more histogram elements, wherein the value of the one or more histogram elements is greater than the calculated number of pixels; If the values ​​of one or more histogram elements are not greater than the calculated number of pixels, the pixel brightness value of the pixel is cropped between the predetermined center threshold and the predetermined maximum threshold, and the y-coordinate value of the second anchor point is set to the average value of the brightness values ​​of all pixels, wherein the average value is between the predetermined center threshold and the predetermined maximum threshold of the HDR video frame.

12. The method according to any one of claims 1-3 and 6-9, characterized in that, Also includes: The tone mapping curve is generated based on the one or more curve parameters.

13. The method according to any one of claims 1-3 and 6-9, characterized in that, The method includes: Receive the metadata and the HDR video frame.

14. The method according to any one of claims 1-3 and 6-9, characterized in that, The method includes: The one or more curve parameters are transmitted as other metadata.

15. The method according to any one of claims 1-3 and 6-9, characterized in that, The method is performed by an encoder and / or a decoder.

16. An encoder for encoding HDR video frames, characterized in that, The encoder is used to perform the method according to any one of claims 1 to 11 and 14 to 15.

17. A decoder for decoding HDR video frames, characterized in that, The decoder is used to perform the method according to any one of claims 1 to 13.

18. A system for generating tone mapping curves, characterized in that, The system includes: The encoder according to claim 16; The decoder according to claim 17.

Images