A high dynamic range HDR video processing method, encoding device and decoding device

Abstract

The present application provides a method for processing a high dynamic range (HDR) video, an encoding device, and a decoding device. The method can determine the mode adopted by the HDR video at the encoding end according to a quality evaluation algorithm. The method includes: obtaining the dynamic metadata of the Nth frame of the HDR video according to a dynamic metadata generation algorithm; calculating the tone-mapping curve parameters of the Nth frame of the HDR video according to the dynamic metadata of the Nth frame of the HDR video; generating a tone-mapping curve according to the curve parameters; determining the distortion D caused by the tone-mapping curve according to a quality evaluation algorithm ′ ; comparing the D' and D T , and determining the mode adopted by the Nth frame of the HDR video, where the mode is an automatic mode or a director mode, and where the D T is a threshold value; determining the metadata of the Nth frame of the HDR video according to the determined mode adopted by the Nth frame of the HDR video

Description

[0001] This application is a divisional application of the Chinese patent application submitted to the China Patent Office on May 8, 2020, with application number 202010390234.7 and application name “A processing method, encoding device and decoding device for high dynamic range HDR video”. Technical Field

[0002] The present application relates to the field of image processing technology, and more specifically, to a processing method, encoding device and decoding device for high dynamic range HDR video. Background Art

[0003] The dynamic range mapping method is mainly used in the adaptation of the front-end high dynamic range (HDR) signal and the back-end HDR terminal display device, generally including the tone mapping process from high brightness to low brightness and the tone mapping process from low brightness to high brightness. Before the HDR video is encoded at the encoding end, it must first be preprocessed. During the preprocessing process, a mode adopted by the HDR video will be used, generally including automatic mode or director mode. The algorithm of the automatic mode is an empirical formula derived from a large amount of test data. In practice, it can cover most of the scenes, but the curve calculated by the automatic mode for some scenes cannot achieve good results. The director mode refers to the director colorist manually adjusting and determining the parameters or predicting and correcting the parameters of the automatic mode to further optimize the curve parameters to obtain the director mode. The corrected director mode can generally achieve better results. Therefore, how to choose the mode adopted by the HDR video is an urgent problem to be solved. Summary of the invention

[0004] The present application provides a high dynamic range HDR video processing method, encoding device and decoding device. The method can determine the mode adopted by the HDR video according to a quality evaluation algorithm at the encoding end.

[0005] In a first aspect, a method for processing a high dynamic range HDR video is provided, the method being applied to an encoding device, comprising: obtaining dynamic metadata of an HDR video of an Nth frame according to a dynamic metadata generation algorithm, wherein N is greater than 0; calculating tone-mapping curve parameters of the HDR video of the Nth frame according to the dynamic metadata of the HDR video of the Nth frame; generating a tone-mapping curve according to the curve parameters; determining a distortion D′ caused by the tone-mapping curve according to a quality evaluation algorithm; and comparing D′ with D T , determining the mode adopted by the HDR video of the Nth frame according to the comparison result, wherein the mode is an automatic mode or a director mode, wherein the HDR Tis a threshold value; and determining metadata of the HDR video of the Nth frame according to the determined mode adopted by the HDR video of the Nth frame.

[0006] By comparing the distortion D′ and D caused by the tone-mapping curve according to the quality evaluation algorithm T , determining whether the mode adopted by the HDR video of the Nth frame is the automatic mode or the director mode, can determine the mode adopted by the HDR video according to the distortion caused by the tone-mapping curve, thereby achieving a better tone mapping effect.

[0007] In combination with the first aspect, in certain implementations of the first aspect, the comparison of D′ and D T , determining the mode adopted by the HDR video of the Nth frame, including: when the D′ is greater than the D T When the HDR video of the Nth frame is determined to adopt the director mode, the D T The HDR video of the Nth frame adopts the distortion in the director mode; or, when the D′ is less than or equal to the D T When the HDR video of the Nth frame is determined to adopt the automatic mode.

[0008] In combination with the first aspect, in some implementations of the first aspect, the method further includes: obtaining an offset deltaC[N] of the tone-mapping curve parameters of the HDR video of the Nth frame based on the metadata of the frame (NM, N) in the M frame window, the tone-mapping curve parameters and the D′, wherein M is greater than 0.

[0009] In combination with the first aspect, in some implementations of the first aspect, an adjusted tone-mapping curve is generated according to the deltaC[N]; and a distortion D caused by the adjusted tone-mapping curve is determined according to the quality evaluation algorithm. T .

[0010] In combination with the first aspect, in certain implementations of the first aspect, determining the metadata of the HDR video of the Nth frame includes: generating the metadata of the HDR video of the Nth frame according to the mode adopted by the HDR video of the Nth frame, the dynamic metadata generation algorithm, and the offset deltaC[N] of the tone-mapping curve parameter.

[0011] In combination with the first aspect, in certain implementations of the first aspect, the method is applied to any one or more of the following preset configuration parameters: 1000nit, 500nit, and low dynamic range SDR.

[0012] In combination with the first aspect, in some implementations of the first aspect, the tone-mapping curve is as shown in formula (1):

[0013]

[0014] The curve parameters a and p are determined according to the dynamic information of the HDR video of the Nth frame, the curve parameters b, n and m are preset values, L is the input brightness, and L′ is the output brightness.

[0015] In combination with the first aspect, in some implementations of the first aspect, the method further includes: determining multiple offsets of the adjusted curve parameter p according to the predicted offset of the curve parameter p and K*basic step value, wherein K is a positive integer and K is greater than or equal to 0; determining curve parameters p″ and a″ corresponding to each of the multiple offsets of the curve parameter p according to the multiple offsets of the curve parameter p; generating multiple adjusted tone-mapping curves according to the parameters p″ and a″ corresponding to each of the multiple offsets; determining distortion D″ caused by each of the multiple adjusted tone-mapping curves according to the quality evaluation algorithm; selecting tone-mapping curve parameters p″ and a″ corresponding to any one D″ that meets the first condition from the multiple D″ as target adjusted curve parameters p′ and a′, the D T Any one D″ is selected.

[0016] In combination with the first aspect, in certain implementations of the first aspect, the multiple offsets of the adjusted curve parameter p are determined based on the predicted offset of the curve parameter p and K*basic step value, including: when K is 1, the determined offset is the offset of the predicted curve parameter p plus the basic step value, and the offset of the predicted curve parameter p minus the basic step value.

[0017] In combination with the first aspect, in some implementations of the first aspect, the first condition is to select a tone-mapping curve parameter with a smaller distortion D″ from two distortions D″.

[0018] In combination with the first aspect, in some implementations of the first aspect, determining the multiple offsets of the adjusted curve parameter p according to the predicted offset of the curve parameter p and the K*basic step value includes:

[0019] According to the basic delta±M*basic step value stepsize, multiple offsets are determined, wherein M is any positive integer less than or equal to K.

[0020] In combination with the first aspect, in certain implementations of the first aspect, the first condition is to select a minimum D" from multiple D"s or to select a D" that is smaller than a first threshold from multiple D".

[0021] In combination with the first aspect, in some implementations of the first aspect, the method further includes:

[0022] The offset of the predicted curve parameter p is determined according to formula (2):

[0023]

[0024] Wherein, deltaC[Nk] represents the offset used by the Nk-th frame, D[Nk] represents the distortion calculated by the quality assessment algorithm of the Nk-th frame, and M is the window length.

[0025] In combination with the first aspect, in certain implementations of the first aspect, determining the metadata of the HDR video of the Nth frame according to the determined mode adopted by the HDR video of the Nth frame includes: when the mode adopted by the HDR video of the Nth frame is the automatic mode, the metadata of the HDR video of the Nth frame includes dynamic metadata of the HDR video of the Nth frame; or, when the mode adopted by the HDR video of the Nth frame is the director mode, the metadata of the HDR video of the Nth frame includes the dynamic metadata of the HDR video of the Nth frame and P[N], where P[N] is a curve parameter P when the mode adopted by the HDR video of the Nth frame is the director mode.

[0026] In combination with the first aspect, in certain implementations of the first aspect, the metadata of the HDR video of the Nth frame includes a flag bit, and the flag bit is used to indicate that the mode adopted by the HDR video of the Nth frame is the director mode or the flag bit is used to indicate that the mode adopted by the HDR video of the Nth frame is the automatic mode.

[0027] In a second aspect, a method for processing high dynamic range HDR video is provided, which is applied to a decoding device and is characterized in that it includes: decoding to obtain an HDR video of the Nth frame and metadata of the HDR video of the Nth frame; and judging a mode adopted by the HDR video of the Nth frame according to a flag bit of the metadata.

[0028] In combination with the second aspect, in certain implementations of the second aspect, when the flag indicates that the mode adopted by the HDR video of the Nth frame is the automatic mode, the method further includes: calculating tone mapping tone-mapping curve parameters in the automatic mode according to the metadata of the HDR video of the Nth frame; generating a tone-mapping curve in the automatic mode according to the tone mapping tone-mapping curve parameters; and displaying the HDR video of the Nth frame according to the tone-mapping curve in the automatic mode and the HDR video of the Nth frame.

[0029] In combination with the second aspect, in certain implementations of the second aspect, when the flag indicates that the mode adopted by the HDR video of the Nth frame is the director mode, the method further includes: extracting tone-mapping curve parameters in the director mode from the metadata; generating a tone-mapping curve in the director mode according to the curve parameters; and displaying the HDR video of the Nth frame according to the tone-mapping curve in the director mode and the HDR video of the Nth frame.

[0030] In a third aspect, a method for processing high dynamic range HDR video is provided, the method being applied to an encoding device, comprising: obtaining dynamic metadata of an HDR video of an Nth frame according to a dynamic metadata generation algorithm; calculating tone-mapping curve parameters of the HDR video of the Nth frame; generating a tone-mapping curve according to the curve parameters; determining a distortion D′ caused by the tone-mapping curve according to a quality evaluation algorithm; obtaining an offset deltaC[N] of the curve parameters of the HDR video of the Nth frame according to metadata of frames (NM, N) in an M-frame window, the tone-mapping curve parameters and the D′, wherein N is greater than 0; generating an adjusted tone-mapping curve according to the offset deltaC[N] of the curve parameters of the HDR video of the Nth frame; generating metadata of the HDR video of the Nth frame in director mode according to the dynamic metadata generation algorithm and the offset deltaC[N] of the curve parameters of the HDR video of the Nth frame.

[0031] In combination with the third aspect, in some implementations of the third aspect, the method further includes: generating an adjusted tone-mapping curve according to an offset deltaC[N] of a curve parameter of the HDR video of the Nth frame; and determining a distortion D caused by the adjusted tone-mapping curve according to a quality evaluation algorithm. T , the D TThe offset deltaC[N+j] used to predict the N+j frame in director mode is used, where j is greater than or equal to 1 and less than or equal to M.

[0032] In combination with the third aspect, in certain implementations of the third aspect, the method is applied to any one or more of the following preset configuration parameters: 1000nit, 500nit, and low dynamic range SDR.

[0033] In conjunction with the third aspect, in some implementations of the third aspect, the tone-mapping curve is as shown in formula (3):

[0034]

[0035] Among them, parameters a and p are determined according to the dynamic metadata of the HDR video of the Nth frame, parameters b, n and m are preset values, L is the input brightness, and L′ is the output brightness.

[0036] In combination with the third aspect, in some implementations of the third aspect, the deltaC[N] is deltap[N], and the method further includes: determining multiple offsets of the adjusted curve parameter p according to the predicted offset of the curve parameter p and K*basic step value, wherein K is a positive integer and K is greater than or equal to 0; determining curve parameters p″ and a″ corresponding to each of the multiple offsets of the curve parameter p according to the multiple offsets of the curve parameter p; generating multiple adjusted tone-mapping curves according to the parameters p″ and a″ corresponding to each of the multiple offsets; determining distortion D″ caused by each of the multiple adjusted tone-mapping curves according to the quality evaluation algorithm; selecting tone-mapping curve parameters p″ and a″ corresponding to any one D″ that meets the first condition from the multiple D″ as target adjusted curve parameters p′ and a′, and the D T Any one D″ is selected.

[0037] In combination with the third aspect, in certain implementations of the third aspect, the multiple offsets of the adjusted curve parameter p are determined based on the predicted offset of the curve parameter p and K*basic step value, including: K is 1, the determined offset is the offset of the predicted curve parameter p plus the basic step value, and the offset of the predicted curve parameter p minus the basic step value.

[0038] In combination with the third aspect, in certain implementations of the third aspect, the first condition is to select a tone-mapping curve parameter with a smaller distortion D″ from two distortions D″.

[0039] In combination with the third aspect, in certain implementations of the third aspect, determining multiple offsets of the adjusted curve parameter p based on the predicted offset of the curve parameter p and K*basic step value includes: determining multiple offsets based on the basic delta±M*basic step value stepsize, where M is each positive integer less than or equal to K.

[0040] In combination with the third aspect, in certain implementations of the third aspect, the first condition is to select a minimum D" from multiple D"s or to select a D" that is smaller than a first threshold from multiple D".

[0041] In conjunction with the third aspect, in some implementations of the third aspect, the method further includes:

[0042] The offset of the predicted curve parameter p is determined according to formula (4):

[0043]

[0044] Wherein, deltaC[Nk] represents the offset used by the Nk-th frame, D[Nk] represents the distortion calculated by the quality assessment algorithm of the Nk-th frame, and M is the window length.

[0045] In combination with the third aspect, in certain implementations of the third aspect, determining the metadata of the HDR video of the Nth frame according to the director mode adopted by the HDR video of the Nth frame includes: when the mode adopted by the HDR video of the Nth frame is the director mode, the metadata of the HDR video of the Nth frame includes basic dynamic metadata and P[N] of the HDR video of the Nth frame, and P[N] is a curve parameter P when the mode adopted by the HDR video of the Nth frame is the director mode.

[0046] In combination with the third aspect, in certain implementations of the third aspect, the dynamic metadata includes a flag bit, and the flag bit is used to indicate that the mode adopted by the HDR video of the Nth frame is the director mode.

[0047] In a fourth aspect, a method for processing a high dynamic range HDR video is provided, the method being applied to a decoding device, comprising: obtaining an HDR video of an Nth frame and dynamic metadata of the HDR video of the Nth frame; calculating tone-mapping curve parameters of the HDR video of the Nth frame according to the dynamic metadata of the HDR video of the Nth frame; generating a tone-mapping curve according to the curve parameters; determining a distortion D′ caused by the tone-mapping curve according to a quality evaluation algorithm; and comparing D′ with D T , determine whether to use the automatic mode, where the D T is the threshold value.

[0048] In conjunction with the fourth aspect, in certain implementations of the fourth aspect, when the D′ is greater than the D T When , according to the metadata of the frame (NM, N) in the M frame window, the tone-mapping curve parameters and the D′, the offset deltaC[N] of the curve parameters of the HDR video of the Nth frame is obtained, wherein the N is greater than 0.

[0049] In combination with the fourth aspect, in some implementations of the fourth aspect, the method further includes: generating an adjusted tone-mapping curve according to the deltaC[N]; and determining a distortion D caused by the adjusted tone-mapping curve according to the quality evaluation algorithm. T .

[0050] In combination with the fourth aspect, in certain implementations of the fourth aspect, the dynamic metadata generation algorithm and the offset deltaC[N] of the curve parameters of the HDR video of the Nth frame are used to generate metadata of the HDR video of the Nth frame.

[0051] In combination with the fourth aspect, in certain implementations of the fourth aspect, the method is applied to any one or more of the following preset configuration parameters: 1000nit, 500nit, and low dynamic range SDR.

[0052] In conjunction with the fourth aspect, in some implementations of the fourth aspect, the tone-mapping curve is as shown in formula (5):

[0053]

[0054] Among them, parameters a and p are determined according to the dynamic metadata of the HDR video of the Nth frame, parameters b, n and m are preset values, L is the input brightness, and L′ is the output brightness.

[0055] In combination with the fourth aspect, in some implementations of the fourth aspect, the deltaC[N] is deltap[N], and the method further includes: determining multiple offsets of the adjusted curve parameter p according to the predicted offset of the curve parameter p and K*basic step value, wherein K is a positive integer and K is greater than or equal to 0; determining curve parameters p″ and a″ corresponding to each of the multiple offsets of the curve parameter p according to the multiple offsets of the curve parameter p; generating multiple adjusted tone-mapping curves according to the parameters p″ and a″ corresponding to each of the multiple offsets; determining distortion D″ caused by each of the multiple adjusted tone-mapping curves according to the quality evaluation algorithm; selecting tone-mapping curve parameters p″ and a″ corresponding to any one D″ that meets the first condition from the multiple D″ as target adjusted curve parameters p′ and a′, and the D T is the selected D″.

[0056] In combination with the fourth aspect, in certain implementations of the fourth aspect, the multiple offsets of the adjusted curve parameter p are determined based on the predicted offset of the curve parameter p and K*basic step value, including: K is 1, the determined offset is the offset of the predicted curve parameter p plus the basic step value, and the offset of the predicted curve parameter p minus the basic step value.

[0057] In combination with the fourth aspect, in certain implementations of the fourth aspect, the first condition is to select a tone-mapping curve parameter with a smaller distortion D″ from two distortions D″.

[0058] In combination with the fourth aspect, in certain implementations of the fourth aspect, the multiple offsets of the adjusted curve parameter p are determined based on the predicted offset of the curve parameter p and K*basic step value, including: determining multiple offsets based on the basic delta±M*basic step value stepsize, where M is each positive integer less than or equal to K.

[0059] In combination with the fourth aspect, in certain implementations of the fourth aspect, the first condition is to select the smallest D" from multiple D" or to select a D" that is smaller than a first threshold from multiple D".

[0060] In conjunction with the fourth aspect, in some implementations of the fourth aspect, the method further includes:

[0061] The offset of the predicted curve parameter p is determined according to formula (6):

[0062]

[0063] Wherein, deltaC[Nk] represents the offset used by the Nk-th frame, D[Nk] represents the distortion calculated by the quality assessment algorithm of the Nk-th frame, and M is the window length.

[0064] In a fifth aspect, an encoding device is provided, comprising: an acquisition unit, the acquisition unit being used to acquire dynamic metadata of an HDR video of an N-th frame according to a dynamic metadata generation algorithm; a processing unit, the processing unit being used to calculate tone-mapping curve parameters of the HDR video of the N-th frame according to the dynamic metadata of the HDR video of the N-th frame; the processing unit being further used to generate a tone-mapping curve according to the curve parameters; the processing unit being further used to determine a distortion D′ caused by the tone-mapping curve according to a quality evaluation algorithm; the processing unit being further used to compare D′ with D T , determine the mode adopted by the HDR video of the Nth frame, the mode being the automatic mode or the director mode, wherein the HDR T is a threshold value; and the processing unit is further used to determine metadata of the HDR video of the Nth frame according to the determined mode adopted by the HDR video of the Nth frame.

[0065] In conjunction with the fifth aspect, in some implementations of the fifth aspect, the processing unit is specifically configured to: when the D′ is greater than the D T When the HDR video of the Nth frame is determined to adopt the director mode, the D T The HDR video of the Nth frame adopts the distortion in the director mode; or, when the D′ is less than or equal to the D T When the HDR video of the Nth frame is determined to adopt the automatic mode.

[0066] In combination with the fifth aspect, in certain implementations of the fifth aspect, the processing unit is further used to: obtain an offset deltaC[N] of the curve parameters of the HDR video of the Nth frame based on the metadata of the frame (NM, N) in the M frame window, the tone-mapping curve parameters and the D′, wherein M is greater than 0.

[0067] In combination with the fifth aspect, in some implementations of the fifth aspect, the processing unit is further used to: generate an adjusted tone-mapping curve according to the deltaC[N]; and determine the distortion D caused by the adjusted tone-mapping curve according to the quality evaluation algorithm. T .

[0068] In combination with the fifth aspect, in certain implementations of the fifth aspect, the processing unit is specifically used to: generate metadata of the HDR video of the Nth frame according to the mode adopted by the HDR video of the Nth frame, the dynamic metadata generation algorithm, and the offset deltaC[N] of the tone-mapping curve parameter.

[0069] In combination with the fifth aspect, in certain implementations of the fifth aspect, the encoding device is applied to any one or more of the following preset configuration parameters: 1000nit, 500nit, and low dynamic range SDR.

[0070] In conjunction with the fifth aspect, in some implementations of the fifth aspect, the tone-mapping curve is as shown in formula (7):

[0071]

[0072] Among them, parameters a and p are determined according to the dynamic information of the HDR video of the Nth frame, parameters b, n and m are preset values, L is the input brightness, and L′ is the output brightness.

[0073] In combination with the fifth aspect, in some implementations of the fifth aspect, the processing unit is further used to: determine multiple offsets of the adjusted curve parameter p according to the predicted offset of the curve parameter p and K*basic step value, wherein K is a positive integer and K is greater than or equal to 0; determine curve parameters p″ and a″ corresponding to each of the multiple offsets of the curve parameter p according to the multiple offsets of the curve parameter p; generate multiple adjusted tone-mapping curves according to the parameters p″ and a″ corresponding to each of the multiple offsets; determine distortion D″ caused by each of the multiple adjusted tone-mapping curves according to the quality evaluation algorithm; select tone-mapping curve parameters p″ and a″ corresponding to any one D″ that meets the first condition from the multiple D″ as target adjusted curve parameters p′ and a′, and the D T is the selected D″.

[0074] In combination with the fifth aspect, in certain implementations of the fifth aspect, the processing unit is specifically used to: when K is 1, the determined offset is the offset of the predicted curve parameter p plus the basic step value, and the offset of the predicted curve parameter p minus the basic step value.

[0075] In combination with the fifth aspect, in certain implementations of the fifth aspect, the first condition is to select a tone-mapping curve parameter with a smaller distortion D″ from two distortions D″.

[0076] In combination with the fifth aspect, in certain implementations of the fifth aspect, the processing unit is specifically used to: determine multiple offsets based on the basic delta±M*basic step value stepsize, where M is each positive integer less than or equal to K.

[0077] In combination with the fifth aspect, in certain implementations of the fifth aspect, the first condition is to select the smallest D" from multiple D" or to select a D" that is smaller than a first threshold from multiple D".

[0078] In conjunction with the fifth aspect, in some implementations of the fifth aspect, the processing unit is further configured to:

[0079] The offset of the predicted curve parameter p is determined according to formula (8):

[0080]

[0081] Wherein, deltaC[Nk] represents the offset used by the Nk-th frame, D[Nk] represents the distortion calculated by the quality assessment algorithm of the Nk-th frame, and M is the window length.

[0082] In combination with the fifth aspect, in certain implementations of the fifth aspect, the processing unit is specifically used to: when the mode adopted by the HDR video of the Nth frame is the automatic mode, the metadata of the HDR video of the Nth frame includes dynamic metadata of the HDR video of the Nth frame; or, when the mode adopted by the HDR video of the Nth frame is the director mode, the metadata of the HDR video of the Nth frame includes the dynamic metadata of the HDR video of the Nth frame and P[N], where P[N] is the curve parameter P when the mode adopted by the HDR video of the Nth frame is the director mode.

[0083] In combination with the fifth aspect, in certain implementations of the fifth aspect, the metadata includes a flag bit, and the flag bit of the HDR video of the Nth frame is used to indicate that the mode adopted by the HDR video of the Nth frame is the director mode or the flag bit is used to indicate that the mode adopted by the HDR video of the Nth frame is the automatic mode.

[0084] In a sixth aspect, a decoding device is provided, comprising: an acquisition unit, which is used to decode and acquire an HDR video of the Nth frame and metadata of the HDR video of the Nth frame; and a processing unit, which is used to determine a mode adopted by the HDR video of the Nth frame according to a flag bit of the metadata.

[0085] In combination with the sixth aspect, in certain implementations of the sixth aspect, when the flag indicates that the mode adopted by the HDR video of the Nth frame is the automatic mode, the processing unit is further used to: calculate the tone mapping tone-mapping curve parameters in the automatic mode according to the metadata of the HDR video of the Nth frame; generate the tone-mapping curve in the automatic mode according to the tone mapping tone-mapping curve parameters; and display the HDR video of the Nth frame according to the tone-mapping curve in the automatic mode and the HDR video of the Nth frame.

[0086] In combination with the sixth aspect, in certain implementations of the sixth aspect, when the flag indicates that the mode adopted by the HDR video of the Nth frame is the director mode, the processing unit is further used to: extract the tone-mapping curve parameters in the director mode from the metadata; generate the tone-mapping curve in the director mode according to the curve parameters; and display the HDR video of the Nth frame according to the tone-mapping curve in the director mode and the HDR video of the Nth frame.

[0087] In the seventh aspect, a coding device is provided, including: an acquisition unit, the acquisition unit is used to acquire dynamic metadata of an HDR video of the Nth frame according to a dynamic metadata generation algorithm; a processing unit, the processing unit is used to calculate tone mapping tone-mapping curve parameters of the HDR video; the processing unit is also used to generate a tone-mapping curve according to the curve parameters; the processing unit is also used to determine the distortion D′ caused by the tone-mapping curve according to a quality evaluation algorithm; the processing unit is also used to obtain an offset deltaC[N] of the curve parameters of the HDR video of the Nth frame according to the metadata of the frames (NM, N) in the M frame window, the tone-mapping curve parameters and the D′, wherein M is greater than 0; the processing unit is also used to generate metadata of the HDR video of the Nth frame in director mode according to the dynamic metadata generation algorithm and the offset deltaC[N] of the curve parameters of the HDR video of the Nth frame.

[0088] In combination with the seventh aspect, in some implementations of the seventh aspect, the processing unit is further used to: generate an adjusted tone-mapping curve according to an offset deltaC[N] of the curve parameter of the HDR video of the Nth frame; and determine a distortion D caused by the adjusted tone-mapping curve according to a quality evaluation algorithm. T , the D TThe offset deltaC[N+j] used to predict the N+j frame in director mode is used, where j is greater than or equal to 1 and less than or equal to M.

[0089] In combination with the seventh aspect, in certain implementations of the seventh aspect, the encoding device is applied to any one or more of the following preset configuration parameters: 1000nit, 500nit, and low dynamic range SDR.

[0090] In conjunction with the seventh aspect, in some implementations of the seventh aspect, the tone-mapping curve is as shown in formula (9):

[0091]

[0092] The curve parameters a and p are determined according to the dynamic metadata of the HDR video of the Nth frame, the curve parameters b, n and m are preset values, L is the input brightness, and L′ is the output brightness.

[0093] In combination with the seventh aspect, in some implementations of the seventh aspect, the processing unit is further used to: determine multiple offsets of the adjusted curve parameter p according to the predicted offset of the curve parameter p and K*basic step value, wherein K is a positive integer and K is greater than or equal to 0; determine curve parameters p″ and a″ corresponding to each of the multiple offsets of the curve parameter p according to the multiple offsets of the curve parameter p; generate multiple adjusted tone-mapping curves according to the parameters p″ and a″ corresponding to each of the multiple offsets; determine distortion D″ caused by each of the multiple adjusted tone-mapping curves according to the quality evaluation algorithm; select tone-mapping curve parameters p″ and a″ corresponding to any one D″ that meets the first condition from the multiple D″ as target adjusted curve parameters p′ and a′, and the D T is the selected D″.

[0094] In combination with the seventh aspect, in certain implementations of the seventh aspect, the processing unit is specifically used to: when K is 1, the determined offset is the offset of the predicted curve parameter p plus the basic step value, and the offset of the predicted curve parameter p minus the basic step value.

[0095] In combination with the seventh aspect, in certain implementations of the seventh aspect, the first condition is to select a tone-mapping curve parameter with a smaller distortion D″ from two distortions D″.

[0096] In combination with the seventh aspect, in certain implementations of the seventh aspect, the processing unit is specifically used to: determine multiple offsets based on the basic delta±M*basic step value stepsize, where M is each positive integer less than or equal to K.

[0097] In combination with the seventh aspect, in certain implementations of the seventh aspect, the first condition is to select the smallest D" from multiple D" or to select a D" that is smaller than a first threshold from multiple D".

[0098] In conjunction with the seventh aspect, in some implementations of the seventh aspect, the processing unit is further configured to:

[0099] The offset of the predicted curve parameter p is determined according to formula (10):

[0100]

[0101] Wherein, deltaC[Nk] represents the offset used by the Nk-th frame, D[Nk] represents the distortion calculated by the quality assessment algorithm of the Nk-th frame, and M is the window length.

[0102] In combination with the seventh aspect, in certain implementations of the seventh aspect, the processing unit is specifically used to: when the mode adopted by the HDR video of the Nth frame is the director mode, the metadata of the HDR video of the Nth frame includes dynamic metadata and P[N] of the HDR video of the Nth frame, and P[N] is the curve parameter P when the mode adopted by the HDR video of the Nth frame is the director mode.

[0103] In combination with the seventh aspect, in certain implementations of the seventh aspect, the dynamic metadata includes a flag bit, and the flag bit is used to indicate that the mode adopted by the HDR video of the Nth frame is the director mode.

[0104] In an eighth aspect, a decoding device is provided, comprising an acquisition unit, the acquisition unit being used to acquire an HDR video of an Nth frame and dynamic metadata of the HDR video of the Nth frame; a processing unit, the processing unit being used to calculate tone-mapping curve parameters of the HDR video of the Nth frame according to the dynamic metadata of the HDR video of the Nth frame; the processing unit being further used to generate a tone-mapping curve according to the curve parameters; the processing unit being further used to determine a distortion D′ caused by the tone-mapping curve according to a quality evaluation algorithm; the processing unit being further used to compare D′ with D T , determine whether to use the automatic mode, where the D T is the threshold value.

[0105] In conjunction with the eighth aspect, in some implementations of the eighth aspect, the processing unit is specifically configured to: when the D′ is greater than the D T When , the offset deltaC[N] of the curve parameters of the HDR video of the Nth frame is obtained according to the metadata of the frame (NM, N) in the M frame window, the tone-mapping curve parameters and the D′, wherein the M is greater than 0.

[0106] In combination with the eighth aspect, in some implementations of the eighth aspect, the processing unit is further used to: generate an adjusted tone-mapping curve according to the deltaC[N]; and determine the distortion D caused by the adjusted tone-mapping curve according to the quality evaluation algorithm. T .

[0107] In combination with the eighth aspect, in certain implementations of the eighth aspect, the decoding device is applied to any one or more of the following preset configuration parameters: 1000nit, 500nit, and low dynamic range SDR.

[0108] In conjunction with the eighth aspect, in some implementations of the eighth aspect, the tone-mapping curve is as shown in formula (11):

[0109]

[0110] Among them, parameters a and p are determined according to the dynamic metadata of the HDR video of the Nth frame, parameters b, n and m are preset values, L is the input brightness, and L′ is the output brightness.

[0111] In combination with the eighth aspect, in certain implementations of the eighth aspect, the processing unit is further used to: determine multiple offsets of the adjusted curve parameter p according to the predicted offset of the curve parameter p and K*basic step value, wherein K is a positive integer and K is greater than or equal to 0; determine curve parameters p″ and a″ corresponding to each of the multiple offsets of the curve parameter p according to the multiple offsets of the curve parameter p; generate multiple adjusted tone-mapping curves according to the parameters p″ and a″ corresponding to each of the multiple offsets; determine distortion D″ caused by each of the multiple adjusted tone-mapping curves according to the quality evaluation algorithm; select tone-mapping curve parameters p″ and a″ corresponding to any one D″ that meets the first condition from the multiple D″ as target adjusted curve parameters p′ and a′, and the D T Any one D″ is selected.

[0112] In combination with the eighth aspect, in certain implementations of the eighth aspect, the processing unit is specifically used to: when K is 1, the determined offset is the offset of the predicted curve parameter p plus the basic step value, and the offset of the predicted curve parameter p minus the basic step value.

[0113] In combination with the eighth aspect, in certain implementations of the eighth aspect, the first condition is to select a tone-mapping curve parameter with a smaller distortion D″ from two distortions D″.

[0114] In combination with the eighth aspect, in certain implementations of the eighth aspect, the processing unit is specifically used to: determine multiple offsets based on the basic delta±M*basic step value stepsize, where M is each positive integer less than or equal to K.

[0115] In combination with the eighth aspect, in certain implementations of the eighth aspect, the first condition is to select the smallest D" from multiple D" or to select a D" that is smaller than a first threshold from multiple D".

[0116] In conjunction with the eighth aspect, in some implementations of the eighth aspect, the processing unit is further configured to:

[0117] The offset of the predicted curve parameter p is determined according to formula (12):

[0118]

[0119] Wherein, deltaC[Nk] represents the offset used by the Nk-th frame, D[Nk] represents the distortion calculated by the quality assessment algorithm of the Nk-th frame, and M is the window length.

[0120] In the ninth aspect, a coding device is provided, which includes at least one processor and a memory, and the at least one processor is used to execute the method in the above first aspect or any possible implementation of the first aspect, or to execute the above third aspect or any possible implementation of the third aspect.

[0121] In the tenth aspect, a decoding device is provided, which includes at least one processor and an interface circuit, and the at least one processor is used to execute the method in the second aspect or any possible implementation of the second aspect, or to execute the method in the fourth aspect or any possible implementation of the fourth aspect.

[0122] In the eleventh aspect, a computer program product is provided, which includes a computer program, which, when executed by a processor, is used to execute the method in the first aspect or any possible implementation of the first aspect, or to execute the method in the second aspect or any possible implementation of the second aspect, or to execute the method in the third aspect or any possible implementation of the third aspect, or to execute the method in the fourth aspect or any possible implementation of the fourth aspect.

[0123] In the twelfth aspect, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed, it is used to execute the method in the first aspect or any possible implementation of the first aspect, or to execute the method in the second aspect or any possible implementation of the second aspect, or to execute the method in the third aspect or any possible implementation of the third aspect, or to execute the method in the fourth aspect or any possible implementation of the fourth aspect.

[0124] In the thirteenth aspect, a chip is provided, comprising: a processor, for calling and running a computer program from a memory, so that a communication device equipped with the chip executes the method in the first aspect or any possible implementation of the first aspect, or executes the method in the second aspect or any possible implementation of the second aspect, or is used to execute the method in the third aspect or any possible implementation of the third aspect, or executes the method in the fourth aspect or any possible implementation of the fourth aspect.

[0125] Optionally, the chip may further include a memory in which instructions are stored, and the processor is used to execute the instructions stored in the memory or instructions derived from other sources.

[0126] In a fourteenth aspect, a computer-readable storage medium stores a bitstream, the bitstream including encoded video data, the bitstream being obtained according to an encoding method, the encoding method comprising: obtaining dynamic metadata of an HDR video of an N-th frame according to a dynamic metadata generation algorithm; calculating tone-mapping curve parameters of the HDR video of the N-th frame according to the dynamic metadata of the HDR video of the N-th frame; generating a tone-mapping curve according to the curve parameters; performing display adaptation on the HDR video of the N-th frame according to the tone-mapping curve; determining a distortion D′ caused by the tone-mapping curve according to a quality evaluation algorithm; if N>0, obtaining an offset deltaC[N] of the tone-mapping curve parameters of the HDR video of the N-th frame according to the metadata of the frame (NM, N) in an M-frame window, the tone-mapping curve parameters and D′; if D′ is greater than DT , determine that the mode adopted by the HDR video of the Nth frame is the director mode, where the D T is a threshold value; when the mode adopted by the HDR video of the Nth frame is the director mode, metadata of the HDR video of the Nth frame is generated according to the dynamic metadata generation algorithm and the deltaC[N].

[0127] In a fifteenth aspect, a method for storing a code stream includes: generating a code stream, the code stream being a code stream generated according to the encoding method of the first aspect; and storing the code stream in a storage medium.

[0128] In a sixteenth aspect, a method for transmitting a code stream includes: obtaining a code stream, which is a code stream generated according to the encoding method of the first aspect; transmitting the code stream to a storage medium or to a display device for decoding and display. BRIEF DESCRIPTION OF THE DRAWINGS

[0129] Figure 1 Schematic diagram of a system for applying a high dynamic range HDR video processing method provided in an embodiment of the present application;

[0130] Figure 2 It is a dynamic mapping schematic diagram;

[0131] Figure 3 is the image of the PQ photoelectric transfer function;

[0132] Figure 4 is the image of HLG photoelectric transfer function;

[0133] Figure 5 is the image of the SLF photoelectric transfer function;

[0134] Figure 6 is a schematic flow chart of a method for processing a high dynamic range HDR video according to an embodiment of the present application;

[0135] Figure 7 This is a schematic block diagram of a system for applying a high dynamic range HDR video processing method provided by the present application;

[0136] Figure 8 This is a system schematic diagram of a method for processing high dynamic range HDR video provided by the present application;

[0137] Fig. 9 is a schematic flow chart of a method for processing a high dynamic range HDR video according to an embodiment of the present application;

[0138] Fig.10 is a schematic flow chart of a method 600 for processing a high dynamic range HDR video according to an embodiment of the present application;

[0139] Fig.11 This is a schematic block diagram of a system for applying a high dynamic range HDR video processing method provided by the present application;

[0140] Fig.12 is a schematic flow chart of a method for processing a high dynamic range HDR video according to an embodiment of the present application;

[0141] Fig.13 This is a schematic block diagram of a system for applying a high dynamic range HDR video processing method provided by the present application;

[0142] Fig.14 is a schematic flow chart of a method for processing a high dynamic range HDR video according to an embodiment of the present application;

[0143] Fig.15 A schematic block diagram of a device according to an embodiment of the present application is shown;

[0144] Fig.16 A schematic diagram of the structure of a terminal device provided for this application. DETAILED DESCRIPTION

[0145] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0146] Figure 1 1 is a schematic diagram of a system 100 for applying a method for processing a high dynamic range HDR video provided in an embodiment of the present application. Figure 1 As shown, an HDR end-to-end processing system is generally divided into four parts: an HDR preprocessing module 110, an HDR video encoding module 120, an HDR video decoding module 130 and an HDR dynamic tone mapping module 140, wherein the HDR preprocessing module 110 and the HDR video encoding module 120 are encoding ends, and the HDR video decoding module 130 and the HDR dynamic tone mapping module 140 are decoding ends.

[0147] The HDR video is input to the HDR preprocessing module 110. The HDR video may be a captured video or an HDR video processed by a colorist or a device using an algorithm. The pixel values ​​of the HDR video have been photoelectrically transferred. For example, the pixel values ​​of the HDR video are all values ​​in the PQ domain.

[0148] The HDR preprocessing module 110 is responsible for extracting static metadata and dynamic metadata. If the HDR video adopts the director mode, the HDR preprocessing module 110 calculates the curve parameters of the tone mapping of the director mode, and the curve parameters of the tone mapping of the director mode will be written into the metadata.

[0149] The HDR video encoding module 120 encodes the HDR video and metadata, and the metadata is embedded in the user-defined part of the bitstream. The HDR video encoding module 120 can use any standard encoder, such as High Efficiency Video Coding (HEVC) or the second generation digital audio and video coding technology standard (Audio Video Coding Standard 2, AVS2). After the HDR video encoding module completes the encoding, the encoding end can transmit the bitstream to the decoding end.

[0150] The HDR video decoding module 130 at the decoding end decodes the bitstream according to the standard corresponding to the bitstream format, and outputs HDR decoded video and HDR metadata.

[0151] The HDR dynamic tone mapping module 140 calculates the tone mapping curve according to the decoded HDR video and metadata, as well as the peak brightness of the display device, performs tone mapping, and finally outputs it to the display. If the HDR video uses the director mode, the HDR dynamic tone mapping module does not need to calculate and directly uses only the curve parameters in the metadata.

[0152] Before the HDR video is input to the HDR preprocessing module 110, the HDR video pixel value needs to be photoelectrically transferred. This is mainly because the maximum brightness information of the display device cannot reach the brightness information of the real world, and we browse the image through the display device, so the photoelectric transfer function is needed. In order to understand the photoelectric transfer more clearly, it is necessary to introduce the concept of dynamic range.

[0153] Dynamic Range is used in many fields to represent the ratio of the maximum value to the minimum value of a variable. In digital images, the dynamic range represents the ratio between the maximum grayscale value and the minimum grayscale value within the displayable range of the image. The dynamic range in nature is quite large. The brightness of a night scene under the starry sky is about 0.001cd / m2, and the brightness of the sun itself is as high as 1,000,000,000cd / m2. Such a dynamic range reaches the order of 1,000,000,000 / 0.001=1013. However, in real scenes in nature, the brightness of the sun and the brightness of the stars will not be obtained at the same time. For natural scenes in the real world, the dynamic range is in the range of 10-3 to 106. In most of the current color digital images, each channel of R, G, and B is stored in an 8-bit byte. That is to say, the representation range of each channel is 0 to 255 grayscale levels. Here, 0 to 255 is the dynamic range of the image. Since the dynamic range of the same scene in the real world is in the range of 10-3 to 106, we call it high dynamic range (HDR). The dynamic range of ordinary pictures is low dynamic range (LDR). The imaging process of a digital camera is actually the mapping of the high dynamic range of the real world to the low dynamic range of the photo. The mapping of the high dynamic range in the real world to the low dynamic range of the photo is often a nonlinear process, such as Figure 2 As shown, Figure 2 A schematic diagram showing a dynamic mapping of a high dynamic range in the real world to a low dynamic range of a display device. Figure 2 The real-world dynamic range is around 80 to 2000, and the dynamic range mapped to the display device is around 1 to 200.

[0154] Standard dynamic range images correspond to high dynamic range images. The 8-bit images in the traditional jpeg format can be considered as standard dynamic range images. Before the emergence of cameras that can shoot HDR images, traditional cameras can only record the captured light information within a certain range by controlling the exposure value. Since the maximum brightness information of the display device cannot reach the brightness information of the real world, and we browse images through the display device, a photoelectric transfer function is required. The early display device was a cathode ray tube (CRT) display, and its photoelectric transfer function was the Gamma function. The ITU-R Recommendation BT.1886 standard defines this photoelectric transfer function based on the "Gamma" function, as shown in formula (1):

[0155]

[0156] After the above conversion, the image quantized to 8 bits is a traditional SDR image. The SDR image and transfer function perform well on traditional display devices (illuminance is around 100 cd / m2).

[0157] However, with the upgrade of display devices, the illumination range of display devices continues to increase. The existing consumer-level HDR display is 600cd / m2, and the illumination information of high-end HDR display can reach 2000cd / m2, which far exceeds the illumination information of SDR display devices. The photoelectric conversion function in the ITU-R Recommendation BT.1886 standard cannot well express the display performance of HDR display devices. Therefore, an improved photoelectric transfer function is needed to adapt to the upgrade of display devices. At present, there are three common photoelectric conversion functions: Perception quantization (PQ), Hybrid Log-Gamma (HLG) and Scene Luminance Fidelity (SLF). The following introduces the three curves.

[0158] PQ photoelectric transfer function, the photoelectric transfer function here is different from the traditional "Gamma" transfer function. It proposes a perceptual quantization transfer function based on the brightness perception model of the human eye. The PQ photoelectric transfer function represents the conversion relationship between the linear signal value of the image pixel and the nonlinear signal value in the PQ domain. The image of the PQ photoelectric transfer function can be referenced Figure 3 , Figure 3 The image of the PQ photoelectric transfer function is shown. The PQ photoelectric transfer function can be expressed as formula (2):

[0159]

[0160] Among them, the parameters in formula (2) can be calculated according to formula (3):

[0161]

[0162] in,

[0163] L represents the linear signal value, and its value is normalized to [0, 1].

[0164] L′ represents the nonlinear signal value, and its value range is [0, 1].

[0165] is the PQ photoelectric transfer coefficient.

[0166] is the PQ photoelectric transfer coefficient.

[0167] PQ photoelectric transfer coefficient.

[0168] PQ photoelectric transfer coefficient.

[0169] PQ photoelectric transfer coefficient.

[0170] The HLG photoelectric transfer function is improved based on the traditional Gamma curve. Figure 4 is the image of the HLG photoelectric transfer function. The HLG photoelectric transfer function applies the traditional Gamma curve in the low segment and supplements the log curve in the high segment. The HLG photoelectric transfer function represents the conversion relationship from the linear signal value of the image pixel to the nonlinear signal value of the HLG domain. The HLG photoelectric transfer function can be expressed as formula (4):

[0171]

[0172] in,

[0173] L represents the linear signal value, and its value range is [0, 12].

[0174] L' represents the nonlinear signal value, and its value range is [0, 1].

[0175] a=0.17883277, HLG photoelectric transfer coefficient,

[0176] b=0.28466892, HLG photoelectric transfer coefficient,

[0177] c=0.55991073, HLG photoelectric transfer coefficient.

[0178] The SLF photoelectric transfer function is the optimal curve obtained based on the brightness distribution of the HDR scene while satisfying the optical characteristics of the human eye. Figure 5 , Figure 5 is the image of SLF photoelectric transfer function. SLF photoelectric transfer curve represents the conversion relationship between the linear signal value of image pixel and the nonlinear signal value in SLF domain. The conversion relationship between the linear signal value of image pixel and the nonlinear signal value in SLF domain is shown in formula (5):

[0179]

[0180] Among them, the parameters in formula (5) can be calculated according to formula (6):

[0181]

[0182] in,

[0183] L represents the linear signal value, and its value is normalized to [0,1].

[0184] L' represents the nonlinear signal value, and its value range is [0,1].

[0185] p = 2.3, SLF photoelectric transfer coefficient,

[0186] m=0.14,SLF photoelectric transfer coefficient,

[0187] a=1.12762,SLF photoelectric transfer coefficient,

[0188] b = -0.12762, SLF photoelectric transfer coefficient.

[0189] The HDR preprocessing module 110 is responsible for extracting static metadata and dynamic metadata, which mainly adapts the front-end HDR signal and the back-end HDR terminal display device through dynamic range mapping. For example, the front end is a 4000nit light signal (nit unit of light intensity), and the HDR display capability of the back-end HDR terminal display device (TV, IPAD) is only 500nit. Therefore, how to map the 4000nit signal to the 500nit device is a high-to-low tone-mapping process. Another is that the front end is a 100nit SDR signal collected, and the display end is a 2000nit TV signal. If the 100nit signal is better displayed on a 2000nit device, it is another low-to-high tone-mapping process.

[0190] Dynamic range mapping can be divided into static and dynamic. The static mapping method is to use a single piece of data to perform the overall tone-mapping process based on the same video content or the same hard disk content, that is, the processing curve is usually the same. The advantage of this method is that it carries less information and the processing flow is relatively simple; the disadvantage of this method is that each scene uses the same curve for tone-mapping, which will cause information loss in some scenes. For example, if the curve focuses on protecting the bright area, some details will be lost in some extremely dark scenes, or simply invisible, which will affect the experience. The dynamic mapping method is to make dynamic adjustments based on the content of a specific area, each scene, or each frame. The advantage of this method is that different curves can be processed according to specific areas, each scene, or each frame, so that the processing results will be better. The disadvantage is that each frame or each scene needs to carry relevant scene information, and the amount of information carried is large.

[0191] During the preprocessing process, a mode used by HDR video will be used, such as the automatic mode or the director mode. The algorithm of the automatic mode is an empirical formula derived from a large amount of test data. In practice, it can cover most scenes, but the curve calculated by the automatic mode for some scenes cannot achieve good results. The director mode refers to the director's mode obtained by manually adjusting and determining the parameters of the automatic mode or predicting and correcting the parameters of the automatic mode to further optimize the curve parameters. The corrected director mode can generally achieve better results.

[0192] In view of this, the present application proposes a method for processing high dynamic range HDR video, which can determine the mode adopted by the HDR video and thus achieve a better tone mapping effect.

[0193] Combine the following Figure 6 A method for processing high dynamic range HDR video provided by the present application is described in detail. Figure 6 is a schematic flow chart of a method 200 for processing a high dynamic range HDR video according to an embodiment of the present application. The method 200 can be applied in Figure 1 The scenario shown can of course also be applied to other scenarios requiring video processing, and the embodiments of the present application are not limited here.

[0194] It should also be understood that in the embodiments of the present application, the method is described by taking the encoding device as the execution subject of the execution method as an example. As an example but not a limitation, the execution subject of the execution method may also be a chip, a chip system, or a processor applied to the encoding device.

[0195] like Figure 6 As shown, Figure 6 The method 200 shown in FIG. 2 may include S210 to S260. Figure 6 Each step in method 200 is described in detail.

[0196] S210, the encoding device obtains dynamic metadata of the Nth frame HDR video according to a dynamic metadata generation algorithm, where N is greater than 0. The Nth frame HDR video is referred to as the Nth frame HDR video.

[0197] S220: The encoding device calculates tone-mapping curve parameters of the HDR video of the Nth frame according to the dynamic metadata of the HDR video of the Nth frame.

[0198] S230: The encoding device generates a tone-mapping curve according to the curve parameters.

[0199] S240: The encoding device determines the distortion D′ caused by the tone-mapping curve according to a quality evaluation algorithm.

[0200] S250, the encoding device compares D′ and D T , determine the mode adopted by the HDR video of the Nth frame, the mode being the automatic mode or the director mode, wherein the HDR T is the threshold value.

[0201] S260: The encoding device determines metadata of the HDR video of the Nth frame according to the determined mode adopted by the HDR video of the Nth frame.

[0202] In the method 200, the distortion D′ and D′ caused by the tone-mapping curve are determined according to the quality evaluation algorithm by comparing T , determining whether the mode adopted by the HDR video of the Nth frame is the automatic mode or the director mode, can determine the mode adopted by the HDR video according to the distortion caused by the tone-mapping curve, thereby achieving a better tone mapping effect.

[0203] In step S210, the dynamic metadata of the HDR video of the Nth frame is obtained according to the dynamic metadata generation algorithm. The dynamic metadata generally refers to the basic dynamic metadata, which generally includes the minimum value of the maxrgb value of all pixels in the current frame or current scene (minimum_maxrgb_pq), the average value of the maxrgb value of all pixels in the current frame or current scene (average_maxrgb_pq), the value corresponding to the 90% percentile of the maxrgb value of all pixels in the current frame or current scene minus the value corresponding to the 10% percentile (variance_maxrgb_pq) and the maximum value of the maxrgb value of all pixels in the current frame or current scene (maximum_maxrgb_pq), where the maxrgb value is the maximum value of the three components of R, G, and B of a certain pixel. Note that here refers to the value of the PQ domain, that is, between 0 and 1.

[0204] In step S220, tone-mapping curve parameters of the HDR video of the Nth frame are calculated according to the dynamic metadata of the HDR video of the Nth frame.

[0205] The tone-mapping curve parameters can be a variety of curve parameters, such as the sigmoidal curve proposed by Dolby and the Bessel curve proposed by Samsung. The curve schemes proposed by Dolby and Samsung are all private schemes, and their specific implementation requires their authorization.

[0206] The tone-mapping curve parameter may also be a dynamic tone-mapping curve based on the SLF curve. The dynamic tone-mapping curve based on the SLF curve is in the form of formula (7):

[0207]

[0208] Among them, the value range of a is between 0.0 and 1.0, the value range of b is between 0.0 and 1.0, and the value range of p and m is between 0.1 and 60. L′ represents the output brightness, which can be a rational number in the range of 0.0 to 1.0, and L represents the input brightness, which can be a rational number in the range of 0.0 to 1.0. The curve shape is not flexible enough and cannot provide more diverse shapes. Specifically, the curve only has C, reverse S and reverse C types, and cannot provide positive S type. If the image is a scene with strong contrast between light and dark, a positive S-type curve is required.

[0209] The tone-mapping curve parameter may also be in the form of a tone-mapping curve as shown in formula (8):

[0210]

[0211] Among them, the value range of a is between 0.0 and 1.0, the value range of b is between 0.0 and 1.0, and the value range of p, n and m is between 0.1 and 60. L′ represents the output brightness, for example, it can be a rational number, ranging from 0.0 to 1.0, and L represents the input brightness, for example, it can be a rational number, ranging from 0.0 to 1.0, and L is a pixel.

[0212] The specific values ​​of a, b, p, n and m are determined based on the dynamic key information of a specific area, each frame or each scene, such as dynamic metadata. Among them, a is mainly related to the maximum brightness value of the scene, or the maximum parameters of the shooting device of the scene and the maximum display capability of the display device. b is related to the minimum value of the scene and the minimum display capability of the display device. P, n, and m are related to the overall style of the scene.

[0213] in,

[0214]

[0215] b = minimum value of display device - F (minimum value of scene or minimum value of shooting device)

[0216] Usually, m is fixed to 2.4, n is fixed to 1, and b is fixed to the minimum brightness of the display. Only parameters a and p need to be calculated. The function of p is to adjust the brightness and the height of the curve. The larger p is, the higher the overall curve is. The function of parameter a is to limit the maximum value of the curve, which must not exceed the maximum value of the display device. Because the maximum value of the display device is a constant, once parameter p is determined, parameter a can be calculated based on p.

[0217] In step S230, the encoding device generates a tone-mapping curve in the automatic mode according to the tone-mapping curve parameters in the automatic mode.

[0218] In step S240, the encoding device determines the distortion D′ caused by the tone-mapping curve in the automatic mode according to the quality assessment algorithm. The quality assessment algorithm can also be called a quality assessment model. The most advanced HDR image quality assessment model [1][2] (where [1] is HDR-VDP-2.2: A Calibrated Method for Objective Quality Prediction of High Dynamic Range and Standard Images, Manish Narwaria, Rafal K.Mantiuk, MattheiuPerreira Da Silva and Patrick Le Callet. In: Journal of Electronic Imaging, 24(1), 2015), [2] is a dynamic range independent quality assessment method (Dynamic Range Independent Image Quality Assessment, O.Aydin, Mantiuk, Karol Myszkowski, Hans-Peter Seidel. In: ACM Transactions on Graphics (Proc. of SIGGRAPH'08), 27(3), article no. 69, 2008), based on HVS, considering the sensitivity of HVS to different brightness and spatial frequencies, the perceptual error of the local contrast of each pixel position in the two images before and after tone mapping is calculated. If the probability is 0, it means that the human eye cannot perceive the difference, and the image quality after tone mapping is very good; if the probability is 1, it means that the human eye can definitely perceive the difference, which means that the image quality after tone mapping is relatively poor. Because the brightness and spatial frequency at each pixel are different, and the results of tone mapping are also different, the perceptual error corresponding to each pixel. Finally, the perceptual errors of each pixel are superimposed together to obtain the average perceptual error Q of the entire image.

[0219] The input of the quality assessment model is the original image before tone mapping, and the output is the result image after tone mapping. Both are linear domain values, in nits. The output Q of the model represents the quality loss, which can be normalized between 0 and 1. The larger the output value, the worse the image quality. 0 means that each pixel has no loss, and the image quality is the highest; 1 means that each pixel has obvious perceptual error, and the image quality is the lowest.

[0220] In step S250, the encoding device compares D′ and D T , determine the mode adopted by the HDR video of the Nth frame, the mode being the automatic mode or the director mode, wherein the HDR T is the threshold value.

[0221] In order to more clearly understand the present application, the automatic mode and the director mode are explained below.

[0222] Automatic mode is a non-director mode, i.e. a non-human intervention mode, a non-human intervention mode. Automatic mode can also be called a non-human intervention tone mapping mode. The parameters of the curve in automatic mode are calculated based on basic dynamic metadata. Color-related dynamic metadata can also be present in automatic mode, but it is not required.

[0223] Director mode, in the usual sense, refers to the director and colorist manually adjusting and determining parameters. This is just one case of the director mode in this application. The director mode in this application also includes using algorithms to calculate curve parameters, because using algorithms to calculate curve parameters is equivalent to replacing the director and colorist to adjust parameters, so it is also a director mode in a broad sense. Basic dynamic metadata still needs to be transmitted in director mode. The parameters of the curve can be manually adjusted by the colorist or calculated by a computer algorithm. In addition to the parameters of the basic curve, there are also parameters of cubic splines such as TH1, TH2, TH3, TH_strength, etc., which also need to be manually adjusted or calculated by an algorithm. These parameters will also be embedded in the dynamic metadata and transmitted to the decoding device. The decoding device will directly use these parameters to generate the tone mapping curve according to the capabilities of the display, and can also make appropriate adjustments. Similarly, color-related dynamic metadata can also be present, but it is not required.

[0224] Therefore, the essential difference between the automatic mode and the director mode is that the automatic mode only transmits basic dynamic metadata and requires the decoder to calculate the curve parameters by itself and then generate the curve; the director mode pre-calculates the curve parameters at the production end or the encoding end and transmits them to the decoder as part of the dynamic metadata. The decoder can directly use these parameters to generate the curve.

[0225] The D T The HDR video of the Nth frame adopts the distortion in the director mode, and the D′ and D T , determining the mode adopted by the HDR video of the Nth frame, including: when the D′ is greater than the D T When D′ is less than or equal to D T When the HDR video of the Nth frame is determined to adopt the automatic mode.

[0226] In step S260, the encoding device determines metadata of the HDR video of the Nth frame according to the determined mode adopted by the HDR video of the Nth frame.

[0227] The metadata includes a flag bit, and the flag bit is used to indicate that the mode adopted by the HDR video of the Nth frame is the director mode or the flag bit is used to indicate that the mode adopted by the HDR video of the Nth frame is the automatic mode. For example, the automatic mode bit stream flag tone_mapping_mode=0, and the director mode bit stream flag tone_mapping_mode=1.

[0228] The method may further include: obtaining an offset deltaC[N] of the curve parameters of the HDR video of the Nth frame according to the metadata of the frame (NM, N) in the M frame window, the tone-mapping curve parameters and the D′, wherein N is greater than 0. Then step S260 may specifically include: generating metadata of the HDR video of the Nth frame according to the mode adopted by the HDR video of the Nth frame, the dynamic metadata generation algorithm, and the offset deltaC[N] of the tone-mapping curve parameters.

[0229] The method may further include: generating an adjusted tone-mapping curve according to the deltaC[N]; determining the distortion D caused by the adjusted tone-mapping curve according to the quality evaluation algorithm. T .

[0230] The method can be applied to any one or more of the following preset configuration parameters: 1000nit, 500nit and low dynamic range SDR.

[0231] It should be understood that the preset configuration parameter is only used for example and does not impose any limitation on the present application. For example, the preset configuration parameter may also be 900 nit, 400 nit, etc.

[0232] In order to more clearly understand the present application, the method of the present application is described in detail below in combination with the tone-mapping curve form shown in the above formula (8).

[0233] Figure 7 1 is a schematic block diagram of a system for applying a high dynamic range HDR video processing method provided by the present application. Figure 7 As shown, the method can be applied to a system including an encoding end 310 and a decoding end 320 .

[0234] The encoding end 310 can perform the following four steps: S311, the encoding end 310 extracts basic dynamic metadata of the HDR source video according to the dynamic metadata extraction algorithm; S312, the encoding end 310 calculates the curve parameters in the automatic mode according to the basic dynamic metadata; S313, the encoding end 310 predicts the curve parameter offset according to the time domain filtering, and corrects the curve parameters according to the quality evaluation algorithm; S314, the encoding end 310 switches the automatic mode and the director mode according to the tone-mapping curve distortion of the corrected curve parameters and the tone-mapping curve distortion of the curve parameters in the automatic mode.

[0235] When the decoding end 320 obtains the code stream transmitted by the encoding end, it needs to determine the mode adopted by the HDR video in the code stream through the flag bit of the metadata in the code stream. Depending on the mode adopted by the HDR video, the decoding end 320 may include two branch steps.

[0236] If the HDR video mode in the bitstream is automatic, you can perform the following steps:

[0237] S321, the decoding end 320 extracts basic dynamic metadata of the HDR source video according to a dynamic metadata extraction algorithm. S322, the decoding end 320 calculates curve parameters according to the basic dynamic metadata.

[0238] S323: The decoding end 320 generates a curve according to the curve parameters of the basic dynamic metadata.

[0239] If the HDR video in the bitstream uses the automatic mode, you can perform the following steps:

[0240] S324: The decoding end 320 extracts curve parameters from the metadata.

[0241] S325, the decoding end 320 generates a curve according to the extracted curve parameters.

[0242] After generating the curve parameters, the decoding end 320 performs display adaptation according to the decoded HDR video and displays the HDR video.

[0243] The method of the present application is described in detail below in conjunction with the tone-mapping curve form shown in equation (8) above. Figure 8 is a schematic flow chart of a method 400 for processing a high dynamic range HDR video according to an embodiment of the present application. The method 400 can be applied in Figure 1 or Figure 7 The scenario shown can of course also be applied to other scenarios requiring video processing, and the embodiments of the present application are not limited here.

[0244] It should also be understood that in the embodiments of the present application, the method is described by taking the encoding device as the execution subject of the execution method as an example. As an example but not a limitation, the execution subject of the execution method may also be a chip, a chip system, or a processor applied to the encoding device.

[0245] like Figure 8 As shown, Figure 8 The method 400 shown in FIG. 4 may include S401 to S415. Figure 8 Each step in method 400 is described in detail.

[0246] S401: The encoding device extracts basic dynamic metadata of the HDR video of the Nth frame according to a dynamic metadata extraction algorithm.

[0247] S402: The encoding device calculates tone-mapping curve parameters p′ and a′ of the Nth frame of HDR video in an automatic mode according to basic dynamic metadata.

[0248] The following specifically describes how to generate and calculate the tone-mapping curve parameters p′ and a′ of the Nth frame of HDR video in automatic mode based on basic dynamic metadata.

[0249] First, the updating process of the maximum brightness correction value max_lum of the frame to be processed, i.e., the Nth frame, is described:

[0250] (1) Convert the maximum display brightness value max_display_mastering_luminance to the PQ domain to obtain the display brightness range MaxRefDisplay of the reference display device.

[0251] (2) Substitute maximum_maxrgb, average_maxrgb, and variance_maxrgb in metadata according to formula (9) to calculate the maximum reference brightness value MAX1 of the frame to be processed.

[0252]

[0253] Among them, A and B are weight coefficients, A is the function of average_maxrgb, A = (1-B)*(1-F(average_maxrgb / maximum_maxrgb)), F(x) is a constant function, and the default values ​​of A and B are 1 / 3.

[0254] (3) Determine the final maximum brightness correction value according to formula (10):

[0255]

[0256] Next, describe the tone-mapping curve parameters p′ and a′ in automatic mode:

[0257] (1) Set m and n to the preset values ​​of 2.4 and 1 respectively. After the above parameters are preset, the curve becomes

[0258]

[0259] (2) Set b to MinDisplay

[0260] (3) Substitute average_maxrgb(avgL) in metadata according to formula (11) to calculate p:

[0261]

[0262] in:

[0263] PvalueH0, PvalueL0, TPH0 and TPL0 are preset values, and the default values ​​are 3.5, 4.0, 0.6, 0.3; g0() is y=xN, and the default value is y=x.

[0264] (4) Update the maximum brightness correction value max_lum according to formula (12)

[0265] in:

[0266] PdeltaH1, PdeltaL1, TPH1 and TPL1 are preset values, and the default values ​​are 0.6, 0.0, 0.9, 0.75; g1() is y=xN, and the default value is y=x.

[0267] (5) According to p, m, n, b, K1, K2, and K3, H(L) is obtained, as shown in formula (13):

[0268]

[0269] Therefore, the parameter a′ is determined according to formula (14):

[0270] a′=(MaxDISPLAY-MinDISPALY) / (H(MaxSource)-H(MinSource)) (14)

[0271] Among them, MaxSource is equal to the maximum brightness correction value max_lum (PQ domain) of the frame to be processed, and MinSource is equal to the minimum brightness min_lum (PQ domain) of the frame to be processed.

[0272] S403: The encoding device generates a tone-mapping curve in the automatic mode according to the curve parameters p′ and a′ in the automatic mode.

[0273] S404: The encoding device determines the distortion D′ caused by the tone-mapping curve in the automatic mode according to a quality evaluation algorithm.

[0274] In this step, the encoding device generates a tone-mapping curve in the automatic mode according to the curve parameters p′ and a′ in the automatic mode to perform tone-mapping, and then inputs the two images before and after tone mapping into the quality assessment model to calculate the subjective distortion D′.

[0275] S405, the encoding device uses a time domain filtering method to predict the offset delta of the optimal curve parameter relative to the parameter p' of the automatic mode. The offset of the optimized curve parameter relative to the parameter p' of the automatic mode can be predicted according to formula (15):

[0276]

[0277] Among them, deltaC[Nk] represents the offset used in the Nkth frame, D[Nk] represents the subjective distortion calculated by the quality model of the Nkth frame, k = 1, 2, ..., M. M is the length of the queue in the cache. The larger M is, the more historical data is used, and the smoother the filtering result is; the smaller M is, the closer the historical data used is to the current frame, and the better it can reflect the trend of changes. In extreme cases, fixed M = 1, which is equivalent to using the previous frame to predict the current frame, delta = deltaC[N-1]. The recommended maximum value of M is between 4 and 8.

[0278] The predicted offset delta and the optimal offset value usually have a certain deviation, so the predicted offset delta needs to be corrected so that the corrected value can be closer to the optimal value. Taking delta as the benchmark, increase one step and decrease one step to calculate the two corrected values ​​of delta, and compare which corrected value has lower distortion. See step S406 and step S407.

[0279] S406, the encoding device corrects the offset of the predicted curve parameter p′, the offset of the corrected curve parameter p′ is the offset of the predicted curve parameter p minus the basic step value stepsize, that is, deltaC1=delta-stepsize, and stepsize is recommended to be 0.1 for parameter p′; the adjusted curve parameter p1 is determined according to the curve parameter p′ in the automatic mode and the offset of the corrected curve parameter p′, p1=p+deltaC1, that is, a1 is calculated according to p1, and the parameters p1 and a1 generate an adjusted tone-mapping curve, and the distortion D1 caused by each tone-mapping curve in the multiple adjusted tone-mapping curves is determined according to the quality evaluation algorithm.

[0280] S407, the encoding device corrects the offset of the predicted curve parameter p′, the offset of the corrected curve parameter p′ is the offset of the predicted curve parameter p′ plus the basic step value stepsize, deltaC2=delta-stepsize; determines the adjusted curve parameter p2 according to the curve parameter p′ in the automatic mode and the offset of the corrected curve parameter p′, calculates a2 according to p2, generates an adjusted tone-mapping curve with parameters p2 and a2, and determines the distortion D2 caused by each of the multiple adjusted tone-mapping curves according to the quality evaluation algorithm.

[0281] In the embodiment of the present application, only two correction values ​​are calculated because each time a correction value is tested, a quality assessment model needs to be calculated, but the amount of calculation of the quality assessment model is large, so it is limited to two times in the embodiment of the present application. This does not guarantee that each frame will obtain the optimal value, but each frame can be closer to the optimal value (one step length) based on the predicted value, so that the predicted value can gradually converge to the vicinity of the optimal value over time.

[0282] S408: The encoding device compares the sizes of D1 and D2.

[0283] S409, if D1 is less than or equal to D2, the offset of the modified curve parameter p′ is deltaC1, the distortion of the modified curve parameter DT=D1, and the parameter set P={p1, a1}.

[0284] S410, if D1 is greater than D2, the offset of the modified curve parameter p is deltaC2, the distortion of the modified curve parameter DT=D2, and the parameter set P={p2, a2}.

[0285] The predicted and corrected deltaC curve parameters are better than the automatic mode in most cases, but there is no guarantee, so a comparison is needed. If the automatic mode is better, then the automatic mode is used for the Nth frame; if the predicted and corrected value is better, then the director mode is used for the current frame.

[0286] S411, comparing the distortion D′ caused by the tone-mapping curve in the automatic mode with the distortion D of the modified curve parameters T .

[0287] S412, when the D′ is greater than the D T When determining that the HDR video of the Nth frame adopts the director mode, the curve parameters of the Nth frame are P[N]=P, the distortion D[N]=DT, and the offset deltaC[N]=deltaC.

[0288] S413, putting the parameter set P[N] into the metadata of the director mode.

[0289] S414, when the D′ is less than or equal to the D T When determining that the HDR video of the Nth frame adopts the automatic mode, the curve parameters of the Nth frame are P[N]=P′, distortion D[N]=D′, offset deltaC[N]=0, and P[N] does not need to be put in the dynamic metadata to be transmitted to the decoding end in the automatic mode. Only basic dynamic metadata needs to be transmitted, and the decoding end can calculate the value of P[N] according to the standard algorithm.

[0290] S415, add deltaC[N] and D[N] to the cache queue to predict the parameter offset of the next frame or multiple frames f(N+j), where j is greater than or equal to 1 and less than or equal to M. It should be noted that the queue here refers to the queue used for time domain filtering in step 3 above. The queue adopts the first-in-first-out rule. If the queue is full, the head of the queue is removed each time, and then new data is added to the end of the queue.

[0291] The encoder sends the bit stream to the decoder, and it should be understood that sending can also be understood as storing in a storage medium. For example, the encoder can burn the bit stream into an optical disc, and the decoder reads the bit stream from the optical disc.

[0292] In the implementation of the present application, based on the automatic mode and taking the quality assessment model as the standard, the curve parameters are optimized to achieve a better tone mapping effect; the quality assessment model in the embodiment of the present application is calculated three times per frame, once in the automatic mode and twice in the correction process, and the amount of calculation is controllable; the automatic mode and the director mode are switched to ensure that better curve parameters are used.

[0293] The method of the present application is described in detail below in conjunction with the tone-mapping curve form shown in equation (8) above. Fig. 9 is a schematic flow chart of a method 500 for processing a high dynamic range HDR video according to an embodiment of the present application. The method 500 can be applied in Figure 1 or Figure 7 The scenario shown can of course also be applied to other scenarios requiring video processing, and the embodiments of the present application are not limited here.

[0294] It should also be understood that in the embodiments of the present application, the method is described by taking the encoding device as the execution subject of the execution method as an example. As an example but not a limitation, the execution subject of the execution method may also be a chip, a chip system, or a processor applied to the encoding device.

[0295] like Fig. 9 As shown, Fig. 9The method 500 shown in FIG. 5 may include S501 to S514. Fig. 9 Each step in method 500 is described in detail.

[0296] S501: The encoding device extracts basic dynamic metadata of the HDR video of the Nth frame according to a dynamic metadata extraction algorithm.

[0297] S502: The encoding device calculates tone-mapping curve parameters p′ and a′ of the HDR video of the Nth frame in an automatic mode according to basic dynamic metadata.

[0298] S503: The encoding device generates a tone-mapping curve in the automatic mode according to the curve parameters p′ and a′ in the automatic mode.

[0299] S504: The encoding device determines the subjective distortion D′ caused by the tone-mapping curve in the automatic mode according to a quality evaluation algorithm.

[0300] S505: The encoding device uses a time domain filtering method to predict an offset delta of an optimal curve parameter relative to a parameter p in an automatic mode.

[0301] S506, determining multiple offsets of the adjusted curve parameter p according to the predicted offset delta of the curve parameter p and K*basic step value, wherein K is a positive integer and K is greater than or equal to 0; determining curve parameters p″ and a″ corresponding to each of the multiple offsets of the curve parameter p according to the multiple offsets of the curve parameter p; generating multiple adjusted tone-mapping curves according to the parameters p″ and a″ corresponding to each of the multiple offsets; determining distortion D″ caused by each of the multiple adjusted tone-mapping curves according to the quality evaluation algorithm; selecting tone-mapping curve parameters p″ and a″ corresponding to any one D″ that meets the first condition from the multiple D″ as target adjusted curve parameters p′ and a′, the D T is the selected D″.

[0302] In one implementation, determining multiple offsets of the adjusted curve parameter p based on the predicted offset of the curve parameter p and K*basic step value includes: determining multiple offsets based on the basic delta±M*basic step value stepsize, where M is each positive integer less than or equal to K.

[0303] In one implementation, the first condition is to select D″ from multiple D″ whose D″ is smaller than a first threshold value, such as if the first threshold value is 0.3. The first condition may also be to select the smallest D″ from multiple D″.

[0304] Specifically, after obtaining the predicted offset delta, an exhaustive search can be used to search for a better offset. In the exhaustive search, K is an integer. For example, if K is 3, seven different values ​​of delta-3*stepsize, delta-2*stepzie, delta-stepsize, delta, delta+stepsize, delta+2*stepsize, and delta+3*stepsize will be searched, and the one with the smallest subjective distortion will be selected. The offset is recorded as deltaC, the parameter p=P′+deltaC, a is calculated by p, the parameter set P={p,a}, and the corresponding subjective distortion is recorded as DT.

[0305] The predicted and corrected deltaC curve parameters are better than the automatic mode in most cases, but there is no guarantee, so a comparison is needed. If the automatic mode is better, then the automatic mode is used for the Nth frame; if the predicted and corrected value is better, then the director mode is used for the current frame.

[0306] S507, compare the D′ and D T .

[0307] S508, when the D′ is greater than the D T When determining that the HDR video of the Nth frame adopts the director mode, the curve parameters of the Nth frame are P[N]=P, the distortion D[N]=DT, and the offset deltaC[N]=deltaC.

[0308] S509, putting the parameter set P[N] into the metadata of the director mode.

[0309] S510, when the D′ is less than or equal to the D T When determining that the HDR video of the Nth frame adopts the automatic mode, the curve parameters of the Nth frame are P[N]=P′, distortion D[N]=D′, offset deltaC[N]=0, and P[N] does not need to be put in the dynamic metadata to be transmitted to the decoding end in the automatic mode. Only the basic dynamic source data needs to be transmitted, and the decoding end can calculate the value of P[N] according to the standard algorithm.

[0310] S511, add deltaC[N] and D[N] to the cache queue to predict the parameter offset of the next frame f(N+1). It should be noted that the queue here refers to the queue used for time domain filtering in step 3 above. The queue adopts the first-in-first-out rule. If the queue is full, then the head of the queue is removed first, and then new data is added to the end of the queue.

[0311] It should be understood that methods 501 to 505 correspond to methods 401 to 405, and methods 507 to 511 correspond to methods 411 to 415. Therefore, the relevant descriptions of methods 501 to 505 and methods 507 to 511 can refer to methods 401 to 405 and methods 411 to 415, and will not be described again here to avoid repetition.

[0312] In the embodiment of the present application, on the basis of the automatic mode, the curve parameters are optimized based on the quality assessment model to achieve a better tone mapping effect; under the condition of sufficient computing resources, the optimal parameters are searched by exhaustive method to maximize the effect; the automatic mode and the director mode are switched to ensure that the better curve parameters are used.

[0313] Fig.10 is a schematic flow chart of a method 600 for processing a high dynamic range HDR video according to an embodiment of the present application. The method 600 can be applied in Figure 1 or Figure 7 The scenario shown can of course also be applied to other scenarios requiring video processing, and the embodiments of the present application are not limited here.

[0314] It should also be understood that in the embodiments of the present application, the method is described by taking the decoding device as the execution subject of the execution method as an example. As an example but not a limitation, the execution subject of the execution method may also be a chip, a chip system, or a processor applied to the decoding device.

[0315] like Fig.10 As shown, Fig.10 The method 600 shown in FIG. 6 may include S610 to S620. Fig.10 Each step in method 600 is described in detail.

[0316] S610: A decoding device obtains an HDR video of an N-th frame and metadata of the HDR video of the N-th frame.

[0317] S620: The decoding device determines the mode adopted by the HDR video of the Nth frame according to the flag bit of the metadata.

[0318] In one implementation, when the flag indicates that the mode adopted by the HDR video of the Nth frame is the automatic mode, the method further includes: calculating tone-mapping curve parameters in the automatic mode according to the metadata of the HDR video of the Nth frame; generating a tone-mapping curve in the automatic mode according to the tone-mapping curve parameters; and displaying the HDR video of the Nth frame according to the tone-mapping curve in the automatic mode and the HDR video of the Nth frame.

[0319] In one implementation, when the flag indicates that the mode adopted by the HDR video of the Nth frame is the director mode, the method further includes: extracting tone-mapping curve parameters in the director mode from the metadata; generating a tone-mapping curve in the director mode according to the curve parameters; and displaying the HDR video of the Nth frame according to the tone-mapping curve in the director mode and the HDR video of the Nth frame.

[0320] The above embodiment of the present application describes that the encoding device needs to switch between the automatic mode and the director mode, but the predicted and corrected parameter curve is likely to be better than the automatic mode, so it is possible to consider omitting the automatic mode to reduce the complexity of the system. In addition, omitting the automatic mode can also omit the calculation of the subjective distortion of the automatic mode. Therefore, the present application also provides an embodiment in which the encoding device does not switch between the automatic mode and the director mode and always uses the director mode.

[0321] Fig.11 1 is a schematic block diagram of a system for applying a high dynamic range HDR video processing method provided by the present application. Fig.11 As shown, the method can be applied to a system including an encoding end 710 and a decoding end 720 .

[0322] The encoding end 710 can perform the following four steps: S711, the encoding end 710 extracts basic dynamic metadata of the HDR source video according to the dynamic metadata extraction algorithm; S712, the encoding end 710 calculates the curve parameters in the automatic mode according to the basic dynamic metadata; S713, the encoding end 710 predicts the curve parameter offset according to the time domain filtering, and corrects the curve parameters according to the quality evaluation algorithm; S714, the encoding end 710 generates director mode metadata.

[0323] S721, the decoding end 720 extracts curve parameters from metadata;

[0324] S722: The decoding end 720 generates a curve according to the extracted curve parameters.

[0325] After generating the curve parameters, the decoding end 720 performs display adaptation according to the decoded HDR video and displays the HDR video.

[0326] The method of the present application is described in detail below in conjunction with the tone-mapping curve form shown in equation (7) above. Fig.12 is a schematic flow chart of a method 800 for processing a high dynamic range HDR video according to an embodiment of the present application. The method 800 can be applied in Figure 1 or Fig.11 The scenario shown can of course also be applied to other scenarios requiring video processing, and the embodiments of the present application are not limited here.

[0327] It should also be understood that in the embodiments of the present application, the method is described by taking the encoding device as the execution subject of the execution method as an example. As an example but not a limitation, the execution subject of the execution method may also be a chip, a chip system, or a processor applied to the encoding device.

[0328] like Fig.12 As shown, Fig.12 The method 800 shown in FIG. 8 may include S801 to S815. Fig.12 Each step in method 800 is described in detail.

[0329] S801: The encoding device extracts basic dynamic metadata of the HDR video of the Nth frame according to a dynamic metadata extraction algorithm.

[0330] S802: The encoding device calculates tone-mapping curve parameters of the HDR video of the Nth frame in automatic mode according to basic dynamic metadata, such as curve parameters p and a.

[0331] S803: The encoding device generates a tone-mapping curve in the automatic mode according to the curve parameters p and a in the automatic mode.

[0332] S804: The encoding device determines the distortion D′ caused by the tone-mapping curve according to a quality evaluation algorithm.

[0333] S805, the encoding device obtains the offset deltaC[N] of the tone-mapping curve parameters of the HDR video of the Nth frame according to the metadata of the frame (NM, N) in the M frame window, the tone-mapping curve parameters and the D′, wherein N is greater than 0.

[0334] S806, the encoding device generates dynamic metadata and P[N] of the HDR video of the Nth frame in director mode according to the dynamic metadata generation algorithm and the offset deltaC[N] of the curve parameters of the HDR video of the Nth frame, wherein P[N] is the curve parameter P when the mode adopted by the HDR video of the Nth frame is the director mode.

[0335] The method further includes: generating an adjusted tone-mapping curve according to an offset deltaC[N] of a curve parameter of the HDR video of the Nth frame; and determining a distortion D caused by the adjusted tone-mapping curve according to a quality evaluation algorithm. T , the D T The offset deltaC[N+j] used to predict the N+j frame in director mode is used, where j is greater than or equal to 1 and less than or equal to M.

[0336] In one implementation, determining multiple offsets of the adjusted curve parameter p based on the predicted offset of the curve parameter p and K*basic step value includes: determining multiple offsets based on the basic delta±M*basic step value stepsize, where M is each positive integer less than or equal to K.

[0337] In one implementation, the first condition is to select D″ from multiple D″ whose D″ is smaller than a first threshold value, such as if the first threshold value is 0.3. The first condition may also be to select the smallest D″ from multiple D″.

[0338] Specifically, after obtaining the predicted offset delta, an exhaustive search can be used to search for a better offset. In the exhaustive search, K is an integer. For example, if K is 3, seven different values ​​of delta-3*stepsize, delta-2*stepzie, delta-stepsize, delta, delta+stepsize, delta+2*stepsize, and delta+3*stepsize will be searched, and the one with the smallest subjective distortion will be selected. The offset is recorded as deltaC, the parameter p=P′+deltaC, a is calculated by p, the parameter set P={p,a}, and the corresponding subjective distortion is recorded as DT.

[0339] S807, add deltaC[N] and D[N] to the cache queue to predict the parameter offset of the next frame f(N+1), D[N]=DT, offset deltaC[N]=deltaC. It should be noted that the queue here refers to the queue used for time domain filtering above. The queue adopts the first-in-first-out rule. If the queue is full, then the head of the queue is removed first, and then new data is added to the end of the queue.

[0340] It should be understood that methods 801 to 804 correspond to methods 501 to 504 , and therefore the relevant descriptions of methods 801 to 804 may refer to methods 501 to 504 , and will not be described again here to avoid repetition.

[0341] In the embodiment of the present application, on the basis of the automatic mode, the curve parameters are optimized with the quality assessment model as the standard, so as to achieve a better tone mapping effect; under the condition of sufficient computing resources, the optimal parameters are searched by exhaustive method to maximize the effect; the switching between the automatic mode and the director mode is omitted, and the system complexity is reduced.

[0342] It should be understood that in method 800, the encoding device can also perform two offset corrections based on delta, respectively increase a step size and decrease a step size to calculate two delta correction values, and compare which correction value has lower distortion. For details, refer to step S406 and step S407.

[0343] The above embodiments of the present application describe that the encoding device needs to switch between the automatic mode and the director mode for the HDR video of the Nth frame, or adopt the director mode for the HDR video of the Nth frame. The encoding needs to switch between the automatic mode and the director mode for the HDR video of the Nth frame, or adopt the director mode for the HDR video of the Nth frame. The action of using the director mode for the HDR video of the Nth frame can also be performed by the decoding device. The present application also provides a method for processing high dynamic range HDR video, which is performed by a decoding device.

[0344] Fig.13 1 is a schematic block diagram of a system for applying a high dynamic range HDR video processing method provided by the present application. Fig.13 As shown, the method can be applied to a system including an encoding end 910 and a decoding end 920 .

[0345] The encoder 910 may perform step S911, where the encoder 910 extracts basic dynamic metadata of the HDR source video according to a dynamic metadata extraction algorithm; S912, the encoder 910 transmits the extracted basic dynamic metadata and the compressed video to the decoder; S921, the decoder 920 calculates the curve parameters of the automatic mode; S922, the decoder 920 predicts and corrects the offset of the curve parameters and updates the curve parameters, and finally generates a curve; S923, the decoder 920 generates a curve according to the updated curve parameters. After generating the curve parameters, the decoder 920 performs display adaptation according to the decoded HDR video and displays the HDR video.

[0346] Fig.14 is a schematic flow chart of a method 1000 for processing a high dynamic range HDR video according to an embodiment of the present application. The method 1000 can be applied in Figure 1 or Fig.13 The scenario shown can of course also be applied to other scenarios requiring video processing, and the embodiments of the present application are not limited here.

[0347] It should also be understood that in the embodiments of the present application, the method is described by taking the decoding device as the execution subject of the execution method as an example. As an example but not a limitation, the execution subject of the execution method may also be a chip, a chip system, or a processor applied to the decoding device.

[0348] like Fig.14 As shown, Fig.14 The method 1000 shown in FIG. 1 may include S1001 to S1015. Fig.14 Each step in method 1000 is described in detail.

[0349] S1001: A decoding device extracts basic dynamic metadata of an HDR video of an Nth frame according to a dynamic metadata extraction algorithm.

[0350] S1002: The decoding device calculates tone-mapping curve parameters p and a of the HDR video of the Nth frame in automatic mode according to basic dynamic metadata.

[0351] S1003: The decoding device generates a tone-mapping curve in the automatic mode according to the curve parameters p and a in the automatic mode.

[0352] S1004: The decoding device determines the subjective distortion D′ caused by the tone-mapping curve in the automatic mode according to a quality evaluation algorithm.

[0353] In this step, the decoding device generates a tone-mapping curve in the automatic mode according to the curve parameters p and a in the automatic mode to perform tone-mapping, and then inputs the two images before and after tone mapping into the quality assessment model to calculate the subjective distortion D′.

[0354] S1005, the decoding device compares D′ and D T , determine whether to use the automatic mode, where the D T Threshold

[0355] S1006, when the D′ is less than or equal to the D T When the HDR video of the Nth frame is adapted, the decoding device uses the automatic mode.

[0356] S1007, when the D′ is greater than the D T When D TAs a preset value, the decoding device uses a time domain filtering method to predict the offset delta of the optimal curve parameter relative to the parameter p of the automatic mode.

[0357] S1008, the decoding device corrects the offset of the predicted curve parameter p, the offset of the corrected curve parameter p is the offset of the predicted curve parameter p minus the basic step value stepsize, that is, deltaC1=delta-stepsize, and stepsize is recommended to be 0.1 for parameter p; the adjusted curve parameter p1 is determined according to the curve parameter p in the automatic mode and the offset of the corrected curve parameter p, p1=p+deltaC1, a1 is calculated according to p1, the parameters p1 and a1 generate an adjusted tone-mapping curve, and the distortion D1 caused by each tone-mapping curve in the multiple adjusted tone-mapping curves is determined according to the quality evaluation algorithm.

[0358] S1009, the decoding device corrects the offset of the predicted curve parameter p, the offset of the corrected curve parameter p is the offset of the predicted curve parameter p plus the basic step value stepsize, deltaC2 = delta-stepsize; the adjusted curve parameter p2 is determined according to the curve parameter p in the automatic mode and the offset of the corrected curve parameter p, a2 is calculated according to p2, the parameters p2 and a2 generate an adjusted tone-mapping curve, and the distortion D2 caused by each tone-mapping curve in the multiple adjusted tone-mapping curves is determined according to the quality evaluation algorithm.

[0359] In the embodiment of the present application, only two correction values ​​are calculated because each time a correction value is tested, a quality assessment model needs to be calculated, but the amount of calculation of the quality assessment model is large, so it is limited to two times in the embodiment of the present application. This does not guarantee that each frame will obtain the optimal value, but each frame can be closer to the optimal value (one step length) based on the predicted value, so that the predicted value can gradually converge to the vicinity of the optimal value over time.

[0360] S1010: The decoding device compares the sizes of D1 and D2.

[0361] S1011, if D1 is less than or equal to D2, the offset of the modified curve parameter p is deltaC1, the distortion of the modified curve parameter DT=D1, the parameter set P1={p1, a1}, the curve parameter of the HDR video of the Nth frame is P[N]=P1, the distortion D[N]=DT=D1, the offset deltaC[N]=deltaC1, and the decoding device generates a tone-mapping curve according to the curve parameter P1 of the HDR video of the Nth frame.

[0362] S1012, if D1 is greater than D2, the offset of the modified curve parameter p is deltaC2, the distortion of the modified curve parameter DT=D2, the parameter set P2={p2, a2}, the curve parameter of the Nth frame is P[N]=P2, the distortion D[N]=DT=D2, the offset deltaC[N]=deltaC2, and the decoding device generates a tone-mapping curve according to the curve parameter P2 of the HDR video of the Nth frame.

[0363] S1013, add deltaC[N] and D[N] to the cache queue to predict the parameter offset of the next frame f(N+1). It should be noted that the queue here refers to the queue used for the above time domain filtering. The queue adopts the first-in-first-out rule. If the queue is full, then the head of the queue is removed first, and then new data is added to the end of the queue.

[0364] In the embodiment of the present application, the decoding end optimizes the curve parameters based on the automatic mode and the quality assessment model as the standard, so as to achieve a better tone mapping effect; the quality assessment model is calculated three times per frame in the embodiment of the present application, once in the automatic mode and twice in the correction process, and the amount of calculation is controllable; the curve calculation and optimization of dynamic tone mapping requires the minimum and maximum brightness of the target display, so compared with the optimization performed on the encoding end in the embodiment, the brightness information of the target display is more accurate and the effect will be better if it is performed on the decoding end.

[0365] It should be understood that the decoding device determines the multiple offsets of the adjusted curve parameter p according to the predicted offset delta of the curve parameter p and the K*basic step value. For details, please refer to method 506.

[0366] In the above embodiment, the decoding device needs to make an automatic mode determination, but the predicted and corrected parameter curve is most likely better than the automatic mode, so it is possible to consider omitting the automatic mode determination to reduce the complexity of the system. In addition, omitting the automatic mode can also omit the calculation of the subjective distortion of the automatic mode. Therefore, the decoding device can directly correct the automatic mode to obtain the offset deltaC[N] of the curve parameters of the HDR video of the Nth frame without making an automatic mode determination. The specific process of the decoding device directly correcting the automatic mode to obtain the offset deltaC[N] of the curve parameters of the HDR video of the Nth frame can refer to the process of the encoding device using the director mode to transmit the HDR video in method 800.

[0367] It should also be understood that the execution process of the decoding device can refer to the encoding device accordingly, the difference being that: first, the parameters P[N] obtained after optimization by the decoding device do not need to be written into the dynamic metadata; second, the decoding device will use the optimized curve parameters P[N] to generate the curve.

[0368] Combination of the above Figures 1 to 14 The communication parameter measurement method of the multi-card terminal device in the embodiment of the present application is described in detail. Figure 15 to Figure 16 The communication device according to the embodiment of the present application is described in detail.

[0369] Fig.15 A schematic block diagram of an apparatus 1100 according to an embodiment of the present application is shown.

[0370] In some embodiments, the apparatus 1100 may be a terminal device, or may be a chip or a circuit, such as a chip or a circuit that may be disposed in a terminal device.

[0371] In one possible manner, the apparatus 1100 may include a processing unit 1110 (ie, an example of a processor) and a transceiver unit 1130. In some possible implementations, the processing unit 1110 may also be referred to as a determination unit. In some possible implementations, the transceiver unit 1130 may include a receiving unit and a sending unit.

[0372] In one implementation, the transceiver unit 1130 may be implemented by a transceiver or a transceiver-related circuit or an interface circuit.

[0373] In one implementation, the device may further include a storage unit 1120. In one possible implementation, the storage unit 1120 is used to store instructions. In one implementation, the storage unit may also be used to store data or information. The storage unit 1120 may be implemented by a memory.

[0374] In some possible designs, the processing unit 1110 is used to execute the instructions stored in the storage unit 1120, so that the apparatus 1100 implements the steps performed by the terminal device in the above method. Alternatively, the processing unit 1110 can be used to call the data of the storage unit 1120, so that the apparatus 1100 implements the steps performed by the terminal device in the above method.

[0375] In some possible designs, the processing unit 1110 is used to execute the instructions stored in the storage unit 1120, so that the apparatus 1100 implements the steps performed by the access network device in the above method. Alternatively, the processing unit 1110 can be used to call the data of the storage unit 1120, so that the apparatus 1100 implements the steps performed by the access network device in the above method.

[0376] For example, the processing unit 1110, the storage unit 1120, and the transceiver unit 1130 can communicate with each other through the internal connection path to transmit control and / or data signals. For example, the storage unit 1120 is used to store a computer program, and the processing unit 1110 can be used to call and run the computer program from the storage unit 1120 to control the transceiver unit 1130 to receive signals and / or send signals, and complete the steps of the terminal device or access network device in the above method. The storage unit 1120 can be integrated in the processing unit 1110, or it can be set separately from the processing unit 1110.

[0377] Optionally, if the apparatus 1100 is a communication device (eg, a terminal device), the transceiver unit 1130 includes a receiver and a transmitter. The receiver and the transmitter may be the same or different physical entities. When they are the same physical entity, they may be collectively referred to as a transceiver.

[0378] When the device 1100 is a terminal device, the transceiver unit 1130 may be a sending unit or a transmitter when sending information, and the transceiver unit 1130 may be a receiving unit or a receiver when receiving information. The transceiver unit may be a transceiver, and the transceiver, transmitter or receiver may be a radio frequency circuit. When the device includes a storage unit, the storage unit is used to store computer instructions, and the processor is connected to the memory for communication. The processor executes the computer instructions stored in the memory, so that the device can execute method 200, method 1100 or method 600. The processor may be a general-purpose central processing unit (CPU), a microprocessor, or an application specific integrated circuit (ASIC).

[0379] Optionally, if the device 1100 is a chip or a circuit, the transceiver unit 1130 includes an input interface and an output interface.

[0380] When the device 1100 is a chip, the transceiver unit 1130 may be an input and / or output interface, a pin or a circuit, etc. The processing unit 1110 may execute the computer-executable instructions stored in the storage unit so that the device may execute method 200, method 1100 or method 600. Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc. The storage unit may also be a storage unit in the terminal located outside the chip, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0381] As an implementation, the function of the transceiver unit 1130 may be implemented by a transceiver circuit or a dedicated chip for transceiver. The processing unit 1110 may be implemented by a dedicated processing chip, a processing circuit, a processing unit or a general chip.

[0382] As another implementation method, it is possible to use a general-purpose computer to implement the communication device (e.g., terminal device, or access network device) provided in the embodiment of the present application. That is, the program code for implementing the functions of the processing unit 1110 and the transceiver unit 1130 is stored in the storage unit 1120, and the general-purpose processing unit implements the functions of the processing unit 1110 and the transceiver unit 1130 by executing the code in the storage unit 1120.

[0383] In some embodiments, the apparatus 1100 may be an encoding device, or a chip or circuit disposed in the encoding device. When the apparatus 1100 is an encoding device, or a chip or circuit disposed in the encoding device, an acquisition unit 1140 is used to acquire dynamic metadata of the HDR video of the Nth frame according to a dynamic metadata generation algorithm, wherein N is greater than 0; a processing unit 1110 is used to calculate tone-mapping curve parameters of the HDR video of the Nth frame according to the dynamic metadata of the HDR video of the Nth frame; the processing unit 1110 is also used to generate a tone-mapping curve according to the curve parameters; the processing unit 1110 is also used to determine the distortion D′ caused by the tone-mapping curve according to a quality evaluation algorithm; the processing unit 1110 is also used to compare D′ and D T , determine the mode adopted by the HDR video of the Nth frame, the mode being the automatic mode or the director mode, wherein the HDR T is a threshold value; the processing unit 1110 is further used to determine the metadata of the HDR video of the Nth frame according to the determined mode adopted by the HDR video of the Nth frame.

[0384] In one implementation, the processing unit 1110 is specifically configured to: when the D′ is greater than the D T When the HDR video of the Nth frame is determined to adopt the director mode, the D T The HDR video of the Nth frame adopts the distortion in the director mode; or, when the D′ is less than or equal to the D T When the HDR video of the Nth frame is determined to adopt the automatic mode.

[0385] In one implementation, the processing unit 1110 is further used to obtain an offset deltaC[N] of the curve parameters of the HDR video of the Nth frame based on the metadata of the frame (NM, N) in the M frame window, the tone-mapping curve parameters and the D′, wherein the M is greater than 0.

[0386] In one implementation, the processing unit 1110 is further configured to: generate an adjusted tone-mapping curve according to the deltaC[N]; and determine the distortion D caused by the adjusted tone-mapping curve according to the quality evaluation algorithm. T .

[0387] In one implementation, the processing unit 1110 is specifically used to generate metadata of the HDR video of the Nth frame according to the mode adopted by the HDR video of the Nth frame, the dynamic metadata generation algorithm and the offset deltaC[N] of the tone-mapping curve parameter.

[0388] In one implementation, the encoding device is applied to any one or more of the following preset configuration parameters: 1000nit, 500nit, and low dynamic range SDR.

[0389] In one implementation, the tone-mapping curve is as shown in formula (16):

[0390]

[0391] The curve parameters a and p are determined according to the dynamic metadata of the HDR video of the Nth frame, the curve parameters b, n and m are preset values, L is the input brightness, and L′ is the output brightness.

[0392] In one implementation, the processing unit 1110 is further used to: determine multiple offsets of the adjusted curve parameter p according to the predicted offset of the curve parameter p and K*basic step value, wherein K is a positive integer and K is greater than or equal to 0; determine curve parameters p″ and a″ corresponding to each of the multiple offsets of the curve parameter p according to the multiple offsets of the curve parameter p; generate multiple adjusted tone-mapping curves according to the parameters p″ and a″ corresponding to each of the multiple offsets; determine the distortion D″ caused by each of the multiple adjusted tone-mapping curves according to the quality evaluation algorithm; select the tone-mapping curve parameters p″ and a″ corresponding to any one D″ that meets the first condition from the multiple D″ as the target adjusted curve parameters p′ and a′, and the D T is the selected D″.

[0393] In one implementation, the processing unit 1110 is specifically configured to: when K is 1, the determined offset is the offset of the predicted curve parameter p plus the basic step value, and the offset of the predicted curve parameter p minus the basic step value.

[0394] In one implementation, the first condition is to select a tone-mapping curve parameter with a smaller distortion D″ from two distortions D″.

[0395] In one implementation, the processing unit 1110 is specifically configured to: determine a plurality of offsets according to the basic delta±M*basic step value stepsize, wherein M is each positive integer less than or equal to K.

[0396] In one implementation, the first condition is to select the smallest D" from a plurality of D" or to select a D" that is smaller than a first threshold from a plurality of D".

[0397] In one implementation, the processing unit 1110 is further configured to:

[0398] The offset of the predicted curve parameter p is determined according to formula (17):

[0399]

[0400] Wherein, deltaC[Nk] represents the offset used by the Nk-th frame, D[Nk] represents the distortion calculated by the quality assessment algorithm of the Nk-th frame, and M is the window length.

[0401] In one implementation, the processing unit 1110 is specifically used to: when the mode adopted by the HDR video of the Nth frame is the automatic mode, the metadata of the HDR video of the Nth frame includes dynamic metadata of the HDR video of the Nth frame; or, when the mode adopted by the HDR video of the Nth frame is the director mode, the metadata of the HDR video of the Nth frame includes the dynamic metadata of the HDR video of the Nth frame and P[N], where P[N] is the curve parameter P when the mode adopted by the HDR video of the Nth frame is the director mode.

[0402] In one implementation, the metadata of the HDR video of the Nth frame includes a flag bit, and the flag bit is used to indicate that the mode adopted by the HDR video of the Nth frame is the director mode or the flag bit is used to indicate that the mode adopted by the HDR video of the Nth frame is the automatic mode.

[0403] When the device 1100 is configured in or is itself a coding device, each module or unit in the device 1100 can be used to execute each action or processing process performed by the coding device in the above method. Here, in order to avoid redundancy, its detailed description is omitted.

[0404] In some embodiments, the apparatus 1100 may be a decoding device, or a chip or circuit disposed in the decoding device. When the apparatus 1100 is a decoding device, or a chip or circuit disposed in the decoding device, the acquisition unit 1140 is used to acquire the HDR video of the Nth frame and the metadata of the HDR video of the Nth frame; the processing unit 1110 is used to determine the mode adopted by the HDR video of the Nth frame according to the flag bit of the metadata.

[0405] In one implementation, when the flag indicates that the mode adopted by the HDR video of the Nth frame is the automatic mode, the processing unit 1110 is further used to: calculate the tone mapping tone-mapping curve parameters in the automatic mode according to the metadata of the HDR video of the Nth frame; generate the tone-mapping curve in the automatic mode according to the tone mapping tone-mapping curve parameters; and display the HDR video of the Nth frame according to the tone-mapping curve in the automatic mode and the HDR video of the Nth frame.

[0406] In one implementation, when the flag indicates that the mode adopted by the HDR video of the Nth frame is the director mode, the processing unit 1110 is further used to: extract tone-mapping curve parameters in the director mode from the metadata; and generate a tone-mapping curve in the director mode according to the curve parameters;

[0407] The HDR video of the Nth frame is displayed according to the tone-mapping curve in the director mode and the HDR video of the Nth frame.

[0408] When the apparatus 1100 is configured in or is itself a decoding device, each module or unit in the apparatus 1100 can be used to execute each action or processing process performed by the decoding device in the above method. Here, in order to avoid redundancy, the detailed description is omitted.

[0409] In some embodiments, the apparatus 1100 may be a decoding device, or a chip or circuit disposed in a decoding device. When the apparatus 1100 is a decoding device, or a chip or circuit disposed in the decoding device, the acquisition unit 1140 is used to acquire dynamic metadata of the Nth frame of HDR video according to a dynamic metadata generation algorithm; the processing unit 1110 is used to calculate tone-mapping curve parameters of the HDR video; the processing unit 1110 is also used to generate a tone-mapping curve according to the curve parameters; the processing unit 1110 is also used to determine the distortion D′ caused by the tone-mapping curve according to a quality evaluation algorithm; the processing unit 1110 is also used to obtain the offset deltaC[N] of the curve parameters of the Nth frame of HDR video according to the metadata of the frame (NM, N) in the M frame window, the tone-mapping curve parameters and the D′, wherein N is greater than 0; the processing unit 1110 is also used to generate metadata of the Nth frame of HDR video in director mode according to the dynamic metadata generation algorithm and the offset deltaC[N] of the curve parameters of the Nth frame of HDR video. .

[0410] In one implementation, the processing unit 1110 is further configured to: generate an adjusted tone-mapping curve according to an offset deltaC[N] of the curve parameter of the HDR video of the Nth frame; and determine a distortion D caused by the adjusted tone-mapping curve according to a quality evaluation algorithm. T , the D T The offset deltaC[N+j] used to predict the N+j frame in director mode is used, where j is greater than or equal to 1 and less than or equal to M.

[0411] In one implementation, the encoding device is applied to any one or more of the following preset configuration parameters: 1000nit, 500nit, and low dynamic range SDR.

[0412] In one implementation, the tone-mapping curve is as shown in formula (18):

[0413]

[0414] The curve parameters a and p are determined according to the dynamic metadata of the HDR video of the Nth frame, the curve parameters b, n and m are preset values, L is the input brightness, and L′ is the output brightness.

[0415] In one implementation, the processing unit 1110 is further used to: determine multiple offsets of the adjusted curve parameter p according to the predicted offset of the curve parameter p and K*basic step value, wherein K is a positive integer and K is greater than or equal to 0; determine curve parameters p″ and a″ corresponding to each of the multiple offsets of the curve parameter p according to the multiple offsets of the curve parameter p; generate multiple adjusted tone-mapping curves according to the parameters p″ and a″ corresponding to each of the multiple offsets; determine the distortion D″ caused by each of the multiple adjusted tone-mapping curves according to the quality evaluation algorithm; select the tone-mapping curve parameters p″ and a″ corresponding to any one D″ that meets the first condition from the multiple D″ as the target adjusted curve parameters p′ and a′, and the D T Any one D″ is selected.

[0416] In one implementation, the processing unit 1110 is specifically configured to: when K is 1, the determined offset is the offset of the predicted curve parameter p plus the basic step value, and the offset of the predicted curve parameter p minus the basic step value.

[0417] In one implementation, the first condition is to select a tone-mapping curve parameter with a smaller distortion D″ from two distortions D″.

[0418] In one implementation, the processing unit 1110 is specifically configured to: determine a plurality of offsets according to the basic delta±M*basic step value stepsize, wherein M is each positive integer less than or equal to K.

[0419] In one implementation, the first condition is to select the smallest D" from a plurality of D" or to select a D" that is smaller than a first threshold from a plurality of D".

[0420] In one implementation, the processing unit 1110 is further configured to:

[0421] The offset of the predicted curve parameter p is determined according to formula (19):

[0422]

[0423] Wherein, deltaC[Nk] represents the offset used by the Nk-th frame, D[Nk] represents the distortion calculated by the quality assessment algorithm of the Nk-th frame, and M is the window length.

[0424] In one implementation, the processing unit 1110 is specifically used to: when the mode adopted by the HDR video of the Nth frame is the director mode, the metadata of the HDR video of the Nth frame includes basic dynamic metadata and P[N], and the P[N] is the curve parameter P when the mode adopted by the HDR video of the Nth frame is the director mode.

[0425] In one implementation, the metadata includes a flag bit, and the flag bit is used to indicate that the mode adopted by the HDR video of the Nth frame is the director mode.

[0426] When the device 1100 is configured in or is itself a coding device, each module or unit in the device 1100 can be used to execute each action or processing process performed by the coding device in the above method. Here, in order to avoid redundancy, its detailed description is omitted.

[0427] In some embodiments, the apparatus 1100 may be a decoding device, or a chip or circuit disposed in the decoding device. When the apparatus 1100 is a decoding device, or a chip or circuit disposed in the decoding device, the acquisition unit 1140 is used to acquire the HDR video of the Nth frame and the dynamic metadata of the HDR video of the Nth frame; the processing unit 1110 is used to calculate the tone-mapping curve parameters of the HDR video of the Nth frame according to the dynamic metadata of the HDR video of the Nth frame; the processing unit 1110 is also used to generate a tone-mapping curve according to the curve parameters; the processing unit 1110 is also used to determine the distortion D′ caused by the tone-mapping curve according to the quality evaluation algorithm; the processing unit 1110 is also used to compare the D′ and D T , determine whether to use the automatic mode, where the D T is the threshold value.

[0428] In one implementation, the processing unit 1110 is specifically configured to: when the D′ is greater than the D T When , the offset deltaC[N] of the curve parameters of the HDR video of the Nth frame is obtained according to the metadata of the frame (NM, N) in the M frame window, the tone-mapping curve parameters and the D′, wherein the M is greater than 0.

[0429] In one implementation, the processing unit 1110 is further configured to: generate an adjusted tone-mapping curve according to the deltaC[N]; and determine the distortion D caused by the adjusted tone-mapping curve according to the quality evaluation algorithm. T .

[0430] In one implementation, the decoding device is applied to any one or more of the following preset configuration parameters: 1000nit, 500nit, and low dynamic range SDR.

[0431] In one implementation, the tone-mapping curve is as shown in formula (20):

[0432]

[0433] The curve parameters a and p are determined according to the dynamic metadata of the HDR video of the Nth frame, the curve parameters b, n and m are preset values, L is the input brightness, and L′ is the output brightness.

[0434] In one implementation, the processing unit 1110 is further used to: determine multiple offsets of the adjusted curve parameter p according to the predicted offset of the curve parameter p and K*basic step value, wherein K is a positive integer and K is greater than or equal to 0; determine curve parameters p″ and a″ corresponding to each of the multiple offsets of the curve parameter p according to the multiple offsets of the curve parameter p; generate multiple adjusted tone-mapping curves according to the parameters p″ and a″ corresponding to each of the multiple offsets; determine the distortion D″ caused by each of the multiple adjusted tone-mapping curves according to the quality evaluation algorithm; select the tone-mapping curve parameters p″ and a″ corresponding to any one D″ that meets the first condition from the multiple D″ as the target adjusted curve parameters p′ and a′, and the D T is the selected D″.

[0435] In one implementation, the processing unit 1110 is specifically configured to: when K is 1, the determined offset is the offset of the predicted curve parameter p plus the basic step value, and the offset of the predicted curve parameter p minus the basic step value.

[0436] In one implementation, the first condition is to select a tone-mapping curve parameter with a smaller distortion D″ from two distortions D″.

[0437] In one implementation, the processing unit 1110 is specifically configured to: determine a plurality of offsets according to the basic delta±M*basic step value stepsize, wherein M is each positive integer less than or equal to K.

[0438] In one implementation, the first condition is to select the smallest D" from a plurality of D" or to select a D" that is smaller than a first threshold from a plurality of D".

[0439] In one implementation, the processing unit 1110 is further configured to:

[0440] The offset of the predicted curve parameter p is determined according to formula (21):

[0441]

[0442] Wherein, deltaC[Nk] represents the offset used by the Nk-th frame, D[Nk] represents the distortion calculated by the quality assessment algorithm of the Nk-th frame, and M is the window length.

[0443] When the apparatus 1100 is configured in or is itself a decoding device, each module or unit in the apparatus 1100 can be used to execute each action or processing process performed by the decoding device in the above method. Here, in order to avoid redundancy, the detailed description is omitted.

[0444] Fig.16 This is a schematic diagram of the structure of a terminal device 1200 provided in the present application. The terminal device 1200 can execute the actions executed by the terminal device in the above method embodiment.

[0445] For ease of explanation, Fig.16 Only the main components of the terminal device are shown. Fig.16 As shown, the terminal device 1200 includes a processor, a memory, a control circuit, an antenna, and an input and output device.

[0446] The processor is mainly used to process the communication protocol and communication data, as well as to control the entire terminal device, execute the software program, and process the data of the software program, such as to support the terminal device to perform the actions described in the above-mentioned transmission precoding matrix indication method embodiment. The processor can realize the functions of the above-mentioned processing unit and acquisition unit. The memory is mainly used to store software programs and data, such as storing the codebook described in the above-mentioned embodiment. The control circuit is mainly used for conversion between baseband signals and radio frequency signals and processing of radio frequency signals. The control circuit and the antenna can also be called a transceiver, which is mainly used to transmit and receive radio frequency signals in the form of electromagnetic waves. Input and output devices, such as touch screens, display screens, keyboards, etc. are mainly used to receive data input by users and output data to users.

[0447] When the terminal device is turned on, the processor can read the software program in the storage unit, interpret and execute the instructions of the software program, and process the data of the software program. When data needs to be sent wirelessly, the processor performs baseband processing on the data to be sent, and outputs the baseband signal to the RF circuit. The RF circuit performs RF processing on the baseband signal and then sends the RF signal outward in the form of electromagnetic waves through the antenna. When data is sent to the terminal device, the RF circuit receives the RF signal through the antenna, converts the RF signal into a baseband signal, and outputs the baseband signal to the processor. The processor converts the baseband signal into data and processes the data.

[0448] Those skilled in the art will appreciate that for ease of description, Fig.16 Only one memory and processor are shown. In an actual terminal device, there may be multiple processors and memories. The memory may also be referred to as a storage medium or a storage device, etc., which is not limited in the embodiments of the present application.

[0449] For example, the processor may include a baseband processor and a central processing unit. The baseband processor is mainly used to process the communication protocol and communication data, and the central processing unit is mainly used to control the entire terminal device, execute software programs, and process data of the software programs. Fig.16 The processor in integrates the functions of the baseband processor and the central processing unit. Those skilled in the art will understand that the baseband processor and the central processing unit may also be independent processors, which are interconnected through technologies such as buses. Those skilled in the art will understand that the terminal device may include multiple baseband processors to adapt to different network standards, and the terminal device may include multiple central processing units to enhance its processing capabilities. The various components of the terminal device may be connected through various buses. The baseband processor may also be described as a baseband processing circuit or a baseband processing chip. The central processing unit may also be described as a central processing circuit or a central processing chip. The function of processing the communication protocol and communication data may be built into the processor, or may be stored in the storage unit in the form of a software program, and the processor executes the software program to implement the baseband processing function.

[0450] For example, in the embodiment of the present application, the antenna and the control circuit having transceiver functions may be regarded as the transceiver unit 1210 of the terminal device 1200, and the processor having a processing function may be regarded as the processing unit 1220 of the terminal device 1200. Fig. 9As shown, the terminal device 1200 includes a transceiver unit 1210 and a processing unit 1220. The transceiver unit may also be referred to as a transceiver, a transceiver, a transceiver device, etc. Optionally, the device used to implement the receiving function in the transceiver unit 1210 may be regarded as a receiving unit, and the device used to implement the sending function in the transceiver unit 1210 may be regarded as a sending unit, that is, the transceiver unit includes a receiving unit and a sending unit. Exemplarily, the receiving unit may also be referred to as a receiver, a receiver, a receiving circuit, etc., and the sending unit may be referred to as a transmitter, a transmitter, or a transmitting circuit, etc.

[0451] It should be understood that in the embodiments of the present application, the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0452] It should also be understood that the memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0453] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0454] An embodiment of the present application further provides a computer-readable medium on which a computer program is stored. When the computer program is executed by a computer, the steps performed by the encoding device or the decoding device in any of the above embodiments are implemented.

[0455] An embodiment of the present application also provides a computer program product, which, when executed by a computer, implements the steps performed by the encoding device or decoding device in any of the above embodiments.

[0456] The embodiment of the present application also provides a system chip, which includes: a communication unit and a processing unit. The processing unit may be, for example, a processor. The communication unit may be, for example, a communication interface, an input / output interface, a pin or a circuit. The processing unit may execute computer instructions to enable the chip in the communication device to perform the steps performed by the encoding device or the decoding device provided in the embodiment of the present application.

[0457] Optionally, the computer instructions are stored in a storage unit.

[0458] The embodiment of the present application further provides a computer-readable storage medium, the computer-readable storage medium storing a bitstream, the bitstream including encoded video data, the bitstream being obtained according to an encoding method, the encoding method comprising: obtaining dynamic metadata of an HDR video of an N-th frame according to a dynamic metadata generation algorithm; calculating tone-mapping curve parameters of the HDR video of the N-th frame according to the dynamic metadata of the HDR video of the N-th frame; generating a tone-mapping curve according to the curve parameters; performing display adaptation on the HDR video of the N-th frame according to the tone-mapping curve; determining a distortion D′ caused by the tone-mapping curve according to a quality evaluation algorithm; if N>0, obtaining an offset deltaC[N] of the tone-mapping curve parameters of the HDR video of the N-th frame according to the metadata of the frame (NM, N) in the M-frame window, the tone-mapping curve parameters and the D′; if D′ is greater than D T , determine that the mode adopted by the HDR video of the Nth frame is the director mode, where the D T is a threshold value; when the mode adopted by the HDR video of the Nth frame is the director mode, metadata of the HDR video of the Nth frame is generated according to the dynamic metadata generation algorithm and the deltaC[N].

[0459] The embodiment of the present application further provides a method for storing a code stream, comprising: generating a code stream, the code stream being a code stream generated according to the encoding method of the first aspect; and storing the code stream in a storage medium.

[0460] The embodiment of the present application also provides a method for transmitting a code stream, comprising: obtaining a code stream, which is a code stream generated according to the encoding method of the first aspect; transmitting the code stream to a storage medium or to a display device for decoding and display.

[0461] The various embodiments in this application may be used independently or in combination, which is not limited here.

[0462] In addition, various aspects or features of the present application can be implemented as methods, devices or products using standard programming and / or engineering techniques. The term "product" used in this application covers computer programs that can be accessed from any computer-readable device, carrier or medium. For example, computer-readable media may include, but are not limited to: magnetic storage devices (e.g., hard disks, floppy disks or tapes, etc.), optical disks (e.g., compact discs (CDs), digital versatile discs (DVDs), etc.), smart cards and flash memory devices (e.g., erasable programmable read-only memory (EPROM), cards, sticks or key drives, etc.). In addition, the various storage media described herein may represent one or more devices and / or other machine-readable media for storing information. The term "machine-readable medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing and / or carrying instructions and / or data.

[0463] It should be understood that "and / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent three situations: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previously associated objects are in an "or" relationship. "At least one" means one or more; "at least one of A and B" is similar to "A and / or B", describing the association relationship of associated objects, indicating that three relationships may exist. For example, at least one of A and B can represent three situations: A exists alone, A and B exist at the same time, and B exists alone.

[0464] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0465] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0466] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0467] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0468] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0469] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0470] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for processing high dynamic range HDR video, characterized in that: include: Acquire dynamic metadata of the HDR video of the Nth frame according to a dynamic metadata generation algorithm; Calculate tone-mapping curve parameters of the HDR video of the Nth frame according to the dynamic metadata of the HDR video of the Nth frame; Generate a tone-mapping curve according to the curve parameters; Performing display adaptation on the HDR video of the Nth frame according to the tone-mapping curve; Determine the distortion D caused by the tone-mapping curve according to the quality evaluation algorithm ′ ; If N>0, according to the metadata of the frame (NM, N-1) in the M frame window, the tone-mapping curve parameters and the D ′ , obtain the offset deltaC[N] of the tone-mapping curve parameter of the HDR video of the Nth frame; If the D ′ Greater than D T , determine that the mode adopted by the HDR video of the Nth frame is the director mode, wherein the D T is the threshold value; When the mode adopted by the HDR video of the Nth frame is the director mode, metadata of the HDR video of the Nth frame is generated according to the dynamic metadata generation algorithm and the deltaC[N].

2. The method according to claim 1, characterized in that The method further comprises: generating an adjusted tone-mapping curve according to the deltaC[N]; Determine the distortion D caused by the adjusted tone-mapping curve according to the quality evaluation algorithm T .

3. The method according to claim 1 or 2, characterized in that: The HDR video of the Nth frame is displayed and adapted according to any one or more of the following preset configuration parameters: 1000nit, 500nit and low dynamic range SDR.

4. The method according to claim 1 or 2, characterized in that: The tone-mapping curve is shown in formula (1): The curve parameters a and p are determined according to the dynamic metadata of the HDR video of the Nth frame, the curve parameters b, n and m are preset values, L is the input brightness, L ′ is the output brightness.

5. The method according to claim 4, characterized in that The method further comprises: Determine multiple offsets of the adjusted curve parameter p according to the predicted offset of the curve parameter p and K*basic step value, where K is a positive integer; According to the multiple offsets of the curve parameter p, determine the curve parameter p corresponding to each of the multiple offsets of the curve parameter p. ″ and a ″ ; According to the parameter p corresponding to each of the multiple offsets ″ and a ″ Generate multiple adjusted tone-mapping curves; Determine the distortion D caused by each tone-mapping curve in the plurality of adjusted tone-mapping curves according to the quality evaluation algorithm. ″ ; From the multiple D ″ Select any D that satisfies the first condition ″ The corresponding tone-mapping curve parameter p ″ and a ″ The curve parameter p adjusted for the target ′ and a ′ , the D T For the selected D ″ .

6. The method according to claim 5, characterized in that The step of determining the multiple offsets of the adjusted curve parameter p according to the predicted offset of the curve parameter p and the K*basic step value comprises: The K is 1, and the determined offset is the offset of the predicted curve parameter p plus the basic step value, and the offset of the predicted curve parameter p minus the basic step value.

7. The method according to claim 6, characterized in that The first condition is that from the two distortions D ″ Select Distortion D ″ Small tone-mapping curve parameters.

8. The method according to claim 5, characterized in that The step of determining the multiple offsets of the adjusted curve parameter p according to the predicted offset of the curve parameter p and the K*basic step value comprises: Multiple offsets are determined according to basic delta±M1*basic step value stepsize, where M1 is any positive integer less than or equal to K.

9. The method according to claim 8, characterized in that: The first condition is that from multiple D ″ Select the smallest D ″ or from multiple D ″ Select D that is less than the first threshold ″ .

10. The method according to any one of claims 5 to 6 and 9, characterized in that: The method further comprises: The offset of the predicted curve parameter p is determined according to formula (2): Wherein, deltaC[Nk] represents the offset used by the Nk-th frame, D[Nk] represents the distortion of the Nk-th frame, and M is the window length.

11. The method according to any one of claims 5 to 6 and 9, characterized in that: When the mode adopted by the HDR video of the Nth frame is the director mode, the metadata of the HDR video of the Nth frame includes dynamic metadata and P[N] of the HDR video of the Nth frame, and P[N] is the curve parameter P when the mode adopted by the HDR video of the Nth frame is the director mode.

12. The method according to any one of claims 1 to 2, 5 to 6 and 9, characterized in that: The method further comprises: When the D ′ Less than or equal to D T When the HDR video of the Nth frame is determined to adopt the automatic mode.

13. The method according to any one of claims 1 to 2, 5 to 6 and 9, characterized in that: The metadata of the HDR video of the Nth frame includes a flag bit, and the flag bit is used to indicate that the mode adopted by the HDR video of the Nth frame is the director mode or the flag bit is used to indicate that the mode adopted by the HDR video of the Nth frame is the automatic mode.

14. A method for processing high dynamic range HDR video, characterized in that: include: Decoding to obtain an HDR video of the Nth frame and metadata of the HDR video of the Nth frame, wherein N is greater than 0, and a mode adopted by the HDR video of the Nth frame is a director mode; Extracting tone-mapping curve parameters in director mode from the metadata; Generate a tone-mapping curve in director mode according to the tone-mapping curve parameters in director mode; The HDR video of the Nth frame is displayed according to the tone-mapping curve in the director mode.

15. The method according to claim 14, characterized in that The metadata of the HDR video of the Nth frame includes a flag bit, and the flag bit is used to indicate that the mode adopted by the HDR video of the Nth frame is the director mode or the flag bit is used to indicate that the mode adopted by the HDR video of the Nth frame is the automatic mode.

16. The method according to claim 15, characterized in that When the flag bit indicates that the mode adopted by the HDR video of the Nth frame is the automatic mode, the method further includes: Calculating tone-mapping curve parameters in automatic mode according to metadata of the HDR video of the Nth frame; Generate a tone-mapping curve in automatic mode according to the tone-mapping curve parameters in automatic mode; The HDR video of the Nth frame is displayed according to the tone-mapping curve in the automatic mode.

17. A device for high dynamic range (HDR) video processing, characterized in that: include: An acquisition unit, the acquisition unit is used to acquire dynamic metadata of the Nth frame of HDR video according to a dynamic metadata generation algorithm, wherein N is greater than 0; A processing unit, the processing unit being configured to calculate tone-mapping curve parameters of the HDR video of the Nth frame according to the dynamic metadata of the HDR video of the Nth frame; The processing unit is further used to generate a tone-mapping curve according to the curve parameters; The processing unit is further used to perform display adaptation on the HDR video of the Nth frame according to the tone-mapping curve; The processing unit is further configured to determine the distortion D caused by the tone-mapping curve according to a quality evaluation algorithm. ′ ; The processing unit is further configured to, when N>0, calculate the frame metadata of the frame in the M frame window (NM, N-1), the tone-mapping curve parameters and the D ′ , obtain the offset deltaC[N] of the tone-mapping curve parameter of the HDR video of the Nth frame; The processing unit is also used to ′ Greater than D T , it is determined that the mode adopted by the HDR video of the Nth frame is the director mode, wherein the D T is the threshold value; The processing unit is further configured to generate metadata of the HDR video of the Nth frame according to the dynamic metadata generation algorithm and the deltaC[N] when the mode adopted by the HDR video of the Nth frame is the director mode.

18. The device according to claim 17, characterized in that The processing unit is also used for: generating an adjusted tone-mapping curve according to the deltaC[N]; Determine the distortion D caused by the adjusted tone-mapping curve according to the quality evaluation algorithm T .

19. The device according to claim 17 or 18, characterized in that The processing unit is specifically configured to perform display adaptation on the HDR video of the Nth frame according to any one or more of the following preset configuration parameters: 1000nit, 500nit and low dynamic range SDR.

20. The device according to claim 17 or 18, characterized in that The tone-mapping curve is shown in formula (3): The curve parameters a and p are determined according to the dynamic metadata of the HDR video of the Nth frame, the curve parameters b, n and m are preset values, L is the input brightness, L ′ is the output brightness.

21. The device according to claim 20, characterized in that The processing unit is also used for: Determine multiple offsets of the adjusted curve parameter p according to the predicted offset of the curve parameter p and K*basic step value, where K is a positive integer; According to the multiple offsets of the curve parameter p, determine the curve parameter p corresponding to each of the multiple offsets of the curve parameter p. ″ and a ″ ; According to the parameter p corresponding to each of the multiple offsets ″ and a ″ Generate multiple adjusted tone-mapping curves; Determine the distortion D caused by each tone-mapping curve in the plurality of adjusted tone-mapping curves according to the quality evaluation algorithm. ″ ; From the multiple D ″ Select any D that satisfies the first condition ″ The corresponding tone-mapping curve parameter p ″ and a ″ The curve parameter p adjusted for the target ′ and a ′ , the D T For the selected D ″ .

22. The device according to claim 21, characterized in that The processing unit is specifically used for: The K is 1, and the determined offset is the offset of the predicted curve parameter p plus the basic step value, and the offset of the predicted curve parameter p minus the basic step value.

23. The device according to claim 22, characterized in that The first condition is that from the two distortions D ″ Select Distortion D ″ Small tone-mapping curve parameters.

24. The device according to claim 21, characterized in that The processing unit is specifically used for: Multiple offsets are determined according to basic delta±M1*basic step value stepsize, where M1 is any positive integer less than or equal to K.

25. The device according to claim 24, characterized in that The first condition is that from multiple D ″ Select the smallest D ″ or from multiple D ″ Select D that is less than the first threshold ″ .

26. The device according to any one of claims 21 to 22 and 25, characterized in that The processing unit is also used for: The offset of the predicted curve parameter p is determined according to formula (4): Wherein, deltaC[Nk] represents the offset used by the Nk-th frame, D[Nk] represents the distortion calculated by the quality assessment algorithm of the Nk-th frame, and M is the window length.

27. The device according to any one of claims 21 to 22 and 25, characterized in that The processing unit is specifically used for: In the case where the mode adopted by the HDR video of the Nth frame is the director mode, the metadata of the HDR video of the Nth frame includes dynamic metadata and P[N] of the HDR video of the Nth frame, and P[N] is the curve parameter P when the mode adopted by the HDR video of the Nth frame is the director mode.

28. The device according to any one of claims 17 to 18, 21 to 22, and 25, characterized in that The processing unit is also used for: When the D ′ Less than or equal to D T When the HDR video of the Nth frame is determined to adopt the automatic mode.

29. The device according to any one of claims 17 to 18, 21 to 22, and 25, characterized in that The metadata of the HDR video of the Nth frame includes a flag bit, and the flag bit is used to indicate that the mode adopted by the HDR video of the Nth frame is the director mode or the flag bit is used to indicate that the mode adopted by the HDR video of the Nth frame is the automatic mode.

30. A device for high dynamic range (HDR) video processing, characterized in that: include: an acquisition unit, configured to decode and acquire an HDR video of an N-th frame and metadata of the HDR video of the N-th frame, wherein N is greater than 0, and a mode adopted by the HDR video of the N-th frame is a director mode; A processing unit, configured to extract tone-mapping curve parameters in director mode from the metadata; The processing unit is further configured to generate a tone-mapping curve in director mode according to the tone-mapping curve parameters in director mode; A display unit is used to display the HDR video of the Nth frame according to the tone-mapping curve in the director mode.

31. The device according to claim 30, characterized in that The metadata of the HDR video of the Nth frame includes a flag bit, and the flag bit is used to indicate that the mode adopted by the HDR video of the Nth frame is the director mode or the flag bit is used to indicate that the mode adopted by the HDR video of the Nth frame is the automatic mode.

32. The device according to claim 31, characterized in that When the flag indicates that the mode adopted by the HDR video of the Nth frame is the automatic mode, The processing unit is further used to calculate tone-mapping curve parameters in automatic mode according to the metadata of the HDR video of the Nth frame; The processing unit is further configured to generate a tone-mapping curve in the automatic mode according to the tone-mapping curve parameters in the automatic mode; The display unit is further configured to display the Nth frame of HDR video according to the tone-mapping curve in the automatic mode.

33. A method for storing a code stream, characterized in that: include: Generate a bitstream, wherein a user-defined portion of the bitstream is embedded with metadata generated according to the method of claim 1; The code stream is stored in a storage medium.

34. A method for transmitting a code stream, characterized in that: include: Acquire a bitstream, wherein a user-defined portion of the bitstream is embedded with metadata generated according to the method of claim 1; Transmitting the code stream to a storage medium for storage; or The code stream is transmitted to a display device for decoding and then displayed.

Images