Video data processing method and device and medium

Abstract

The embodiment of the invention provides a video data processing method and device and a medium. The method comprises the following steps: acquiring metadata of video frames in a standard dynamic range video; obtaining display parameters of the display equipment; determining a mapping curve corresponding to the video frame according to the display parameters and the metadata; the mapping curve is used for representing a mapping relation between a source color intensity value and a target color intensity value in an electric signal space; wherein the expansion upper limit value of the target color intensity value in the mapping curve is determined according to the display parameters and the metadata. The embodiment of the invention can effectively improve the display effect of the video frame.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of video processing technology, and in particular to a video data processing method, device, and medium. Background Art

[0002] Dynamic range refers to the range from the darkest to the brightest parts of an image, typically expressed as the ratio of the brightest to the darkest parts. Generally speaking, the larger the dynamic range, the closer the image's display will resemble the visual experience of a real scene. Standard Dynamic Range (SDR) is a common video image standard that limits the displayed light intensity based on the brightness, contrast, and color characteristics of cathode ray tube displays.

[0003] At present, the digital signals of SDR videos are usually converted into light intensity values ​​of the display device according to the EOTF (Electro-Optical Transfer Function) specified in the ITU-BT (International Telecommunication Union-Radio Communication Sector Broadcasting Service) 709 standard, so that the display device can control the pixel emission according to these light intensity values, and finally present the picture content of the corresponding brightness.

[0004] However, the maximum display brightness specified in the above standard is 100 nits, and the peak brightness of most display devices on the market is usually greater than 100 nits. Therefore, directly using the electro-optical transfer function specified in this standard cannot fully utilize the high brightness performance of the display device, resulting in poor display quality when the display device displays SDR video. Summary of the Invention

[0005] Embodiments of the present invention provide a video data processing method, device, and medium, which can effectively improve the display effect of video frames.

[0006] In a first aspect, an embodiment of the present invention discloses a method for processing video data, the method comprising:

[0007] Get metadata for video frames in standard dynamic range video;

[0008] Get the display parameters of the display device;

[0009] A mapping curve corresponding to the video frame is determined based on the display parameters and the metadata; the mapping curve is used to characterize the mapping relationship between the source color intensity value and the target color intensity value in the electrical signal space; wherein the extended upper limit value of the target color intensity value in the mapping curve is determined based on the display parameters and the metadata.

[0010] In a second aspect, an embodiment of the present invention discloses a method for processing video data, the method comprising:

[0011] Get the YUV signal of the video frame in the standard dynamic range video;

[0012] Converting the YUV signal into an RGB electrical signal;

[0013] Determine the source color intensity value of the video frame at the pixel point based on the RGB electrical signal;

[0014] The source color intensity value is mapped using a mapping curve corresponding to the video frame to obtain a target color intensity value; the mapping curve is used to represent a mapping relationship between the source color intensity value and the target color intensity value in the electrical signal space; wherein an extended upper limit value of the target color intensity value in the mapping curve is determined based on display parameters of a display device and metadata of the video frame;

[0015] According to the target color intensity value, color correction is performed on the RGB optical signal converted from the RGB electrical signal, so that the color-corrected RGB optical signal is displayed on a display device.

[0016] In a third aspect, an embodiment of the present invention discloses a video data processing method, characterized in that the method is applied to a decoding end and includes:

[0017] Get the metadata of the video frame from the standard dynamic range video bitstream sent by the encoder;

[0018] Get the display parameters of the display device;

[0019] A mapping curve corresponding to the video frame is determined based on the display parameters and the metadata; the mapping curve is used to characterize the mapping relationship between the source color intensity value and the target color intensity value in the electrical signal space; wherein the extended upper limit value of the target color intensity value in the mapping curve is determined based on the display parameters and the metadata.

[0020] In a fourth aspect, an embodiment of the present invention discloses a video data processing method, which is applied to an encoding end and includes:

[0021] Get metadata for video frames in standard dynamic range video;

[0022] Encoding the metadata and the standard dynamic range video to obtain a bitstream;

[0023] The code stream is sent to a decoding end, so that the decoding end determines a mapping curve corresponding to the video frame based on display parameters of a display device and the metadata; the mapping curve is used to represent a mapping relationship between source color intensity values ​​and target color intensity values ​​in an electrical signal space; wherein an extended upper limit value of the target color intensity value in the mapping curve is determined based on the display parameters and the metadata.

[0024] In a fifth aspect, an embodiment of the present invention discloses a video data processing device, comprising:

[0025] A metadata acquisition module, used to obtain metadata of video frames in a standard dynamic range video;

[0026] A display parameter acquisition module, used to obtain display parameters of a display device;

[0027] A mapping curve determination module is used to determine a mapping curve corresponding to the video frame based on the display parameters and the metadata; the mapping curve is used to represent the mapping relationship between the source color intensity value and the target color intensity value in the electrical signal space; wherein the extended upper limit value of the target color intensity value in the mapping curve is determined based on the display parameters and the metadata.

[0028] In a sixth aspect, an embodiment of the present invention discloses a video data processing device, the device comprising:

[0029] YUV acquisition module, used to obtain the YUV signal of the video frame in the standard dynamic range video;

[0030] A conversion module, used for converting the YUV signal into an RGB electrical signal;

[0031] A source intensity determination module is used to determine the source color intensity value of a pixel in a video frame based on the RGB electrical signal;

[0032] a mapping module, configured to map the source color intensity values ​​using a mapping curve corresponding to the video frame to obtain a target color intensity value; the mapping curve is configured to represent a mapping relationship between the source color intensity values ​​and the target color intensity values ​​in an electrical signal space; wherein an extended upper limit value of the target color intensity value in the mapping curve is determined based on display parameters of a display device and metadata of the video frame;

[0033] The color correction module is used to perform color correction on the RGB light signal converted from the RGB electrical signal according to the target color intensity value, so as to display the color-corrected RGB light signal on a display device.

[0034] In a seventh aspect, an embodiment of the present invention discloses a video data processing device, which is applied to a decoding end and includes:

[0035] The metadata acquisition module is used to obtain the metadata of the video frame from the standard dynamic range video bitstream sent by the encoder;

[0036] A display parameter acquisition module, used to obtain display parameters of a display device;

[0037] A mapping curve determination module is used to determine a mapping curve corresponding to the video frame based on the display parameters and the metadata; the mapping curve is used to represent the mapping relationship between the source color intensity value and the target color intensity value in the electrical signal space; wherein the extended upper limit value of the target color intensity value in the mapping curve is determined based on the display parameters and the metadata.

[0038] In an eighth aspect, an embodiment of the present invention discloses a video data processing device, which is applied to an encoding end and includes:

[0039] A metadata acquisition module, used to obtain metadata of video frames in a standard dynamic range video;

[0040] An encoding module, configured to encode the metadata and the standard dynamic range video to obtain a bit stream;

[0041] A sending module is configured to send the code stream to a decoding end, so that the decoding end determines a mapping curve corresponding to the video frame based on display parameters of a display device and the metadata; the mapping curve is configured to represent a mapping relationship between source color intensity values ​​and target color intensity values ​​in an electrical signal space; wherein an extended upper limit value of the target color intensity value in the mapping curve is determined based on the display parameters and the metadata.

[0042] In a ninth aspect, an embodiment of the present invention discloses a device for video data processing, comprising a processor and a non-volatile memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to execute the aforementioned method.

[0043] In a tenth aspect, an embodiment of the present invention discloses a non-transitory computer-readable storage medium storing instructions, wherein the instructions enable a processor to execute the aforementioned method.

[0044] In the eleventh aspect, an embodiment of the present invention discloses a non-transitory computer-readable recording medium storing a bitstream generated by a method executed by an apparatus for video data processing, wherein the method comprises: obtaining metadata of video frames in a standard dynamic range video; encoding the metadata and the standard dynamic range video to obtain a bitstream.

[0045] In the twelfth aspect, an embodiment of the present invention discloses a method for storing a bit stream of video data, comprising: obtaining metadata of video frames in a standard dynamic range video; encoding the metadata and the standard dynamic range video to obtain a bit stream; and storing the bit stream in a non-transitory computer-readable recording medium.

[0046] In a thirteenth aspect, an embodiment of the present invention discloses a method for storing a bitstream, comprising: executing the video data processing method described in the fourth aspect to generate a bitstream; and storing the bitstream.

[0047] Compared with the prior art, the embodiments of the present invention have the following advantages:

[0048] In the technical solution of the embodiment of the present invention, a mapping curve corresponding to a video frame is determined based on display parameters of a display device and metadata of the video frame, and an extended upper limit value of a target color intensity value in the mapping curve is determined based on the display parameters and metadata.

[0049] Since the metadata of the video frame can reflect the enhancement requirements of the highlight area of ​​the video frame content, the display parameters of the display device limit the brightness expansion capability supported by the display device. Determining the upper limit of the expansion of the mapping curve based on these two factors can not only ensure that the target color intensity value does not exceed the capability range of the display device and prevent display anomalies due to hardware limitations, but also make full use of the brightness potential of the display device and reasonably expand the brightness of the light signal of the video frame according to the actual needs of the video content. Therefore, the mapping curve corresponding to the video frame in the embodiment of the present invention can improve the brightness level of the video frame based on the expansion of the brightness range of the light signal, such as making the dark details more clearly visible and the highlight parts not overexposed, thereby effectively improving the display effect of the video frame. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 1 is a schematic flow chart of steps of a video data processing method according to an embodiment of the present invention;

[0051] Figure 2 is a schematic diagram of a mapping curve according to an embodiment of the present invention;

[0052] Figure 3 1 is a schematic flow chart of steps of a video data processing method according to an embodiment of the present invention;

[0053] Figure 4 This is a flow chart of a video data processing method in an end-to-end scenario according to an embodiment of the present invention;

[0054] Figure 5 1 is a schematic flow chart of steps of a video data processing method according to an embodiment of the present invention;

[0055] Figure 6 1 is a schematic flow chart of steps of a video data processing method according to an embodiment of the present invention;

[0056] Figure 7 is a structural diagram of a video data processing device according to an embodiment of the present invention;

[0057] Figure 8 is a structural diagram of a video data processing device according to an embodiment of the present invention;

[0058] Figure 9 is a structural diagram of a video data processing device according to an embodiment of the present invention;

[0059] Figure 10 is a structural diagram of a video data processing device according to an embodiment of the present invention;

[0060] Figure 11 FIG. 1 is a schematic structural diagram of an electronic device 1100 according to an embodiment of the present invention. DETAILED DESCRIPTION

[0061] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0062] The video data processing method of the embodiment of the present invention can be used to process SDR video to improve the display effect of the SDR video on a display device.

[0063] The video signal processing method provided in the embodiment of the present invention can be applied to electronic devices. In specific applications, the electronic devices can be set-top boxes, smart TVs, smart phones, personal computers, multimedia players, etc.

[0064] It is understood that electronic devices with screens can be used to display videos, meaning they also function as display devices. For example, devices with screens, such as smart TVs, smartphones, and laptops, can function as display devices to display videos. Electronic devices without screens can display videos via connected display devices. For example, set-top boxes, multimedia players, and desktop computers can display videos via connected monitors.

[0065] The video data processing method according to the embodiment of the present invention is described below through specific embodiments.

[0066] Reference Figure 1 , which shows a schematic flow chart of the steps of a video data processing method according to an embodiment of the present invention. The method may specifically include the following steps:

[0067] Step 101: Obtain metadata of video frames in a standard dynamic range video;

[0068] Step 102: Obtain display parameters of the display device;

[0069] Step 103: Determine a mapping curve corresponding to the video frame based on the display parameters and the metadata. The mapping curve is used to characterize the mapping relationship between the source color intensity value and the target color intensity value in the electrical signal space. The extended upper limit value of the target color intensity value in the mapping curve is determined based on the display parameters and the metadata.

[0070] In the technical solution of the embodiment of the present invention, a mapping curve corresponding to a video frame is determined based on display parameters of a display device and metadata of the video frame, and an extended upper limit value of a target color intensity value in the mapping curve is determined based on the display parameters and metadata.

[0071] Since the metadata of the video frame can reflect the enhancement requirements of the highlight area of ​​the video frame content, the display parameters of the display device limit the brightness expansion capability supported by the display device. Determining the upper limit of the expansion of the mapping curve based on these two factors can not only ensure that the target color intensity value does not exceed the capability range of the display device and prevent display anomalies due to hardware limitations, but also make full use of the brightness potential of the display device and reasonably expand the brightness of the light signal of the video frame according to the actual needs of the video content. Therefore, the mapping curve corresponding to the video frame in the embodiment of the present invention can improve the brightness level of the video frame based on the expansion of the brightness range of the light signal, such as making the dark details more clearly visible and the highlight parts not overexposed, thereby effectively improving the display effect of the video frame.

[0072] In step 101, a video frame is a static image in an SDR video. It is the basic unit of SDR video. Playing multiple video frames in sequence forms a dynamic video image. Metadata is data used to describe the attributes and characteristics of data. Video frame metadata describes relevant information such as the content, characteristics, and time position of the video frame.

[0073] The electronic device, acting as a decoder, can obtain metadata for video frames from the standard dynamic range video bitstream sent by the encoder. In this case, the metadata for the video frames can be obtained by the encoder using image processing methods and encapsulated into the SDR video bitstream. Alternatively, the electronic device can use image processing methods to obtain metadata for the video frames.

[0074] In a specific implementation, the metadata specifically includes at least one of the following brightness features: a bright part feature value, a dark part feature value, a maximum brightness value, an average brightness value, and a dark part compensation coefficient.

[0075] In a specific implementation, the process of obtaining metadata of a video frame in a standard dynamic range video in step 101 specifically includes the following steps:

[0076] Step A1: determining a histogram distribution of a luminance component of a video frame, and determining a mid-gray value of the histogram distribution;

[0077] Step A2: determining a bright feature value based on the sum of the brightness of pixels whose brightness values ​​are greater than the mid-gray value and the number of pixels in the histogram distribution;

[0078] Step A3, determining a dark feature value based on the sum of the brightness of pixels whose brightness values ​​are not greater than the middle gray value in the histogram distribution, the number of pixels, and the middle gray value;

[0079] Step A4: determining an average brightness value based on the sum of the product of the brightness value of the light signal of the video frame and the number of pixels corresponding to the brightness value, and the total number of pixels in the video frame;

[0080] Step A5: Use the middle gray value, the bright feature value, the dark feature value, and the average brightness value as metadata of the video frame.

[0081] In step A1, in the YUV (Luminance Chrominance) color space, the Y component (luminance component) represents the brightness information (i.e., the degree of lightness and darkness) of the video frame. It is a single-channel signal that describes the brightness of the image. It is independent of the chrominance information (represented by the UV component) and can directly reflect the brightness and darkness levels of the image.

[0082] The luminance component histogram (HistY) is a graph showing the relationship between the number of pixels and luminance values, calculated by counting the pixel values ​​of the Y component (i.e., luminance values) in a video frame. Generally speaking, the horizontal axis of a histogram represents luminance values, covering a range from darkest to brightest; the vertical axis represents the number or percentage of pixels with corresponding luminance values.

[0083] The brightness value of the Y component usually ranges from 0 to 255. To facilitate subsequent processing, the brightness value of the Y component can be normalized. In this case, the brightness value of the Y component ranges from 0 to 1.

[0084] Middle Gray represents the brightness value that achieves a visually balanced balance of light and dark within a video frame's luminance distribution. This value varies depending on factors such as the specific content of the video frame and shooting conditions. It serves as a key reference value for achieving a balanced brightness relationship across the frame, consistent with visual perception.

[0085] The embodiment of the present invention does not limit the specific method for determining the mid-gray value.

[0086] For example, the middle gray value may be a set value such as 118. In another example, the square root of the product of the brightness value of the first brightness percentile and the brightness value of the second brightness percentile in the histogram distribution may be determined as the middle gray value of the histogram distribution.

[0087] The brightness values ​​of all pixels in the video frame can be sorted in ascending order to obtain a sorting result. The first brightness percentile and the second brightness percentile respectively represent the ranking position of the brightness value in the sorting result. For example, the 0.05% position can be used as the first brightness percentile, and the 99.95% position can be used as the second brightness percentile. In this way, the brightness value of the first brightness percentile is smaller than the brightness value of the second brightness percentile.

[0088] In an optional implementation of the present invention, the process of determining the middle gray value of the histogram distribution in step A1 specifically includes the following steps:

[0089] Step A11: determining an initial value of the mid-gray value based on the brightness value of the first brightness percentile and the brightness value of the second brightness percentile in the histogram distribution;

[0090] Step A12: Divide the brightness range into a fourth brightness range and a fifth brightness range according to the initial value of the middle gray value;

[0091] Step A13: determining a current value of the mid-gray value based on a first brightness average value of the histogram distribution of the fourth brightness interval in the electrical signal space and a second brightness average value of the histogram distribution of the fifth brightness interval in the optical signal space;

[0092] Step A14: If the current value of the middle gray value does not meet the set conditions, the fourth brightness interval and the fifth brightness interval are updated according to the current value of the middle gray value, and according to the updated fourth brightness interval and fifth brightness interval, the first brightness average value of the histogram distribution of the fourth brightness interval in the electrical signal space and the second brightness average value of the histogram distribution of the fifth brightness interval in the optical signal space are executed to determine the current value of the middle gray value in step A13.

[0093] From step A11 to step A14, the middle gray value is determined in an iterative manner, which can improve the accuracy and robustness of the middle gray value.

[0094] In step A11 , the square root of the product of the brightness value of the first brightness percentile and the brightness value of the second brightness percentile in the histogram distribution may be determined as the initial value of the middle gray value of the histogram distribution.

[0095] In step A12 , the brightness interval may be divided into a fourth brightness interval [0, MildGray) and a fifth brightness interval [MildGray, 1] according to the initial value of the middle gray value MildGray.

[0096] In step A13, assuming that AverageE_s represents the first brightness average value of the histogram distribution of the fourth brightness interval [0, MildGray) in the electrical signal space, and AverageO_l represents the second brightness average value of the histogram distribution of the fifth brightness interval [MildGray, 1] in the optical signal space, the square root of the product of the first brightness average value and the second brightness average value can be determined as the current value of the middle gray value of the histogram distribution, see formula (1) for details.

[0097]

[0098] In step A14, a condition is set to indicate the condition for terminating the iteration. The condition can be determined by a person skilled in the art based on actual application requirements. For example, the condition can be set to indicate that the absolute value of the difference between the jth current value of the mid-gray value and the (j-1)th current value of the mid-gray value is less than a first threshold, where j represents the number of iterations and can be a positive integer, and the first threshold can be a positive number.

[0099] If the current value of the middle gray value meets the set conditions, the iteration can be ended and the current value of the middle gray value is used as the final middle gray value.

[0100] If the current value of the middle gray value does not meet the set condition, the fourth and fifth brightness intervals are updated according to the current value of the middle gray value, and step A13 is repeated according to the updated fourth and fifth brightness intervals to obtain the next current value.

[0101] Step A2 is used to determine the highlight feature value. Assuming the brightness range is divided into a highlight range and a dark range based on the mid-gray value, the highlight range corresponds to a brightness value greater than the mid-gray value, while the dark range corresponds to a brightness value less than or equal to the mid-gray value. The highlight feature value is used to characterize the brightness characteristics corresponding to the highlight range.

[0102] In a specific implementation, the first expectation of the pixels having brightness values ​​i greater than the middle gray value MildGray in the histogram distribution can be determined based on the sum of the brightness of the pixels having brightness values ​​greater than the middle gray value in the histogram distribution and the number of pixels having brightness values ​​greater than the middle gray value in the histogram distribution, as the bright feature value LightE. The specific determination process is referred to formula (2).

[0103] LightE=∑ o>MildGray HistY[i]*i / ∑ i＞MildGrayHistY[i] (2)

[0104] Where HistY[i] represents the number of pixels with brightness value i in the histogram distribution. i＞MildGray HistY[i] represents the number of pixels whose brightness value is greater than the middle gray value in the histogram distribution. i＞MildGray HistY[i]*i represents the total brightness of pixels whose brightness values ​​are greater than the middle gray value in the histogram distribution.

[0105] Step A3 is used to determine the dark area feature value. The dark area feature value is used to characterize the brightness feature corresponding to the dark area interval.

[0106] In a specific implementation, the second expected MeanDark of the pixel whose brightness value i is not greater than the middle gray value MildGray in the histogram distribution can be first determined based on the sum of the brightness of the pixel whose brightness value is not greater than the middle gray value in the histogram distribution and the number of pixel whose brightness value is not greater than the middle gray value in the histogram distribution. The specific determination process is referred to formula (3).

[0107] MeanDark=∑ i∈[0,Mi1dGray] HistY[i]*i / ∑ i∈[0,MildGray] HistY[i] (3)

[0108] Among them, ∑ i∈[0,MildGray] HistY[i] represents the number of pixels in the histogram distribution whose brightness value is not greater than the middle gray value. i∈[0,MildGray] HistY[i]*i represents the total brightness of pixels whose brightness values ​​are not greater than the middle gray value in the histogram distribution.

[0109] Furthermore, the dark feature value DarkE can be determined according to the second expectation and the middle gray value using formula (4).

[0110]

[0111] Step A4 is used to determine an average brightness value. The average brightness value is a weighted average value of the brightness distribution of the video frame and can represent the overall brightness characteristics of the video frame.

[0112] In a specific implementation, the electrical signal of the video frame can be first converted into an optical signal using formula (5) according to the electro-optical conversion function.

[0113] Luma[i]=EOTF(i / 255) (5)

[0114] Wherein, i / 255 represents the brightness value of the normalized electrical signal; Luma[i] represents the brightness value of the optical signal of the video frame; and the function EOTF() represents the electro-optical conversion function.

[0115] In a specific implementation, the brightness values ​​of the light signal can be traversed to calculate the sum of the products of each brightness value Luma[i] and the number of pixels with brightness value i in the histogram distribution HistY[i]; the ratio of the sum of the products to the total number of pixels RES in the video frame is determined as the average brightness value AvergO. The specific calculation process is shown in formula (6):

[0116] AvergO=∑HistY[i]*Luma[i] / HES (6)

[0117] The total number of pixels RES of a video frame is equal to the product of the horizontal number of pixels (width) and the vertical number of pixels (height) of the video frame. Taking a video frame with a resolution of 1920×1080 as an example, the total number of pixels is: 1920×1080=2073600.

[0118] The maximum brightness value MaxE can be the maximum brightness value of all pixels in the video frame. Specifically, it can be determined using formula (7):

[0119] MaxE=Max(HistY[i]) (7)

[0120] The dark compensation coefficient CompS is used to adjust the mapping slope of the dark portion corresponding to the first brightness range. The dark compensation coefficient can be set by the user based on the overall brightness characteristics of the SDR video (such as the average brightness value). For example, if the overall brightness characteristics of the SDR video are less than the second threshold, the dark compensation coefficient is set by the user to 0.05; if the overall brightness of the SDR video is greater than the third threshold, the dark compensation coefficient is set by the user to 0. The second and third thresholds can both be positive numbers, and the third threshold is greater than the second threshold.

[0121] In step 102, the display parameters of the display device refer to the technical characteristics of the display device used to adjust and control the image output. In a specific implementation, the display parameters specifically involve technical characteristics of the display device such as brightness, contrast, color gamut, and response time.

[0122] In an optional implementation, the display parameters specifically include: maximum screen brightness MaxDisplay, and / or screen brightness extension parameter EdrN.

[0123] The maximum screen brightness may refer to the maximum screen brightness that a display device can achieve when playing an SDR video. For example, under standard ambient light, a luminance meter may be used to measure the peak brightness (in nits) of a display device outputting a full-white SDR image. The measurement result may be used as the maximum screen brightness. Of course, the embodiments of the present invention do not limit the specific method for determining the maximum screen brightness.

[0124] The screen brightness expansion parameter is used to quantify the extent to which the display device increases video brightness when playing SDR video. The screen brightness expansion parameter is typically a value greater than 1 and can be set by the user. It will be understood that the embodiments of the present invention do not limit the specific method for determining the screen brightness expansion parameter.

[0125] In one implementation, the display parameters include a screen brightness extension parameter, and the metadata includes a maximum brightness value. The process of determining an extension upper limit value of a target color intensity value in a mapping curve according to the embodiment of the present invention specifically includes:

[0126] Step B1: determining a video frame brightness expansion parameter according to the maximum brightness value;

[0127] Step B2: Select the larger one from the screen brightness expansion parameter and the video frame brightness expansion parameter as the brightness expansion parameter EN;

[0128] Step B3: Determine an upper limit value of the expansion of the target color intensity value in the mapping curve according to the brightness expansion parameter.

[0129] In step B1, the video frame brightness expansion parameter may be a brightness expansion parameter of the video frame dimension, which can reflect the enhancement requirement of the video frame content for the highlight area.

[0130] In practice, the electro-optical conversion function can be used to convert the maximum brightness value to the maximum brightness value in the optical signal space. The inverse of the maximum brightness value in the optical signal space is then taken. Since the target color intensity value in the mapping curve is typically capped at 1, the inverse result reflects the need for highlight enhancement within the video frame.

[0131] The inverse result can be used as the video frame brightness expansion parameter. Alternatively, the inverse result can be compared with the video frame brightness expansion value, and the smaller of the two values ​​can be selected as the video frame brightness expansion parameter. The video frame brightness expansion value can be determined by the user, for example, it can be a value greater than 1, such as 1.5.

[0132] In step B2, a larger one may be selected from the screen brightness expansion parameter EdrN and the video frame brightness expansion parameter as the brightness expansion parameter EN.

[0133] Formula (8) shows the calculation process of the brightness extension parameter EN.

[0134]

[0135] In step B3, the brightness expansion parameter can be used as the upper limit of the expansion of the target color intensity value in the mapping curve. Alternatively, the brightness expansion parameter can be adjusted, and the adjustment result can be used as the upper limit of the expansion of the target color intensity value. For example, the adjustment can be achieved by performing a mathematical operation on the brightness expansion parameter, such as multiplying or dividing it by a set coefficient.

[0136] In embodiments of the present invention, the extended upper limit value represents the upper limit to which the brightness of a video frame can be extended on a display device. Because the extended upper limit value is determined based on display parameters and metadata, different display parameters may correspond to different extended upper limits, and different metadata may also correspond to different extended upper limits.

[0137] In the embodiment of the present invention, the mapping curve is used to represent the mapping relationship between the source color intensity value and the target color intensity value in the electrical signal space, so the mapping curve can play a role in color correction of the video frame.

[0138] The extended upper limit value is used to determine the upper limit value of the target color intensity value in the mapping curve. In the process of color correction of video frames based on the mapping curve, the extended upper limit value can constrain the target color intensity value of the corrected video frame and optimize the color space distribution of the corrected video frame. In addition, the extended upper limit value can adapt to the characteristics of different display devices and prevent the color of the corrected video frame from being over-enhanced and causing distortion. In summary, the extended upper limit value can make the picture color of the corrected video frame adapt to the display device and the video frame content, thereby improving the overall visual effect of the video frame.

[0139] The electrical signal space refers to the mathematical space in video technology that represents pixel brightness and color information using digital or analog electrical signal values. The optical signal space refers to the physical space in video technology that represents visual effects using light intensity and color distribution that the human eye can perceive.

[0140] In particular, the upper limit of the target color intensity value expansion in the mapping curve in the embodiment of the present invention is determined based on display parameters and metadata. This upper limit ensures that the target color intensity value does not exceed the capabilities of the display device, while fully utilizing the display device's brightness potential, and rationally expanding the optical signal brightness of the video frame based on the actual requirements of the video content. Therefore, the mapping curve corresponding to the video frame in the embodiment of the present invention can achieve richer brightness levels and clearer details in the video frame, thereby effectively improving the display quality of the video frame.

[0141] In a specific implementation, the mapping curve specifically includes: mapping curves corresponding to at least two brightness intervals; at a boundary point between two adjacent brightness intervals, the mapping curves corresponding to the two adjacent brightness intervals are smoothed.

[0142] A brightness interval may refer to a continuous range of brightness values ​​divided into segments, and is used to describe or process a set of pixels at a specific brightness level.

[0143] The embodiment of the present invention determines mapping curves for at least two brightness intervals respectively, and can use different mapping curves to perform differentiated conversions on the color intensity values ​​of the brightness ranges corresponding to different brightness intervals. In addition, the mapping curves are smoothed at the boundary point between two adjacent intervals, which can reduce the color mutation during the transition of the brightness intervals, thereby achieving fine-grained adjustment of the brightness and color of the video picture, and further making the display effect of the video frame reach a uniform and natural state.

[0144] In one implementation, the display parameters specifically include: maximum screen brightness; the metadata specifically includes: highlight feature values ​​and average brightness values; the at least two brightness intervals specifically include: a first brightness interval and a second adjacent brightness interval; the lower limit of the first brightness interval is 0. The first brightness interval may correspond to a dark region of a color space. When the number of brightness intervals is two, the second brightness interval corresponds to a bright region of the color space. When the number of brightness intervals is greater than two, the second brightness interval corresponds to a mid-gray region of the color space.

[0145] The process of determining the mapping curve corresponding to the video frame according to the display parameters and the metadata specifically includes:

[0146] Step C1: Determine a first parameter of a second setting curve function corresponding to a second brightness range according to the maximum brightness, average brightness value, and bright portion characteristic value of the screen.

[0147] The second setting curve function corresponding to the second brightness range can be determined by those skilled in the art according to actual application requirements. For example, the second setting curve function can be a curve provided by CUVA (China Ultra HD Video Alliance).

[0148] Formula (9) shows the form of the second setting curve function:

[0149] G(L)=((mp*L) / ((mp-1)*L+1)) 2.4 (9)

[0150] Where L represents the source color intensity value in the electrical signal space, which can be determined based on the normalized pixel value of the RGB (red, green, and blue) electrical signal corresponding to the video frame. Assuming the pixel value of the RGB electrical signal is p, then L = p / 255. G(L) represents the second set curve function, the function result of which represents the target color intensity value in the electrical signal space. mp represents the first parameter of the second set curve function.

[0151] When the number of brightness intervals is 2, the first parameter mp of the second setting curve function corresponding to the second brightness interval can be determined according to the maximum brightness, average brightness value and bright part characteristic value of the screen.

[0152] If the number of brightness intervals is greater than two, the brightness intervals may further include a third brightness interval. A second dividing point may be first determined based on the brightness expansion parameter and the highlight feature value. Then, the first parameter mp of the second setting curve function corresponding to the second brightness interval may be determined based on the maximum brightness, average brightness, highlight feature value, and the second dividing point. The second dividing point P2 is the dividing point between the second and third brightness intervals.

[0153] Formula (10) shows the process of determining the second dividing point according to the brightness extension parameter EN and the bright part characteristic value LightE.

[0154]

[0155] The process of determining the first parameter mp of the second setting curve function corresponding to the second brightness range according to the maximum brightness, average brightness value, bright part characteristic value and second dividing point of the screen specifically includes the following steps:

[0156] Step D1: Determine the upper limit mpMax of the first parameter mp according to the maximum brightness MaxDisplay of the screen. For details, see formula (11):

[0157]

[0158] Step D2: Determine the first parameter mp of the second setting curve function corresponding to the second brightness interval according to the bright feature value LightE and the second dividing point P2. For details, see formula (12):

[0159]

[0160] Step D3: Compensate the first parameter mp according to the average brightness value Averg0. For details, see formula (13):

[0161]

[0162] In an optional implementation of the present invention, the metadata may further include: a dark feature value and a dark compensation coefficient; the first brightness range and the second brightness range correspond to a first dividing point; and the process of determining the mapping curve corresponding to the video frame based on the display parameters and the metadata may further include:

[0163] Step C2: determining a lower limit value of a slope of a first setting curve function corresponding to the first brightness range according to the dark portion characteristic value and the dark portion compensation coefficient;

[0164] Step C3, determining the slope of the first set curve function corresponding to the first brightness range at the first dividing point according to the slope lower limit value, the maximum brightness of the screen, and the average brightness value;

[0165] Step C4, determining a first dividing point according to the slope of the first setting curve function corresponding to the first brightness range at the first dividing point and the second setting curve function corresponding to the second brightness range;

[0166] Step C5: Determine the parameters of the first setting curve function corresponding to the first brightness interval according to the slope of the first setting curve function corresponding to the first brightness interval at the first dividing point, the slope of the second setting curve function corresponding to the second brightness interval at the first dividing point, and the first dividing point.

[0167] Those skilled in the art may determine the first setting curve function corresponding to the first brightness range according to actual application requirements. The embodiment of the present invention does not limit the specific function form.

[0168] For example, formula (14) shows an example of the functional form of the first setting curve function corresponding to the first brightness interval.

[0169]

[0170] Among them, D(L) represents the first setting curve function corresponding to the first brightness interval, and its function result represents the target color intensity value in the electrical signal space; L represents the source color intensity value in the electrical signal space; P1 represents the first dividing point between the first brightness interval and the second brightness interval; S1 and S2 represent the second parameter and third parameter of the first setting curve function, respectively.

[0171] In step C2, the lower limit value MinSLope of the slope of the first setting curve function corresponding to the first brightness range may be determined based on the dark portion characteristic value, as shown in formula (15).

[0172]

[0173] Among them, OETF represents the optoelectronic conversion function; DarkO represents the dark eigenvalue in the optical signal space. The specific calculation process is as follows:

[0174] DarkO=EOTF)Max(DarkE,0.18))*200(16)

[0175] Furthermore, step C2 can compensate the slope lower limit value MinSLope according to the dark portion compensation coefficient CompS. The compensation process is shown in formula (17).

[0176] MinSlope=Min(MinSlope+CompS,1.0) (17)

[0177] In step C3, the slope K1 of the first setting curve function corresponding to the first brightness interval at the first dividing point is determined according to the slope lower limit value MinSlope, the maximum screen brightness MaxDisplay and the average brightness value AvergO.

[0178] If MinSlope=1, then K1=1; otherwise, refer to formula (18) to calculate the slope K1.

[0179]

[0180] Wherein, m=1.1+MaxDisplay / 10000.

[0181] In step C4, based on the slope of the first setting curve function corresponding to the first brightness interval at the first dividing point and the second setting curve function corresponding to the second brightness interval, the equation shown in formula (19) is constructed and the equation is solved to determine the first dividing point.

[0182] K1=G(P1) / P1 (19)

[0183] In step C5, the third parameter S2 of the first setting curve function corresponding to the first brightness range can be determined according to formula (20).

[0184] S2=(K2-ad*K1)*P1 (20)

[0185] The determination process of ad is shown in formula (21):

[0186] ad=1-Max(1.5-EN,0)*Min(AvergO / 0.05,1) (21)

[0187] K2 represents the slope of the second setting curve function corresponding to the second brightness range at the first dividing point. The determination process is shown in formula (22):

[0188]

[0189] In step C5, the second parameter S1 of the first setting curve function corresponding to the first brightness interval can be determined according to formula (23).

[0190] S1=K1*ad (23)

[0191] In an optional implementation of the present invention, the process of determining the mapping curve corresponding to the video frame based on the display parameters and the metadata may further include:

[0192] Step C6: Compensate the second setting curve function corresponding to the second brightness interval according to the slope K1 of the first setting curve function corresponding to the first brightness interval at the first dividing point, the slope K2 of the second setting curve function corresponding to the second brightness interval at the first dividing point, and the first dividing point P1, so as to achieve smoothing of the first setting curve function corresponding to the first brightness interval and the second setting curve function corresponding to the second brightness interval at the first dividing point.

[0193] Specifically, according to formula (24), the compensation value mb can be added on the basis of the second setting curve function corresponding to formula (9).

[0194] GN(L)=G(L)+mb (24)

[0195] Wherein, GN(L) represents the second setting curve function after compensation; the calculation process of the compensation value mb is as follows:

[0196] mb=P1*(0.5*K2-K1+0.5*ad*K1) (25)

[0197] As shown in formula (26), based on the above process of determining the mapping curve corresponding to the above video frame, the mapping curve determined by the embodiment of the present invention specifically includes: a first setting curve function corresponding to the first brightness interval and a second setting curve function corresponding to the second brightness interval, and the dividing point between the first brightness interval and the second brightness interval is the first dividing point P1.

[0198]

[0199] In an optional implementation of the present invention, the process of determining the mapping curve corresponding to the video frame based on the display parameters and the metadata may further include:

[0200] Step C7: Perform brightness compensation on the first set curve function in the first brightness interval and the second set curve function in the second brightness interval according to the brightness expansion parameter. The specific brightness compensation process is as follows:

[0201]

[0202] In an optional implementation of the present invention, the at least two brightness intervals may further include a third brightness interval adjacent to the second brightness interval. The process of determining the mapping curve corresponding to the video frame based on the display parameters and the metadata may further include:

[0203] Step C8. Determine the parameters of the function corresponding to the third brightness interval based on the boundary conditions; wherein the boundary conditions specifically include: the function value of the function corresponding to the third brightness interval at the second boundary point is equal to the function value of the second set curve function corresponding to the second brightness interval at the second dividing point; the derivative of the function corresponding to the third brightness interval at the second boundary point is equal to the derivative of the second set curve function corresponding to the second brightness interval at the second dividing point; and the function value of the function corresponding to the third brightness interval at the upper limit value of the source color intensity value is equal to the extended upper limit value.

[0204] It should be noted that the embodiment of the present invention does not limit the function corresponding to the third brightness range. For example, the function corresponding to the third brightness range can be an exponential function or a power function.

[0205] The boundary condition of the embodiment of the present invention can not only realize smooth continuity of the function corresponding to the third brightness range at the second dividing point, but also enable the function value of the function corresponding to the third brightness range to be expanded based on the expansion upper limit value.

[0206] The boundary condition corresponding to formula (28) is specifically: the function value of the function corresponding to the third brightness interval at the second boundary point is equal to the function value of the second setting curve function corresponding to the second brightness interval at the second dividing point. Wherein, E(L) represents the function corresponding to the third brightness interval, and DE(L) represents the derivative form of the function corresponding to the third brightness interval.

[0207]

[0208] The boundary condition corresponding to formula (29) is specifically: the derivative of the function corresponding to the third brightness range at the second boundary point is equal to the derivative of the second setting curve function corresponding to the second brightness range at the second dividing point. DE(P2) represents the derivative of the function corresponding to the third brightness range at the second boundary point.

[0209]

[0210] The boundary condition corresponding to formula (30) is specifically: the function value of the third brightness interval corresponding function at the upper limit of the source color intensity value is equal to the extended upper limit value. Wherein, the upper limit of the source color intensity value is specifically 1.

[0211] E(1)=EN (30)

[0212] As shown in formula (31), based on the above process of determining the mapping curve corresponding to the above video frame, the mapping curve determined by the embodiment of the present invention specifically includes: a first setting curve function of the first brightness interval, a second setting curve function of the second brightness interval, and a corresponding function of the third brightness interval.

[0213]

[0214] Reference Figure 2 , which shows a schematic diagram of a mapping curve according to an embodiment of the present invention, wherein the horizontal axis represents the source color intensity value in the electrical signal space, and its value range is [0, 1]. The vertical axis represents the target color intensity value in the electrical signal space, and its value range is [0, 4].

[0215] In the first luminance range 201 corresponding to [0, P1], the mapping curve exhibits a near-linear growth, effectively preserving details in the dark areas of the color space. In the second luminance range 202 corresponding to [P1, P2], the mapping curve transitions to a relatively gentle nonlinear change. In the third luminance range 203 corresponding to [P2, 1], the mapping curve accelerates upward, effectively expanding the luminance values ​​in the third luminance range and enhancing the layering and detail in the bright areas of the image. Figure 2 In the mapping curve, the upper limit of the target color intensity value is specifically 4. This upper limit can make the brightness levels of the video frame richer and the details clearer, thereby effectively improving the display effect of the video frame.

[0216] It should be noted that the SDR video includes multiple video frames. The embodiment of the present invention can determine a corresponding mapping curve for each video frame in the SDR video.

[0217] The mapping curve of the embodiment of the present invention can be used for display processing of video frames to effectively improve the display effect of the video frames. It is understood that the embodiment of the present invention does not limit the specific display processing process corresponding to the mapping curve.

[0218] In one implementation, the display processing of the video frame specifically includes: obtaining a YUV signal of a video frame in a standard dynamic range video; converting the YUV signal into a first RGB electrical signal; converting the first RGB electrical signal into a second RGB electrical signal according to a mapping curve; and displaying the RGB optical signal converted from the second RGB electrical signal on a display device.

[0219] In another implementation, the display processing of the video frame specifically includes: obtaining a YUV signal of a video frame in a standard dynamic range video; converting the YUV signal into an RGB electrical signal; determining a source color intensity value of the video frame at a pixel point based on the RGB electrical signal; mapping the source color intensity value using a mapping curve corresponding to the video frame to obtain a target color intensity value; and performing color correction on an RGB light signal converted from the RGB electrical signal based on the target color intensity value, so as to display the color-corrected RGB light signal on a display device.

[0220] In summary, the video data processing method of the embodiment of the present invention determines a mapping curve corresponding to a video frame based on the display parameters of the display device and the metadata of the video frame, and the extended upper limit value of the target color intensity value in the mapping curve is determined based on the display parameters and the metadata.

[0221] Since the metadata of the video frame can reflect the enhancement requirements of the highlight area of ​​the video frame content, the display parameters of the display device limit the brightness expansion capability supported by the display device. Determining the upper limit of the expansion of the mapping curve based on these two factors can not only ensure that the target color intensity value does not exceed the capability range of the display device and prevent display anomalies due to hardware limitations, but also make full use of the brightness potential of the display device and reasonably expand the brightness of the light signal of the video frame according to the actual needs of the video content. Therefore, the mapping curve corresponding to the video frame in the embodiment of the present invention can improve the brightness level of the video frame based on the expansion of the brightness range of the light signal, such as making the dark details more clearly visible and the highlight parts not overexposed, thereby effectively improving the display effect of the video frame.

[0222] Furthermore, this embodiment of the present invention takes into account the metadata of the video frame and the display parameters of the display device when determining the mapping curve. This allows video frames containing different color intensities to be adapted to display devices with different display parameters. In other words, this embodiment of the present invention enables relatively accurate color intensity mapping of video frames on display devices with different display parameters. Therefore, this embodiment of the present invention can fully leverage the performance advantages of the display device, thereby improving the display quality of the video frames.

[0223] Reference Figure 3 , which shows a schematic flow chart of the steps of a video data processing method according to an embodiment of the present invention. The method may specifically include the following steps:

[0224] Step 301: Acquire a YUV signal of a video frame in a standard dynamic range video;

[0225] Step 302: convert the YUV signal into an RGB electrical signal;

[0226] Step 303: Determine the source color intensity value of the video frame at the pixel point based on the RGB electrical signal;

[0227] Step 304: Map the source color intensity values ​​using the mapping curve corresponding to the video frame to obtain a target color intensity value; the mapping curve is used to represent the mapping relationship between the source color intensity values ​​and the target color intensity values ​​in the electrical signal space; wherein the extended upper limit value of the target color intensity value in the mapping curve is determined based on the display parameters of the display device and the metadata of the video frame;

[0228] Step 305 : Perform color correction on the RGB light signal converted from the RGB electrical signal according to the target color intensity value, so as to display the color-corrected RGB light signal on a display device.

[0229] An embodiment of the present invention determines a source color intensity value at a pixel point of a video frame, maps the source color intensity value using a mapping curve corresponding to the video frame, obtains a target color intensity value, and performs color correction on an RGB optical signal converted from the RGB electrical signal based on the target color intensity value.

[0230] Because the upper limit of the target color intensity value in the mapping curve is determined based on display parameters and metadata, the target color intensity value can map the brightness-limited light signal in SDR video to a wider brightness range supported by both the video frame content and the display device. Therefore, embodiments of the present invention can enhance the brightness levels of video frames based on the expansion of the brightness range of the light signal, for example, making dark details more visible and preventing overexposure of highlights, thereby effectively improving the display quality of the video frames.

[0231] In step 301, if SDR video transmission is involved, the decoder can obtain the YUV signal of the video frame in the standard dynamic range video from the bitstream sent by the encoder. If SDR video transmission is not involved, the YUV signal of the video frame in the standard dynamic range video can be directly obtained.

[0232] In step 302, the YUV signal may be converted into an RGB electrical signal according to the ITU-BT.709 standard. Formula (32) shows an example of converting the YUV signal into an RGB electrical signal.

[0233]

[0234] In step 303 , the source color intensity value represents the color intensity of the video frame at the pixel point.

[0235] In one implementation, the source color intensity value at a pixel point in the video frame can be selected from the pixel values ​​of the three channels of the RGB electrical signal. The three channels specifically include a red channel, a green channel, and a blue channel. For example, the maximum value among the pixel values ​​of the three channels of the RGB electrical signal can be selected as the source color intensity value at the pixel point in the video frame.

[0236] In another implementation, the source color intensity value at the pixel point of the video frame can be determined based on the RGB electrical signal and the brightness component of the YUV signal. In determining the source color intensity value at the pixel point of the video frame, the RGB electrical signal and the brightness component of the YUV signal are comprehensively considered, so that the source color intensity value conforms to the characteristics of the SDR image.

[0237] In a specific implementation, the process of determining the source color intensity value of a pixel in a video frame based on the RGB electrical signal and the brightness component of the YUV signal specifically includes:

[0238] Step E1, selecting the RGB feature value of the pixel point from the pixel values ​​of the RGB electrical signal in the three channels;

[0239] Step E2: Fusing the RGB feature value and the brightness component of the pixel point to obtain the source color intensity value of the video frame at the pixel point.

[0240] In step E1 , the maximum value may be selected from the pixel values ​​of the three channels of the RGB electrical signal as the RGB feature value of the pixel point.

[0241] In step E2, the fusion method of the RGB feature value and the brightness component specifically includes: a weighted average method, etc. It can be understood that the embodiment of the present invention does not limit the specific fusion method.

[0242] Formula (33) shows an example of determining the source color intensity value RGBF at a pixel point in a video frame. This example adds the maximum value of the RGB electrical signal among the pixel values ​​of the three channels to the luminance component and divides the result by 2 to obtain the source color intensity value RGBF. Where Y represents the luminance component.

[0243] RGBF=(max(R,G,B)+Y) / 2 (33)

[0244] In step 304, formula (33) shows the process of mapping the above-mentioned source color intensity value, wherein tmk represents the mapped target color intensity value.

[0245] tmk=TM(RGBF) (34)

[0246] In step 305, the process of performing color correction on the RGB optical signal converted from the RGB electrical signal according to the target color intensity value specifically includes:

[0247] Step F1: When the target color intensity value is not greater than 1, a first adjustment is performed on the source pixel value of the RGB light signal at the pixel point according to the target color intensity value, and the obtained first adjustment result is used as the target pixel value of the RGB light signal at the pixel point after color correction; or

[0248] Step F2: When the target color intensity value is greater than 1, a first adjustment is performed on the source pixel value of the RGB light signal at the pixel point according to the target color intensity value to obtain a first adjustment result; and a second adjustment is performed on the first adjustment result according to the adjustment coefficient and the brightness value of the RGB light signal at the pixel point. The obtained second adjustment result is used as the target pixel value of the RGB light signal at the pixel point after color correction.

[0249] Formula (35) shows the process of performing a first adjustment on the source pixel value of the RGB light signal at the pixel point. Here, EOTF(R) represents the source pixel value of the RGB light signal in the red channel, EOTF(G) represents the source pixel value of the RGB light signal in the green channel, and EOTF(B) represents the source pixel value of the RGB light signal in the blue channel. Rt represents the first adjustment result of the RGB light signal in the red channel, Gt represents the first adjustment result of the RGB light signal in the green channel, and Bt represents the first adjustment result of the RGB light signal in the blue channel. The first adjustment process can first calculate the ratio of the target color intensity value to the source color intensity value, and then multiply the ratio by the source pixel value of the RGB light signal at the pixel point. The product obtained can be used as the first adjustment result.

[0250]

[0251] Formula (36) shows the process of performing the second adjustment on the first adjustment result.

[0252]

[0253] Luma represents the brightness of the RGB light signal at the pixel point. Luma = 0.2126*Rt + 0.7152*Gt + 0.0722*Bt, where 0.2126, 0.7152, and 0.0722 represent the preset weight coefficients for the red, green, and blue channels, respectively. The preset weight coefficients for the red, green, and blue channels can be flexibly set based on historical experience or experimental data.

[0254] adjustS represents the adjustment coefficient, which is calculated as follows:

[0255]

[0256] Among them, max(R, G, B) is the maximum value of the pixel values ​​of the three channels R, G, and B in the RGB electrical signal, and min(R, G, B) is the minimum value of the pixel values ​​of the three channels R, G, and B.

[0257] max(Rt, Gt, Bt) is the maximum value of the pixel values ​​of the three channels R, G, and B in the RGB light signal, and min(Rt, Gt, Bt) is the minimum value of the pixel values ​​of the three channels R, G, and B.

[0258] When the target color intensity value exceeds 1 (i.e., exceeds the standard dynamic range), an embodiment of the present invention first proportionally amplifies the source pixel value of the RGB light signal according to the target color intensity value, so that the brightness of the highlight area is preliminarily expanded. On this basis, a second adjustment is made to the first adjustment result based on the adjustment coefficient and the brightness value of the RGB light signal at the pixel point. The second adjustment is used to avoid detail loss and color oversaturation caused by excessive stretching of the highlight area. The above-mentioned staged adjustment method can not only break through the brightness limit of SDR to make the highlight brighter, but also maintain color accuracy, and ultimately achieve a balanced optimization of highlight details and overall contrast.

[0259] The video data processing method according to the embodiment of the present invention can be applied to end-to-end application scenarios.

[0260] Reference Figure 4 , which shows a schematic flow diagram of a video data processing method in an end-to-end scenario according to an embodiment of the present invention. In this diagram, the encoder obtains metadata of the video frame based on the YUV signal of the SDR video, encodes the YUV signal and metadata of the SDR video, and generates a bitstream. The metadata can be dynamic metadata that changes as the video frame changes.

[0261] The code stream may be a bit stream, which is a binary data stream generated by encoding and compressing video data.

[0262] A bitstream may include a bit sequence representing an encoded representation of video data. A bitstream may include encoded video frames and associated data. An encoded video frame is an encoded representation of a video frame. The associated data may include parameter sets for the bit sequence, parameter sets for the video frame, and the like. The parameter sets for the video frame may include metadata for the video frame.

[0263] After receiving the bitstream from the encoder, the decoder decodes it to obtain the YUV signal and metadata for the SDR video. Based on the metadata and the display parameters of the display device, the decoder determines the mapping curve for the video frame. Based on the mapping curve and the SDR video's YUV signal, the decoder determines the RGB optical signal suitable for displaying the SDR video on the display device.

[0264] Reference Figure 5 , which shows a schematic flow chart of the steps of a video data processing method according to an embodiment of the present invention. The method is applied to an encoding end and may specifically include the following steps:

[0265] Step 501: Obtain metadata of video frames in a standard dynamic range video;

[0266] Step 502: Encode the metadata and the standard dynamic range video to obtain a bitstream;

[0267] Step 503: Send the code stream to a decoder so that the decoder determines a mapping curve corresponding to the video frame based on display parameters of the display device and the metadata. The mapping curve is used to represent a mapping relationship between source color intensity values ​​and target color intensity values ​​in an electrical signal space. An extended upper limit value of the target color intensity value in the mapping curve is determined based on the display parameters and the metadata.

[0268] The encoding end in the embodiment of the present invention may be a terminal device with an image acquisition device, including but not limited to: a mobile phone, a tablet computer, a laptop computer, a calculator, a phone watch, etc.

[0269] The applicable scenarios of the embodiments of the present invention include but are not limited to:

[0270] 1. The SDR video at the encoding end is mapped to the CRT (Cathode Ray Tube) display at the decoding end for display;

[0271] 2. The SDR video at the encoding end is mapped to the LCD (Liquid Crystal Display) display at the decoding end for display;

[0272] 3. The SDR video on the encoding side is mapped to the OLED (Organic Light-Emitting Diode) display on the decoding side for display;

[0273] 4. The SDR video on the encoding side is mapped to the minLED (Mini Light Emitting Diode) display on the decoding side for display.

[0274] The technical solution of the encoding end provided by the embodiment of the present invention, combined with the technical solution of the decoding end, can significantly improve the color distortion problem of the video picture when displaying SDR video on a high-brightness screen (such as a display screen with a maximum display brightness of more than 300nit).

[0275] In a specific implementation, the metadata specifically includes at least one of the following brightness features: a bright part feature value, a dark part feature value, a maximum brightness value, an average brightness value, and a dark part compensation coefficient.

[0276] For the process of obtaining metadata of video frames in standard dynamic range video, you can refer to Figure 1 The relevant descriptions in the illustrated method embodiment are omitted here for brevity.

[0277] The encoder can convert the metadata into a 16-bit integer and write it into the SEI (Supplemental Enhancement Information) part of the bitstream.

[0278] In an example of a bitstream in an embodiment of the present invention, the first codeword SPS (Sequence Parameter Set) retains the global parameters of the video frame after encoding in the SDR video; the second codeword PPS (Picture Parameter Set) retains the parameters related to the entire video frame. PPS is usually saved in the file header of the video file together with SPS; the third codeword IDR (Immediate Decoding Refresh) is used to save the decoding parameter set; the fourth codeword SLICE (Slice) is used to save frame encoding information; the fifth codeword SEI is used to save supplementary information of the video bitstream, that is, the metadata mentioned in this embodiment. The encoder writes the metadata into the fifth codeword SEI, and then encodes it into a bitstream together with the data in other codewords.

[0279] Reference Figure 6 , which shows a schematic flow chart of the steps of a video data processing method according to an embodiment of the present invention. The method is applied to a decoding end and may specifically include the following steps:

[0280] Step 601: Obtain metadata of video frames from a standard dynamic range video bitstream sent by an encoder.

[0281] Step 602: Obtain display parameters of the display device;

[0282] Step 603: Determine a mapping curve corresponding to the video frame based on the display parameters and the metadata. The mapping curve is used to characterize the mapping relationship between the source color intensity value and the target color intensity value in the electrical signal space. The extended upper limit value of the target color intensity value in the mapping curve is determined based on the display parameters and the metadata.

[0283] The technical solution provided in this embodiment is applicable to a decoding terminal, which may specifically include a computer terminal (including a server) and / or a mobile terminal (including but not limited to a smartphone, tablet computer, VR (Virtual Reality) helmet, VR glasses, etc.).

[0284] As can be seen from the above technical solution of the encoder, since the code stream sent from the encoder to the decoder includes video frames and metadata of the video frames, the decoder can decode the video frames and metadata of the video frames from the code stream.

[0285] For the process of determining the mapping curve corresponding to the video frame according to the display parameters and the metadata in step 603, reference may be made to Figure 1 The relevant descriptions in the illustrated method embodiment are omitted here for brevity.

[0286] The decoding end of the embodiment of the present invention can also perform display processing of video frames.

[0287] In one implementation, the display processing of the video frame specifically includes: obtaining a YUV signal of a video frame in a standard dynamic range video; converting the YUV signal into a first RGB electrical signal; converting the first RGB electrical signal into a second RGB electrical signal according to a mapping curve; and displaying the RGB optical signal converted from the second RGB electrical signal on a display device.

[0288] In another implementation, the display processing of the video frame specifically includes: obtaining the YUV signal of the video frame in the standard dynamic range video; converting the YUV signal into an RGB electrical signal; determining the source color intensity value of the video frame at the pixel point according to the RGB electrical signal; mapping the source color intensity value using the mapping curve corresponding to the video frame to obtain the target color intensity value; performing color correction on the RGB light signal converted from the RGB electrical signal according to the target color intensity value, so as to display the color-corrected RGB light signal on the display device. For details, please refer to Figure 3 The relevant description of the embodiment of the method shown is not repeated here.

[0289] It should be noted that, for the sake of simplicity, the aforementioned method embodiments are described as a series of actions. However, those skilled in the art should be aware that the present invention is not limited by the order of the actions described, as certain steps may be performed in other orders or simultaneously according to the present invention. Furthermore, those skilled in the art should also be aware that the embodiments described in this specification are preferred embodiments, and the actions involved are not necessarily required by the present invention.

[0290] Based on the description of the above method embodiment, the present invention also provides a corresponding video data processing device embodiment to implement the content described in the above method embodiment.

[0291] Reference Figure 7 , which shows a schematic structural diagram of a video data processing device according to an embodiment of the present invention, wherein the device specifically includes the following modules:

[0292] The metadata acquisition module 701 is used to acquire metadata of video frames in a standard dynamic range video;

[0293] A display parameter acquisition module 702 is used to acquire display parameters of a display device;

[0294] A mapping curve determination module 703 is configured to determine a mapping curve corresponding to the video frame based on the display parameters and the metadata; the mapping curve is configured to characterize a mapping relationship between source color intensity values ​​and target color intensity values ​​within an electrical signal space; wherein an extended upper limit value of the target color intensity value in the mapping curve is determined based on the display parameters and the metadata.

[0295] Optionally, the display parameter includes a screen brightness extension parameter, and the metadata includes a maximum brightness value; the device may further include:

[0296] a video frame brightness expansion parameter determination module, configured to determine the video frame brightness expansion parameter according to the maximum brightness value;

[0297] a brightness expansion parameter determination module, configured to select a larger one from a screen brightness expansion parameter and a video frame brightness expansion parameter as the brightness expansion parameter;

[0298] The expansion upper limit value determining module is used to determine the expansion upper limit value of the target color intensity value in the mapping curve according to the brightness expansion parameter.

[0299] Optionally, the mapping curve includes: mapping curves corresponding to at least two brightness intervals; and at a boundary point between two adjacent brightness intervals, the mapping curves corresponding to the two adjacent brightness intervals are smoothed.

[0300] Optionally, the display parameter includes: maximum screen brightness; the metadata includes: a bright feature value and an average brightness value; the at least two brightness intervals include: an adjacent first brightness interval and a second brightness interval; the lower limit value of the first brightness interval is 0;

[0301] The mapping curve determination module 703 includes:

[0302] The first parameter determination module is used to determine the first parameter of the second setting curve function corresponding to the second brightness range according to the maximum brightness, average brightness value and bright part characteristic value of the screen.

[0303] Optionally, the metadata further includes: a dark feature value and a dark compensation coefficient; the first brightness range and the second brightness range correspond to a first dividing point; the mapping curve determination module 703 further includes:

[0304] a slope lower limit value determining module, configured to determine a slope lower limit value of a first setting curve function corresponding to the first brightness range according to a dark portion characteristic value and a dark portion compensation coefficient;

[0305] a slope determination module, configured to determine the slope of a first set curve function corresponding to a first brightness interval at a first dividing point according to the slope lower limit value, the maximum brightness of the screen, and the average brightness value;

[0306] a first dividing point determining module, configured to determine the first dividing point according to the slope of the first setting curve function corresponding to the first brightness interval at the first dividing point and the second setting curve function corresponding to the second brightness interval;

[0307] The second parameter determination module is used to determine the parameters of the first setting curve function corresponding to the first brightness interval based on the slope of the first setting curve function corresponding to the first brightness interval at the first dividing point, the slope of the second setting curve function corresponding to the second brightness interval at the first dividing point, and the first dividing point.

[0308] Optionally, the mapping curve determination module 703 further includes:

[0309] The first compensation module is used to compensate the second set curve function corresponding to the second brightness interval according to the slope of the first set curve function corresponding to the first brightness interval at the first dividing point, the slope of the second set curve function corresponding to the second brightness interval at the first dividing point, and the first dividing point, so as to achieve smoothing processing of the first set curve function corresponding to the first brightness interval and the second set curve function corresponding to the second brightness interval at the first dividing point.

[0310] Optionally, the mapping curve determination module 703 further includes:

[0311] The second compensation module is configured to perform brightness compensation on the first setting curve function corresponding to the first brightness interval and the second setting curve function corresponding to the second brightness interval according to the brightness extension parameter.

[0312] Optionally, the at least two brightness intervals further include: a third brightness interval adjacent to the second brightness interval, and the mapping curve determining module 703 further includes:

[0313] A third parameter determination module, configured to determine parameters of a function corresponding to a third brightness range according to boundary conditions;

[0314] The boundary conditions include: the function value of the third brightness interval corresponding function at the second boundary point is equal to the function value of the second setting curve function corresponding to the second brightness interval at the second dividing point; the derivative of the third brightness interval corresponding function at the second boundary point is equal to the derivative of the second setting curve function corresponding to the second brightness interval at the second dividing point; and the function value of the third brightness interval corresponding function at the upper limit value of the source color intensity value is equal to the extended upper limit value.

[0315] Optionally, the metadata includes at least one of the following brightness features: a bright part feature value, a dark part feature value, a maximum brightness value, an average brightness value, and a dark part compensation coefficient.

[0316] Optionally, the metadata acquisition module 701 includes:

[0317] a histogram processing module, configured to determine a histogram distribution of a luminance component of a video frame and determine a mid-gray value of the histogram distribution;

[0318] a bright feature value determination module, configured to determine a bright feature value based on the sum of the brightness of pixels having brightness values ​​greater than the mid-gray value and the number of pixels in the histogram distribution;

[0319] a dark portion feature value determination module, configured to determine a dark portion feature value based on the sum of the brightness of pixels whose brightness values ​​are not greater than the middle gray value in the histogram distribution, the number of pixels, and the middle gray value;

[0320] an average brightness value determination module, configured to determine an average brightness value based on the sum of the products of the brightness value of the light signal of the video frame and the number of pixels corresponding to the brightness value, and the total number of pixels in the video frame;

[0321] The metadata determination module is configured to use the middle gray value, the bright part feature value, the dark part feature value, and the average brightness value as the metadata.

[0322] Optionally, the histogram processing module includes:

[0323] a first middle gray value determination module, configured to determine an initial value of the middle gray value according to the brightness value of the first brightness percentile and the brightness value of the second brightness percentile in the histogram distribution;

[0324] a division module, configured to divide the brightness interval into a fourth brightness interval and a fifth brightness interval according to an initial value of the middle gray value;

[0325] a second middle gray value determination module, configured to determine a current value of the middle gray value based on a first brightness average value of the histogram distribution of the fourth brightness interval in the electrical signal space and a second brightness average value of the histogram distribution of the fifth brightness interval in the optical signal space;

[0326] an updating module, configured to update the fourth brightness interval and the fifth brightness interval according to the current value of the middle gray value if the current value of the middle gray value does not meet the set condition;

[0327] An iterative execution module is used to determine the current value of the middle gray value based on the updated fourth brightness interval and fifth brightness interval, the first brightness average value of the histogram distribution of the fourth brightness interval in the electrical signal space, and the second brightness average value of the histogram distribution of the fifth brightness interval in the optical signal space.

[0328] Reference Figure 8 , which shows a schematic structural diagram of a video data processing device according to an embodiment of the present invention, wherein the device specifically includes the following modules:

[0329] The YUV acquisition module 801 is used to obtain the YUV signal of the video frame in the standard dynamic range video;

[0330] A conversion module 802 is configured to convert the YUV signal into an RGB electrical signal;

[0331] The source intensity determination module 803 is used to determine the source color intensity value of the video frame at the pixel point according to the RGB electrical signal;

[0332] a mapping module 804 configured to map the source color intensity values ​​using a mapping curve corresponding to the video frame to obtain a target color intensity value; the mapping curve being configured to represent a mapping relationship between the source color intensity values ​​and the target color intensity values ​​in an electrical signal space; wherein an extended upper limit value of the target color intensity value in the mapping curve is determined based on display parameters of a display device and metadata of the video frame;

[0333] The color correction module 805 is configured to perform color correction on the RGB optical signal converted from the RGB electrical signal according to the target color intensity value, so as to display the color-corrected RGB optical signal on a display device.

[0334] Optionally, the source strength determination module 803 includes:

[0335] The intensity calculation module is used to determine the source color intensity value of the video frame at the pixel point according to the RGB electrical signal and the brightness component of the YUV signal.

[0336] Optionally, the intensity calculation module includes:

[0337] A selection module is used to select the RGB feature value of the pixel point from the pixel values ​​of the RGB electrical signal in the three channels;

[0338] The fusion module is used to fuse the RGB feature value and the brightness component of the pixel point to obtain the source color intensity value of the video frame at the pixel point.

[0339] Optionally, the color correction module 805 includes:

[0340] A first color correction module is configured to, when a target color intensity value is not greater than 1, perform a first adjustment on a source pixel value of the RGB light signal at a pixel point according to the target color intensity value, and use the obtained first adjustment result as a target pixel value of the RGB light signal at the pixel point after color correction; or

[0341] The second color correction module is configured to, when a target color intensity value is greater than 1, perform a first adjustment on the source pixel value of the RGB light signal at the pixel point according to the target color intensity value to obtain a first adjustment result; and perform a second adjustment on the first adjustment result according to the adjustment coefficient and the brightness value of the RGB light signal at the pixel point, and the obtained second adjustment result is used as the target pixel value of the RGB light signal at the pixel point after color correction.

[0342] Reference Figure 9 , which shows a schematic structural diagram of a video data processing device according to an embodiment of the present invention. The device is applied to a decoding end and specifically includes the following modules:

[0343] The metadata acquisition module 901 is used to obtain metadata of video frames from the bit stream of the standard dynamic range video sent by the encoding end;

[0344] A display parameter acquisition module 902 is used to acquire display parameters of a display device;

[0345] A mapping curve determination module 903 is configured to determine a mapping curve corresponding to the video frame based on the display parameters and the metadata; the mapping curve is configured to characterize a mapping relationship between source color intensity values ​​and target color intensity values ​​within an electrical signal space; wherein an extended upper limit value of the target color intensity value in the mapping curve is determined based on the display parameters and the metadata.

[0346] Reference Figure 10 , which shows a schematic structural diagram of a video data processing device according to an embodiment of the present invention, wherein the device is applied to an encoding end and includes:

[0347] The metadata acquisition module 1001 is used to acquire metadata of video frames in a standard dynamic range video;

[0348] The encoding module 1002 is configured to encode the metadata and the standard dynamic range video to obtain a bit stream;

[0349] A sending module 1003 is configured to send the code stream to a decoding end, so that the decoding end determines a mapping curve corresponding to the video frame based on display parameters of a display device and the metadata; the mapping curve is used to represent a mapping relationship between source color intensity values ​​and target color intensity values ​​in an electrical signal space; wherein an extended upper limit value of the target color intensity value in the mapping curve is determined based on the display parameters and the metadata.

[0350] As for the above-mentioned device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0351] An embodiment of the present invention discloses a device for video data processing, comprising a processor and a non-volatile memory having instructions thereon, wherein the instructions, when executed by the processor, enable the processor to perform the aforementioned method.

[0352] An embodiment of the present invention discloses a non-transitory computer-readable storage medium storing instructions, wherein the instructions enable a processor to execute the aforementioned method.

[0353] An embodiment of the present invention discloses a non-transitory computer-readable recording medium storing a bitstream generated by a method executed by an apparatus for video data processing, wherein the method comprises: obtaining metadata of video frames in a standard dynamic range video; and encoding the metadata and the standard dynamic range video to obtain a bitstream.

[0354] An embodiment of the present invention discloses a method for storing a bitstream of video data, comprising: obtaining metadata of video frames in a standard dynamic range video; encoding the metadata and the standard dynamic range video to obtain a bitstream; and storing the bitstream in a non-transitory computer-readable recording medium.

[0355] The embodiment of the present invention discloses a method for storing a bit stream, comprising: executing Figure 5 The video data processing method shown generates a bit stream; and stores the bit stream.

[0356] Non-volatile memory: Components or devices within computer hardware used for non-temporary storage of data and instructions. The emphasis on "non-volatile" storage means that data is not lost due to transient factors such as the loss of electrical signals. Devices like ROM (read-only memory) and the memory chips in solid-state drives (SSDs) can store data for a long time.

[0357] Non-transitory computer-readable recording media: Physical media that can store computer data permanently and can be read. Data storage is stable and non-temporary. These media, such as hard drives, USB flash drives, and optical disks, can store programs, documents, videos, and other types of data. Even if the device loses power or is restarted, the data remains available for subsequent access by the computer.

[0358] Non-transitory computer-readable storage media: These are part of the computer storage system and are capable of storing computer-readable data over a long period of time. These media include hard drives, solid-state drives, and flash memory. Unlike transient storage, these media retain data permanently or over a long period of time. Computers can access the stored instructions and data through appropriate interfaces and protocols. These media can be used for program execution and data processing, such as storing operating systems, application code, and user files, providing stable data support for computer systems.

[0359] The apparatus for video data processing may be an electronic device. Figure 11 A structural schematic diagram of an electronic device 1100 according to an embodiment of the present invention is shown. The electronic device 1100 specifically includes: one or more processors 1102, a control module (chip set) 1104 coupled to at least one of the (one or more) processors 1102, a memory 1106 coupled to the control module 1104, a non-volatile memory / storage device 1108 coupled to the control module 1104, one or more input / output devices 1110 coupled to the control module 1104, and a network interface 1112 coupled to the control module 1104.

[0360] The processor 1102 may include one or more single-core or multi-core processors, and the processor 1102 may include any combination of general-purpose processors or dedicated processors (e.g., graphics processors, application processors, baseband processors, etc.). In some embodiments, the electronic device 1100 can serve as a terminal device, server (cluster), or other device described in the embodiments of the present invention.

[0361] In some embodiments, the electronic device 1100 may include one or more computer-readable media (e.g., memory 1106 or non-volatile memory / storage device 1108) having instructions 1114 and one or more processors 1102 configured in combination with the one or more computer-readable media to execute the instructions 1114 to implement a module to perform the actions described in the present disclosure.

[0362] For one embodiment, the control module 1104 may include any suitable interface controller to provide any suitable interface to at least one of the processor(s) 1102 and / or any suitable device or component in communication with the control module 1104 .

[0363] The control module 1104 may include a memory controller module to provide an interface to the memory 1106. The memory controller module may be a hardware module, a software module, and / or a firmware module.

[0364] The memory 1106 can be used, for example, to load and store data and / or instructions 1114 for the electronic device 1100. For one embodiment, the memory 1106 can include any suitable volatile memory, such as a suitable DRAM (Dynamic Random Access Memory). In some embodiments, the memory 1106 can include a double data rate type four synchronous dynamic random access memory.

[0365] For one embodiment, the control module 1104 may include one or more input / output controllers to provide an interface to the non-volatile memory / storage device 1108 and the input / output device(s) 1110 .

[0366] For example, the non-volatile memory / storage device 1108 may be used to store data and / or instructions 1114. The non-volatile memory / storage device 1108 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable non-volatile storage device(s) (e.g., one or more hard disk drives, one or more optical disk drives, and / or one or more digital versatile optical disk drives).

[0367] The non-volatile memory / storage device 1108 may include storage resources that are physically part of the device on which the electronic device 1100 is installed, or it may be accessible to the device without being part of the device. For example, the non-volatile memory / storage device 1108 may be accessed via one or more input / output devices 1110 over a network.

[0368] (One or more) input / output devices 1110 may provide an interface for electronic device 1100 to communicate with any other appropriate device. Input / output devices 1110 may include communication components, audio components, sensor components, etc. Network interface 1112 may provide an interface for electronic device 1100 to communicate via one or more networks. Electronic device 1100 may wirelessly communicate with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols, for example, accessing a wireless network based on a communication standard such as WiFi (WIreless Fidelity), 2G (2-Generation wireless telephone technology, first generation wireless communication technology), 3G (3-Generation wireless telephone technology, third generation wireless communication technology), 4G (4-Generation wireless telephone technology, fourth generation wireless communication technology), 5G (5-Generation wireless telephone technology, fifth generation wireless communication technology), etc., or a combination thereof for wireless communication.

[0369] For one embodiment, at least one of the processor(s) 1102 may be packaged together with the logic of one or more controllers (e.g., a memory controller module) of the control module 1104. For one embodiment, at least one of the processor(s) 1102 may be packaged together with the logic of one or more controllers of the control module 1104 to form a system-in-package. For one embodiment, at least one of the processor(s) 1102 may be integrated on the same die with the logic of one or more controllers of the control module 1104. For one embodiment, at least one of the processor(s) 1102 may be integrated on the same die with the logic of one or more controllers of the control module 1104 to form a system-on-chip.

[0370] In various embodiments, the electronic device 1100 may be, but is not limited to, a terminal device such as a server, a desktop computing device, or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a touchscreen device, a netbook, etc.). In various embodiments, the electronic device 1100 may have more or fewer components and / or a different architecture. For example, in some embodiments, the electronic device 1100 includes one or more cameras, a keyboard, an LCD screen (including a touchscreen display), a non-volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit, and a speaker.

[0371] An embodiment of the present invention provides a machine-readable medium having instructions stored thereon, which, when executed by one or more processors, causes an electronic device to execute one or more of the methods described in the above embodiments.

[0372] Alternatively, the machine-readable medium may be a non-transitory computer-readable storage medium, for example, a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0373] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0374] Those skilled in the art will readily appreciate that any combination of the above-described embodiments is feasible, and therefore any combination of the above-described embodiments is an embodiment of the present invention. However, due to space limitations, this specification does not describe each of the embodiments in detail. Although preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they understand the basic inventive concepts. Therefore, the appended claims are intended to be interpreted as covering the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0375] The above is a detailed introduction to a video data processing method, device and medium provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the contents of this specification should not be understood as limiting the present invention.

Claims

1. A video data processing method, characterized in that: The method comprises: Get metadata for video frames in standard dynamic range video; Get the display parameters of the display device; A mapping curve corresponding to the video frame is determined based on the display parameters and the metadata; the mapping curve is used to characterize the mapping relationship between the source color intensity value and the target color intensity value in the electrical signal space; wherein the extended upper limit value of the target color intensity value in the mapping curve is determined based on the display parameters and the metadata.

2. The method according to claim 1, characterized in that The display parameters include: screen brightness extension parameters, and the metadata include: maximum brightness value; the method further includes: Determining a video frame brightness expansion parameter according to the maximum brightness value; Selecting a larger one from the screen brightness expansion parameter and the video frame brightness expansion parameter as the brightness expansion parameter; An extended upper limit value of the target color intensity value in the mapping curve is determined according to the brightness extension parameter.

3. The method according to claim 1, characterized in that The mapping curves include: mapping curves corresponding to at least two brightness intervals; and at a boundary point between two adjacent brightness intervals, the mapping curves corresponding to the two adjacent brightness intervals are smoothed.

4. The method according to claim 3, characterized in that The display parameters include: maximum screen brightness; the metadata include: bright feature value and average brightness value; the at least two brightness intervals include: adjacent first brightness interval and second brightness interval; the lower limit value of the first brightness interval is 0; The determining, according to the display parameters and the metadata, a mapping curve corresponding to the video frame includes: According to the maximum brightness, average brightness value and bright part characteristic value of the screen, a first parameter of the second setting curve function corresponding to the second brightness range is determined.

5. The method according to claim 4, characterized in that The metadata further includes: a dark feature value and a dark compensation coefficient; a first dividing point corresponding to the first brightness range and the second brightness range; and determining the mapping curve corresponding to the video frame based on the display parameters and the metadata further includes: Determining a lower limit value of a slope of a first setting curve function corresponding to the first brightness range according to a dark portion characteristic value and a dark portion compensation coefficient; Determining the slope of the first set curve function corresponding to the first brightness range at the first dividing point according to the slope lower limit value, the maximum brightness of the screen, and the average brightness value; determining a first dividing point according to a slope of a first setting curve function corresponding to the first brightness range at the first dividing point and a second setting curve function corresponding to the second brightness range; The parameters of the first setting curve function corresponding to the first brightness interval are determined according to the slope of the first setting curve function corresponding to the first brightness interval at the first dividing point, the slope of the second setting curve function corresponding to the second brightness interval at the first dividing point, and the first dividing point.

6. The method according to claim 5, characterized in that The determining, according to the display parameters and the metadata, a mapping curve corresponding to the video frame further includes: According to the slope of the first setting curve function corresponding to the first brightness interval at the first dividing point, the slope of the second setting curve function corresponding to the second brightness interval at the first dividing point, and the first dividing point, the second setting curve function corresponding to the second brightness interval is compensated to achieve smoothing processing of the first setting curve function corresponding to the first brightness interval and the second setting curve function corresponding to the second brightness interval at the first dividing point.

7. The method according to any one of claims 3 to 6, characterized in that The determining, according to the display parameters and the metadata, a mapping curve corresponding to the video frame further includes: According to the brightness extension parameter, brightness compensation is performed on a first setting curve function corresponding to the first brightness interval and a second setting curve function corresponding to the second brightness interval.

8. The method according to any one of claims 3 to 6, characterized in that The at least two brightness intervals further include: a third brightness interval adjacent to the second brightness interval, and determining the mapping curve corresponding to the video frame according to the display parameter and the metadata further includes: Determining parameters of a function corresponding to a third brightness range according to boundary conditions; The boundary conditions include: the function value of the third brightness interval corresponding function at the second boundary point is equal to the function value of the second setting curve function corresponding to the second brightness interval at the second dividing point; the derivative of the third brightness interval corresponding function at the second boundary point is equal to the derivative of the second setting curve function corresponding to the second brightness interval at the second dividing point; and the function value of the third brightness interval corresponding function at the upper limit value of the source color intensity value is equal to the extended upper limit value.

9. The method according to any one of claims 1 to 6, characterized in that The metadata includes at least one of the following brightness features: a bright portion feature value, a dark portion feature value, a maximum brightness value, an average brightness value, and a dark portion compensation coefficient.

10. The method according to any one of claims 1 to 6, characterized in that: The step of obtaining metadata of a video frame in a standard dynamic range video includes: Determining a histogram distribution of a luminance component of a video frame, and determining a mid-gray value of the histogram distribution; Determine a bright feature value according to the sum of the brightness of pixels whose brightness values ​​are greater than the middle gray value and the number of pixels in the histogram distribution; Determining a dark feature value according to the sum of the brightness of pixels whose brightness values ​​are not greater than the middle gray value in the histogram distribution, the number of pixels, and the middle gray value; Determining an average brightness value based on the sum of the product of the brightness value of the light signal of the video frame and the number of pixels corresponding to the brightness value, and the total number of pixels in the video frame; The middle gray value, the bright feature value, the dark feature value, and the average brightness value are used as the metadata.

11. The method according to claim 10, characterized in that Determining the middle gray value of the histogram distribution includes: Determining an initial value of a mid-gray value based on a brightness value of a first brightness percentile and a brightness value of a second brightness percentile in the histogram distribution; Dividing the brightness range into a fourth brightness range and a fifth brightness range according to the initial value of the middle gray value; Determining a current value of the mid-gray value based on a first brightness average value of the histogram distribution of the fourth brightness interval in the electrical signal space and a second brightness average value of the histogram distribution of the fifth brightness interval in the optical signal space; If the current value of the middle gray value does not meet the set conditions, the fourth brightness interval and the fifth brightness interval are updated according to the current value of the middle gray value, and according to the updated fourth brightness interval and the fifth brightness interval, the step of determining the current value of the middle gray value based on the first brightness average value of the histogram distribution of the fourth brightness interval in the electrical signal space and the second brightness average value of the histogram distribution of the fifth brightness interval in the optical signal space is performed.

12. A video data processing method, characterized in that: The method comprises: Get the YUV signal of the video frame in the standard dynamic range video; Converting the YUV signal into an RGB electrical signal; Determine the source color intensity value of the video frame at the pixel point based on the RGB electrical signal; The source color intensity value is mapped using a mapping curve corresponding to the video frame to obtain a target color intensity value; the mapping curve is used to represent a mapping relationship between the source color intensity value and the target color intensity value in the electrical signal space; wherein an extended upper limit value of the target color intensity value in the mapping curve is determined based on display parameters of a display device and metadata of the video frame; According to the target color intensity value, color correction is performed on the RGB optical signal converted from the RGB electrical signal, so that the color-corrected RGB optical signal is displayed on a display device.

13. The method according to claim 12, characterized in that Determining the source color intensity value of a pixel in a video frame according to the RGB electrical signal includes: The source color intensity value of the video frame at the pixel point is determined according to the RGB electrical signal and the brightness component of the YUV signal.

14. The method according to claim 13, characterized in that Determining the source color intensity value of a pixel of a video frame according to the RGB electrical signal and the brightness component of the YUV signal includes: Select the RGB feature value of the pixel point from the pixel values ​​of the RGB electrical signal in the three channels; The RGB feature value and brightness component of the pixel are fused to obtain the source color intensity value of the pixel in the video frame.

15. The method according to claim 14, characterized in that The step of performing color correction on the RGB optical signal converted from the RGB electrical signal according to the target color intensity value comprises: When the target color intensity value is not greater than 1, a first adjustment is performed on the source pixel value of the RGB light signal at the pixel point according to the target color intensity value, and the obtained first adjustment result is used as the target pixel value of the RGB light signal at the pixel point after color correction; or When the target color intensity value is greater than 1, a first adjustment is performed on the source pixel value of the RGB light signal at the pixel point according to the target color intensity value to obtain a first adjustment result; and a second adjustment is performed on the first adjustment result according to the adjustment coefficient and the brightness value of the RGB light signal at the pixel point. The second adjustment result is used as the target pixel value of the color-corrected RGB light signal at the pixel point.

16. A video data processing method, characterized in that: The method is applied to a decoding end and includes: Get the metadata of the video frame from the standard dynamic range video bitstream sent by the encoder; Get the display parameters of the display device; A mapping curve corresponding to the video frame is determined based on the display parameters and the metadata; the mapping curve is used to characterize the mapping relationship between the source color intensity value and the target color intensity value in the electrical signal space; wherein the extended upper limit value of the target color intensity value in the mapping curve is determined based on the display parameters and the metadata.

17. A video data processing method, characterized in that: The method is applied to the encoding end and includes: Get metadata for video frames in standard dynamic range video; Encoding the metadata and the standard dynamic range video to obtain a bitstream; The code stream is sent to a decoding end, so that the decoding end determines a mapping curve corresponding to the video frame based on display parameters of a display device and the metadata; the mapping curve is used to represent a mapping relationship between source color intensity values ​​and target color intensity values ​​in an electrical signal space; wherein an extended upper limit value of the target color intensity value in the mapping curve is determined based on the display parameters and the metadata.

18. A video data processing device, characterized in that: The device comprises: A metadata acquisition module, used to obtain metadata of video frames in a standard dynamic range video; A display parameter acquisition module, used to obtain display parameters of a display device; A mapping curve determination module is used to determine a mapping curve corresponding to the video frame based on the display parameters and the metadata; the mapping curve is used to represent the mapping relationship between the source color intensity value and the target color intensity value in the electrical signal space; wherein the extended upper limit value of the target color intensity value in the mapping curve is determined based on the display parameters and the metadata.

19. A video data processing device, characterized in that: The device comprises: YUV acquisition module, used to obtain the YUV signal of the video frame in the standard dynamic range video; A conversion module, used for converting the YUV signal into an RGB electrical signal; A source intensity determination module is used to determine the source color intensity value of a pixel in a video frame based on the RGB electrical signal; a mapping module, configured to map the source color intensity values ​​using a mapping curve corresponding to the video frame to obtain a target color intensity value; the mapping curve is configured to represent a mapping relationship between the source color intensity values ​​and the target color intensity values ​​in an electrical signal space; wherein an extended upper limit value of the target color intensity value in the mapping curve is determined based on display parameters of a display device and metadata of the video frame; The color correction module is used to perform color correction on the RGB light signal converted from the RGB electrical signal according to the target color intensity value, so as to display the color-corrected RGB light signal on a display device.

20. A video data processing device, characterized in that: The device is applied to a decoding end and includes: The metadata acquisition module is used to obtain the metadata of the video frame from the standard dynamic range video bitstream sent by the encoder; A display parameter acquisition module, used to obtain display parameters of a display device; A mapping curve determination module is used to determine a mapping curve corresponding to the video frame based on the display parameters and the metadata; the mapping curve is used to represent the mapping relationship between the source color intensity value and the target color intensity value in the electrical signal space; wherein the extended upper limit value of the target color intensity value in the mapping curve is determined based on the display parameters and the metadata.

21. A video data processing device, characterized in that: The device is applied to an encoding end and includes: A metadata acquisition module, used to obtain metadata of video frames in a standard dynamic range video; An encoding module, configured to encode the metadata and the standard dynamic range video to obtain a bit stream; A sending module is configured to send the code stream to a decoding end, so that the decoding end determines a mapping curve corresponding to the video frame based on display parameters of a display device and the metadata; the mapping curve is configured to represent a mapping relationship between source color intensity values ​​and target color intensity values ​​in an electrical signal space; wherein an extended upper limit value of the target color intensity value in the mapping curve is determined based on the display parameters and the metadata.

22. An apparatus for video data processing, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 17.

23. A non-transitory computer-readable storage medium storing instructions, wherein the instructions cause a processor to execute the method according to any one of claims 1 to 17.

24. A non-transitory computer-readable recording medium storing a bitstream generated by a method performed by an apparatus for video data processing, wherein the method comprises: Get metadata for video frames in standard dynamic range video; The metadata and the standard dynamic range video are encoded to obtain a bitstream.

25. A method for storing a bitstream of video data, comprising: Get metadata for video frames in standard dynamic range video; Encoding the metadata and the standard dynamic range video to obtain a bitstream; And, the bit stream is stored in a non-transitory computer-readable recording medium.

26. A method for storing a bit stream, characterized in that: include: executing the video data processing method according to claim 17 to generate a bit stream; And, storing the bit stream.

Images