Video data processing method, video display method and device, electronic device and readable storage medium

Abstract

The present invention relates to a video data processing method, a video display method and device, an electronic device and a readable storage medium. The method improves the display effect of a terminal device when displaying an SDR video by improving the mapping curve construction method. The specific reason is: compared with the existing technology, the mapping curve of each frame of video image is dynamically constructed in real time, rather than a single unchanging gamma curve; in addition, the technical solution provided by the present invention takes into account the display characteristics of the frame of video image before mapping and the display characteristics of the terminal device after mapping when constructing the mapping curve of each frame of video image. This makes it possible for video images containing different picture tones to adapt to terminal devices with different display characteristics, ensuring that the displayed picture is not distorted. Therefore, the problem of directly using the gamma curve to map the SDR video to the terminal device in the existing technology is overcome, resulting in poor display effect of the terminal device.

Description

Technical Field

[0001] The present invention relates to the field of image display technology, and in particular to a video data processing method, a video display method and device, an electronic device, and a readable storage medium. Background Art

[0002] With the rapid development of the electronics industry, display technology has undergone major innovations, from CRT (Cathode Ray Tube) to LCD (Liquid Crystal Display), OLED (Organic Light Emitting Display), minLED (sub-millimeter light emitting diode, LED devices with chip sizes between 50 and 200 μm), etc. The display capabilities of electronic terminals have been greatly improved. For example, the maximum display brightness of a standard CRT display is 100 nit, while the maximum display brightness of the current mainstream LCD and OLED displays can reach more than 300 nit.

[0003] However, compared to display chips, the development of display technology is still relatively lagging. Currently, the internationally accepted standard for Standard Dynamic Range (SDR) video playback is still the BT709 standard. This standard specifies the electro-optical transfer function (EOTF), also commonly known as the gamma curve, which defines how electrical signals (8-bit, 0-255) are converted into optical signals (0.01-100 nits). Currently, the maximum display brightness of mainstream LCD and OLED display devices can reach over 300 nits, far exceeding the 100 nit specified by the REC.709 SDR video display standard.

[0004] It is not difficult to find that REC.709 does not match the display capabilities of current terminals. Most electronic products on the market ignore this problem and generally directly use the gamma curve to map SDR video to the terminal device. As a result, to a certain extent, when the terminal device displays SDR video, the display effect is poor. The specific manifestations include:

[0005] For images with large overall brightness and dark differences and rich dark details, the dark details will be smoothed out to a certain extent after being processed by the gamma curve, resulting in loss of image information, inability to truly reproduce the information of the original SDR video, and the emergence of problems such as image color distortion. Summary of the Invention

[0006] In order to overcome the problems existing in the related art to at least a certain extent, the present invention provides a video data processing method, a video display method and device, an electronic device and a readable storage medium to solve the problem in the prior art that directly using the gamma curve to map the SDR video to the terminal device results in poor display effect of the terminal device.

[0007] According to a first aspect of an embodiment of the present invention, a video data processing method is provided, applicable to a decoding end, comprising:

[0008] Extracting the brightness feature of each frame of video image from the video stream of the standard dynamic range video sent by the encoder;

[0009] Get the display characteristics of the terminal device;

[0010] Normalizing the brightness feature to obtain a normalized brightness feature;

[0011] Based on the display characteristics and the normalized brightness characteristics, a mapping curve is constructed for each frame of the video image. The mapping curve is used to characterize the mapping relationship between the original RGB electrical signal and the corrected RGB electrical signal; the original RGB electrical signal is an RGB electrical signal obtained by converting the YUV electrical signal of the video image to be displayed; and the corrected RGB electrical signal is an RGB electrical signal adapted to the display characteristics of the terminal device.

[0012] According to a second aspect of an embodiment of the present invention, a video data processing method is provided, applicable to an encoding end, comprising:

[0013] For the standard dynamic range video to be displayed, the brightness characteristics of each frame of the video image are calculated;

[0014] Writing the brightness feature into metadata of each frame of video image;

[0015] Encoding the metadata and each frame of video image to form a video stream;

[0016] The video stream is sent to a decoding end so that the decoding end constructs a mapping curve for each frame of the video image based on the brightness characteristics and the display characteristics of the terminal device, wherein the mapping curve is used to characterize the mapping relationship between the original RGB electrical signal and the corrected RGB electrical signal; the original RGB electrical signal is an RGB electrical signal obtained by converting the YUV electrical signal of the video image to be displayed; and the corrected RGB electrical signal is an RGB electrical signal adapted to the display characteristics of the terminal device.

[0017] According to a third aspect of an embodiment of the present invention, a video display method is provided, applicable to a decoding end, comprising:

[0018] Extracting the YUV electrical signal of each frame of video image from the video stream of the standard dynamic range video sent by the encoding end, and converting the YUV electrical signal into the original RGB electrical signal;

[0019] Converting the original RGB electrical signal into a modified RGB electrical signal according to a mapping curve constructed according to the above-mentioned video data processing method;

[0020] According to a preset electro-optical conversion function, the corrected RGB electrical signal is converted into an RGB optical signal for display on a terminal device.

[0021] According to a fourth aspect of an embodiment of the present invention, a video display method is provided, applicable to a decoding end, comprising:

[0022] Extracting the YUV electrical signal of each frame of video image from the video stream of the standard dynamic range video sent by the encoding end, and converting the YUV electrical signal into the original RGB electrical signal;

[0023] According to the mapping curve, the original RGB electrical signal is converted into a modified RGB electrical signal, so that the modified RGB electrical signal is adapted to the RGB electrical signal of the display characteristics of the terminal device; the mapping curve is constructed according to the above-mentioned video data processing method;

[0024] Determine the brightness adjustment coefficient according to the corrected RGB electrical signal;

[0025] According to the brightness adjustment coefficient, the RGB optical signal converted from the original RGB electrical signal is subjected to hue correction, and the hue-corrected RGB optical signal is displayed on a terminal device.

[0026] According to a fifth aspect of an embodiment of the present invention, there is provided a video data processing apparatus, disposed at a decoding end, comprising:

[0027] An acquisition module is used to extract the brightness characteristics of each frame of video image from the video stream of the standard dynamic range video sent by the encoding end;

[0028] It is also used to obtain the display characteristics of the terminal device;

[0029] A normalization module, configured to normalize the brightness feature to obtain a normalized brightness feature;

[0030] A construction module is used to construct a mapping curve for each frame of the video image based on the display characteristics and the normalized brightness characteristics, wherein the mapping curve is used to represent the mapping relationship between the original RGB electrical signal and the corrected RGB electrical signal; the original RGB electrical signal is an RGB electrical signal obtained by converting the YUV electrical signal of the video image to be displayed; and the corrected RGB electrical signal is an RGB electrical signal adapted to the display characteristics of the terminal device.

[0031] According to a sixth aspect of an embodiment of the present invention, there is provided a video data processing apparatus, disposed at an encoding end, comprising:

[0032] A calculation module, for calculating the brightness characteristics of each frame of the standard dynamic range video to be displayed;

[0033] A reading and writing module, used for writing the brightness feature into the metadata of each frame of video image;

[0034] An encoding module, configured to encode the metadata and each frame of video image to form a video stream;

[0035] A sending module is configured to send the video stream to a decoding end, so that the decoding end constructs a mapping curve for each frame of the video image based on the brightness characteristics and the display characteristics of the terminal device, wherein the mapping curve is used to represent the mapping relationship between the original RGB electrical signal and the corrected RGB electrical signal; the original RGB electrical signal is an RGB electrical signal obtained by converting the YUV electrical signal of the video image to be displayed; and the corrected RGB electrical signal is an RGB electrical signal adapted to the display characteristics of the terminal device.

[0036] According to a seventh aspect of an embodiment of the present invention, there is provided a video display device, which is provided at a decoding end and includes:

[0037] An extraction module is used to extract the YUV electrical signal of each frame of video image from the video stream of the standard dynamic range video sent by the encoding end, and convert the YUV electrical signal into the original RGB electrical signal;

[0038] a correction module, configured to convert the original RGB electrical signal into a corrected RGB electrical signal according to a mapping curve, so that the corrected RGB electrical signal is adapted to the RGB electrical signal of the display characteristics of the terminal device; the mapping curve is constructed according to the above-mentioned video data processing method;

[0039] The conversion module is used to convert the corrected RGB electrical signal into an RGB optical signal according to a preset electro-optical conversion function for display on the terminal device.

[0040] According to an eighth aspect of an embodiment of the present invention, there is provided an end-to-end video display system, including:

[0041] An encoding end, used to execute the above method;

[0042] The decoding end is used to execute the above method.

[0043] According to a ninth aspect of an embodiment of the present invention, there is provided an electronic device, including:

[0044] at least one processor; and

[0045] a memory communicatively connected to the at least one processor; wherein,

[0046] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the above method.

[0047] According to a tenth aspect of an embodiment of the present invention, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the above method.

[0048] According to an eleventh aspect of an embodiment of the present invention, there is provided a computer program product, comprising a computer program, wherein the computer program implements the above method when executed by a processor.

[0049] The technical solutions provided by the embodiments of the present invention may have the following beneficial effects:

[0050] By improving the mapping curve construction method, the display effect of the terminal device when displaying SDR video is improved. The specific reasons are: compared with the existing technology, the mapping curve of each frame of video image is dynamically constructed in real time, rather than a single unchanging gamma curve; in addition, the existing technology does not consider the brightness characteristics of the video image to be mapped and the display characteristics of the terminal device when mapping. The technical solution provided by the present invention takes into account the display characteristics of the frame of video image before mapping and the display characteristics of the terminal device after mapping when constructing the mapping curve of each frame of video image. This allows video images containing different picture tones to adapt to terminal devices with different display characteristics, ensuring that the displayed picture is not distorted. Therefore, it overcomes the problem in the existing technology of directly using the gamma curve to map the SDR video to the terminal device, resulting in poor display effect of the terminal device.

[0051] In addition, one of the video display methods and devices provided by the present invention uses an improved mapping curve to correct the brightness of the original RGB electrical signal, and the mapping curve takes into account the brightness characteristics of each frame of video image and the display characteristics of the terminal device when constructing it. Therefore, after the corrected RGB electrical signal is converted into an optical signal through a standard electro-optical conversion function, the converted optical signal can be adapted to the display characteristics of the terminal device, and can also overcome the problem in the prior art of directly using the gamma curve to map the SDR video to the terminal device, resulting in poor display effect of the terminal device.

[0052] Another video display method and device provided by the present invention uses an improved mapping curve to correct the brightness of the original RGB electrical signal. The mapping curve takes into account the brightness characteristics of each frame of video image and the display characteristics of the terminal device when constructing it. Therefore, the brightness adjustment coefficient determined according to the corrected RGB electrical signal can accurately adjust the hue of the converted RGB light signal, so that the light signal after hue correction can be adapted to the display characteristics of the terminal device. It can also overcome the problem in the prior art of directly using the gamma curve to map the SDR video to the terminal device, resulting in poor display effect of the terminal device.

[0053] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0055] Figure 1 is a flowchart of a method for processing video data according to an exemplary embodiment;

[0056] Figure 2 1 is a schematic diagram showing a bitstream format of a video stream sent by an encoding end according to an exemplary embodiment;

[0057] Figure 3 is a flowchart of a method for processing video data according to an exemplary embodiment;

[0058] Figure 4 is a schematic diagram of an S-shaped mapping curve according to an exemplary embodiment;

[0059] Figure 5 is a schematic diagram of a mapping curve according to an exemplary embodiment;

[0060] Figure 6 is a flow chart showing a video display method according to an exemplary embodiment;

[0061] Figure 7 is a flow chart showing a video display method according to an exemplary embodiment;

[0062] Figure 8 is a schematic block diagram of a video data processing device according to an exemplary embodiment;

[0063] Figure 9 is a schematic block diagram of a video data processing device according to an exemplary embodiment;

[0064] Figure 10 is a schematic block diagram of a video display device according to an exemplary embodiment;

[0065] Figure 11 The present invention is a schematic block diagram of an end-to-end video display system according to an exemplary embodiment. DETAILED DESCRIPTION

[0066] Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. In the following description, when referring to the drawings, like numbers in different figures represent like or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present invention, as detailed in the appended claims.

[0067] As described in the background technology, in the prior art, directly using a gamma curve to map SDR video to a terminal device results in the terminal device being unable to perfectly display the production effect of the SDR video. The present invention provides the following exemplary embodiments. It should be noted that:

[0068] 1. The "minimum screen brightness" mentioned in each embodiment of the present invention refers to the minimum screen brightness when playing SDR video when the terminal device does not implement the technical solution provided by the present invention; the "maximum screen brightness" refers to the maximum screen brightness that can be achieved when playing SDR video when the terminal device does not implement the technical solution provided by the present invention; and the "screen resolution" refers to the screen resolution that comes with the terminal device when it leaves the factory.

[0069] 2. The “preset” mentioned in the embodiments of the present invention refers to the REC.709 standard currently accepted by mainstream televisions and monitors.

[0070] Figure 1 A video data processing method according to an exemplary embodiment is shown, which is applicable to the encoding end. Figure 1 , the method comprising:

[0071] Step S11: Calculate the brightness characteristics of each frame of the standard dynamic range video to be displayed;

[0072] Step S12: writing the brightness feature into the metadata of each frame of video image;

[0073] Step S13: Encode the metadata and each frame of video image to form a video stream;

[0074] Step S14: Send the video stream to the decoding end, so that the decoding end constructs a mapping curve for each frame of the video image according to the brightness characteristics and the display characteristics of the terminal device, wherein the mapping curve is used to characterize the mapping relationship between the original RGB electrical signal and the corrected RGB electrical signal; the original RGB electrical signal is an RGB electrical signal obtained by converting the YUV electrical signal of the video image to be displayed; and the corrected RGB electrical signal is an RGB electrical signal adapted to the display characteristics of the terminal device.

[0075] It should be noted that the technical solution provided in this embodiment is applicable to the encoding end, which is a terminal device with an image acquisition device, including but not limited to: mobile phones, tablet computers, laptops, calculators, and phone watches. Applicable scenarios include but are not limited to:

[0076] 1. The SDR video at the encoding end is mapped to the CRT display at the decoding end for display;

[0077] 2. The SDR video at the encoding end is mapped to the LCD display at the decoding end for display;

[0078] 3. The SDR video on the encoding side is mapped to the OLED display on the decoding side for display;

[0079] 4. The SDR video on the encoding side is mapped to the minLED display on the decoding side for display.

[0080] In addition, the technical solution of the encoding end provided in this embodiment, in conjunction with the technical solution of the decoding end (the technical solution of the decoding end is that the decoding end recorded in step S14 constructs a mapping curve for each frame of the video image according to the brightness characteristics and the display characteristics of the terminal device), can be used to enhance SDR video. It is suitable for mid- and low-end terminal devices with video playback capabilities. As one of the code streams of high-audio and video standards (for example, frame-by-frame images), it can increase the coverage of high-audio and video standards.

[0081] The technical solution of the encoding end provided in this embodiment, combined with the technical solution of the decoding end, can significantly improve the color distortion of the video picture when displaying SDR video on a high-brightness display screen (a display screen with a maximum display brightness of more than 300 nits), and can to a certain extent solve the problem of mismatch between the display screen's display capabilities and the SDR standard.

[0082] The technical solution of the encoding end provided in this embodiment, combined with the technical solution of the decoding end, fully considers the characteristics of the video material (i.e., the brightness characteristics mentioned in this embodiment) and the display capabilities of the terminal (i.e., the display characteristics mentioned in this embodiment), and can significantly improve the playback effect of SDR video.

[0083] In specific practice, the brightness characteristics include at least brightness intensity StrengthL, average brightness AvergLuma, and a boundary value RegionD between bright and dark areas. The display characteristics include at least maximum screen brightness MaxDisplay, minimum screen brightness MinDisplay, and screen resolution RES.

[0084] The step S11 of "calculating the brightness characteristics of each frame of the standard dynamic range video to be displayed" at least includes:

[0085] Step 1: Calculate the brightness intensity of each frame of video image, including:

[0086] Calculating the histogram distribution HistY of the brightness component Y of each frame of the video image, and determining the mid-gray value MildGray of the histogram distribution HistY, including:

[0087] 1) Set the middle gray value of the histogram distribution HistY to a fixed value, for example, MildGray=118;

[0088] 2) The square root of the product of the maximum brightness value and the minimum brightness value in the histogram distribution HistY is determined as the middle gray value of the histogram distribution HistY. For example,

[0089] Among them, maxY is the maximum brightness value in the histogram distribution HistY (that is, the maximum coordinate value on the horizontal axis), and minY is the minimum brightness value in the histogram distribution HistY (that is, the minimum coordinate value on the horizontal axis).

[0090] The expectation of the pixel points whose brightness value i is greater than the middle gray value MildGray in the histogram distribution HistY is calculated and recorded as the brightness intensity StrengthL, specifically:

[0091] StrengthL=∑ i>MildGray HistY[i]*i / ∑ i>MildGray HistY[i].

[0092] Step 2: Calculate the average brightness of each frame of video image, including:

[0093] Convert the electrical signal of the video image into an optical signal according to a preset electro-optical conversion function;

[0094] Luma[i]=EOTF709(i / 255); i / 255 represents the pixel value of the normalized electrical signal, and Luma[i] represents the pixel value after conversion to an optical signal; the function EOTF709() represents the electro-optical conversion function specified in the REC.709 standard, specifically:

[0095] L represents the input variable, L=i / 255.

[0096] Traversing all brightness values ​​of the light signal, calculating the sum of the products of each brightness value Luma[i] and its corresponding number of pixels HistY[i] in the histogram distribution HistY;

[0097] The ratio of the sum of the products to the screen resolution RES of the terminal device is determined as the average brightness AvergLuma, specifically:

[0098] AvergLuma=∑HistY[i]*Luma[i] / RES.

[0099] Step 3: Calculate the boundary value between the bright area and the dark area of ​​each frame of the video image, including:

[0100] Calculate the expectation of pixels whose brightness value i is between zero and mid-gray value [0-MildGray] in the histogram distribution HistY, and record it as the average darkness MeanDark;

[0101] MeanDark=∑ i∈[0，MildGray] HistY[i]*i / ∑ i∈[0，MildGray] HistY[i]

[0102] Calculate the average value of the average darkness and the middle gray value, and determine the average value as the demarcation value RegionD, specifically:

[0103]

[0104] The steps of "writing the brightness feature into the metadata of each frame of video image" in step S12 and "encoding the metadata and each frame of video image to form a video stream" in step S13 are as follows:

[0105] See also Figure 2 , Figure 2This example shows a video stream sent by the encoder. In this video stream, the first symbol, SPS, stores the global parameters of the encoded video image. The second symbol, PPS, stores parameters related to the entire image. PPS is usually stored in the video file header together with SPS. The third symbol, IDR, stores the decoding parameter set. The fourth symbol, SLICE, stores frame encoding information. The fifth symbol, SEI, stores supplementary information of the video stream, namely the metadata mentioned in this embodiment. Subsequent symbols are repetitions of SLICE and SEI, and will not be described in detail.

[0106] After the encoder calculates the brightness feature, it writes it to Figure 2 The fifth codeword SEI shown is then encoded into a video stream together with the data in other codewords.

[0107] It is understandable that the technical solution provided by this embodiment, because the video images on the encoder side are lossless and uncompressed, by calculating the brightness characteristics of each video frame on the encoder side, can ensure maximum alignment with the actual state of each video frame, reduce calculation errors, and provide accurate data support for the mapping curve constructed by the decoder side. At the same time, because the calculation step of the brightness characteristics is transferred to the encoder side, the computational pressure on the decoder side is also reduced, allowing the decoder side to construct the dynamic mapping curve more quickly and display high-quality video images with lossless tones more quickly.

[0108] Figure 3 A video data processing method according to an exemplary embodiment is shown, which is applicable to a decoding end. Figure 3 , the method comprising:

[0109] Step S21: extracting the brightness feature of each frame of video image from the video stream of the standard dynamic range video sent by the encoding end;

[0110] Step S22: Obtain display characteristics of the terminal device;

[0111] Step S23: normalizing the brightness feature to obtain a normalized brightness feature;

[0112] Step S24: Construct a mapping curve for each frame of the video image based on the display characteristics and the normalized brightness characteristics, wherein the mapping curve is used to characterize the mapping relationship between the original RGB electrical signal and the corrected RGB electrical signal; the original RGB electrical signal is an RGB electrical signal obtained by converting the YUV electrical signal of the video image to be displayed; and the corrected RGB electrical signal is an RGB electrical signal adapted to the display characteristics of the terminal device.

[0113] It should be noted that the technical solution provided in this embodiment is applicable to the decoding end. The decoding end includes: a computer end (including a server), and / or a mobile terminal (including but not limited to: a smartphone, a tablet computer, a VR helmet, VR glasses, etc.), and applicable scenarios include but are not limited to:

[0114] 1. The SDR video at the encoding end is mapped to the CRT display at the decoding end for display;

[0115] 2. The SDR video at the encoding end is mapped to the LCD display at the decoding end for display;

[0116] 3. The SDR video on the encoding side is mapped to the OLED display on the decoding side for display;

[0117] 4. The SDR video on the encoding side is mapped to the minLED display on the decoding side for display.

[0118] As can be seen from the technical solution of the encoding end, since the video stream sent by the encoding end to the decoding end includes metadata, and the metadata records the brightness characteristics of each frame of the video image, the "extracting the brightness characteristics of each frame of the video image from the video stream of the standard dynamic range video sent by the encoding end" in step S21 can be:

[0119] A video stream of a standard dynamic range video is obtained, each frame of video image and metadata of each frame of video image are decoded therefrom, and brightness characteristics of each frame of video image are read from the metadata.

[0120] It can be seen from the technical solution of the above encoding end that the brightness characteristics include at least the brightness intensity StrengthL, the average brightness AvergLuma, and the boundary value RegionD between the bright area and the dark area, and the display characteristics include at least: the maximum screen brightness MaxDisplay, the minimum screen brightness MinDisplay and the screen resolution RES.

[0121] In step S23, “normalizing the brightness feature to obtain a normalized brightness feature” includes:

[0122] Normalized brightness intensity StrengthL'=StrengthL / 255;

[0123] Normalized average intensity AvergLuma'=AvergLuma / 255;

[0124] The normalized boundary value RegionD'=RegionD / 255;

[0125] 255 represents the maximum pixel value.

[0126] In step S24, “constructing a mapping curve for each frame of the video image according to the display characteristics and the normalized brightness characteristics” includes:

[0127] Step 1: Determine, based on the display characteristics and the normalized brightness characteristics of the current frame video image, a function expression of an S-shaped mapping curve for the brightness region of the current frame video image and a function expression of a linear mapping line for the dark region of the current frame video image;

[0128] Step 2: Determine the piecewise function composed of the linear mapping line and the S-shaped mapping curve as the mapping curve of the current frame video image.

[0129] The step 1 of “determining a function expression of an S-shaped mapping curve of a brightness region of the current frame video image based on the display characteristics and the normalized brightness characteristics of the current frame video image” includes:

[0130] Select any one of the preset standard S-shaped mapping curves;

[0131] The parameter values ​​of the curve are calculated according to a pre-generated parameter calculation formula, wherein the parameter calculation formula is related to the display characteristics and the normalized brightness characteristics of the current frame video image.

[0132] The parameter calculation formula is pre-generated by the following method, including:

[0133] Acquire multiple sets of display features and normalized brightness features, repeat the following operations for each set of display features and normalized brightness features until the output S-shaped mapping curve has a curve shape that meets the expected curve shape under different display features and normalized brightness features, and determine the parameter calculation formula at this time as the parameter calculation formula of the current standard S-shaped mapping curve; the operations include:

[0134] For any standard S-shaped mapping curve, a set of display features and normalized brightness features are substituted into a preset parameter calculation formula to obtain a set of parameter values;

[0135] Substituting the parameter value into the function expression of the current standard S-shaped mapping curve, and outputting an S-shaped mapping curve;

[0136] The parameter calculation formula is adjusted until the curve shape of the output S-shaped mapping curve meets the curve shape of the S-shaped mapping curve of the expected curve shape.

[0137] The expected curve shape refers to the curve shape that satisfies preset constraints, and the preset constraints include:

[0138] (1) The lower the screen brightness, the smaller the curvature of the S-shaped mapping curve, that is, the overall shape is close to linear;

[0139] (2) The greater the brightness intensity StrengthL', the smaller the curvature of the S-shaped mapping curve;

[0140] (3) The smaller the average brightness AvergLuma' is, the greater the curvature of the S-shaped mapping curve is.

[0141] In specific practice, the calculation of the parameter value of the curve according to the pre-generated parameter calculation formula includes:

[0142] See also Figure 4 ( Figure 4 The 1 on the horizontal axis represents a pixel value of 255, the 0.2 on the horizontal axis represents a pixel value of 0.2*255, and so on; the 1 on the vertical axis represents a pixel value of 255, the 0.1 on the vertical axis represents a pixel value of 0.1*255, and so on; Figure 4 The solid curve in the figure is the curve shape diagram of the selected standard S-shaped mapping curve. The dotted straight line on the diagonal is an auxiliary line, which is drawn to help users more intuitively see the curvature of the selected standard S-shaped mapping curve. If the function expression of the selected standard S-shaped mapping curve is:

[0143] F(L)=((mp*L) / ((mp-1)*L+1)) 3.2+deltaM (1), L represents the pixel value after normalization of the original RGB electrical signal, L=i / 255, F(L) represents the pixel value after normalization of the modified RGB electrical signal; and the brightness characteristics include at least: brightness intensity, average brightness, and the boundary value between the bright area and the dark area; the display characteristics include at least: maximum screen brightness, minimum screen brightness and screen resolution, then:

[0144] The specific value of the parameter mp is solved according to the following formula (3):

[0145] strengthL' represents the normalized brightness intensity, and mpMax represents the upper limit of mp;

[0146] Solve the parameter deltaM according to the following formula (5):

[0147] MinDisplay represents the minimum screen brightness, RegionD' represents the boundary value between the normalized bright area and the dark area, and EOTF709() represents the electro-optical transfer function.

[0148] In practice, the upper limit mpMax of the parameter mp is obtained according to the following formula (2):

[0149] MaxDisplay indicates the maximum screen brightness;

[0150] The compensation value Δmp of the parameter mp is solved according to the following formula (4):

[0151] AvergLuma' represents the normalized average brightness;

[0152] And update the specific value of mp to:

[0153] AvergLuma' represents the normalized average brightness.

[0154] In step 1, "determining a function expression of a linear mapping line of a dark region of the current frame video image based on the display characteristics and the normalized brightness characteristics of the current frame video image" includes:

[0155] (1) determining a lower limit value of the slope of the linear mapping line according to the display characteristics and the normalized brightness characteristics, and compensating the lower limit value of the slope with a preset slope compensation value, including:

[0156] Step A: Calculate the product of the reciprocal of the maximum screen brightness MaxDisplay and a preset fixed constant (e.g., 100);

[0157] Calculate a preset power of the product, and determine the result as the limit value SlopeLimit of the slope of the linear mapping line, wherein the preset power C0 is related to the minimum screen brightness, specifically:

[0158] Among them, the value of C0 is related to the minimum screen brightness MinDisplay:

[0159]

[0160] Step B: If the normalized average brightness AvergLuma' is less than a first threshold (the first threshold is set according to experimental data or empirical value, for example, set to 0.01), the lower limit value MinSlope of the slope is determined to be a constant (for example, 0.9), specifically: AvergLuma'<0.01, MinSlope=0.9.

[0161] Step C: If the normalized average brightness AvergLuma' is greater than or equal to the first threshold and less than the second threshold (the second threshold is set according to experimental data or empirical value, for example, set to 0.1, and the second threshold is greater than the first threshold), the lower limit value MinSlope of the slope is determined as a function related to the limit value SlopeLimit of the slope, specifically:

[0162] 0.1>AvergLuma′≥0.01, MinSlope=0.9-(0.9-SlopeLimit)*avglog 1.1 ,avglog=2+log10(AvergLuma′).

[0163] Step D: If the normalized average brightness AvergLuma' is greater than the second threshold, determine the lower limit value MinSlope of the slope as the limit value SlopeLimit of the slope, specifically:

[0164] AvergLuma′≥0.1, MinSlope=SlopeLimit.

[0165] Combine the results from steps A to D, that is:

[0166]

[0167] (2) Determine the vertical coordinate F(RegionD') corresponding to the normalized boundary value RegionD' on the S-shaped mapping curve, that is, substitute RegionD' into the value obtained by the above formula (1).

[0168] (3) The ratio F(RegionD') / (RegionD') of the vertical coordinate F(RegionD') and the normalized boundary value RegionD' is determined as the reference slope.

[0169] (4) Compensate the lower limit value MinSlope of the slope with a preset slope compensation value CompS, compare the lower limit value of the compensated slope with the reference slope, and determine the minimum value of the two as the slope Slope of the linear mapping line, specifically:

[0170] CompS represents a preset slope compensation value, which is read from the metadata when the decoding end decodes the video stream and is written by the encoding end during encoding. The slope compensation value is manually set by the encoding end user based on the overall brightness characteristics of the SDR video to be transmitted (for example, if the overall brightness of the current SDR video to be transmitted is less than the first threshold, the slope compensation value is set to 0.05 by the encoding end user; if the overall brightness of the current SDR video to be transmitted is greater than the second threshold, the slope compensation value is set to 0 by the encoding end user; wherein the second threshold is greater than the first threshold).

[0171] (5) According to the slope and the normalized demarcation value, a function expression of the linear mapping line of the dark area of ​​the current frame video image is determined, specifically:

[0172] TM(L)=Slope*L, L≤RegionD′.

[0173] Furthermore, the method further includes:

[0174] Find a point on the S-shaped mapping curve such that the slope of the line connecting the point and the origin is equal to the slope of the linear mapping line, that is, satisfying:

[0175] Update the horizontal coordinate RegionDnew of the point to the normalized boundary value between the bright area and the dark area;

[0176] According to the slope and the updated boundary value between the bright area and the dark area, the function expression of the linear mapping line is updated, that is:

[0177] TM(L)=Slope*L,L≤RegionDnew.

[0178] In step 2, “determining the piecewise function composed of the linear mapping line and the S-shaped mapping curve as the mapping curve of the current frame video image” is specifically as follows:

[0179] L represents the pixel value after normalization of the original RGB electrical signal, L=i / 255.

[0180] Figure 5 This is a schematic diagram of the mapping curve of the current frame video image generated according to the technical solution provided by this embodiment. Figure 5 , Figure 5 1 on the horizontal axis represents a pixel value of 255, 0.2 on the horizontal axis represents a pixel value of 0.2*255, and so on; 1 on the vertical axis represents a pixel value of 255, 0.1 on the vertical axis represents 0.1*255, and so on. Figure 5 The mapping curve shown in the example has an inflection point when the horizontal coordinate is 0.2. When the horizontal coordinate is less than or equal to 0.2, the corresponding straight line is the linear mapping straight line of the dark area, and when the horizontal coordinate is greater than 0.2, the corresponding curve is the S-shaped mapping curve of the bright area.

[0181] It can be understood that the technical solution provided in this embodiment improves the display effect of the terminal device when displaying SDR video by improving the mapping curve construction method. The specific reason is: compared with the existing technology, the mapping curve of each frame of video image is dynamically constructed in real time, rather than a single unchanging gamma curve; in addition, the existing technology does not consider the brightness characteristics of the video image to be mapped and the display characteristics of the terminal device when mapping. The technical solution provided by the present invention takes into account the display characteristics of the frame of video image before mapping and the display characteristics of the terminal device after mapping when constructing the mapping curve of each frame of video image. This allows video images containing different picture tones to adapt to terminal devices with different display characteristics, ensuring that the displayed picture is not distorted. Therefore, it overcomes the problem of directly using the gamma curve to map the SDR video to the terminal device in the existing technology, resulting in poor display effect of the terminal device.

[0182] Figure 6 is a flow chart of a video display method according to an exemplary embodiment. Figure 6 As shown, the method is applicable to the decoding end, and the method includes:

[0183] Step S31: extracting a YUV electrical signal of each frame of video image from the video stream of the standard dynamic range video sent by the encoding end, and converting the YUV electrical signal into an original RGB electrical signal;

[0184] Step S32: converting the original RGB electrical signal into a modified RGB electrical signal according to a mapping curve constructed according to the above-mentioned video data processing method;

[0185] Step S33: According to a preset electro-optical conversion function, the corrected RGB electrical signal is converted into an RGB optical signal for display on a terminal device.

[0186] It should be noted that the technical solution provided in this embodiment is applicable to the decoding end. The decoding end includes: a computer end (including a server), and / or a mobile terminal (including but not limited to: a smartphone, a tablet computer, a VR helmet, VR glasses, etc.), and applicable scenarios include but are not limited to:

[0187] 1. The SDR video at the encoding end is mapped to the CRT display at the decoding end for display;

[0188] 2. The SDR video at the encoding end is mapped to the LCD display at the decoding end for display;

[0189] 3. The SDR video on the encoding side is mapped to the OLED display on the decoding side for display;

[0190] 4. The SDR video on the encoding side is mapped to the minLED display on the decoding side for display.

[0191] It can be understood that the technical solution provided in this embodiment uses an improved mapping curve to correct the brightness of the original RGB electrical signal, and the mapping curve takes into account the brightness characteristics of each frame of video image and the display characteristics of the terminal device when constructing it. Therefore, after the corrected RGB electrical signal is converted into an optical signal through a standard electro-optical conversion function, the converted optical signal can be adapted to the display characteristics of the terminal device, and can also overcome the problem in the prior art of directly using the gamma curve to map the SDR video to the terminal device, resulting in poor display effect of the terminal device.

[0192] Figure 7 is a flow chart of a video display method according to an exemplary embodiment. Figure 7 As shown, the method is applicable to the decoding end, and the method includes:

[0193] Step S41: extracting a YUV electrical signal of each frame of video image from the video stream of the standard dynamic range video sent by the encoding end, and converting the YUV electrical signal into an original RGB electrical signal;

[0194] Step S42: Convert the original RGB electrical signal into a modified RGB electrical signal according to a mapping curve, so that the modified RGB electrical signal is adapted to the RGB electrical signal of the display characteristics of the terminal device; the mapping curve is constructed according to the above-mentioned video data processing method;

[0195] Step S43: determining a brightness adjustment coefficient based on the corrected RGB electrical signal;

[0196] Step S44: performing color tone correction on the RGB optical signal converted from the original RGB electrical signal according to the brightness adjustment coefficient, and displaying the color tone corrected RGB optical signal on a terminal device.

[0197] It should be noted that the technical solution provided in this embodiment is applicable to the decoding end. The decoding end includes: a computer end (including a server), and / or a mobile terminal (including but not limited to: a smartphone, a tablet computer, a VR helmet, VR glasses, etc.), and applicable scenarios include but are not limited to:

[0198] 1. The SDR video at the encoding end is mapped to the CRT display at the decoding end for display;

[0199] 2. The SDR video at the encoding end is mapped to the LCD display at the decoding end for display;

[0200] 3. The SDR video on the encoding side is mapped to the OLED display on the decoding side for display;

[0201] 4. The SDR video on the encoding side is mapped to the minLED display on the decoding side for display.

[0202] It can be understood that the technical solution provided in this embodiment uses an improved mapping curve to correct the brightness of the original RGB electrical signal, and the mapping curve takes into account the brightness characteristics of each frame of the video image and the display characteristics of the terminal device when constructing it. Therefore, the brightness adjustment coefficient determined according to the corrected RGB electrical signal can accurately adjust the hue of the converted RGB light signal, so that the light signal after hue correction can be adapted to the display characteristics of the terminal device. It can also overcome the problem in the prior art of directly using the gamma curve to map the SDR video to the terminal device, resulting in poor display effect of the terminal device.

[0203] In specific practice, the step S41 of “converting the YUV electrical signal into the original RGB electrical signal” may be:

[0204] According to the preset conversion formula of YUV electrical signal to RGB electrical signal, the YUV electrical signal is converted into the original RGB signal:

[0205]

[0206] In specific practice, the step S43 of “determining the brightness adjustment coefficient according to the corrected RGB electrical signal” may include:

[0207] Compare the original pixel maximum values ​​of the three channels R, G, and B in the original RGB electrical signal, that is:

[0208] RGBmax=max(R, G, B);

[0209] The pixel maximum value TM(RGBmax) obtained by mapping the original pixel maximum value RGBmax through the mapping curve is recorded as the corrected pixel maximum value;

[0210] The ratio of the corrected pixel maximum value to the original pixel maximum value is determined as the brightness adjustment coefficient tmk, that is, tmk=TM(RGBmax) / RGBmax.

[0211] In specific practice, in step S44, “performing hue correction on the RGB light signal converted from the original RGB electrical signal according to the brightness adjustment coefficient” may include:

[0212] According to the brightness adjustment coefficient, brightness correction is performed on the RGB optical signals converted from the original RGB electrical signals (the optical signal of the R channel is corrected to (tmk*EOTF709(R), the optical signal of the G channel is corrected to tmK*EOTF709(G), and the optical signal of the B channel is corrected to tmK*EOTF709(B), where EOTF7099() represents a preset electro-optical conversion function);

[0213] According to the preset photoelectric conversion function, the brightness-corrected RGB optical signal is converted into a brightness-corrected RGB electrical signal (the brightness-corrected electrical signal of the R channel is OETF709(tmK*EOTF709(R)), the brightness-corrected electrical signal of the G channel is OETF709(tmK*EOTF709(G)), and the brightness-corrected electrical signal of the B channel is OETF709(tmK*EOTF709(B)), where OETF709() represents the preset photoelectric conversion function), that is: Among them, R, G, B are original RGB electrical signals, and Rt, Gt, Bt are RGB electrical signals after brightness correction.

[0214] The saturation of the RGB electrical signal after brightness correction is corrected as follows:

[0215] in,

[0216] Luma is the saturation adjustment value, Luma = 0.2126*Rt + 0.7152*Gt + 0.0722*Bt, where 0.2126, 0.7152, and 0.0722 are preset weight coefficients, set based on historical experience or experimental data;

[0217] adjustS is the saturation adjustment coefficient, in,

[0218] max(R, G, B) is the maximum value of the original pixel in the R, G, and B channels of the original RGB electrical signal, and min(R, G, B) is the minimum value of the original pixel in the R, G, and B channels;

[0219] max(Rt, Gt, Bt) is the maximum value of the pixel in the R, G, and B channels in the RGB electrical signal after brightness correction, and min(Rt, Gt, Bt) is the minimum value of the pixel in the R, G, and B channels.

[0220] According to the preset electro-optical conversion function, the saturation-corrected RGB electrical signal is converted into an RGB optical signal to obtain a hue-corrected RGB optical signal, that is:

[0221] Among them, Rnew, Gnew, and Bnew are RGB electrical signals after saturation correction, and oR, oG, and oB are RGB optical signals after hue correction.

[0222] It can be understood that the technical solution provided by this embodiment adds brightness correction and saturation correction of the RGB light signal compared to the video display method provided by the previous embodiment, and the brightness adjustment coefficient for brightness correction and the saturation adjustment coefficient for saturation correction are both related to the mapping curve dynamically constructed by the decoding end. Since the mapping curve takes into account the brightness characteristics of the current frame video image and the display characteristics of the terminal device when constructing it, the brightness adjustment coefficient and saturation adjustment coefficient determined based on the mapping curve can both correct the hue of the light signal to be displayed to match the display characteristics of the video terminal device, so that the high-brightness terminal device can display the image of the standard dynamic range video without distortion, thereby improving the picture hue quality.

[0223] Figure 8 is a schematic block diagram of a video data processing device 100 according to an exemplary embodiment. Figure 8 As shown, the device 100 is provided at the encoding end, and the device 100 includes:

[0224] A calculation module 101 is used to calculate the brightness characteristics of each frame of the standard dynamic range video to be displayed;

[0225] The reading and writing module 102 is used to write the brightness feature into the metadata of each frame of video image;

[0226] The encoding module 103 is used to encode the metadata and each frame of the video image to form a video stream;

[0227] The sending module 104 is configured to send the video stream to a decoding end so that the decoding end constructs a mapping curve for each frame of the video image based on the brightness characteristics and the display characteristics of the terminal device. The mapping curve is used to represent the mapping relationship between the original RGB electrical signal and the corrected RGB electrical signal; the original RGB electrical signal is an RGB electrical signal obtained by converting the YUV electrical signal of the video image to be displayed; and the corrected RGB electrical signal is an RGB electrical signal adapted to the display characteristics of the terminal device.

[0228] It is understandable that the technical solution provided by this embodiment, because the video images on the encoder side are lossless and uncompressed, by calculating the brightness characteristics of each video frame on the encoder side, can ensure maximum alignment with the actual state of each video frame, reduce calculation errors, and provide accurate data support for the mapping curve constructed by the decoder side. At the same time, because the calculation step of the brightness characteristics is transferred to the encoder side, the computational pressure on the decoder side is also reduced, allowing the decoder side to construct the dynamic mapping curve more quickly and display high-quality video images with lossless tones more quickly.

[0229] Figure 9 is a schematic block diagram of a video data processing device 200 according to an exemplary embodiment. Figure 9 As shown, the device 200 is provided at a decoding end, and the device 200 includes:

[0230] An acquisition module 201 is configured to extract a brightness feature of each frame of a video image from a video stream of a standard dynamic range video sent by an encoding end;

[0231] It is also used to obtain the display characteristics of the terminal device;

[0232] A normalization module 202 is configured to normalize the brightness feature to obtain a normalized brightness feature;

[0233] A construction module 203 is configured to construct a mapping curve for each frame of the video image based on the display characteristics and the normalized brightness characteristics, wherein the mapping curve is used to characterize the mapping relationship between the original RGB electrical signal and the corrected RGB electrical signal; the original RGB electrical signal is an RGB electrical signal obtained by converting the YUV electrical signal of the video image to be displayed; and the corrected RGB electrical signal is an RGB electrical signal adapted to the display characteristics of the terminal device.

[0234] It can be understood that the technical solution provided in this embodiment improves the display effect of the terminal device when displaying SDR video by improving the mapping curve construction method. The specific reason is: compared with the existing technology, the mapping curve of each frame of video image is dynamically constructed in real time, rather than a single unchanging gamma curve; in addition, the existing technology does not consider the brightness characteristics of the video image to be mapped and the display characteristics of the terminal device when mapping. The technical solution provided by the present invention takes into account the display characteristics of the frame of video image before mapping and the display characteristics of the terminal device after mapping when constructing the mapping curve of each frame of video image. This allows video images containing different picture tones to adapt to terminal devices with different display characteristics, ensuring that the displayed picture is not distorted. Therefore, it overcomes the problem of directly using the gamma curve to map the SDR video to the terminal device in the existing technology, resulting in poor display effect of the terminal device.

[0235] Figure 10 is a schematic block diagram of a video display device 300 according to an exemplary embodiment. Figure 10 As shown, the device 300 is provided at a decoding end, and the device 300 includes:

[0236] The extraction module 301 is used to extract the YUV electrical signal of each frame of video image from the video stream of the standard dynamic range video sent by the encoding end, and convert the YUV electrical signal into the original RGB electrical signal;

[0237] A correction module 302 is configured to convert the original RGB electrical signal into a corrected RGB electrical signal according to a mapping curve, so that the corrected RGB electrical signal is adapted to the RGB electrical signal of the display characteristics of the terminal device; the mapping curve is constructed according to the above-mentioned video data processing method;

[0238] The conversion module 303 is configured to convert the corrected RGB electrical signal into an RGB optical signal according to a preset electrical-optical conversion function for display on a terminal device.

[0239] It can be understood that the technical solution provided in this embodiment uses an improved mapping curve to correct the brightness of the original RGB electrical signal, and the mapping curve takes into account the brightness characteristics of each frame of video image and the display characteristics of the terminal device when constructing it. Therefore, after the corrected RGB electrical signal is converted into an optical signal through a standard electro-optical conversion function, the converted optical signal can be adapted to the display characteristics of the terminal device, and can also overcome the problem in the prior art of directly using the gamma curve to map the SDR video to the terminal device, resulting in poor display effect of the terminal device.

[0240] Figure 11 is a schematic block diagram of an end-to-end video display system 400 according to an exemplary embodiment. Figure 11 As shown, the system 400 includes:

[0241] The encoding end 401 is used to execute the above method;

[0242] The decoding end 402 is configured to execute the above method.

[0243] It is understandable that the technical solution provided by this embodiment, because the video images on the encoder side are lossless and uncompressed, by calculating the brightness characteristics of each video frame on the encoder side, can ensure maximum alignment with the actual state of each video frame, reduce calculation errors, and provide accurate data support for the mapping curve constructed by the decoder side. At the same time, because the calculation step of the brightness characteristics is transferred to the encoder side, the computational pressure on the decoder side is also reduced, allowing the decoder side to construct the dynamic mapping curve more quickly and display high-quality video images with lossless tones more quickly.

[0244] The decoding end improves the display effect of the terminal device when displaying SDR video by improving the mapping curve construction method. The specific reasons are: compared with the existing technology, the mapping curve of each frame of video image is dynamically constructed in real time, rather than a single unchanging gamma curve; in addition, the existing technology does not consider the brightness characteristics of the video image to be mapped and the display characteristics of the terminal device when mapping. The technical solution provided by the present invention takes into account the display characteristics of the frame of video image before mapping and the display characteristics of the terminal device after mapping when constructing the mapping curve of each frame of video image. This allows video images containing different picture tones to adapt to terminal devices with different display characteristics, ensuring that the displayed picture is not distorted. Therefore, it overcomes the problem of directly using the gamma curve to map the SDR video to the terminal device in the existing technology, resulting in poor display effect of the terminal device.

[0245] According to an exemplary embodiment, an electronic device includes:

[0246] at least one processor; and

[0247] a memory communicatively connected to the at least one processor; wherein,

[0248] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the above method.

[0249] It should be noted that electronic devices include but are not limited to: smart terminals (for example, mobile phones, tablet computers, smart watches, etc.) and computer devices.

[0250] The processor includes but is not limited to: CPU, single chip microcomputer, PLC controller, FPGA controller, etc.

[0251] The memory may include computer system readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory; it may also include other removable / non-removable, volatile / non-volatile computer system storage media. The memory may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of various embodiments of the present invention.

[0252] It is understandable that the technical solution provided by this embodiment, because the video images on the encoder side are lossless and uncompressed, by calculating the brightness characteristics of each video frame on the encoder side, can ensure maximum alignment with the actual state of each video frame, reduce calculation errors, and provide accurate data support for the mapping curve constructed by the decoder side. At the same time, because the calculation step of the brightness characteristics is transferred to the encoder side, the computational pressure on the decoder side is also reduced, allowing the decoder side to construct the dynamic mapping curve more quickly and display high-quality video images with lossless tones more quickly.

[0253] The decoding end improves the display effect of the terminal device when displaying SDR video by improving the mapping curve construction method. The specific reasons are: compared with the existing technology, the mapping curve of each frame of video image is dynamically constructed in real time, rather than a single unchanging gamma curve; in addition, the existing technology does not consider the brightness characteristics of the video image to be mapped and the display characteristics of the terminal device when mapping. The technical solution provided by the present invention takes into account the display characteristics of the frame of video image before mapping and the display characteristics of the terminal device after mapping when constructing the mapping curve of each frame of video image. This allows video images containing different picture tones to adapt to terminal devices with different display characteristics, ensuring that the displayed picture is not distorted. Therefore, it overcomes the problem of directly using the gamma curve to map the SDR video to the terminal device in the existing technology, resulting in poor display effect of the terminal device.

[0254] According to an exemplary embodiment, a non-transitory computer-readable storage medium storing computer instructions is shown, wherein the computer instructions are used to enable the computer to execute the above method.

[0255] According to an exemplary embodiment, a computer program product includes a computer program. When the computer program is executed by a processor, the method described above is implemented.

[0256] The computer-readable storage medium disclosed in this embodiment includes, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or components, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or device.

[0257] It is understandable that the technical solution provided by this embodiment, because the video images on the encoder side are lossless and uncompressed, by calculating the brightness characteristics of each video frame on the encoder side, can ensure maximum alignment with the actual state of each video frame, reduce calculation errors, and provide accurate data support for the mapping curve constructed by the decoder side. At the same time, because the calculation step of the brightness characteristics is transferred to the encoder side, the computational pressure on the decoder side is also reduced, allowing the decoder side to construct the dynamic mapping curve more quickly and display high-quality video images with lossless tones more quickly.

[0258] The decoding end improves the display effect of the terminal device when displaying SDR video by improving the mapping curve construction method. The specific reasons are: compared with the existing technology, the mapping curve of each frame of video image is dynamically constructed in real time, rather than a single unchanging gamma curve; in addition, the existing technology does not consider the brightness characteristics of the video image to be mapped and the display characteristics of the terminal device when mapping. The technical solution provided by the present invention takes into account the display characteristics of the frame of video image before mapping and the display characteristics of the terminal device after mapping when constructing the mapping curve of each frame of video image. This allows video images containing different picture tones to adapt to terminal devices with different display characteristics, ensuring that the displayed picture is not distorted. Therefore, it overcomes the problem of directly using the gamma curve to map the SDR video to the terminal device in the existing technology, resulting in poor display effect of the terminal device.

[0259] It can be understood that the same or similar parts of the above embodiments can be referenced to each other, and the contents not described in detail in some embodiments can refer to the same or similar contents in other embodiments.

[0260] It should be noted that, in the description of the present invention, the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance. In addition, in the description of the present invention, unless otherwise specified, the meaning of "plurality" is at least two.

[0261] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention pertain.

[0262] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0263] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0264] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing module, or each unit may exist physically separately, or two or more units may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or in the form of software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

[0265] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0266] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0267] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A video data processing method, applicable to a decoding end, characterized in that: include: Extracting the brightness feature of each frame of video image from the video stream of the standard dynamic range video sent by the encoder; Get the display characteristics of the terminal device; Normalizing the brightness feature to obtain a normalized brightness feature; Determining, based on the display characteristics and the normalized brightness characteristics of the current frame video image, a function expression for an S-shaped mapping curve for a brightness region of the current frame video image and a function expression for a linear mapping line for a dark region of the current frame video image; determining a piecewise function consisting of the linear mapping line and the S-shaped mapping curve as a mapping curve for the current frame video image; the mapping curve being used to represent a mapping relationship between an original RGB electrical signal and a corrected RGB electrical signal; The original RGB electrical signal is an RGB electrical signal obtained by converting the YUV electrical signal of the video image to be displayed; The modified RGB electrical signal is an RGB electrical signal adapted to the display characteristics of the terminal device; The brightness feature includes a boundary value between a bright area and a dark area. The function expression for determining a linear mapping line of the dark area of ​​the current frame video image based on the display feature and the normalized brightness feature of the current frame video image includes: Determining a lower limit value of the slope of the linear mapping line according to the display characteristics and the normalized brightness characteristics, and compensating the lower limit value of the slope with a preset slope compensation value; Determining the vertical coordinate corresponding to the normalized demarcation value on the S-shaped mapping curve; Determine the ratio of the ordinate to the normalized cutoff value as a reference slope; comparing the lower limit of the compensated slope with the reference slope, and determining the minimum value of the two as the slope of the linear mapping line; According to the slope and the normalized demarcation value, a function expression of a linear mapping line of a dark area of ​​the current frame video image is determined.

2. The method according to claim 1, characterized in that The function expression of determining the S-shaped mapping curve of the brightness area of ​​the current frame video image according to the display characteristics and the normalized brightness characteristics of the current frame video image includes: Select any one of the preset standard S-shaped mapping curves; The parameter values ​​of the curve are calculated according to a pre-generated parameter calculation formula, wherein the parameter calculation formula is related to the display characteristics and the normalized brightness characteristics of the current frame video image.

3. The method according to claim 2, characterized in that The parameter calculation formula is pre-generated by the following method, including: Acquire multiple sets of display features and normalized brightness features, repeat the following operations for each set of display features and normalized brightness features until the output S-shaped mapping curve has a curve shape that meets the expected curve shape under different display features and normalized brightness features, and determine the parameter calculation formula at this time as the parameter calculation formula of the current standard S-shaped mapping curve; the operations include: For any standard S-shaped mapping curve, a set of display features and normalized brightness features are substituted into a preset parameter calculation formula to obtain a set of parameter values; Substituting the parameter value into the function expression of the current standard S-shaped mapping curve, and outputting an S-shaped mapping curve; The parameter calculation formula is adjusted until the curve shape of the output S-shaped mapping curve meets the curve shape of the S-shaped mapping curve of the expected curve shape.

4. The method according to claim 2, characterized in that Calculating the parameter value of the curve according to the pre-generated parameter calculation formula includes: If the function expression of the selected standard S-shaped mapping curve is: F(L)=((mp*L) / ((mp-1)*L+1)) 3.2+deltaM (1), L represents the pixel value after normalization of the original RGB electrical signal, L=i / 255, F(L) represents the pixel value after normalization of the modified RGB electrical signal; and the brightness characteristics include at least: brightness intensity, average brightness, and the boundary value between the bright area and the dark area; the display characteristics include at least: maximum screen brightness, minimum screen brightness and screen resolution, then: The specific value of the parameter mp is solved according to the following formula (3): strengthL' represents the normalized brightness intensity, and mpMax represents the upper limit of mp; Solve the parameter deltaM according to the following formula (5): MinDisplay represents the minimum screen brightness, RegionD' represents the boundary value between the normalized bright area and the dark area, and EOTF709() represents the electro-optical transfer function.

5. The method according to claim 4, characterized in that The upper limit value mpMax of the parameter mp is solved according to the following formula (2): MaxDisplay indicates the maximum screen brightness; and / or, The compensation value Δmp of the parameter mp is solved according to the following formula (4): AvergLuma' represents the normalized average brightness; And update the specific value of mp to: AvergLuma' represents the normalized average brightness.

6. The method according to claim 1, characterized in that The brightness feature includes average brightness, and determining the lower limit of the slope of the linear mapping line according to the display feature and the normalized brightness feature includes: determining a limit value of the slope of the linear mapping line according to the display characteristics; If the normalized average brightness is less than a first threshold, determining the lower limit of the slope to be a constant; If the normalized average brightness is greater than or equal to a first threshold and less than a second threshold, determining a lower limit of the slope as a function related to a limit value of the slope; and the second threshold is greater than the first threshold; If the normalized average brightness is greater than a second threshold, the lower limit value of the slope is determined as the limit value of the slope.

7. The method according to claim 6, characterized in that The display characteristics include: maximum screen brightness and minimum screen brightness. Determining the limit value of the slope of the linear mapping line according to the display characteristics includes: Calculate the product of the reciprocal of the maximum screen brightness and a preset fixed constant; A preset power of the product is obtained, and the obtained result is determined as a limit value of the slope of the linear mapping line, wherein the preset power is related to the minimum screen brightness.

8. The method according to any one of claims 6 to 7, characterized in that Also includes: Find a point on the S-shaped mapping curve such that the slope of the line connecting the point and the origin is equal to the slope of the linear mapping line; Update the horizontal coordinate of the point to the normalized boundary value between the bright area and the dark area; The function expression of the linear mapping line is updated according to the slope and the updated boundary value between the bright area and the dark area.

9. A video data processing method, applicable to an encoding end, characterized in that: include: For the standard dynamic range video to be displayed, the brightness characteristics of each frame of the video image are calculated; Writing the brightness feature into metadata of each frame of video image; Encoding the metadata and each frame of video image to form a video stream; Sending the video stream to a decoding end, so that the decoding end constructs a mapping curve for each frame of the video image according to the brightness characteristics and the display characteristics of the terminal device, wherein the mapping curve is used to represent the mapping relationship between the original RGB electrical signal and the corrected RGB electrical signal; The original RGB electrical signal is an RGB electrical signal obtained by converting the YUV electrical signal of the video image to be displayed; the modified RGB electrical signal is an RGB electrical signal adapted to the display characteristics of the terminal device; The decoding end constructs a mapping curve for each frame of the video image according to the brightness characteristics and the display characteristics of the terminal device, including: Determining, based on the display characteristics and the brightness characteristics of the current frame video image, a function expression of an S-shaped mapping curve for a brightness region of the current frame video image and a function expression of a linear mapping line for a dark region of the current frame video image; and determining a piecewise function composed of the linear mapping line and the S-shaped mapping curve as a mapping curve for the current frame video image; The brightness feature includes a boundary value between a bright area and a dark area. The function expression for determining a linear mapping line of the dark area of ​​the current frame video image based on the display feature and the brightness feature of the current frame video image includes: Determining a lower limit value of the slope of the linear mapping line according to the display characteristics and the brightness characteristics, and compensating the lower limit value of the slope with a preset slope compensation value; Determining the vertical coordinate corresponding to the normalized demarcation value on the S-shaped mapping curve; Determine the ratio of the ordinate to the normalized cutoff value as a reference slope; comparing the lower limit of the compensated slope with the reference slope, and determining the minimum value of the two as the slope of the linear mapping line; According to the slope and the normalized demarcation value, a function expression of a linear mapping line of a dark area of ​​the current frame video image is determined.

10. The method according to claim 9, characterized in that The display characteristics include: screen resolution; the calculation of the brightness characteristics of each frame of the standard dynamic range video to be displayed includes: Calculate the histogram distribution HistY of the brightness component Y of each frame of video image, and determine the middle gray value of the histogram distribution HistY; Counting the expected value of pixels whose brightness values ​​are greater than the middle gray value in the histogram distribution HistY, and recording it as brightness intensity; Convert the electrical signal of the video image into an optical signal according to a preset electro-optical conversion function; Calculate the sum of the products of all brightness values ​​of the light signal and the number of pixels corresponding to each; Determine the ratio of the product and the screen resolution as the average brightness; Counting the expected number of pixels with brightness values ​​between zero and mid-gray values ​​in the histogram distribution HistY, recorded as average darkness; Calculating an average value of the average darkness and the middle gray value, and determining the average value as a dividing value between the bright area and the dark area; The brightness intensity, the average brightness, and the boundary value between the bright area and the dark area are used as brightness features.

11. The method according to claim 10, characterized in that Determining the middle gray value of the histogram distribution HistY includes: The middle gray value of the histogram distribution HistY is taken as a fixed value; or, The square root of the product of the maximum brightness value and the minimum brightness value in the histogram distribution HistY is determined as the middle gray value of the histogram distribution HistY.

12. A video display method, applicable to a decoding end, characterized in that: include: Extracting the YUV electrical signal of each frame of video image from the video stream of the standard dynamic range video sent by the encoding end, and converting the YUV electrical signal into the original RGB electrical signal; converting the original RGB electrical signal into a modified RGB electrical signal according to a mapping curve constructed by the video data processing method according to any one of claims 1 to 8; According to a preset electro-optical conversion function, the corrected RGB electrical signal is converted into an RGB optical signal for display on a terminal device.

13. A video display method, applicable to a decoding end, characterized in that: include: Extracting the YUV electrical signal of each frame of video image from the video stream of the standard dynamic range video sent by the encoding end, and converting the YUV electrical signal into the original RGB electrical signal; Converting the original RGB electrical signal into a modified RGB electrical signal according to the mapping curve, so that the modified RGB electrical signal is adapted to the RGB electrical signal of the display characteristic of the terminal device; The mapping curve is constructed according to the video data processing method according to any one of claims 1 to 8; Determine the brightness adjustment coefficient according to the corrected RGB electrical signal; According to the brightness adjustment coefficient, the RGB optical signal converted from the original RGB electrical signal is subjected to hue correction, and the hue-corrected RGB optical signal is displayed on a terminal device.

14. The method according to claim 13, characterized in that The step of performing hue correction on the RGB light signal converted from the original RGB electrical signal according to the brightness adjustment coefficient includes: Performing brightness correction on the RGB optical signal converted from the original RGB electrical signal according to the brightness adjustment coefficient; Convert the brightness-corrected RGB optical signal into a brightness-corrected RGB electrical signal according to a preset photoelectric conversion function; Perform saturation correction on the RGB electrical signal after brightness correction; According to a preset electro-optical conversion function, the saturation-corrected RGB electrical signal is converted into an RGB optical signal to obtain a hue-corrected RGB optical signal.

15. The method according to claim 13, characterized in that Determining the brightness adjustment coefficient according to the corrected RGB electrical signal includes: Compare the original pixel maximum values ​​of the three channels R, G, and B in the original RGB electrical signal; The pixel maximum value obtained by mapping the original pixel maximum value through the mapping curve is recorded as the corrected pixel maximum value; The ratio of the corrected pixel maximum value to the original pixel maximum value is determined as the brightness adjustment coefficient.

16. The method according to claim 14, characterized in that The saturation correction of the brightness-corrected RGB electrical signal comprises: Determining a saturation adjustment coefficient and a saturation adjustment value of the RGB electrical signal after brightness correction; Subtracting the pixel values ​​of the R, G, and B channels of the brightness-corrected RGB electrical signal from the saturation adjustment value, and multiplying the obtained differences by the saturation adjustment coefficient; The obtained products are added to the saturation adjustment value respectively to obtain the pixel values ​​of the three channels R, G, and B after saturation correction.

17. The method according to claim 16, characterized in that The step of determining the saturation adjustment coefficient and the saturation adjustment value of the brightness-corrected RGB electrical signal includes: Determine the maximum value of the original pixel in the R, G, and B channels and the minimum value of the original pixel in the R, G, and B channels in the original RGB electrical signal, and record the difference between the two as a first difference; Determine the maximum pixel value of the R, G, and B channels and the minimum pixel value of the R, G, and B channels in the RGB electrical signal after brightness correction, and record the difference between the two as the second difference; determining a ratio of the first difference to the second difference as a saturation adjustment coefficient; The pixel values ​​of the R, G, and B channels of the RGB electrical signal after brightness correction are weighted and summed according to the preset weight coefficients, and the obtained result is determined as the saturation adjustment value.

18. A video data processing device, provided at a decoding end, characterized in that: include: An acquisition module is used to extract the brightness characteristics of each frame of video image from the video stream of the standard dynamic range video sent by the encoding end; It is also used to obtain the display characteristics of the terminal device; A normalization module, configured to normalize the brightness feature to obtain a normalized brightness feature; A construction module, configured to construct a mapping curve for each frame of video image based on the display characteristics and the normalized brightness characteristics, wherein the mapping curve is used to represent a mapping relationship between the original RGB electrical signal and the corrected RGB electrical signal; The original RGB electrical signal is an RGB electrical signal obtained by converting the YUV electrical signal of the video image to be displayed; The modified RGB electrical signal is an RGB electrical signal adapted to the display characteristics of the terminal device; The constructing a mapping curve for each frame of the video image according to the display characteristics and the normalized brightness characteristics includes: Determining, based on the display characteristics and the normalized brightness characteristics of the current frame video image, a function expression for an S-shaped mapping curve for a brightness region of the current frame video image and a function expression for a linear mapping line for a dark region of the current frame video image; and determining a piecewise function composed of the linear mapping line and the S-shaped mapping curve as a mapping curve for the current frame video image; The brightness feature includes a boundary value between a bright area and a dark area. The function expression for determining a linear mapping line of the dark area of ​​the current frame video image based on the display feature and the normalized brightness feature of the current frame video image includes: Determining a lower limit value of the slope of the linear mapping line according to the display characteristics and the normalized brightness characteristics, and compensating the lower limit value of the slope with a preset slope compensation value; Determining the vertical coordinate corresponding to the normalized demarcation value on the S-shaped mapping curve; Determine the ratio of the ordinate to the normalized cutoff value as a reference slope; comparing the lower limit of the compensated slope with the reference slope, and determining the minimum value of the two as the slope of the linear mapping line; According to the slope and the normalized demarcation value, a function expression of a linear mapping line of a dark area of ​​the current frame video image is determined.

19. A video data processing device, provided at an encoding end, characterized in that: include: A calculation module, for calculating the brightness characteristics of each frame of the standard dynamic range video to be displayed; A reading and writing module, used for writing the brightness feature into the metadata of each frame of video image; An encoding module, configured to encode the metadata and each frame of video image to form a video stream; a sending module, configured to send the video stream to a decoding end, so that the decoding end constructs a mapping curve for each frame of the video image based on the brightness characteristics and the display characteristics of the terminal device, wherein the mapping curve is used to represent the mapping relationship between the original RGB electrical signal and the corrected RGB electrical signal; The original RGB electrical signal is an RGB electrical signal obtained by converting the YUV electrical signal of the video image to be displayed; the modified RGB electrical signal is an RGB electrical signal adapted to the display characteristics of the terminal device; The decoding end constructs a mapping curve for each frame of the video image according to the brightness characteristics and the display characteristics of the terminal device, including: Determining, based on the display characteristics and the brightness characteristics of the current frame video image, a function expression of an S-shaped mapping curve for a brightness region of the current frame video image and a function expression of a linear mapping line for a dark region of the current frame video image; and determining a piecewise function composed of the linear mapping line and the S-shaped mapping curve as a mapping curve for the current frame video image; The brightness feature includes a boundary value between a bright area and a dark area. The function expression for determining a linear mapping line of the dark area of ​​the current frame video image based on the display feature and the brightness feature of the current frame video image includes: Determining a lower limit value of the slope of the linear mapping line according to the display characteristics and the brightness characteristics, and compensating the lower limit value of the slope with a preset slope compensation value; Determining the vertical coordinate corresponding to the normalized demarcation value on the S-shaped mapping curve; Determine the ratio of the ordinate to the normalized cutoff value as a reference slope; comparing the lower limit of the compensated slope with the reference slope, and determining the minimum value of the two as the slope of the linear mapping line; According to the slope and the normalized demarcation value, a function expression of a linear mapping line of a dark area of ​​the current frame video image is determined.

20. A video display device, provided at a decoding end, characterized in that: include: An extraction module is used to extract the YUV electrical signal of each frame of video image from the video stream of the standard dynamic range video sent by the encoding end, and convert the YUV electrical signal into the original RGB electrical signal; a correction module, configured to convert the original RGB electrical signal into a corrected RGB electrical signal according to a mapping curve, so that the corrected RGB electrical signal is adapted to an RGB electrical signal of a display characteristic of a terminal device; The mapping curve is constructed according to the video data processing method according to any one of claims 1 to 8; The conversion module is used to convert the corrected RGB electrical signal into an RGB optical signal according to a preset electro-optical conversion function for display on the terminal device.

21. An end-to-end video display system, characterized in that: include: An encoding end, configured to execute the method according to any one of claims 9 to 11; The decoding end is used to execute the method described in any one of claims 1 to 8 or any one of claims 12 to 17.

22. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 17.

23. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 17.

24. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method according to any one of claims 1 to 17.

Images