Video data processing method, video display method and device, electronic equipment and readable storage medium

Abstract

This invention relates to a video data processing method, a video display method and apparatus, an electronic device, and a readable storage medium. The method corrects the tone mapping curve dynamically generated by the decoding end by calculating the information loss value at the encoding end. Since the information loss value takes into account the tone loss of the video image frame to be displayed when displayed on the terminal device, the video image mapped by the tone mapping curve after correction by the information loss value can reduce the problem of poor visual effect caused by the mismatch between the upper limit brightness of the video image frame and the peak brightness of the terminal device.

Description

Technical Field

[0001] This invention relates to the field of image display technology, specifically to a video data processing method, a video display method and apparatus, an electronic device, and a readable storage medium. Background Technology

[0002] The past decade has seen rapid development in video-related hardware and software technologies. On one hand, video production and playback technologies have undergone significant updates, with video moving from Standard Dynamic Range (SDR) to High Dynamic Range (HDR) video. On the other hand, display material technology has also made significant progress, with LCD (Liquid Crystal Display) technology emerging as a prime example.

[0003] LCD displays and OLED (Organic Light Emitting Display) have completely dominated various terminal products.

[0004] However, hardware development lags slightly behind software development. Currently, the peak brightness of mainstream LCD and OLED displays can only reach 500 nits, which is significantly lower than the HDR standard's upper limit of 10,000 nits. To achieve perfect HDR effects on existing hardware, image tone mapping technology has been developed, which can solve the adaptation problem between high dynamic range and low dynamic range.

[0005] There are two types of tone mapping techniques: static tone mapping and dynamic tone mapping. Static tone mapping refers to a fixed shape of the mapping curve for a given video sequence, while dynamic tone mapping dynamically generates the mapping curve based on the video scene and the display performance of the terminal device. Therefore, generally speaking, the effect of dynamic tone mapping is significantly better than that of static tone mapping. Currently, the mainstream international dynamic tone mapping standards include ST2094-20, ST2094-40, and T / UWA005.1-2022. However, the mapping curves corresponding to these standards still have problems in large-scale applications. In some scenarios, such as scenes with large overall brightness differences and rich details in the shadows, these tone mapping curves can smooth out the shadow details to some extent, resulting in a loss of image information and a poor visual effect. Summary of the Invention

[0006] To at least partially overcome the problems existing in related technologies, the present invention provides a video data processing method, a video display method and apparatus, an electronic device, and a readable storage medium to solve the problem that when dynamic tone mapping curves are used to map video images to be displayed in the prior art, there is loss of image information, resulting in poor visual effects.

[0007] According to a first aspect of the present invention, a video data processing method is provided, applicable to a decoding end, comprising:

[0008] The device receives a video stream sent by the encoding end and decodes video image frames from the video stream, as well as the brightness characteristics and information loss value of each video image frame. The information loss value is the estimated value of the color tone loss when the video image frame is displayed on the terminal device.

[0009] Obtain the display characteristics of the terminal device;

[0010] Based on the brightness and display characteristics, generate the tone mapping curve for the current video image frame;

[0011] Based on the information loss value, the tone mapping curve is corrected to obtain a corrected tone mapping curve. The corrected tone mapping curve is used to characterize the mapping relationship between the original RGB electrical signal and the corrected RGB electrical signal. The original RGB electrical signal is the RGB electrical signal obtained by converting the YUV electrical signal of the video image frame to be displayed, and the corrected RGB electrical signal is the RGB electrical signal adapted to the display characteristics of the terminal device.

[0012] According to a second aspect of the present invention, a video data processing method is provided, applicable to an encoding end, comprising:

[0013] For the video to be sent, calculate the brightness characteristics of each frame of the video image;

[0014] Determine the information loss value for each frame of video image, wherein the information loss value is an estimated value of the tone loss when the video image frame is displayed on the terminal device;

[0015] The brightness features and information loss values ​​are written into the metadata of each frame of video image;

[0016] The metadata and each frame of video image are encoded to form a video stream;

[0017] The video stream is sent to the decoding end, so that the decoding end can correct the tone mapping curve of the current video image frame according to the brightness characteristics and the estimated amount of tone loss. The corrected tone mapping curve is used to characterize the mapping relationship between the original RGB electrical signal and the corrected RGB electrical signal. The original RGB electrical signal is the RGB electrical signal obtained by converting the YUV electrical signal of the video image frame to be displayed, and the corrected RGB electrical signal is the RGB electrical signal adapted to the display characteristics of the terminal device.

[0018] According to a third aspect of the present invention, a video display method is provided, applicable to a decoding end, comprising:

[0019] Extract the YUV electrical signal of each frame of video image from the video stream sent from the encoding end, and convert the YUV electrical signal into the original RGB electrical signal;

[0020] The original RGB electrical signal is converted into a corrected RGB electrical signal according to the tone mapping curve, and the tone mapping curve is generated according to the video data processing method described above.

[0021] Based on a preset electro-optical conversion function, the corrected RGB electrical signal is converted into an RGB light signal for display on the terminal device.

[0022] According to a fourth aspect of the present invention, a video data processing apparatus is provided, disposed at a decoding end, comprising:

[0023] The decoding module is used to receive the video stream sent by the encoding end and decode the video image frames from the video stream, as well as the brightness characteristics and information loss value of each video image frame. The information loss value is the estimated value of the color tone loss when the video image frame is displayed on the terminal device.

[0024] The acquisition module is used to acquire the display characteristics of the terminal device;

[0025] The generation module is used to generate the tone mapping curve of the current video image frame based on the brightness characteristics and display characteristics.

[0026] The correction module is used to correct the tone mapping curve according to the information loss value to obtain a corrected tone mapping curve. The corrected tone mapping curve is used to characterize the mapping relationship between the original RGB electrical signal and the corrected RGB electrical signal. The original RGB electrical signal is the RGB electrical signal obtained by converting the YUV electrical signal of the video image frame to be displayed, and the corrected RGB electrical signal is the RGB electrical signal adapted to the display characteristics of the terminal device.

[0027] According to a fifth aspect of the present invention, a video data processing apparatus is provided, disposed at an encoding end, comprising:

[0028] The calculation module is used to calculate the brightness characteristics of each frame of the video to be sent.

[0029] It is also used to determine the information loss value of each video image frame, wherein the information loss value is an estimated value of the tonal loss of the video image frame when it is displayed on the terminal device;

[0030] The read / write module is used to write the brightness features and information loss values ​​into the metadata of each frame of video image;

[0031] The encoding module is used to encode the metadata and each frame of video image to form a video stream;

[0032] The sending module is used to send the video stream to the decoding end, so that the decoding end can correct the tone mapping curve of the current video image frame according to the brightness characteristics and the estimated value of tone loss. The corrected tone mapping curve is used to characterize the mapping relationship between the original RGB electrical signal and the corrected RGB electrical signal. The original RGB electrical signal is the RGB electrical signal obtained by converting the YUV electrical signal of the video image frame to be displayed, and the corrected RGB electrical signal is the RGB electrical signal adapted to the display characteristics of the terminal device.

[0033] According to a sixth aspect of the present invention, a video display system is provided, comprising:

[0034] The aforementioned video data processing apparatus is provided at the encoding end, and the aforementioned video data processing apparatus is provided at the decoding end.

[0035] According to a seventh aspect of the present invention, an electronic device is provided, comprising:

[0036] At least one processor; and

[0037] A memory communicatively connected to the at least one processor; wherein,

[0038] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method described above.

[0039] According to an eighth aspect of the present invention, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are configured to cause the computer to perform the method described above.

[0040] According to a ninth aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the above-described method.

[0041] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects:

[0042] The information loss value calculated at the encoding end is used to correct the tone mapping curve dynamically generated at the decoding end. Since the information loss value takes into account the tone loss of the video image frame when it is displayed on the terminal device, the video image mapped by the tone mapping curve after correction by the information loss value can reduce the problem of poor visual effect caused by the mismatch between the upper limit brightness of the video image frame and the peak brightness of the terminal device.

[0043] Furthermore, the video display method provided by this invention corrects the tone of the original RGB electrical signal by means of a modified tone mapping curve. Since the tone mapping curve takes into account the tone loss of the video image frame to be displayed when it is displayed on the terminal device, the modified RGB electrical signal is then converted into an optical signal by a preset electro-optical conversion function. The converted optical signal can be adapted to the display characteristics of the terminal device, reducing the problem of poor visual effect caused by the loss of image information when the video image frame is displayed on the terminal device due to the mismatch between the upper limit brightness of the video image frame and the peak brightness of the terminal device.

[0044] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0045] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0046] Figure 1 This is a flowchart illustrating a video data processing method according to an exemplary embodiment;

[0047] Figure 2 This is a schematic diagram illustrating an S-shaped mapping curve according to an exemplary embodiment;

[0048] Figure 3 This is a schematic diagram of an S-shaped mapping curve according to another exemplary embodiment;

[0049] Figure 4 This is a schematic diagram of a mapping curve according to an exemplary embodiment;

[0050] Figure 5 This is a schematic diagram illustrating the bitstream format of a video stream transmitted from an encoding end according to an exemplary embodiment;

[0051] Figure 6 This is a flowchart illustrating a video data processing method according to another exemplary embodiment;

[0052] Figure 7 This is a flowchart illustrating a video display method according to an exemplary embodiment;

[0053] Figure 8 This is a schematic block diagram illustrating a video data processing apparatus according to an exemplary embodiment;

[0054] Figure 9 This is a schematic block diagram illustrating a video data processing apparatus according to another exemplary embodiment;

[0055] Figure 10 This is a schematic block diagram illustrating a video display system according to an exemplary embodiment. Detailed Implementation

[0056] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0057] As described in the background section, when using dynamic tone mapping curves directly for video image mapping in the prior art, in certain scenarios, such as scenes with large overall brightness differences and rich details in the dark areas, the dark details will be smoothed out to some extent after processing by these tone mapping curves, resulting in a loss of image information and poor visual effects.

[0058] To address this technical problem, the present invention provides the following exemplary embodiments, which should be noted as follows:

[0059] 1. The "minimum screen brightness" mentioned in the embodiments of the present invention refers to the minimum screen brightness when playing SDR video when the terminal device does not implement the technical solution provided by the present invention; the "maximum screen brightness" refers to the maximum screen brightness that the terminal device can achieve when playing SDR video when it does not implement the technical solution provided by the present invention; and the "screen resolution" refers to the screen resolution that the terminal device comes with at the factory.

[0060] 2. The term "preset" mentioned in the embodiments of the present invention refers to the REC.709 standard, which is currently accepted by mainstream televisions and monitors.

[0061] Figure 1 This is a flowchart illustrating a video data processing method according to an exemplary embodiment. The method is applicable to the encoding end. See also... Figure 1 The method includes:

[0062] Step S11: Calculate the brightness features of each frame of the video to be sent;

[0063] Step S12: Determine the information loss value of each video frame, wherein the information loss value is the estimated value of the tone loss when the video frame is displayed on the terminal device;

[0064] Step S13: Write the brightness feature and information loss value into the metadata of each frame of video image;

[0065] Step S14: Encode the metadata and each frame of video image to form a video stream;

[0066] Step S15: Send the video stream to the decoding end so that the decoding end can correct the tone mapping curve of the current video image frame according to the brightness characteristics and the estimated amount of tone loss. The corrected tone mapping curve is used to characterize the mapping relationship between the original RGB electrical signal and the corrected RGB electrical signal. The original RGB electrical signal is the RGB electrical signal obtained by converting the YUV electrical signal of the video image frame to be displayed, and the corrected RGB electrical signal is the RGB electrical signal adapted to the display characteristics of the terminal device.

[0067] It should be noted that the technical solution provided in this embodiment is applicable to the encoding end, which is a terminal device with an image acquisition device, including but not limited to: mobile phones, tablets, laptops, calculators, and smartwatches, etc., and the applicable scenarios include but are not limited to:

[0068] 1. The SDR video at the encoding end is mapped to the CRT monitor at the decoding end for display;

[0069] 2. The SDR / HDR video at the encoding end is mapped to the LCD display at the decoding end for display;

[0070] 3. The SDR / HDR video at the encoding end is mapped to the OLED display at the decoding end for display;

[0071] 4. The SDR / HDR video at the encoding end is mapped to the minLED display at the decoding end for display.

[0072] It is understandable that SDR / HDR video playback technology currently relies on tone mapping curves. The technical solution provided in this embodiment can adaptively correct tone mapping curves to avoid the loss of detail information during the mapping process, thereby ensuring the playback effect of SDR / HDR videos.

[0073] In practice, the brightness characteristics include at least the brightness intensity StrengthL, the average brightness AvergLuma, and the boundary value between the bright and dark areas RegionD. The display characteristics include at least the maximum screen brightness MaxDisplay, the minimum screen brightness MinDisplay, and the screen resolution RES.

[0074] Step S11, "Calculate the luminance features of each frame of the video to be sent," includes at least:

[0075] Step 1) Calculate the luminance intensity of each frame of the video image, including:

[0076] Calculating the histogram distribution HistY of the luminance component Y in each frame of video image, and determining the mid-gray value MildGray of the histogram distribution HistY, includes:

[0077] 1) Set the mid-gray value of the histogram distribution HistY to a fixed value, for example, set MildGray = 118;

[0078] 2) The square root of the product of the maximum and minimum brightness values ​​in the histogram distribution HistY is determined as the mid-gray value of the histogram distribution HistY. For example, take... ;

[0079] Where maxY is the maximum brightness value in the histogram distribution HistY (i.e., the maximum value of the coordinate on the horizontal axis), and minY is the minimum brightness value in the histogram distribution HistY (i.e., the minimum value of the coordinate on the horizontal axis).

[0080] Calculate the expected value of pixels in the histogram distribution HistY where the luminance value i is greater than the mid-gray value MildGray, denoted as the luminance intensity StrengthL, specifically as follows:

[0081] StrengthL=∑ i＞MildGray HistY[i]*i / ∑ i＞MildGray HistY[i].

[0082] Step 2) Calculate the average brightness of each frame of the video image, including:

[0083] Based on a preset electro-optical conversion function, the electrical signals of the video image are converted into optical signals;

[0084] Luma[i] = EOTF709(i / 255); i / 255 represents the pixel value of the normalized electrical signal, and Luma[i] represents the pixel value after conversion to an optical signal; the function EOTF709() represents the electro-optical conversion function specified in the REC.709 standard, specifically:

[0085] L represents the input variable, L = i / 255.

[0086] Iterate through all the brightness values ​​of the light signal and calculate the sum of the products of each brightness value Luma[i] and the number of pixels HistY[i] in the histogram distribution HistY;

[0087] The ratio of the sum of the products to the screen resolution RES of the terminal device is determined as the average brightness AvergLuma, specifically:

[0088] AvergLuma=∑HistY[i]*Luma[i] / RES.

[0089] Step 3) Calculate the boundary value between the bright and dark areas of each frame of video image, including:

[0090] Calculate the expected value of pixels in the histogram distribution HistY whose brightness value i is between zero and medium gray [0~MildGray], and denote it as the mean darkness MeanDark;

[0091] MeanDark=∑ i∈[0,MildGray] HistY[i]*i / ∑ i∈[0,MildGray] HistY[i]

[0092] Calculate the average of the average darkness and the average gray value, and determine the average value as the boundary value RegionD, specifically as follows:

[0093]

[0094] Step S12, "determining the information loss value of each video frame, wherein the information loss value is an estimated value of the tone loss when the video frame is displayed on the terminal device," includes:

[0095] Step 1) Convert the YUV electrical signal of each frame of video image into an RGB electrical signal, including:

[0096] According to the following conversion formula (1) for converting YUV electrical signals to RGB electrical signals, the YUV electrical signals are converted into the original RGB signals:

[0097]

[0098] According to the following conversion formula (2) for converting YUV electrical signals to RGB electrical signals, the YUV electrical signals are converted into the original RGB signals:

[0099]

[0100] Step 2) Obtain the default values ​​of the terminal device's display characteristics. Here, the default values ​​of the display characteristics can be the default values ​​of the maximum screen brightness, MaxDisplay.

[0101] For HDR video, the default value is 500 nits; for SDR video, the default value is 300 nits.

[0102] Step 3) Generate a tone mapping curve based on the default values ​​of the brightness and display features, including:

[0103] Based on the default values ​​of the display features and the normalized brightness features of the current video image frame, determine the function expression of the S-shaped mapping curve of the brightness region of the current video image frame, and the function expression of the linear mapping line of the dark region of the current video image frame.

[0104] The piecewise function composed of the linear mapping line and the S-shaped mapping curve is determined as the tone mapping curve of the current video image frame.

[0105] The functional expression for determining the S-shaped mapping curve of the brightness region of the current frame video image based on the display features and the normalized brightness features of the current frame video image includes:

[0106] Choose any one of the preset standard S-shaped mapping curves;

[0107] Calculate the parameter values ​​of the curve based on the pre-generated parameter calculation formula, including:

[0108] (1) See Figure 2 ( Figure 2 In the horizontal axis, 1 represents a pixel value of 255, 0.2 represents a pixel value of 0.2 * 255, and so on; in the vertical axis, 1 represents a pixel value of 255, 0.1 represents a pixel value of 0.1 * 255, and so on. If the function expression of the selected standard S-curve is:

[0109] F(L)=((mp*L) / ((mp-1)*L+1)) 3.2+deltaM (3), L represents the pixel value after normalization of the original RGB electrical signal, L=i / 255, F(L) represents the pixel value after normalization of the corrected RGB electrical signal; the upper limit value of parameter mp is solved according to the following formula (4):

[0110] MaxDisplay represents the default value for the maximum screen brightness;

[0111] The specific value of parameter mp can be obtained using the following formula (5):

[0112]

[0113] strengthL' represents the normalized luminance intensity, and mpMax represents the upper limit of mp.

[0114] Update the specific value of parameter mp according to the following formula (6):

[0115] (6), AvergLuma' represents the normalized average brightness;

[0116] The parameter deltaM is solved according to the following formula (7):

[0117]

[0118] MinDisplay represents the default value for minimum screen brightness, RegionD' represents the normalized boundary value between bright and dark areas, and EOTF709() represents the electro-optical conversion function specified in the REC.709 standard.

[0119] See Figure 3 ( Figure 3 In the horizontal axis, 1 represents a pixel value of 255, 0.2 represents a pixel value of 0.2 * 255, and so on; in the vertical axis, 1 represents a pixel value of 255, 0.1 represents a pixel value of 0.1 * 255, and so on. If the function expression of the selected standard S-curve is:

[0120] F(L)=(3+deltaP)*L 2 *(1-L) 2 +p*L 3 *(1-L)+L 4 (8), L represents the pixel value after normalization of the original RGB electrical signal, L=i / 255, F(L) represents the pixel value after normalization of the corrected RGB electrical signal; the upper limit value of parameter p is solved according to the following formula (9):

[0121] MaxDisplay represents the default value for the maximum screen brightness;

[0122] The specific value of parameter p can be determined using the following formula (10):

[0123] strengthL' represents the normalized luminance intensity, and pMax represents the upper limit of p;

[0124] Update the specific value of parameter p according to the following formula (11):

[0125]

[0126] (11), AvergLuma' represents the normalized average brightness;

[0127] The parameter deltaP is solved according to the following formula (12):

[0128]

[0129] MinDisplay represents the default value for minimum screen brightness, RegionD' represents the normalized boundary value between bright and dark areas, and EOTF709() represents the electro-optical conversion function specified in the REC.709 standard.

[0130] In practice, based on the default values ​​of the display features and the normalized brightness features of the current video image frame, the functional expression for the linear mapping line of the dark region of the current video image frame is determined, including:

[0131] Calculate the reciprocal of the default value of the maximum screen brightness MaxDisplay, and multiply it by a preset fixed constant (e.g., 100);

[0132] Calculate the product to a preset power, for example, to the power of C0, and determine the result as the limit value of the slope of the linear mapping line, SlopeLimit. The preset power C0 is related to the default value of the minimum screen brightness, specifically:

[0133] The value of C0 is related to the default value of the minimum screen brightness, MinDisplay:

[0134]

[0135] Specifically, the ordinate corresponding to the normalized boundary value on the S-shaped mapping curve of the bright part is determined as follows:

[0136] Substitute RegionD' into the above formula (3) or formula (8) to obtain the value F(RegionD').

[0137] The ratio F(RegionD') / (RegionD') of the ordinate to the normalized boundary value is determined as the reference slope.

[0138] By comparing the limiting value of the slope with the reference slope, the minimum value of the two is determined as the slope of the linear mapping line in the dark area, specifically:

[0139]

[0140] Based on the slope and the normalized boundary value, the functional expression for the linear mapping line of the dark region of the current frame video image is determined as follows:

[0141] TM(L)=Slope*L, L≤RegionD′.

[0142] See Figure 4 ( Figure 4In the horizontal axis, 1 represents a pixel value of 255, 0.2 represents a pixel value of 0.2*255, and so on; in the vertical axis, 1 represents a pixel value of 255, 0.1 represents a pixel value of 0.1*255, and so on. Figure 4 The tone mapping curve shown in the example has an inflection point at an x-coordinate of 0.2. The straight line corresponding to the x-coordinate less than or equal to 0.2 is the linear mapping line for the dark region, and the curve corresponding to the x-coordinate greater than 0.2 is the S-shaped mapping curve for the bright region. The piecewise function composed of the linear mapping line and the S-shaped mapping curve is determined as the tone mapping curve TM(L) of the current frame video image, specifically:

[0143] L represents the pixel value after normalization of the original RGB electrical signal, L = i / 255.

[0144] Step 4) Correct the YUV and RGB electrical signals according to the tone mapping curve, including:

[0145] Based on the tone mapping curve, the luminance component Y of the YUV electrical signal is mapped to the luminance component Y' (specifically, by searching...). Figure 4 The ordinate of the point whose horizontal coordinate is equal to the normalized luminance component Y / 255 is the mapped luminance component Y'.

[0146] According to the hue mapping curve, the red component R of the RGB electrical signal is mapped to a red component R'; the green component G of the RGB electrical signal is mapped to a green component G'; and the blue component B of the RGB electrical signal is mapped to a blue component B' (specifically, searching...). Figure 4 The ordinate of the point whose x-coordinate is equal to the normalized green component G / 255 is the mapped green component G'. The other components, red R' and blue B', are obtained in the same way as green component G', and will not be repeated here.

[0147] Step 5) Calculate the information entropy loss value of luminance based on the YUV electrical signals before and after correction, including:

[0148] Calculate the information entropy of the histogram distribution HistY of the luminance component Y, and the information entropy of the histogram distribution HistYtm of the luminance component Y'.

[0149] The absolute value of the difference between the information entropy of HistY and the information entropy of HistYtm is denoted as the information entropy loss value of brightness.

[0150] Step 6) Calculate the information entropy loss value of chromaticity based on the RGB electrical signals before and after correction, including:

[0151] Calculate the information entropy of the histogram distribution HistR of the red component R and the information entropy of the histogram distribution HistRtm of the red component R' respectively, and denote the absolute value of the difference between the information entropy of HistR and the information entropy of HistRtm as the information entropy loss value of the red component R.

[0152] Calculate the information entropy of the histogram distribution HistG of the green component G and the information entropy of the histogram distribution HistGtm of the green component G' respectively, and denote the absolute value of the difference between the information entropy of HistG and the information entropy of HistGtm as the information entropy loss value of the green component G.

[0153] Calculate the information entropy of the histogram distribution HistB of the blue component B and the information entropy of the histogram distribution HistBtm of the blue component B' respectively, and denote the absolute value of the difference between the information entropy of HistB and the information entropy of HistBtm as the information entropy loss value of the blue component B.

[0154] The sum of the information entropy loss values ​​of the red component R, the green component G, and the blue component B is denoted as the chromaticity information entropy loss value.

[0155] Step 7) Determine the information loss value for each frame of video image based on the information entropy loss value, including:

[0156] For each frame of video image, the information entropy loss value of luminance and the information entropy loss value of chrominance of that frame are summed, and the summation result is compared with a preset loss coefficient (the preset loss coefficient is set according to empirical values ​​or experimental data, for example, 0.1).

[0157] Multiply them, and determine the product as the amount of information loss of the video image in that frame.

[0158] Steps 5) to 7) above can be expressed mathematically as follows:

[0159]

[0160] Where deltaInfo represents the amount of information loss;

[0161] X = Y, R, G, B, Ytm, Rtm, Gtm, Btm; En(HistX) represents the information entropy, abs() represents the absolute value, Res represents the total number of pixels in the current frame of the video image, and N represents the maximum pixel value in the current frame of the video image. For HDR video, N = 1023; for SDR video, N = 255. L represents the normalized pixel value. For SDR video, L = i / 255; for HDR video, L = i / 1023, where i represents the pixel value.

[0162] Step S13, "writing the brightness feature and information loss value into the metadata of each frame of video image," specifically means:

[0163] The brightness characteristics and information loss values ​​are converted to int16 integers and then written into the metadata.

[0164] In step S14, "the metadata and each frame of video image are encoded to form a video stream".

[0165] See Figure 5 , Figure 5 An example of a video stream sent by the encoding end is given. In this video stream, the first symbol SPS stores the global parameters of the encoded video image; the second symbol PPS stores the parameters related to the overall image. PPS is usually stored together with SPS in the file header of the video file; the third symbol IDR is used to store the set of decoding parameters; the fourth symbol SLICE is used to store frame encoding information; the fifth symbol SEI is used to store supplementary information of the video stream, that is, the metadata mentioned in this embodiment; ... the subsequent symbols are just repetitions of SLICE and SEI, and will not be described in detail.

[0166] After the encoder calculates the luminance features, it writes them to... Figure 5 The fifth code element, SEI, is then encoded together with the data from other code elements into a video stream.

[0167] It is understood that the technical solution provided in this embodiment, since the video image at the encoding end is an uncompressed and untransmitted lossless video image, can ensure maximum accuracy in matching the true state of each video image by calculating the estimated amount of tone loss for each frame at the encoding end, reducing calculation errors and providing accurate data support for the decoding end to correct the tone mapping curve. At the same time, since the calculation step of the estimated amount of tone loss is transferred to the encoding end, it also reduces the computational burden on the decoding end, enabling the decoding end to construct a dynamic tone mapping curve more quickly and display high-quality, tone-lossless video images more rapidly.

[0168] Figure 6 This is a flowchart illustrating a video data processing method according to another exemplary embodiment, applicable to the decoding end. See also... Figure 6 The method includes:

[0169] Step S21: Receive the video stream sent by the encoding end, and decode the video image frames from the video stream, as well as the brightness characteristics and information loss value of each video image frame. The information loss value is the estimated value of the color tone loss when the video image frame is displayed on the terminal device.

[0170] Step S22: Obtain the display characteristics of the terminal device;

[0171] Step S23: Generate the tone mapping curve of the current video image frame based on the brightness characteristics and display characteristics;

[0172] Step S24: Based on the information loss value, the tone mapping curve is corrected to obtain a corrected tone mapping curve. The corrected tone mapping curve is used to characterize the mapping relationship between the original RGB electrical signal and the corrected RGB electrical signal. The original RGB electrical signal is the RGB electrical signal obtained by converting the YUV electrical signal of the video image frame to be displayed, and the corrected RGB electrical signal is the RGB electrical signal adapted to the display characteristics of the terminal device.

[0173] It should be noted that the technical solution provided in this embodiment is applicable to the decoding end. The decoding end includes: a computer (including a server), and / or a mobile terminal (including but not limited to: smartphones, tablets, VR headsets, VR glasses, etc.), and applicable scenarios include but are not limited to:

[0174] 1. The SDR video at the encoding end is mapped to the CRT monitor at the decoding end for display;

[0175] 2. The SDR / HDR video at the encoding end is mapped to the LCD display at the decoding end for display;

[0176] 3. The SDR / HDR video at the encoding end is mapped to the OLED display at the decoding end for display;

[0177] 4. The SDR / HDR video at the encoding end is mapped to the minLED display at the decoding end for display.

[0178] It is understandable that SDR / HDR video playback technology currently relies on tone mapping curves. The technical solution provided in this embodiment can adaptively correct tone mapping curves to avoid the loss of detail information during the mapping process, thereby ensuring the playback effect of SDR / HDR videos.

[0179] In practice, the brightness characteristics include at least the brightness intensity StrengthL, the average brightness AvergLuma, and the boundary value between the bright and dark areas RegionD. The display characteristics include at least the maximum screen brightness MaxDisplay, the minimum screen brightness MinDisplay, and the screen resolution RES.

[0180] Step S23, "generating the tone mapping curve of the current video image frame based on the brightness features and display features," includes:

[0181] Based on the display characteristics and the normalized brightness characteristics of the current video image frame, determine the function expression of the S-shaped mapping curve of the brightness region of the current video image frame, and the function expression of the linear mapping line of the dark region of the current video image frame.

[0182] The piecewise function composed of the linear mapping line and the S-shaped mapping curve is determined as the tone mapping curve of the current video image frame.

[0183] It should be noted that the method for generating tone mapping curves at the decoding end is the same as that at the encoding end. The only difference is that the encoding end uses the default value of the display characteristics of the terminal device, while the decoding end uses the actual value of the display characteristics of the terminal device. All other steps are the same and will not be repeated here.

[0184] Step S24, "correcting the tone mapping curve based on the information loss value," includes:

[0185] Step 1) Based on the information loss value, correct the slope of the linear mapping line in the dark area of ​​the current video image frame, including:

[0186] If the maximum screen brightness is equal to the default value of the maximum screen brightness, the slope of the linear mapping line is compensated with the information loss value, and the compensated slope is updated to the slope of the linear mapping line in the dark area of ​​the current video image frame.

[0187] If the maximum screen brightness is not equal to the default value of the maximum screen brightness, the information loss value is adjusted according to the maximum screen brightness (the information loss value is adjusted by a preset adjustment function, which is a linear or nonlinear function with the information loss quantification value and the maximum screen brightness as independent variables and the adjusted information loss value as the dependent variable). The adjusted information loss value is used to compensate for the slope of the linear mapping line, and the compensated slope is updated to the slope of the linear mapping line in the dark area of ​​the current video image frame.

[0188] like Figure 5 As shown, if the VUI parameter value of the first symbol SPS in the video stream received by the decoding end is equal to 16 or 18, the decoding end defaults to the currently received video stream being an HDR video stream, and the default value of the maximum screen brightness is 500; if the VUI parameter value of the first symbol SPS in the video stream received by the decoding end is equal to 1 or empty, the decoding end defaults to the currently received video stream being an SDR video stream, and the default value of the maximum screen brightness is 300.

[0189] For HDR video, MaxDisplay = 500; for SDR video, MaxDisplay = 300, SlopeN = Slope + deltaInfoN, where SlopeN is the slope of the linear mapping line of the updated dark region, Slope is the slope of the linear mapping line of the dark region before the update, and deltaInfoN is the information loss value, deltaInfo < 1.

[0190] If MaxDisplay≠500 for HDR video and MaxDisplay≠300 for SDR video, SlopeN=Slope+deltaInfoN', where SlopeN is the slope of the linear mapping line of the dark region after the update, Slope is the slope of the linear mapping line of the dark region before the update, and deltaInfoN' is the adjusted information loss value.

[0191] or,

[0192]

[0193] The F'() preset adjustment function is Y=500 for HDR video and Y=300 for SDR video.

[0194] Step 2) Based on the corrected slope, update the boundary value between the bright and dark areas of the current video image frame, including:

[0195] Find a point on the S-shaped mapping curve such that the slope of the line connecting this point and the origin is equal to the corrected slope, i.e., satisfying: F() is the function expression for the S-shaped mapping curve in the tone mapping curve generated in step S23;

[0196] Update the x-coordinate RegionDnew of this point to the boundary value between the bright and dark regions of the current video image frame;

[0197] Based on the corrected slope SlopeN and the updated boundary value RegionDnew between the bright and dark regions, the function expression of the linear mapping line is updated, i.e.:

[0198] TM(L)=SlopeN*L,L≤RegionDnew,L represents the pixel value after normalization of the original RGB electrical signal,L=i / 255.

[0199] It is understood that the technical solution provided in this embodiment corrects the tone mapping curve dynamically generated by the decoding end by calculating the information loss value at the encoding end. Since the information loss value takes into account the tone loss of the video image frame to be displayed when it is displayed on the terminal device, the video image mapped by the tone mapping curve after correction by the information loss value can reduce the problem of poor visual effect caused by the mismatch between the upper limit brightness of the video image frame and the peak brightness of the terminal device when the video image frame is displayed on the terminal device.

[0200] Figure 7 This is a flowchart illustrating a video display method according to an exemplary embodiment. The method is applicable to the decoding end. See also... Figure 7 The method includes:

[0201] Step S31: Extract the YUV electrical signal of each frame of video image from the video stream sent from the encoding end, and convert the YUV electrical signal into the original RGB electrical signal;

[0202] Step S32: Convert the original RGB electrical signal into a corrected RGB electrical signal according to the tone mapping curve, wherein the tone mapping curve is generated according to the video data processing method described above;

[0203] Step S33: According to the preset electro-optic conversion function, the corrected RGB electrical signal is converted into an RGB light signal for display on the terminal device.

[0204] It should be noted that the technical solution provided in this embodiment is applicable to the decoding end. The decoding end includes: a computer (including a server), and / or a mobile terminal (including but not limited to: smartphones, tablets, VR headsets, VR glasses, etc.), and applicable scenarios include but are not limited to:

[0205] 1. The SDR video at the encoding end is mapped to the CRT monitor at the decoding end for display;

[0206] 2. The SDR / HDR video at the encoding end is mapped to the LCD display at the decoding end for display;

[0207] 3. The SDR / HDR video at the encoding end is mapped to the OLED display at the decoding end for display;

[0208] 4. The SDR / HDR video at the encoding end is mapped to the minLED display at the decoding end for display.

[0209] It is understandable that SDR / HDR video playback technology currently relies on tone mapping curves. The technical solution provided in this embodiment can adaptively correct tone mapping curves to avoid the loss of detail information during the mapping process, thereby ensuring the playback effect of SDR / HDR videos.

[0210] It is understood that the technical solution provided in this embodiment corrects the tone of the original RGB electrical signal by modifying the tone mapping curve. The tone mapping curve takes into account the tone loss of the video image frame when it is displayed on the terminal device. Therefore, after converting the modified RGB electrical signal into an optical signal by a preset electro-optical conversion function, the converted optical signal can be adapted to the display characteristics of the terminal device. This reduces the problem of poor visual effect caused by the loss of image information when the video image frame is displayed on the terminal device due to the mismatch between the upper limit brightness of the video image frame and the peak brightness of the terminal device.

[0211] Figure 8 This is a schematic block diagram of a video data processing apparatus 100 according to an exemplary embodiment, such as... Figure 8 As shown, the device 100 is disposed at the encoding end, and the device 100 includes:

[0212] The calculation module 101 is used to calculate the brightness characteristics of each frame of the video to be transmitted.

[0213] It is also used to determine the information loss value of each video image frame, wherein the information loss value is an estimated value of the tonal loss of the video image frame when it is displayed on the terminal device;

[0214] The read / write module 102 is used to write the brightness features and information loss values ​​into the metadata of each frame of video image;

[0215] Encoding module 103 is used to encode the metadata and each frame of video image to form a video stream;

[0216] The sending module 104 is used to send the video stream to the decoding end, so that the decoding end can correct the tone mapping curve of the current video image frame according to the brightness characteristics and the estimated value of tone loss. The corrected tone mapping curve is used to characterize the mapping relationship between the original RGB electrical signal and the corrected RGB electrical signal. The original RGB electrical signal is the RGB electrical signal obtained by converting the YUV electrical signal of the video image frame to be displayed, and the corrected RGB electrical signal is the RGB electrical signal adapted to the display characteristics of the terminal device.

[0217] It should be noted that the technical solution provided in this embodiment is applicable to the encoding end, which is a terminal device with an image acquisition device, including but not limited to: mobile phones, tablets, laptops, calculators, and smartwatches, etc., and the applicable scenarios include but are not limited to:

[0218] 1. The SDR video at the encoding end is mapped to the CRT monitor at the decoding end for display;

[0219] 2. The SDR / HDR video at the encoding end is mapped to the LCD display at the decoding end for display;

[0220] 3. The SDR / HDR video at the encoding end is mapped to the OLED display at the decoding end for display;

[0221] 4. The SDR / HDR video at the encoding end is mapped to the minLED display at the decoding end for display.

[0222] It is understandable that SDR / HDR video playback technology currently relies on tone mapping curves. The technical solution provided in this embodiment can adaptively correct tone mapping curves to avoid the loss of detail information during the mapping process, thereby ensuring the playback effect of SDR / HDR videos.

[0223] It is understood that the technical solution provided in this embodiment, since the video image at the encoding end is an uncompressed and untransmitted lossless video image, can ensure maximum accuracy in matching the true state of each video image by calculating the estimated amount of tone loss for each frame at the encoding end, reducing calculation errors and providing accurate data support for the decoding end to correct the tone mapping curve. At the same time, since the calculation step of the estimated amount of tone loss is transferred to the encoding end, it also reduces the computational burden on the decoding end, enabling the decoding end to construct a dynamic tone mapping curve more quickly and display high-quality, tone-lossless video images more rapidly.

[0224] Figure 9 This is a schematic block diagram of a video data processing apparatus 200 according to an exemplary embodiment, such as... Figure 9 As shown, the device 200 is disposed at the decoding end, and the device 200 includes:

[0225] The decoding module 201 is used to receive the video stream sent by the encoding end, and decode the video image frames from the video stream, as well as the brightness characteristics and information loss value of each video image frame. The information loss value is the estimated value of the color tone loss when the video image frame is displayed on the terminal device.

[0226] The acquisition module 202 is used to acquire the display features of the terminal device;

[0227] The generation module 203 is used to generate the tone mapping curve of the current video image frame based on the brightness characteristics and display characteristics;

[0228] The correction module 204 is used to correct the tone mapping curve according to the information loss value to obtain a corrected tone mapping curve. The corrected tone mapping curve is used to characterize the mapping relationship between the original RGB electrical signal and the corrected RGB electrical signal. The original RGB electrical signal is the RGB electrical signal obtained by converting the YUV electrical signal of the video image frame to be displayed, and the corrected RGB electrical signal is the RGB electrical signal adapted to the display characteristics of the terminal device.

[0229] It should be noted that the technical solution provided in this embodiment is applicable to the decoding end. The decoding end includes: a computer (including a server), and / or a mobile terminal (including but not limited to: smartphones, tablets, VR headsets, VR glasses, etc.), and applicable scenarios include but are not limited to:

[0230] 1. The SDR video at the encoding end is mapped to the CRT monitor at the decoding end for display;

[0231] 2. The SDR / HDR video at the encoding end is mapped to the LCD display at the decoding end for display;

[0232] 3. The SDR / HDR video at the encoding end is mapped to the OLED display at the decoding end for display;

[0233] 4. The SDR / HDR video at the encoding end is mapped to the minLED display at the decoding end for display.

[0234] It is understandable that SDR / HDR video playback technology currently relies on tone mapping curves. The technical solution provided in this embodiment can adaptively correct tone mapping curves to avoid the loss of detail information during the mapping process, thereby ensuring the playback effect of SDR / HDR videos.

[0235] It is understood that the technical solution provided in this embodiment corrects the tone of the original RGB electrical signal by modifying the tone mapping curve. The tone mapping curve takes into account the tone loss of the video image frame when it is displayed on the terminal device. Therefore, after converting the modified RGB electrical signal into an optical signal by a preset electro-optical conversion function, the converted optical signal can be adapted to the display characteristics of the terminal device. This reduces the problem of poor visual effect caused by the loss of image information when the video image frame is displayed on the terminal device due to the mismatch between the upper limit brightness of the video image frame and the peak brightness of the terminal device.

[0236] Figure 10 This is a schematic block diagram illustrating an end-to-end video display system 300 according to an exemplary embodiment, such as... Figure 10 As shown, the system 300 includes:

[0237] The aforementioned video data processing device is installed at the encoding end 301, and,

[0238] The aforementioned video data processing device is installed at the decoding end 302.

[0239] It is understood that the technical solution provided in this embodiment, since the video image at the encoding end is an uncompressed and untransmitted lossless video image, can ensure maximum accuracy in matching the true state of each video image by calculating the estimated amount of tone loss for each frame at the encoding end, reducing calculation errors and providing accurate data support for the decoding end to correct the tone mapping curve. At the same time, since the calculation step of the estimated amount of tone loss is transferred to the encoding end, it also reduces the computational burden on the decoding end, enabling the decoding end to construct a dynamic tone mapping curve more quickly and display high-quality, tone-lossless video images more rapidly.

[0240] The decoding end corrects the original RGB electrical signal by modifying the tone mapping curve. This tone mapping curve takes into account the tone loss of the video image frame when it is displayed on the terminal device. Therefore, after converting the modified RGB electrical signal into an optical signal through a preset electro-optical conversion function, the converted optical signal can be adapted to the display characteristics of the terminal device. This reduces the problem of poor visual effect caused by the loss of image information when the video image frame is displayed on the terminal device due to the mismatch between the upper limit brightness of the video image frame and the peak brightness of the terminal device.

[0241] An electronic device according to an exemplary embodiment includes:

[0242] At least one processor; and

[0243] A memory communicatively connected to the at least one processor; wherein,

[0244] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method described above.

[0245] It should be noted that electronic devices include, but are not limited to: smart terminals (e.g., mobile phones, tablets, smartwatches, etc.) and computer equipment.

[0246] The processor includes, but is not limited to, CPU, microcontroller, PLC controller, FPGA controller, etc.

[0247] The memory may include computer system readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory; it may also include other removable / non-removable, volatile / non-volatile computer system storage media. The memory may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0248] It is understood that the technical solution provided in this embodiment, since the video image at the encoding end is an uncompressed and untransmitted lossless video image, can ensure maximum accuracy in matching the true state of each video image by calculating the estimated amount of tone loss for each frame at the encoding end, reducing calculation errors and providing accurate data support for the decoding end to correct the tone mapping curve. At the same time, since the calculation step of the estimated amount of tone loss is transferred to the encoding end, it also reduces the computational burden on the decoding end, enabling the decoding end to construct a dynamic tone mapping curve more quickly and display high-quality, tone-lossless video images more rapidly.

[0249] The decoding end corrects the original RGB electrical signal by modifying the tone mapping curve. This tone mapping curve takes into account the tone loss of the video image frame when it is displayed on the terminal device. Therefore, after converting the modified RGB electrical signal into an optical signal through a preset electro-optical conversion function, the converted optical signal can be adapted to the display characteristics of the terminal device. This reduces the problem of poor visual effect caused by the loss of image information when the video image frame is displayed on the terminal device due to the mismatch between the upper limit brightness of the video image frame and the peak brightness of the terminal device.

[0250] An exemplary embodiment illustrates a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method described above.

[0251] A computer program product according to an exemplary embodiment includes a computer program that, when executed by a processor, implements the method described above.

[0252] The computer-readable storage media disclosed in this embodiment include, but are not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0253] It is understood that the technical solution provided in this embodiment, since the video image at the encoding end is an uncompressed and untransmitted lossless video image, can ensure maximum accuracy in matching the true state of each video image by calculating the estimated amount of tone loss for each frame at the encoding end, reducing calculation errors and providing accurate data support for the decoding end to correct the tone mapping curve. At the same time, since the calculation step of the estimated amount of tone loss is transferred to the encoding end, it also reduces the computational burden on the decoding end, enabling the decoding end to construct a dynamic tone mapping curve more quickly and display high-quality, tone-lossless video images more rapidly.

[0254] The decoding end corrects the original RGB electrical signal by modifying the tone mapping curve. This tone mapping curve takes into account the tone loss of the video image frame when it is displayed on the terminal device. Therefore, after converting the modified RGB electrical signal into an optical signal through a preset electro-optical conversion function, the converted optical signal can be adapted to the display characteristics of the terminal device. This reduces the problem of poor visual effect caused by the loss of image information when the video image frame is displayed on the terminal device due to the mismatch between the upper limit brightness of the video image frame and the peak brightness of the terminal device.

[0255] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0256] It should be noted that in the description of this invention, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this invention, unless otherwise stated, "a plurality of" means at least two.

[0257] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0258] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0259] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0260] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0261] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0262] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0263] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A video data processing method, applicable to a decoding end, characterized in that, include: The device receives a video stream sent by the encoding end and decodes video image frames from the video stream, as well as the brightness characteristics and information loss value of each video image frame. The information loss value is the estimated value of the color tone loss when the video image frame is displayed on the terminal device. The process of determining the information loss value of each video frame includes: converting the YUV electrical signal of each video frame into an RGB electrical signal; obtaining the default values ​​of the display features of the terminal device; generating a tone mapping curve based on the brightness features and the default values ​​of the display features; correcting the YUV electrical signal and the RGB electrical signal based on the tone mapping curve; calculating the information entropy loss value of brightness based on the YUV electrical signal before and after correction; calculating the information entropy loss value of chrominance based on the RGB electrical signal before and after correction; and determining the information loss value of each video frame based on the information entropy loss value. Obtain the display characteristics of the terminal device; Based on the brightness and display characteristics, generate the tone mapping curve for the current video image frame; Based on the information loss value, the tone mapping curve is corrected to obtain a corrected tone mapping curve. The corrected tone mapping curve is used to characterize the mapping relationship between the original RGB electrical signal and the corrected RGB electrical signal. The original RGB electrical signal is the RGB electrical signal obtained by converting the YUV electrical signal of the video image frame to be displayed, and the corrected RGB electrical signal is the RGB electrical signal adapted to the display characteristics of the terminal device.

2. The method according to claim 1, characterized in that, The step of generating the tone mapping curve of the current video image frame based on the brightness and display features includes: Based on the display characteristics and the normalized brightness characteristics of the current video image frame, determine the function expression of the S-shaped mapping curve of the brightness region of the current video image frame, and the function expression of the linear mapping line of the dark region of the current video image frame. The piecewise function composed of the linear mapping line and the S-shaped mapping curve is determined as the tone mapping curve of the current video image frame.

3. The method according to claim 2, characterized in that, The step of correcting the tone mapping curve based on the information loss value includes: Based on the information loss value, the slope of the linear mapping line in the dark area of ​​the current video image frame is corrected; Based on the corrected slope, update the boundary value between the bright and dark areas of the current video image frame.

4. The method according to claim 3, characterized in that, The display features include: the maximum screen brightness of the terminal device; and the correction of the slope of the linear mapping line of the dark area of ​​the current video image frame based on the information loss value includes: If the maximum screen brightness is equal to the default value of the maximum screen brightness, the slope of the linear mapping line is compensated with the information loss value, and the compensated slope is updated to the slope of the linear mapping line in the dark area of ​​the current video image frame. If the maximum screen brightness is not equal to the default value of the maximum screen brightness, the information loss value is adjusted according to the maximum screen brightness, and the slope of the linear mapping line is compensated with the adjusted information loss value. The compensated slope is then updated to the slope of the linear mapping line in the dark area of ​​the current video image frame.

5. The method according to claim 3, characterized in that, The step of updating the boundary value between the bright and dark regions of the current video image frame based on the corrected slope includes: Find a point on the S-shaped mapping curve such that the slope of the line connecting that point and the origin is equal to the corrected slope; Update the x-coordinate of this point to the boundary between the bright and dark areas of the current video image frame.

6. The method according to claim 4, characterized in that, The adjustment of the information loss value based on the maximum screen brightness specifically involves: The information loss value is adjusted by a preset adjustment function, which is a linear or nonlinear function with the information loss value and the maximum screen brightness as independent variables and the adjusted information loss value as the dependent variable.

7. A video data processing method, applicable to the encoding end, characterized in that, include: For the video to be sent, calculate the brightness characteristics of each frame of the video image; Determine the information loss value for each frame of video image, wherein the information loss value is an estimated value of the tone loss when the video image frame is displayed on the terminal device; Determining the information loss value of each video frame includes: converting the YUV electrical signal of each video frame into an RGB electrical signal; obtaining the default values ​​of the display features of the terminal device; generating a tone mapping curve based on the brightness features and the default values ​​of the display features; correcting the YUV electrical signal and the RGB electrical signal based on the tone mapping curve; calculating the information entropy loss value of brightness based on the YUV electrical signal before and after correction; calculating the information entropy loss value of chrominance based on the RGB electrical signal before and after correction; and determining the information loss value of each video frame based on the information entropy loss value. The brightness features and information loss values ​​are written into the metadata of each frame of video image; The metadata and each frame of video image are encoded to form a video stream; The video stream is sent to the decoding end, so that the decoding end can correct the tone mapping curve of the current video image frame according to the brightness characteristics and the estimated amount of tone loss. The corrected tone mapping curve is used to characterize the mapping relationship between the original RGB electrical signal and the corrected RGB electrical signal. The original RGB electrical signal is the RGB electrical signal obtained by converting the YUV electrical signal of the video image frame to be displayed, and the corrected RGB electrical signal is the RGB electrical signal adapted to the display characteristics of the terminal device.

8. The method according to claim 7, characterized in that, The step of generating a tone mapping curve based on the default values ​​of the brightness characteristics and display characteristics includes: Based on the default values ​​of the display features and the normalized brightness features of the current video image frame, determine the function expression of the S-shaped mapping curve of the brightness region of the current video image frame, and the function expression of the linear mapping line of the dark region of the current video image frame. The piecewise function composed of the linear mapping line and the S-shaped mapping curve is determined as the tone mapping curve of the current video image frame.

9. The method according to claim 7, characterized in that, The step of correcting the YUV and RGB electrical signals according to the tone mapping curve includes: According to the tone mapping curve, the luminance component Y of the YUV electrical signal is mapped to the luminance component Y'. According to the hue mapping curve, the red component R of the RGB electrical signal is mapped to the red component R'; the green component G of the RGB electrical signal is mapped to the green component G'; and the blue component B of the RGB electrical signal is mapped to the blue component B'.

10. The method according to claim 9, characterized in that, The step of calculating the information entropy loss value of luminance based on the YUV electrical signals before and after correction includes: Calculate the information entropy of the histogram distribution HistY of the luminance component Y, and the information entropy of the histogram distribution HistYtm of the luminance component Y'. The absolute value of the difference between the information entropy of HistY and the information entropy of HistYtm is denoted as the information entropy loss value of brightness.

11. The method according to claim 9, characterized in that, The step of calculating the information entropy loss value of chromaticity based on the RGB electrical signals before and after correction includes: Calculate the information entropy of the histogram distribution HistR of the red component R and the information entropy of the histogram distribution HistRtm of the red component R' respectively, and denote the absolute value of the difference between the information entropy of HistR and the information entropy of HistRtm as the information entropy loss value of the red component R. Calculate the information entropy of the histogram distribution HistG of the green component G and the information entropy of the histogram distribution HistGtm of the green component G' respectively, and denote the absolute value of the difference between the information entropy of HistG and the information entropy of HistGtm as the information entropy loss value of the green component G. Calculate the information entropy of the histogram distribution HistB of the blue component B and the information entropy of the histogram distribution HistBtm of the blue component B' respectively, and denote the absolute value of the difference between the information entropy of HistB and the information entropy of HistBtm as the information entropy loss value of the blue component B. The sum of the information entropy loss values ​​of the red component R, the green component G, and the blue component B is denoted as the chromaticity information entropy loss value.

12. The method according to claim 7, characterized in that, The step of determining the information loss value of each video frame based on the information entropy loss value includes: For each frame of video image, the information entropy loss value of brightness and the information entropy loss value of chrominance of the video image frame are summed, and the summation result is multiplied by a preset loss coefficient. The product result is determined as the information loss value of the video image frame.

13. A video display method, applicable to a decoding end, characterized in that, include: Extract the YUV electrical signal of each frame of video image from the video stream sent from the encoding end, and convert the YUV electrical signal into the original RGB electrical signal; The original RGB electrical signal is converted into a corrected RGB electrical signal according to the tone mapping curve, wherein the tone mapping curve is generated by the video data processing method according to any one of claims 1 to 6; Based on a preset electro-optical conversion function, the corrected RGB electrical signal is converted into an RGB light signal for display on the terminal device.

14. A video data processing device, disposed at a decoding end, characterized in that, include: The decoding module is used to receive the video stream sent by the encoding end and decode the video image frames from the video stream, as well as the brightness characteristics and information loss value of each video image frame. The information loss value is the estimated value of the color tone loss when the video image frame is displayed on the terminal device. The process of determining the information loss value of each video frame includes: converting the YUV electrical signal of each video frame into an RGB electrical signal; obtaining the default values ​​of the display features of the terminal device; generating a tone mapping curve based on the brightness features and the default values ​​of the display features; correcting the YUV electrical signal and the RGB electrical signal based on the tone mapping curve; calculating the information entropy loss value of brightness based on the YUV electrical signal before and after correction; calculating the information entropy loss value of chrominance based on the RGB electrical signal before and after correction; and determining the information loss value of each video frame based on the information entropy loss value. The acquisition module is used to acquire the display characteristics of the terminal device; The generation module is used to generate the tone mapping curve of the current video image frame based on the brightness characteristics and display characteristics. The correction module is used to correct the tone mapping curve according to the information loss value to obtain a corrected tone mapping curve. The corrected tone mapping curve is used to characterize the mapping relationship between the original RGB electrical signal and the corrected RGB electrical signal. The original RGB electrical signal is the RGB electrical signal obtained by converting the YUV electrical signal of the video image frame to be displayed, and the corrected RGB electrical signal is the RGB electrical signal adapted to the display characteristics of the terminal device.

15. A video data processing apparatus, disposed at an encoding end, characterized in that, include: The calculation module is used to calculate the brightness characteristics of each frame of the video to be sent. It is also used to determine the information loss value of each video image frame, wherein the information loss value is an estimated value of the tonal loss of the video image frame when it is displayed on the terminal device; Determining the information loss value of each video frame includes: converting the YUV electrical signal of each video frame into an RGB electrical signal; obtaining the default values ​​of the display features of the terminal device; generating a tone mapping curve based on the brightness features and the default values ​​of the display features; correcting the YUV electrical signal and the RGB electrical signal based on the tone mapping curve; calculating the information entropy loss value of brightness based on the YUV electrical signal before and after correction; calculating the information entropy loss value of chrominance based on the RGB electrical signal before and after correction; and determining the information loss value of each video frame based on the information entropy loss value. The read / write module is used to write the brightness features and information loss values ​​into the metadata of each frame of video image; The encoding module is used to encode the metadata and each frame of video image to form a video stream; The sending module is used to send the video stream to the decoding end, so that the decoding end can correct the tone mapping curve of the current video image frame according to the brightness characteristics and the estimated value of tone loss. The corrected tone mapping curve is used to characterize the mapping relationship between the original RGB electrical signal and the corrected RGB electrical signal. The original RGB electrical signal is the RGB electrical signal obtained by converting the YUV electrical signal of the video image frame to be displayed, and the corrected RGB electrical signal is the RGB electrical signal adapted to the display characteristics of the terminal device.

16. An end-to-end video display system, characterized in that, include: The video data processing apparatus as described in claim 14 is disposed at the decoding end, and the video data processing apparatus as described in claim 15 is disposed at the encoding end.

17. An electronic device comprising: At least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 13.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 13.

19. A computer program product comprising a computer program that, when executed by a processor, implements the method of any one of claims 1 to 13.

Images