Video processing method, device, equipment and medium

Abstract

Embodiments of the present disclosure relate to a video processing method, device, equipment and medium, comprising: obtaining a first nonlinear video frame by decoding a nonlinear video of a first color space color gamut, and obtaining a second nonlinear video frame by decoding a nonlinear video of a second color space color gamut; processing the second nonlinear video frame to generate a second linear video frame, and performing color space conversion processing on the second linear video frame to generate a third linear video frame using the first color space color gamut; processing the first nonlinear video frame to generate a first linear video frame, and obtaining a first linear target video frame of the first color space according to the first linear video frame and the third linear video frame; and performing fusion processing on the first linear target video frame and a linear special effect resource to generate a first linear special effect video frame of the first color space. Embodiments of the present disclosure ensure the color accuracy and richness of the first linear special effect video frame, make the added special effect resource more natural, and improve the realism of the special effect video.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a video processing method, apparatus, device, and medium. Background Art

[0002] With the development of computer technology, the application scenarios of video processing technology are becoming more and more extensive. In the process of video processing, it is common to need to edit and splice multiple videos and then perform special effects processing, such as adding special effects stickers after splicing the videos.

[0003] However, current video processing methods can cause color differences in the spliced ​​video, resulting in insufficient color accuracy in the generated special effects video. Summary of the Invention

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a video processing method, apparatus, device and medium.

[0005] In a first aspect, an embodiment of the present disclosure provides a video processing method, the method comprising:

[0006] Decoding a nonlinear video in a first color space gamut to obtain a corresponding first nonlinear video frame, and decoding a nonlinear video in a second color space gamut to obtain a corresponding second nonlinear video frame, wherein the color gamut of the first color space is larger than the color gamut of the second color space;

[0007] Processing the second nonlinear video frame to generate a corresponding second linear video frame, and performing color space conversion on the second linear video frame to generate a corresponding third linear video frame using the first color space gamut;

[0008] Processing the first nonlinear video frame to generate a corresponding first linear video frame, and acquiring a first linear target video frame using the first color space gamut according to the first linear video frame and the third linear video frame;

[0009] A linear special effect resource using the first color space gamut is acquired, and the first linear target video frame and the linear special effect resource are fused to generate a first linear special effect video frame using the first color space gamut.

[0010] In an optional implementation:

[0011] The nonlinear video using the first color space gamut includes: a nonlinear HDR video using the Rec.2020 color space;

[0012] The nonlinear video using the second color space color gamut includes: a nonlinear SDR video using the sRGB color space.

[0013] In an optional implementation, obtaining a first linear target video frame using the first color space gamut according to the first linear video frame and the third linear video frame includes:

[0014] performing splicing processing on the first linear video frame and the third linear video frame to obtain a first linear target video frame using the first color space gamut; and / or,

[0015] Pixels of the first linear video frame and the third linear video frame are superimposed to obtain a first linear target video frame using the first color space color gamut.

[0016] In an optional implementation, obtaining a linear special effect resource using the first color space gamut includes:

[0017] It is detected whether a nonlinear special effect resource adopts the first color space gamut; if so, the nonlinear special effect resource is processed to generate a linear special effect resource adopting the first color space gamut.

[0018] In an optional implementation, after detecting whether the nonlinear special effect resource adopts the first color space gamut, the method further includes:

[0019] If the nonlinear special effect resource adopts the second color space gamut, processing the nonlinear special effect resource to generate a corresponding linear special effect resource adopting the second color space gamut;

[0020] Color space conversion processing is performed on the linear special effect resource using the second color space color gamut to generate a corresponding linear special effect resource using the first color space color gamut.

[0021] In an optional implementation, after generating the first linear special effect video frame using the first color space gamut, the method further includes:

[0022] The first linear special effects video frame using the first color space color gamut is encoded to generate a first linear special effects video for display on a display device.

[0023] In an optional implementation, after generating the first linear special effect video frame using the first color space gamut, the method further includes:

[0024] Processing the first linear special effect video frame to generate a first nonlinear special effect video frame using the first color space gamut;

[0025] The first nonlinear special effects video frame is encoded to generate a first nonlinear special effects video stored in the first color space color gamut.

[0026] In an optional implementation, after generating the first linear special effect video frame using the first color space gamut, the method further includes:

[0027] Performing color space conversion on a first linear special effect video frame using the first color space gamut to generate a second linear special effect video frame using the second color space gamut;

[0028] The second linear special effect video frame is encoded to generate a second linear special effect video for display on a display device.

[0029] In an optional implementation, after generating the second linear special effect video frame using the second color space gamut, the method further includes:

[0030] Processing the second linear special effect video frame to generate a second nonlinear special effect video frame using the second color space gamut;

[0031] The second nonlinear special effects video frame is encoded to generate a second nonlinear special effects video stored in the second color space color gamut.

[0032] In an optional embodiment, the method further includes:

[0033] The data storage accuracy of the video frame is determined according to a storage device or a display device.

[0034] In a second aspect, an embodiment of the present disclosure provides a video processing device, the device comprising:

[0035] a decoding module, configured to decode a nonlinear video in a first color space gamut to obtain a corresponding first nonlinear video frame, and decode a nonlinear video in a second color space gamut to obtain a corresponding second nonlinear video frame, wherein the color gamut of the first color space is larger than the color gamut of the second color space;

[0036] a first conversion module, configured to process the second nonlinear video frame to generate a corresponding second linear video frame, and perform color space conversion on the second linear video frame to generate a corresponding third linear video frame using the first color space gamut;

[0037] a first generating module, configured to process the first nonlinear video frame to generate a corresponding first linear video frame, and obtain a first linear target video frame using the first color space gamut according to the first linear video frame and the third linear video frame;

[0038] The second generation module is used to obtain a linear special effect resource using the first color space color gamut, fuse the first linear target video frame and the linear special effect resource, and generate a first linear special effect video frame using the first color space color gamut.

[0039] In a third aspect, the present disclosure provides a computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium. When the instructions are executed on a terminal device, the terminal device implements the above method.

[0040] In a fourth aspect, the present disclosure provides an electronic device, comprising: a processor; a memory for storing instructions executable by the processor; and the processor for reading the executable instructions from the memory and executing the instructions to implement the above method.

[0041] In a fifth aspect, the present disclosure provides a computer program product, which includes a computer program / instructions, and the computer program / instructions implement the above method when executed by a processor.

[0042] The technical solution provided by the embodiments of the present disclosure has at least the following advantages compared with the prior art:

[0043] The video processing method provided by the embodiments of the present disclosure performs linear processing on a second non-linear video frame to generate a second linear video frame, so that special effects processing can be performed based on the second linear video frame. The second linear video frame is color-space converted to a first color space to generate a third linear video frame, thereby ensuring the uniformity of the color space of the video frame to be processed with special effects and expanding the color space, making the video frame more colorful. The first non-linear video frame is processed to generate a first linear video frame, so that special effects processing can be performed based on the first linear video frame. The first linear video frame and the third linear video frame are unified into a linear space, so that a first linear target video frame generated based on the above two video frames is still a linear space. The first linear target video frame and the linear special effects resource are fused, and both the first linear target video frame and the linear special effects resource are in a linear space and adopt the color gamut of the first color space, thereby ensuring the color uniformity, accuracy, and richness of the first linear special effects video frame, making the added special effects resource more natural and improving the realism of the special effects video. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

[0045] Figure 1 A flowchart of a video processing method provided by an embodiment of the present disclosure;

[0046] Figure 2 A schematic diagram of a color space provided in an embodiment of the present disclosure;

[0047] Figure 3a A schematic diagram of a linear space from black to white provided in an embodiment of the present disclosure;

[0048] Figure 3b A schematic diagram of a nonlinear space from black to white provided in an embodiment of the present disclosure;

[0049] Figure 3c A schematic diagram of a nonlinear space and a nonlinear space comparison provided by an embodiment of the present disclosure;

[0050] Figure 4 A schematic diagram of the correspondence between a linear space and a nonlinear space provided in an embodiment of the present disclosure;

[0051] Figure 5a A schematic diagram of splicing a first linear video frame and a third linear video frame provided in an embodiment of the present disclosure;

[0052] Figure 5b A schematic diagram of another embodiment of the present disclosure for splicing a first linear video frame and a third linear video frame;

[0053] Figure 6 A schematic diagram of superimposing a first linear video frame and a third linear video frame provided in an embodiment of the present disclosure;

[0054] Figure 7 A schematic diagram of different data storage accuracies of a video frame provided by an embodiment of the present disclosure;

[0055] Figure 8 A flowchart of another video processing method provided by an embodiment of the present disclosure;

[0056] Figure 9 A schematic structural diagram of a video processing device provided in an embodiment of the present disclosure;

[0057] Figure 10 A schematic structural diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0058] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0059] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0060] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

[0061] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0062] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0063] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0064] In order to solve the above problems, the embodiments of the present disclosure provide a video processing method, which is introduced below in conjunction with specific embodiments.

[0065] Figure 1 This is a flow chart of a video processing method provided by an embodiment of the present disclosure. The method can be executed by a video processing device, wherein the device can be implemented using software and / or hardware and can generally be integrated into an electronic device. Figure 1 As shown, the method includes:

[0066] Step 101: decode a nonlinear video using a first color space gamut to obtain a corresponding first nonlinear video frame, and decode a nonlinear video using a second color space gamut to obtain a corresponding second nonlinear video frame, wherein the color gamut of the first color space is larger than the color gamut of the second color space.

[0067] In this embodiment, the multiple nonlinear videos to be processed, obtained by shooting with a mobile phone and / or forwarding through other communication applications, include: nonlinear videos using a first color space gamut and nonlinear videos using a second color space gamut, wherein the color gamut of the first color space is larger than the color gamut of the second color space. For example: a nonlinear high-dynamic range (HDR) video using the Rec.2020 (Recommendation ITU-R-BT.2020) color space is obtained by shooting with a mobile phone, and a nonlinear standard dynamic range (SDR) video using the standard red, green, and blue (sRGB) color space is obtained and forwarded through a communication application.

[0068] It should be noted that the color space is a model used to represent color. Different color spaces have corresponding color ranges. For example, the three-dimensional color space Figure 2 As shown in the diagram, the color area covered by the triangles corresponding to different color spaces represents the color range that the color space can represent, and the area of ​​the color area represents the size of the color space. Figure 2 As can be seen in the figure, the color range of Rec.2020 is larger than that of sRGB. Taking the three-dimensional color space as an example, the color is represented by three-dimensional coordinates, where the value of each dimension coordinate is 0 to 1, where 0 means that the color is not taken and 1 means that the color is taken to the maximum value of the color in this color space. Figure 2 , the upper vertex of the triangle corresponding to Rec.2020 is the point representing the green primary color in Rec.2020, which is (0,1,0), and the upper vertex of the triangle corresponding to sRGB is the point representing the green primary color in sRGB, which is (0,1,0). Figure 2As can be seen in the figure, although the green primitive in Rec.2020 and sRGB are both represented as (0,1,0), they actually represent different colors. This shows that even if the color representations of the first color space and the second color space used by nonlinear video are the same, the actual colors are different. Therefore, mixing images from different color spaces may cause color inaccuracy. To ensure accurate color and high-quality special effect colors in video frames after special effects processing, it is necessary to ensure that the color space used by each video frame to be processed is consistent before special effects processing.

[0069] Furthermore, it should be noted that videos captured by mobile phones and / or videos forwarded and processed by communication applications are generally nonlinear videos. In this embodiment, nonlinear video frames using the first color space gamut and the second color space gamut refer to videos that have been converted from a linear space to a nonlinear space. In linear videos, the relationship between the value corresponding to a pixel and the luminous power of the pixel is linear, such as Figure 3a As shown, Figure 3a This is a schematic diagram of a linear space from black to white provided by the embodiment of the present disclosure. However, because the human eye is more sensitive to dark colors, the human eye sees Figure 3a The bright areas in the image are larger than the dark areas. In order to balance the areas of the bright and dark areas so that the areas of the bright and dark areas seen by the human eye are similar, the linear space can be processed nonlinearly.

[0070] In nonlinear space, the relationship between the value corresponding to a pixel and the luminous power of the pixel is nonlinear, such as Figure 3b As shown, Figure 3b A schematic diagram of a nonlinear space from black to white provided by an embodiment of the present disclosure. Figure 3a , the dark areas seen by the human eye become more and the light areas become less, so that the areas of the light areas and dark areas seen by the human eye are similar. In an optional embodiment, the nonlinear space is a space that has been gamma-corrected and the corresponding gamma value is 2.2. In order to more clearly illustrate the relationship between the linear space and the nonlinear space, as shown in FIG. Figure 3c As shown, Figure 3c In the figure, the dotted line represents the dividing line between the dark and light areas seen by the human eye. The scale value corresponding to this dividing line in the linear space is 21.76%, and the scale value corresponding to this dividing line in the nonlinear space is 50%. It can be seen that, except for the scales of 0 and 100%, the scale values ​​corresponding to the same human visual color in the linear space and the nonlinear space are different. In order to ensure the accuracy of the color, the video frame needs to be unified in the linear space or nonlinear space before image processing.

[0071] To perform special effects processing on nonlinear video, it is necessary to decode the nonlinear video in a first color space to obtain the corresponding first nonlinear video frame, and to decode the nonlinear video in a second color space to obtain the corresponding second nonlinear video frame. There are various decoding methods available, which can be selected based on the application scenario and are not limited in this embodiment, such as soft decoding and hard decoding. Furthermore, the nonlinear videos in different color spaces can be a single video or multiple videos.

[0072] Step 102: Process the second nonlinear video frame to generate a corresponding second linear video frame, and perform color space conversion on the second linear video frame to generate a corresponding third linear video frame using the first color space gamut.

[0073] In order to meet the needs of video processing, such as special effects processing, such as lighting effects, facial skin smoothing and other computational processing, it is necessary to perform special effects processing on the video frames in a linear space. However, since the color gamut of the first color space used by the first nonlinear video frame to be processed is larger than the color gamut of the second color space used by the second nonlinear video frame, as mentioned above, in order to ensure the uniformity of the color space and in order to use the first color space color gamut with richer image colors, it is necessary to convert the second nonlinear video frame using the second color space color gamut to the first color space color gamut for processing. Since the color space conversion of the video frame also needs to be processed in a linear space, it is necessary to first perform a linear conversion on the second nonlinear video frame to generate a corresponding second linear video frame, and then perform a color space conversion on the second linear video frame. The specific processing process is as follows:

[0074] First, it is necessary to determine the video format of the second nonlinear video frame to be processed, and then call the conversion function corresponding to the video format to convert the nonlinear video frame to generate a corresponding second linear video frame. It should be noted that the conversion functions corresponding to different video formats are pre-set according to the application scenario of video processing. For example, the nonlinear HDR video frame is converted into a linear HDR video frame through the HDR conversion function corresponding to the HDR video format, or the nonlinear SDR video frame is converted into a linear SDR video frame through the SDR conversion function corresponding to the SDR video format. In order to more clearly illustrate the linear conversion process of the nonlinear video frame, the SDR nonlinear video frame is taken as an example as follows: Figure 4 As shown, Figure 4 The horizontal axis is a color diagram of the linear video frame from black to white, and the vertical axis is a color diagram of the SDR non-linear video frame from black to white. The curve in the figure represents the correspondence and conversion relationship between the colors of the linear video frame and the colors of the SDR non-linear video frame. According to this relationship, the SDR non-linear video frame can be converted into an SDR linear video frame.

[0075] Furthermore, it is necessary to determine a conversion relationship between the first color space and the second color space, and perform color space conversion on the second linear video frame based on the conversion relationship to generate a corresponding third linear video frame using the color gamut of the first color space. For example, assuming the first color space is Rec.2020, the second color space is sRGB, and the second linear video frame is an SDR video frame using the sRGB color space, then the color space of the second linear video frame is converted from sRGB to Rec.2020 based on a color space conversion function or color space conversion matrix between Rec.2020 and sRGB, thereby obtaining a third linear video frame that is an HDR video frame using Rec.2020.

[0076] Step 103: Process the first nonlinear video frame to generate a corresponding first linear video frame, and obtain a first linear target video frame using a first color space gamut according to the first linear video frame and the third linear video frame.

[0077] In this embodiment, in order to meet the needs of video processing, it is also necessary to perform linear processing on the first nonlinear video frame to generate a corresponding first linear video frame. This processing method is similar to the processing method of processing the second nonlinear video frame to generate a second linear video frame, and will not be repeated here.

[0078] After obtaining the first linear video frame and the third linear video frame having the same color space and being both linear spaces, color deviation caused by different color spaces is avoided, thereby ensuring color accuracy. A first linear target video frame can be obtained based on the first linear video frame and the third linear video frame, and the color space of the first linear target video frame is also the first color space.

[0079] It should be noted that there are multiple methods for obtaining the first linear target video frame, and examples are described as follows:

[0080] In one embodiment, the first linear video frame and the third linear video frame are spliced ​​to obtain a first linear target video frame using a first color space gamut.

[0081] First, N first linear video frames (N is an integer) are selected, and M third linear video frames (M is an integer) are selected. Further, a preset splicing method is used to combine the selected N first linear video frames and the M third linear video frames according to the preset splicing method. The combined video frame is the first linear target video frame. There are multiple preset splicing methods, and examples are described as follows:

[0082] Method 1: If Figure 5a As shown, a portion of the third linear video frame is spliced ​​after a portion of the first linear video frame, thereby obtaining a first linear target video frame. Figure 5a, 100 frames of the first linear video frames are selected, and 80 frames of the third linear video frames are selected. The spliced ​​first linear target video frames have a total of 100 frames, and frames 1 to 50 of the first linear target video frames are frames 1 to 50 of the first linear video frames, and frames 51 to 100 are frames 31 to 80 of the third linear video frames.

[0083] Method 2: If Figure 5b As shown, the first linear video frame and the third linear video frame are cross-joined to obtain a first linear target video frame. Figure 5b , 100 frames of the first linear video frames are selected, and 80 frames of the third linear video frames are selected. The stitched first linear target video frames have a total of 80 frames. Among the first linear target video frames, frames 1 to 20 are frames 1 to 20 of the first linear video frames, frames 21 to 40 are frames 21 to 40 of the third linear video frames, frames 41 to 60 are frames 21 to 40 of the first linear video frames, and frames 61 to 80 are frames 41 to 60 of the third linear video frames.

[0084] In another embodiment, pixels of the first linear video frame and the third linear video frame are superimposed to obtain a first linear target video frame using the first color space gamut. An example is shown below:

[0085] In an optional embodiment, as Figure 6 As shown, the number of frames of the first linear video frame is the same as that of the third linear video frame, and the pixels of each frame of the first linear video frame and each frame of the third linear video frame can be superimposed to obtain the first linear target video frame. Figure 6 In the embodiment, there are three first linear video frames and three third linear video frames. The pixels of each frame of the first linear video frame and each frame of the third linear video frame are superimposed. When the pixels of the first linear video frame and the pixels of the third linear video frame overlap, the pixels of the third linear video frame are retained, thereby achieving pixel superposition and obtaining the corresponding first linear target video frame.

[0086] Step 104 : Acquire a linear special effect resource using the first color space gamut, fuse the first linear target video frame and the linear special effect resource, and generate a first linear special effect video frame using the first color space gamut.

[0087] In this embodiment, in order to ensure color accuracy, the color space used by the linear special effects resource is the first color space gamut. After determining the first linear target video frame to be processed, the linear special effects resource is obtained, and the first linear target video frame and the linear special effects resource are fused to generate a first linear special effects video frame using the first color space gamut. Through the fusion process, the linear special effects resource is applied to the first linear target video frame, and virtual stickers and / or blurring are added to the first linear target video frame according to the special effects application requirements, thereby generating a first linear special effects video frame. For example, the color space of the linear special effects resource and the first linear target video frame is the same Rec.2020. The linear special effects resource is used to add sticker special effects. The linear special effects resource is used to perform fusion processing with the first linear target video frame, thereby adding stickers to the corresponding positions of the first linear target video frame, and generating a first linear special effects video frame using the first color space gamut.

[0088] It should be noted that each video frame in the above embodiment has a corresponding data storage precision, which indicates the accuracy of the color that each pixel in the video frame can represent. For the same color range, the more bits of data storage precision, the finer the granularity of the color range can be divided. In an optional implementation, Figure 7 As shown, the data storage accuracy is divided into 10bit and 8bit. Figure 7 It can be seen that the color granularity corresponding to 10 bits is finer, and the color granularity corresponding to 8 bits is coarser. However, in some application scenarios, the display device is not sufficient to display colors with finer granularity, or the storage space of the storage device is not sufficient to provide enough storage space to store video frames with more bits of data storage precision. Therefore, it is necessary to determine the data storage precision of each video frame in this embodiment based on the storage device or display device. For example, if the display device cannot show the difference between data storage precision of 10 bits and 8 bits, the data storage precision of the video frame is determined to be 8 bits.

[0089] In summary, the video processing method of the embodiment of the present disclosure performs linear processing on the second non-linear video frame to generate a second linear video frame, so that processing can be performed based on the second linear video frame, and the second linear video frame is converted into the first color space to generate a third linear video frame, thereby ensuring the uniformity of the color space of the video frame to be processed, while expanding the color space, making the color of the video frame richer; processing the first non-linear video frame to generate a first linear video frame, so that special effects processing can be performed based on the first linear video frame, while unifying the first linear video frame and the third linear video frame into linear space, so that the first linear target video frame generated based on the above two video frames is still linear space, and the first linear target video frame and the linear special effects resource are fused. The first linear target video frame and the linear special effects resource are both in linear space and both use the first color space color gamut, thereby ensuring the color uniformity, accuracy and richness of the first linear special effects video frame, making the added special effects resource more natural and improving the realism of the special effects video.

[0090] Based on the above embodiment, in order to ensure the color accuracy of the processed video frame and the quality of the special effect color, it is necessary to obtain a linear special effect resource in a linear space that is consistent with the color space used by the first linear target video frame to be processed before processing, and then perform special effect processing on the first linear target video frame based on the linear special effect resource. However, in actual application scenarios, the color space used by the special effect resource may be inconsistent with the first linear target video frame, and the special effect resource may also be nonlinear. Therefore, it is also necessary to obtain a linear special effect resource that is consistent with the color space gamut used by the first linear target video frame to be processed. Taking nonlinear special effect resources as an example, Figure 8 As shown, in the above embodiment, obtaining a linear special effect resource using the first color space gamut includes:

[0091] Step 801 : Detect whether a nonlinear special effect resource adopts a first color space gamut. If so, process the nonlinear special effect resource to generate a linear special effect resource adopting the first color space gamut.

[0092] In this embodiment, in order to ensure the color accuracy of the special effects processing, it is detected whether the nonlinear special effects resource adopts the first color space color gamut. If the first color space color gamut is adopted, it means that the color space of the nonlinear special effects resource is the same as the color space of the first linear target video frame to be processed by the special effects. Furthermore, in order to ensure that the special effects processing can be performed, it is necessary to linearly process the nonlinear special effects resource to generate a linear special effects resource. The color space adopted by the nonlinear special effects resource is also the first color space. For example, if the first color space is Rec.2020, the nonlinear special effects resource is parsed to obtain the color space parameters of the nonlinear special effects resource. If the nonlinear special effects resource is a nonlinear HDR special effects resource adopting Rec.2020, the nonlinear HDR special effects resource adopting Rec.2020 is further linearized to obtain a linear special effects resource that is a linear HDR special effects resource adopting Rec.2020.

[0093] Step 802: If the nonlinear special effect resource adopts the second color space gamut, the nonlinear special effect resource is processed to generate a corresponding linear special effect resource adopting the second color space gamut.

[0094] If the nonlinear special effects resource uses the second color space gamut, it means that the color space of the nonlinear special effects resource is different from the color space of the first linear target video frame to be processed. If the special effects processing is performed directly, the color of the special effects processing will be inaccurate, resulting in a poor sense of realism of the special effects processing. At the same time, in order to ensure the realism of the special effects processing and to be able to perform special effects processing methods based on mathematical operations, such as blur processing, it is necessary to perform color space conversion and linear processing on the nonlinear special effects resource. First, the nonlinear special effects resource is linearly processed to generate a corresponding linear special effects resource using the second color space gamut. For example, if the second color space gamut is sRGB, and the nonlinear special effects resource is a nonlinear SDR special effects resource using sRGB, the nonlinear SDR special effects resource using sRGB is first linearly processed, and the generated linear special effects resource is a linear SDR special effects resource using sRGB.

[0095] Step 803 : Perform color space conversion processing on the linear special effect resource using the second color space gamut to generate a corresponding linear special effect resource using the first color space gamut.

[0096] Furthermore, a color space conversion process is performed on the linear special effects resource using the second color space gamut to generate a corresponding linear special effects resource using the first color space gamut. In an optional embodiment, the color space conversion process can be implemented by a conversion function, and the conversion function can be set according to the first color space and the second color space. For example, the linear special effects resource is a linear SDR special effects resource using sRGB, and the linear SDR special effects resource using sRGB is subjected to color space conversion through a conversion function. The conversion function can convert a video frame using the SDR format of sRGB into a video frame using the HDR format of Rec.2020, and the generated linear special effects resource is a linear HDR special effects resource using Rec.2020.

[0097] In summary, the video processing method of the embodiment of the present invention determines the special effect resource as linear and the color space of the special effect resource as the first color space, ensuring that the color range of the special effect resource is large, ensuring the color richness of the generated first linear special effect video frame, and consistent with the first linear target video frame to be subjected to special effect processing as a linear space and the first color space, thereby ensuring the color accuracy of the image and video after special effect processing, making the added special effect resource more natural, and improving the realism of the generated first linear special effect video frame.

[0098] Based on the above embodiment, further, after generating the first linear special effect video frame, it is also necessary to generate a corresponding video, which can be used for displaying on a display device or for storing on a storage device. Depending on different application scenarios, there are corresponding methods for generating video frames, and examples are described as follows:

[0099] Scenario 1: In this scenario, a video is generated for display on a display device, and the display device is suitable for displaying a video using a first color space color gamut, including: encoding and processing a first linear special effects video frame using the first color space color gamut, and generating a first linear special effects video for display on the display device.

[0100] Linear video can be displayed on a display device. To obtain a linear special effects video, the first linear special effects video frames using the first color space gamut are encoded to generate the first linear special effects video for display on the display device. The encoding process can synthesize the linear special effects video frames into a corresponding linear special effects video. There are multiple encoding processes that can be selected based on the application scenario, and this embodiment does not limit them, such as soft encoding and hard encoding. In this scenario, displaying the linear video using the first color space gamut on the display device can achieve richer and more accurate colors.

[0101] Scenario 2: In this scenario, the generated video is stored in a storage device, and the storage device is suitable for storing videos using the first color space gamut, including:

[0102] The first linear special effect video frame is processed to generate a first nonlinear special effect video frame using a first color space gamut. Since the video stored in the storage medium is nonlinear, the first linear special effect video frame needs to be processed to generate the first nonlinear special effect video frame. This processing can be used to convert the linear video frame into a nonlinear video frame. The processing process is described in the above embodiment and is not further described here.

[0103] Furthermore, the first nonlinear special effects video frame is encoded to generate a first nonlinear special effects video stored in the first color space gamut. The encoding process can synthesize the first nonlinear special effects video frame into a corresponding first nonlinear special effects video. There are multiple encoding processes that can be selected based on the application scenario. The first nonlinear special effects video frame is processed using the encoding method to generate a corresponding first nonlinear special effects video, and the video is stored using a storage device.

[0104] Scenario 3: In this scenario, a video is generated for display on a display device. The color space used by the video in this scenario is the second color space gamut, including:

[0105] A first linear special effects video frame using a first color space gamut is subjected to color space conversion to generate a second linear special effects video frame using a second color space gamut. The first linear special effects video frame is processed to generate a second linear special effects video frame. There are multiple methods for implementing this color space conversion, which can be selected based on the application scenario, and this embodiment does not limit this. For example, a conversion function is used to implement color space conversion and / or a matrix is ​​used to implement color space conversion. The conversion function or matrix can be selected and designed based on the second color space and the first color space. In an optional embodiment, the first color space is Rec.2020, the second color space is sRGB, the first linear special effects video frame is a linear HDR special effects video frame using Rec.2020, and the linear HDR special effects video frame using Rec.2020 is processed using a conversion function. The conversion function can convert the linear HDR video frame using Rec.2020 into a linear SDR video frame using sRGB, thereby generating a second linear special effects video frame as a linear SDR special effects video frame using sRGB.

[0106] Furthermore, the second linear special effect video frame is encoded to generate a second linear special effect video for display on a display device. The encoding process can synthesize the second linear special effect video frame into a corresponding second linear special effect video. There are various encoding processes that can be selected based on the application scenario and are not limited in this embodiment, such as soft encoding and hard encoding.

[0107] It should be noted that when displaying a video, if a special effects video with a first color space (for example, Rec. 2020) is displayed on a display that can only display videos with a second color space (for example, sRGB), the color quality will not be improved, and overexposure may be caused (that is, as long as the color to be displayed exceeds the color that the pixel can display, the pixel will show maximum brightness, thereby reducing the color accuracy). Therefore, the first color space is converted to the second color space, which improves the color accuracy, reduces the storage space occupied by the special effects video frames, and improves the transmission efficiency of the special effects video.

[0108] Scenario 4: In this scenario, based on the above embodiment, before displaying the second linear special effect video on the display device, the special effect video needs to be stored in a storage device. The video stored in the storage medium is a non-linear video, so the linear special effect video frame needs to be processed to generate a non-linear special effect video frame. Specifically, in the above embodiment, after generating the second linear special effect video frame using the second color space gamut, the following is further included:

[0109] The second linear special effect video frame is nonlinearly processed to generate a second nonlinear special effect video frame using the second color space gamut. This process can be used to convert the linear video frame into a nonlinear video frame. The conversion process is described in the above embodiment and will not be repeated here.

[0110] Furthermore, the second nonlinear special effects video frame is encoded to generate a second nonlinear special effects video stored in the second color space gamut. The encoding process can synthesize the second nonlinear special effects video frame into a corresponding second nonlinear special effects video. There are multiple encoding processes that can be selected based on the application scenario. The second nonlinear special effects video frame is processed using the encoding method to generate a corresponding second nonlinear special effects video, and the video is stored using a storage device.

[0111] Figure 9 This is a schematic diagram of the structure of a video processing device provided by an embodiment of the present disclosure. The device can be implemented by software and / or hardware and can generally be integrated into an electronic device. Figure 9 As shown, the device comprises:

[0112] A decoding module 901 is configured to decode a nonlinear video in a first color space gamut to obtain a corresponding first nonlinear video frame, and decode a nonlinear video in a second color space gamut to obtain a corresponding second nonlinear video frame, wherein the color gamut of the first color space is larger than the color gamut of the second color space;

[0113] A first conversion module 902 is configured to process the second nonlinear video frame to generate a corresponding second linear video frame, and perform color space conversion on the second linear video frame to generate a corresponding third linear video frame using the first color space gamut;

[0114] A first generating module 903 is configured to process the first nonlinear video frame to generate a corresponding first linear video frame, and obtain a first linear target video frame using the first color space gamut according to the first linear video frame and the third linear video frame;

[0115] The second generating module 904 is configured to obtain a linear special effect resource using the first color space gamut, fuse the first linear target video frame with the linear special effect resource, and generate a first linear special effect video frame using the first color space gamut.

[0116] Optionally, in the device:

[0117] The nonlinear video using the first color space gamut includes: a nonlinear HDR video using the Rec.2020 color space;

[0118] The nonlinear video using the second color space color gamut includes: a nonlinear SDR video using the sRGB color space.

[0119] Optionally, the first generating module 903 is configured to:

[0120] performing splicing processing on the first linear video frame and the third linear video frame to obtain a first linear target video frame using the first color space gamut; and / or,

[0121] Pixels of the first linear video frame and the third linear video frame are superimposed to obtain a first linear target video frame using the first color space color gamut.

[0122] Optionally, the second generating module 904 is configured to:

[0123] It is detected whether a nonlinear special effect resource adopts the first color space gamut; if so, the nonlinear special effect resource is processed to generate a linear special effect resource adopting the first color space gamut.

[0124] Optionally, the device further includes:

[0125] A first processing module is configured to, if the nonlinear special effect resource adopts the second color space gamut, process the nonlinear special effect resource to generate a corresponding linear special effect resource adopting the second color space gamut;

[0126] The second processing module is used to perform color space conversion processing on the linear special effect resource using the second color space color gamut to generate a corresponding linear special effect resource using the first color space color gamut.

[0127] Optionally, the device further includes:

[0128] The first encoding module is used to encode the first linear special effect video frame using the first color space color gamut to generate a first linear special effect video for display on a display device.

[0129] Optionally, the device further includes:

[0130] a third processing module, configured to process the first linear special effect video frame to generate a first nonlinear special effect video frame using the first color space gamut;

[0131] The second encoding module is used to encode the first nonlinear special effect video frame to generate a first nonlinear special effect video storage using the first color space color gamut.

[0132] Optionally, the device further includes:

[0133] a second conversion module, configured to perform color space conversion on the first linear special effect video frame using the first color space gamut to generate a second linear special effect video frame using the second color space gamut;

[0134] The third encoding module is used to encode the second linear special effect video frame to generate a second linear special effect video for display on a display device.

[0135] Optionally, the device further includes:

[0136] a fourth processing module, configured to process the second linear special effect video frame to generate a second nonlinear special effect video frame using the second color space gamut;

[0137] The fourth encoding module is used to encode the second nonlinear special effect video frame to generate a second nonlinear special effect video storage using the second color space color gamut.

[0138] Optionally, the device further includes:

[0139] The determination module is used to determine the data storage accuracy of the video frame according to a storage device or a display device.

[0140] The video processing device provided in the embodiments of the present disclosure can execute the video processing method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0141] An embodiment of the present disclosure further provides a computer program product, including a computer program / instruction, which implements the video processing method provided by any embodiment of the present disclosure when executed by a processor.

[0142] Figure 10 A schematic structural diagram of an electronic device provided in an embodiment of the present disclosure.

[0143] The following specific reference Figure 10 , which shows a schematic structural diagram of an electronic device 1000 suitable for implementing the embodiments of the present disclosure. The electronic device 1000 in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), wearable electronic devices, and fixed terminals such as digital TVs, desktop computers, smart home devices, and the like. Figure 10 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0144] like Figure 10 As shown, the electronic device 1000 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1008 into a random access memory (RAM) 1003. Various programs and data required for the operation of the electronic device 1000 are also stored in the RAM 1003. The processing device 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0145] Typically, the following devices may be connected to the I / O interface 1005: an input device 1006 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1007 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1008 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the electronic device 1000 to communicate with other devices wirelessly or by wire to exchange data.Figure 10 The electronic device 1000 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0146] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network via the communication device 1009, or installed from the storage device 1008, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the video processing method of the embodiment of the present disclosure are performed.

[0147] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0148] In some embodiments, the client and server can communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0149] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0150] The computer-readable medium carries one or more programs. When executed by the electronic device, the electronic device: decodes a nonlinear video in a first color space to obtain a first nonlinear video frame, decodes a nonlinear video in a second color space to obtain a second nonlinear video frame; processes the second nonlinear video frame to generate a second linear video frame, and performs color space conversion on the second linear video frame to generate a third linear video frame using the color gamut of the first color space; processes the first nonlinear video frame to generate a first linear video frame, and obtains a first linear target video frame in the first color space based on the first linear video frame and the third linear video frame; obtains a linear special effect resource in the first color space, and fuses the first linear target video frame with the linear special effect resource to generate a first linear special effect video frame in the first color space. The disclosed embodiments ensure the color accuracy and richness of the first linear special effect video frame, making the added special effect resource more natural and improving the realism of the special effect video.

[0151] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0152] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0153] The units involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a unit does not necessarily limit the unit itself.

[0154] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0155] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0156] According to one or more embodiments of the present disclosure, the present disclosure provides a video processing method, including:

[0157] Decoding a nonlinear video in a first color space gamut to obtain a corresponding first nonlinear video frame, and decoding a nonlinear video in a second color space gamut to obtain a corresponding second nonlinear video frame, wherein the color gamut of the first color space is larger than the color gamut of the second color space;

[0158] Processing the second nonlinear video frame to generate a corresponding second linear video frame, and performing color space conversion on the second linear video frame to generate a corresponding third linear video frame using the first color space gamut;

[0159] Processing the first nonlinear video frame to generate a corresponding first linear video frame, and acquiring a first linear target video frame using the first color space gamut according to the first linear video frame and the third linear video frame;

[0160] A linear special effect resource using the first color space gamut is acquired, and the first linear target video frame and the linear special effect resource are fused to generate a first linear special effect video frame using the first color space gamut.

[0161] According to one or more embodiments of the present disclosure, in a video processing method provided by the present disclosure,

[0162] The nonlinear video using the first color space gamut includes: a nonlinear HDR video using the Rec.2020 color space;

[0163] The nonlinear video using the second color space color gamut includes: a nonlinear SDR video using the sRGB color space.

[0164] According to one or more embodiments of the present disclosure, in a video processing method provided by the present disclosure, obtaining a first linear target video frame using the first color space gamut according to the first linear video frame and the third linear video frame includes:

[0165] performing splicing processing on the first linear video frame and the third linear video frame to obtain a first linear target video frame using the first color space gamut; and / or,

[0166] Pixels of the first linear video frame and the third linear video frame are superimposed to obtain a first linear target video frame using the first color space color gamut.

[0167] According to one or more embodiments of the present disclosure, in a video processing method provided by the present disclosure, obtaining a linear special effect resource using the first color space color gamut includes:

[0168] It is detected whether a nonlinear special effect resource adopts the first color space gamut; if so, the nonlinear special effect resource is processed to generate a linear special effect resource adopting the first color space gamut.

[0169] According to one or more embodiments of the present disclosure, a video processing method provided by the present disclosure further includes, after detecting whether the nonlinear special effect resource adopts the first color space gamut:

[0170] If the nonlinear special effect resource adopts the second color space gamut, processing the nonlinear special effect resource to generate a corresponding linear special effect resource adopting the second color space gamut;

[0171] Color space conversion processing is performed on the linear special effect resource using the second color space color gamut to generate a corresponding linear special effect resource using the first color space color gamut.

[0172] According to one or more embodiments of the present disclosure, a video processing method provided by the present disclosure further includes, after generating a first linear special effect video frame using the first color space gamut:

[0173] The first linear special effects video frame using the first color space color gamut is encoded to generate a first linear special effects video for display on a display device.

[0174] According to one or more embodiments of the present disclosure, a video processing method provided by the present disclosure further includes, after generating a first linear special effect video frame using the first color space gamut:

[0175] Processing the first linear special effect video frame to generate a first nonlinear special effect video frame using the first color space gamut;

[0176] The first nonlinear special effects video frame is encoded to generate a first nonlinear special effects video stored in the first color space color gamut.

[0177] According to one or more embodiments of the present disclosure, a video processing method provided by the present disclosure further includes, after generating a first linear special effect video frame using the first color space gamut:

[0178] Performing color space conversion on a first linear special effect video frame using the first color space gamut to generate a second linear special effect video frame using the second color space gamut;

[0179] The second linear special effect video frame is encoded to generate a second linear special effect video for display on a display device.

[0180] According to one or more embodiments of the present disclosure, a video processing method provided by the present disclosure further includes, after generating a second linear special effect video frame using the second color space gamut:

[0181] Processing the second linear special effect video frame to generate a second nonlinear special effect video frame using the second color space gamut;

[0182] The second nonlinear special effects video frame is encoded to generate a second nonlinear special effects video stored in the second color space color gamut.

[0183] According to one or more embodiments of the present disclosure, a video processing method provided by the present disclosure further includes:

[0184] The data storage accuracy of the video frame is determined according to a storage device or a display device.

[0185] According to one or more embodiments of the present disclosure, the present disclosure provides a video processing device, including:

[0186] a decoding module, configured to decode a nonlinear video in a first color space gamut to obtain a corresponding first nonlinear video frame, and decode a nonlinear video in a second color space gamut to obtain a corresponding second nonlinear video frame, wherein the color gamut of the first color space is larger than the color gamut of the second color space;

[0187] a first conversion module, configured to process the second nonlinear video frame to generate a corresponding second linear video frame, and perform color space conversion on the second linear video frame to generate a corresponding third linear video frame using the first color space gamut;

[0188] a first generating module, configured to process the first nonlinear video frame to generate a corresponding first linear video frame, and obtain a first linear target video frame using the first color space gamut according to the first linear video frame and the third linear video frame;

[0189] The second generation module is used to obtain a linear special effect resource using the first color space color gamut, fuse the first linear target video frame and the linear special effect resource, and generate a first linear special effect video frame using the first color space color gamut.

[0190] According to one or more embodiments of the present disclosure, in a video processing device provided by the present disclosure:

[0191] The nonlinear video using the first color space gamut includes: a nonlinear HDR video using the Rec.2020 color space;

[0192] The nonlinear video using the second color space color gamut includes: a nonlinear SDR video using the sRGB color space.

[0193] According to one or more embodiments of the present disclosure, in a video processing device provided by the present disclosure, the first generating module is configured to:

[0194] performing splicing processing on the first linear video frame and the third linear video frame to obtain a first linear target video frame using the first color space gamut; and / or,

[0195] Pixels of the first linear video frame and the third linear video frame are superimposed to obtain a first linear target video frame using the first color space color gamut.

[0196] According to one or more embodiments of the present disclosure, in a video processing device provided by the present disclosure, the second generating module is configured to:

[0197] It is detected whether a nonlinear special effect resource adopts the first color space gamut; if so, the nonlinear special effect resource is processed to generate a linear special effect resource adopting the first color space gamut.

[0198] According to one or more embodiments of the present disclosure, a video processing device provided by the present disclosure further includes:

[0199] A first processing module is configured to, if the nonlinear special effect resource adopts the second color space gamut, process the nonlinear special effect resource to generate a corresponding linear special effect resource adopting the second color space gamut;

[0200] The second processing module is used to perform color space conversion processing on the linear special effect resource using the second color space color gamut to generate a corresponding linear special effect resource using the first color space color gamut.

[0201] According to one or more embodiments of the present disclosure, a video processing device provided by the present disclosure further includes:

[0202] The first encoding module is used to encode the first linear special effect video frame using the first color space color gamut to generate a first linear special effect video for display on a display device.

[0203] According to one or more embodiments of the present disclosure, a video processing device provided by the present disclosure further includes:

[0204] a third processing module, configured to process the first linear special effect video frame to generate a first nonlinear special effect video frame using the first color space gamut;

[0205] The second encoding module is used to encode the first nonlinear special effect video frame to generate a first nonlinear special effect video storage using the first color space color gamut.

[0206] According to one or more embodiments of the present disclosure, a video processing device provided by the present disclosure further includes:

[0207] a second conversion module, configured to perform color space conversion on the first linear special effect video frame using the first color space gamut to generate a second linear special effect video frame using the second color space gamut;

[0208] The third encoding module is used to encode the second linear special effect video frame to generate a second linear special effect video for display on a display device.

[0209] According to one or more embodiments of the present disclosure, a video processing device provided by the present disclosure further includes:

[0210] A fourth processing module, configured to process the second linear special effect video frame to generate a second nonlinear special effect video frame using the second color space gamut;

[0211] The fourth encoding module is used to encode the second nonlinear special effect video frame to generate a second nonlinear special effect video storage using the second color space color gamut.

[0212] According to one or more embodiments of the present disclosure, a video processing device provided by the present disclosure further includes:

[0213] The determination module is used to determine the data storage accuracy of the video frame according to a storage device or a display device.

[0214] According to one or more embodiments of the present disclosure, the present disclosure provides an electronic device, including:

[0215] processor;

[0216] a memory for storing instructions executable by the processor;

[0217] The processor is configured to read the executable instructions from the memory and execute the instructions to implement any one of the video processing methods provided in the present disclosure.

[0218] According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute any video processing method provided by the present disclosure.

[0219] The above description merely illustrates the preferred embodiments of the disclosure and a principle for applying the technologies. It is understood by those skilled in the art that the disclosed scope of the disclosure is not limited to the technical solutions formed by the specific combinations of the technical features described above, and should also cover other technical solutions formed by the combinations of the technical features described above or their equivalent features without departing from the disclosed concept. For example, the technical solutions formed by the mutual replacement of the above-described features and the technical features with similar functions disclosed in the disclosure (but not limited to) can be formed.

[0220] Furthermore, although operations are depicted in a particular, sequential order, this should not be understood as requiring or implying that the operations are performed in the order illustrated or sequentially. In certain circumstances, multitasking and parallel processing can be advantageous. Likewise, although specific implementation details are included for the purpose of providing a thorough disclosure, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable sub-combination.

[0221] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A video processing method, characterized in that: include: Decoding a nonlinear video in a first color space gamut to obtain a corresponding first nonlinear video frame, and decoding a nonlinear video in a second color space gamut to obtain a corresponding second nonlinear video frame, wherein the color gamut of the first color space is larger than the color gamut of the second color space; Processing the second nonlinear video frame to generate a corresponding second linear video frame, and performing color space conversion on the second linear video frame to generate a corresponding third linear video frame using the first color space gamut; Processing the first nonlinear video frame to generate a corresponding first linear video frame; Acquire a first linear target video frame using the first color space gamut according to the first linear video frame and the third linear video frame; wherein the first linear target video frame is determined based on a splicing process and / or an overlay process of video frames; Acquiring a linear special effect resource using the first color space gamut; wherein acquiring the linear special effect resource using the first color space gamut includes: detecting whether a nonlinear special effect resource uses the first color space gamut, and if so, processing the nonlinear special effect resource to generate a linear special effect resource using the first color space gamut; The first linear target video frame and the linear special effect resource are fused to generate a first linear special effect video frame using the first color space color gamut.

2. The method according to claim 1, characterized in that The nonlinear video using the first color space gamut includes: a nonlinear HDR video using the Rec.2020 color space; The nonlinear video using the second color space color gamut includes: a nonlinear SDR video using the sRGB color space.

3. The method according to claim 1, characterized in that The step of obtaining a first linear target video frame using the first color space gamut according to the first linear video frame and the third linear video frame includes: performing splicing processing on the first linear video frame and the third linear video frame to obtain a first linear target video frame using the first color space gamut; and / or, Pixels of the first linear video frame and the third linear video frame are superimposed to obtain a first linear target video frame using the first color space color gamut.

4. The method according to claim 1, wherein After detecting whether the nonlinear special effect resource adopts the first color space gamut, the method further includes: If the nonlinear special effect resource adopts the second color space gamut, processing the nonlinear special effect resource to generate a corresponding linear special effect resource adopting the second color space gamut; Color space conversion processing is performed on the linear special effect resource using the second color space color gamut to generate a corresponding linear special effect resource using the first color space color gamut.

5. The method according to claim 1, wherein After generating the first linear special effect video frame using the first color space gamut, the method further includes: The first linear special effects video frame using the first color space color gamut is encoded to generate a first linear special effects video for display on a display device.

6. The method according to claim 1, characterized in that After generating the first linear special effect video frame using the first color space gamut, the method further includes: Processing the first linear special effect video frame to generate a first nonlinear special effect video frame using the first color space gamut; The first nonlinear special effects video frame is encoded to generate a first nonlinear special effects video stored in the first color space color gamut.

7. The method according to claim 1, characterized in that After generating the first linear special effect video frame using the first color space gamut, the method further includes: Performing color space conversion on a first linear special effect video frame using the first color space gamut to generate a second linear special effect video frame using the second color space gamut; The second linear special effect video frame is encoded to generate a second linear special effect video for display on a display device.

8. The method according to claim 7, characterized in that After generating the second linear special effect video frame using the second color space gamut, the method further includes: Processing the second linear special effect video frame to generate a second nonlinear special effect video frame using the second color space gamut; The second nonlinear special effects video frame is encoded to generate a second nonlinear special effects video stored in the second color space color gamut.

9. The method according to any one of claims 1 to 8, characterized in that: Also includes: The data storage accuracy of the video frame is determined according to a storage device or a display device.

10. A video special effects processing device, characterized in that: The device comprises: a decoding module, configured to decode a nonlinear video in a first color space gamut to obtain a corresponding first nonlinear video frame, and decode a nonlinear video in a second color space gamut to obtain a corresponding second nonlinear video frame, wherein the color gamut of the first color space is larger than the color gamut of the second color space; a first conversion module, configured to process the second nonlinear video frame to generate a corresponding second linear video frame, and perform color space conversion on the second linear video frame to generate a corresponding third linear video frame using the first color space gamut; A first generating module is configured to process the first nonlinear video frame to generate a corresponding first linear video frame, and obtain a first linear target video frame using the first color space gamut based on the first linear video frame and the third linear video frame; wherein the first linear target video frame is determined based on a splicing process and / or an overlay process of the video frames; The second generation module is used to obtain linear special effects resources using the first color space color gamut, fuse the first linear target video frame and the linear special effects resources, and generate a first linear special effects video frame using the first color space color gamut; wherein, the obtaining of the linear special effects resources using the first color space color gamut includes: detecting whether a nonlinear special effects resource uses the first color space color gamut, and if so, processing the nonlinear special effects resource to generate a linear special effects resource using the first color space color gamut.

11. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing instructions executable by the processor; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the video processing method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device implements the video processing method according to any one of claims 1 to 9.

13. A computer program product, characterized in that The computer program product comprises a computer program / instruction, and when the computer program / instruction is executed by a processor, the video processing method according to any one of claims 1 to 9 is implemented.

Images