Video processing method and apparatus, device, and medium

Abstract

Embodiments of the present disclosure relate to a video processing method, device, equipment and medium, wherein the method comprises: decoding a nonlinear video to obtain a corresponding nonlinear video frame; processing the nonlinear video frame to generate a corresponding linear video frame; obtaining a target linear special effect resource consistent with a color space adopted by the linear video frame; and performing fusion processing on the linear video frame and the target linear special effect resource to generate a linear special effect video frame. Embodiments of the present disclosure process the nonlinear video frame to generate a corresponding linear video frame, and the special effect resource for special effect processing is also linear. Therefore, the fusion processing is performed on the linear video frame and the target linear special effect resource in the linear space, thereby ensuring the color accuracy of the generated linear special effect video frame, making the added special effect resource more natural, and improving the realism of the generated linear special effect video frame.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of data processing, and particularly relates to a video processing method and device, equipment and medium. BACKGROUND

[0002] With the development of computer technology, the application scenarios of video processing technology are increasingly wide, such as adding virtual stickers to a to-be-processed video, virtual try-on and the like.

[0003] However, the current video processing method may cause a color deviation between the to-be-processed video and the added virtual effect, and the color accuracy of the generated special effect video is insufficient. SUMMARY

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a video processing method, device, equipment and medium.

[0005] In a first aspect, the embodiments of the present disclosure provide a video processing method, and the method comprises:

[0006] decoding a nonlinear video to obtain a corresponding nonlinear video frame;

[0007] processing the nonlinear video frame to generate a corresponding linear video frame;

[0008] obtaining a target linear special effect resource consistent with a color space adopted by the linear video frame;

[0009] performing fusion processing on the linear video frame and the target linear special effect resource to generate a linear special effect video frame.

[0010] In an optional implementation, the decoding of the nonlinear video to obtain the corresponding nonlinear video frame comprises:

[0011] decoding a nonlinear HDR video to obtain a corresponding nonlinear HDR video frame;

[0012] Alternatively,

[0013] decoding a nonlinear SDR video to obtain a corresponding nonlinear SDR video frame.

[0014] In an optional implementation, the processing of the nonlinear video frame to generate the corresponding linear video frame comprises:

[0015] obtaining a video format of the nonlinear video frame;

[0016] calling a conversion function corresponding to the video format to perform conversion processing on the nonlinear video frame to generate a corresponding linear video frame.

[0017] In an optional implementation, the obtaining the target linear effect resource consistent with the color space adopted by the linear video frame comprises:

[0018] detecting whether the nonlinear effect resource is consistent with the color space adopted by the linear video frame;

[0019] if consistent, processing the nonlinear effect resource to generate the corresponding target linear effect resource.

[0020] In an optional implementation, the method further comprises:

[0021] if inconsistent, processing the nonlinear effect resource to generate an initial linear effect resource;

[0022] converting the initial linear effect resource into the target linear effect resource consistent with the color space adopted by the linear video frame according to a preset color space conversion function.

[0023] In an optional implementation, the fusing processing of the linear video frame and the target linear effect resource to generate a linear effect video frame comprises:

[0024] superimposing the target linear effect resource and a target region of the linear video frame to generate a linear effect video frame; and / or,

[0025] calculating the target linear effect resource and a target region pixel of the linear video frame to generate a linear effect video frame.

[0026] In an optional implementation, after the linear effect video frame is generated, the method further comprises:

[0027] encoding the linear effect video frame to generate a linear effect video for display on a display device.

[0028] In an optional implementation, after the linear effect video frame is generated, the method further comprises:

[0029] processing the linear effect video frame to generate a corresponding nonlinear effect video frame;

[0030] encoding the nonlinear effect video frame to generate a nonlinear effect video for storage.

[0031] In a second aspect, the embodiments of the present disclosure provide a video processing device, the device comprising:

[0032] a decoding module configured to decode a nonlinear video to obtain a corresponding nonlinear video frame;

[0033] The first processing module is configured to process the nonlinear video frame to generate a corresponding linear video frame.

[0034] The acquisition module is configured to acquire a target linear special effect resource consistent with a color space adopted by the linear video frame.

[0035] The fusion module is configured to perform fusion processing on the linear video frame and the target linear special effect resource to generate a linear special effect video frame.

[0036] In a third aspect, the present disclosure provides a computer-readable storage medium, which stores instructions, and when the instructions are executed on a terminal device, the terminal device implements the method described above.

[0037] In a fourth aspect, the present disclosure provides an electronic device, which includes a processor, a memory for storing executable instructions of the processor, and the processor is configured to read the executable instructions from the memory and execute the instructions to implement the method described above.

[0038] In a fifth aspect, the present disclosure provides a computer program product, which includes computer programs / instructions, and when the computer programs / instructions are executed by a processor, the method described above is implemented.

[0039] The technical solution provided by the embodiments of the present disclosure has at least the following advantages compared with the prior art:

[0040] The video processing method provided by the embodiments of the present disclosure processes a nonlinear video frame to generate a corresponding linear video frame, and the special effect resource for special effect processing is also linear. Therefore, the linear video frame and the target linear special effect resource are fused in a linear space, so as to ensure the color accuracy of the generated linear special effect video frame, make the added special effect resource more natural, and improve the realism of the generated linear special effect video frame. BRIEF DESCRIPTION OF DRAWINGS

[0041] The above and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the original and elements are not necessarily drawn according to the scale.

[0042] Figure 1 A flowchart of a video processing method provided by the embodiments of the present disclosure;

[0043] Figure 2a A schematic diagram of a linear space from black to white provided by the embodiments of the present disclosure;

[0044] Figure 2bA diagram of a non-linear space from black to white is provided for an embodiment of the present disclosure;

[0045] Figure 2c A diagram of a non-linear space and a comparison of non-linear spaces is provided for an embodiment of the present disclosure;

[0046] Figure 3 A diagram of a correspondence between a linear space and a non-linear space is provided for an embodiment of the present disclosure;

[0047] Figure 4 A diagram of a three-dimensional color space is provided for an embodiment of the present disclosure;

[0048] Figure 5a A diagram of generating a linear feature video frame is provided for an embodiment of the present disclosure;

[0049] Figure 5b Another diagram of generating a linear feature video frame is provided for an embodiment of the present disclosure;

[0050] Figure 6 A structural diagram of a video processing apparatus is provided for an embodiment of the present disclosure;

[0051] Figure 7 A structural diagram of an electronic device is provided for an embodiment of the present disclosure. DETAILED DESCRIPTION

[0052] Embodiments of the present disclosure will be described more fully hereinafter with reference to the accompanying drawings. While several embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be construed as being limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and fully convey the scope of the present disclosure to those skilled in the art. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of the present disclosure.

[0053] It should be understood that the various steps of the method embodiments of the present disclosure can be performed in different orders and / or in parallel. In addition, the method embodiments can include additional steps and / or omit performing the steps shown. The scope of the present disclosure is not limited in this regard.

[0054] The term "comprising" and variations thereof as used herein are open-ended, that is, "comprising but not limited to." The term "based on" is "based, at least in part, on." The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments." Related terms are defined as follows.

[0055] It should be noted that the terms "first", "second", and the like in the present disclosure are only used to distinguish different devices, modules or units, and are not intended to limit the order or interdependence of the functions performed by these devices, modules or units.

[0056] It should be noted that the terms "one", "multiple" in the present disclosure are illustrative and not restrictive, and those skilled in the art should understand that unless otherwise explicitly indicated in the context, it should be understood as "one or more".

[0057] The names of the messages or information exchanged between the plurality of devices in the embodiments of the present disclosure are only for illustrative purposes, and are not intended to limit the scope of the messages or information.

[0058] To solve the above problems, the present disclosure provides a video processing method, which will be introduced below in combination with specific embodiments.

[0059] Figure 1 A flowchart of a video processing method provided by the embodiments of the present disclosure is shown in the figure. The method can be executed by a video processing device, which can be implemented by software and / or hardware, and can be integrated in an electronic device. As shown in the figure, the method comprises the following steps. Figure 1

[0060] Step 101: decoding a nonlinear video to obtain a corresponding nonlinear video frame.

[0061] Generally, the video obtained by shooting with a mobile phone and / or edited and processed is a nonlinear video. In some embodiments of the present disclosure, the video format of the nonlinear video is various, for example, a nonlinear High Dynamic Range Imaging (HDR) video with a video format of High Dynamic Range Imaging (HDR), or a nonlinear Standard Dynamic Range (SDR) video with a video format of Standard Dynamic Range (SDR).

[0062] In the present embodiment, the nonlinear video refers to the conversion of the linear space of the video into a nonlinear space. In the linear space, the relationship between the value corresponding to the pixel point and the luminous power of the pixel point is linear, as shown in the figure. Figure 2a Figure 2a A schematic diagram of a linear space from black to white is provided in the embodiments of the present disclosure. Since the human eye is more sensitive to dark colors, the human eye sees the nonlinear video as shown in the figure. Figure 2a ​​The bright color region in the linear space is more than the dark color region. In order to balance the area of the bright color region and the dark color region, and make the area of the bright color region and the dark color region similar to the human eye, the linear space can be processed in a non-linear manner. In the non-linear space, the relationship between the value corresponding to the pixel point and the luminous power of the pixel point is non-linear, as shown in Figure 2b Figure 2b A schematic diagram of a non-linear space from black to white is provided for the embodiments of the present disclosure. Compared with Figure 2a , the dark color region seen by the human eye becomes more and the bright color region becomes less, so that the area of the bright color region and the dark color region seen by the human eye is similar.

[0063] In an optional implementation, the non-linear space is a gamma space and the corresponding gamma value is 2.2. In order to more clearly illustrate the relationship between the linear space and the non-linear space, as shown in Figure 2c Figure 2c In the figure, the dashed line represents the boundary line between the dark color region and the bright color region seen by the human eye. In the linear space, the value corresponding to the boundary line is 21.76%, and in the non-linear space, the value corresponding to the boundary line is 50%. It can be seen that, except for the color with a value of 0 or 100%, the values corresponding to the same human visual color in the linear space and the non-linear space are different. Similarly, the human visual color corresponding to the same value in the linear space and the non-linear space can also be different.

[0064] It can be seen that the video processed in a non-linear manner can better balance the proportion of bright color and dark color, so as to meet the visual perception of the human eye. It is easy to understand that the video frames in the non-linear video are also non-linear. The decoding process can decode the video into corresponding frames. In the present embodiment, in order to perform special effect processing on the non-linear video, the non-linear video needs to be decoded to obtain corresponding non-linear video frames. The decoding manner can be selected according to the application scenario, and the present embodiment does not limit the decoding manner. For example, soft decoding and hard decoding. For example, a non-linear HDR video is decoded to obtain corresponding non-linear HDR video frames; or a non-linear SDR video is decoded to obtain corresponding non-linear SDR video frames.

[0065] Step 102, processing the non-linear video frame to generate a corresponding linear video frame.

[0066] In order to meet the needs of video processing such as special effect processing, for example, illumination special effect, face skinning, etc. calculation processing, the video frame needs to be processed in a linear space for special effect processing. Therefore, the non-linear video frame needs to be processed to generate a corresponding linear video frame.

[0067] ​​In some embodiments of the present disclosure, different conversion functions are used to process the non-linear video frames of different video formats to generate corresponding linear video frames according to the needs of the application scenarios. For example, in some embodiments of the present disclosure, the video format of the non-linear video frame to be processed is first determined, and then a conversion function corresponding to the video format is called to convert and process the non-linear video frame to generate a corresponding linear video frame. It should be noted that the conversion functions corresponding to different video formats are pre-set according to the application scenarios of video processing, for example, a non-linear HDR video frame is converted into a linear HDR video frame by using an HDR conversion function corresponding to the HDR video format, or a non-linear SDR video frame is converted into a linear SDR video frame by using an SDR conversion function corresponding to the SDR video format. In order to more clearly illustrate the linear conversion process of the non-linear video frame, the SDR non-linear video frame is taken as an example for illustration as follows: Figure 3 Figure 3 In the figure, the horizontal coordinate is a color diagram of a linear video frame from black to white, and the vertical coordinate is a color diagram of an SDR non-linear video frame from black to white. The curve in the figure represents the conversion and correspondence relationship between the color of the linear video frame and the color of the SDR non-linear video frame. According to the relationship, the SDR non-linear video frame can be converted into an SDR linear video frame.

[0068] In step 103, a target linear special effect resource consistent with the color space of the linear video frame is obtained.

[0069] In the present embodiment, the color space is a model used to represent colors, and the color ranges that can be represented by different color spaces are different. For example, as shown in Figure 4 Figure 4 FIG. 1 is a schematic diagram of a three-dimensional color space provided by an embodiment of the present disclosure. In the diagram, the color range covered by the triangle corresponding to each color space represents the color range that can be represented by the color space. As can be seen from the diagram, the color range that can be represented by Rec. 2020 (Recommendation ITU-R-BT. 2020) is larger than the color range that can be represented by the standard Red Green Blue (sRGB).

[0070] It should be noted that the colors represented by the same numerical value in different color spaces can be different. For example, in a three-dimensional color space, the color can be represented by three-dimensional coordinates, where each dimension coordinate has a value of 0-1, where 0 represents that the color is not taken, and 1 represents that the color is taken to the maximum in the color space. See Figure 4 ​​, the upper vertex of the triangle corresponding to Rec. 2020 is the point representing the green primary color in Rec. 2020, which is represented as (0, 1, 0), and the upper vertex of the triangle corresponding to sRGB is the point representing the green primary color in Rec. 709, which is also represented as (0, 1, 0). It can be seen from Figure 4 that although the green primary color in Rec. 2020 and the green primary color in sRGB are both represented as (0, 1, 0), the green primary color in Rec. 2020 actually represents a different color from the green primary color in Rec. 709.

[0071] Therefore, in order to ensure that the color of the video frame after special effect processing is accurate and good in quality, it is necessary to ensure that the target linear special effect resource obtained before special effect processing is consistent with the color space adopted by the linear video frame to be processed, and then to perform special effect processing on the linear video frame according to the target linear special effect resource. However, in actual scene applications, the special effect resource to be processed may be linear or nonlinear, and the color space adopted by the special effect resource to be processed may be consistent or inconsistent with the color space adopted by the linear video frame. Therefore, in order to obtain a target linear special effect resource consistent with the color space adopted by the linear video frame to be processed, the nonlinear special effect resource can be linearly processed into a linear special effect resource, and it is determined that the color space corresponding to the linear special effect resource is consistent with the color space corresponding to the linear video frame to be processed. The nonlinear special effect resource is specifically described as follows:

[0072] In one embodiment, it is detected whether the nonlinear special effect resource to be processed is consistent with the color space adopted by the linear video frame. If it is consistent, the nonlinear special effect resource is processed to generate a corresponding target linear special effect resource. For example, it is assumed that the linear video frame to be processed is a linear SDR video frame adopting sRGB color space. If it is detected that the nonlinear special effect resource is a nonlinear SDR format map adopting sRGB color space, it is determined that the color space adopted by the linear SDR video frame and the nonlinear SDR format map is consistent. Then, the nonlinear SDR format map is linearly processed according to the SDR conversion function to generate a corresponding linear SDR format map, and the linear SDR video frame is processed in the linear space according to the linear SDR format map.

[0073] In another embodiment, based on the previous embodiment, if it is detected that the color space adopted by the nonlinear special effect resource is inconsistent with the color space adopted by the linear video frame, the nonlinear special effect resource is processed to generate a corresponding initial linear special effect resource. For example, assuming that the linear video frame to be processed is a linear HDR video frame adopting a Rec. 2020 color space, if it is detected that the nonlinear special effect resource is a nonlinear SDR format map adopting an sRGB color space, it is determined that the color space adopted by the linear HDR video frame and the nonlinear SDR format map is inconsistent. Then, the nonlinear SDR format map is linearly processed according to an SDR conversion function to generate a corresponding linear SDR format map, and the linear SDR format map is converted into a linear HDR format map according to a color space conversion relationship, so as to perform special effect processing on the linear HDR video frame in a unified color space in the linear space according to the linear HDR format map.

[0074] It should be noted that if the special effect resource to be processed is linear, it is necessary to detect whether the color space adopted by the linear special effect resource to be processed is consistent with the color space adopted by the linear video frame. If consistent, the linear special effect resource is a target linear special effect resource. For example, assuming that the linear video frame to be processed is a linear HDR video frame adopting a Rec. 2020 color space, if it is detected that the linear special effect resource is a linear HDR format map adopting a Rec. 2020 color space, the linear resource can be applied to perform special effect processing on the linear video frame.

[0075] Further, if it is detected that the color space adopted by the linear special effect resource to be processed is inconsistent with the color space adopted by the linear video frame, the linear special effect resource to be processed is color space converted to generate a target linear special effect resource. For example, assuming that the linear video frame to be processed is a linear HDR video frame adopting a Rec. 2020 color space, if it is detected that the linear special effect resource is a linear SDR format map adopting an sRGB color space, it is determined that the color space adopted by the linear HDR video frame and the linear SDR format map is inconsistent. Then, the linear SDR format map is converted into a linear HDR format map according to a color space conversion relationship, so as to perform special effect processing on the linear HDR video frame in a unified color space in the linear space according to the linear HDR format map.

[0076] Step 104, performing fusion processing on the linear video frame and the target linear special effect resource to generate a linear special effect video frame.

[0077] After determining the target linear special effect resource consistent with the color space adopted by the linear video frame and linear, the linear video frame and the target linear special effect resource are fused, the target linear special effect resource is applied to the linear video frame through the fusion processing, and the linear video frame is processed according to the special effect application needs, such as adding virtual stickers and / or blurring, so as to generate a linear special effect video frame, which will be described in detail as follows:

[0078] In one embodiment, as shown in Figure 5a After the foregoing steps, the target linear special effect resource consistent with the color space adopted by the linear video frame is obtained, wherein the target linear special effect resource is a heart-shaped picture, the linear video frame is a thumbs-up gesture, and the target area is the area above the intersection of the index finger and the thumb in the thumbs-up gesture. The heart-shaped picture is superimposed with the target area of the thumbs-up gesture, and when the pixels of the linear video frame and the pixels of the target linear special effect resource coincide, the pixels of the target linear special effect resource are retained, so that the superimposition processing in the target area is realized. A linear feature video frame as shown in Figure 5a A new sticker effect is generated above the intersection of the index finger and the thumb in the thumbs-up gesture.

[0079] In another embodiment, as shown in Figure 5b After the foregoing steps, the target linear special effect resource consistent with the color space adopted by the linear video frame is obtained, wherein the target linear special effect resource is a brown picture, and the linear video frame is a portrait. The target area is a rectangular area containing the portrait. The brown picture is calculated with the target area of the portrait, and when the pixels of the linear video frame in the target area and the pixels of the target linear special effect resource coincide, the average value of each coincident pixel is taken, so that the calculation processing in the target area is realized. A linear feature video frame as shown in Figure 5b The portrait area is blurred.

[0080] In summary, the video processing method of the embodiments of the present disclosure generates a corresponding linear video frame by processing a non-linear video frame, and the special effect resource for special effect processing is also linear. Therefore, the linear video frame and the target linear special effect resource are fused in the linear space, thereby ensuring the color accuracy of the generated linear special effect video frame, making the added special effect resource more natural, and improving the realism of the generated linear special effect video frame.

[0081] Based on the above embodiments, further, after generating the linear special effect video frame, it further includes: encoding the linear special effect video frame to generate a linear special effect video displayed on a display device.

[0082] In order to obtain the linear special effect video, the linear special effect video frame is encoded and processed to generate the linear special effect video. The encoding processing can synthesize the linear special effect video frame into the corresponding linear special effect video. The encoding processing can be selected according to the application scenario, and the embodiment is not limited, for example, soft encoding and hard encoding. Thus, through the aforementioned linear processing, color space conversion and other technical means, the accuracy of the color of the special effect video displayed on the display device is enhanced, and the effect of the special effect is more realistic.

[0083] Based on the above embodiments, further, after generating the linear special effect video frame, in another optional embodiment, the video obtained after processing needs to be stored in a storage medium. Generally, the video stored in the storage medium is a nonlinear video. Therefore, based on the above embodiments, the method further includes:

[0084] The linear special effect video frame is processed to generate a corresponding nonlinear special effect video frame. The processing can be used to convert the linear video frame into the nonlinear video frame. The processing method can be selected according to the application scenario, and the embodiment is not limited, for example, using a transfer function processing. In an optional embodiment, the processing method is using an optical-to-electric transfer function (Opo-electronic, OETF) for processing.

[0085] Further, the nonlinear special effect video frame is encoded and processed to generate nonlinear special effect video storage. The encoding processing can synthesize the nonlinear video frame into the corresponding nonlinear video. The encoding processing can be selected according to the application scenario, and the embodiment is not limited, for example, soft encoding and hard encoding. Thus, the nonlinear special effect video frame can be processed by using the encoding method to generate the corresponding nonlinear special effect video, and the storage device is used for storage. Thus, the storage of the special effect video is realized. If the special effect video is needed in the future, it can be obtained from the storage device, so that the acquisition of the special effect video is more convenient.

[0086] Figure 6 A structure diagram of a video processing device provided by the embodiment of the present disclosure is provided. The device can be realized by software and / or hardware, and can be integrated in an electronic device. As shown in the figure, the device includes: Figure 6

[0087] The decoding module 601 is configured to decode the nonlinear video to obtain the corresponding nonlinear video frame.

[0088] The first processing module 602 is configured to process the nonlinear video frame to generate the corresponding linear video frame.

[0089] The acquisition module 603 is configured to acquire a target linear special effect resource consistent with the color space of the linear video frame. ​

[0090] fusing the linear video frame and the target linear special effect resource to generate a linear special effect video frame.

[0091] Optionally, the decoding module 601 is configured to:

[0092] decode a nonlinear HDR video to obtain a corresponding nonlinear HDR video frame;

[0093] Optionally, the decoding module 601 is configured to:

[0094] decode a nonlinear SDR video to obtain a corresponding nonlinear SDR video frame.

[0095] Optionally, the first processing module 602 is configured to:

[0096] obtain a video format of the nonlinear video frame;

[0097] invoke a conversion function corresponding to the video format to convert the nonlinear video frame to generate a corresponding linear video frame.

[0098] Optionally, the obtaining module 603 is configured to:

[0099] detect whether a color space adopted by the nonlinear special effect resource and the linear video frame is consistent;

[0100] if the color spaces are consistent, process the nonlinear special effect resource to generate a corresponding target linear special effect resource.

[0101] Optionally, the obtaining module 603 is further configured to:

[0102] if the color spaces are inconsistent, process the nonlinear special effect resource to generate a corresponding initial linear special effect resource;

[0103] convert the initial linear special effect resource into the target linear special effect resource consistent with the color space adopted by the linear video frame according to a preset color space conversion function.

[0104] Optionally, the fusing module 604 is configured to:

[0105] superimpose the target linear special effect resource and a target region of the linear video frame to generate a linear special effect video frame; and / or,

[0106] perform calculation processing on the target linear special effect resource and a target region pixel of the linear video frame to generate a linear special effect video frame.

[0107] Optionally, the apparatus further includes:

[0108] The first encoding module is configured to encode the linear special effect video frame to generate a linear special effect video for display on a display device.

[0109] Optionally, the apparatus further includes:

[0110] The second processing module is configured to process the linear special effect video frame to generate a corresponding nonlinear special effect video frame.

[0111] The second encoding module is configured to encode the nonlinear special effect video frame to generate a nonlinear special effect video for storage.

[0112] The video processing apparatus provided by the embodiments of the present disclosure can execute the video processing method provided by any of the embodiments of the present disclosure, and has the corresponding function modules and beneficial effects of the execution method.

[0113] The embodiments of the present disclosure further provide a computer program product, which includes computer programs / instructions, and the computer programs / instructions are executed by a processor to implement the video processing method provided by any of the embodiments of the present disclosure.

[0114] Figure 7 A structural schematic diagram of an electronic device provided by the embodiments of the present disclosure is provided.

[0115] Reference will be made in detail to Figure 7 , which shows a structural schematic diagram of an electronic device 700 suitable for implementing the embodiments of the present disclosure. The electronic device 700 in the embodiments of the present disclosure can include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (Personal Digital Assistant), a PAD (Tablet Personal Computer), a PMP (Portable Multimedia Player), a vehicle terminal (such as a vehicle navigation terminal), a wearable electronic device, and the like, and a fixed terminal such as a digital TV, a desktop computer, a smart home device, and the like. Figure 7 The electronic device shown is only an example, and should not bring any limitation to the functions and use range of the embodiments of the present disclosure.

[0116] As shown in Figure 7 , the electronic device 700 can include a processing apparatus (such as a central processor, a graphics processor, etc.) 701, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 702 or programs loaded from a storage apparatus 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 are also stored. The processing apparatus 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0117] In general, the following devices can be connected to the I / O interface 705: input devices 706 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, and the like; output devices 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, and the like; storage devices 708 including, for example, a magnetic tape, a hard disk, and the like; and communication devices 709. The communication devices 709 can allow the electronic device 700 to communicate wirelessly or wired with other devices to exchange data. Although Figure 7 The electronic device 700 is shown with various devices, but it is understood that all of the illustrated devices are not required to implement or have the electronic device. More or less devices can alternatively be implemented or have.

[0118] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for executing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network through the communication devices 709, or installed from the storage devices 708, or installed from the ROM 702. When the computer program is executed by the processing devices 701, the above-mentioned functions defined in the video processing method of embodiments of the present disclosure are performed.

[0119] It should be noted that the computer-readable medium described above can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program used by or in connection with an instruction execution system, apparatus or device. In the present disclosure, the computer-readable signal medium can include a data signal propagated in baseband or propagated as a carrier wave in a propagated data signal, in which the computer-readable program code is contained. Such a propagated data signal can take many forms, including but not limited to, an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium that can send, propagate or transfer the program for use by or in connection with the instruction execution system, apparatus or device. The program code contained in the computer-readable medium can be transmitted by any suitable medium, including but not limited to, wire, cable, RF (radio frequency), etc., or any suitable combination of the above.

[0120] In some embodiments, the client, server, or both can communicate using any current known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet, and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any current known or future developed networks.

[0121] The computer-readable medium described above can be included in the electronic device described above; or can exist separately from the electronic device, and is not assembled into the electronic device.

[0122] The computer readable medium carries one or more programs when the one or more programs are executed by the electronic device, the electronic device is caused to: decode the nonlinear video to obtain a corresponding nonlinear video frame; process the nonlinear video frame to generate a corresponding linear video frame; obtain a target linear special effect resource consistent with a color space adopted by the linear video frame; and perform fusion processing on the linear video frame and the target linear special effect resource to generate a linear special effect video frame. The nonlinear video frame is processed to generate a corresponding linear video frame in the embodiment of the disclosure, and the special effect resource for special effect processing is also linear. Therefore, the linear video frame and the target linear special effect resource are fused in the linear space, so as to ensure the color accuracy of the generated linear special effect video frame, make the added special effect resource more natural, and improve the realism of the generated linear special effect video frame.

[0123] Computer program code for carrying out operations of the present disclosure can be written in one or more programming languages or combinations of languages including object oriented programming languages such as Java, Smalltalk, C++ or conventional procedural programming languages such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0124] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of present disclosure. In this regard, each block in the flow diagrams or block diagrams can represent a module, a procedure, or a portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may

[0125] The units described in the embodiments of the present disclosure can be implemented in the form of software, or can be implemented in the form of hardware. In some cases, the name of the unit does not constitute a limitation on the unit itself.

[0126] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that can be used include: field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-chips (SOCs), complex programmable logic devices (CPLDs), etc.

[0127] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0128] According to one or more embodiments of the present disclosure, the present disclosure provides a video processing method, comprising:

[0129] decoding a nonlinear video to obtain a corresponding nonlinear video frame;

[0130] processing the nonlinear video frame to generate a corresponding linear video frame;

[0131] obtaining a target linear special effect resource consistent with a color space adopted by the linear video frame;

[0132] fusing the linear video frame and the target linear special effect resource to generate a linear special effect video frame.

[0133] According to one or more embodiments of the present disclosure, in a video processing method provided by the present disclosure, the decoding of the nonlinear video to obtain the corresponding nonlinear video frame comprises:

[0134] decoding a nonlinear HDR video to obtain a corresponding nonlinear HDR video frame;

[0135] Alternatively,

[0136] decoding the nonlinear SDR video to obtain a corresponding nonlinear SDR video frame.

[0137] According to one or more embodiments of the present disclosure, the present disclosure provides a video processing method, wherein the processing the nonlinear video frame to generate a corresponding linear video frame comprises:

[0138] obtaining a video format of the nonlinear video frame;

[0139] calling a conversion function corresponding to the video format to convert the nonlinear video frame to generate a corresponding linear video frame.

[0140] According to one or more embodiments of the present disclosure, the present disclosure provides a video processing method, wherein the obtaining a target linear special effect resource consistent with a color space adopted by the linear video frame comprises:

[0141] detecting whether the nonlinear special effect resource is consistent with a color space adopted by the linear video frame;

[0142] if consistent, processing the nonlinear special effect resource to generate a corresponding target linear special effect resource.

[0143] According to one or more embodiments of the present disclosure, the present disclosure provides a video processing method, wherein the processing the nonlinear video frame to generate a corresponding linear video frame comprises:

[0144] if inconsistent, processing the nonlinear special effect resource to generate a corresponding initial linear special effect resource;

[0145] converting the initial linear special effect resource into the target linear special effect resource consistent with the color space adopted by the linear video frame according to a preset color space conversion function.

[0146] According to one or more embodiments of the present disclosure, the present disclosure provides a video processing method, wherein the processing the nonlinear video frame to generate a corresponding linear video frame comprises:

[0147] superimposing the target linear special effect resource on a target region of the linear video frame to generate a linear special effect video frame; and / or,

[0148] calculating the target linear special effect resource and pixels of the target region of the linear video frame to generate a linear special effect video frame.

[0149] According to one or more embodiments of the present disclosure, the present disclosure provides a video processing method, wherein after the linear special effect video frame is generated, the method further comprises:

[0150] The linear special effect video frame is encoded to generate a linear special effect video displayed on a display device.

[0151] According to one or more embodiments of the present disclosure, the video processing method provided by the present disclosure further includes, after the linear special effect video frame is generated:

[0152] processing the linear special effect video frame to generate a corresponding nonlinear special effect video frame;

[0153] encoding the nonlinear special effect video frame to generate a nonlinear special effect video storage.

[0154] According to one or more embodiments of the present disclosure, the video processing device provided by the present disclosure includes:

[0155] a decoding module configured to decode a nonlinear video to obtain a corresponding nonlinear video frame;

[0156] a first processing module configured to process the nonlinear video frame to generate a corresponding linear video frame;

[0157] an obtaining module configured to obtain a target linear special effect resource consistent with a color space of the linear video frame;

[0158] a fusion module configured to fuse the linear video frame and the target linear special effect resource to generate a linear special effect video frame.

[0159] According to one or more embodiments of the present disclosure, the video processing device provided by the present disclosure includes:

[0160] decoding a nonlinear HDR video to obtain a corresponding nonlinear HDR video frame;

[0161] or,

[0162] decoding a nonlinear SDR video to obtain a corresponding nonlinear SDR video frame.

[0163] According to one or more embodiments of the present disclosure, the video processing device provided by the present disclosure includes:

[0164] obtaining a video format of the nonlinear video frame;

[0165] calling a conversion function corresponding to the video format to convert the nonlinear video frame to generate a corresponding linear video frame.

[0166] According to one or more embodiments of the present disclosure, the video processing device provided by the present disclosure includes:

[0167] detect whether the color space adopted by the nonlinear special effect resource is consistent with the color space adopted by the linear video frame;

[0168] If consistent, the nonlinear special effect resource is processed to generate the corresponding target linear special effect resource.

[0169] According to one or more embodiments of the present disclosure, the present disclosure provides a video processing device, wherein the acquisition module is further configured to:

[0170] If inconsistent, the nonlinear special effect resource is processed to generate an initial linear special effect resource;

[0171] According to a preset color space conversion function, the initial linear special effect resource is converted into the target linear special effect resource consistent with the color space adopted by the linear video frame.

[0172] According to one or more embodiments of the present disclosure, the present disclosure provides a video processing device, wherein the fusion module is configured to:

[0173] superimpose the target linear special effect resource and a target region of the linear video frame to generate a linear special effect video frame; and / or,

[0174] calculate and process the target linear special effect resource and a target region pixel of the linear video frame to generate a linear special effect video frame.

[0175] According to one or more embodiments of the present disclosure, the present disclosure provides a video processing device, wherein the device further comprises:

[0176] a first encoding module configured to encode and process the linear special effect video frame to generate a linear special effect video for display on a display device.

[0177] According to one or more embodiments of the present disclosure, the present disclosure provides a video processing device, wherein the device further comprises:

[0178] a second processing module configured to process the linear special effect video frame to generate a corresponding nonlinear special effect video frame;

[0179] a second encoding module configured to encode and process the nonlinear special effect video frame to generate a nonlinear special effect video for storage.

[0180] According to one or more embodiments of the present disclosure, the present disclosure provides an electronic device, comprising:

[0181] a processor;

[0182] a memory for storing instructions executable by the processor;

[0183] The processor is configured to read the executable instructions from the memory and execute the instructions to implement any of the video processing methods provided by the present disclosure.

[0184] According to one or more embodiments of the present disclosure, the present disclosure provides a computer readable storage medium, which stores a computer program for executing any of the video processing methods provided by the present disclosure.

[0185] The above description is merely illustrative of the exemplary embodiments of the present disclosure and the principles of the technology involved. It is understood that the disclosed scope of the present disclosure is not limited to the technical solutions formed by the specific combinations of the above technical features, and also covers other technical solutions formed by the combinations of the above technical features or equivalent features thereof without departing from the above disclosed concept. For example, the technical solutions formed by the mutual replacement of the above features and the technical features disclosed in the present disclosure (but not limited to) having similar functions.

[0186] In addition, although each operation is depicted in a particular order, this should not be understood as requiring the operations to be performed in the particular order shown or in a sequential order. In certain circumstances, multitasking and parallel processing can be advantageous. Similarly, although several implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments can also be combined in a single embodiment. Conversely, various features described in the context of a single embodiment can also be implemented separately or in any suitable sub-combination.

[0187] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A method of video processing, the method comprising: The method comprises: decoding a nonlinear video to obtain a corresponding nonlinear video frame; wherein the nonlinear video and the nonlinear video frame are in a nonlinear space, and a relationship between a value corresponding to a pixel point and a luminous power of the pixel point in the nonlinear space is nonlinear; processing the nonlinear video frame to generate a corresponding linear video frame; wherein the linear video frame is in a linear space, and a relationship between a value corresponding to a pixel point and a luminous power of the pixel point in the linear space is linear; obtaining a target linear special effect resource consistent with a color space adopted by the linear video frame; fusing the linear video frame and the target linear special effect resource to generate a linear special effect video frame.

2. The method of claim 1, wherein, The decoding of the nonlinear video to obtain the corresponding nonlinear video frame comprises: decoding a nonlinear HDR video to obtain a corresponding nonlinear HDR video frame; or, decoding a nonlinear SDR video to obtain a corresponding nonlinear SDR video frame.

3. The method of claim 1, wherein, The processing of the nonlinear video frame to generate the corresponding linear video frame comprises: obtaining a video format of the nonlinear video frame; calling a conversion function corresponding to the video format to convert and process the nonlinear video frame to generate the corresponding linear video frame.

4. The method of claim 1, wherein, The obtaining of the target linear special effect resource consistent with the color space adopted by the linear video frame comprises: detecting whether a nonlinear special effect resource is consistent with a color space adopted by the linear video frame; if consistent, processing the nonlinear special effect resource to generate the corresponding target linear special effect resource.

5. The method of claim 4, wherein, The method further comprises: if inconsistent, processing the nonlinear special effect resource to generate a corresponding initial linear special effect resource; converting the initial linear special effect resource into the target linear special effect resource consistent with the color space adopted by the linear video frame according to a preset color space conversion function.

6. The method of claim 1, wherein, The fusing of the linear video frame and the target linear special effect resource to generate the linear special effect video frame comprises: superimposing the target linear special effect resource on a target region of the linear video frame to generate the linear special effect video frame; and / or, calculating the target linear special effect resource and pixels of the target region of the linear video frame to generate the linear special effect video frame.

7. The method of claim 1, wherein, After the generation of the linear special effect video frame, the method further comprises: encoding and processing the linear special effect video frame to generate a linear special effect video for display on a display device.

8. The method of claim 1, wherein, After the generation of the linear special effect video frame, the method further comprises: processing the linear special effect video frame to generate a corresponding nonlinear special effect video frame; encoding and processing the nonlinear special effect video frame to generate a nonlinear special effect video for storage.

9. A video processing apparatus, comprising: The apparatus comprises: a decoding module configured to decode a nonlinear video to obtain a corresponding nonlinear video frame; wherein the nonlinear video and the nonlinear video frame are in a nonlinear space, and a relationship between a value corresponding to a pixel point and a luminous power of the pixel point in the nonlinear space is nonlinear; The first processing module is configured to process the nonlinear video frame to generate a corresponding linear video frame; wherein the linear video frame is in a linear space, and a relationship between a value of a pixel point in the linear space and a luminous power of the pixel point is linear; The acquisition module is configured to acquire a target linear special effect resource consistent with a color space of the linear video frame; The fusion module is configured to perform fusion processing on the linear video frame and the target linear special effect resource to generate a linear special effect video frame.

10. An electronic device, comprising: The electronic device comprises: a processor; a memory configured to store executable instructions of the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the video processing method according to any one of claims 1-8.

11. A computer readable storage medium characterized by, The computer readable storage medium stores instructions, and when the instructions run on a terminal device, the terminal device implements the video processing method according to any one of claims 1-8.

12. A computer program product, characterised in that, The computer program product comprises computer programs / instructions, and when the computer programs / instructions are executed by a processor, the video processing method according to any one of claims 1-8 is implemented.

Images