Video processing method and device, and electronic equipment

Abstract

The embodiment of the invention provides a video processing method. The video processing method comprises the following steps: judging whether video frames of a foreground video material and a background video material have blank frames or not; if neither the foreground video material nor the background video material exists, aliasing the video frames of the foreground video material and the background video material; and if yes, skipping the blank frame, and performing aliasing processing on video frames, except the blank frame, of the foreground video material and the background video material. According to the video processing method provided by the embodiment of the invention, the blank frame in the video frames of the foreground video material and the background video material is identified, and the blank frame is skipped to perform aliasing processing, so that the calculation speed of aliasing processing or color mixing processing is accelerated. The embodiment of the invention further provides a video processing device and electronic equipment.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing technology, and in particular, to a video processing method, device, and electronic device. Background Art

[0002] In digital video processing, color mixing technology is used to adjust the hue, saturation and brightness of videos, as well as to implement special effects and filters. It is essential for video content production and optimization in areas such as video post-production, online education and training, and telemedicine.

[0003] Color mixing is a frequent and computationally intensive task, especially when processing high-resolution and high-frame-rate videos, the amount of computation and the required computing resources will increase significantly. Therefore, how to speed up the calculation of color mixing and reduce the resources required for color mixing calculations is an urgent problem to be solved. Summary of the invention

[0004] In response to the problems existing in the above-mentioned prior art, the embodiments of the present application provide a video processing method, device, and electronic device, which accelerates the calculation speed of aliasing processing or color mixing processing by identifying blank frames in the video frames of foreground video materials and background video materials, and skipping the blank frames for aliasing processing.

[0005] In a first aspect, an embodiment of the present application provides a video processing method, comprising the following steps:

[0006] Determine whether there are blank frames in the video frames of the foreground video material and the background video material;

[0007] If neither exists, performing aliasing processing on the video frames of the foreground video material and the background video material; and

[0008] If it exists, the blank frame is skipped, and aliasing processing is performed on the video frames of the foreground video material and the background video material except the blank frame.

[0009] Furthermore, before determining whether there are blank frames in the video frames of the foreground video material and the background video material, the method further includes:

[0010] For the foreground video material and the background video material, a frame attribute table sequence is created to determine whether there are blank frames in the video frames of the foreground video material and the background video material.

[0011] Furthermore, the frame attribute table sequence includes the numbers of the video frames of the foreground video material and the background video material, whether they are blank frames, timestamps, and key frame identifiers.

[0012] Furthermore, before creating a frame attribute table sequence for the foreground video material and the background video material, the method further includes:

[0013] Mark the blank frames of the foreground video material and the background video material.

[0014] Further, marking the blank frames of the foreground video material and the background video material includes:

[0015] When the transparency values ​​of the video frames of the foreground video material and the background video material are all 0, they are marked as blank frames.

[0016] Further, marking the blank frames of the foreground video material and the background video material includes:

[0017] When the RGB values ​​of the video frames of the foreground video material and the background video material are all 0, they are marked as blank frames.

[0018] Further, marking the blank frames of the foreground video material and the background video material includes:

[0019] If the slices and / or macroblocks of the encoded foreground video material and the background video material are blank, they are marked as blank frames.

[0020] In a second aspect, an embodiment of the present application further provides a video processing device, including:

[0021] A blank frame determination module is used to determine whether there are blank frames in the video frames of the foreground video material and the background video material;

[0022] A video aliasing module, configured to perform aliasing processing on video frames of the foreground video material and the background video material; and

[0023] The blank frame processing module is used to skip the blank frames and perform aliasing processing on the video frames of the foreground video material and the background video material except the blank frames.

[0024] In a third aspect, an embodiment of the present application further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is configured to implement the video processing method according to the first aspect when executing the program.

[0025] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored, wherein the computer program is used to implement the video processing method according to the first aspect above.

[0026] The embodiments of the present application bring the following beneficial effects:

[0027] In the video processing method provided in the embodiment of the present application, before performing aliasing processing, it is first determined whether there are blank frames in the video frames of the foreground video material and the background video material; if neither of them exists, aliasing processing is performed on the video frames of the foreground video material and the background video material; and if they exist, the blank frames are skipped, and aliasing processing is performed on the video frames of the foreground video material and the background video material other than the blank frames. Therefore, the video processing method provided in the embodiment of the present application speeds up the calculation speed of aliasing processing or color mixing processing by identifying blank frames in the video frames of the foreground video material and the background video material, and skipping the blank frames for aliasing processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0029] Figure 1 A schematic diagram of a video processing method according to an embodiment of the present invention;

[0030] Figure 2 A structural block diagram of a video processing device provided in an embodiment of the present application;

[0031] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0032] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments described in the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0034] In the specification and claims of this application and the above-mentioned drawings, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, unless otherwise specified, "multiple" means two or more. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to the specific circumstances.

[0035] Figure 1 FIG. 1 is a flow chart of a video processing method according to an embodiment of the present application. Figure 1 As shown, the video processing method of the embodiment of the present application includes the following steps:

[0036] S101: Determine whether there are blank frames in the video frames of the foreground video material and the background video material;

[0037] Common video aliasing or color mixing is the image mixing and superposition of video materials on multiple tracks. In video editing, materials are arranged in tracks. The stacking of tracks allows users to combine multiple video or image materials in the same video project to create complex visual effects and animations. For example, the upper track can contain a mask that defines the area where the content of the lower track is visible. In this way, a specific image or video clip can be accurately placed at a specific position in another video, or a dynamic mapping effect can be achieved. The core principle of the upper and lower track image aliasing method is also color mixing, that is, the content of the upper track is combined with the content of the lower track according to specific mixing rules. Existing image aliasing technology calculates all pixels of all frames in the video, and uses different color mixing modes according to needs when aliasing each frame. The color mixing modes used can be simply divided into linear mixing and nonlinear mixing.

[0038] In the video processing method provided in the embodiment of the present application, when performing image aliasing processing, the content of the upper track is defined as the foreground video material, and the content of the lower track is defined as the background video material. Before performing aliasing processing or color mixing processing, it is first necessary to determine whether there are blank frames in the video frames of the foreground video material and the background video material to avoid aliasing processing on the blank frames.

[0039] S102: If neither of them exists, performing aliasing processing on the video frames of the foreground video material and the background video material; and

[0040] Before performing aliasing or color mixing processing, it is first necessary to determine whether there are blank frames in the video frames of the foreground video material and the background video material to avoid aliasing processing on the blank frames. When there are no blank frames in the video frames of the foreground video material and the background video material, aliasing processing is performed on the video frames of the foreground video material and the background video material.

[0041] S103: If it exists, skip the blank frame, and perform aliasing processing on the video frames of the foreground video material and the background video material except the blank frame.

[0042] That is, when there are blank frames in the video frames of the foreground video material and / or the background video material, aliasing processing is performed on the video frames of the foreground video material and the background video material except the blank frames.

[0043] Therefore, in the video processing method provided in the embodiment of the present application, before performing aliasing processing, it is first determined whether there are blank frames in the video frames of the foreground video material and the background video material; if neither of them exists, aliasing processing is performed on the video frames of the foreground video material and the background video material; and if they exist, the blank frames are skipped, and aliasing processing is performed on the video frames of the foreground video material and the background video material other than the blank frames. Therefore, the video processing method provided in the embodiment of the present application speeds up the calculation speed of aliasing processing or color mixing processing by identifying blank frames in the video frames of the foreground video material and the background video material, and skipping the blank frames for aliasing processing.

[0044] Furthermore, in some embodiments of the present application, before determining whether there are blank frames in the video frames of the foreground video material and the background video material, the process further includes:

[0045] For the foreground video material and the background video material, a frame attribute table sequence is created to determine whether there are blank frames in the video frames of the foreground video material and the background video material.

[0046] In the video processing method provided in the embodiment of the present application, by establishing a sequence of frame attribute tables, it is used to efficiently store and retrieve relevant information of video frames. Specifically, the information of each frame is put into a structure, and the structure follows the memory alignment rules to reduce the additional operations during addressing. The structures of all frames are stored in order to form a continuous memory block, so that they can be accessed by the CPU at high speed. Because the structures follow the memory alignment rules, their layout in the memory is optimized, which helps to improve the hit rate of the CPU cache. The sequential storage design of the frame attribute table allows the use of a simple address offset to achieve fast data retrieval, quickly locate the attributes of any frame, without traversing the entire video data stream, and such a fast access mechanism greatly reduces the processing time and avoids invalid search and reading of large amounts of data.

[0047] The above data management method can not only simplify the memory addressing process, reduce the generation of memory fragmentation, and thus improve the addressing efficiency, but also enable video processing to better utilize the principle of locality, thereby improving the cache hit rate. Since the data access pattern in the frame attribute table is highly local, the CPU can predict and preload the data to be used, so that more data access can be completed directly from the cache instead of reading from the slower main memory, which can significantly reduce the number of times the CPU accesses the main memory or swaps in data from the virtual memory, which can reduce latency and increase processing speed.

[0048] Furthermore, in some embodiments of the present application, the frame attribute table sequence includes the video frame number of the foreground video material and the background video material, whether it is a blank frame, a timestamp, and a key frame identifier.

[0049] Specifically, in the video processing method provided in the embodiment of the present application, a frame attribute table sequence is established to efficiently store and retrieve relevant information of the video frames, including the video frame numbers of the foreground video material and the background video material, whether they are blank frames, timestamps and key frame identifiers, thereby simplifying the memory addressing process, reducing the generation of memory fragmentation, and thereby improving the addressing efficiency.

[0050] Furthermore, in some embodiments of the present application, before creating a frame attribute table sequence for the foreground video material and the background video material, the process further includes:

[0051] Mark the blank frames of the foreground video material and the background video material.

[0052] Specifically, in the video processing method provided in the embodiment of the present application, it is first necessary to mark the blank frames of the foreground video material and the background video material, and then create a frame attribute table sequence for the foreground video material and the background video material, so as to simplify the memory addressing process, reduce the generation of memory fragmentation, and thus improve the addressing efficiency.

[0053] Further, in some embodiments of the present application, marking the blank frames of the foreground video material and the background video material includes:

[0054] When the transparency values ​​of the video frames of the foreground video material and the background video material are all 0, they are marked as blank frames.

[0055] According to different scenes and rendering modes, the definition of blank frames can be extended and classified using different tags. From the scene point of view, the foreground and background of the video frame are distinguished. If normal blending only considers the Alpha value (transparency value), the frame with a transparency value of all 0 can be marked as a transparent blank frame, which can be used as a standard for determining blank frames.

[0056] Further, in some embodiments of the present application, marking the blank frames of the foreground video material and the background video material includes:

[0057] When the RGB values ​​of the video frames of the foreground video material and the background video material are all 0, they are marked as blank frames.

[0058] According to different scenes and rendering modes, the definition of blank frames can be expanded and classified using different tags. RGB values ​​need to be considered during additive mixing. If the RGB values ​​of a frame are all 0, it can be considered as an RGB blank frame. If the RGB values ​​of a frame are all 1 (assuming the color value range is 0 to 1), no matter what color the other frame is, the mixing result is a completely white picture, which can also be considered as a blank frame.

[0059] Further, in some embodiments of the present application, marking the blank frames of the foreground video material and the background video material includes:

[0060] If the slices and / or macroblocks of the encoded foreground video material and the background video material are blank, they are marked as blank frames.

[0061] Specifically, for the video material added by the user, it is inefficient to directly determine whether the RGB values ​​of each frame after decoding are all 0, because it needs to check each pixel of each frame one by one, which is time-consuming for video materials with huge data volumes. Therefore, an intelligent analysis method based on video codec can be used to identify blank frames. During the encoding process, the video codec will compress the spatial redundancy within the frame and the temporal redundancy between frames. Using this feature, blank frames can be identified by analyzing the differences between the previous and next frames. Specifically, the data at the slice and macroblock levels after encoding can be directly compared. If it is found that all slices or macroblocks of the frame are blank, the frame can be considered to be a blank frame.

[0062] The above method utilizes the inherent mechanism of the video codec, and does not need to decode and compare every pixel of each frame, thereby greatly improving the processing speed. In addition, since the codec has performed inter-frame prediction when processing the video, it can accurately identify the blank frame.

[0063] Figure 2 2 is a structural block diagram of a video processing device 200 provided in an embodiment of the present application. Figure 2 As shown, the video processing device 200 of the embodiment of the present application includes: a blank frame determination module 210, a video aliasing module 220 and a blank frame processing module 230, wherein:

[0064] A blank frame determination module 210 is used to determine whether there are blank frames in the video frames of the foreground video material and the background video material;

[0065] A video aliasing module 220, configured to perform aliasing processing on the video frames of the foreground video material and the background video material; and

[0066] The blank frame processing module 230 is used to skip the blank frames and perform aliasing processing on the video frames of the foreground video material and the background video material except the blank frames.

[0067] In the video processing device provided in the embodiment of the present application, before performing aliasing processing, it is first determined whether there are blank frames in the video frames of the foreground video material and the background video material; if none of them exist, aliasing processing is performed on the video frames of the foreground video material and the background video material; and if they exist, the blank frames are skipped, and aliasing processing is performed on the video frames of the foreground video material and the background video material other than the blank frames. Therefore, the video processing method provided in the embodiment of the present application speeds up the calculation speed of aliasing processing or color mixing processing by identifying blank frames in the video frames of the foreground video material and the background video material, and skipping the blank frames for aliasing processing.

[0068] It should be noted that the specific implementation of the video processing device of the embodiment of the present application is similar to the specific implementation of the video processing method of the embodiment of the present application. Please refer to the description of the method part for details, and no further details will be given here.

[0069] Figure 3 Schematic diagram of the structure of an electronic device 300 according to an embodiment of the present application.

[0070] like Figure 3 As shown, the electronic device 300 includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from the storage part 302 to a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The CPU 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0071] The following components are connected to the I / O interface 305: an input section 306 including a keyboard, a mouse, etc.; an output section 307 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN card, a modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the I / O interface 305 as needed. A removable medium 311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 310 as needed, so that a computer program read therefrom is installed into the storage section 308 as needed.

[0072] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a machine-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 309, and / or installed from a removable medium 311. When the computer program is executed by a central processing unit (CPU) 301, the above-mentioned functions defined in the electronic device of the present application are executed.

[0073] It should be noted that the computer-readable medium shown in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electronic device, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0074] In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction-executing electronic device, apparatus, or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction-executing electronic device, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0075] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the processing receiving device, method and computer program product according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the aforementioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based electronic device that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0076] The units or modules involved in the embodiments of the present application may be implemented by software or hardware. The units or modules described may also be arranged in a processor, and the processor is used to implement the video processing method when executing the program:

[0077] Determine whether there are blank frames in the video frames of the foreground video material and the background video material;

[0078] If neither exists, performing aliasing processing on the video frames of the foreground video material and the background video material; and

[0079] If it exists, the blank frame is skipped, and aliasing processing is performed on the video frames of the foreground video material and the background video material except the blank frame.

[0080] As another aspect, the present application further provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiment; or may exist independently and not be assembled into the electronic device. The above computer-readable storage medium stores one or more programs, and when the above programs are used by one or more processors to execute the video processing method described in the present application:

[0081] Determine whether there are blank frames in the video frames of the foreground video material and the background video material;

[0082] If neither exists, performing aliasing processing on the video frames of the foreground video material and the background video material; and

[0083] If it exists, the blank frame is skipped, and aliasing processing is performed on the video frames of the foreground video material and the background video material except the blank frame.

[0084] As another aspect, the present application further provides a computer program product, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer program product stores one or more programs, and when the above programs are used by one or more processors to execute the video processing method described in the present application:

[0085] Determine whether there are blank frames in the video frames of the foreground video material and the background video material;

[0086] If neither exists, performing aliasing processing on the video frames of the foreground video material and the background video material; and

[0087] If it exists, the blank frame is skipped, and aliasing processing is performed on the video frames of the foreground video material and the background video material except the blank frame.

[0088] The above description is only a preferred embodiment of the present application, and does not limit the patent scope of the present application. All equivalent structural changes made by using the contents of the present application specification and drawings under the application concept of the present application, or directly / indirectly used in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A video processing method, characterized in that: The following steps are involved: Determine whether there are blank frames in the video frames of the foreground video material and the background video material; If neither exists, performing aliasing processing on the video frames of the foreground video material and the background video material; and If it exists, the blank frame is skipped, and aliasing processing is performed on the video frames of the foreground video material and the background video material except the blank frame.

2. The video processing method according to claim 1, characterized in that: Before determining whether there are blank frames in the video frames of the foreground video material and the background video material, the method further includes: A frame attribute table sequence is created for the foreground video material and the background video material to determine whether there are blank frames in the video frames of the foreground video material and the background video material.

3. The video processing method according to claim 2, characterized in that: The frame attribute table sequence includes the video frame numbers of the foreground video material and the background video material, whether they are blank frames, timestamps, and key frame identifiers.

4. The video processing method according to claim 3, characterized in that: Before creating a frame attribute table sequence for the foreground video material and the background video material, the method further includes: Mark the blank frames of the foreground video material and the background video material.

5. The video processing method according to claim 4, characterized in that: The marking of the blank frames of the foreground video material and the background video material comprises: When the transparency values ​​of the video frames of the foreground video material and the background video material are all 0, they are marked as blank frames.

6. The video processing method according to claim 4, characterized in that: The marking of the blank frames of the foreground video material and the background video material comprises: When the RGB values ​​of the video frames of the foreground video material and the background video material are all 0, they are marked as blank frames.

7. The video processing method according to claim 4, characterized in that: The marking of the blank frames of the foreground video material and the background video material comprises: If the slices and / or macroblocks of the encoded foreground video material and the background video material are blank, they are marked as blank frames.

8. A video processing device, characterized in that: include: A blank frame determination module is used to determine whether there are blank frames in the video frames of the foreground video material and the background video material; A video aliasing module, used for performing aliasing processing on the video frames of the foreground video material and the background video material; and The blank frame processing module is used to skip the blank frames and perform aliasing processing on the video frames of the foreground video material and the background video material except the blank frames.

9. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor is configured to implement the video processing method according to any one of claims 1 to 7 when executing the program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is used to implement the video processing method according to any one of claims 1-7.

Images