A video processing method, apparatus, device and medium

Abstract

Embodiments of the present specification disclose a video processing method, device, equipment and medium, the video processing method comprises: acquiring video frames obtained by decomposing an original video; for any video frame, determining the transparency of the video frame, performing a matting operation on the video frame according to the transparency of the video frame to generate a post-matting image corresponding to the video frame; generating a post-matting video according to the post-matting images corresponding to the video frames obtained by decomposing the original video; wherein for any video frame, determining the transparency of the video frame comprises: for the first video frame, determining the transparency of the first video frame by using a deep learning model; for any non-first video frame, performing a feature fusion operation on the video frame by using the transparency of the previous video frame of the video frame, and determining the transparency of the video frame after the feature fusion operation by using a deep learning model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a video processing method and device, equipment and medium. BACKGROUND

[0002] With the development of technology, video matting has become an important technology in the field of video processing, and has been widely used in the fields of video editing, film production, short video production, live broadcast, etc. Correspondingly, how to improve the matting effect has become an important issue.

[0003] Therefore, a more effective video matting scheme is needed. SUMMARY

[0004] The embodiments of the present specification provide a video processing method, device, equipment and medium to solve the technical problem of how to more effectively perform video matting.

[0005] To solve the above technical problems, the embodiments of the present specification provide the following technical solutions:

[0006] The embodiments of the present specification provide a video processing method, comprising:

[0007] obtaining video frames decomposed from an original video;

[0008] For any video frame, determining the transparency of the video frame, performing a matting operation on the video frame according to the transparency of the video frame, and generating a post-matting image corresponding to the video frame;

[0009] generating a post-matting video according to the post-matting images corresponding to the video frames decomposed from the original video;

[0010] For any video frame, determining the transparency of the video frame comprises:

[0011] For the first video frame, using a deep learning model to determine the transparency of the first video frame;

[0012] For any non-first video frame, using the transparency of the previous video frame of the video frame to perform a feature fusion operation on the video frame, and using a deep learning model to determine the transparency of the video frame after the feature fusion operation.

[0013] The embodiments of the present specification provide a video processing device, comprising:

[0014] a decomposing module configured to obtain video frames decomposed from an original video;

[0015] a matting module configured to, for any video frame, determine the transparency of the video frame, perform a matting operation on the video frame according to the transparency of the video frame, and generate a post-matting image corresponding to the video frame.

[0016] a combination module configured to generate a matting video according to the matting images corresponding to the video frames decomposed from the original video;

[0017] For any video frame, the transparency of the video frame is determined, including:

[0018] For the first video frame, the transparency of the first video frame is determined by using a deep learning model;

[0019] For any non-first video frame, the transparency of the non-first video frame is determined by using a deep learning model after a feature fusion operation is performed on the non-first video frame using the transparency of a previous video frame of the non-first video frame.

[0020] An embodiment of the present specification provides a video processing device, including:

[0021] at least one processor;

[0022] and,

[0023] a memory in communication connection with the at least one processor;

[0024] wherein,

[0025] the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the video processing method described above.

[0026] An embodiment of the present specification provides a computer readable storage medium, which stores computer executable instructions, and the computer executable instructions are executed by a processor to implement the video processing method described above.

[0027] The above at least one technical solution adopted by the embodiment of the present specification can achieve the following beneficial effects:

[0028] By feature fusion between adjacent video frames, the transparency of the video frames is determined, so that the transparency of each non-first video frame is constrained by the transparency of the previous video frame when the transparency of the non-first video frame is calculated, the difference between the transparencies of the video frames is reduced, the difference between the matting images corresponding to the video frames is reduced, the flicker problem of the matting video is avoided, and the video matting effect is improved. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to make the technical solutions in the specification clearer, the following will briefly introduce the drawings needed to be used in the description of the embodiments or prior art in the specification. Obviously, the drawings introduced below are only some of the drawings that the embodiments described in the specification may involve, and other drawings can also be obtained by those of ordinary skill in the art without creative labor on the basis of these drawings.

[0030] Figure 1 is a schematic diagram of an execution subject of a video processing method in the first embodiment of the specification.

[0031] Figure 2 is a flowchart of a video processing method in the first embodiment of the specification.

[0032] Figure 3 is a schematic diagram of video frame feature fusion in the first embodiment of the specification.

[0033] Figure 4 is a structural schematic diagram of a video processing device in the second embodiment of the specification. DETAILED DESCRIPTION

[0034] In order to make those skilled in the art better understand the technical solutions in the specification, the technical solutions of the embodiments of the specification will be clearly and completely described below in combination with the drawings of the embodiments of the specification. Obviously, the embodiments described in the specification are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments of the specification, all other embodiments obtained by those of ordinary skill in the art without creative labor should be within the scope of protection of the present application.

[0035] With the development of technology, video matting has become an important technology in the field of video processing, and has been widely used in the fields of video editing, film production, short video production, live broadcast, etc. In fact, due to the reasons such as video exposure, too fast action of the person, etc., the video after matting will have the problem of flicker. In order to solve this problem, the prior art proposes methods such as video post-processing or adding video timing information.

[0036] Among them, the post-processing method is to further process the result obtained by the deep learning model, such as dilating erosion, edge sharpening, maximum connected domain operation, etc. to avoid the problem of video flicker. However, the method of video post-processing leads to a non-end-to-end overall solution (i.e. first through the deep learning model, then post-processing, divided into multiple stages of processing), the speed will be greatly reduced and cannot reach the real-time effect (i.e. there is a large delay between the original video and the video after matting).

[0037] While incorporating video temporal information into deep learning models can improve flickering, if the calculated transparency differences between adjacent video frames are too large, the temporal information becomes meaningless, and flickering will still occur.

[0038] The first embodiment of this specification (hereinafter referred to as "Embodiment 1") provides a video processing method. The executing entity of Embodiment 1 can be a terminal (including but not limited to mobile phones, computers, tablets, and televisions), a server, an operating system, an application, a video processing platform, or a video processing system, etc. That is, the executing entity can be diverse and can be set, used, or changed as needed. Alternatively, a third-party application can assist the executing entity in executing Embodiment 1. For example... Figure 1 As shown, the video processing method in Embodiment 1 can be executed by a server, and a corresponding application can be installed on the terminal (held by the user). Data can be transmitted between the terminal or the application and the server. Data can be collected, input, or output, or pages or information can be processed (to the user) through the terminal or the application, thereby assisting the server in executing the video processing method in Embodiment 1.

[0039] like Figure 2 As shown, the video processing method provided in Embodiment 1 includes:

[0040] S101: (Execution subject) Obtains video frames obtained from the decomposition of the original video;

[0041] In Example 1, the original video (referring to the video stream) can be decomposed into frames of images, each frame being a video frame. The execution entity in Example 1 can obtain the video frames obtained from the decomposition of the original video.

[0042] Obtaining video frames from the decomposition of the original video may include:

[0043] The original video may be pre-stored by the execution entity of Embodiment 1. The execution entity of Embodiment 1 decomposes the original video to obtain video frames.

[0044] Alternatively, the original video can be decomposed into video frames by other entities or devices. The execution entity in Embodiment 1 obtains the video frames decomposed from the original video from the other entities or devices.

[0045] Alternatively, the original video can be a continuously generated video stream, and the execution body of embodiment one obtains video frames decomposed from the continuously generated video stream; for example, the original video is a video stream continuously captured by the execution body of embodiment one, the execution body of embodiment one continuously captures the original video and decomposes the captured original video into video frames; alternatively, the original video is a video stream continuously captured by another subject or device, and the captured video is continuously transmitted to the execution body of embodiment one, and the execution body of embodiment one continuously decomposes the received original video into video frames; alternatively, the original video is continuously captured by another subject or device, and the another subject or device continuously decomposes the captured original video into video frames and continuously transmits the decomposed video frames to the execution body of embodiment one.

[0046] It should be noted that the video frames decomposed from the original video have time sequence information, and the execution body of embodiment one can determine the time sequence information of each video frame decomposed from the original video. The time sequence information indicates which two video frames are adjacent, which video frame is in front and which video frame is behind in the adjacent video frames.

[0047] S103: (execution body) determines the transparency of any video frame, performs a cutout operation on the video frame according to the transparency of the video frame, and generates a cutout image corresponding to the video frame; wherein, for any video frame, determining the transparency of the video frame comprises: for the first video frame, determining the transparency of the first video frame using a deep learning model; for any non-first video frame, performing a feature fusion operation on the video frame using the transparency of the previous video frame of the video frame, and determining the transparency of the video frame after the feature fusion operation using a deep learning model.

[0048] For any video frame, the execution body of embodiment one can determine the transparency of the video frame (the transparency is the alpha value or alpha). Wherein, for any video frame, determining the transparency of the video frame can include:

[0049] 1.1, for the first video frame (the first video frame is the video frame with the highest order), the execution body of embodiment one determines the transparency of the first video frame using a deep learning model. Specifically, the first video frame can be input into the deep learning model, and the deep learning model performs convolution, feature extraction, segmentation, regression and other calculations on the first video frame, so as to determine and output the transparency of the first video frame from the deep learning model.

[0050] 1.2 For any video frame, if it is not the first video frame, the execution entity in Embodiment 1 can use the transparency of the previous video frame to perform feature fusion on that video frame, and then use a deep learning model to determine the transparency of the video frame after feature fusion. That is, the execution entity in Embodiment 1 can use the transparency of the previous video frame to perform feature fusion on that video frame, and then input the video frame after feature fusion into a deep learning model, which determines and outputs the transparency of the video frame.

[0051] For example, for the next video frame adjacent to the first video frame (denoted as video frame a1), the transparency of the first video frame is used to perform feature fusion on video frame a1. The video frame a1 after feature fusion is then input into a deep learning model. The deep learning model performs convolution, feature extraction, segmentation, regression, and other calculations on video frame a1 to determine and output the transparency of "video frame a1 after feature fusion." Similarly, for the next video frame adjacent to video frame a1 (denoted as video frame a2), the transparency of video frame a1 is used to perform feature fusion on video frame a2. The video frame a2 after feature fusion is then input into the deep learning model. The algorithm performs convolution, feature extraction, segmentation, and regression calculations on video frame a2 to determine and output the transparency of "video frame a2 after feature fusion." Similarly, for the next video frame adjacent to video frame ai (i≥1) (denoted as video frame a(i+1)), the transparency of video frame ai is used to perform feature fusion on video frame a(i+1). The video frame a(i+1) after feature fusion is then input into a deep learning model. The deep learning model performs convolution, feature extraction, segmentation, and regression calculations on video frame a(i+1) to determine and output the transparency of "video frame a(i+1) after feature fusion," and so on. For example... Figure 3 As shown, assuming the first video frame is used as the current input video frame to the deep learning model (denoted as the "current frame"), the deep learning model outputs the transparency alpha of the current frame. The transparency of the current frame is then concatenated to the next frame, and a feature fusion operation is performed on the next frame using the transparency of the current frame.

[0052] In Embodiment 1, determining the transparency of any video frame can include: determining the transparency of the most recently acquired video frame after acquiring each video frame. That is, after acquiring the first video frame, the transparency of the first video frame is determined as described above; after acquiring video frame a1, the transparency of video frame a1 is determined as described above; after acquiring video frame a2, the transparency of video frame a2 is determined as described above; and so on, after acquiring video frame a(i+1), the transparency of video frame a(i+1) is determined as described above.

[0053] In the embodiment one, for any non-first video frame, the feature fusion operation is used to fuse the transparency feature of the previous video frame of the video frame to the video frame, so as to constrain the calculation of the transparency of the video frame. Wherein, for any non-first video frame, using the transparency of the previous video frame of the video frame, the feature fusion operation on the video frame can include: for any non-first video frame, using the transparency of the previous video frame of the video frame, the feature fusion operation on the video frame is performed by the concat method. Under the concat method, the transparency of the previous video frame of the video frame is merged into the video frame, so that the feature dimension of the video frame describing the image is increased.

[0054] In the above 1.2, for any video frame, if the video frame is not the first video frame, the execution subject of the embodiment one uses the transparency of the previous video frame of the video frame to perform the feature fusion operation on the video frame, which can be represented by the following formula (1):

[0055]

[0056] Wherein, t represents time or time; A(t-1) represents the transparency of a certain video frame determined by the deep learning model at t-1 time; I(t-1) represents the next video frame of the “certain video frame” at t-1 time, which has been obtained The original picture (so-called “original picture” refers to the video frame obtained by decomposing the original video before feature fusion operation) of the next video frame; I(t) represents the next video frame of the “certain video frame” after the feature fusion operation, which needs to be input into the deep learning model at t time, A(t-1) is connected with I(t-1), and I(t) is obtained by performing feature fusion operation on I(t-1) using A(t-1).

[0057] For any video frame, the transparency of the video frame embodies and contains the object of interest in the video frame, such as a person, an animal, an article. Figure 3 An example of transparency alpha is shown in Figure 3 In the embodiment one, the person image is taken as the object of interest.

[0058] The deep learning model can be constructed on the execution subject of the embodiment one, and the deep learning model is run by the execution subject of the embodiment one. Specifically, the deep learning model is constructed by a deep learning network, that is, the deep learning model is a model obtained by training the deep learning network. The deep learning model is composed of a series of deep learning weights. When data enters the deep learning model, it will be calculated and analyzed by the weights in the model, and finally the predicted result of the model is obtained. This process can be called inference or deduction or prediction, so using the deep learning model to determine the transparency of the video frame is equivalent to inferring or deducing or predicting the transparency of the video frame by the deep learning model.

[0059] In the embodiment one, for any video frame, after determining the transparency of the video frame, the execution subject of the embodiment one can perform a matting operation on the video frame according to the transparency of the video frame to generate a post-matting image corresponding to the video frame.

[0060] In the embodiment one, for any video frame, after determining the transparency of the video frame, the execution subject of the embodiment one can perform a matting operation on the video frame according to the transparency of the video frame to generate a post-matting image corresponding to the video frame.

[0061] I = aF + (1-a)B; (2)

[0062] In the embodiment one, for any video frame, after determining the transparency of the video frame, the execution subject of the embodiment one can perform a matting operation on the video frame according to the transparency of the video frame to generate a post-matting image corresponding to the video frame.

[0063] S105: (execution subject) generating a post-matting video according to the post-matting images corresponding to the video frames decomposed from the original video.

[0064] In the embodiment one, for any video frame, after determining the transparency of the video frame, the execution subject of the embodiment one can perform a matting operation on the video frame according to the transparency of the video frame to generate a post-matting image corresponding to the video frame.

[0065] In the embodiment one, for any video frame, after determining the transparency of the video frame, the execution subject of the embodiment one can perform a matting operation on the video frame according to the transparency of the video frame to generate a post-matting image corresponding to the video frame.

[0066] In the embodiment one, for any pair of adjacent video frames, the feature of the later video frame in the pair of adjacent video frames is fused through the transparency of the earlier video frame in the pair of adjacent video frames, so that when the deep learning model calculates the transparency of the later video frame after the feature fusion, the transparency of the earlier video frame is constrained, thereby reducing the difference between the transparency of the earlier video frame and the transparency of the later video frame, and reducing the difference between the post-matting image corresponding to the earlier video frame and the post-matting image corresponding to the later video frame, i.e. reducing the difference between the post-matting images corresponding to any pair of adjacent video frames. In this way, the matting video composed of the post-matting images corresponding to the video frames will not have the flicker problem, and the video matting effect is effectively improved.

[0067] In the first embodiment, the difference between the transparency of the front video frame and the transparency of the rear video frame is reduced, and the difference between the image after matting corresponding to the front video frame and the image after matting corresponding to the rear video frame is reduced, so that the stability of the deep learning model can be effectively improved.

[0068] The content in the first embodiment can be automatically executed by the execution subject in the first embodiment and the deep learning model thereon, realizing automatic execution from the original video to the output video after matting, without the need for user processing and intervention, which is an end-to-end processing process, can effectively reduce the delay from the original video to the video after matting, and realizes real-time processing of the original video (i.e., video stream).

[0069] As shown in Figure 4 The second embodiment of the present specification provides a video processing device corresponding to the first embodiment, comprising:

[0070] The decomposition module 202 is configured to obtain video frames decomposed from the original video;

[0071] The matting module 204 is configured to determine the transparency of any video frame, perform matting operation on the video frame according to the transparency of the video frame, and generate the image after matting corresponding to the video frame;

[0072] The combination module 206 is configured to generate the video after matting according to the images after matting corresponding to each video frame decomposed from the original video;

[0073] For any video frame, the transparency of the video frame is determined, comprising:

[0074] For the first video frame, the transparency of the first video frame is determined by using the deep learning model;

[0075] For any non-first video frame, the feature fusion operation is performed on the video frame using the transparency of the previous video frame of the video frame, and the transparency of the video frame after the feature fusion operation is determined by using the deep learning model.

[0076] Optionally, for any non-first video frame, the feature fusion operation performed on the video frame using the transparency of the previous video frame of the video frame comprises:

[0077] For any non-first video frame, the feature fusion operation is performed on the video frame using the transparency of the previous video frame of the video frame by the concat method.

[0078] Optionally, the feature fusion operation is configured to fuse the transparency of the previous video frame of the video frame into the video frame.

[0079] Optionally, the video after matting is generated according to the images after matting corresponding to the video frames decomposed from the original video, and the video after matting is generated by:

[0080] According to the time sequence information of the video frames decomposed from the original video, the images after matting corresponding to the video frames decomposed from the original video are combined to generate the video after matting.

[0081] Optionally, the decomposing module 202 is further configured to determine the time sequence information of the video frames decomposed from the original video.

[0082] Optionally, the video frames decomposed from the original video are obtained by:

[0083] The video frames decomposed from the continuously generated video stream are obtained.

[0084] Optionally, for any video frame, the transparency of the video frame is determined by:

[0085] The transparency of the newly obtained video frame is determined each time a video frame is obtained.

[0086] The third embodiment of the present specification provides a video processing device, comprising:

[0087] at least one processor;

[0088] and,

[0089] a memory in communication connection with the at least one processor;

[0090] wherein,

[0091] the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the video processing method of the first embodiment.

[0092] The fourth embodiment of the present specification provides a computer readable storage medium, and the computer readable storage medium stores computer executable instructions, and the computer executable instructions are executed by a processor to implement the video processing method of the first embodiment.

[0093] The above embodiments can be used in combination, and the same named modules in different embodiments or the same embodiment can be the same or different modules.

[0094] The above-described embodiments of the application have been described in connection with what are presently considered to be the most practical and preferred embodiments, and the best modes contemplated by the inventors. As indicated, however, the application is directed to each individual feature, system, article, material, kit, and / or method described herein. Thus, any single supply chain optimization system feature, system, article, material, kit, and / or method described herein can be included in or per se form an independent application. In addition, combinations of features, systems, articles, materials, kits, and / or methods described herein can also form an independent application. Likewise, still other applications within the scope of the present application can be set forth in the following claims, and equivalents thereto. Further, the narratives herein are intended to provide a single description of the many features and aspects of what are described and claimed. Various changes can be made and equivalents employed as known in the art without departing from the scope of the application as described.

[0095] Each of the embodiments in the present specification is described in a progressive manner, and the same or similar parts between embodiments can be referred to each other. Each embodiment mainly explains the difference from other embodiments. Especially, the device, equipment, and nonvolatile computer readable storage medium embodiments are basically similar to the method embodiments, and thus the description is relatively simple, and the relevant part can be referred to the part of the method embodiment.

[0096] The device, equipment, nonvolatile computer readable storage medium and method provided by the embodiments of the present specification are corresponding, and thus the device, equipment, and nonvolatile computer readable storage medium also have similar beneficial technical effects as the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the corresponding device, equipment, and nonvolatile computer readable storage medium will not be described here.

[0097] In the 1990s, it was quite obvious to distinguish whether an improvement in a technology was in hardware (e.g., improvement in circuit structures of diodes, transistors, switches, etc.) or in software (improvement in method flow). However, as technology has evolved, many improvements in method flow today can be considered as direct improvements in hardware circuit structures. Designers almost always obtain the corresponding hardware circuit structures by programming the improved method flow into hardware circuits. Therefore, it cannot be said that an improvement in a method flow cannot be implemented by hardware entity modules. For example, a programmable logic device (PLD) (e.g., a field programmable gate array (FPGA)) is an integrated circuit whose logic function is determined by user programming of the device. A digital system is "integrated" on a PLD by the designer programming it, rather than by asking a chip manufacturer to design and fabricate a custom integrated circuit chip. Moreover, instead of manually fabricating integrated circuit chips, this programming is now mostly implemented by "logic compiler" software, which is similar to software compilers used in program development, and the original code to be compiled is written in a specific programming language, which is called a hardware description language (HDL), and there are many such languages, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., and the most commonly used are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should be aware that, as long as the method flow is logically programmed in the above-mentioned hardware description languages and programmed into an integrated circuit, a hardware circuit implementing the logical method flow can be easily obtained.

[0098] The controller can be implemented in any suitable way, for example the controller can take the form of a microprocessor or processor and a computer readable medium storing computer readable program code, such as software or firmware, executable by the (micro)processor, logic gates, switches, an application specific integrated circuit (ASIC), a programmable logic controller and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicon Labs C8051F320, the memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that, in addition to being implemented in pure computer readable program code, the controller can equally well be implemented to perform the same functions using logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded microcontrollers, by way of logical programming of the method steps. The controller can thus be considered a hardware component, and the means included therein for performing the various functions can be considered structures within the hardware component. Alternatively, or even additionally, the means for performing the various functions can be considered both software modules that implement the method and structures within the hardware component.

[0099] The systems, apparatuses, modules or units illustrated by the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0100] For the sake of description, the above apparatuses are described in functional division and are described respectively. Of course, the functions of the units can be implemented in the same or multiple software and / or hardware when implementing the present specification.

[0101] Those skilled in the art will understand that the embodiments of the present specification can be provided as a method, a system or a computer program product. Therefore, the embodiments of the present specification can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present specification can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0102] The specification is presented with reference to flow diagrams and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the specification. It will be understood that each block of the flow diagrams and / or block diagrams, and combinations of blocks in the flow diagrams and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 one or more flow or multiple flows and / or blocks Figure 1 one or more flow or multiple flows and / or blocks

[0103] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 one or more flow or multiple flows and / or blocks Figure 1 one or more flow or multiple flows and / or blocks

[0104] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 one or more flow or multiple flows and / or blocks Figure 1 one or more flow or multiple flows and / or blocks

[0105] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0106] The memory can include non-persistent memory and / or volatile memory, such as a random access memory (RAM) including a cache area for the temporary storage of data. The memory can also include non-volatile memory, such as a read only memory (ROM) device, a floppy disk, a hard disk, or a flash memory. The memory can store an operating system including procedures and data used to manage the computer's operation, as well as programs and data used by the operating system. The memory can also store programs and data used by the operating system and other programs, such as programs 1-4.

[0107] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device. According to the definition herein, computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0108] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusion, such that processes, methods, articles or devices that include a series of elements not only include those elements, but also include other elements not explicitly listed or inherent to such processes, methods, articles or devices. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0109] The specification can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The specification can also be practiced in a distributed computing environment, in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0110] Each embodiment in the specification is described in a progressive manner, and the same or similar parts between each embodiment can be referred to each other, and each embodiment focuses on the difference from other embodiments. In particular, for system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0111] The above merely provides a description of the embodiments of the present application, but should not be taken as a limitation. Various changes and modifications can be made to the present application by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall fall within the scope of claims of the present application.

Claims

1. A method for video processing, comprising: obtaining video frames decomposed from an original video; determining, for any of the video frames, a transparency of the video frame, performing a matting operation on the video frame according to the transparency of the video frame, and generating a post-matted image corresponding to the video frame; generating a post-matted video according to the post-matted images corresponding to the video frames decomposed from the original video; wherein the determining, for any of the video frames, the transparency of the video frame comprises: determining, for a first video frame, the transparency of the first video frame using a deep learning model; determining, for any non-first video frame, the transparency of the non-first video frame using a feature fusion operation on the non-first video frame with the transparency of a previous video frame of the non-first video frame, and determining the transparency of the non-first video frame after the feature fusion operation using the deep learning model; and wherein the performing, for any non-first video frame, the feature fusion operation on the non-first video frame using the transparency of the previous video frame of the non-first video frame comprises: performing, for any non-first video frame, the feature fusion operation on the non-first video frame using the transparency of the previous video frame of the non-first video frame by a concat manner. 2.The method of claim 1, wherein the generating the post-matted video according to the post-matted images corresponding to the video frames decomposed from the original video comprises: combining the post-matted images corresponding to the video frames decomposed from the original video according to time sequence information of the video frames decomposed from the original video, and generating the post-matted video. 3.The method of claim 1, further comprising: determining the time sequence information of the video frames decomposed from the original video. 4.The method of claim 1, wherein the obtaining the video frames decomposed from the original video comprises: obtaining the video frames decomposed from a continuously generated video stream. 5.The method of claim 1, wherein the determining, for any of the video frames, the transparency of the video frame comprises: determining the transparency of a newly obtained video frame each time a video frame is obtained. 6.An apparatus for video processing, comprising: a decomposing module configured to obtain video frames decomposed from an original video; a matting module configured to determine, for any of the video frames, a transparency of the video frame, perform a matting operation on the video frame according to the transparency of the video frame, and generate a post-matted image corresponding to the video frame; and a combining module configured to generate a post-matted video according to the post-matted images corresponding to the video frames decomposed from the original video; wherein the determining, for any of the video frames, the transparency of the video frame comprises: determining, for a first video frame, the transparency of the first video frame using a deep learning model; determining, for any non-first video frame, the transparency of the non-first video frame using a feature fusion operation on the non-first video frame with the transparency of a previous video frame of the non-first video frame, and determining the transparency of the non-first video frame after the feature fusion operation using the deep learning model; and wherein the performing, for any non-first video frame, the feature fusion operation on the non-first video frame using the transparency of the previous video frame of the non-first video frame comprises: performing, for any non-first video frame, the feature fusion operation on the non-first video frame using the transparency of the previous video frame of the non-first video frame by a concat manner. ​ ​ ​ ​ The video frame is subjected to a feature fusion operation using the transparency of the previous video frame of the video frame, by the formula ; wherein t represents time or time point; A(t-1) represents the transparency of a video frame determined by the deep learning model at time t-1; I(t-1) represents the original image of the next video frame of the video frame obtained at time t-1; and I(t) represents the next video frame of the video frame which needs to be input into the deep learning model at time t after the feature fusion operation. ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ For any non-first video frame, performing a feature fusion operation on the video frame using the transparency of a previous video frame of the video frame by a concat manner; The feature fusion operation is configured to fuse the transparency feature of the previous video frame of the video frame into the video frame; The video frame is subjected to a feature fusion operation using the transparency of the previous video frame of the video frame, by the formula ; wherein t represents time or time point; A(t-1) represents the transparency of a video frame determined by the deep learning model at time t-1; I(t-1) represents the original image of the next video frame of the video frame obtained at time t-1; and I(t) represents the next video frame of the video frame which needs to be input into the deep learning model at time t after the feature fusion operation.

7. A video processing device, comprising: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the video processing method in any one of claims 1 to 5.

8. A computer readable storage medium storing computer executable instructions, and the computer executable instructions, when executed by a processor, implement the video processing method in any one of claims 1 to 5.

Images