Method, device, equipment and medium for adding animation to video

Abstract

The embodiments of the present disclosure relate to a method, device, equipment and medium for adding animation to a video, and relate to the technical field of video processing. The method includes the steps of: detecting whether an object is held by a user's hand in a current video frame currently being shot; determining a target animation corresponding to the object according to whether the object is being held; determining an animation adding area in the current video frame corresponding to the target animation, and adding the target animation to the animation adding area. This adds an animation special effect corresponding to the object held by the user's hand in the video, intuitively reflects the interaction between the user and the hand-held object, improves the interest of video shooting, and provides technical support for improving the user's adhesion to video products.

Description

[Technical field]

[0001] [CROSS REFERENCE TO RELATED APPLICATIONS] This disclosure claims priority to a Chinese patent application bearing application number 202110450098.0 and entitled “Method, Apparatus, Device and Medium for Adding Animation to Video,” filed with the China Patent Office on April 25, 2021, the entire contents of which are incorporated herein by reference.

[0002] [Technical field] The present disclosure relates to the technical field of video processing, and in particular to a method, apparatus, device and medium for adding animation to video. [Background technology]

[0003] With the development of video products (such as short video applications), users' requirements for video products are becoming more and more diverse, for example, the interest of video products is becoming more common.

[0004] In the related art, various video editing props are provided to meet the interest requirements of users, but the use of various video editing props requires high editing skills, and the editing props are not differentiated for different videos, and there is no sense of interaction with the video, so there are still relatively high limitations in meeting the interest requirements of users for video products. Summary of the Invention [Means for solving the problem]

[0005] To solve or at least partially solve the above problem, the present disclosure provides a method for adding animation to a video, the method including the steps of: detecting whether an object is being held by a user's hand in a current video frame currently being captured; determining a target animation corresponding to the object in response to the object being held; determining an animation addition area in the current video frame corresponding to the target animation; and adding the target animation to the animation addition area.

[0006] An embodiment of the present disclosure further provides an animation adding device for a video, the device including: a first detection module for detecting whether an object is held by a user's hand in a current video frame currently being captured; a first determination module for determining a target animation corresponding to the object according to the object being held; and an animation adding module for determining an animation adding area in the current video frame corresponding to the target animation, and adding the target animation to the animation adding area;

[0007] An embodiment of the present disclosure further provides an electronic device, the electronic device including a processor and a memory for storing instructions executable by the processor, the processor reading the executable instructions from the memory and executing the instructions to implement the method for adding animation to a video according to an embodiment of the present disclosure.

[0008] An embodiment of the present disclosure further provides a computer-readable storage medium, the storage medium storing a computer program, the computer program being used to execute the method for adding animation to a video according to an embodiment of the present disclosure.

[0009] Compared with the prior art, the technical solutions according to the embodiments of the present disclosure have the following advantages: Detect whether an object is held by the user's hand in the current video frame currently being shot, and if an object is held, determine a target animation corresponding to the object, and finally determine an animation adding area in the current video frame corresponding to the target animation, and add the target animation to the animation adding area, thereby adding an animation special effect corresponding to the object held by the user's hand in the video, intuitively reflecting the interaction between the user and the hand-held object, improving the interest of video shooting, and providing technical support for improving user adhesion to video products. [Brief description of the drawings]

[0010] The above and other features, advantages and aspects of each embodiment of the present disclosure will become more apparent with reference to the following specific embodiments in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals refer to the same or similar elements. It should be understood that the drawings are schematic and that the components and elements are not necessarily drawn to scale.

[0011] [Figure 1] 1 is a flowchart of a method for adding animation to a video according to an embodiment of the present disclosure. [Diagram 2] 4 is a flowchart of a method for adding animation to a video according to another embodiment of the present disclosure. [Diagram 3] 1 is a schematic diagram of an animated additional scene of a video according to an embodiment of the present disclosure. [Figure 4] FIG. 13 is a schematic diagram of an animated additional scene of a video according to another embodiment of the present disclosure. [Diagram 5] FIG. 13 is a schematic diagram of an animated additional scene of a video according to another embodiment of the present disclosure. [Figure 6] FIG. 13 is a schematic diagram of an animated additional scene of a video according to another embodiment of the present disclosure. [Figure 7] FIG. 13 is a schematic diagram of an animated additional scene of a video according to another embodiment of the present disclosure. [Figure 8] 4 is a flowchart of a method for adding animation to a video according to another embodiment of the present disclosure. [Figure 9] FIG. 13 is a schematic diagram of an animated additional scene of a video according to another embodiment of the present disclosure. [Figure 10] 4 is a flowchart of a method for adding animation to a video according to another embodiment of the present disclosure. [Figure 11] FIG. 13 is a schematic diagram of an animated additional scene of a video according to another embodiment of the present disclosure. [Figure 12] FIG. 2 is a schematic structural diagram of a video animation adding device according to an embodiment of the present disclosure; [Figure 13] FIG. 11 is a schematic structural diagram of a video animation adding device according to another embodiment of the present disclosure; [Figure 14] FIG. 11 is a schematic structural diagram of a video animation adding device according to another embodiment of the present disclosure; [Figure 15] 1 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, the embodiments of the present disclosure will be described in more detail with reference to the drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be realized in various forms and should not be construed as being limited to the embodiments described herein, but rather that these embodiments are provided for a deeper and more complete understanding of the present disclosure. It should also be understood that the drawings and embodiments of the present disclosure are only given for illustrative purposes and are not intended to limit the scope of protection of the present disclosure.

[0013] It should be understood that the steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. Also, method embodiments may include additional steps and / or omit the performance of illustrated steps. The scope of the present disclosure is not limited in this respect.

[0014] As used herein, the term "comprising" and variations thereof are open ended, i.e., meaning "including but not limited to." The term "based on" means "based at least in part on." The term "in one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one other embodiment," and the term "some embodiments" means "at least some embodiments." Additionally, definitions relating to other terms are provided below.

[0015] It should be noted that concepts such as "first" and "second" referred to in this disclosure are used only to distinguish different devices, modules, or units, and are not intended to limit the order or interdependence of functions performed by these devices, modules, or units.

[0016] It should be understood by those of skill in the art that modifications such as "a" and "a plurality" referred to in the present disclosure are exemplary rather than limiting and should be interpreted as "one or more" unless the context clearly dictates otherwise.

[0017] The names of messages or information exchanged among devices in the embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0018] In order to solve the problem that it is difficult to meet the user's interest in video editing in the above-mentioned later video editing, the embodiment of the present disclosure provides an animation adding method, in which animation effects can be flexibly added to video frames, the interaction between the user and the handheld object is intuitively reflected, and various animation adding effects can be realized from multiple dimensions such as added animation and added area of ​​animation, so as to meet the user's interest requirements in video editing.

[0019] The method is described below with reference to a specific embodiment. The method for adding animation to a video in this embodiment is applicable to any video product, such as short video application.

[0020] FIG. 1 is a flow chart of a method for adding animation to a video according to an embodiment of the present invention. The method can be performed by a device for adding animation to a video. The device can be realized by software and / or hardware, and can be generally integrated into electronic equipment. As shown in FIG. 1, the method includes the following steps:

[0021] Step 101: Detect whether an object is being held by the user's hand in the currently captured video frame.

[0022] In this embodiment, the processing target is the current video frame currently being shot, so by adding animation to each video frame currently being shot, real-time addition of animation effects is realized, thereby satisfying the need for editing video in real time.

[0023] Of course, in some possible implementations, animation may be added later to video frames in the captured video, and the method of adding animation is similar to that of the current video frames captured in real time, so it will not be repeated here.

[0024] In this embodiment, after detecting that the current video frame contains the user's hand, it is detected whether the user's hand in the currently captured video frame holds an object. That is, the subsequent process is targeted at the animation addition process when there is hand-held interaction between the user and the object, and the object may be any object held by the user's hand, or may be a specific object such as an object of a specific shape or color. If the object is any object, a corresponding animation is added simply by the object being held by the user's hand, and if the object is a specific object, a corresponding animation is added simply by the specific object being held by the user's hand, which further enhances the interest.

[0025] It should be noted that, in different application scenarios, the methods for detecting whether an object is held by the user's hand in the currently captured video frame are different.

[0026] Example 1: In this example, the hand contour of the user's hand is detected, the object contour of the object is detected, and it is determined whether the hand contour and the object contour overlap, and if there is an overlapping area, it is considered that the object is being held in the user's hand.

[0027] Example 2: In this example, a deep learning model is pre-trained according to deep learning techniques, and the deep learning model learns to obtain image features of an image in which an object is held in a user's hand, and then a current video frame is input into the deep learning model, and whether an object is held in the user's hand is determined according to the output of the deep learning model.

[0028] Step 102: If the object is being held, determine a target animation corresponding to the object.

[0029] In this embodiment, when it is detected that an object is being held in the user's hand, the animation addition condition is met and a target animation corresponding to the object is determined.

[0030] Step 103: Determine an animation adding area in the current video frame that corresponds to the target animation, and add the target animation to the animation adding area.

[0031] In this embodiment, in order to further enhance the interest of adding animation, an animation adding area corresponding to the target animation in the current video frame is determined, which includes, but is not limited to, an area where an object is located, all areas of the current video frame, areas where other objects are located, etc.

[0032] Furthermore, by adding a target animation to the animation addition area, when the user holds an object in his or her hand, such interaction can be reflected through the animation, which is very interesting.

[0033] In order to allow those skilled in the art to more clearly understand the animation adding method according to the embodiment of the present disclosure, the following takes as an example a prop adding scene in which the animation adding method is applied to a short video application.

[0034] In this embodiment, when a user is shooting a video, triggering the hand-held object animation add-on prop in the toolbar will detect whether an object is held by the user's hand in the current video frame being shot. If no object is detected, a prompt message "try holding an object" will be displayed, prompting the user to hold an object.

[0035] When it is detected that an object is being held in the user's hand, a target animation corresponding to the object is determined, an animation addition area corresponding to the target animation in the current video frame is determined, and the target animation is added to the animation addition area.

[0036] According to the above, the video animation adding method of the embodiment of the present disclosure detects whether an object is held by the user's hand in the current video frame currently being shot, and determines a corresponding target animation if an object is held, and finally determines an animation adding area in the current video frame corresponding to the target animation, and adds the target animation to the animation adding area, thereby adding an animation special effect corresponding to the object held by the user's hand in the video, intuitively reflecting the interaction between the user and the handheld object, improving the interest of video shooting, and providing technical support for improving the user's adhesion to video products.

[0037] When adding an animation to a scene where a user holds an object in his / her hand, if the added target animation is different, the displayed editing effect will be different, so the method of determining the target animation corresponding to the object will vary depending on the application scene. For example, as follows.

[0038] Example 1: In this example, as shown in FIG. 2, determining a target animation corresponding to an object includes the following steps.

[0039] Step 201: Extract preset image features corresponding to an object.

[0040] In this embodiment, image processing techniques can be used to extract preset image features corresponding to the object. The preset image features may be any features that determine the target animation. In some possible embodiments, the preset image features include at least one of the following: a shape feature of the object (e.g., a square feature, a circular feature, etc.), a color feature of the object (e.g., a red feature, a green feature, etc.), a size feature of the object (e.g., much larger or much smaller than the user's hand, etc.), an object type of the object (e.g., lucky money, a gift, a diamond ring, etc.), a position feature of the object (e.g., located at the top or bottom of the hand, located at the top or bottom of the image, etc.), and a gesture feature of the user's hand holding the object (e.g., a grasping gesture feature, a fingertip grabbing gesture feature, etc.).

[0041] Step 202: Determine a target animation corresponding to a preset image feature.

[0042] It should be understood that determining the target animation corresponding to the preset image feature may be realized by consulting a preset database, or may be automatically generated by an animation generating device according to the preset image feature. Since the preset database includes a correspondence between the preset image feature and the corresponding target animation, the target animation corresponding to the preset image feature can be determined by consulting the preset database.

[0043] As an example, if the preset image feature is a circle feature, the corresponding target animation is a flame, and if the preset image feature is green, the corresponding target animation is a flower.

[0044] For example, if the preset image feature is a user's hand gesture feature, e.g., if the user's hand gesture is clicking an object, the target animation can be determined to be a texture change animation such as a freezing animation or a gold animation, and combined with the target animation addition area being the area where the object is located, animation effects such as "turning a stone into gold" or "touching it with your hand" can be realized.

[0045] Example 2: In this example, multiple material sets corresponding to a target animation are predetermined, and after it is determined that the current video frame includes an object being held by a user's hand, one target material is randomly determined from each of the multiple material sets, and all the target materials are combined to generate a target animation.

[0046] For example, as shown in FIG. 3, when multiple material sets corresponding to a target animation include an eye animation material set, a nose animation material set, a limb animation material set, etc., the eye animation, nose animation, limb animation corresponding to an object are randomly determined from the corresponding material sets, and then combined to obtain the final target animation.

[0047] In view of the above, the method for adding animation to a video of an embodiment of the present disclosure can flexibly determine a target animation corresponding to an object, thereby further improving the realism of the interaction between the user's hands and the object and increasing the interest of the video.

[0048] It is easy to understand that the display effect of the current video frame will be different depending on the animation adding area, and the following will be illustratively described by combining different animation adding areas.

[0049] Example 1: In this example, an image area in which an object is located is identified, and the image area in which the object is located is further determined as an animation addition area, thereby adding a target animation to the object to enhance the interactive realism of the user's hands and the object.

[0050] For example, referring to FIG. 4, when a user holds a “cup” in his / her hand and the corresponding target animation is a facial expression animation, adding a facial expression animation to the “cup” can reflect the interactive feeling that the “cup” is being held in the user’s hand.

[0051] In addition, by matching different target animations to different current video frames, different animation effects can be reflected by the animation changes of the object. For example, referring to Fig. 4, by adding different facial expression animations to the "cup" in different video frames, the anthropomorphic expression of "facial expression changes" in which the different "cups" are held by the user's hands can be realized.

[0052] Example 2: In this example, a related object of the object is determined. The related object may be an object that is pre-matched with the image features of the object. For example, if the object is circular, the related object is circular. The related object may be a pre-defined specific object, for example, a human face. The related object may be a pre-defined area of ​​the video frame, for example, the area in the top left corner of the video.

[0053] For example, if the related object is a human face and the target animation corresponds to the shape of the object, as shown in FIG. 5, when the user holds a fan-shaped object in his hand, the user's face will display a fan-shaped animation, thus realizing the interesting effect of mapping the animation.

[0054] Of course, in this example, both the area where the object is located and the area where the related object is located may be used as the animation adding area. That is, as shown in Fig. 6, when the target animation corresponds to the shape of the object, as shown in Fig. 6, when the user holds a fan-shaped object in his hand, the fan-shaped animation is displayed on both the user's face and the object in the user's hand, thereby realizing the interesting effect of mapping the animation.

[0055] Example 3: In this embodiment, for a certain object type, all image areas in the current video frame are determined as animation adding areas to create a sense of atmosphere.

[0056] That is, in this embodiment, for determining the object type of an object, the image features of the object can be learned according to deep learning technology to obtain the corresponding object type. If the object type belongs to a preset type, all image areas of the current video frame are determined as animation adding areas.

[0057] For example, if the object type of the object is determined to be "lucky money", the entire image area in the current video frame is determined as the animation adding area to create a sense of atmosphere. Furthermore, as shown in Fig. 7, the animation effect of "starlight" can be realized on the entire screen of the current video frame corresponding to opening the gift, so as to create the sense of atmosphere of the gift.

[0058] In the above example, the method of determining the target animation adding area of ​​the current frame is described. In another embodiment of the present invention, in order to further enhance the interest of the video, the animation of each video frame previously shot may be retained in the current video frame to form a trajectory-following animation display effect.

[0059] In one embodiment of the present disclosure, as shown in FIG. 8, before adding the target animation to the animation adding area, the method further includes the following steps:

[0060] Step 801: Detect whether the video frames captured before the current video frame include a historical video frame in which an object is held in the user's hand.

[0061] Step 802: if the history video frame is included, obtain the history animation additional area and the history target animation of the history video frame.

[0062] That is, it is determined whether the current video frame is the first video frame of the captured video, and if not, it determines the history video frame, and obtains the history animation addition area and the history target animation of the history video frame.

[0063] Step 803: Display the history target animation in the history animation addition area in the current video frame.

[0064] In this embodiment, when the history animation addition area and history target animation are obtained when an object is held in the user's hand, the history target animation is displayed in the history animation addition area in the current video frame, thereby realizing the effect of animation memory.

[0065] For example, when the animation-added areas are all areas where an object is located, the target animation is a Flash animation, and a Flash animation corresponding to the historical movement trajectory of the object is displayed in the current video frame, as shown in FIG. 9, to realize the effect of trajectory tracking. By using this animation display method, the user can realize interactive operations such as "drawing a heart" with the object.

[0066] The above is only an interesting video processing method for adding animation, and in practical application, other contents such as audio data can also be added along with the addition of the target animation, i.e., audio data corresponding to the target animation is determined. In some possible embodiments, the correspondence between the target animation and the audio data can be stored in advance, and the corresponding audio data can be determined by checking the correspondence, and the audio data can be played when the target animation is added. For example, in the example where the user holds a "sugar cube", the audio data after the "sugar cube" is held can be added to further enhance the sense of interactivity and improve the interest of the video shooting.

[0067] Based on the above, the method for adding animation to video in the embodiment of the present disclosure flexibly selects the animation adding area, and realizes various animation display effects based on the coordination between the animation adding area and the target animation, thereby further enhancing the interest of shooting video frames.

[0068] Based on the above embodiment, in order to further enhance the interest of video shooting, the movement trajectory of an object held by a user in his / her hand can be tracked, and even if the object leaves the user's hand, an animation can be displayed by tracking the movement of the object.

[0069] In one embodiment of the present disclosure, as shown in FIG. 10, the method further includes the following steps:

[0070] Step 1001: Detect whether an object has been released from the user's hand in a video frame captured after the current video frame.

[0071] In this embodiment, it may be determined whether the object has left the hand, for example by detecting whether the contour on which the object is located overlaps with the contour of the user's hand, or a deep learning method may be used to identify deep image features of a video frame taken after the current video frame, and the deep image features may be used to determine whether the object has left the user's hand.

[0072] Step 1002: If the object is separated from the user's hand, determine the movement trajectory of the object in the subsequent corresponding video frame after the object is separated from the user's hand.

[0073] To realize the addition of animation according to the object's departure trajectory, the movement trajectory of the object in the corresponding video frame after it leaves the user's hand is determined.

[0074] In some possible implementations, after the object leaves the user's hand, the position of the object in each corresponding video frame is determined and the object's position in all previous corresponding video frames in the shooting order is superimposed to obtain the object's movement trajectory in the corresponding video frame.

[0075] In some other possible embodiments, a first position in a subsequent corresponding video frame after the object is released from the user's hand is determined, a second position in a previous video frame where the object is held by the user's hand is determined, and a motion trajectory from the second position to the first position is directly generated. The trajectory shape of the motion trajectory can be any shape.

[0076] Step 1003: Determine a trajectory animation corresponding to the object, and add the trajectory animation according to the movement trajectory.

[0077] In this embodiment, the trajectory animation corresponding to the object may be a uniform animation preset by a video shooting application, or may be a trajectory animation corresponding to the image features of the object, that is, a correspondence between the image features of the object and the trajectory animation is preset, and the trajectory animation corresponding to the object is determined based on the correspondence. Of course, the trajectory animation corresponding to the object may be a trajectory animation randomly selected from a preset trajectory material library.

[0078] In this embodiment, a trajectory animation is added following the object area where the object is located, and a trajectory animation corresponding to the image area covered by the object's movement trajectory is added.

[0079] For example, as shown in FIG. 11, if the current video frame is “the dancer is holding a magic wand”, in the next video frame, when the “magic wand” leaves the “dancer’s” hand, the corresponding trajectory animation will be displayed directly according to the movement trajectory of the “magic wand” leaving the hand.

[0080] In view of the above, the animation adding method of the embodiment of the present disclosure can add animation according to the movement trajectory of an object after it leaves the hand, creating an animation effect of an object being thrown, further enhancing the interest of adding animation to a video.

[0081] To realize the above embodiment, the present disclosure further provides a video animation adding device.

[0082] 12 is a schematic structural diagram of a video animation adding device according to an embodiment of the present disclosure. The device can be realized by software and / or hardware, and can be generally integrated into electronic devices. As shown in FIG. 12, the device includes a first detection module 1210, a first determination module 1220, and an animation adding module 1230. The first detection module 1210 detects whether an object is being held by the user's hand in the current video frame currently being captured. A first determination module 1220 determines a target animation that corresponds to the object when the object is being held. The animation addition module 1230 determines an animation addition area in the current video frame that corresponds to the target animation, and adds the target animation to the animation addition area.

[0083] The video animation adding device according to an embodiment of the present disclosure can execute the video animation adding method provided by any embodiment of the present disclosure, and has corresponding functional modules for executing the method and beneficial effects.

[0084] According to the above, the animation adding device according to the embodiment of the present disclosure detects whether an object is held by the user's hand in the current video frame currently being shot, and determines a target animation corresponding to the object if the object is held, and finally determines an animation adding area in the current video frame corresponding to the target animation, and adds the target animation to the animation adding area, thereby adding an animation special effect corresponding to the object held by the user's hand in the video, intuitively reflecting the interaction between the user and the hand-held object, improving the interest of video shooting, and providing technical support for improving the user's adhesion to video products.

[0085] In one embodiment of the present disclosure, the first determination module 1220 specifically determines: Extracting preset image features corresponding to the object; A preset database is consulted to determine a target animation that corresponds to the preset image features.

[0086] The video animation adding device according to an embodiment of the present disclosure can execute the video animation adding method provided by any embodiment of the present disclosure, and has corresponding functional modules for executing the method and beneficial effects.

[0087] Based on the above, the video animation adding device of an embodiment of the present disclosure flexibly determines a target animation corresponding to an object, further improving the realism of the interaction between the user's hands and the object, and increasing the interest of the video.

[0088] In one embodiment of the present disclosure, the animation addition module 1230 specifically: determining associated objects of the object; Detect whether the current video frame contains an object of interest; If a related object is included, the area in which the related object exists and is located is determined as the animation addition area.

[0089] In one embodiment of the present disclosure, the animation addition module 1230 specifically: determining an object type for the object; If the object type belongs to the preset type, determine all the image areas of the current video frame as the animation adding area.

[0090] 13, the video animation adding device includes a first detection module 1310, a first determination module 1320, an animation adding module 1330, a second detection module 1340, and an acquisition module 1350. The first detection module 1310, the first determination module 1320, and the animation adding module 1330 have the same functions as the first detection module 1210, the first determination module 1220, and the animation adding module 1230 in the above embodiment, so they will not be repeated here.

[0091] In this embodiment, the second detection module 1340 obtains a historical video frame that was captured before the current video frame. The acquisition module 1350 acquires the historical animation additional area and the historical target animation of the historical video frame if the historical video frame includes the historical video frame. The animation addition module 1330 further displays the historical target animation in a historical animation addition area in the current video frame.

[0092] The video animation adding device according to an embodiment of the present disclosure can execute the video animation adding method provided by any embodiment of the present disclosure, and has corresponding functional modules for executing the method and beneficial effects.

[0093] Based on the above, the video animation adding device of the embodiment of the present disclosure flexibly selects the animation adding area, and by coordinating the animation adding area with the target animation, various animation display effects can be realized, further enhancing the interest of shooting video frames.

[0094] 14, the animation adding device includes a first detection module 1410, a first determination module 1420, an animation adding module 1430, a third detection module 1440, and a second determination module 1450. The first detection module 1410, the first determination module 1420, and the animation adding module 1430 have the same functions as the first detection module 1210, the first determination module 1220, and the animation adding module 1230 in the above embodiment, so they will not be repeated here.

[0095] The third detection module 1440 detects whether the object is released from the user's hand in a video frame captured after the current video frame. The second determination module 1450 determines, upon release from the user's hand, the movement trajectory of the object after it has been released from the user's hand in a subsequent corresponding video frame. The animation adding module 1430 further determines a trajectory animation corresponding to the object, and adds the trajectory animation according to the movement trajectory.

[0096] The video animation adding device according to an embodiment of the present disclosure can execute the video animation adding method provided by any embodiment of the present disclosure, and has corresponding functional modules for executing the method and beneficial effects.

[0097] In view of the above, the animation adding device according to an embodiment of the present disclosure can add animation according to the movement trajectory of the object after it leaves the hand, thereby creating an animation effect of the object being thrown, further enhancing the interest of adding animation to a video.

[0098] To realize the above embodiment, the present disclosure further provides a computer program, which includes instructions, which, when executed by a processor, realizes the method for adding animation to a video described in the above embodiment.

[0099] FIG. 15 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.

[0100] Hereinafter, referring specifically to FIG. 15, a schematic structural diagram suitable for realizing an electronic device 1500 in an embodiment of the present disclosure is shown. The electronic device 1500 in the embodiment of the present disclosure includes, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablets), PMPs (portable multimedia players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. The electronic device shown in FIG. 15 is merely an example and should not impose any limitations on the functions and scope of use of the embodiment of the present disclosure.

[0101] 15, the electronic device 1500 may include a processing unit (e.g., a central processing unit, a graphic processor, etc.) 1501, which can perform various appropriate operations and processes according to a program stored in a read only memory (ROM) 1502 or a program loaded from a storage device 1508 into a random access memory (RAM) 1503. The RAM 1503 also stores various programs and data necessary for the operation of the electronic device 1500. The processing unit 1501, the ROM 1502, and the RAM 1503 are connected to each other via a bus 1504. An input / output (I / O) interface 1505 is also connected to the bus 1504.

[0102] Typically, the following devices can be connected to the I / O interface 1505: input devices 1506 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 1507 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1508 including, for example, a magnetic tape, hard disk, etc.; and communication devices 1509. The communication devices 1509 allow the electronic device 1500 to communicate and exchange data with other devices, wirelessly or via wires. Although FIG. 15 illustrates the electronic device 1500 having various devices, it should be understood that it is not necessary to implement or include all of the devices shown. Alternatively, more or fewer devices can be implemented or included.

[0103] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowcharts can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product including a computer program embodied in a non-transitory computer readable medium, the computer program including program code for performing the method illustrated in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network via the communication device 1509, or installed from the storage device 1508, or installed from the ROM 1502. When the computer program is executed by the processing device 1510, it performs the above-mentioned functions limited in the method for adding animation to a video according to an embodiment of the present disclosure.

[0104] It should be noted that the above computer readable medium of the present disclosure may be a computer readable signal medium or a computer readable storage medium, or any combination of the above two. The computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, apparatus or device, or any combination of the above. More specific examples of the computer readable storage medium may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory (EPROM or flash memory), an optical fiber, a portable compact disk read only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer readable storage medium may be any tangible medium that contains or stores a program, which can be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, the computer readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, and is loaded with computer readable program code. Such propagated data signals may take various forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may transmit, propagate, or transmit a program for use by or in combination with an instruction execution system, device, or apparatus. The program code contained in the computer-readable medium may be transmitted using any suitable medium, including, but not limited to, electrical wires, optical cables, RF (radio frequency), or any suitable combination of these.

[0105] The computer-readable medium may be included in the electronic device, or may exist separately from the electronic device.

[0106] The computer-readable medium includes one or more programs, which, when executed by an electronic device, allow the electronic device to detect whether an object is held by the user's hand in the current video frame currently being shot, determine a corresponding target animation if an object is held, determine an animation adding area corresponding to the target animation in the current video frame, and add the target animation to the animation adding area, thereby adding an animation special effect corresponding to the object held by the user's hand in the video, intuitively reflecting the interaction between the user and the hand-held object, improving the interest of video shooting, and providing technical support for improving the user's stickiness to video products.

[0107] The computer program code for carrying out the operations of the present disclosure can be written in one or more programming languages ​​or combinations thereof. Such programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and also include traditional process-based programming languages ​​such as "C" or similar programming languages. The program code can run entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. When a remote computer is involved, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or connected to an external computer (such as connected via an Internet connection by an Internet service provider).

[0108] The flowcharts and block diagrams in the drawings illustrate possible system architectures, functions, and operations of the systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams may represent a module, program segment, or part of code, which includes one or more executable instructions for implementing a certain logical function. It should be noted that in some alternative implementations, the functions noted in the blocks may occur in a different order than the order shown. For example, two blocks shown in succession may actually be executed substantially in parallel, or may be executed in the reverse order, depending on the functionality involved. It should be noted that each block in the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, may be implemented by a dedicated hardware-based system for executing a certain function or operation, or may be implemented by a combination of dedicated hardware and computer instructions.

[0109] The units according to the embodiments of the present disclosure may be implemented in the form of software or hardware, and the names of the units may not be limitations on the units themselves in some cases.

[0110] The functions described herein may be performed, at least in part, by one or more hardware logic components. For example, but not limited to, exemplary types of hardware logic components that may be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), etc.

[0111] In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain or store a program for use by or in combination with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium includes, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the above. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), optical fibers, compact disk read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0112] The above description is only a description of the preferred embodiments of the present disclosure and the technical principles used. As can be understood by those skilled in the art, the disclosure scope of the present disclosure is not limited to the technical solution consisting of a specific combination of the above technical features, and also covers other technical solutions that combine the above technical features or their equivalent features in any way without departing from the concept of the above disclosure. For example, a technical solution formed by replacing the above features with technical features having similar functions disclosed in the present disclosure (not limited to them).

[0113] Also, although operations are described in a particular order, this should not be understood as requiring that these operations be performed in the particular order or sequence shown. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although the above description includes some specific implementation details, these should not be construed as limitations on the scope of the disclosure. Some features described in separate embodiments may be implemented in a single embodiment in combination. Rather, various features described in a single embodiment may be implemented in multiple embodiments alone or in any suitable subcombination.

[0114] Although the present subject matter has been described in language specific to structural features and / or logical operations of methods, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or operations described above. Rather, the specific features and operations described above are merely example forms for implementing the claims.

Claims

1. 1. A method for adding animation to a video, the method being performed by an apparatus for adding animation to a video, the method comprising: detecting whether an object is being held by a user's hand in a currently captured video frame; determining a target animation corresponding to the object in response to the object being held; determining an animation addition area in the current video frame corresponding to the target animation, and adding the target animation to the animation addition area; The step of determining an animation addition area in the current video frame corresponding to the target animation includes: determining an associated object of the object; detecting whether the associated object is included in the current video frame; and determining an area where the associated object is located as an animation addition area according to whether the associated object is included; or determining an object type of the object; and determining all image areas of the current video frame as the animation adding area according to the object type belonging to a preset type; A method comprising:

2. The step of determining a target animation corresponding to the object comprises: extracting preset image features corresponding to the object; and determining a target animation corresponding to the preset image characteristics.

3. The preset image features include:

3. The method of claim 2, further comprising at least one of a shape characteristic of the object, a color characteristic of the object, a size characteristic of the object, an object type of the object, a position characteristic of the object, and a gesture characteristic of the user's hand with which the object is being held.

4. The step of determining an animation addition area in the current video frame corresponding to the target animation includes:

2. The method of claim 1, further comprising determining an image area in which the object is located as the animation addition area.

5. before the step of adding the target animation to the animation adding area, detecting whether video frames captured prior to the current video frame include a historical video frame in which the object is held in the user's hand; According to including the history video frame, obtaining a history animation additional area and a history target animation of the history video frame; 5. The method according to claim 1, further comprising the step of: displaying the history target animation in the history animation additional area in the current video frame.

6. detecting whether the object has left the user's hand in a video frame captured after the current video frame; In response to the object being released from the user's hand, determining a movement trajectory of the object in a subsequent corresponding video frame after the object is released from the user's hand; 2. The method of claim 1, further comprising determining a trajectory animation corresponding to the object and adding the trajectory animation according to the movement trajectory.

7. determining audio data corresponding to the target animation; 2. The method of claim 1, further comprising the step of: playing said audio data when said target animation is added.

8. A video animation adding device, comprising: a first detection module for detecting whether an object is being held by a user's hand in a current video frame currently being captured; a first determination module for determining a target animation corresponding to the object in response to the object being held; an animation adding module for determining an animation adding area in the current video frame corresponding to the target animation, and adding the target animation to the animation adding area; The animation adding module specifically includes: Determine an associated object of the object, detect whether the associated object is included in the current video frame, and, according to the inclusion of the associated object, determine an area in which the associated object is located as an animation addition area; or determining an object type of the object; and determining all image areas of the current video frame as the animation adding area according to the object type belonging to a preset type; 13. An apparatus for adding animation to a video.

9. The first determination module specifically comprises: Extracting preset image features corresponding to the object; 9. The apparatus of claim 8, further comprising: determining a target animation corresponding to the preset image characteristics.

10. a second detection module for detecting whether the video frames captured before the current video frame include a historical video frame in which an object is held in the user's hand; an acquisition module for acquiring a history animation additional area and a history target animation of the history video frame in response to including the history video frame; 10. The apparatus of claim 8 or 9, further comprising: the animation adding module for displaying the historical target animation in the historical animation adding area in the current video frame.

11. a third detection module for detecting whether the object is released from the user's hand in a video frame captured after the current video frame; a second determination module for determining, in response to the object being released from the user's hand, a movement trajectory of the object in a subsequent corresponding video frame after the object is released from the user's hand; 9. The apparatus of claim 8, further comprising: an animation adding module for determining a path animation corresponding to the object and adding the path animation according to the movement path.

12. An electronic device, A processor; a memory for storing instructions executable by said processor; The processor is operable to read the executable instructions from the memory and execute the instructions to implement the method for adding animation to a video according to any one of claims 1 to 7.

13. 8. A computer-readable storage medium having a computer program stored therein, the computer program being used to perform the method for adding animation to a video according to any one of claims 1 to 7.

Images