Method, apparatus, electronic device and storage medium for determining special effect video

Abstract

Embodiments of the present disclosure provide a method, apparatus, electronic device, and storage medium for determining a special effect video. The method includes: obtaining special effect operation information including at least one of an audio special effect operation, a touch control special effect operation, and a gesture special effect operation during a video shooting process; calling a target additional waiting special effect corresponding to the special effect operation information from a special effect resource library; performing a fusion process on the target additional waiting special effect for each of a plurality of processing waiting video frames to determine a plurality of target special effect video frames; and determining a target special effect video based on the plurality of target special effect video frames.

Description

Technical Field

[0001] This disclosure claims the priority of a Chinese patent application with the application number 202210540730.5, which was filed with the Chinese Patent Office on May 17, 2022. All the contents of the above application are incorporated herein by reference.

[0002] The embodiments of this disclosure relate to the technology of adding special effects, for example, a special effect video determination method, apparatus, electronic device, and storage medium.

Background Art

[0003] Currently, both short videos and live broadcasts are popular Internet content dissemination methods. During the shooting process of short videos and live broadcasts, special effects can be added to enhance the visual effect.

[0004] However, the addition of special effects in related technologies has a relatively fixed method and cannot add special effects according to the personalized needs of users. This leads to the problems of low interactivity between users and videos and low interest in shooting videos, which may affect the visual effect of the videos and the user experience.

Summary of the Invention

[0005] This disclosure provides a method, apparatus, electronic device, and storage medium for determining a special effect video to improve the interaction flexibility of adding special effects to a video and enhance the interest in shooting the video.

[0006] In a first aspect, embodiments of this disclosure are A method for determining a special effect video, the method comprising: In the process of shooting a video, obtaining special effect operation information including at least one of an audio special effect operation, a touch control special effect operation, and a gesture special effect operation; calling, from the special effect resource library, a target pending special effect corresponding to the special effect operation information; performing a fusion process on the target pending special effect for each of a plurality of video frames waiting for processing, to determine a plurality of target special effect video frames; determining a target special effect video based on the plurality of target special effect video frames, including: A method for determining a special effect video is provided.

[0007] In a second aspect, embodiments of the present disclosure provide an apparatus for determining a special effect video, the apparatus including: a special effect operation information acquisition module configured to acquire special effect operation information including at least one of an audio special effect operation, a touch control special effect operation, and a gesture special effect operation during a video shooting process; a target pending special effect determination module configured to call, from the special effect resource library, a target pending special effect corresponding to the special effect operation information; a target special effect video frame determination module configured to perform a fusion process on the target pending special effect for each of a plurality of video frames waiting for processing, to determine a plurality of target special effect video frames; a target special effect video determination module configured to determine a target special effect video based on the plurality of target special effect video frames. An apparatus for determining a special effect video is further provided.

[0008] In a third aspect, embodiments of the present disclosure provide an electronic device, the electronic device including: one or more processors; a storage device configured to store one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement a method for determining a special effect video according to any of the embodiments of the present disclosure. An electronic device is further provided.

[0009] In a fourth aspect, A storage medium including computer-executable instructions used to execute a method for determining a special effect video according to any of the embodiments of the present disclosure when executed by a processor of a computer is provided.

Brief Description of the Drawings

[0010] Throughout the drawings, the same or similar reference numerals indicate the same or similar elements. It should be understood that the drawings are schematic and the elements and elements are not necessarily drawn to scale.

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Embodiments for Carrying Out the Invention

[0012] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be realized in various forms. It should be understood that the drawings and embodiments of the present disclosure are merely exemplary.

[0013] It should be understood that each step described in the embodiments of the method of the present disclosure may be executed in a different order and / or in parallel. Also, the method embodiments may include additional steps and / or omit the execution of the steps shown.

[0014] The term "including" and its variations used in this document are open inclusion, that is, "including but not limited to these". The term "based on" means "at least partially based on". The term "one embodiment" represents "at least one embodiment", the term "another embodiment" represents "at least one another embodiment", and the term "some embodiments" represents "at least some embodiments". Related definitions of other terms are given in the following description.

[0015] Note that the concepts such as "first", "second", etc. mentioned in the present disclosure are only for distinguishing different devices, modules or units, and are not for limiting the order or interdependence relationship of the functions executed by these devices, modules or units.

[0016] Note that the modifiers "one" and "a plurality" mentioned in the present disclosure are schematic, and those skilled in the art should understand that, unless otherwise specified in the context, it should be understood as "at least one".

[0017] The names of the messages or information that interact among multiple devices in the embodiments of the present disclosure are merely for the purpose of description and are not intended to limit the scope of these messages or information.

[0018] Before using the technical solutions disclosed in each example of the present disclosure, it can be understood that the type, scope of use, usage scenarios, etc. of the personal information related to the present disclosure should be notified to the user in an appropriate manner in accordance with the relevant laws and regulations, and the user's permission should be obtained.

[0019] For example, when responding to the receipt of the user's active request, the user should be clearly presented with the fact that it is necessary to obtain and use the user's personal information for the operation required to be executed by sending the presentation information to the user. Thereby, based on the presentation information, the user can independently select whether to provide personal information to software or hardware such as an electronic device, an application program, a server, or a storage medium that executes the operations of the technical solution of the present disclosure.

[0020] As one preferred implementation form, the method of sending presentation information to the user in response to the receipt of the user's active request may be, for example, in the form of a pop-up window, and the pop-up window can show the presentation information in text form. Also, the pop-up window may be equipped with a selection widget for the user to select whether to "agree" or "disagree" to provide personal information to the electronic device.

[0021] As described above, it can be understood that the process of notifying the user and obtaining the user's permission is merely exemplary, and other methods that comply with the relevant laws and regulations are also applicable to the implementation forms of the present disclosure.

[0022] It can be understood that for the data related to the technical solution of the present disclosure (for example, including the data itself, the acquisition or use of the data), it should comply with the requirements of the corresponding laws, regulations, and relevant regulations.

[0023] Before introducing this technical solution, an exemplary description of the application scenario may be given first. The technical solution of the present disclosure can be applied to any scene where it is necessary to shoot a special effect video, interact with the content of the video, or perform special effect processing during the image shooting process. For example, the content being shot during the video shooting process may be applied to a scene where special effects can be demonstrated, such as the shooting scene of a short video.

[0024] Video shooting may be a dynamic shooting scene, and image shooting may be a static shooting scene. That is, in either a dynamic shooting scene or a static shooting scene, if it is desired to give a certain special effect to the content of the image, the technical solution according to the embodiments of the present disclosure can be adopted and implemented. It can be understood that this technical solution can be integrated into any existing shooting device. For example, this method may be integrated into a mobile terminal with a camera head, or into other dedicated cameras. Of course, for the sake of convenience, this method can be integrated into a personal computer (PC). FIG. 1 is a flowchart of a special effect video determination method according to an embodiment of the present disclosure. The embodiments of the present disclosure are applicable when there can be a certain interactivity between the content of the video during the video shooting process. It is also applicable when interacting with the content of the video when shooting with video shooting software such as live streaming software and video chat software. This method can be executed by a special effect video determination device, and the device can be realized in the form of software and / or hardware. Preferably, it is realized by an electronic device, and the electronic device may be a mobile terminal, a PC terminal, a server, or the like.

[0025] As shown in FIG. 1, the method includes the following.

[0026] In S110, during the video shooting process, special effect operation information is obtained.

[0027] An apparatus for executing a method for determining a special effect video according to an embodiment of the present disclosure can be integrated into application software that supports a special effect image processing function, and the software can be installed on an electronic device. Preferably, the electronic device may be a mobile terminal or a PC terminal, etc. The application software may be a type of software for image / video processing, as long as this type of software can realize image / video processing. The application software may be a specially developed application program realized in software that adds special effects to display special effects or integrated into a corresponding page, and the user can realize special effect addition processing through the page integrated on the PC terminal.

[0028] Shooting of a video by a camera in a smart device, or shooting of a video by a video chat or video recording function in any software, or live streaming by any live streaming software, etc., can all make it possible to understand that it is in the process of shooting a video. The special effect operation information may be related operation information for special effects to be added subsequently.

[0029] The special effect operation information includes at least one of a voice special effect operation, a touch control special effect operation, and a gesture special effect operation.

[0030] The voice special effect operation may be a special effect trigger operation executed by the user in a voice manner. The touch control special effect operation may be a special effect trigger operation executed by the user in a manner of touching the screen. The gesture special effect operation may be a special effect trigger operation executed by the user by demonstrating a specific gesture in the shooting screen during shooting.

[0031] Exemplarily, in the process of shooting a video, the special effect operation information of the user is captured and obtained. For example, a voice special effect operation such as "shooting start" is detected, or a touch control special effect operation such as a click on a specific position of the screen is detected, or a gesture special effect operation such as an "OK" gesture is detected.

[0032] Note that multiple types of special effect operation information can be detected simultaneously. For example, voice information, or touch control operations, or gesture information can be obtained.

[0033] That is, it is possible to intelligently determine which one or more of the current operation information corresponds to among the special effect operations. Of course, in order to improve the efficiency and accuracy of the interaction, at least one interaction mode can be combined to obtain multiple types of trigger operation modes, and the special effect operation information can be determined based on the selection of the trigger operation mode.

[0034] Preferably, at least one special effect operation mode waiting for selection is displayed on the display interface, the triggered special effect operation mode waiting for selection is set as the target special effect operation mode, and the corresponding special effect operation information is obtained based on the target special effect operation mode.

[0035] The special effect operation mode waiting for selection may be a mode corresponding to at least one special effect operation information such as, for example, a voice special effect operation mode, a touch control special effect operation mode, a gesture special effect operation mode, a voice + touch control special effect operation mode, and a voice + gesture special effect operation mode. The display interface may be, for example, the display interface of a shooting device such as the shooting interface displayed on a mobile phone. The target special effect operation mode may be the special effect operation mode waiting for selection triggered by the user, that is, the special effect operation mode waiting for selection to be used subsequently.

[0036] Exemplarily, according to the user's selection, a plurality of special effect operation modes waiting for selection may be displayed on the display interface. Further, the triggered special effect operation mode waiting for selection is set as the target special effect operation mode, and then, when determining whether a special effect addition operation has been triggered, the target special effect operation mode is used to obtain corresponding special effect operation information based on the target special effect operation mode.

[0037] Note that on the display interface, various special effect operation modes waiting for selection can be displayed in a form where the view is swiped, and on the display interface, various special effect operation modes waiting for selection can also be displayed in the form of a selection widget.

[0038] Note that the reason for determining the target special effect mode is to quickly respond to specific special effect operation information. By responding to the target special effect mode, interference from special effect operation information of other special effect operation modes waiting for selection is avoided. For example, when the target special effect operation mode is a gesture special effect operation mode, the special effect operation information corresponding to the voice special effect operation mode will not be processed even if it exists, and when the target operation mode is a voice + touch control special effect operation mode, the special effect operation information corresponding to the voice special effect operation mode will not be processed even if it exists.

[0039] Preferably, before video shooting, it can be determined that video shooting has been triggered by any one or more of the following methods.

[0040] In Method 1, it is detected that the video shooting widget has been triggered.

[0041] The video shooting widget may be a button for triggering shooting, and the button may be a physical button or a virtual button such as, for example, a shooting button on a camera, a live streaming start button in live streaming software, or a special effect addition button. Preferably, the video shooting widget may be a button corresponding to any special effect tool among the special effect tools.

[0042] Exemplarily, when it is detected that the video shooting widget is triggered, it is considered that the shooting of the video is triggered, that is, it is determined that the shooting process of the video has already started.

[0043] In Method 2, it is detected that the captured screen contains the target object.

[0044] The captured screen may be the screen captured by the lens. The target object may be a preset object, and the target object may be, for example, a specific object or object type such as a person, an animal, a tree, a vehicle, etc.

[0045] Exemplarily, when the target object is a specific object, an image of the specific object can be uploaded in advance and the feature information of the specific object can be learned. When the captured screen contains the same object as the learned feature information, it indicates that the shooting of the video is triggered, that is, it is determined that the shooting process of the video has already started. When the target object is an object type, when the captured screen contains any object corresponding to the object type, it can be determined that the shooting of the video is triggered, that is, it can be determined that the shooting process of the video has already started.

[0046] In Method 3, it is detected that the facial information matches the preset facial information.

[0047] The facial information may be expression information such as a smile, a blink, or a duck mouth. The pre-set facial information may be facial information for triggering the shooting of a pre-set video.

[0048] Exemplarily, detect the facial information in the captured screen, and when the facial information matches the pre-set facial information, it is determined that the shooting of the video has been triggered, that is, it is determined that the shooting process of the video has already started.

[0049] In Method 4, it is detected that the audio information triggers a video shooting command.

[0050] The audio information may be the audio of the user being collected.

[0051] Exemplarily, receive the user's audio, determine the information related to the start of shooting from the audio, for example, when detecting the audio information of "start shooting", it can be determined that the shooting of the video has been triggered, that is, it is determined that the shooting process of the video has already started.

[0052] In Method 5, it is detected that the limb movement of the target object in the captured screen matches the limb movement corresponding to the pre-set limb movement information.

[0053] The limb movement may include movements such as nodding, waving, and jumping performed by human body parts such as the head, neck, hands, elbows, arms, body, hip sitting, and feet. The pre-set limb movement information may be limb movement information for triggering the shooting of a pre-set video.

[0054] Exemplarily, detect the limb movement of the target object in the captured screen, and determine that the limb movement matches the limb movement corresponding to the preset limb movement information. For example, when the detected limb movement is the "OK" gesture of the hand movement, it is determined that the video shooting has been triggered, that is, it is determined that the video shooting process has already started.

[0055] In S120, call the target additional waiting special effect corresponding to the special effect operation information from the special effect resource library.

[0056] The target additional waiting special effect may be a special effect corresponding to the special effect operation information, that is, a special effect to be added to the subsequent video frame. The special effect resource library may be a storage space stored locally or in the cloud, and is used to store various special effects corresponding to various special effect operation information.

[0057] Note that the special effect resource library may include various special effects added by the program developer during development, or various special effects uploaded and created by the user. For example, the user uploads a special effect and sets the special effect operation information corresponding to the special effect, so that the special effect can be called by the special effect operation information during subsequent use. The special effect uploaded by the user may be a single picture, and the user can process the picture to obtain the desired special effect. For example, the user uploads a picture with a hat, trims the picture, and obtains the area where the hat is located as a hat special effect.

[0058] Exemplarily, after obtaining the special effect operation information, based on the special effect operation information, the target additional waiting special effect corresponding to the special effect operation information, that is, which special effect to add to the subsequent video frame, can be determined from the special effect resource library.

[0059] Preferably, since the special effect operation information can include at least one of a voice special effect operation, a touch control special effect operation, and a gesture special effect operation, the target additional waiting special effect may be a special effect that matches at least one special effect operation among the special effect operation information.

[0060] Preferably, the target additional waiting special effect includes a dynamic special effect and / or a static special effect.

[0061] Among the dynamic special effect and the static special effect, the dynamic special effect may be a special effect having dynamics. For example, it is a moving special effect having a certain movement direction and movement speed, a special effect having a shape change, a change in light and shadow, etc. The static special effect may be a special effect having a fixed position and a fixed shape, and the relative position between the static special effect and the target object may be fixed. The target additional waiting special effect may be a special effect determined for the relevance of subsequent special effects.

[0062] Note that whether to add a dynamic special effect or a static special effect may be determined by the user's selection, may be determined by the form of the special effect stored in advance, may be determined by analyzing based on the shooting screen, or may be determined based on the user's operation information.

[0063] Preferably, when it is detected that the dynamic special effect is triggered, the related special effect corresponding to the target additional waiting special effect is displayed.

[0064] The related special effect may be another special effect that is associated with the target additional waiting special effect and is distinguished from the target additional waiting special effect.

[0065] Exemplarily, when the dynamic special effect is triggered, the related special effect corresponding to the target additional waiting special effect can be determined and the related special effect can be displayed.

[0066] Exemplarily, when detecting that a dynamic special effect is triggered, if it is determined that the target addition waiting special effect is "addition of floating heart-shaped bubbles" and the related special effect corresponding to the target addition waiting special effect is "heart-shaped bubbles that grow from small to large and then burst", the related special effect can be displayed, that is, the dynamic special effect of the heart-shaped bubbles can be displayed.

[0067] In S130, a target addition waiting special effect is fused to each of a plurality of processing waiting video frames to determine a plurality of target special effect video frames.

[0068] The processing waiting video frame may be a video frame captured after acquiring special effect operation information. The target special effect video frame may be a video frame after adding a special effect.

[0069] Exemplarily, after determining the target addition waiting special effect, the target addition waiting special effect is superimposed and fused onto the processing waiting video frame, so that the target addition waiting special effect can be added to the processing waiting video frame, and the processed video frame can be used as the target special effect video frame.

[0070] Preferably, By determining the target display position in the processing waiting video frame of the target addition waiting special effect, fusing the target addition waiting special effect at the target display position, and obtaining the target special effect video frame, The target addition waiting special effect can be fused to the processing waiting video frame to determine the target special effect video frame.

[0071] The target display position may be the position where the target addition waiting special effect is added, or the position determined based on the special effect operation information, or the position determined based on the processing waiting video frame, etc.

[0072] Exemplarily, when the special effect operation information is "put a hat on a person", it is possible to determine that the target display position in the video frame waiting for processing is the head position of the person. Thereby, a special effect waiting for target addition can be fused with the target display position to obtain a target special effect video frame. When it is determined that the special effect waiting for target addition is "addition of fish", it is possible to detect whether the information on the water area is included in the video frame waiting for processing. When the information on the water area is included, the special effect waiting for target addition can be added to the water area.

[0073] In S140, based on a plurality of target special effect video frames, a target special effect video is determined.

[0074] Among them, the target special effect video may be a video in which a plurality of target special effect video frames are continuously played exceeding the number of frames set in advance per second. The number of frames set in advance is usually greater than 24 frames. The target special effect video may be a video after the target special effect video is added.

[0075] Exemplarily, after determining a plurality of target special effect video frames, the plurality of target special effect video frames can be continuously played in order to obtain a target special effect video.

[0076] Preferably, according to the generation time stamp of the target special effect video frame, by stitching a plurality of target special effect video frames to determine the target special effect video, Based on a plurality of target special effect video frames, a target special effect video can be determined.

[0077] The generation time stamp may be the time information at the time of generating the target special effect video frame.

[0078] Exemplarily, when generating a target special effect video frame, a generation timestamp is added to the target special effect video frame. Further, a plurality of target special effect video frames are rearranged and stitched based on the generation timestamp, and the video after stitching is used as the target special effect video.

[0079] The technical solution of the embodiments of the present disclosure is to obtain special effect operation information during the video shooting process, call a target additional special effect corresponding to the special effect operation information from the special effect resource library, identify the special effect that the user desires to add, perform a fusion process on the target additional special effect in the video frame waiting to be processed, determine the target special effect video frame, and determine the target special effect video based on a plurality of target special effect video frames, thereby overlaying a special effect on the captured video and solving the problem that the flexibility of adding special effects to the video is small and the interestingness is low, achieving the technical effect of improving the interaction flexibility of adding special effects to the video and enhancing the interestingness of shooting the video.

[0080] Based on the above embodiments, the step of calling a target additional special effect corresponding to the special effect operation information from the special effect resource library may be: It may also be a step of determining a target additional special effect from the special effect resource library based on the screen content and the special effect operation information in the video frame waiting to be processed.

[0081] The screen content may be the current display screen information in the video frame waiting to be processed, and each video frame waiting to be processed corresponds to one screen content.

[0082] Preferably, the screen content may include target information, current position information, current time information, etc.

[0083] Exemplarily, after determining and obtaining the special effect operation information and the screen content of the video frame waiting for processing, based on the comprehensive special effect operation information and the screen content, matching processing is performed in the special effect resource library, and the matched special effect can be set as the special effect waiting for target addition.

[0084] FIG. 2 is a flowchart of a method for determining a special effect video when the special effect operation information according to an embodiment of the present disclosure is an audio special effect operation. Based on the above technical solution, for the case where the special effect operation information is an audio special effect operation, the method for determining the special effect waiting for target addition can refer to the detailed description of this technical solution. Among them, the interpretation of the same or corresponding terms as the above technical solutions will not be repeated here.

[0085] As shown in FIG. 2, the method includes the following.

[0086] In S210, during the video shooting process, special effect operation information is obtained.

[0087] In S220, when the special effect operation information is an audio special effect operation, an audio data stream corresponding to the audio special effect operation is obtained.

[0088] The audio data stream may be a data stream of the user's voice collected during the video shooting process.

[0089] Exemplarily, after determining the audio special effect operation, an audio data stream corresponding to the audio special effect operation can be obtained and used for the subsequent determination of the special effect waiting for target addition.

[0090] In S230, based on the audio data stream, a special effect waiting for target addition is called from the special effect resource library.

[0091] Exemplarily, based on the audio data stream, a special effect that matches the audio data stream is determined from the special effect resource library as the target additional waiting special effect, and the target additional waiting special effect can be called for subsequent processing.

[0092] Preferably, the target additional waiting special effect can be called according to the screen content and the audio data stream in the video frame waiting for processing. For example, the target additional waiting special effect may be called from the special effect resource library based on the screen content and the audio data stream in the video frame waiting for processing.

[0093] Exemplarily, after obtaining the audio data stream, the screen content of the video frame waiting for processing is determined, and matching is performed in the special effect resource library according to the audio data stream and the screen content, and the target additional waiting special effect is matched and called.

[0094] Preferably, when the screen content includes target information and current position information, Based on at least one keyword corresponding to the audio data stream, the target information of the target object in the screen content, and the current position information, in the manner of calling the target additional waiting special effect from the special effect resource library, Based on the screen content and the audio data stream in the video frame waiting for processing, the target additional waiting special effect can be called from the special effect resource library.

[0095] Among the target information and the current position information, the target information may be information for describing the target. The target information includes, for example, target basic information such as hairstyle, clothing color, and whether or not wearing glasses. The current position information includes the scene position to which the target belongs and / or the current geographical position of the target. The scene position to which the target belongs may be the position of the target in the scene screen. The current geographical position may be the geographical position where the video is shot, for example, the geographical position determined by a positioning system. Regarding the keyword, the audio data stream can be processed, decomposed into different words with actual meanings, and these words can be used as keywords.

[0096] In addition, processing the audio data stream may be to perform speech-to-text conversion processing, and perform processing such as word decomposition, removal of meaningless words, and part-of-speech analysis on the text, whereby the remaining nouns can be used as keywords.

[0097] Exemplarily, when the character information obtained by processing the audio data stream is "a woman with long hair wears a hat", the keywords obtained by processing may be "a woman with long hair" and "hat".

[0098] Exemplarily, after processing the audio data stream to obtain each keyword, based on each keyword, the target information of the target in the screen content, and the current position information, as a special effect waiting for target addition, one special effect that meets various needs can be called from the special effect resource library.

[0099] Exemplarily, when the keyword is "hat", the target information of the target is long hair, and the current position information is a certain ethnic minority area, a hat special effect with the style of the ethnic minority that matches the long hair can be called from the special effect resource library.

[0100] Preferably, at least one keyword corresponding to the audio data stream, the target information of the target object in the screen content, and the current time information may be used to call a target additional waiting special effect from the special effect resource library.

[0101] The current time information may be, for example, date information such as holiday information.

[0102] Exemplarily, when the keyword is "hat", the target information of the target object is long hair, the current time information is January 1st, and the holiday corresponding to the current time information can be set as New Year's Day, a Spring Festival hat suitable for long hair can be called from the special effect resource library. For example, the Spring Festival hat may be a hat with a tiger head.

[0103] Preferably, the following steps can be used to call a target additional waiting special effect from the special effect resource library based on at least one keyword.

[0104] In step 1, when at least one keyword in the audio data stream includes target name information and the screen content includes at least one target object, a target special effect addition target corresponding to the target name information is determined from the at least one target object.

[0105] The target name information may be a name for representing the target. For example, the target name information may be a name, a nickname, a number, etc. The target special effect addition target may be the target object corresponding to the target name information in the screen content, that is, the object to which the special effect will be added later.

[0106] Exemplarily, based on a pre-stored correspondence between a target name and a target, such as a correspondence between a stored target name and a target facial image, a facial image corresponding to the target name in the screen content is searched, and the target object with the facial image is set as the target special effect addition target.

[0107] Exemplarily, when the target name information A corresponds to the facial image a, the target name information B corresponds to the facial image b, and the screen content includes the facial image a and the facial image b, the targets corresponding to the target name information A and the target name information B are the target targets. When the keyword includes the target name information A, the target target corresponding to the facial image >a is set as the target special effect addition target.

[0108] In step 2, based on at least one keyword and the target special effect addition target, the target additional pending special effect is called from the special effect resource library.

[0109] Exemplarily, based on at least one keyword and the target special effect addition target, a special effect that simultaneously meets the needs of the keyword and the target special effect addition target is determined from the special effect resource library, the determined special effect is set as the target additional pending special effect, the target additional pending special effect is called, and the target additional pending special effect can be used for the subsequent addition of the special effect.

[0110] In S240, the target additional pending special effect is fused with the processing pending video frame to determine the target special effect video frame.

[0111] In S250, based on each target special effect video frame, the target special effect video is determined.

[0112] In the technical solution of the embodiments of the present disclosure, during the video shooting process, special effect operation information is obtained. When the special effect operation information is an audio special effect operation, an audio data stream corresponding to the audio special effect operation is obtained, and based on the audio data stream, a target additional waiting special effect is called from the special effect resource library, thereby determining the target additional waiting special effect by the audio special effect operation, fusing the target additional waiting special effect with the video frame waiting for processing, determining the target special effect video frame, and determining the target special effect video based on each target special effect video frame, so as to solve the problem that the flexibility of adding special effects to the video is small and the interestingness is low, improve the interaction flexibility of adding special effects to the video, and achieve the technical effect of enhancing the interestingness of shooting the video.

[0113] FIG. 3 is a flowchart of a method for determining a special effect video when the special effect operation information according to the embodiments of the present disclosure is a touch control special effect operation. Based on the above technical solution, when the special effect operation information is a touch control special effect operation, the determination method of the target additional waiting special effect can refer to the detailed description of this technical solution. Among them, the interpretation of the same or corresponding terms as the above technical solutions will not be repeated here.

[0114] As shown in FIG. 3, the method includes the following.

[0115] In S310, during the video shooting process, special effect operation information is obtained.

[0116] In S320, when the special effect operation information is a touch control special effect operation, based on the trigger operation on the display interface, the screen content in the video frame waiting for processing is obtained.

[0117] The trigger operation corresponds to a touch control special effect operation. The trigger operation may be a click operation on the display interface or the like. The screen content may be various information in the current video frame waiting for processing, that is, various information related to the current shooting screen.

[0118] Exemplarily, when adding a special effect by a touch control special effect operation, the user can execute a trigger operation through the display interface. When the trigger operation is detected, the screen content in the video frame waiting for processing can be obtained. The current shooting screen may be used as the screen content, or the current shooting screen may be elementarily divided to obtain the screen content.

[0119] In S330, based on the visual elements in the screen content, the target additional waiting special effect is determined.

[0120] The visual elements may be object information or may be scene information or the like.

[0121] Exemplarily, by determining visual elements from the screen content and analyzing the visual elements, the target additional waiting special effect corresponding to the analysis result can be determined from the special effect resource library.

[0122] Preferably, based on the visual elements in the screen content and the position information corresponding to the touch point, in the manner of determining the target additional waiting special effect, the target additional waiting special effect can be determined based on the visual elements in the screen content.

[0123] The touch point may be, for example, a point corresponding to a trigger operation such as a point where the user clicks on the screen. The position information corresponding to the touch point may be the position information on the display interface of the touch point.

[0124] Exemplarily, based on the position information corresponding to the touch point, it is possible to determine which visual element in the screen content the position information corresponds to. Further, based on the visual element, a target additional special effect can be determined from the special effect resource library.

[0125] Exemplarily, when the visual element corresponding to the position information corresponding to the touch point is "ground", the target additional special effect to be searched for may be "mushrooms, flowers or grass growing from the ground", etc. When the visual element corresponding to the position information corresponding to the touch point is "balloon", the target additional special effect searched from the special effect resource library may be "the balloon being burst", etc.

[0126] In addition, when the special effect operation information is a touch control special effect operation, the camera head used when shooting a video may be a front camera head or a rear camera head.

[0127] In S340, the target additional special effect is fused with the video frame waiting for processing to determine the target special effect video frame.

[0128] In S350, based on each target special effect video frame, the target special effect video is determined.

[0129] In the technical solution of the embodiments of the present disclosure, during the video shooting process, special effect operation information is obtained. When the special effect operation information is a touch control special effect operation, based on the trigger operation on the display interface, the screen content in the video frame waiting to be processed is obtained, and based on the visual elements in the screen content, the target additional waiting special effect is determined. Thereby, the target additional waiting special effect is determined by the touch control special effect operation, and the target additional waiting special effect is fused and processed in the video frame waiting to be processed to determine the target special effect video frame, and the target special effect video is determined based on each target special effect video frame, thereby solving the problem that the flexibility of adding special effects to the video is small and the interestingness is low, improving the interaction flexibility of adding special effects to the video, and achieving the technical effect of enhancing the interestingness of shooting the video.

[0130] FIG. 4 is a flowchart of a method for determining a special effect video when the special effect operation information according to the embodiments of the present disclosure is a gesture special effect operation. Based on the above technical solution, for the case where the special effect operation information is a gesture special effect operation, the determination method of the target additional waiting special effect can refer to the detailed description of this technical solution. Among them, the interpretation of the same or corresponding terms as the above technical solutions will not be repeated here.

[0131] As shown in FIG. 4, the method includes the following.

[0132] In S410, during the video shooting process, special effect operation information is obtained.

[0133] In S420, when the special effect operation information is a gesture special effect operation, a gesture special effect operation on the display interface is detected, and a target gesture pose is determined.

[0134] The target gesture pose may be a gesture pose captured on the display interface, or the target gesture pose may be a gesture pose of any target object on the display interface.

[0135] Exemplarily, when detecting a gesture special effect operation on a display interface and detecting the gesture special effect operation, a target gesture pose corresponding to the gesture special effect operation is determined.

[0136] Exemplarily, the target gesture pose may be a static gesture such as a peace sign, a finger heart, etc., or may be a dynamic gesture such as pinching with a finger, grasping with a hand, or waving a hand.

[0137] In S430, based on the target gesture pose, a target additional waiting special effect is called from a special effect resource library.

[0138] Exemplarily, based on the target gesture pose, a special effect corresponding to the target gesture pose is searched in a special effect resource library, the searched special effect is used as the target additional waiting special effect, and the target additional waiting special effect can be called and used for adding subsequent special effects.

[0139] Preferably, based on the target gesture pose, the position information of the target gesture pose, at least one display object corresponding to the screen content in the video frame waiting to be processed, and the scene information of the scene to which the at least one display object belongs, by the method of calling a target additional waiting special effect from a special effect resource library, Based on the target gesture pose, a target additional waiting special effect can be called from a special effect resource library.

[0140] The position information of the target gesture pose may be the position information in the video frame waiting to be processed of the target gesture pose. The display object may be a target object. The scene information may be information for describing scenes such as a holiday scene in a display interface, a scene with a wide viewing angle, etc.

[0141] Exemplarily, based on the target gesture pose, the position information of the target gesture pose, at least one display target corresponding to the screen content in the video frame waiting to be processed, and the scene information of the scene to which the at least one display target belongs, one special effect that meets various needs can be called from the special effect resource library as the target additional waiting special effect.

[0142] Note that in order to determine the target additional waiting special effect based on the target gesture pose hereinafter, the correspondence relationship between various gesture poses and the additional waiting special effects can be stored in advance.

[0143] Exemplarily, when the target gesture pose is to spread the fingers, the position information of the target gesture pose corresponds to the display target A in the display interface, and the scene information to which the display target A belongs is a wide scene, based on the target gesture pose, the special effects of "adding large wings" and "adding small wings" can be determined from the special effect resource library, and the special effect of "adding large wings" can be determined based on the scene information. Furthermore, based on at least one of the position information of the target gesture pose and the display target in the display interface, the target special effect addition target is determined, and the special effect of the large wings added to the target special effect addition target is used as the target additional waiting special effect.

[0144] Exemplarily, at least one display target in the display interface is a pedestrian photographed by the camera head, the target gesture pose is to put the hand into the screen and make a finger heart, when the position information of the target gesture pose is on the pedestrian's body and the scene information is a normal scene, a special effect of "a heart mark appears" is called from the special effect resource library, and the special effect of "a heart mark appears" can be displayed on the pedestrian. At least one display target in the display interface is a hand photographed by the camera head, the target gesture pose is a gesture of imitating Spider-Man and shooting spider silk, the position information of the target gesture pose is position G on the display screen, that is, the position of the hand, when the scene information is a street scene, a special effect of "spider silk special effect" is called from the special effect resource library, and the "spider silk special effect" can be emitted from position G.

[0145] Note that when the special effect operation information is a gesture special effect operation, the camera head used when shooting the video may be a front camera head or a rear camera head.

[0146] In S440, a target additional waiting special effect is fusion-processed into the processing-waiting video frame to determine the target special effect video frame.

[0147] In S450, based on each target special effect video frame, the target special effect video is determined.

[0148] The technical solution of the embodiments of the present disclosure is as follows: during the video shooting process, obtain special effect operation information. When the special effect operation information is a gesture special effect operation, detect the gesture special effect operation on the display interface, determine the target gesture pose, and based on the target gesture pose, call the target pending special effect from the special effect resource library, thereby determining the target pending special effect through the gesture special effect operation, performing a fusion process on the target pending special effect on the video frame waiting to be processed, determining the target special effect video frame, and determining the target special effect video based on each target special effect video frame, so as to solve the problem that the flexibility of adding special effects to the video is small and the interestingness is low, improve the interaction flexibility of adding special effects to the video, and achieve the technical effect of enhancing the interestingness of shooting the video.

[0149] FIG. 5 is a flowchart of a method for determining a special effect video when the special effect operation information according to the embodiments of the present disclosure includes an audio special effect operation and a touch control special effect operation. Based on the above technical solution, when the special effect operation information includes an audio special effect operation and a touch control special effect operation, the method for determining the target pending special effect can refer to the detailed description of this technical solution. Among them, the interpretation of the same or corresponding terms as the above technical solutions will not be repeated here.

[0150] As shown in FIG. 5, the method includes the following.

[0151] In S510, during the video shooting process, obtain special effect operation information.

[0152] In S520, when the special effect operation information includes an audio special effect operation and a touch control special effect operation, determine the pending special effect from the special effect resource library based on the audio data stream corresponding to the audio special effect operation.

[0153] The additional pending special effect may also be a special effect corresponding to the audio data stream determined from the special effect resource library.

[0154] Exemplarily, when the special effect operation information includes a voice special effect operation and a touch control special effect operation, an audio data stream of the voice special effect operation is acquired, the audio data stream is processed, and an additional pending special effect is determined based on the processing result.

[0155] Note that processing the audio data stream may be to perform voice character conversion processing and perform processing such as word decomposition, removal of meaningless words, and part-of-speech analysis on the characters.

[0156] In S530, based on the screen content corresponding to the touch point in the display interface of the touch control special effect operation, the additional pending special effect is processed to determine the target additional pending special effect.

[0157] The screen content includes at least one visual element. The visual element includes at least one of a target object element, an environmental element, and an object element.

[0158] Exemplarily, based on the touch point in the display interface of the touch control special effect operation, the screen content corresponding to the touch point is determined, the visual element in the screen content is determined, and based on the visual element, the additional pending special effect after processing is processed so as to match the current visual element, and the additional pending special effect after processing is set as the target additional pending special effect.

[0159] Exemplarily, the visual elements included in the screen content are empty and grassland, the audio data stream corresponding to the voice special effect operation is "Place some birds here", and it can be determined that the additional waiting special effect is "bird". When the visual element in the screen content corresponding to the touch point is "empty", the additional waiting special effect of "bird" is processed, and the additional waiting special effect can be processed so as to become the target additional waiting special effect of "a flock of flying birds". When the visual element in the screen content corresponding to the touch point is "grassland", the additional waiting special effect of "bird" is processed, and the additional waiting special effect can be processed so as to become the target additional waiting special effect of "a bird standing and jumping".

[0160] Note that when the special effect operation information is a voice special effect operation and a touch control special effect operation, usually, the user describes the content related to the special effect by the voice special effect operation, determines information such as the position and size where the special effect is added according to the touch control special effect operation, and adds the special effect required by the user more precisely and accurately.

[0161] In S540, the target additional waiting special effect is fused with the video frame waiting for processing to determine the target special effect video frame.

[0162] In S550, based on each target special effect video frame, the target special effect video is determined.

[0163] The technical solution of the embodiments of the present disclosure is as follows: during the video shooting process, special effect operation information is obtained. When the special effect operation information includes voice special effect operation and touch control special effect operation, based on the audio data stream corresponding to the voice special effect operation, an additional pending special effect is determined from the special effect resource library, and the additional pending special effect is processed based on the screen content corresponding to the touch point in the display interface of the touch control special effect operation, and a target additional pending special effect is determined. Thus, the target additional pending special effect is determined by the voice special effect operation and the touch control special effect operation, and the target additional pending special effect is fused into the video frame to be processed to determine the target special effect video frame, and based on each target special effect video frame, a target special effect video is determined, thereby solving the problem that the flexibility of adding special effects to the video is small and the interestingness is low, improving the interaction flexibility of adding special effects to the video, and achieving the technical effect of enhancing the interestingness of shooting the video.

[0164] FIG. 6 is a flowchart of a method for determining a special effect video when the special effect operation information according to the embodiments of the present disclosure includes a voice special effect operation and a gesture special effect operation. Based on the above technical solution, when the special effect operation information includes a voice special effect operation and a gesture special effect operation, the method for determining the target additional pending special effect can refer to the detailed description of the present technical solution. Among them, the interpretation of the same or corresponding terms as the above technical solutions will not be repeated here.

[0165] As shown in FIG. 6, the method includes the following.

[0166] In S610, during the video shooting process, special effect operation information is obtained.

[0167] In S620, when the special effect operation information includes a voice special effect operation and a gesture special effect operation, an additional pending special effect is determined from the special effect resource library based on the audio data stream corresponding to the voice special effect operation.

[0168] Exemplarily, when the special effect operation information includes a voice special effect operation and a gesture special effect operation, an audio data stream of the voice special effect operation is acquired, the audio data stream is processed, and an additional pending special effect is determined from the special effect resource library based on the processing result.

[0169] In S630, based on the gesture position information corresponding to the gesture special effect operation and the screen content, a target addition position of the additional pending special effect is determined, the additional pending special effect is processed based on the target addition position, and a target additional pending special effect is acquired.

[0170] The gesture position information may be a position indicated by a gesture operation. The target addition position may be a position of the additional pending special effect.

[0171] Exemplarily, based on the gesture special effect operation, gesture position information corresponding to the gesture special effect operation is determined, and a position in the screen content corresponding to the gesture position information can be set as the target addition position. Further, based on the surrounding scene information of the target addition position, the additional pending special effect can be processed to acquire a target additional pending special effect.

[0172] Exemplarily, when the display interface is a street block and the audio data stream corresponding to the voice special effect operation is "Place a car here", the additional pending special effect can be determined to be "car". When the gesture position information corresponding to the gesture special effect operation and the screen content is an empty space in the street block, it can be determined that the empty space is the target addition position. Further, based on the size of the empty space, the placement angle of the car and the size of the special effect can be adjusted, and the adjusted additional pending special effect can be set as the target additional pending special effect. When the gesture position information corresponding to the screen content and the gesture special effect operation is a wall in the street block, it can be determined that the wall is the target addition position. Further, based on the size of the wall, the car can be adjusted to be a graffiti-style hand-drawn car, the placement angle and the size of the special effect can be adjusted, and the adjusted additional pending special effect can be set as the target additional pending special effect.

[0173] In addition, when the special effect operation information is voice special effect operation and gesture special effect operation, usually, the user describes the content related to the special effect through the voice special effect operation, determines information such as the position and size where the special effect is added according to the gesture special effect operation, and adds the special effect required by the user more precisely and accurately.

[0174] In S640, the target additional waiting special effect is fused with the processing-waiting video frame to determine the target special effect video frame.

[0175] In S650, based on each target special effect video frame, the target special effect video is determined.

[0176] The technical solution method of the embodiment of the present disclosure is to obtain special effect operation information during the video shooting process. When the special effect operation information includes voice special effect operation and gesture special effect operation, based on the audio data stream corresponding to the voice special effect operation, the additional waiting special effect is determined from the special effect resource library, and based on the gesture position information and the screen content corresponding to the gesture special effect operation, the target additional position of the additional waiting special effect is determined. Based on the target additional position, the additional waiting special effect is processed to obtain the target additional waiting special effect. Thereby, the target additional waiting special effect is determined by the voice special effect operation and the gesture special effect operation, the target additional waiting special effect is fused with the processing-waiting video frame to determine the target special effect video frame, and based on each target special effect video frame, the target special effect video is determined, so as to solve the problem that the flexibility of adding special effects to the video is small and the interestingness is low, improve the interaction flexibility of adding special effects to the video, and achieve the technical effect of enhancing the interestingness of shooting the video.

[0177] FIG. 7 is a structural schematic diagram of an apparatus for determining a special effect video according to an embodiment of the present disclosure. For example, as shown in FIG. 7, the apparatus includes a special effect operation information acquisition module 710, a target additional waiting special effect determination module 720, a target special effect video frame determination module 730, and a target special effect video determination module 740.

[0178] Among these modules, the special effect operation information acquisition module 710 is configured to acquire special effect operation information including at least one of voice special effect operation, touch control special effect operation, and gesture special effect operation during the video shooting process. The target additional waiting special effect determination module 720 is configured to call a target additional waiting special effect corresponding to the special effect operation information from a special effect resource library. The target special effect video frame determination module 730 is configured to perform a fusion process on each of a plurality of video frames waiting for processing with the target additional waiting special effect to determine a plurality of target special effect video frames. The target special effect video determination module 740 is configured to determine a target special effect video based on the plurality of target special effect video frames.

[0179] Preferably, the apparatus includes a method for detecting that a video shooting widget is triggered, a method for detecting that a captured screen includes a target object, a method for detecting that facial information matches pre-set facial information, a method for detecting that voice information triggers a video shooting command, a method for detecting that a limb movement of a target object in a captured screen matches a limb movement corresponding to pre-set limb movement information, and further includes a trigger determination module configured to determine that video shooting is triggered by at least one of the methods.

[0180] Preferably, the apparatus The target special effect operation mode determination module is further included, which is configured to display at least one waiting-for-selection special effect operation mode on the display interface, set the triggered waiting-for-selection special effect operation mode as the target special effect operation mode, and obtain corresponding special effect operation information based on the target special effect operation mode.

[0181] Preferably, the special effect operation information is voice special effect operation, and the target additional waiting-for-selection special effect determination module 720 is configured to obtain an audio data stream corresponding to the voice special effect operation, and based on the audio data stream, call a target additional waiting-for-selection special effect from the special effect resource library in such a way that the target additional waiting-for-selection special effect corresponding to the voice special effect operation is called from the special effect resource library.

[0182] Preferably, the target additional waiting-for-selection special effect determination module 720 is configured to call the target additional waiting-for-selection special effect from the special effect resource library based on the screen content in the video frame waiting for processing and the audio data stream, that is, based on the audio data stream, call the target additional waiting-for-selection special effect from the special effect resource library.

[0183] Preferably, the target additional waiting-for-selection special effect determination module 720 is configured to call the target additional waiting-for-selection special effect from the special effect resource library based on at least one keyword corresponding to the audio data stream, the target information of the target object in the screen content, and the current position information, that is, based on the screen content and the audio data stream in the video frame waiting for processing, call the target additional waiting-for-selection special effect from the special effect resource library. The target information includes target basic information, and the current position information includes the scene position to which the target object belongs and / or the current geographical position of the target object.

[0184] Preferably, the target additional waiting special effect determination module 720 further When at least one keyword in the audio data stream includes target name information and the screen content includes at least one target object, a target special effect additional target corresponding to the target name information is determined from the at least one target object, and based on the at least one keyword and the target special effect additional target, the target additional waiting special effect is called from the special effect resource library. In this way, based on the screen content and the audio data stream in the video frame waiting for processing, it is configured to call the target additional waiting special effect from the special effect resource library.

[0185] Preferably, the target additional waiting special effect determination module 720 Based on the screen content and the special effect operation information in the video frame waiting for processing, by the method of determining the target additional waiting special effect from the special effect resource library, it is configured to call the target additional waiting special effect corresponding to the special effect operation information from the special effect resource library.

[0186] Preferably, the special effect operation information is a touch control special effect operation, and the target additional waiting special effect determination module 720 Based on the trigger operation corresponding to the touch control special effect operation on the display interface, by the method of obtaining the screen content in the video frame waiting for processing, it calls the target additional waiting special effect corresponding to the touch control special effect operation from the special effect resource library, and based on the visual elements in the screen content, it is configured to determine the target additional waiting special effect.

[0187] Preferably, the target additional waiting special effect determination module 720 Based on the visual elements and the position information corresponding to the touch points in the screen content, by the method of determining the target additional waiting special effect, it is configured to determine the target additional waiting special effect based on the visual elements in the screen content.

[0188] Preferably, the special effect operation information is a gesture special effect operation, and the target addition waiting special effect determination module 720 is configured to detect a gesture special effect operation on the display interface, determine a target gesture pose, and call a target addition waiting special effect from the special effect resource library based on the target gesture pose, in such a way as to call a target addition waiting special effect corresponding to the gesture special effect operation from the special effect resource library.

[0189] Preferably, the target addition waiting special effect determination module 720 is configured to call the target addition waiting special effect from the special effect resource library based on the target gesture pose, in such a way as to call the target addition waiting special effect from the special effect resource library based on at least one of the target gesture pose, position information of the target gesture pose, at least one display object corresponding to the screen content in the video frame waiting for processing, and scene information of the scene to which the at least one display object belongs.

[0190] Preferably, the special effect operation information includes an audio special effect operation and a touch control special effect operation, and the target addition waiting special effect determination module 720 is configured to determine an addition waiting special effect from the special effect resource library based on the audio data stream corresponding to the audio special effect operation, process the addition waiting special effect based on the screen content corresponding to the touch point on the display interface of the touch control special effect operation, and determine a target addition waiting special effect, in such a way as to call a target addition waiting special effect corresponding to the audio special effect operation and the touch control special effect operation from the special effect resource library. The screen content includes at least one visual element, and the visual element includes at least one of a target object element, an environmental element, and an object element.

[0191] Preferably, the special effect operation information includes voice special effect operations and gesture special effect operations, and the target additional waiting special effect determination module 720 Based on the audio data stream corresponding to the voice special effect operation, determine the additional waiting special effect from the special effect resource library, and based on the gesture position information and the screen content corresponding to the gesture special effect operation, determine the target additional position of the additional waiting special effect, and process the additional waiting special effect based on the target additional position to obtain the target additional waiting special effect. In this way, the target additional waiting special effects corresponding to the voice special effect operation and the gesture special effect operation are configured to be called from the special effect resource library.

[0192] Preferably, the target special effect video frame determination module 730 Determine the target display position in the processing waiting video frame of the target additional waiting special effect, fuse the target additional waiting special effect at the target display position, and obtain the target special effect video frame. In this way, the target additional waiting special effect is fused and processed in the processing waiting video frame to determine the target special effect video frame.

[0193] Preferably, the target additional waiting special effect includes dynamic special effects and / or static special effects.

[0194] Preferably, the device Further includes a related special effect display module configured to display the related special effect corresponding to the target additional waiting special effect when it detects that a dynamic special effect is triggered.

[0195] The technical solution of the embodiments of the present disclosure is as follows: in the video shooting process, obtain special effect operation information, call the target additional waiting special effect corresponding to the special effect operation information from the special effect resource library, identify the special effect that the user desires to add, perform a fusion process on the target additional waiting special effect for the video frame to be processed, determine the target special effect video frame, and determine the target special effect video based on a plurality of target special effect video frames, so as to superimpose special effects on the captured video, solve the problem that the flexibility of adding special effects to the video is small and the interestingness is low, improve the interaction flexibility of adding special effects to the video, and achieve the technical effect of enhancing the interestingness of shooting the video.

[0196] The apparatus for determining a special effect video according to an embodiment of the present disclosure can execute the method for determining a special effect video according to any embodiment of the present disclosure, and achieve the functional modules and beneficial effects corresponding to the execution of the method.

[0197] It should be noted that the multiple units and modules included in the apparatus are merely divided according to functional logic, but are not limited to the above-divided units and modules, as long as the corresponding functions can be realized. Also, the names of the functional units are only for facilitating the distinction between each other.

[0198] FIG. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure. Hereinafter, with reference to FIG. 8, a schematic structural diagram of an electronic device 800 (for example, a terminal device or a server in FIG. 8) suitable for realizing the embodiment of the present disclosure is shown. The terminal device in the embodiment of the present disclosure may include mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), in-vehicle terminals (for example, in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. The electronic device shown in FIG. 8 is merely an example.

[0199] As shown in FIG. 8, the electronic device 800 may include a processing device 801 (for example, a central processing unit, a graphic processor, etc.). The processing device 801 can execute various appropriate operations and processes based on a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 into a random access memory (RAM) 803. The RAM 803 further stores various programs and data necessary for the operation of the electronic device 800. The processing device 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0200] Typically, the I / O interface 805 can be connected to an input device 806 including, for example, a touch panel, a touch pad, a keyboard, a mouse, a camera head, a microphone, an accelerometer, a gyroscope, etc., an output device 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc., a storage device 808 including, for example, a magnetic tape, a hard disk, etc., and a communication device 809. The communication device 809 enables the electronic device 800 to communicate with other devices wirelessly or wiredly to exchange data. Although the electronic device 800 having various devices is shown in FIG. 8, it should be understood that it is not required to implement or include all the shown devices. Instead, more or fewer devices may be implemented or included.

[0201] In one embodiment, according to an embodiment of the present disclosure, the process described with reference to the above flowchart can be realized as a computer software program. For example, an embodiment of the present disclosure includes a computer program product including a computer program carried on a non-transitory computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network by the communication device 809, or may be installed from the storage device 808, or may be installed from the ROM 802. When the computer program is executed by the processing device 801, the above functions limited by the method of the embodiment of the present disclosure are executed.

[0202] The electronic device according to an embodiment of the present disclosure and the special effect video determination method according to the above embodiment belong to the same disclosure concept. Technical details not described in detail in this embodiment can refer to the above embodiment, and this embodiment includes the same beneficial effects as the above embodiment.

[0203] An embodiment of the present disclosure provides a computer storage medium storing a computer program which, when executed by a processor, implements a method for determining a special effect video according to the above embodiment.

[0204] Note that the computer-readable medium described above in the present disclosure may be a computer-readable signal medium, a computer-readable storage medium, or a combination of both. The computer-readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. Examples of computer-readable storage media may include electrical connections having one or more leads, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory (FLASH), optical fibers, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or suitable combinations of the above. In the present disclosure, the computer-readable storage medium may be a tangible medium that contains or stores a program that can be used in an instruction execution system, apparatus, or device, or in combination with an instruction execution system, apparatus, or device. In the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which computer-readable program code is carried. Such a propagated data signal can take various forms, including electromagnetic signals, optical signals, or suitable combinations of the above. The computer-readable signal medium may further be any computer-readable medium other than the computer-readable storage medium, and the computer-readable signal medium can transmit, propagate, or transmit a program used in an instruction execution system, apparatus, or device, or in combination with an instruction execution system, apparatus, or device. The program code included in the computer-readable medium can be transmitted via any suitable medium, including electric wires, optical cables, radio frequency (RF), etc., or suitable combinations of the above.

[0205] In some embodiments, the client and the server can communicate using any known or future-developed network protocol, such as the Hyper Text Transfer Protocol (HTTP), and can be interconnected with digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include Local Area Networks (LANs), Wide Area Networks (WANs), network off-networks (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), and networks known currently or developed in the future.

[0206] The computer-readable medium may be included in the electronic device or may exist alone and not be attached to the electronic device.

[0207] The computer-readable medium carries at least one program, and when the at least one program is executed by the electronic device, the electronic device acquires special effect operation information including at least one of voice special effect operations, touch control special effect operations, and gesture special effect operations during the video shooting process, calls the target additional waiting special effect corresponding to the special effect operation information from the special effect resource library, performs a fusion process on the target additional waiting special effect for each of a plurality of processing-waiting video frames, determines a plurality of target special effect video frames, and determines a target special effect video based on the plurality of target special effect video frames.

[0208] Computer program code for performing the operations of the present disclosure can be created in one or more programming languages or combinations thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and further including conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a single independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer via any type of network including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., connected via the Internet using an Internet service provider).

[0209] Flowcharts and block diagrams in the drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram can represent a module, program segment, or part of code, and the module, program segment, or part of code includes one or more executable instructions for implementing a given logic function. It should be noted that in some alternative implementations, the functions represented by the blocks may occur in an order different from the order shown in the drawings. For example, two consecutively shown blocks may actually be executed substantially in parallel by such functions, and they may be executed in the reverse order in some cases. It should be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, may be implemented by a system based on dedicated hardware for performing a given function or operation, or may be implemented by a combination of dedicated hardware and computer instructions.

[0210] The units described in the embodiments of the present disclosure may be implemented in software or in hardware. Here, the name of the unit does not limit the unit itself in some cases. For example, the first acquisition unit may be described as "a unit for acquiring at least two Internet protocol addresses".

[0211] The functions described above of the present invention may be executed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that may be used include Field Programmable Gate Array (FPGA), Application Specific Integrated Circuit (ASIC), Application Specific Standard Parts (ASSP), System on Chip (SOC), Complex Programmable Logic Device (CPLD), and the like.

[0212] In the context of the present disclosure, a machine-readable medium may be a tangible medium that contains or stores a program for use in or in conjunction with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a suitable combination of the foregoing. Examples of machine-readable storage media may include electrical connections by one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only disk (CD-ROM), optical storage device, magnetic storage device, or a suitable combination of the foregoing.

[0213] According to one or more embodiments of the present disclosure, [Example 1] is A method for determining a special effect video, the method comprising In the video shooting process, a step of obtaining special effect operation information including at least one of voice special effect operation, touch control special effect operation, and gesture special effect operation; A step of calling a target additional waiting special effect corresponding to the special effect operation information from a special effect resource library; A step of performing a fusion process on the target additional waiting special effect for each of a plurality of video frames waiting for processing to determine a plurality of target special effect video frames; A step of determining a target special effect video based on the plurality of target special effect video frames, including: Providing a method for determining a special effect video.

[0214] According to one or more embodiments of the present disclosure, [Example 2] is Preferably, the step of determining that video shooting is triggered includes A step of detecting that a video shooting widget is triggered; A step of detecting that the captured screen includes a target object; A step of detecting that the facial information matches the pre-set facial information; A step of detecting that the voice information triggers a video shooting command; A step of detecting that the limb movement of the target object in the captured screen matches the limb movement corresponding to the pre-set limb movement information, including at least one of: Providing a method for determining a special effect video.

[0215] According to one or more embodiments of the present disclosure, [Example 3] is Preferably, before the step of obtaining special effect operation information, the method includes Display at least one select-wait special effect operation mode on the display interface, set the triggered select-wait special effect operation mode as the target special effect operation mode, and further include the step of obtaining corresponding special effect operation information based on the target special effect operation mode. Provide a method for determining a special effect video.

[0216] According to one or more embodiments of the present disclosure, [Example 4] is Preferably, the special effect operation information is voice special effect operation. The step of calling the target additional wait special effect corresponding to the special effect operation information from the special effect resource library is The step of obtaining the audio data stream corresponding to the voice special effect operation, and The step of calling the target additional wait special effect from the special effect resource library based on the audio data stream, and includes Provide a method for determining a special effect video.

[0217] According to one or more embodiments of the present disclosure, [Example 5] is Preferably, the step of calling the target additional wait special effect from the special effect resource library based on the audio data stream is The step of calling the target additional wait special effect from the special effect resource library based on the screen content in the video frame waiting for processing and the audio data stream, and includes Provide a method for determining a special effect video.

[0218] According to one or more embodiments of the present disclosure, [Example 6] is Preferably, the screen content includes target information and current position information. The step of calling the target additional wait special effect from the special effect resource library based on the screen content in the video frame waiting for processing and the audio data stream is including the step of calling the target additional waiting special effect from the special effect resource library based on at least one keyword corresponding to the audio data stream, target information of a target object in the screen content, and current position information; The target information includes target basic information, and the current position information includes the scene position to which the target object belongs and / or the current geographical position of the target object. Provided is a method for determining a special effect video.

[0219] According to one or more embodiments of the present disclosure, [Example 7] is Preferably, the screen content includes target information and current position information, and based on the screen content in the video frame waiting to be processed and the audio data stream, the step of calling the target additional waiting special effect from the special effect resource library is when at least one keyword of the audio data stream includes target name information and the screen content includes at least one target object, determining a target special effect additional target corresponding to the target name information from the at least one target object; including: calling the target additional waiting special effect from the special effect resource library based on the at least one keyword and the target special effect additional target. Provided is a method for determining a special effect video.

[0220] According to one or more embodiments of the present disclosure, [Example 8] is Preferably, the step of calling the target additional waiting special effect corresponding to the special effect operation information from the special effect resource library is including the step of determining a target additional waiting special effect from the special effect resource library based on the screen content in the video frame waiting to be processed and the special effect operation information. Provided is a method for determining a special effect video.

[0221] According to one or more embodiments of the present disclosure, [Example 9] is Preferably, the special effect operation information is a touch control special effect operation. Based on the screen content in the video frame waiting to be processed and the special effect operation information, the step of determining the target additional waiting special effect from the special effect resource library is as follows: Obtaining the screen content in the video frame waiting to be processed based on a trigger operation corresponding to the touch control special effect operation on the display interface; Determining the target additional waiting special effect based on the visual elements in the screen content, and includes: A method for determining a special effect video is provided.

[0222] According to one or more embodiments of the present disclosure, [Example 10] is Preferably, the step of determining the target additional waiting special effect based on the visual elements in the screen content is Determining the target additional waiting special effect based on the visual elements in the screen content and the position information corresponding to the touch point, and includes: A method for determining a special effect video is provided.

[0223] According to one or more embodiments of the present disclosure, [Example 11] is Preferably, the special effect operation information is a gesture special effect operation. The step of calling the target additional waiting special effect corresponding to the special effect operation information from the special effect resource library is Detecting a gesture special effect operation on the display interface and determining a target gesture pose; Calling the target additional waiting special effect from the special effect resource library based on the target gesture pose, and includes: A method for determining a special effect video is provided.

[0224] According to one or more embodiments of the present disclosure, [Example 12] is Preferably, based on the target gesture pose, the step of calling a target additional waiting special effect from the special effect resource library is: including the step of calling the target additional waiting special effect from the special effect resource library based on the target gesture pose, the position information of the target gesture pose, at least one display target corresponding to the screen content in the video frame waiting to be processed, and the scene information of the scene to which the at least one display target belongs; A method for determining a special effect video is provided.

[0225] According to one or more embodiments of the present disclosure, [Example 13] is: Preferably, the special effect operation information includes a voice special effect operation and a touch control special effect operation. The step of calling a target additional waiting special effect corresponding to the special effect operation information from the special effect resource library is: determining an additional waiting special effect from the special effect resource library based on the audio data stream corresponding to the voice special effect operation; and processing the additional waiting special effect based on the screen content corresponding to the touch point in the display interface of the touch control special effect operation, and determining a target additional waiting special effect, wherein the screen content includes at least one visual element, and the visual element includes at least one of a target object element, an environmental element, and an object element. A method for determining a special effect video is provided.

[0226] According to one or more embodiments of the present disclosure, [Example 14] is: Preferably, the special effect operation information includes a voice special effect operation and a gesture special effect operation. The step of calling a target additional waiting special effect corresponding to the special effect operation information from the special effect resource library is: determining an additional waiting special effect from the special effect resource library based on the audio data stream corresponding to the voice special effect operation; and Based on the gesture position information corresponding to the gesture special effect operation and the screen content, determining the target addition position of the to-be-added special effect, and based on the target addition position, processing the to-be-added special effect to obtain the target to-be-added special effect; and the like, Provide a method for determining a special effect video.

[0227] According to one or more embodiments of the present disclosure, [Example 15] is, Preferably, the step of fusing the target to-be-added special effect with the to-be-processed video frame to determine the target special effect video frame includes: Determining the target display position of the target to-be-added special effect in the to-be-processed video frame, fusing the target to-be-added special effect at the target display position, and obtaining the target special effect video frame. Provide a method for determining a special effect video.

[0228] According to one or more embodiments of the present disclosure, [Example 16] is, Preferably, the target to-be-added special effect includes a dynamic special effect and / or a static special effect. Provide a method for determining a special effect video.

[0229] According to one or more embodiments of the present disclosure, [Example 17] is, Preferably, when it is detected that the dynamic special effect is triggered, further including the step of displaying the related special effect corresponding to the target to-be-added special effect. Provide a method for determining a special effect video.

[0230] According to one or more embodiments of the present disclosure, [Example 18] is, Preferably, the step of determining the target special effect video based on each target special effect video frame includes: According to the generation timestamps of each target special effect video frame, stitching each target special effect video frame to determine the target special effect video. Provide a method for determining a special effect video.

[0231] According to one or more embodiments of the present disclosure, [Example 19] is An apparatus for determining a special effect video, the apparatus comprising: A special effect operation information acquisition module configured to acquire special effect operation information including at least one of an audio special effect operation, a touch control special effect operation, and a gesture special effect operation during a video shooting process; A target additional waiting special effect determination module configured to call a target additional waiting special effect corresponding to the special effect operation information from a special effect resource library; A target special effect video frame determination module configured to perform a fusion process on the target additional waiting special effect for each of a plurality of processing waiting video frames to determine a plurality of target special effect video frames; A target special effect video determination module configured to determine a target special effect video based on the plurality of target special effect video frames. Provide an apparatus for determining a special effect video.

[0232] Also, although the operations are described in a specific order, it should not be understood that these operations need to be performed in the specific order or the forward order shown. In certain environments, multitasking and parallel processing may be advantageous.

Claims

1. A method for determining a special effect video, the method comprising: In the video shooting process, obtaining special effect operation information including at least one of voice special effect operation, touch control special effect operation, and gesture special effect operation; Calling, from a special effect resource library, a target additional waiting special effect corresponding to the special effect operation information; Performing a fusion process on the target additional waiting special effect for each of a plurality of processing waiting video frames to determine a plurality of target special effect video frames; Determining a target special effect video based on the plurality of target special effect video frames. The method.

2. The method further comprises: Before shooting the video, determining that the shooting of the video is triggered, The step of determining that the shooting of the video is triggered includes: Detecting that a video shooting widget is triggered; Detecting that the captured screen includes a target object; Detecting that the facial information matches the preset facial information; Detecting that the voice information triggers a video shooting command; Detecting that the limb movement of the target object in the captured screen matches the limb movement corresponding to the preset limb movement information, including at least one of the above steps. The method according to claim 1.

3. Before the step of obtaining special effect operation information, the method further comprises: Displaying at least one selectable waiting special effect operation mode on a display interface, setting the triggered selectable waiting special effect operation mode as a target special effect operation mode, and obtaining corresponding special effect operation information based on the target special effect operation mode. The method according to claim 1.

4. The special effect operation information is a voice special effect operation. The step of calling, from a special effect resource library, a target additional waiting special effect corresponding to the special effect operation information includes: Obtaining an audio data stream corresponding to the voice special effect operation; Calling a target additional waiting special effect from the special effect resource library based on the audio data stream. The method according to claim 1.

5. Based on the audio data stream, the step of calling a target additional waiting special effect from the special effect resource library is including the step of calling the target additional waiting special effect from the special effect resource library based on the screen content in the video frame waiting for processing and the audio data stream. The method according to claim 4.

6. The screen content includes target information and current position information. Based on the screen content in the video frame waiting for processing and the audio data stream, the step of calling the target additional waiting special effect from the special effect resource library is including the step of calling the target additional waiting special effect from the special effect resource library based on at least one keyword corresponding to the audio data stream, the target information of the target object in the screen content, and the current position information. The target information includes target basic information, and the current position information includes at least one of the scene position to which the target object belongs and the current geographical position of the target object. The method according to claim 5.

7. The screen content includes target information and current position information. Based on the screen content in the video frame waiting for processing and the audio data stream, the step of calling the target additional waiting special effect from the special effect resource library is in response to at least one keyword of the audio data stream including target name information and the screen content including at least one target object, determining a target special effect additional target corresponding to the target name information from the at least one target object; including the step of calling the target additional waiting special effect from the special effect resource library based on the at least one keyword and the target special effect additional target. The method according to claim 5.

8. The step of calling a target additional waiting special effect corresponding to the special effect operation information from the special effect resource library is including the step of determining a target additional waiting special effect from the special effect resource library based on the screen content in the plurality of video frames waiting for processing and the special effect operation information. The method according to claim 1.

9. The special effect operation information is a touch control special effect operation. The step of determining a target additional waiting special effect from the special effect resource library based on the screen content in the plurality of video frames waiting for processing and the special effect operation information is as follows: Based on a trigger operation corresponding to the touch control special effect operation on the display interface, the step of obtaining the screen content in the video frame waiting for processing; Based on the visual elements in the screen content, the step of determining the target additional waiting special effect, including: The method according to claim 8.

10. Based on the visual elements in the screen content, the step of determining the target additional waiting special effect is as follows: Based on the visual elements in the screen content and the position information corresponding to the touch points, the step of determining the target additional waiting special effect is included. The method according to claim 9.

11. The special effect operation information is a gesture special effect operation. The step of calling a target additional waiting special effect corresponding to the special effect operation information from the special effect resource library is as follows: Detecting a gesture special effect operation on the display interface and determining a target gesture pose; Based on the target gesture pose, the step of calling a target additional waiting special effect from the special effect resource library, including: The method according to claim 1.

12. Based on the target gesture pose, the step of calling a target additional waiting special effect from the special effect resource library is as follows: Based on the target gesture pose, the position information of the target gesture pose, at least one display object corresponding to the screen content in the video frame waiting for processing, and the scene information of the scene to which the at least one display object belongs, the step of calling the target additional waiting special effect from the special effect resource library is included. The method according to claim 11.

13. The special effect operation information includes an audio special effect operation and a touch control special effect operation. The step of calling a target additional waiting special effect corresponding to the special effect operation information from the special effect resource library is as follows: Based on the audio data stream corresponding to the audio special effect operation, the step of determining an additional waiting special effect from the special effect resource library; processing the additional pending special effect and determining a target additional pending special effect based on the screen content corresponding to the touch point in the display interface of the touch control special effect operation; the screen content includes at least one visual element, and the visual element includes at least one of a target object element, an environmental element, and an object element; The method according to claim 1.

14. the special effect operation information includes a voice special effect operation and a gesture special effect operation; The step of calling a target additional pending special effect corresponding to the special effect operation information from a special effect resource library includes: determining an additional pending special effect from the special effect resource library based on an audio data stream corresponding to the voice special effect operation; determining a target additional position of the additional pending special effect based on gesture position information and screen content corresponding to the gesture special effect operation, and processing the additional pending special effect based on the target additional position to obtain the target additional pending special effect; The method according to claim 1.

15. The step of performing a fusion process on the target additional pending special effect on a video frame to be processed to determine a target special effect video frame includes: determining a target display position of the target additional pending special effect in the video frame to be processed, fusing the target additional pending special effect at the target display position, and obtaining the target special effect video frame; The method according to claim 1.

16. the target additional pending special effect includes at least one of a dynamic special effect and a static special effect; The method according to claim 1.

17. further including the step of displaying a related special effect corresponding to the target additional pending special effect when it is detected that a dynamic special effect is triggered; The method according to claim 16.

18. An apparatus for determining a special effect video, the apparatus comprising: a special effect operation information acquisition module configured to acquire special effect operation information including at least one of a voice special effect operation, a touch control special effect operation, and a gesture special effect operation during a video shooting process; a target additional pending special effect determination module configured to call a target additional pending special effect corresponding to the special effect operation information from a special effect resource library; A target special effect video frame determination module configured to perform a fusion process of the target additional waiting special effect on each of a plurality of processing-waiting video frames to determine a plurality of target special effect video frames; A target special effect video determination module configured to determine a target special effect video based on the plurality of target special effect video frames; and An apparatus.

19. An electronic device, comprising: One or more processors; and A storage device configured to store one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 17. An electronic device.

20. A computer-executable instruction including a method according to any one of claims 1 to 17, which is used when executed by a processor of a computer. A storage medium.

Images