Method, apparatus, computer device and storage medium for generating special effects

Abstract

The present disclosure provides a method, an apparatus, a computer device, and a storage medium for generating special effects, including the steps of: obtaining a target dynamic content selected by a user and control action information set for the target dynamic content, where the control action information is used to control the playback state of the target dynamic content; determining an association relationship between the target dynamic content and the first reference object on a special effect editing page where a first reference object is displayed; and generating a special effect for the target dynamic content based on the association relationship, the target dynamic content, and the control action information.

Description

[Technical Field]

[0001] This application claims priority from Chinese Patent Application No. 2023101388210, filed on February 10, 2023, the entire contents of which are hereby incorporated by reference.

[0002] SUMMARY OF THE INVENTION The disclosed embodiments relate to a method, apparatus, computer device, and storage medium for generating special effects. [Background technology]

[0003] With the development of short video technology, more and more users are using short videos to obtain information or share their lives. When users shoot short videos, they need to use special effects to make the videos more beautiful. In practical applications, various special effects may have different expression forms, and most of the video playback effect depends on the selection of special effects, which poses a serious challenge to the production of special effects. Summary of the Invention [Means for solving the problem]

[0004] The disclosed embodiments provide at least a method, an apparatus, a computer device, and a storage medium for generating special effects.

[0005] According to a first aspect, embodiments of the present disclosure provide a method for generating special effects, the method comprising: acquiring a target dynamic content selected by a user and control action information set for the target dynamic content, the control action information being used to control the playback state of the target dynamic content; determining an association relationship between the target dynamic content and the first reference object on a special effect editing page where the first reference object is displayed; generating a special effect for the target dynamic content based on the association relationship, the target dynamic content, and the control action information.

[0006] In one possible embodiment, the step of obtaining the user-selected target dynamic content comprises: The method includes receiving targeted dynamic content uploaded by a user in response to a first trigger operation for the target type.

[0007] In one possible embodiment, the first reference object of the special effect editing page is obtained by photographing a target entity object in a real scene in real time, or is a pre-set virtual object.

[0008] In one possible embodiment, the step of determining an association relationship between the target dynamic content and the first reference object on a special effect editing page where the first reference object is displayed includes: determining a set target display area of ​​the target dynamic content on the special effect editing page; determining an association relationship between the target dynamic content and the first reference object based on an association manner that matches a target type to which the target dynamic content belongs and the target display area;

[0009] In one possible embodiment, when the target type is a first type, determining an association relationship between the target dynamic content and the first reference object based on an association manner that matches the target type to which the target dynamic content belongs and the target display area includes: According to the target display area of ​​the target dynamic content on the special effect editing page, determine first position information of a first key point of the target dynamic content, and determine second position information of a second key point of the first reference object on the special effect editing page; determining an association relationship between the target dynamic content and the first reference object based on the first location information and the second location information.

[0010] In one possible embodiment, the step of determining first position information of a first key point of the target dynamic content based on the target display area of ​​the target dynamic content on the special effect editing page includes: creating a dynamic graph layer corresponding to the target dynamic content; determining first position information of a first key point of the target dynamic content in the dynamic graph layer according to a target display area of ​​the target dynamic content in the special effect editing page; determining second position information of second key points and / or key areas of the first reference object on the special effect editing page; creating a two-dimensional keypoint graph layer corresponding to a first reference object of the special effect edit page; determining second position information of the second keypoints and / or key regions in the two-dimensional keypoint graph layer.

[0011] In one possible embodiment, when the target type is a second type, determining an association relationship between the target dynamic content and the first reference object based on an association manner that matches the target type to which the target dynamic content belongs and the target display area includes: creating a three-dimensional grid graph layer corresponding to the first reference object; determining a target grid associated with the target dynamic content based on a target display area of ​​the target dynamic content on the special effect editing page; determining an association relationship between the target dynamic content and the first reference object based on the target grid.

[0012] In one possible embodiment, the step of determining a target grid associated with the target dynamic content based on a target display area of ​​the target dynamic content on the special effect editing page includes: projecting a three-dimensional grid corresponding to the first reference object onto a two-dimensional plane; determining a target two-dimensional grid corresponding to the target display area based on a target display area of ​​the target dynamic content on the special effect editing page and a two-dimensional grid obtained by projecting the three-dimensional grid onto the two-dimensional plane; determining a three-dimensional grid corresponding to the target two-dimensional grid as a target grid associated with the target dynamic content.

[0013] In one possible embodiment, the method comprises: obtaining the special effect; acquiring a video feed and determining a second referent in the video feed; Displaying the target dynamic content with the special effect based on the position or area information of the second reference object; The method further includes adjusting a playback state of the target dynamic content in response to detecting a control action on the video screen that matches control action information included in the special effect.

[0014] In one possible embodiment, the control action comprises one or more of an appear action, a disappear action, a pause action; when the control action includes the appearing action and the pause action, a response time to the pause action is later than a response time to the appearing action; When the control actions include the disappear action and the pause action, the response time to the pause action is earlier than the response time to the disappear action.

[0015] In one possible embodiment, when the control action includes a pause action, the step of adjusting the playback state of the target dynamic content in response to detecting a control action on the video screen that matches control action information included in the special effect includes: determining a currently displayed image frame of the target dynamic content in response to detecting a pause action in the video screen that matches control action information included in the special effect; stopping the playback of the target dynamic content and displaying the image frame at a current display position of the target dynamic content.

[0016] According to a second aspect, the presently disclosed embodiments further provide an apparatus for generating special effects, the apparatus comprising: an acquisition module for acquiring a target dynamic content selected by a user and control action information set for the target dynamic content, the control action information being used to control the playback state of the target dynamic content; a determining module for determining an association relationship between the target dynamic content and the first reference object on a special effect editing page where the first reference object is displayed; a generating module for generating a special effect for the target dynamic content based on the association relationship, the target dynamic content, and the control action information.

[0017] According to a third aspect, an embodiment of the present disclosure further provides a computing device, the computing device including a processor, a memory, and a bus, the memory storing machine-readable instructions executable by the processor, the computing device communicating between the processor and the memory via the bus when the computing device is running, the machine-readable instructions, when executed by the processor, performing the steps of the first aspect above or any possible embodiment of the first aspect.

[0018] According to a fourth aspect, the presently disclosed embodiments further provide a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the first aspect or any possible embodiment of the first aspect.

[0019] In order to make the above objects, features, and advantages of the present disclosure clear and easily understandable, preferred embodiments are shown and described in detail below with reference to the accompanying drawings.

[0020] In order to more clearly explain the technical solutions of the embodiments of the present disclosure, the following briefly introduces drawings used in the embodiments, which are incorporated in the specification and form part of this specification, and which illustrate embodiments consistent with the present disclosure and are used to explain the technical solutions of the present disclosure together with the specification. It is understood that the following drawings only illustrate certain embodiments of the present disclosure, and therefore should not be considered as limiting the scope, and those skilled in the art can further obtain other drawings based on these drawings without any creative work. [Brief explanation of the drawings]

[0021] [Figure 1]FIG. 1 shows a flowchart of a method for generating special effects according to an embodiment of the present disclosure. [Figure 2] FIG. 2 shows a schematic diagram of a special effect editing page in the special effect generating method according to the disclosed embodiment. [Figure 3] FIG. 3 shows a flowchart of a special effect display method according to an embodiment of the present disclosure. [Figure 4] FIG. 4 shows a structural schematic diagram of a special effect generating device according to an embodiment of the present disclosure. [Figure 5] FIG. 5 shows a structural diagram of a computer device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0022] In order to clarify the objectives, technical solutions, and advantages of the disclosed embodiments, the technical solutions of the disclosed embodiments will be clearly and completely described below with reference to the drawings of the disclosed embodiments. It is obvious that the described embodiments are only some embodiments of the disclosed embodiments, not all embodiments. Generally, the assemblies of the disclosed embodiments described and shown in the drawings herein may be arranged and designed in various different configurations. Therefore, the following detailed description of the disclosed embodiments provided in the drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of the present invention. Based on the disclosed embodiments, all other embodiments that can be obtained by those skilled in the art without requiring creative work are also within the scope of protection of the present disclosure.

[0023] It should be noted that like reference numbers and letters represent like items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0024] The term "and / or" used herein is merely for explaining the association relationship and indicates that there are three types of relationships, for example, A and / or B refers to three situations: A exists alone, A and B exist simultaneously, and B exists alone. Also, the term "at least one" used herein means any combination of any one of a plurality of types or at least two of a plurality of types, for example, including at least one of A, B, and C refers to including any one or more elements selected from the group consisting of A, B, and C.

[0025] It is to be understood that, before using the technical solutions disclosed in each embodiment of the present disclosure, the type, scope of use, use scenarios, etc. of personal information related to the present disclosure should be properly notified to users in accordance with relevant laws and regulations, and their consent should be obtained.

[0026] For example, in response to receiving a user's active request, prompt information may be sent to the user to clearly prompt the user that the operation it requests to perform requires the acquisition and use of the user's personal information, so that the user can autonomously choose whether to provide personal information to software or hardware, such as an electronic device, an application program, a server, or a storage medium, that performs the operation of the technical solution of the present disclosure based on the prompt information.

[0027] As an optional, non-limiting example, the method of sending prompt information to the user in response to receiving the user's active request may be, for example, a pop-up window method, in which the prompt information may be presented in text form, and the pop-up window may also include a selection control for the user to select "Agree" or "Disagree" to provide personal information to the electronic device.

[0028] It should be understood that the above notification and user consent obtaining process is merely exemplary and does not limit the implementation manner of the present disclosure, and other methods that comply with relevant laws and regulations may also be applied to the implementation manner of the present disclosure.

[0029] To facilitate understanding of this embodiment, the special effect generation method disclosed in the presently disclosed embodiment will first be described in detail. The execution body of the special effect generation method according to the presently disclosed embodiment is generally a terminal device, which may be, for example, a smartphone, a tablet computer, a smart wearable device, etc.

[0030] FIG. 1 is a flowchart of a method for generating special effects according to an embodiment of the present disclosure, which includes steps 101 to 103.

[0031] Step 101: Obtaining a target dynamic content selected by a user and control action information set for the target dynamic content, and the control action information is used to control the playback state of the target dynamic content.

[0032] Step 102: on a special effect editing page where a first reference object is displayed, determine an association relationship between the target dynamic content and the first reference object.

[0033] Step 103: generating a special effect for the target dynamic content based on the association relationship, the target dynamic content, and the control action information.

[0034] Below is a detailed description of the above steps.

[0035] For step 101, The target dynamic content is dynamic content that needs to be added to the special effect to be generated, i.e., sticker content that needs to be displayed when using the final generated special effect, and the target dynamic content may be, for example, a dynamic image (such as a gif image), animation, video, etc.

[0036] In one possible embodiment, when a user obtains selected target dynamic content, the target dynamic content being uploaded by the user can be received in response to a first trigger operation for the target type.

[0037] Here, the target dynamic content being uploaded by the user may be target dynamic content uploaded locally from the client, or may be video content captured in real time by calling the client's image collection device.

[0038] For example, multiple types can be displayed on the special effect editing entry page, and different types correspond to different display effects. A user can trigger different types to enter the corresponding special effect editing page.

[0039] Optionally, the target type may include a first type for performing 2D display and a second sticker type for performing 3D display. The first type sticker may be a simple image superposition when displaying, while the second type sticker may be a 3D display using a 3D projection method when displaying. Specific display realization methods will be described below, so explanations will be omitted here.

[0040] In another possible embodiment, when the user obtains the target dynamic content selected by the user, a plurality of selectable dynamic contents are preset, and in response to the user's selection operation on any of the dynamic content, the dynamic content corresponding to the selection operation is set as the target dynamic content selected by the user, and the target dynamic content selected by the user is obtained.

[0041] Since the sticker content added to the generated special effect is dynamic content, control action information for the target dynamic content can also be added to enhance the display effect of the special effect, and the control action information is used to control the playback state of the target dynamic content, for example, to pause the playback of the target dynamic content or to start the playback of the target dynamic content.

[0042] Specifically, the control action information may be action information that needs to be detected in the process of the user using the special effect after the special effect is generated, such as smiling, blinking, waving, etc.

[0043] In one possible scenario, the user may select multiple target dynamic contents, and the control action information set for different target dynamic contents may be the same or different. In another possible scenario, the user may select static content, and control action information may be set corresponding to the static content, and the control action information of the static content allows the user to control the display or disappearance of the static content, and the control action information of the static content may be the same or different from the control action information of the target dynamic content.

[0044] In practical applications, multiple selectable control action information can be provided for any dynamic content or static content, and the user can set corresponding control action information for each dynamic content or static content from the multiple selectable control action information.

[0045] In other possible embodiments, the control action information may be empty, in which case, after a special effect for the target dynamic content is generated, the target dynamic content may be played and displayed continuously when the special effect is applied.

[0046] In another possible embodiment, the control action information may further include a random pause, and the random pause may include two possible situations: on the one hand, when the special effect is applied, the target dynamic content is paused / played at a random time, for example, the target dynamic content pauses playback after playing for N seconds, where N is a random number; on the other hand, when the special effect is applied, the control action to control the target dynamic content is random, i.e., when the special effect is applied, one control action is randomly determined.

[0047] For step 102, The special effect editing page is a page for performing special effect editing, and in one possible embodiment, the special effect editing page may display a first reference object, and the first reference object is obtained by photographing a target entity object in a real scene in real time, or is a pre-set virtual object.

[0048] The preset virtual object may be, for example, a two-dimensional virtual object or a three-dimensional virtual object, and the present invention is not limited thereto.

[0049] Here, the purpose of displaying the first reference object on the special effect editing page is to allow the user to make real-time adjustments based on the displayed target dynamic content during the process of creating the special effect, such as adjusting the position and size.

[0050] Furthermore, when the first virtual object is a preset virtual object, in order to display the target dynamic content more realistically, a collected video image can be further acquired, and position information of a second reference object in the video image can be determined. In this way, after the association relationship is determined based on step 102, a preview result can be displayed that displays the special effect finally generated based on the position information of the second reference object and the association relationship.

[0051] Because different target types correspond to different display schemes, different target types can be matched to different association schemes and thereby displayed in different dimensions.

[0052] In one possible embodiment, when determining the association relationship between the target dynamic content and the first reference object on a special effect editing page where a first reference object is displayed, first, a set target display area of ​​the target dynamic content is determined on the special effect editing page, and then, the association relationship between the target dynamic content and the first reference object is determined based on the association method and the target display area that match the target type to which the target dynamic content belongs.

[0053] The target type to which the target dynamic content belongs may be a target type that is determined when a user selects the target dynamic content, or in another possible embodiment, the target type to which the target dynamic content belongs may be a target type that matches the target dynamic content after identifying the target dynamic content.

[0054] Exemplarily, when identifying the target dynamic content and matching a target type to the target dynamic content, a correspondence between the dynamic content type and each target type is pre-established, and then the target type to which the target dynamic content belongs is determined based on the correspondence and the dynamic content type of the target dynamic content.

[0055] For example, the target type may be a sticker type, a sticker type corresponding to a dynamic image may be the first type, a sticker type corresponding to an animation may be the second type, and when the target dynamic content is an animation, the target type may be the second type, and when the target dynamic content is a gif image, the target type may be the first type.

[0056] Specifically, when determining the association relationship between the target dynamic content and the first referent, the following two situations may be classified based on the target type.

[0057] Situation 1: the target type is the first type for two-dimensional display.

[0058] In this situation, the association manner that matches the target type to which the target dynamic content belongs is the association between the first keypoint of the target dynamic content and the second keypoint of the first referent.

[0059] When determining the association relationship between the target dynamic content and the first reference object based on the association method matching the target type to which the target dynamic content belongs and the target display area, it can be performed, for example, by steps a1 and a2.

[0060] Step a1: based on the target display area of ​​the target dynamic content on the special effects editing page, determine first position information of a first key point of the target dynamic content, and determine second position information of a second key point of the first reference object on the special effects editing page.

[0061] Step a2: determining an association relationship between the target dynamic content and the first reference object based on the first location information and the second location information;

[0062] The first key point of the target dynamic content may refer to a position key point of the target dynamic content, for example, may refer to the center point of the target dynamic content, and the first key points of different target dynamic contents may be the same, for example, the same first key point refers to the type of the first key point being the same, not the first position information of the first key point being the same.

[0063] When determining the first position information of the first key point of the target dynamic content, for example, a dynamic graph layer corresponding to the target dynamic content can be first created, and the size of the dynamic graph layer is the same as the size of the area for displaying the dynamic content of the special effects editing page; and then, the first position information of the first key point of the target dynamic content can be determined in the dynamic graph layer based on the target display area of ​​the target dynamic content on the special effects editing page.

[0064] The second key point of the first reference object on the special effect editing page may be preset, for example, when the first reference object is a human face, the second key point may be the center of an eyebrow. In the process of special effect editing performed by different users, the corresponding second key points may be the same.

[0065] In one possible embodiment, when determining the second position information of the second key point of the first reference object, a two-dimensional key point graph layer corresponding to the first reference object of the special effect editing page can be created, and then the second position information of the second key point can be determined in the two-dimensional key point graph layer.

[0066] In another possible embodiment, when the target type is the first type, the association manner matching the target type to which the target dynamic content belongs may be an association between a first key point of the target dynamic content and a key area of ​​the first reference object. The key area may refer to a reference area when displaying a special effect, such as an area where a hand detection frame is located, an area where an eye detection frame is located, etc.

[0067] In one possible embodiment, the user can select the association method on the special effect editing page, i.e., whether the first position information of the first key point of the target dynamic content is associated with the second position information of the second key point, or whether the first position information of the first key point of the target dynamic content is associated with the second position information of the key region.

[0068] For example, after selecting a target dynamic content, as shown in FIG. 2, on the special effect editing page, when the user adjusts the display position of the target dynamic content, association guidelines can be displayed simultaneously. For example, a first guideline connecting the first key point and the second key point is displayed, and a second guideline connecting the first key point and the key area is displayed. The user can select the corresponding association method by triggering the guidelines.

[0069] It should be noted that the step of selecting an association method here refers to the selection when the target type is the first type, and the association methods are essentially the same type.

[0070] In one possible embodiment, the association relationship between the target dynamic content and the first reference object can be characterized by a relative positional relationship, and when determining the association relationship between the target dynamic content and the first reference object based on the first position information and the second position information, the relative positional relationship between the target dynamic content and the first reference object can be directly determined based on the first position information and the second position information.

[0071] Situation 2: the target type is the second type for three-dimensional display.

[0072] In this situation, the association manner that matches the target type to which the target dynamic content belongs is that the target dynamic content is associated with the target region of the first reference object.

[0073] In one possible embodiment, when determining the association relationship between the target dynamic content and the first reference object based on the association method matching the target type to which the target dynamic content belongs and the target display area, it can be performed, for example, by steps b1 to b3.

[0074] Step b1: creating a three-dimensional grid graph layer corresponding to the first reference object;

[0075] Step b2: determining a target grid associated with the target dynamic content according to a target display area of ​​the target dynamic content on the special effect editing page;

[0076] Step b3: determining an association relationship between the target dynamic content and the first reference object based on the target grid.

[0077] Specifically, when determining a target grid associated with the target dynamic content based on the target display area of ​​the target dynamic content on the special effects editing page, for example, first project a three-dimensional grid corresponding to the first reference object in the three-dimensional grid graph layer onto a two-dimensional plane, and then determine a target two-dimensional grid corresponding to the target display area based on the target display area of ​​the target dynamic content on the special effects editing page and the two-dimensional grid projected onto the two-dimensional plane, and then determine the three-dimensional grid corresponding to the target two-dimensional grid as the target grid associated with the target dynamic content.

[0078] When determining the association relationship between the target dynamic content and the first reference object based on the target grid, for example, the target dynamic content and the target grid can be directly bound, and the target grid bound to the target dynamic content is, that is, the association relationship between the target dynamic content and the first reference object.

[0079] Here, since the target dynamic content is a sticker type for three-dimensional display, when the target dynamic content is bound to the target grid, the target dynamic content can be displayed in combination with the visibility / invisibility of the part area corresponding to the target grid when the target dynamic content is displayed, and further, three-dimensional display of the target dynamic content can be realized.

[0080] For step 103, In one possible embodiment, when generating a special effect for the target dynamic content based on the association relationship, the target dynamic content, and the control action information, for example, the association relationship, the target dynamic content, and the control action information can be packaged to generate a special effect for the target dynamic content.

[0081] Optionally, other related information such as the display size of the target dynamic content can also be set on the special effect editing page, and further, the association relationship, the target dynamic content, and the control action information can be packaged, and when generating special effects for the target dynamic content, other related information such as the display size of the target dynamic content can be synchronously packaged, thereby generating special effects for the target dynamic content.

[0082] The method, apparatus, computer device, and storage medium for generating special effects provided by the disclosed embodiments provide users with multiple choices, allowing them to freely select different types of target dynamic content and freely generate special effects to control the target dynamic content by setting the display position and control action information of the target dynamic content, thereby enriching the sources of special effects and the interaction forms of special effect production, reducing the difficulty of special effect production, and further improving the quantity and quality of special effect production.

[0083] Corresponding to the method for generating special effects described in FIG. 1, the present disclosure further provides a method for displaying special effects, and FIG. 3 shows a flowchart of the method for displaying special effects according to the present disclosure, which method includes steps 301 to 304.

[0084] Step 301: obtain a special effect, which is generated by the method for generating a special effect described in the above embodiment.

[0085] Step 302: Collect a video image and determine a second referent in the video image.

[0086] Step 303: displaying the target dynamic content with the special effect according to the position or region information of the second reference object.

[0087] Step 304: Adjusting the playback state of the target dynamic content in response to detecting a control action on the video screen that matches the control action information included in the special effect.

[0088] Specifically, when the target dynamic content is displayed with the special effect based on the position or area information of the second reference object, the display position of the target dynamic content is first determined based on the position or area information of the second reference object and the association relationship between the target dynamic content included in the special effect and the specified reference object, and the target dynamic content is then displayed at the determined display position.

[0089] When the target dynamic content is displayed, if the target types to which the target dynamic content belongs are different, the target dynamic content can be displayed in different display manners.

[0090] For example, when the target type is the first type, after determining the display position of the target dynamic content on the video screen, the target dynamic content can be directly displayed at the display position.

[0091] When the target type is the second type, the position information of the target grid associated with the target dynamic content can be determined based on the association relationship and the position or area information of the second reference object, and then the target dynamic content can be mapped to the target grid as a texture based on the UV mapping corresponding to the second reference object, and further a three-dimensional display of the target dynamic content can be realized.

[0092] In one possible embodiment, after collecting a video screen or displaying the target dynamic content, prompt information including the control action information is displayed, thereby prompting the user to perform a control action corresponding to the control action information and control the target dynamic content.

[0093] In one possible embodiment, the control action includes an appearance action, and before displaying the target dynamic content with the special effect based on the position or area information of the second referent, the control action may further detect whether the appearance action appears on the video screen, and if so, display the target dynamic content.

[0094] For example, if the appearance action is a blink, after collecting the video screen, it is detected whether the second reference object in the video screen performs a blink action (whether it is blinking), and if it is detected that the second reference object performs a blink action, the target dynamic content can be displayed according to the above step 303.

[0095] In another possible embodiment, the control action may further include a disappearance action, and when the disappearance action is detected in the collected video screen, the display of the target dynamic content in the video screen can be canceled.

[0096] For example, if the disappearing action is waving a hand, after displaying the target dynamic content on the video screen, it is possible to detect whether a second referent in the video screen performs a waving action, and if it is detected that the second referent performs the waving action, it is possible to cancel displaying the target dynamic content on the video screen.

[0097] In another possible embodiment, the control action may further include a pause action, and after detecting the pause action, playback of the target dynamic content can be controlled to be paused.

[0098] Specifically, in response to detecting a pause action in the video screen that matches the control action information included in the special effect, the currently displayed image frame of the target dynamic content is determined, and then the playback of the target dynamic content is stopped and the image frame is displayed at the current display position of the target dynamic content.

[0099] The target dynamic content can be understood as a plurality of consecutive image frames, and the playback of the target dynamic content can be understood as consecutive image frames played in order. After detecting a pause action, the currently displayed image frame of the target dynamic content can be directly determined and the image frame can be directly displayed. After pausing the playback of the target dynamic content, the displayed image frame can be understood as static content.

[0100] The control action may further include a start action, which, after stopping playback of the target dynamic content, continues to play the target dynamic content based on the currently displayed image frame in response to detecting a start action in the video screen that matches the control action information included in the special effect.

[0101] Alternatively, in another possible embodiment, after detecting a pause action and pausing the target dynamic content, the target dynamic content can continue to be played directly based on the currently displayed image frame when the pause time length reaches the target time length.

[0102] The target duration may be a predetermined fixed length, or may be a duration calculated based on the full playback duration of the target dynamic content, for example, the ratio between the target duration and the full playback duration may be a predetermined ratio.

[0103] For example, when the full playback time length of the target dynamic content is 10 seconds and the preset ratio is 1 / 2, the target time length may be 5 seconds, i.e., after pausing the playback of the target dynamic content for 5 seconds, the target dynamic content can continue to be played.

[0104] The method of automatically playing the target dynamic content after pausing can guide the user to check the complete target dynamic content and also induce the user to trigger the pause again, thereby increasing the interaction between users and improving the display effect of special effects.

[0105] Thus, the control action may include one or more of an appearing action, a disappearing action, and a pause action, and when the control action includes the appearing action and the pause action, the response time to the pause action is later than the response time to the appearing action, and when the control action includes the disappearing action and the pause action, the response time to the pause action is earlier than the response time to the disappearing action.

[0106] That is, when responding to a pause action, it is necessary to confirm that the target dynamic content is being displayed on the current video screen, i.e., that it has already previously responded to an appear action but has not yet responded to a disappear action.

[0107] The above-mentioned appearance action, disappearance action, pause action, etc. can be the same action, for example, they can all be blinking actions, and in this situation, the number of times the action is performed can be detected and a response can be made in the corresponding order.

[0108] For example, if the response order is appearance action → pause action → disappearance action, the target dynamic content can be displayed when a blink action is detected for the first time, the playback of the target dynamic content can be paused when a blink action is detected for the second time, and the display of the target dynamic content can be canceled when a blink action is detected for the third time, etc.

[0109] As can be understood by those skilled in the art, in the above methods in the embodiments for implementing the invention, the order of the steps described does not mean to limit the implementation process to a strict execution order, and the specific execution order of the steps should be determined by the functions and possible internal logic.

[0110] Based on the same inventive concept, the presently disclosed embodiment further provides a special effect generating device corresponding to the special effect generating method, and the problem-solving principle of the device in the presently disclosed embodiment is similar to the above-mentioned special effect generating method in the presently disclosed embodiment, so the implementation of the device can refer to the implementation of the method, and repeated details will not be described again.

[0111] FIG. 4 is a structural schematic diagram of an apparatus for generating special effects according to an embodiment of the present disclosure, the apparatus including: an acquisition module 401, a determination module 402, a generation module 403, a collection module 404, and a display module 405; The acquisition module 401 is used to acquire target dynamic content selected by a user and control action information set for the target dynamic content, and the control action information is used to control the playback state of the target dynamic content; The determining module 402 is used to determine an association relationship between the target dynamic content and the first reference object on the special effect editing page where the first reference object is displayed; The generating module 403 is used to generate a special effect for the target dynamic content according to the association relationship, the target dynamic content, and the control action information.

[0112] In one possible embodiment, when the acquisition module 401 acquires the target dynamic content selected by the user, It is used to receive target dynamic content uploaded by a user in response to a first trigger operation for a target type.

[0113] In one possible embodiment, the first reference object of the special effect editing page is obtained by photographing a target entity object in a real scene in real time, or is a pre-set virtual object.

[0114] In one possible embodiment, the determining module 402, when determining the association relationship between the target dynamic content and the first referent on the special effect editing page on which the first referent is displayed, determining the set target display area of ​​the target dynamic content on the special effect editing page; and determining an association relationship between the target dynamic content and the first reference object based on an association manner that matches the target type to which the target dynamic content belongs and the target display area.

[0115] In one possible embodiment, when the target type is a first type, the determination module 402 determines the association relationship between the target dynamic content and the first referent based on the association manner that matches the target type to which the target dynamic content belongs and the target display area: Determine first position information of a first key point of the target dynamic content according to the target display area of ​​the target dynamic content on the special effect editing page, and determine second position information of a second key point of the first reference object on the special effect editing page; and determining an association relationship between the target dynamic content and the first reference object based on the first location information and the second location information.

[0116] In one possible embodiment, when the determination module 402 determines the first position information of the first key point of the target dynamic content based on the target display area of ​​the target dynamic content on the special effect editing page, creating a dynamic graph layer corresponding to the target dynamic content; Determine first position information of a first key point of the target dynamic content in the dynamic graph layer according to a target display area of ​​the target dynamic content in the special effect editing page; The determining module 402, when determining the second position information of the second key point and / or key area of ​​the first reference object on the special effect editing page, creating a two-dimensional keypoint graph layer corresponding to a first reference object of the special effect edit page; and determining second position information of the second keypoints and / or key regions in the two-dimensional keypoint graph layer.

[0117] In one possible embodiment, when the target type is a second type, the determining module 402 determines the association relationship between the target dynamic content and the first referent based on the association manner that matches the target type to which the target dynamic content belongs and the target display area: creating a three-dimensional grid graph layer corresponding to the first reference object; determining a target grid associated with the target dynamic content based on a target display area of ​​the target dynamic content on the special effect editing page; and determining an association relationship between the target dynamic content and the first reference object based on the target grid.

[0118] In one possible embodiment, when the determination module 402 determines a target grid associated with the target dynamic content based on a target display area of ​​the target dynamic content on the special effect editing page, projecting a three-dimensional grid corresponding to the first reference object onto a two-dimensional plane; Determine a target two-dimensional grid corresponding to the target display area based on a target display area of ​​the target dynamic content on the special effect editing page and a two-dimensional grid obtained by projecting the three-dimensional grid onto the two-dimensional plane; and determining a three-dimensional grid corresponding to the target two-dimensional grid as a target grid associated with the target dynamic content.

[0119] In one possible embodiment, the acquisition module 401 is further used for acquiring the special effects; The apparatus further includes a collection module 404 for collecting a video image and determining a second referent in the video image; The device further includes a display module 405 for displaying the target dynamic content with the special effect according to the position or area information of the second reference object; The display module 405 is further used for adjusting the playback state of the target dynamic content in response to detecting a control action on the video screen that matches the control action information included in the special effect.

[0120] In one possible embodiment, the control action comprises one or more of an appear action, a disappear action, a pause action; when the control action includes the appearing action and the pause action, a response time to the pause action is later than a response time to the appearing action; When the control actions include the disappear action and the pause action, the response time to the pause action is earlier than the response time to the disappear action.

[0121] In one possible embodiment, when the control action includes a pause action, the display module, in response to detecting a control action on the video screen that matches control action information included in the special effect, adjusts the playback state of the target dynamic content by: determining a currently displayed image frame of the target dynamic content in response to detecting a pause action in the video screen that matches control action information included in the special effect; stopping the playback of the target dynamic content and displaying the image frame at the current display position of the target dynamic content.

[0122] For the description of the processing process of each module in the device and the interaction process between each module, please refer to the relevant description in the above method embodiment, and detailed description will be omitted here.

[0123] Based on the same technical idea, the presently disclosed embodiment further provides a computer device. Figure 5 is a structural diagram of a computer device 500 according to the presently disclosed embodiment, which includes a processor 501, a memory 502, and a bus 503. The memory 502 is used to store execution instructions, and includes a memory 5021 and an external memory 5022. The memory 5021 is also called an internal memory and is used to temporarily store calculation data in the processor 501 and data to be exchanged with the external memory 5022 such as a hard disk. The processor 501 exchanges data with the external memory 5022 through the memory 5021. When the computer device 500 is executed, the processor 501 and the memory 502 communicate with each other via the bus 503, thereby allowing the processor 501 to An instruction to acquire a target dynamic content selected by a user and control action information set for the target dynamic content, the control action information being used to control the playback state of the target dynamic content; an instruction to determine an association relationship between the target dynamic content and the first reference object on a special effect editing page where the first reference object is displayed; and an instruction to generate a special effect for the target dynamic content based on the association relationship, the target dynamic content, and the control action information.

[0124] In one possible embodiment, the instructions executed by the processor 501 for obtaining the user selected target dynamic content include: Responsive to a first trigger operation for the target type, receiving targeted dynamic content uploaded by the user.

[0125] In one possible embodiment, in the instructions executed by the processor 501, the first reference object of the special effect editing page is obtained by photographing a target entity object in a real scene in real time, or is a pre-set virtual object.

[0126] In one possible embodiment, the instructions executed by the processor 501 for determining an association relationship between the target dynamic content and the first referent on a special effect editing page on which the first referent is displayed include: determining the set target display area of ​​the target dynamic content on the special effect editing page; determining an association relationship between the target dynamic content and the first reference object based on an association manner that matches a target type to which the target dynamic content belongs and the target display area.

[0127] In one possible embodiment, in the instructions executed by the processor 501, when the target type is a first type, the instructions for determining an association relationship between the target dynamic content and the first referent based on an association scheme matching the target type to which the target dynamic content belongs and the target display area include: Determine first position information of a first key point of the target dynamic content according to the target display area of ​​the target dynamic content on the special effect editing page, and determine second position information of a second key point of the first reference object on the special effect editing page; determining an association relationship between the target dynamic content and the first reference object based on the first location information and the second location information.

[0128] In one possible embodiment, in the instructions executed by the processor 501, the instruction for determining, based on the target display area of ​​the target dynamic content on the special effect editing page, first position information of a first key point of the target dynamic content is: creating a dynamic graph layer corresponding to the target dynamic content; determining first position information of a first key point of the target dynamic content in the dynamic graph layer according to a target display area of ​​the target dynamic content in the special effect editing page; The instruction for determining second position information of a second key point and / or a key area of ​​the first reference object on the special effect editing page includes: creating a two-dimensional keypoint graph layer corresponding to a first reference object of the special effect edit page; determining second position information of the second keypoint and / or key region in the two-dimensional keypoint graph layer.

[0129] In one possible embodiment, the instructions executed by the processor 501 include, when the target type is a second type, determining an association relationship between the target dynamic content and the first referent based on an association scheme matching the target type to which the target dynamic content belongs and the target display area: creating a three-dimensional grid graph layer corresponding to the first reference object; determining a target grid associated with the target dynamic content based on a target display area of ​​the target dynamic content on the special effect editing page; determining an association relationship between the target dynamic content and the first reference object based on the target grid.

[0130] In one possible embodiment, the instructions executed by the processor 501 for determining a target grid associated with the target dynamic content based on a target display area of ​​the target dynamic content on the special effect editing page include: projecting a three-dimensional grid corresponding to the first reference object onto a two-dimensional plane; Determine a target two-dimensional grid corresponding to the target display area based on a target display area of ​​the target dynamic content on the special effect editing page and a two-dimensional grid obtained by projecting the three-dimensional grid onto the two-dimensional plane; determining a three-dimensional grid corresponding to the target two-dimensional grid as a target grid associated with the target dynamic content.

[0131] In one possible embodiment, the instructions executed by processor 501 include: obtaining the special effect; acquiring a video feed and determining a second referent in the video feed; Displaying the target dynamic content with the special effect based on the position or area information of the second reference object; The method further includes adjusting a playback state of the target dynamic content in response to detecting a control action on the video screen that matches control action information included in the special effect.

[0132] In one possible embodiment, the instructions executed by the processor 501 include: the control action includes one or more of an appear action, a disappear action, and a pause action; when the control action includes the appearing action and the pause action, a response time to the pause action is later than a response time to the appearing action; When the control actions include the disappear action and the pause action, the response time to the pause action is earlier than the response time to the disappear action.

[0133] In one possible embodiment, instructions executed by the processor 501 include, when the control action includes a pause action, instructions for adjusting a playback state of the target dynamic content in response to detecting a control action on the video screen that matches control action information included in the special effect: determining a currently displayed image frame of the target dynamic content in response to detecting a pause action in the video screen that matches control action information included in the special effect; stopping the playback of the target dynamic content and displaying the image frame at a current display position of the target dynamic content.

[0134] The disclosed embodiments further provide a computer-readable storage medium, which stores a computer program, which, when executed by a processor, performs the steps of the method for generating special effects described in the above method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0135] The disclosed embodiments further provide a computer program product, the computer product including a program code, and instructions included in the program code can be used to perform the steps of the method for generating special effects described in the above method embodiments, which can be specifically referred to in the above method embodiments and will not be described in detail herein.

[0136] The computer program product may be specifically implemented in hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically a computer storage medium, and in another alternative embodiment, the computer program product is specifically a software product such as a software development kit (SDK).

[0137] Those skilled in the art will understand that for convenience and simplicity of explanation, the specific operating processes of the systems and devices described above can refer to the corresponding processes in the method embodiments, and will not be described in detail here. It should be understood that the systems, devices, and methods presented in the various embodiments of the present disclosure may be realized in other ways. The device embodiments described above are merely examples. For example, the division of the units is merely a logical functional division, and other division methods may be used in actual implementation. For example, multiple units or assemblies may be combined or aggregated into other systems, or some features may be omitted or not implemented. Furthermore, the couplings, direct couplings, or communication connections between devices or units shown or discussed may be indirect couplings or communication connections between devices or units via some communication interfaces, which may be electrical, mechanical, or other types of communication interfaces.

[0138] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place or distributed across multiple network units. To achieve the objectives of the solution of this embodiment, some or all of the units can be selected according to actual needs.

[0139] Furthermore, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, each unit may exist physically alone, or two or more units may be integrated into one unit.

[0140] When the functions are realized as software functional units and sold or used as independent products, they may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on this understanding, the essence of the technical solutions of the present disclosure, or a portion of the technical solutions or a portion thereof, can be realized in the form of a software product, and the computer software product is stored in a storage medium and includes a plurality of instructions, thereby enabling a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present disclosure. The storage medium includes any medium capable of storing program code, such as a U disk, a mobile disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0141] It should be noted that the above examples are merely specific embodiments of the present disclosure, and are used only to illustrate the technical solutions of the present disclosure, without limiting them. Although the present disclosure has been described in detail with reference to the above examples, the protection scope of the present disclosure is not limited thereto. As can be understood by those skilled in the art, those skilled in the art can modify or easily come up with variations of the technical solutions described in the above examples, or make equivalent substitutions for some technical features, within the technical scope disclosed in the present disclosure. Such modifications, variations, or substitutions do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the examples of the present disclosure, and should all be encompassed by the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be determined by the protection scope of the claims.

Claims

1. 1. A method for the generation of special effects, the method comprising: acquiring a target dynamic content selected by a user and control action information set for the target dynamic content, the control action information being used to control the playback state of the target dynamic content; determining an association relationship between the target dynamic content and the first reference object on a special effect editing page where the first reference object is displayed; generating a special effect for the target dynamic content based on the association relationship, the target dynamic content, and the control action information; method.

2. The step of obtaining the user-selected target dynamic content includes: The method of claim 1 , further comprising receiving user-uploaded targeted dynamic content in response to a first trigger operation for the target type.

3. The method of claim 1 , wherein the first reference object of the special effect editing page is obtained by photographing a target entity object in a real scene in real time, or is a preset virtual object.

4. The step of determining an association relationship between the target dynamic content and the first reference object on a special effect editing page where the first reference object is displayed includes: determining a set target display area of ​​the target dynamic content on the special effect editing page; The method of claim 1, further comprising: determining an association relationship between the target dynamic content and the first referent based on an association scheme matching a target type to which the target dynamic content belongs and the target display area.

5. When the target type is a first type, determining an association relationship between the target dynamic content and the first reference object based on an association manner that matches the target type to which the target dynamic content belongs and the target display area, According to the target display area of ​​the target dynamic content on the special effect editing page, determine first position information of a first key point of the target dynamic content, and determine second position information of a second key point of the first reference object on the special effect editing page; and determining an association relationship between the target dynamic content and the first reference object based on the first location information and the second location information.

6. The step of determining first position information of a first key point of the target dynamic content according to the target display area of ​​the target dynamic content on the special effect editing page includes: creating a dynamic graph layer corresponding to the target dynamic content; determining first position information of a first key point of the target dynamic content in the dynamic graph layer according to a target display area of ​​the target dynamic content in the special effect editing page; determining second position information of second key points and / or key areas of the first reference object on the special effect editing page; creating a two-dimensional keypoint graph layer corresponding to a first reference object of the special effect edit page; and determining second position information of the second keypoints and / or key regions in the two-dimensional keypoint graph layer.

7. When the target type is a second type, determining an association relationship between the target dynamic content and the first reference object according to an association manner that matches the target type to which the target dynamic content belongs and the target display area, creating a three-dimensional grid graph layer corresponding to the first reference object; determining a target grid associated with the target dynamic content based on a target display area of ​​the target dynamic content on the special effect editing page; and determining an association relationship between the target dynamic content and the first referent based on the target grid.

8. determining a target grid associated with the target dynamic content based on a target display area of ​​the target dynamic content on the special effect editing page, projecting a three-dimensional grid corresponding to the first reference object onto a two-dimensional plane; determining a target two-dimensional grid corresponding to the target display area based on a target display area of ​​the target dynamic content on the special effect editing page and a two-dimensional grid obtained by projecting the three-dimensional grid onto the two-dimensional plane; determining a three-dimensional grid corresponding to the target two-dimensional grid as a target grid associated with the target dynamic content.

9. The method comprises: obtaining the special effect; acquiring a video feed and determining a second referent in the video feed; displaying the target dynamic content with the special effect according to position or area information of the second reference object; 2. The method of claim 1, further comprising: adjusting a playback state of the target dynamic content in response to detecting a control action on the video screen that matches control action information included in the special effect.

10. the control action includes one or more of an appear action, a disappear action, and a pause action; when the control action includes the appearing action and the pause action, a response time to the pause action is later than a response time to the appearing action; The method of claim 9 , wherein when the control actions include the disappear action and the pause action, the response time to the pause action is earlier than the response time to the disappear action.

11. When the control action includes a pause action, adjusting the playback state of the target dynamic content in response to detecting a control action on the video screen that matches control action information included in the special effect includes: determining a currently displayed image frame of the target dynamic content in response to detecting a pause action in the video screen that matches control action information included in the special effect; stopping playback of the target dynamic content and displaying the image frame at a current display position of the target dynamic content.

12. 1. An apparatus for producing special effects, the apparatus comprising: an acquisition module configured to acquire target dynamic content selected by a user and control action information set for the target dynamic content, wherein the control action information is configured to control a playback state of the target dynamic content; a determining module configured to determine an association relationship between the target dynamic content and the first reference object on a special effect editing page where the first reference object is displayed; a generating module configured to generate a special effect for the target dynamic content based on the association relationship, the target dynamic content, and the control action information. Device.

13. A computing device comprising a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and wherein when the computing device is running, communication is established between the processor and the memory via the bus, and wherein the machine-readable instructions, when executed by the processor, cause the computing device to perform the method of any one of claims 1 to 11.

14. A computer readable storage medium storing a computer program which, when executed by a processor, performs the method of any one of claims 1 to 11.

Images