Method and device for generating special effect video frame, electronic equipment and storage medium

Abstract

Embodiments of the present disclosure provide a special effect video frame generation method and device, electronic equipment and a storage medium. The method comprises: in response to a special effect video generation instruction, displaying at least two moving special effect elements in a current video frame; determining the motion characteristic information of the special effect elements according to whether the target object is included in the current video frame; determining the motion parameter adjustment mode based on the motion characteristic information, to determine the target motion parameter corresponding to the corresponding special effect element based on the motion parameter adjustment mode; and controlling the corresponding special effect element to move based on the target motion parameter of the special effect element, to obtain a special effect video frame. The technical solution of the embodiments of the present disclosure realizes the interaction between the target object and the special effect element, determines the motion characteristic information of the special effect element, and renders and displays the special effect element based on the motion characteristic information, thereby enhancing the richness and interest of the special effect video and improving the interactive experience between the user and the special effect prop.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the technical field of image processing, and particularly relate to a special effect video frame generation method and device, electronic equipment and a storage medium. BACKGROUND

[0002] With the development of network technology, more and more application programs have entered the life of users, especially a series of software that can shoot short videos, which are deeply loved by users.

[0003] In the prior art, software developers can add various special effect props in the application for users to use in the process of shooting videos. However, the special effect props currently provided for users are very limited, and the quality and richness of the content of the videos need to be further improved. Meanwhile, when a special effect prop includes multiple special effect elements of the same form and motion, the rendering effect of each special effect element is poor, resulting in poor special effect video effect and slow rendering speed based on the special effect prop, which affects the user experience of using the application software. SUMMARY

[0004] The present disclosure provides a special effect video frame generation method, device, electronic equipment and storage medium to determine the motion characteristic information of a special effect element based on the interaction between a target object and the special effect element, to enhance the richness and interest of the special effect video, and to improve the interactive experience between the user and the special effect prop.

[0005] In a first aspect, the embodiments of the present disclosure provide a special effect video frame generation method, which comprises:

[0006] In response to a special effect video generation instruction, displaying at least two moving special effect elements in a current video frame;

[0007] Determining motion characteristic information of the special effect elements according to whether the current video frame includes a target object;

[0008] Based on the motion characteristic information, determining a motion parameter adjustment mode to determine a target motion parameter corresponding to a corresponding special effect element based on the motion parameter adjustment mode;

[0009] Controlling the corresponding special effect element to move based on the target motion parameter of the special effect element to obtain a special effect video frame.

[0010] In a second aspect, the embodiments of the present disclosure provide a special effect video frame generation device, which comprises:

[0011] A special effect element display module configured to display at least two moving special effect elements in a current video frame in response to a special effect video generation instruction;

[0012] motion characteristic information determination module, configured to determine motion characteristic information of the special effect element according to whether the target object is included in the current video frame;

[0013] motion parameter adjustment mode determination module, configured to determine a motion parameter adjustment mode based on the motion characteristic information, so as to determine the target motion parameter corresponding to the special effect element based on the motion parameter adjustment mode;

[0014] special effect video frame determination module, configured to control the corresponding special effect element to move based on the target motion parameter of the special effect element, so as to obtain a special effect video frame.

[0015] In a third aspect, the embodiments of the present disclosure further provide an electronic device, which comprises:

[0016] one or more processors;

[0017] a storage device configured to store one or more programs,

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the method for generating a special effect video frame according to any of the embodiments of the present disclosure.

[0019] In a fourth aspect, the embodiments of the present disclosure further provide a storage medium containing computer executable instructions, which are used to execute the method for generating a special effect video frame according to any of the embodiments of the present disclosure when executed by a computer processor.

[0020] The technical solution of the embodiments of the present disclosure solves the problems in the prior art that when a special effect prop includes multiple special effect elements of the same form and motion, the rendering effect of each special effect element is poor, resulting in poor special effect video effect and slow rendering speed based on the special effect prop, and achieves the following effects: based on the interaction between a target object and a special effect element, the motion characteristic information of the special effect element is determined, the special effect element is rendered and displayed based on the motion characteristic information, the richness and interest of the special effect video are enhanced, and the interactive experience between a user and a special effect prop is improved. BRIEF DESCRIPTION OF DRAWINGS

[0021] The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings. The same or similar components have the same or similar reference labels. It should be understood that the drawings are not necessarily to scale, with emphasis instead being placed upon illustrating the principles of the embodiments of the present disclosure.

[0022] Figure 1 is a schematic diagram of a method for generating a special effect video frame according to an embodiment of the present disclosure;

[0023] Figure 2 is a schematic diagram of a method for generating a special effect video frame according to an embodiment of the present disclosure;

[0024] Figure 3 is a schematic diagram of a method for determining a target motion parameter according to an embodiment of the present disclosure;

[0025] Figure 4 is a schematic diagram of a method for generating a special effect video frame according to an embodiment of the present disclosure;

[0026] Figure 5 is a schematic diagram of a natural interaction behavior implementation process according to an embodiment of the present disclosure;

[0027] Figure 6 is a schematic diagram of a method for generating a special effect video frame according to an embodiment of the present disclosure;

[0028] Figure 7 is a schematic diagram of a method for determining rendering information according to an embodiment of the present disclosure;

[0029] Figure 8 is a schematic diagram of a method for generating a special effect video frame according to an embodiment of the present disclosure;

[0030] Figure 9 is a schematic diagram of a device for generating a special effect video frame according to an embodiment of the present disclosure;

[0031] Figure 10 is a schematic diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] Embodiments of the present disclosure will be described more fully hereinafter with reference to the accompanying drawings. While several embodiments of the present disclosure have been illustrated and described, it will be appreciated that various changes can be made therein without departing from the spirit and scope of the present disclosure. It is to be understood that the drawings and descriptions are not to be taken literally, and that elements of the drawings and descriptions are not necessarily to scale.

[0033] It should be understood that each step recited in the method embodiments of the present disclosure can be performed in different orders and / or in parallel. In addition, the method embodiments can include additional steps and / or omit performing the steps shown. The scope of the present disclosure is not limited in this respect.

[0034] The term "comprising" and variations thereof as used herein are used inclusively, i.e., "comprising, but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment." The term "another embodiment" means "at least one additional embodiment." The term "some embodiments" means "at least some embodiments." Related definitions are given below.

[0035] It should be noted that the terms "first", "second", and the like in the present disclosure are merely used to distinguish different devices, modules or units, and do not imply the order or interdependence of the functions performed by these devices, modules or units.

[0036] It should be noted that the terms "one", "multiple" in the present disclosure are illustrative and not restrictive, and those skilled in the art should understand that "one or more" should be understood unless otherwise explicitly indicated in the context.

[0037] The names of the messages or information exchanged between the devices in the embodiments of the present disclosure are only for illustrative purposes, and are not intended to limit the scope of the messages or information.

[0038] It can be understood that, before using the technical solutions disclosed in the embodiments of the present disclosure, the type, use range, use scenario, etc. of the personal information involved in the present disclosure should be informed to the user and the authorization of the user should be obtained through appropriate means according to relevant laws and regulations.

[0039] For example, in response to receiving the active request of the user, prompt information is sent to the user to explicitly prompt the user that the operation requested to be performed will require obtaining and using the personal information of the user. Thus, the user can voluntarily choose whether to provide the personal information to the software or hardware such as electronic device, application program, server or storage medium, etc. performing the operation of the technical solutions of the present disclosure according to the prompt information.

[0040] As an optional but not limited implementation manner, in response to receiving the active request of the user, the manner of sending prompt information to the user may, for example, be the manner of pop-up window, and the prompt information may, for example, be presented in the form of text in the pop-up window. In addition, the pop-up window may, for example, carry selection controls for the user to select "agree" or "disagree" to provide the personal information to the electronic device.

[0041] It can be understood that the above notification and user authorization process is only illustrative, and does not limit the implementation of the present disclosure, and other methods that meet relevant laws and regulations can also be applied to the implementation of the present disclosure.

[0042] It can be understood that the data involved in the technical solution (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of relevant laws and regulations and relevant provisions.

[0043] Before introducing the technical solution, the application scenario can be exemplarily described. The technical solution of the embodiment of the present disclosure can be applied to any scene that needs to generate a special effect video. For example, when a user uploads a recorded multimedia data stream to a corresponding server of an application, or when a mobile terminal including a camera device collects video pictures in real time, at least two moving special effect elements can be displayed in the video pictures displayed on the terminal device display interface when a special effect video generation instruction is detected. Further, the application can detect the picture content (i.e., multiple objects in the picture) of the obtained multiple video frames, determine the motion characteristic information of the special effect elements according to whether the target object is included in the video picture, determine the motion parameter adjustment mode corresponding to the motion characteristic information, adjust the motion parameters of the special effect elements based on the motion parameter adjustment mode, obtain the target motion parameters corresponding to the special effect elements, and control each special effect element to move according to the corresponding target motion parameters, thereby obtaining special effect video frames. The interaction between the target object and the special effect elements is determined, the motion characteristic information of the special effect elements is determined, the special effect elements are rendered and displayed based on the motion characteristic information, and the use experience of the user in the process of making a special effect video is improved.

[0044] Figure 1 is a flowchart of a method for generating a special effect video frame provided by an embodiment of the present disclosure. The embodiment of the present disclosure is applicable to a situation where an application software processes a current video frame to generate a special effect video, for example, displaying multiple moving special effect elements in a current video picture, determining the motion characteristic information of the special effect elements according to whether a target object is included in the current video picture, and adjusting the motion behavior state of the special effect elements to obtain a special effect video. The method can be executed by a special effect video frame generation device. The device can be implemented in the form of software and / or hardware. Optionally, the device can be implemented by an electronic device, which can be a mobile terminal, a PC terminal, or a server.

[0045] As shown in Figure 1 , the method comprises:

[0046] S110, in response to a special effect video generation instruction, displaying at least two moving special effect elements in a current video frame.

[0047] The device for executing the method for generating a special effect video provided in the embodiments of the present disclosure can be integrated in application software supporting a special effect video processing function, and the software can be installed in an electronic device. The electronic device can be a mobile terminal or a PC terminal, etc. The application software can be a kind of software for image / video processing, and specific application software will not be described here, as long as it can realize image / video processing. It can also be a specially developed application program to realize the software for adding and displaying special effects, or be integrated in a corresponding page, and the user can realize the processing of the special effect video through the integrated page in the PC terminal.

[0048] In the embodiments, in the application software or application program supporting the special effect video processing function, a control for generating a special effect video can be developed in advance, or a preset body action for generating a special effect video is preset. When it is detected that the user triggers the control, or when it is detected that the body action information of the user is consistent with the preset body action, it is determined that a special effect video generation instruction is generated, and then a response is made, so as to display at least two moving special effect elements in the current video frame.

[0049] It should be noted that the technical solution of the embodiments can be executed in the process of the user shooting a video, that is, a special effect video is generated in real time according to the special effect prop selected by the user and the video shot, or a video uploaded by the user can be taken as original data, and then a special effect video is generated based on the solution of the embodiments.

[0050] In the embodiments, the user can shoot a video in real time based on the camera of the mobile terminal, or actively upload a video based on the control developed in advance in the application, and further, the video is analyzed based on the pre-written program, so as to obtain a plurality of video frames. At this time, the video frame displayed in the display interface can be taken as a current video frame. In actual application, a video shooting control can be developed in advance in the application, and when it is detected that the user triggers the control, the terminal device can be directed to the user in real time to realize the collection of the video, and the video frame collected and displayed in the display interface of the terminal device can be taken as a current video frame. Further, when it is detected that the user triggers a special effect video generation operation, at least two moving special effect elements can be displayed in the current video frame.

[0051] In the embodiments, the special effect element can be a special effect object developed in advance and integrated in the application, and the special effect object can be added in the image. Alternatively, the special effect element can be at least one of fire, explosion, smoke, water flow, spark, falling leaves, cloud, fog, snow, dust, meteor and phototactic organism. For example, the special effect element can be a firefly in the phototactic organism, and when the special effect video generation instruction is received, a response is made, and at least two moving fireflies are displayed in the current video frame.

[0052] It should be noted that the number of special effect elements displayed in the current video frame can be two or more. The advantage of this setting is that the special effect elements displayed in the current video frame not only have individual motion behaviors, but also have cluster motion behaviors, thereby increasing the diversity of the special effect elements.

[0053] In actual application, the respective motion parameters of the special effect elements can be set in the development stage. Further, when the special effect video generation instruction is detected, the special effect elements displayed in the current video frame can move according to the pre-set motion parameters.

[0054] Optionally, the at least two moving special effect elements displayed in the current video frame include: controlling the special effect elements to move in the current video frame according to initial motion parameters.

[0055] In the embodiment, the initial motion parameters can be pre-set parameters for controlling the corresponding special effect elements to move. The initial motion parameters can include various parameters, and optionally include initial positions, initial speeds, initial orientations, target orientations and / or target speeds of the special effect elements. The initial position can be the position of the special effect element in the current video frame when the special effect element does not move, or can be understood as the default position of the special effect element when the special effect element does not move, which is set in the development stage. The initial speed can be the movement speed of the special effect element when the special effect element starts to move, or can be understood as the default movement speed of the special effect element, which is set in the development stage. For example, the initial speed can be set as the maximum movement speed. The initial orientation can be the movement direction of the special effect element when the special effect element starts to move. The target orientation can be the final orientation of the special effect element after adjustment when the special effect element moves. The target speed can be the final speed of the special effect element after adjustment when the special effect element moves.

[0056] In actual application, when it is detected that the user triggers the special effect video generation instruction, a response can be performed, and then at least two special effect elements are displayed in the current video frame, and initial motion parameters of the special effect elements are determined respectively to control the special effect elements to move in the current video frame according to the corresponding initial motion parameters. The advantage of this setting is that the initial motion state of the special effect elements is controlled, and the simulation of the motion behavior of the special effect elements is realized, thereby improving the interest of the special effect video.

[0057] S120, determining motion feature information of the special effect elements according to whether the target object is included in the current video frame.

[0058] In the embodiment, the target object can be a target part or a target pose. The target part can be any one or more parts on the trunk of the object's limbs. Alternatively, the target part can be a hand, an arm, or a leg, etc. The target pose can be a facial pose or a limb pose of the object. Alternatively, the facial pose can be blinking or opening the mouth, etc. The limb pose can be turning the head, raising the arm, or opening the palm, etc. The object can be a user or an animal, etc. The motion characteristic information can be information for representing the motion of the special effect element in the current video frame. Alternatively, the motion characteristic information can include individual motion information or cluster motion information, etc. The individual motion information can be information for representing the individual motion of the special effect element. For example, if the special effect element is a glowworm in a phototactic organism, the individual motion information can be the random Brownian motion of the individual glowworm. The cluster motion information can be information for representing the cluster motion of the special effect element. For example, if the special effect element is a glowworm in a phototactic organism, the cluster motion information can be the dispersion or cohesion motion of the cluster of glowworms.

[0059] In actual applications, when the special effect video generation instruction is detected and a response is made, at least two special effect elements are displayed in the current video frame, and each special effect element is controlled to move in the current video frame according to the initial motion parameter corresponding to the special effect element. Further, it is determined whether the target object is included in the current video frame, and according to the determination result, the motion characteristic information of the special effect element in the current video frame is determined.

[0060] It should be noted that when it is determined whether the target object is included in the current video frame, the current video frame can be processed. For example, one or more target parts or one or more target poses can be pre-set in the development stage as the target object. When it is detected that the target part or the target pose is included in the current video frame, it is determined that the target object is included in the current video frame.

[0061] It should be further noted that the number of target objects in the same video frame can be one or more. Whether it is one or more, the technical solution provided in the embodiment of the present disclosure can be used to determine the special effect video frame.

[0062] In S130, a motion parameter adjustment manner is determined based on the motion characteristic information, so as to determine the target motion parameter corresponding to the corresponding special effect element based on the motion parameter adjustment manner.

[0063] In the embodiment, the motion parameter adjustment manner can be a manner of adjusting the motion parameters of the special effect elements to determine the corresponding target motion parameters. It should be noted that for different motion characteristic information, the corresponding motion parameter adjustment manner is also different. For example, if the motion characteristic information is individual motion information, the corresponding motion parameter adjustment manner can be a manner of adjusting the motion parameters of each special effect element as an independent individual to determine the target motion parameters corresponding to each special effect element. The target motion parameter can be a final motion parameter corresponding to the special effect element when the special effect element moves in the current video frame. Optionally, the target motion parameter can include a speed, a direction, and a position, etc. The speed can be a final speed of the special effect element when the special effect element moves; the direction can be a final motion direction of the special effect element; and the position can be a display position of the special effect element in each current video frame when the special effect element moves.

[0064] In actual application, after the motion characteristic information of the special effect element displayed in the current video frame is determined, the motion parameter adjustment manner of adjusting the motion parameters of the special effect element can be determined according to the motion characteristic information, so that the target motion parameter corresponding to the special effect element can be determined according to the motion parameter adjustment manner.

[0065] S140, controlling the corresponding special effect element to move based on the target motion parameter of the special effect element to obtain a special effect video frame.

[0066] In the embodiment, after the target motion parameters corresponding to the special effect elements are determined, the motion process of the corresponding special effect element can be adjusted according to the target motion parameters of the special effect elements, so that each characteristic element moves according to the corresponding target motion parameter, and thus the special effect video frame in which each special effect element moves according to a certain rule in the current video frame can be obtained.

[0067] It should be noted that if the current video frame includes the target object, the special effect video frame includes the target object and the special effect element; and if the current video frame does not include the target object, the special effect video frame only includes the special effect element.

[0068] The technical scheme of the embodiment of the present disclosure is that, in response to a special effect video generation instruction, at least two moving special effect elements are displayed in a current video frame, then, according to whether a target object is included in the current video frame, motion characteristic information of the special effect elements is determined, further, based on the motion characteristic information, a motion parameter adjustment mode is determined, so as to determine target motion parameters corresponding to the respective special effect elements based on the motion parameter adjustment mode, finally, the respective special effect elements are controlled to move based on the target motion parameters of the special effect elements, so as to obtain a special effect video frame, which solves the problems in the prior art that when a special effect prop includes multiple special effect elements of the same form and motion, the rendering effect of each special effect element is poor, which leads to poor special effect video effect and slow rendering speed based on the special effect prop, and achieves the interaction between a target object and a special effect element, determines the motion characteristic information of the special effect element, and renders and displays the special effect element based on the motion characteristic information, so as to enhance the richness and interest of the special effect video and improve the interactive experience between a user and a special effect prop.

[0069] Figure 2 is a flowchart of a method for generating a special effect video frame provided by the embodiment of the present disclosure, and on the basis of the foregoing embodiment, if the current video frame does not include a target object, the motion characteristic information of the special effect element is first motion information, and the target motion parameters are determined based on the motion parameter adjustment mode corresponding to the individual motion information. The specific implementation can be referred to the technical scheme of the present embodiment. The same or corresponding technical terms as the foregoing embodiment are not described herein.

[0070] As shown in Figure 2 , the method specifically includes the following steps:

[0071] S210, in response to a special effect video generation instruction, at least two moving special effect elements are displayed in a current video frame.

[0072] S220, when the current video frame does not include a target object, the motion characteristic information of the special effect element is determined as first motion information.

[0073] In this embodiment, after the at least two special effect elements are controlled to move in the current video frame with the initial motion parameters, it can be detected whether the target object is included in the current video frame, and if it is detected that the target object is not included in the current video frame, the motion feature information of the special effect elements can be determined as the first motion information. The first motion information can be individual motion information, that is, information corresponding to the motion of the special effect elements as independent individuals, that is, the motion feature information between the special effect elements can not be associated. It should be noted that the individual motion information can correspond to the random behavior in the fish school algorithm, that is, the individual fish usually randomly swims in the water, and under the influence of other behaviors, presents a free and disordered swimming state.

[0074] In S230, if the motion feature information is the first motion information, the motion parameter adjustment mode is an individual motion parameter adjustment mode, and the target motion parameter corresponding to the special effect element is determined based on the motion parameter adjustment mode.

[0075] The individual motion parameter adjustment mode is a mode of determining the target motion parameter of each special effect element as an independent individual.

[0076] In this embodiment, after it is determined that the motion feature information of the special effect elements is the individual motion information, the motion parameter adjustment mode is determined as the individual motion parameter adjustment mode of determining the target motion parameter of each special effect element as an independent individual, and then the target motion parameter corresponding to the special effect element can be determined based on the individual motion parameter adjustment mode.

[0077] It should be noted that when the motion parameter adjustment mode is the individual motion parameter adjustment mode, the determination mode of the target motion parameter corresponding to each special effect element displayed in the current video frame is the same, and the determination mode of the target motion parameter can be described by taking any one of the special effect elements as an example.

[0078] Optionally, the target motion parameter corresponding to the special effect element is determined based on the motion parameter adjustment mode, including: for each special effect element, determining the superposition displacement corresponding to the current special effect element in the current video frame; determining the target position of the current special effect element in the current video frame based on the superposition displacement and the position information of the current special effect element in the previous video frame of the current video frame; and determining the target motion parameter of the current special effect element based on the target position and the pre-set region constraint condition.

[0079] The target motion parameter includes a target speed and a target motion direction.

[0080] In the embodiment, the superimposed displacement can be used to represent the position change of the special effect element in the current video frame, that is, the directed line segment from the initial position to the target position of the special effect element in the current video frame. In actual application, the superimposed displacement can be determined according to the initial orientation and the initial speed of the special effect element. Specifically, the product of the initial orientation and the initial speed of the special effect element is the superimposed displacement. The position information of the current special effect element in the previous video frame of the current video frame can be understood as the target position of the current special effect element in the previous video frame of the current video frame. The target position is the display position of the current special effect element in the video frame. The region constraint condition can be pre-set, which is a condition for constraining the movement range of the special effect element in the current video frame. In actual application, a certain spatial range can be pre-set for each special effect element to control the random movement of each special effect element within the spatial range. The spatial range can be determined according to the camera view frustum range and the cube bounding box range. The cube bounding box range includes the camera view frustum range. Both the individual motion information and the cluster motion information are generated and moved within the cube bounding box range and are rendered within the camera view frustum range. That is, only the objects within the camera view frustum range can be rendered to reduce the rendering performance pressure caused by a large number of special effect element instances. Specifically, first, the spatial coordinates of each special effect element in the world space coordinate system are determined based on the cube bounding box as the boundary. For each special effect element, it is detected whether the coordinate values of the spatial coordinates of the current special effect element are within the boundary of the cube bounding box, that is, whether the coordinate values of the three dimensions of xyz are within the [min, max] interval of the cube bounding box. Second, the spatial coordinates are converted to the camera coordinates in the camera coordinate system based on the transformation matrix to determine whether the camera coordinates are within the camera view frustum range. If yes, the object should be rendered. If not, the object is not rendered.

[0081] It should be noted that the advantage of setting the region constraint condition is that when the technical solution is applied to a device with limited computing power, the rendering effect and rendering performance can be combined, and the rendering performance pressure can be relieved to a certain extent.

[0082] In actual application, for each special effect element, the superimposed displacement of the current special effect element in the current video frame can be determined according to the initial orientation and the initial speed of the current special effect element.

[0083] For example, the superimposed displacement of the current special effect element in the current video frame can be determined based on the following formula:

[0084] pos_add = direct * speed

[0085] pos_add can represent the superimposed displacement, direct can represent the initial orientation of the current special effect element, speed can represent the initial speed of the current special effect element, and * represents multiplication.

[0086] Further, the position information of the current special effect element in the previous video frame of the current video frame is determined, and then the position information is added to the superimposed displacement to obtain the target position of the current special effect element in the current video frame.

[0087] For example, the target position of the current special effect element in the current video frame can be determined based on the following formula:

[0088] aim_pos = pos_last_frame + pos_add

[0089] Wherein, aim_pos can represent the target position, and pos_last_frame can represent the position information of the current special effect element in the previous video frame.

[0090] Finally, after obtaining the target position, the target position can be compared with the pre-set region constraint condition to determine whether the target position meets the region constraint condition, so as to determine the target motion parameter of the current special effect element in the current video frame based on the comparison result. The advantage of such setting is that the individual motion behavior simulation effect of the special effect element is realized, and by setting the region constraint condition, the pressure on the rendering performance of the terminal device is reduced under the premise of limited computing power of the terminal device, and the rendering effect of the terminal device on the special effect element is improved.

[0091] In actual application, the target motion parameter of the special effect element in the current video frame can be determined by judging whether the target position of the special effect element meets the region constraint condition. The determination process of the target motion parameter when the target position meets the region constraint condition and the determination process of the target motion parameter when the target position does not meet the region constraint condition will be described below.

[0092] Optionally, based on the target position and the pre-set region constraint condition, the target motion parameter of the current special effect element is determined, including: if the target position meets the region constraint condition, updating the target position of the current special effect element, and adjusting the target motion speed and the target motion direction of the current special effect element based on the pre-set adjustment step.

[0093] In this embodiment, the preset adjustment step can be a pre-set adjustment parameter, and the parameter is a value repeatedly superimposed in each operation process. Optionally, the preset adjustment step can include a speed adjustment step and a direction adjustment step. The speed adjustment step can be an adjustment parameter corresponding to the motion rate of the special effect element. For example, the speed adjustment step can be a random number with a value range of (-0.05, 0.05). The direction adjustment step can be an adjustment parameter corresponding to the motion direction of the special effect element. For example, the direction adjustment step can be a 0.1-degree rotation in the direction from the initial orientation to the target orientation. Correspondingly, the target motion rate can be the final motion rate adjusted based on the speed adjustment step. For example, if the speed adjustment step is a random number in (-0.05, 0.05), the target motion rate is the sum of the initial speed and the random number. The target motion direction can be the final motion direction adjusted based on the direction adjustment step. For example, if the direction adjustment step is a 0.1-degree rotation in the direction from the initial orientation to the target orientation, the target motion direction is the motion direction corresponding to the target orientation after a 0.1-degree rotation.

[0094] It should be noted that whether the target position meets the region constraint condition can be determined by determining whether the distance between the target position and the region boundary set in the region constraint condition is less than a preset distance threshold. For example, taking the region constraint condition as a cube bounding box and the target position of the current special effect element as the origin, a ray is constructed with the target motion direction as the vector, the ray intersects the cube bounding box, and an intersection point intersect_point is obtained. The distance d between the target position and intersect_point is calculated. If d is less than the preset distance threshold, it can be determined that the current special effect element is close to the edge of the cube bounding box, i.e., the target position does not meet the region constraint condition. On the contrary, if d is not less than the preset distance threshold, it can be determined that there is a certain distance between the current special effect element and the edge of the cube bounding box, i.e., the target position meets the region constraint condition.

[0095] In actual application, if the target position meets the region constraint condition, the initial position of the current special effect element in the current video frame can be updated to the target position, and the target motion rate of the current special effect element is determined according to the speed adjustment step, and the target motion direction of the current special effect element is determined according to the direction adjustment step, on the premise that the target motion rate does not exceed the maximum motion rate. The advantage of this setting is that the individual motion behavior simulation effect of the special effect element is realized, and by setting the region constraint condition, the pressure on the rendering performance of the terminal device is reduced under the premise of limited computing power of the terminal device, and the rendering effect of the terminal device on the special effect element is improved.

[0096] For example, the above process can be explained based on the following formula:

[0097] Update the target position: position = aim_pos;

[0098] Determine the target motion speed: speed ′ = speed + random(-0.05, 0.05), wherein speed ′ represents the target motion speed, speed represents the initial speed, and random(-0.05, 0.05) represents the speed adjustment step.

[0099] Optionally, based on the target position and the preset region constraint condition, the target motion parameter of the current special effect element is determined, including: if the target position does not satisfy the region constraint condition, the target motion speed and the target motion direction of the current special effect element are adjusted based on a second function.

[0100] In this embodiment, the second function can be preset, which is a function for adjusting the motion parameter of the special effect element that does not satisfy the region constraint condition.

[0101] In actual application, if the target position does not satisfy the region constraint condition, that is, the target position of the current special effect element is close to the region boundary set in the region constraint condition, at this time, the target motion speed and the target motion direction of the current special effect element can be determined based on the second function, so as to determine the target position of the current special effect element based on the target motion speed and the target motion direction. The advantage of this setting is that the individual motion behavior simulation effect of the special effect element is realized, and by setting the region constraint condition, the pressure on the rendering performance of the terminal device is reduced under the premise that the computing power of the terminal device is limited, and the rendering effect of the terminal device on the special effect element is improved.

[0102] It should be noted that when the current special effect element is close to the region boundary set in the region constraint condition, a huge reverse speed can be given to the current special effect element to make the subsequent motion of the current special effect element move in the opposite direction. The motion of the current special effect element can be reversed by superimposing a huge reverse speed on the current motion speed of the current special effect element. When the current special effect element moves in the opposite direction, the distance d between the new intersect_point and the target position of the current special effect element will be greater than the set threshold value, at this time, the reverse speed is cancelled.

[0103] Illustratively, adjusting the target motion speed and the target motion direction based on the second function can be setting the target motion direction as the direction opposite to the initial direction, that is, direct ′ = direct, wherein direct ′Indicates the target movement direction, direct indicates the initial direction; set the target movement speed to half of the maximum movement speed, that is, speed ′ =max_speed / 2, where speed ′ Indicates the target movement rate, and max_speed indicates the maximum movement rate.

[0104] Furthermore, after determining the target motion direction and target motion rate, the target position of the current special effects element can be re-determined based on the target motion direction and target motion rate. Specifically, the superposition vector can be first determined based on the target motion direction and target motion rate, and then the target position can be determined based on the superposition vector and the position information of the current special effects element in the previous video frame of the current video frame.

[0105] For example, the target position of the current special effect element may be determined based on the following formula:

[0106] vec_add=direct ′ *speed ′

[0107] aim_pos=pos_last_frame+vec_add

[0108] Among them, vec_add can represent the superposition vector.

[0109] For example, it can be combined with Figure 3 The overall process of determining the target motion parameters corresponding to the corresponding feature elements based on the individual motion parameter adjustment method is described: For each video frame, Figure 3 The flowchart shown determines the target motion parameters corresponding to the special effect element. For each special effect element, first, determine the superimposed displacement pos_add = direct * speed corresponding to the current special effect element in the current video frame, the target position aim_pos = pos_last_frame + pos_add of the current special effect element in the current video frame, and the distance distance = dis(aim_pos, pos_ori) between the target position and the initial position. Then, determine whether the distance is greater than or equal to the pre-set area constraint distance ≥ area_limit. If so, reverse the initial direction of the special effect element to direct, reduce the initial motion rate of the special effect element, and recalculate the target position: 1. Reverse direct: direct ′ =direct; 2.speed ′ =max_speed / 2; 3. vec_add=direct ′ *speed′ ; 4. aim_pos = pos_last_frame + vec_add; If not, update the target position of the current special effect element position = aim_pos, and further, when speed does not exceed max_speed, randomly adjust the target movement speed ′ =speed+random(-0.05,0.05), update the direction and target direction of the current special effect element: 1. Rotate the direction 0.1 degrees toward the target direction; 2. Randomly define a new unit vector as the target direction direct_towards.

[0110] S240 : Control the corresponding special effect element to move based on the target motion parameter of the special effect element to obtain a special effect video frame.

[0111] The technical solution of the disclosed embodiment is to display at least two moving special effects elements in the current video frame in response to a special effects video generation instruction, and then, when the target object is not included in the current video frame, determine the motion feature information of the special effects element as the first motion information. Furthermore, if the motion feature information is the first motion information, the motion parameter adjustment method is the individual motion parameter adjustment method, so as to determine the target motion parameters corresponding to the corresponding special effects element based on the motion parameter adjustment method. Finally, the corresponding special effects element is controlled to move based on the target motion parameters of the special effects element to obtain a special effects video frame. This solves the problem in the prior art that when a special effects prop includes multiple special effects elements of the same form and movement, the rendering effect of each special effects element is poor, resulting in poor special effects video effect and slow rendering speed of the special effects video generated based on the special effects prop, and realizes the simulation of the individual random behavior of the special effects elements, so that the special effects elements are closer to the real display effect, thereby improving the rendering effect of the special effects video.

[0112] Figure 4 This is a flow chart of a method for generating special effects video frames provided by an embodiment of the present disclosure. Based on the aforementioned embodiment, if the current video frame includes a target object, the motion characteristic information of the special effects element is the second motion information. The target motion parameters are determined based on the motion parameter adjustment method corresponding to the second motion information. For specific implementation methods, please refer to the technical solution of this embodiment. Technical terms that are identical or corresponding to those in the aforementioned embodiments are not repeated here.

[0113] like Figure 4 As shown, the method specifically includes the following steps:

[0114] S310 : In response to a special effect video generation instruction, display at least two moving special effect elements in a current video frame.

[0115] S320, when the target object is included in the current video frame, determining the motion characteristic information of the special effect element as second motion information.

[0116] In the embodiment, the second motion information can be cluster motion information, that is, the motion characteristic information between the special effect elements can be associated with each other, or can be understood as the cluster behavior in the fish school algorithm, that is, the behavior corresponding to the fish gathering into a school when swimming in order to ensure their survival and avoid harm. Optionally, the second motion information can include cohesive motion behavior, dispersion motion behavior, and unified direction motion behavior.

[0117] In actual application, when the target object is detected in the current video frame, the motion characteristic information of the special effect element can be determined as the cluster motion information, so that the corresponding motion parameter adjustment mode can be determined based on the cluster motion information.

[0118] For example, the hand can be combined with the special effect element to trigger the cluster motion information. Figure 5 The triggering process of the cluster motion information is described: taking the hand as an example, first, the hand recognition algorithm is used to process the current video frame to determine the hand key point, and further, the palm center coordinate is determined based on the hand key point, and the key point corresponding to the palm center coordinate is set as the target point in the cluster motion information. At the same time, the opening and closing hand action in the current video frame is identified based on the hand recognition algorithm, and it is determined whether the hand action in the current video frame is opening hand. If yes, the motion characteristic information of the special effect element is set as the cluster motion information, if not, the motion characteristic information of the special effect element is set as the individual motion information, and finally, when the motion characteristic information is the cluster motion information, the cluster motion behavior of the special effect element is triggered based on the target point.

[0119] S330, if the motion characteristic information is the cohesive motion behavior or the dispersion motion behavior in the second motion information, determining the motion parameter adjustment mode as the first adjustment mode, and determining the target motion parameter corresponding to the special effect element based on the motion parameter adjustment mode.

[0120] The cohesion movement behavior can be a behavior of gathering and moving towards the center of the adjacent special effect element. The dispersion movement behavior can be a behavior of dispersing and moving within a certain range of safe distance to avoid being too crowded with the adjacent special effect element. The dispersion movement behavior is determined based on the disappearance of the target object in the current video frame, that is, when the target object included in the current video frame disappears, the movement feature information of each special effect element can be determined as the dispersion movement behavior in the cluster movement information. The first adjustment mode is determined based on the movement information of the plurality of special effect elements. In actual application, when the movement feature information of the special effect element displayed in the current video frame is determined as the cohesion movement behavior or the dispersion movement behavior in the cluster movement information, the movement information of the plurality of special effect elements displayed in the current video frame can be determined, and based on the movement information, the first adjustment mode is determined. The movement information can be information for representing the movement of the corresponding special effect element in the current video frame.

[0121] In actual application, if the movement feature information of the special effect element displayed in the current video frame is the cohesion movement behavior or the dispersion movement behavior, the movement parameter adjustment mode can be determined as the first adjustment mode determined based on the movement information of the plurality of special effect elements, and then the target movement parameter corresponding to the corresponding feature element can be determined based on the first adjustment mode. The advantage of such setting is that the simulation of the cluster movement behavior of the special effect element is realized, and the dynamic interaction feedback effect between the target object and the special effect element is realized, and thus the diversity of the special effect video is enhanced.

[0122] It should be noted that for each special effect element displayed in the current video frame, the process of determining the target movement parameter based on the first adjustment mode is the same, and thus any one of the special effect elements can be taken as an example for illustration.

[0123] Optionally, determining the target movement parameter corresponding to the corresponding special effect element based on the movement parameter adjustment mode comprises: for each special effect element, determining the relative distance information between the current special effect element and other special effect elements, and the distance difference value between the first position information of the current special effect element and the preset position information; determining the average moving direction of the current special effect element based on the relative distance information, the first position information and the second position information of the other special effect elements; determining the target movement direction of the current special effect element based on the distance difference value and the average moving direction; and determining the target moving position in the target movement parameter according to the first position information and the distance of each frame movement.

[0124] In the embodiment, the relative distance information can be a distance between a display position of the current special effect element in the current video frame and a display position of the other special effect element in the current video frame. It should be noted that, no matter the cohesion motion behavior or the dispersion motion behavior, there is relative distance information between the current special effect element and the other special effect element. The relative distance information can include the cohesion relative distance and the dispersion relative distance, and therefore, when the relative distance information is determined, the cohesion relative distance and the dispersion relative distance can be determined respectively. The first position information can be spatial position information of the current special effect element in the world space. Correspondingly, the second position information can be spatial position information of the other special effect element in the world space. The first position information includes the cohesion position information and the dispersion position information. The cohesion position information can be a display position of each special effect element in the current video frame when the cohesion motion behavior occurs. The dispersion position information can be a display position of each special effect element in the current video frame when the dispersion motion behavior occurs. The preset position information can be a display position of a target point in the current video frame, which is used to control the cohesion motion behavior and the dispersion motion behavior. It should be noted that, the preset position information can be determined based on a target object, for example, if the target object included in the current video frame is a user's hand, the preset position information can be position information corresponding to a center point of the user's hand. The average movement direction can include the cohesion movement direction and the dispersion movement direction.

[0125] In actual application, the first position information of the current special effect element and the preset position information can be determined first, and then a distance difference between the first position information and the preset position information is determined. It should be noted that, when the distance difference is determined, it is necessary to determine whether a distance value between the preset position information and the first position information is greater than a preset threshold value, and only when the distance value is greater than the preset threshold value, the distance difference can be determined, otherwise, the distance difference is 0. The advantage of such setting is that it can avoid the slight jitter phenomenon caused when the special effect element is extremely close to the target point.

[0126] For example, the distance difference can be determined based on the following formula:

[0127] targetDir=normalize(targetPos-pos)

[0128] Wherein, targetDir can represent the distance difference, normalize(·) can represent normalization processing, targetPos can represent the preset position information, and pos can represent the first position information.

[0129] It should be noted that, since the movement directions of the current special effect element and other special effect elements are different in the cohesive movement behavior and the dispersed movement behavior, they can be described separately when determining the relative distance information and the average moving direction.

[0130] Furthermore, for cohesive movement behavior, first determine the first position information of the current special effect element and the second position information of other special effect elements. Then, perform difference processing on the second position information and the first position information to obtain a difference vector. Determine the modulus of the obtained difference vector to obtain relative distance information. After obtaining each relative distance information, determine the ratio between the difference vector and the corresponding relative distance information. Add up each ratio to obtain the total cohesive movement direction of the special effect element. Then, determine the ratio between the total cohesive movement direction and the number of special effect elements, so that the average cohesive movement direction of the current special effect element can be finally determined.

[0131] For example, assuming that the current video frame includes n special effect elements (special effect element 1, special effect element 2, ..., special effect element n), for each special effect element, the relative distance information between the current special effect element and the other special effect elements is calculated. If the relative distance information is less than a preset single proximity threshold, and the current special effect element and the other special effect elements belong to the same group sequence number, the average cohesive movement direction of the current special effect element can be determined based on the following formula:

[0132]

[0133]

[0134] Among them, cohesionDirSum can represent the total cohesive movement direction, P i It can represent the first position information of the current special effect element, P j Indicates the second position information of other special effect elements, |P j -P i | can represent relative distance information, and cohesionDir can represent the average cohesive moving direction.

[0135] Furthermore, for dispersed movement behavior, after determining the first position information of the current special effect element and the second position information of other special effect elements, the first position information and the second position information can be differenced to obtain a difference vector, and the modulus of the difference vector can be determined to obtain relative distance information. After obtaining each relative distance information, the ratio between the difference vector and the corresponding relative distance information can be determined. By adding up each ratio, the total dispersed movement direction of the special effect element can be obtained. Then, the ratio between the total movement direction and the number of special effect elements can be determined, so that the average dispersed movement direction of the current special effect element can be finally determined.

[0136] For example, assuming that n special effect elements (special effect element 1, special effect element 2, …, special effect element n) are included in the current video frame, for each special effect element, the relative distance information between the current special effect element and other special effect elements is calculated, and if the relative distance information is less than a preset single proximity threshold, the average dispersion moving direction of the current special effect element can be determined based on the following formula:

[0137]

[0138]

[0139] Wherein, separationDirSum can represent the total dispersion moving direction, P i may represent the first position information of the current special effect element, P j may represent the second position information of other special effect elements, |P j -P i may represent the relative distance information, and separationDir can represent the average dispersion moving direction.

[0140] Further, after obtaining the distance difference and the average cohesion moving direction or the average dispersion moving direction in the average moving direction, the corresponding synthesis parameters can be set according to the motion characteristic information corresponding to the special effect element, then the product between the distance difference and the corresponding synthesis parameter and the product between the average moving direction and the corresponding synthesis parameter are determined respectively, and finally, the products are combined, that is, the target motion direction of the current special effect element can be obtained.

[0141] For example, continuing to refer to the above example, the target motion direction can be determined based on the following formula:

[0142] finalDir = a * targetDir + b * cohesionDir + c * separationDir

[0143] Wherein, finalDir can represent the target motion direction, a, b, and c can be set to different values according to different motion characteristic information, and a + b + c = 1 is satisfied. For example, the flocking motion information can set a to a value greater than b and c; the individual motion information can set a to 0.

[0144] It should be noted that the cohesion motion behavior and the dispersion motion behavior occur in all processes of the flocking motion, and theoretically, an extreme cohesion and dispersion state can be realized by setting b or c to 0. However, in practice, b or c is generally not 0.

[0145] Further, after the target motion direction is obtained, the target moving position can be determined according to the first position information, the target motion direction and a per-frame moving distance. The per-frame moving distance can be a distance that the special effect element moves in a unit time. The target moving position can be a position that the special effect element needs to move to in the unit time.

[0146] It should be noted that the advantage of determining the target motion parameter corresponding to the special effect element based on the above process is that the simulation of the cluster motion behavior of the special effect element is realized, and the effect of dynamically adjusting the motion direction of the dependent special effect element based on the motion behavior state of the special effect element is realized.

[0147] In a specific implementation, the target motion direction can be normalized first, then the normalized value is multiplied by the per-frame moving distance, and the product is added to the first position information to obtain the target moving position.

[0148] For example, the first position information is startPos, and the per-frame moving distance is perDist. The target moving position can be determined based on the following formula:

[0149] moveDirTarget = startPos + normalize(finalDir) * perDist

[0150] Wherein, moveDirTarget can represent the target moving position.

[0151] S340, control the corresponding special effect element to move based on the target motion parameter of the special effect element, to obtain a special effect video frame.

[0152] The technical scheme of the embodiment of the present disclosure, in response to the special effect video generation instruction, displays at least two moving special effect elements in the current video frame, then determines the motion characteristic information of the special effect elements as the second motion information when the target object is included in the current video frame, further, if the motion characteristic information is the cohesive motion behavior or the dispersive motion behavior in the second motion information, determines the motion parameter adjustment mode as the first adjustment mode, determines the target motion parameter corresponding to the corresponding special effect element based on the motion parameter adjustment mode, finally, controls the corresponding special effect element to move based on the target motion parameter of the special effect element, to obtain the special effect video frame, solves the problems in the prior art that when multiple special effect elements of the same form and motion are included in one special effect prop, the rendering effect of each special effect element is poor, resulting in poor special effect video effect and slow rendering speed based on the special effect prop, realizes the simulation of the cluster motion behavior of the special effect elements, so that the special effect elements are closer to the real display effect, improves the authenticity of the special effect video, and at the same time, through the interaction feedback between the special effect prop and the target object, improves the user experience of using the special effect prop.

[0153] Figure 6 is a special effect video frame generation method flowchart provided by the embodiment of the present disclosure, based on the foregoing embodiment, when the special effect elements and the target object in the special effect video frame are displayed, the relative depth information between the special effect elements and the target object can be determined, and the special effect elements and the target object can be rendered and displayed based on the relative depth information. The specific implementation can be referred to the technical scheme of the present embodiment. Among them, the same or corresponding technical terms as the above embodiments are not described here.

[0154] As Figure 6 shown, the method specifically includes the following steps:

[0155] S410, in response to the special effect video generation instruction, displaying at least two moving special effect elements in the current video frame.

[0156] S420, determining the motion characteristic information of the special effect elements according to whether the target object is included in the current video frame.

[0157] S430, determining the motion parameter adjustment mode based on the motion characteristic information, to determine the target motion parameter corresponding to the corresponding special effect element based on the motion parameter adjustment mode.

[0158] S440, controlling the corresponding special effect element to move based on the target motion parameter of the special effect element, to obtain the special effect video frame.

[0159] S450, when the special effect elements and the target object in the special effect video frame are displayed, determining the relative depth information between the special effect elements and the target object.

[0160] It should be noted that when the special effect video frame includes both the target object and the special effect element, and when the target object and the special effect element are rendered and displayed, there may be a case of occlusion rendering, and in this case, the rendering and display can be performed according to the depth information of the special effect element and the depth information of the target object.

[0161] In the embodiment, the relative depth information can be information for representing a size relationship between a depth value corresponding to the special effect element in the special effect video frame and a depth value corresponding to the target object.

[0162] In actual application, when the special effect element and the target object in the special effect video frame are rendered and displayed, the depth value corresponding to the special effect element in the current video frame and the depth value corresponding to the target object in the current video frame can be determined first, and then the relative depth information can be determined according to the depth values.

[0163] Optionally, determining the relative depth information between the special effect element and the target object includes: determining an object depth map corresponding to the target object and an element depth map corresponding to the special effect element; and determining the relative depth information based on an element depth value in the element depth map and an object depth value in the object depth map.

[0164] The depth image is also called a distance image. Unlike that each pixel point in a gray-scale image stores a corresponding brightness value, each pixel point in a depth image stores a depth value. For each pixel point, the depth value represents the distance from the point to the camera. Further, through the pixel values of multiple points, the distance between the special effect element or the target object and the camera in the current video frame can be determined. In the embodiment, the object depth map can be used to represent the distance between the target object and the camera. Correspondingly, the object depth value can be the distance value between the target object and the camera. The element depth map can be used to represent the distance between the special effect element and the camera. The element depth value can be the distance value between the special effect element and the camera.

[0165] In practical applications, after obtaining a special effects video frame, the special effects video frame can be first processed based on an image segmentation algorithm to determine a segmented image corresponding to the target object. Further, the segmented image can be processed according to a depth estimation algorithm to obtain an object depth image corresponding to the target object. At the same time, based on the distance information between the world space position corresponding to the special effects element and the camera, an element depth map is constructed. Further, the object depth value in the object depth map and the element depth value in the element depth map are determined. Then, the relative depth information can be determined based on the object depth value and the element depth value, so that the rendering information corresponding to the target object and the special effects element can be determined based on the relative depth information. The advantage of this setting is that it achieves the effect of accurately estimating the depth value of the special effects element and the depth value of the target object, thereby improving the rendering effect of the special effects element and the target object in the special effects video frame.

[0166] S460: Determine rendering information corresponding to the target object and the special effect element based on the relative depth information.

[0167] In this embodiment, the rendering information may be information obtained when rendering the target object and the special effect element, or may be information representing the rendering status of the target object and the special effect element in the current video frame. The rendering information may include rendering of the target object and rendering of the special effect element.

[0168] In practical applications, after relative depth information is determined based on the object depth value and the element depth value, the target object and special effect element in the current video frame can be rendered based on the relative depth information.

[0169] It should be noted that each pixel corresponding to the target object and special effect element in the current video frame has corresponding relative depth information. For each pixel, the process of determining the rendering information based on the relative depth information is the same. Therefore, the relative depth information corresponding to any pixel can be used as an example for explanation.

[0170] Optionally, based on the relative depth information, the rendering information corresponding to the target object and the special effect element is determined, including: if the relative depth information shows that the element depth value is less than the object depth value, then the rendering information is determined to be the color information for rendering the special effect element; if the relative depth information shows that the element depth value is greater than the object depth value, then the rendering information is determined to be the color information for rendering the target object.

[0171] In actual application, after obtaining the element depth value and the object depth value, the element depth value and the object depth value can be compared to determine the relative depth information. If the relative depth information is that the element depth value is less than the object depth value, it can be considered that the distance between the special effect element and the camera is less than the distance between the target object and the camera, at this time, the rendering information can be determined as rendering the color information of the special effect element to make the special effect element displayed in the current video frame; if the relative depth information is that the element depth value is greater than the object depth value, it can be considered that the distance between the special effect element and the camera is greater than the distance between the target object and the camera, at this time, the rendering information can be determined as rendering the color information of the target object to make the target object displayed in the current video frame. The advantage of such setting is that the relative depth information between the special effect element and the target object is used to render and display the special effect element and the target object, so that the display effect of the special effect video is closer to the real display effect, and the authenticity of the special effect video is improved.

[0172] For example, the element depth value can be determined based on the element depth map and the object depth value can be determined based on the object depth map. Figure 7 The rendering process of the target object and the special effect element is described. Taking a portrait as an example. First, the special effect video frame is processed based on a portrait segmentation algorithm to obtain a portrait segmentation mask image, then the portrait segmentation mask image is processed based on a depth estimation algorithm to obtain a portrait depth map and determine a portrait depth value, at the same time, an element depth map is constructed and an element depth value is determined, further, it is determined whether the element depth value is less than the portrait depth value, if yes, the color information of the special effect element is rendered, if not, the color information of the portrait is rendered, finally, the color is outputted, and the rendering and display of the special effect video frame is completed.

[0173] The technical scheme of the embodiment of the present disclosure is that, in response to a special effect video generation instruction, at least two moving special effect elements are displayed in a current video frame, then, according to whether a target object is included in the current video frame, motion characteristic information of the special effect elements is determined, further, based on the motion characteristic information, a motion parameter adjustment mode is determined, so as to determine target motion parameters corresponding to the corresponding special effect elements based on the motion parameter adjustment mode, finally, based on the target motion parameters of the special effect elements, the corresponding special effect elements are controlled to move, so as to obtain a special effect video frame, when the special effect elements and the target object in the special effect video frame are displayed, relative depth information between the special effect elements and the target object is determined, based on the relative depth information, rendering information corresponding to the target object and the special effect elements is determined, the problems in the prior art that when multiple special effect elements of the same form and motion are included in one special effect prop, the rendering effect of each special effect element is poor, which leads to poor special effect video effect based on the special effect prop and slow rendering speed are solved, natural interaction between the special effect elements and the target object is realized, and by determining the relative depth information, the special effect elements and the target object are rendered and displayed based on the relative depth information, the rendering effect of the special effect video is ensured on the premise of reducing the pressure of the rendering engine of the terminal device.

[0174] Figure 8 is a special effect video frame generation method flowchart provided by the embodiment of the present disclosure, on the basis of the foregoing embodiment, when the special effect element is a phototaxis element and a light-emitting light source, the target object is a hand part, the hand part corresponds to a target light source, and the target motion parameters further include display light intensity, when the special effect element is rendered and displayed, the display light intensity of the special effect element can also be determined, and the corresponding special effect element is rendered and displayed based on the display light intensity. The specific implementation can be referred to the technical scheme of the embodiment. Wherein, the same or corresponding technical terms as the above embodiments are not described here.

[0175] As shown in Figure 8 , the method specifically includes the following steps:

[0176] S510, in response to a special effect video generation instruction, at least two moving special effect elements are displayed in a current video frame.

[0177] S520, according to whether a target object is included in the current video frame, motion characteristic information of the special effect elements is determined.

[0178] S530, based on the motion characteristic information, a motion parameter adjustment mode is determined, so as to determine target motion parameters corresponding to the corresponding special effect elements based on the motion parameter adjustment mode.

[0179] S540, based on the target motion parameters of the special effect elements, the corresponding special effect elements are controlled to move, so as to obtain a special effect video frame.

[0180] S570, determine the to-be-stacked light intensity corresponding to each special effect element under the action of other special effect elements.

[0181] In this embodiment, for each special effect element, when it is detected that other special effect elements appear in the field of view range of the current special effect element, it can be determined that the other special effect elements have a certain influence on the light intensity of the current special effect element. At this time, the to-be-stacked light intensity corresponding to the current special effect element under the action of other special effect elements can be determined. When the special effect element is a light-emitting light source, the to-be-stacked light intensity of the special effect element can be the light intensity that needs to be stacked on the corresponding special effect element.

[0182] It should be noted that the target object corresponds to the target light source, so when the target object appears in the field of view range of the special effect element, it will also have an influence on the light intensity of the special effect element.

[0183] In actual application, the light intensity of the special effect element is influenced by the light source, the behavior, and the density corresponding to the special effect element. Therefore, when determining the to-be-stacked light intensity of each special effect element, for each special effect element, the distance information between the current special effect element and other special effect elements can be determined, based on which the influence of other special effect elements on the light source of the current special effect element can be determined. Meanwhile, the density information between the current special effect element and other special effect elements can also be determined. Furthermore, the to-be-stacked light intensity of the current special effect element can be determined according to the distance information and the density information.

[0184] It should be noted that the determination method of the to-be-stacked light intensity corresponding to each special effect element is the same, so any one of the special effect elements can be taken as an example for illustration.

[0185] Optionally, the to-be-stacked light intensity corresponding to each special effect element under the action of other special effect elements is determined, including: for each special effect element, determining the relative distance information between the current special effect element and other special effect elements and the density information corresponding to other special effect elements; and determining the to-be-stacked light intensity of the current special effect element according to the relative distance information and the density information.

[0186] In this embodiment, for each special effect element, when it is detected that at least one other special effect element appears in the field of view range of the current special effect element, the relative distance information between the current special effect element and other special effect elements can be determined. At this time, if the target object also appears in the field of view range of the current special effect element, the target object can be regarded as one of the other special effect elements, and then the relative distance information between the current special effect element and the special effect element can be determined. Furthermore, the first to-be-stacked light intensity of the current special effect element can be determined according to the inverse relationship between the square of the absolute value of each relative distance information and the light intensity of the current special effect element.

[0187] Exemplarily, the first to-be-stacked light intensity can be determined based on the following formula:

[0188]

[0189] wherein, Intensity light may represent the first to-be-stacked light intensity, Intensity const may represent a constant of light intensity, may represent a square of an absolute value of the relative distance information.

[0190] Further, the light intensity of the current special effect element is also related to the density information of other special effect elements existing in the visual range of the current special effect element. Specifically, for each special effect element, when it is detected that at least one other special effect element appears in the visual range of the current special effect element, the density information corresponding to the other special effect element can be determined, and further, the second to-be-stacked light intensity can be determined according to the inverse relationship between the density information of the other special effect element and the light intensity of the current special effect element.

[0191] Exemplarily, the second to-be-stacked light intensity can be determined based on the following formula:

[0192] Intensity others =Intensity const / Count others

[0193] wherein, Intensity others may represent the second to-be-stacked light intensity, Count others may represent the density information of the other special effect element.

[0194] In actual application, the first to-be-stacked light intensity and the second to-be-stacked light intensity are both the to-be-stacked light intensity of the current special effect element. The advantage of such setting is that the interaction between special effect elements is realized, and thus the display effect of the special effect video is improved, and the user experience in the process of making the special effect video is improved.

[0195] S560, based on the to-be-stacked light intensity and the target light intensity of the target light source, determining the display light intensity of the special effect element and displaying.

[0196] In this embodiment, after obtaining the to-be-stacked light intensity of each special effect element, the display light intensity of the special effect element can be determined according to the to-be-stacked light intensity and the target light intensity of the target light source. The target light source is the light source corresponding to the target object in the current video frame. The display light intensity can be the light intensity of the special effect element when it is displayed in the current video frame

[0197] In actual application, when the target light source appears in the field of view of the special effect element, the target light intensity of the target light source will have a certain influence on the display light intensity of the special effect element. For example, the closer the distance between the special effect element and the target light source, the higher the display light intensity of the special effect element; the farther the distance between the special effect element and the target light source, the lower the display light intensity of the special effect element. Therefore, when determining the display light intensity of the special effect element, the target light intensity of the target light source and the to-be-stacked light intensity can be used for determination.

[0198] Optionally, the display light intensity of the special effect element is determined based on the to-be-stacked light intensity and the target light intensity of the target light source, including: determining the display light intensity of the special effect element based on the to-be-stacked light intensity, the to-be-stacked light intensity of other special effect elements, the target light intensity, and corresponding weight values.

[0199] In actual application, after the to-be-stacked light intensity of each special effect element is determined, for each special effect element, the weight value corresponding to the to-be-stacked light intensity of the current special effect element and the weight value corresponding to the to-be-stacked light intensity of other special effect elements associated with the current special effect element can be determined, and then the to-be-stacked light intensity and the corresponding weight value are multiplied respectively, and the products are added, so as to determine the display light intensity of the special effect element. At the same time, the display light intensity of the special effect element can be adjusted based on the target light intensity of the target light source, so that the special effect element is displayed in the special effect video frame with the adjusted display light intensity. The advantage of this setting is that the interaction between special effect elements and between special effect elements and target objects is realized, and the interest and authenticity of the special effect video are improved.

[0200] For example, the display light intensity of the special effect element can be determined based on the following formula:

[0201] Intensity=Intensity light *a+Intensity others *(1-a)

[0202] Wherein, Intensity can represent the display light intensity of the special effect element, and a and 1-a represent weight values.

[0203] The technical scheme of the embodiment of the present disclosure is that, in response to a special effect video generation instruction, at least two moving special effect elements are displayed in a current video frame, then, according to whether a target object is included in the current video frame, motion characteristic information of the special effect elements is determined, further, based on the motion characteristic information, a motion parameter adjustment mode is determined, so as to determine target motion parameters corresponding to the corresponding special effect elements based on the motion parameter adjustment mode, finally, based on the target motion parameters of the special effect elements, the corresponding special effect elements are controlled to move, so as to obtain a special effect video frame, the corresponding target light intensity of each special effect element under the action of other special effect elements is determined, based on the target light intensity of the target light source and the light intensity to be superimposed, the display light intensity of the special effect element is determined and displayed, the problems in the prior art that when a special effect prop includes multiple special effect elements of the same form and motion, the rendering effect of each special effect element is poor, resulting in that the special effect video effect generated based on the special effect prop is poor and the rendering speed is slow, and the like are solved, the interaction between the target object and the special effect element is realized, the display light intensity of the special effect element is controlled, the interestingness of the special effect video is enhanced, and the use experience of a user in the process of making a special effect video is improved.

[0204] Figure 9 is a structural schematic diagram of a special effect video frame generation device provided by the embodiment of the present disclosure, as shown in Figure 9 The device comprises a special effect element display module 610, a motion characteristic information determination module 620, a motion parameter adjustment mode determination module 630, and a special effect video frame determination module 640.

[0205] The special effect element display module 610 is configured to display at least two moving special effect elements in a current video frame in response to a special effect video generation instruction.

[0206] The motion characteristic information determination module 620 is configured to determine motion characteristic information of the special effect elements according to whether a target object is included in the current video frame.

[0207] The motion parameter adjustment mode determination module 630 is configured to determine a motion parameter adjustment mode based on the motion characteristic information, so as to determine target motion parameters corresponding to the corresponding special effect elements based on the motion parameter adjustment mode.

[0208] The special effect video frame determination module 640 is configured to control the corresponding special effect elements to move based on the target motion parameters of the special effect elements, so as to obtain a special effect video frame.

[0209] On the basis of the above technical schemes, the special effect element display module 610 is specifically configured to control the special effect elements to move in the current video frame with initial motion parameters.

[0210] On the basis of each of the above technical solutions, the initial motion parameter comprises an initial position, an initial speed, an initial orientation, a target orientation and / or a target speed of each special effect element.

[0211] On the basis of each of the above technical solutions, the motion feature information determination module 620 comprises an individual motion information determination unit and a cluster motion information determination unit.

[0212] The first motion information determination unit is configured to determine the motion feature information of the special effect element as first motion information when the target object is not included in the current video frame.

[0213] The second motion information determination unit is configured to determine the motion feature information of the special effect element as second motion information when the target object is included in the current video frame.

[0214] On the basis of each of the above technical solutions, the target object is a target part or a target posture.

[0215] On the basis of each of the above technical solutions, the motion parameter adjustment mode determination module 630 comprises a motion parameter adjustment mode determination unit.

[0216] The motion parameter adjustment mode determination unit is configured to determine the motion parameter adjustment mode as an individual motion parameter adjustment mode if the motion feature information is the first motion information, wherein the individual motion parameter adjustment mode is a mode of determining target motion parameters for each special effect element as an independent individual.

[0217] On the basis of each of the above technical solutions, the motion parameter adjustment mode determination module 630 further comprises an overlay displacement determination unit, a target position determination unit and a target motion parameter determination unit.

[0218] The overlay displacement determination unit is configured to determine, for a special effect element, an overlay displacement corresponding to the current special effect element in the current video frame.

[0219] The target position determination unit is configured to determine, based on the overlay displacement and position information of the current special effect element in a previous video frame of the current video frame, a target position of the current special effect element in the current video frame.

[0220] The target motion parameter determination unit is configured to determine, based on the target position and a pre-set region constraint condition, a target motion parameter of the current special effect element, wherein the target motion parameter comprises a target speed and a target motion direction.

[0221] On the basis of each of the above technical solutions, the target motion parameter determination unit is specifically configured to: if the target position satisfies the region constraint condition, updating the target position of the current special effect element, and adjusting the target motion speed and the target motion direction of the current special effect element based on a preset adjustment step length; and wherein the preset adjustment step length comprises a speed adjustment step length and a direction adjustment step length.

[0222] On the basis of each of the above technical solutions, the target motion parameter determination unit is specifically configured to: if the target position does not satisfy the region constraint condition, adjusting the target motion speed and the target motion direction of the current special effect element based on a second function.

[0223] On the basis of each of the above technical solutions, the second motion information comprises a cohesive motion behavior or a dispersive motion behavior, and the motion parameter adjustment mode determination module 630 further comprises a first adjustment mode determination unit.

[0224] The first adjustment mode determination unit is configured to: if the motion feature information is the cohesive motion behavior or the dispersive motion behavior in the second motion information, determine that the motion parameter adjustment mode is the first adjustment mode.

[0225] The dispersive motion behavior is determined based on disappearance of a target object in a current video frame, and the first adjustment mode is determined based on motion information of a plurality of special effect elements.

[0226] On the basis of each of the above technical solutions, the motion parameter adjustment mode determination module 630 further comprises a relative distance information determination unit, an average moving direction determination unit, a target motion direction determination unit, and a target moving position determination unit.

[0227] The relative distance information determination unit is configured to: for each special effect element, determine relative distance information between the current special effect element and other special effect elements, and a distance difference value between first position information of the current special effect element and preset position information.

[0228] The average moving direction determination unit is configured to: based on each relative distance information, the first position information, and second position information of other special effect elements, determine an average moving direction of the current special effect element.

[0229] The target motion direction determination unit is configured to: based on the distance difference value and the average moving direction, determine a target motion direction of the current special effect element.

[0230] The target moving position determination unit is configured to: according to the first position information, the target motion direction, and a per-frame moving distance, determine a target moving position in the target motion parameter.

[0231] On the basis of each of the technical solutions above, the device further comprises a relative depth information determination module and a rendering information determination module.

[0232] The relative depth information determination module is configured to determine relative depth information between the special effect element and the target object when the special effect element and the target object in the special effect video frame are displayed.

[0233] The rendering information determination module is configured to determine rendering information corresponding to the target object and the special effect element based on the relative depth information.

[0234] On the basis of each of the technical solutions above, the relative depth information determination module comprises an object depth map determination unit and a relative depth information determination unit.

[0235] The object depth map determination unit is configured to determine an object depth map corresponding to the target object and an element depth map corresponding to the special effect element.

[0236] The relative depth information determination unit is configured to determine the relative depth information based on an element depth value in the element depth map and an object depth value in the object depth map.

[0237] On the basis of each of the technical solutions above, the rendering information determination module comprises a rendering information first determination unit and a rendering information second determination unit.

[0238] The rendering information first determination unit is configured to determine the rendering information as color information for rendering the special effect element if the relative depth information is that the element depth value is less than the object depth value.

[0239] The rendering information second determination unit is configured to determine the rendering information as color information for rendering the target object if the relative depth information is that the element depth value is greater than the object depth value.

[0240] On the basis of each of the technical solutions above, the special effect element is at least one of fire, explosion, smoke, water flow, spark, falling leaves, cloud, fog, snow, dust, meteor and phototactic organism.

[0241] On the basis of each of the technical solutions above, the special effect element is a phototactic element and is a light-emitting light source, the target object is a hand part, the hand part corresponds to a target light source, the target motion parameter further comprises display light intensity, and the device further comprises a to-be-overlaid light intensity determination module and a display light intensity determination module.

[0242] The to-be-overlaid light intensity determination module is configured to determine to-be-overlaid light intensity corresponding to each special effect element under the action of other special effect elements.

[0243] The display light intensity determination module is configured to determine the display light intensity of the special effect element based on the to-be-overlapped light intensity and the target light intensity of the target light source, and display the display light intensity.

[0244] On the basis of the above technical solutions, the to-be-overlapped light intensity determination module comprises a relative distance information determination unit and a to-be-overlapped light intensity determination unit.

[0245] The relative distance information determination unit is configured to determine, for each special effect element, relative distance information between the current special effect element and other special effect elements and density information corresponding to the other special effect elements.

[0246] The to-be-overlapped light intensity determination unit is configured to determine the to-be-overlapped light intensity of the current special effect element according to the relative distance information and the relative density information.

[0247] On the basis of the above technical solutions, the display light intensity determination module is specifically configured to determine the display light intensity of the special effect element based on the to-be-overlapped light intensity, the to-be-overlapped light intensity of the other special effect elements, the target light intensity, and corresponding weight values.

[0248] The technical solution of the embodiment of the present disclosure solves the problems in the prior art that when a special effect prop includes multiple special effect elements of the same form and motion, the rendering effect of each special effect element is poor, resulting in poor special effect video effect and slow rendering speed based on the special effect prop, and the like, achieves interaction between a target object and a special effect element, determines the motion characteristic information of the special effect element, and renders and displays the special effect element based on the motion characteristic information, thereby enhancing the richness and interest of the special effect video and improving the interactive experience between the user and the special effect prop.

[0249] The special effect video frame generation apparatus provided in the embodiments of the present disclosure can perform the special effect video frame generation method provided in any of the embodiments of the present disclosure, and has the corresponding function modules and beneficial effects of performing the method.

[0250] It should be noted that each unit and module included in the apparatus is only divided according to the function logic, but is not limited to the above division, as long as the corresponding function can be implemented; in addition, the specific names of each functional unit are only for convenient mutual distinction, and do not limit the protection scope of the embodiments of the present disclosure.

[0251] Figure 10 is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. The following refers to Figure 10 , which shows a structural schematic diagram of an electronic device (e.g., a terminal device or a server) 700 suitable for implementing an embodiment of the present disclosure. The terminal device in an embodiment of the present disclosure can include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (Personal Digital Assistant), a PAD (Tablet Personal Computer), a PMP (Portable Multimedia Player), a vehicle terminal (e.g., a car navigation terminal), and the like, and a fixed terminal such as a digital TV, a desktop computer, and the like. Figure 10 The electronic device shown is merely an example and should not impose any limitation on the functions and use range of an embodiment of the present disclosure. Figure 10 As shown in

[0252] As shown in Figure 10 , the electronic device 700 can include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 are also stored. The processing device 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0253] Generally, the following devices can be connected to the I / O interface 705: an input device 706 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, and the like; an output device 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, and the like; a storage device 708 including, for example, a magnetic tape, a hard disk, and the like; and a communication device 709. The communication device 709 can allow the electronic device 700 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 10 The electronic device 700 is shown with various devices, but it should be understood that all the shown devices are not required to be implemented or possessed. More or fewer devices can be alternatively implemented or possessed.

[0254] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for executing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by the communication apparatus 709, or installed from the storage apparatus 708, or installed from the ROM 702. When the computer program is executed by the processing apparatus 701, the above-mentioned functions defined in the methods of embodiments of the present disclosure are executed.

[0255] The names of the messages or information exchanged between the plurality of devices in the embodiments of the present disclosure are only for illustrative purposes, and are not intended to limit the scope of the messages or information.

[0256] The electronic device provided by the embodiments of the present disclosure and the method for generating special effect video frames provided by the above-mentioned embodiments belong to the same inventive concept, and the technical details not described in detail in the present embodiment can be referred to the above-mentioned embodiments, and the present embodiment has the same beneficial effects as the above-mentioned embodiments.

[0257] The embodiments of the present disclosure provide a computer storage medium, which stores a computer program, and the program is executed by a processor to implement the method for generating special effect video frames provided by the above-mentioned embodiments.

[0258] It should be noted that the computer-readable medium described above can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium, for example, can be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the foregoing. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program used by or in connection with an instruction execution system, apparatus or device. In the disclosure, the computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, in which the computer-readable program code is contained. Such a propagated data signal can take any of a variety of forms, including, but not limited to, an electromagnetic signal, an optical signal, or any suitable combination of the foregoing. The computer-readable signal medium can also be any computer-readable medium that is not a storage medium and that can communicate, propagate or transport a program for use by or in connection with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including, but not limited to, wire, cable, RF (radio frequency), etc., or any suitable combination of the foregoing.

[0259] In some embodiments, the client, server, or both can communicate using any current known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet, and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any current known or future developed networks.

[0260] The computer-readable medium described above can be included in the electronic device described above; or can exist separately from the electronic device described above, and can be accessed via the electronic device described above.

[0261] The computer-readable medium described above carries one or more programs, which, when executed by the electronic device described above, cause the electronic device to:

[0262] display, in response to the special effect video generation instruction, the special effect elements of at least two motions in the current video frame;

[0263] determine motion characteristic information of the special effect elements according to whether the target object is included in the current video frame;

[0264] determine a motion parameter adjustment manner based on the motion characteristic information, to determine a target motion parameter corresponding to a corresponding special effect element based on the motion parameter adjustment manner;

[0265] control the corresponding special effect element to move based on the target motion parameter of the special effect element, to obtain a special effect video frame.

[0266] Alternatively, the computer readable medium carries one or more programs, when the one or more programs are executed by the electronic device, the electronic device is caused to:

[0267] display, in response to the special effect video generation instruction, the special effect elements of at least two motions in the current video frame;

[0268] determine motion characteristic information of the special effect elements according to whether the target object is included in the current video frame;

[0269] determine a motion parameter adjustment manner based on the motion characteristic information, to determine a target motion parameter corresponding to a corresponding special effect element based on the motion parameter adjustment manner;

[0270] control the corresponding special effect element to move based on the target motion parameter of the special effect element, to obtain a special effect video frame.

[0271] Computer program code for carrying out operations of the present disclosure can be written in any one or more of a variety of programming languages or combinations of languages, including an object-oriented programming language such as Java, Smalltalk, C++, or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0272] The computer program product of the first aspect can include one or more non-transitory computer-readable media storing instructions that, when executed, cause one or more processors to perform the operations of the first aspect. The one or more non-transitory computer-readable media can include, for example, magnetic media such as one or more magnetic disks, magnetic tapes or cassettes; optical media such as one or more compact discs, optical discs or Blu-ray discs; magneto-optical media such as one or more floptical discs; solid state media such as one or more solid state drives or other flash memory arrays; or any suitable combination of these. The one or more non-transitory computer-readable media can be encoded with instructions that, when executed, cause one or more processors to perform the operations of the first aspect.

[0273] The units described in the embodiments of the present disclosure can be implemented by software, or by hardware, or by a combination of software and hardware. In some cases, the name of the unit does not constitute a limitation on the unit itself. For example, the first obtaining unit can also be described as a unit for obtaining at least two Internet protocol addresses.

[0274] The functions described in this document can be implemented in part or in whole using one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Program-specific Integrated Circuits (ASICs), Program-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0275] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more of: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0276] According to one or more embodiments of the present disclosure, Example One provides a method for generating a special effect video frame, the method comprising:

[0277] displaying, in response to a special effect video generation instruction, at least two moving special effect elements in a current video frame;

[0278] determining motion characteristic information of the special effect elements according to whether the target object is included in the current video frame;

[0279] determining a motion parameter adjustment manner based on the motion characteristic information, to determine target motion parameters of corresponding special effect elements based on the motion parameter adjustment manner;

[0280] controlling the corresponding special effect elements to move based on the target motion parameters of the special effect elements, to obtain a special effect video frame.

[0281] According to one or more embodiments of the present disclosure, Example Two provides a method for generating a special effect video frame, the method further comprising:

[0282] Optionally, the special effect elements are controlled to move in the current video frame with initial motion parameters.

[0283] According to one or more embodiments of the present disclosure, Example Three provides a method for generating a special effect video frame, the method further comprising:

[0284] Optionally, the initial motion parameters include initial positions, initial velocities, initial orientations, target orientations, and / or target velocities of the special effect elements.

[0285] According to one or more embodiments of the present disclosure, Example Four provides a method for generating a special effect video frame, the method further comprising:

[0286] Optionally, when the target object is not included in the current video frame, the motion characteristic information of the special effect elements is determined as first motion information.

[0287] When the target object is included in the current video frame, the motion characteristic information of the special effect elements is determined as second motion information.

[0288] According to one or more embodiments of the present disclosure, Example Five provides a method for generating a special effect video frame, the method further comprising:

[0289] Optionally, the target object is a target part or a target posture.

[0290] According to one or more embodiments of the present disclosure, Example Six provides a method for generating a special effect video frame, the method further comprising:

[0291] Optionally, if the motion feature information is first motion information, the motion parameter adjustment manner is an individual motion parameter adjustment manner.

[0292] The individual motion parameter adjustment manner is a manner of determining a target motion parameter for each special effect element as an independent individual.

[0293] According to one or more embodiments of the present disclosure, Example Seven provides a method for generating a special effect video frame, the method further comprising:

[0294] Optionally, for a special effect element, a superimposed displacement corresponding to the current special effect element in a current video frame is determined.

[0295] Based on the superimposed displacement and position information of the current special effect element in a previous video frame of the current video frame, a target position of the current special effect element in the current video frame is determined.

[0296] Based on the target position and a preset regional constraint condition, a target motion parameter of the current special effect element is determined.

[0297] The target motion parameter includes a target speed and a target motion direction.

[0298] According to one or more embodiments of the present disclosure, Example Eight provides a method for generating a special effect video frame, the method further comprising:

[0299] Optionally, if the target position satisfies the regional constraint condition, the target position of the current special effect element is updated, and a target motion speed and a target motion direction of the current special effect element are adjusted based on a preset adjustment step.

[0300] The preset adjustment step includes a speed adjustment step and a direction adjustment step.

[0301] According to one or more embodiments of the present disclosure, Example Nine provides a method for generating a special effect video frame, the method further comprising:

[0302] Optionally, if the target position does not satisfy the regional constraint condition, a target motion speed and a target motion direction of the current special effect element are adjusted based on a second function.

[0303] According to one or more embodiments of the present disclosure, Example Ten provides a method for generating a special effect video frame, the second motion information including cohesive motion behavior or dispersive motion behavior, the method further comprising:

[0304] Optionally, if the motion feature information is cohesive motion behavior or dispersive motion behavior in the second motion information, the motion parameter adjustment manner is determined as a first adjustment manner.

[0305] wherein the dispersion motion behavior is determined based on disappearance of the target object in a current video frame, and the first adjustment manner is determined based on motion information of the plurality of special effect elements.

[0306] According to one or more embodiments of the present disclosure, Example Eleven provides a method for generating a special effect video frame, the method further comprising:

[0307] Optionally, for the special effect element, relative distance information between the current special effect element and other special effect elements is determined, and a distance difference value between first position information of the current special effect element and preset position information is determined.

[0308] Based on the relative distance information, the first position information, and second position information of other special effect elements, an average moving direction of the current special effect element is determined.

[0309] Based on the distance difference value and the average moving direction, a target motion direction of the current special effect element is determined.

[0310] According to the first position information, the target motion direction, and a moving distance of each frame, a target moving position in the target motion parameter is determined.

[0311] According to one or more embodiments of the present disclosure, Example Twelve provides a method for generating a special effect video frame, the method further comprising:

[0312] Optionally, relative depth information between the special effect element and the target object is determined.

[0313] Based on the relative depth information, rendering information corresponding to the target object and the special effect element is determined.

[0314] According to one or more embodiments of the present disclosure, Example Thirteen provides a method for generating a special effect video frame, the method further comprising:

[0315] Optionally, an object depth map corresponding to the target object and an element depth map corresponding to the special effect element are determined.

[0316] Based on an element depth value in the element depth map and an object depth value in the object depth map, the relative depth information is determined.

[0317] According to one or more embodiments of the present disclosure, Example Fourteen provides a method for generating a special effect video frame, the method further comprising:

[0318] Optionally, if the relative depth information indicates that the element depth value is less than the object depth value, determining that the rendering information is color information for rendering the special effect element;

[0319] If the relative depth information indicates that the element depth value is greater than the object depth value, the rendering information is determined to be color information for rendering the target object.

[0320] According to one or more embodiments of the present disclosure, [Example 15] provides a method for generating a special effects video frame, the method further comprising:

[0321] The special effect element is at least one of fire, explosion, smoke, water flow, sparks, fallen leaves, clouds, fog, snow, dust, meteors and phototactic organisms.

[0322] According to one or more embodiments of the present disclosure, [Example 16] provides a method for generating a special effect video frame, wherein the special effect element is a phototactic element and a luminous light source, the target object is a hand part, and the hand part corresponds to the target light source, and the target motion parameter also includes display light intensity. The method further includes:

[0323] Optionally, determining the light intensity to be superimposed corresponding to each special effect element under the action of other special effect elements;

[0324] Based on the light intensity to be superimposed and the target light intensity of the target light source, the display light intensity of the special effect element is determined and displayed.

[0325] According to one or more embodiments of the present disclosure, [Example 17] provides a method for generating a special effects video frame, the method further comprising:

[0326] Optionally, for each special effect element, determine the relative distance information between the current special effect element and other special effect elements and the density information corresponding to the other special effect elements;

[0327] The light intensity to be superimposed of the current special effect element is determined according to the relative distance information and the density information.

[0328] According to one or more embodiments of the present disclosure, [Example 18] provides a method for generating a special effects video frame, the method further comprising:

[0329] Optionally, the display light intensity of the special effect element is determined based on the light intensity to be superimposed, the light intensity of other special effect elements to be superimposed, the target light intensity and corresponding weight values.

[0330] According to one or more embodiments of the present disclosure, [Example 19] provides a device for generating a special effect video frame, the device comprising:

[0331] The special effect element display module is configured to display at least two moving special effect elements in the current video frame in response to the special effect video generation instruction.

[0332] The motion characteristic information determination module is configured to determine motion characteristic information of the special effect elements according to whether the target object is included in the current video frame.

[0333] The motion parameter adjustment mode determination module is configured to determine a motion parameter adjustment mode based on the motion characteristic information, so as to determine the target motion parameter corresponding to the corresponding special effect element based on the motion parameter adjustment mode.

[0334] The special effect video frame determination module is configured to control the corresponding special effect element to move based on the target motion parameter of each special effect element, so as to obtain a special effect video frame.

[0335] The above description is merely preferred embodiments of the present disclosure and a description of the principles of the technology applied. It should be understood by those skilled in the art that the disclosed range of the present disclosure is not limited to the technical solutions formed by the specific combinations of the above technical features, and also covers other technical solutions formed by any combination of the above technical features or equivalent features without departing from the above disclosed concept. For example, the technical solutions formed by replacing the above features with the technical features disclosed in the present disclosure (but not limited to) having similar functions.

[0336] In addition, although each operation is described in a particular order, this should not be understood as requiring the operations to be performed in the specific order shown or in a sequential order. In certain circumstances, multi-tasking and parallel processing can be advantageous. Similarly, although several implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments can also be combined in a single embodiment. Conversely, various features described in the context of a single embodiment can also be separated and implemented in multiple embodiments.

[0337] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely illustrative of example forms of implementing the claims.

Claims

1. A method for generating a special effects video frame, characterized in that: include: In response to the special effect video generation instruction, display at least two moving special effect elements in the current video frame; Determining motion feature information of the special effect element according to whether the current video frame includes the target object; Determining a motion parameter adjustment method based on the motion feature information, and determining a target motion parameter corresponding to the corresponding special effect element based on the motion parameter adjustment method; Controlling the corresponding special effect element to move based on the target motion parameter of the special effect element to obtain a special effect video frame; The determining of the motion feature information of the special effect element according to whether the current video frame includes the target object includes: When the target object is not included in the current video frame, determining that the motion characteristic information of the special effect element is first motion information; wherein the first motion information is information corresponding to controlling the motion of the special effect element as an independent individual; When the target object is included in the current video frame, the motion feature information of the special effect element is determined to be second motion information; wherein the second motion information is information corresponding to the cluster motion of at least two special effect elements.

2. The method according to claim 1, characterized in that The displaying of at least two moving special effect elements in the current video frame includes: The special effect element is controlled to move in the current video frame with initial motion parameters.

3. The method according to claim 2, characterized in that The initial motion parameters include the initial position, initial speed, initial orientation, target orientation and / or target speed of each special effect element.

4. The method according to any one of claims 1 to 3, characterized in that The target object is a target part or a target posture.

5. The method according to claim 1, wherein The determining of a motion parameter adjustment method based on the motion feature information includes: If the motion characteristic information is first motion information, the motion parameter adjustment method is an individual motion parameter adjustment method; The individual motion parameter adjustment method is a method of treating each special effect element as an independent individual to determine the target motion parameter.

6. The method according to claim 5, characterized in that The determining of the target motion parameters corresponding to the corresponding special effect elements based on the motion parameter adjustment method includes: For a special effect element, determining the corresponding superimposed displacement of the current special effect element in the current video frame; Determining a target position of the current special effect element in the current video frame based on the superimposed displacement and position information of the current special effect element in a video frame previous to the current video frame; Determining target motion parameters of the current special effect element based on the target position and pre-set regional constraints; The target motion parameters include target speed and target motion direction.

7. The method according to claim 1, characterized in that The second motion information includes a cohesive motion behavior or a dispersed motion behavior, and determining the motion parameter adjustment method based on the motion feature information includes: If the motion characteristic information is a cohesive motion behavior or a dispersed motion behavior in the second motion information, determining that the motion parameter adjustment method is the first adjustment method; The dispersed motion behavior is determined based on the disappearance of the target object in the current video frame, and the first adjustment method is determined based on the motion information of multiple special effect elements.

8. The method according to claim 1, characterized in that When displaying the special effect elements and target objects in the special effect video frame, the method further includes: Determining relative depth information between the special effect element and the target object; Based on the relative depth information, rendering information corresponding to the target object and the special effect element is determined.

9. The method according to claim 8, characterized in that The determining the relative depth information between the special effect element and the target object includes: Determining an object depth map corresponding to the target object and an element depth map corresponding to the special effect element; The relative depth information is determined based on element depth values ​​in the element depth map and object depth values ​​in the object depth map.

10. The method according to claim 1, characterized in that The special effect element is at least one of fire, explosion, smoke, water flow, sparks, fallen leaves, clouds, fog, snow, dust, meteors and phototactic organisms.

11. The method according to claim 1, characterized in that The special effect element is a phototactic element and a luminous light source, the target object is a hand, the hand corresponds to a target light source, the target motion parameter also includes display light intensity, and the method further includes: Determine the light intensity to be superimposed corresponding to each special effect element under the action of other special effect elements; Based on the light intensity to be superimposed and the target light intensity of the target light source, the display light intensity of the special effect element is determined and displayed.

12. The method according to claim 11, characterized in that Determining the light intensity to be superimposed corresponding to each special effect element and other special effect elements includes: For each special effect element, determine the relative distance information between the current special effect element and other special effect elements and the density information corresponding to other special effect elements; The light intensity to be superimposed of the current special effect element is determined according to the relative distance information and the density information.

13. A device for generating special effect video frames, characterized in that: include: A special effect element display module, configured to display at least two moving special effect elements in a current video frame in response to a special effect video generation instruction; A motion feature information determination module, configured to determine the motion feature information of the special effect element according to whether the current video frame includes a target object; a motion parameter adjustment method determining module, configured to determine a motion parameter adjustment method based on the motion feature information, so as to determine a target motion parameter corresponding to a corresponding special effect element based on the motion parameter adjustment method; a special effect video frame determination module, configured to control the movement of a corresponding special effect element based on a target motion parameter of the special effect element to obtain a special effect video frame; The motion feature information determination module includes: a first motion information determination unit and a second motion information determination unit; The first motion information determining unit is configured to determine, when the target object is not included in the current video frame, the motion characteristic information of the special effect element as first motion information; wherein the first motion information is information corresponding to controlling the motion of the special effect element as an independent individual; The second motion information determination unit is used to determine the motion feature information of the special effect element as second motion information when the target object is included in the current video frame; wherein the second motion information is information corresponding to the cluster motion of at least two special effect elements.

14. An electronic device, characterized in that: The electronic device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method for generating special effect video frames as described in any one of claims 1-12.

15. A storage medium comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a computer processor, are used to perform the method for generating a special effect video frame according to any one of claims 1 to 12.

Images