Special effect display method and device, electronic equipment and storage medium

Abstract

Embodiments of the present disclosure provide a special effect display method and device, electronic equipment and storage medium. The method comprises: obtaining live data of a target live room, the live data being used to display a live picture containing a target special effect, the target special effect including a cluster of special effect objects moving within a first time length, the cluster of special effect objects including at least two special effect objects; generating rendering information of the target special effect based on the live data, the rendering information representing a first rendering position corresponding to the special effect objects; and rendering the target special effect in the live picture according to the rendering information of the target special effect. Since the rendering information of the target special effect is synchronized on the terminal device side by using the live data, the display of the cluster simulation special effect is matched with the content of the live picture played in the live room, avoiding the problem of mismatch and asynchronization of the cluster simulation special effect in the display process, and improving the display effect of the cluster special effect.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to a special effects display method, apparatus, electronic device, and storage medium. Background Technology

[0002] Cluster-based special effects display is a technique that uses computer and image processing technology to simulate the images of clusters of objects, such as schools of fish and flocks of birds, which have clustered movement characteristics. It is widely used in various application scenarios that use visual effects.

[0003] However, in live streaming scenarios, cluster-simulated special effects suffer from display asynchrony and mismatch issues during the display process, affecting the display effect of cluster-simulated special effects. Summary of the Invention

[0004] This disclosure provides a special effects display method, apparatus, electronic device, and storage medium to overcome the problem of asynchronous special effects display.

[0005] In a first aspect, embodiments of this disclosure provide a method for displaying special effects, including:

[0006] Acquire live data from the target live stream, the live data being used to display a live stream containing a target special effect, the target special effect comprising a cluster of special effect objects moving within a first duration, the cluster of special effect objects including at least two special effect objects; based on the live data, generate rendering information for the target special effect, the rendering information representing a first rendering position corresponding to the special effect object; render the target special effect within the live stream according to the rendering information for the target special effect.

[0007] Secondly, embodiments of this disclosure provide a special effects display device, including:

[0008] The acquisition module is used to acquire the live data of the target live room. The live data is used to display the live screen containing the target special effects. The target special effects include a cluster of special effects objects moving within a first duration. The cluster of special effects objects includes at least two special effects objects.

[0009] The generation module is used to generate rendering information of the target effect based on the live broadcast data, wherein the rendering information represents the first rendering position corresponding to the effect object;

[0010] The rendering module is used to render the target effect within the live broadcast frame based on the rendering information of the target effect.

[0011] Thirdly, embodiments of this disclosure provide an electronic device, including:

[0012] A processor, and a memory communicatively connected to the processor;

[0013] The memory stores computer-executed instructions;

[0014] The processor executes computer execution instructions stored in the memory to implement the special effects display method described in the first aspect and various possible designs of the first aspect.

[0015] Fourthly, embodiments of this disclosure provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the special effects display method described in the first aspect and various possible designs of the first aspect.

[0016] Fifthly, embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the special effects display method described in the first aspect and various possible designs of the first aspect.

[0017] The special effects display method, apparatus, electronic device, and storage medium provided in this embodiment acquire live data from a target live stream. This live data is used to display a live stream containing target special effects. The target special effects include a cluster of special effects objects moving within a first duration, with each cluster containing at least two special effects objects. Based on the live data, rendering information for the target special effects is generated, representing a first rendering position corresponding to each special effects object. The target special effects are then rendered within the live stream based on this rendering information. By utilizing live data and synchronizing the rendering information of the target special effects on the terminal device side, the display of the cluster-simulated special effects matches the content of the live stream played in the live stream, avoiding mismatches and desynchronization issues between the cluster-simulated special effects and the live content during display, thus improving the display effect of the cluster special effects. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 An application scenario diagram of the special effects display method provided in the embodiments of this disclosure;

[0020] Figure 2 Flowchart of the special effects display method provided in the embodiments of this disclosure Figure 1 ;

[0021] Figure 3 for Figure 2 The flowchart of the specific implementation method of step S102 in the embodiment shown is as follows;

[0022] Figure 4 This is a schematic diagram of a live streaming interface corresponding to rendering information provided in an embodiment of the present disclosure;

[0023] Figure 5 Flowchart of the special effects display method provided in the embodiments of this disclosure Figure 2 ;

[0024] Figure 6 for Figure 5 The flowchart of the specific implementation method of step S202 in the embodiment shown is as follows;

[0025] Figure 7 A schematic diagram of orientation information provided in an embodiment of this disclosure;

[0026] Figure 8 for Figure 5 The flowchart illustrating the specific implementation of step S204 in the illustrated embodiment is shown.

[0027] Figure 9 A schematic diagram of a complete motion trajectory corresponding to a first preset rule provided in an embodiment of this disclosure;

[0028] Figure 10 for Figure 5 A flowchart illustrating a specific implementation of step S205 in the illustrated embodiment;

[0029] Figure 11 for Figure 5 A flowchart illustrating another specific implementation of step S205 in the illustrated embodiment;

[0030] Figure 12 A structural block diagram of the special effects display device provided in the embodiments of this disclosure;

[0031] Figure 13 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure;

[0032] Figure 14 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0034] The application scenarios of the embodiments of this disclosure are explained below:

[0035] Figure 1 This diagram illustrates an application scenario of the special effects display method provided in this disclosure. The special effects display method provided in this disclosure can be applied to live streaming scenarios based on Augmented Reality (AR) and Mixed Reality (MR). More specifically, it can be applied to the display of props and special effects in live streaming rooms. Figure 1 As shown, the method provided in this disclosure can be applied to terminal devices, such as VR glasses. The terminal device runs a live streaming client and plays a live stream. During this process, when the live stream triggers a target effect ("fish school" effect), the terminal device's display interface dynamically displays the moving "fish school" while simultaneously showing the live stream video. This type of effect is the target effect referred to in this disclosure. The "fish school" in the target effect is a cluster of effect objects, consisting of multiple "fish." Each "fish" is an effect object within the effect object cluster. The terminal device calculates the movement trajectories of the "fish" (effect objects) in the "fish school" (effect object cluster) in real time and renders the texture corresponding to each "fish" in the "fish school," thereby simulating the visual effect of the "fish school" moving in the space corresponding to the live stream video.

[0036] In related technologies, the simulation of biological communities such as "schools of fish" and "flocks of birds" using image technology typically involves controlling the movement trajectory of each special effect object within the cluster using preset cluster movement rules, thereby generating a cluster simulation effect (i.e., a target effect) that moves according to certain rules. However, to simulate the movement characteristics of a realistic biological cluster, the target effect requires real-time calculation of the movement state and position of the special effect objects within the cluster, resulting in significant computational resource overhead. Therefore, in existing technologies, the calculation and rendering of target effects are usually completed on the terminal device side. This leads to asynchronous display of the target effect when users enter the live stream at different times, causing the live stream content to be out of sync and potentially causing occlusion of key objects, such as the broadcaster or exhibits, thus affecting the display of effective information.

[0037] This disclosure provides a special effects display method to solve the above problems.

[0038] refer to Figure 2 , Figure 2 Flowchart of the special effects display method provided in the embodiments of this disclosure Figure 1 The method of this embodiment can be applied to terminal devices, and the special effects display method includes:

[0039] Step S101: Obtain the live data of the target live room. The live data is used to display the live screen containing the target special effects. The target special effects include a cluster of special effects objects moving within the first duration. The cluster of special effects objects includes at least two special effects objects.

[0040] For example, the execution subject of the method provided in this embodiment can be a terminal device, wherein a client of a live streaming application runs within the terminal device. When a user enters a target live streaming room by operating the client, the terminal device receives the live streaming data corresponding to the target live streaming room. Further, the live streaming data includes a video stream corresponding to the live streaming screen. By decoding and playing the video stream, the purpose of displaying the currently playing live streaming screen of the target live streaming room within the client of the terminal device can be achieved. The above-described steps of acquiring live streaming data and converting it into video for playback are existing technologies in the field of live streaming technology and will not be described in detail here.

[0041] Furthermore, the live stream data also includes information used to characterize the target special effects, such as special effect identifiers. Based on these identifiers, the corresponding cluster effect type can be determined, more specifically, such as fish school effects, bird flock effects, etc. The target special effect includes a cluster of special effect objects moving within a first duration. Each cluster includes at least two special effect objects; specifically, the cluster is, for example, a "fish school," and the corresponding special effect object is, for example, a "fish" within the "fish school." For example, unlike background cluster effects, the target special effect has a duration of the first duration, meaning the target special effect ends after the first duration. For details on the special effect object cluster and the corresponding special effect object, please refer to [link to relevant documentation]. Figure 1 The relevant illustrations and descriptions in the embodiments shown will not be repeated here.

[0042] Furthermore, after receiving the live data, the terminal device can obtain the effect texture of the target effect corresponding to the effect identifier based on the video stream and the corresponding effect identifier in the live data. The effect texture of the target effect can be preset locally on the terminal device. In subsequent steps, the terminal device can render the effect texture on the live screen while playing the live screen based on the live data, thereby achieving the purpose of displaying the target effect in the live screen. The specific implementation process will be described in detail in subsequent steps.

[0043] Step S102: Based on the live broadcast data, generate rendering information for the target effect. The rendering information represents the first rendering position corresponding to the effect object.

[0044] For example, after obtaining live data, the terminal device uses the live data to determine the rendering information of the target effect, that is, the location of the effect objects in the effect object cluster, i.e., the first rendering position. Specifically, the rendering information may include the first rendering position corresponding to one or more effect objects in the effect object cluster. More specifically, taking the "fish school" effect as an example, after obtaining live data, the terminal device determines the position (first rendering position) of each "fish" in the "fish school" effect in the live screen based on the live data, i.e., generates the rendering information of the target effect. Then, based on the rendering information, the effect texture is rendered, rendering each "fish" to the corresponding first rendering position, thereby displaying the "fish school" effect in the live screen. In another possible implementation, the rendering information may also include the first rendering positions of some (not all) special effects objects. During subsequent rendering, based on these first rendering positions, the special effects objects and their surrounding objects are rendered separately. That is, multiple special effects objects share a single first rendering position, and this position is changed by translation to obtain the rendering positions of the surrounding special effects objects. This allows for control over the rendering positions of each special effects object in the cluster, thereby reducing computational load.

[0045] In one possible implementation, such as Figure 3 As shown, the specific implementation steps of step S102 include:

[0046] Step S1021: Based on the live streaming data, determine the progress information. The progress information is used to characterize the current playback progress of the target special effect in the target live streaming room.

[0047] For example, the live stream data includes progress information representing the current playback progress of the target effect. This progress information can be a progress indicator representing the percentage of playback progress of the target effect; for example, a progress indicator of 0.1 indicates that the current playback progress of the target effect is 10%; a progress indicator of 0.7 indicates that the current playback progress of the target effect is 70%. Alternatively, the progress information can also represent the duration or remaining duration of the current playback progress of the target effect; for example, a progress indicator of 10 indicates that the current playback progress of the target effect has lasted for 10 seconds; a progress indicator of 5 indicates that the remaining duration of the current playback progress of the target effect is 10 seconds. The specific implementation of the progress information can be set as needed, and will not be elaborated here.

[0048] Furthermore, the live data includes Supplemental Enhancement Information (SEI). The specific implementation of step S1021 includes:

[0049] Step S1021A: parse the live data to obtain the supplementary enhancement information corresponding to the current live frame;

[0050] Step S1021B: Obtain progress information based on the supplementary enhancement information corresponding to the current live frame.

[0051] The supplementary enhancement information is stored along with the video stream in the live data and can be used to carry additional information. This supplementary enhancement information is bound to video frames and can be used to characterize relevant information for the corresponding video frames. Specifically, by setting information representing the playback progress of the target special effect in the supplementary enhancement information, the terminal device, after receiving the live data, can parse the live data to obtain the supplementary enhancement information corresponding to the current video frame, and then obtain the progress information corresponding to that current video frame, thus achieving special effect progress judgment based on video frames.

[0052] Among them, supplementary enhancement information is one of the features of the H.264 standard, and its specific data structure and implementation will not be elaborated here.

[0053] Step S1022: Generate rendering information for the target effect based on the progress information.

[0054] For example, after obtaining the progress information, based on the playback progress of the target effect described by the progress information, the cluster of effect objects corresponding to the current target effect and the positions of the effect objects in the cluster are determined, i.e., rendering information. This allows the cluster of effect objects corresponding to the target effect and the corresponding effect to no longer start moving from the initial position, but rather to move dynamically from the corresponding first rendering position according to the progress information, achieving multi-device synchronization of the target effect.

[0055] Step S103: Render the target effect within the live stream frame based on the rendering information of the target effect.

[0056] For example, after obtaining the rendering information corresponding to the target effect, the effect object is rendered in the live broadcast screen based on the first rendering position of the effect object represented by the rendering information, so that the position of the effect object displayed in the live broadcast screen matches the live broadcast screen. Figure 4 This is a schematic diagram of a live streaming interface corresponding to rendering information provided in an embodiment of this disclosure, such as... Figure 4As shown, after entering the target live stream room, terminal device #1 (the corresponding virtual user) triggers the target effect (shown as the "fish school effect") by using a prop in the first instant, and displays the target effect in its client's live stream interface. Then, in the second instant, terminal device #2 enters the target live stream room by clicking on its icon. In the third instant, terminal device #2 receives live stream data from the live stream server, which includes progress information. Based on this progress information, terminal device #2 determines the rendering information, namely the first rendering position corresponding to the "fish" in the "fish school" effect, and renders the effect so that it is displayed on the live stream screen. Simultaneously, terminal device #1 also renders the "fish school" effect, displaying it on the live stream screen. Since terminal device #2 synchronizes based on the progress information in the live stream data, the position of the "fish school" displayed on terminal device #2 in the live stream screen is consistent with the position of the "fish school" displayed on terminal device #1, achieving synchronization of the target effect between terminal devices #1 and #2. This avoids the problem in existing technologies where, after terminal device #2 enters the target live stream room, when the target effect, including the cluster of effect objects, has already been triggered, it must render from the initial position, ensuring the synchronization of the target effect display between different terminal devices.

[0057] In this embodiment, live data from the target live stream is acquired. This live data is used to display a live stream containing a target special effect. The target special effect includes a cluster of special effect objects moving within a first duration, and the cluster includes at least two special effect objects. Based on the live data, rendering information for the target special effect is generated, and this rendering information represents the first rendering position corresponding to each special effect object. The target special effect is then rendered within the live stream based on this rendering information. By utilizing the live data to synchronize the rendering information of the target special effect on the terminal device side, the display of the cluster simulated special effect matches the content of the live stream played in the live stream, avoiding mismatches and desynchronization between the cluster simulated special effect and the live stream content during display, thus improving the display effect of the cluster special effect.

[0058] refer to Figure 5 , Figure 5 Flowchart of the special effects display method provided in the embodiments of this disclosure Figure 2 This embodiment is in Figure 2 Based on the illustrated embodiment, further steps S102-S103 are refined, and the special effects display method includes:

[0059] Step S201: Obtain the live streaming data of the target live streaming room, which includes progress information.

[0060] Step S202: Generate rendering information for the target effect based on the progress information.

[0061] For example, rendering information is information used to characterize the first rendering position of special effects objects in a cluster of special effects objects within the live stream frame. In one possible implementation, the rendering information includes position information, which characterizes the spatial position of the cluster of special effects objects within the live stream frame. Specifically, when the live stream frame is a two-dimensional video, the position information can be two-dimensional coordinates within the live stream frame; when the live stream frame is a three-dimensional video, such as a live stream frame within a virtual reality-based live streaming scene, the position information can be three-dimensional coordinates within virtual reality space. For example, as... Figure 6 As shown, the specific implementation of step S202 includes:

[0062] Step S2021: Determine the playback timestamp of the target effect based on the progress information;

[0063] Step S2022: Based on the playback timestamp, obtain the target video frame. The target video frame is the video frame corresponding to the playback timestamp in the pre-recorded video corresponding to the target effect.

[0064] Step S2023: Generate rendering information based on the target video frame.

[0065] For example, based on Figure 2 The embodiment shown describes the implementation of progress information. Progress information can be a progress marker representing the percentage of playback progress of the target effect, or the duration or remaining duration of the current playback progress of the target effect. By combining the progress marker corresponding to the progress information with the pre-recorded video corresponding to the target effect, a specific playback moment, i.e., a playback timestamp, can be determined in the pre-recorded video. Then, based on this playback timestamp, the corresponding video frame is extracted from the pre-recorded video to obtain the target video frame. The pre-recorded video is a pre-recorded video used to describe the effect content of the target effect, that is, a video describing the cluster of effect objects and the dynamic changes of the effect objects in the target effect. For example, the number of effect objects in the cluster of effect objects in the pre-recorded video can be the same as the number of effect objects in the cluster of effect objects in the target effect. For example, the video duration of the pre-recorded video is consistent with the duration of the target video, both being the first duration. In one possible implementation, the target video frame is selected based on the playback timestamp, and the obtained target video frame is image parsed to obtain the position of each special effects object in the target video frame, thereby obtaining the rendering information.

[0066] In another possible implementation, progress information (such as an identifier representing the percentage of progress) and rendering information (the first rendering position corresponding to each effect object) have a preset mapping relationship. After obtaining the progress information, the corresponding rendering information can be obtained by looking up the table through this mapping relationship. This situation will not be elaborated further.

[0067] For example, the rendering information also includes at least one of the following: angle information, size information, and orientation information, wherein the position information represents the spatial position of the special effects object cluster within the live broadcast screen; the angle information represents the display angle of at least one special effects object; the size information represents the texture size of at least one special effects object; and the orientation information represents the movement direction of the special effects object cluster.

[0068] For example, the rendering information corresponds one-to-one with the special effects object; that is, each special effects object corresponds to a set of rendering information. Specifically, when the special effects object is a 3D texture model, it has multiple display angles, such as the orientation of the "fish" in a "school of fish." These display angles are represented by the angle information in the rendering information. Size information characterizes the texture size of the special effects object, for example, a magnification factor. When the magnification factor is greater than 1, the texture size corresponding to the special effects object is enlarged based on the base size of the special effects object, and the magnification ratio is determined by this factor. Conversely, when the magnification factor is less than 1, the texture size corresponding to the special effects object is reduced based on the base size of the special effects object, and the reduction ratio is determined by this factor. The size information allows for the adjustment of the special effects object's size.

[0069] Furthermore, the method for obtaining angle and size information in the rendering information is similar to the method for obtaining position information in the rendering information. The corresponding target video frame can be obtained based on the progress information, and then image recognition can be performed on the target video frame to obtain the angle and size information corresponding to each special effects object. The specific implementation method has been introduced in the previous steps and will not be repeated here.

[0070] Furthermore, orientation information represents the direction of movement of a cluster of special effects objects. In live video streaming applications, there are often moving target objects in the live stream, such as the streamer or a football in a match. To further enable interaction between the target video and the live stream content, orientation information can be set in the rendering information. This causes the cluster of special effects objects (such as a "school of fish") to move towards or away from the target object, thereby achieving content-based interaction between the target special effects and the live stream, enhancing the visual expressiveness and interactive effect of the target special effects.

[0071] The specific methods for obtaining directional information include: determining the target location corresponding to the target object based on live broadcast data, and generating directional information based on the target location.

[0072] Specifically, after acquiring the live streaming data, image processing is performed on the current video frame corresponding to the live streaming data to identify target objects in the current video frame, such as the "live streamer"; then, based on the live streamer's location, the target location is determined, and orientation information is generated according to the relationship between the target location and the location information. For example, the orientation information can be a direction vector. Figure 7 This is a schematic diagram of location information provided in an embodiment of the present disclosure, such as... Figure 7 As shown, at the first moment, in the current video frame obtained from the live broadcast data, the target object is, for example, the "anchor's face". The "anchor's face" is located on the left side of the screen (field of view), while the spatial position of the special effects object cluster (e.g., "fish school") corresponding to the position information is located on the right side of the screen (field of view). Then, the orientation information is a vector pointing from the spatial position of the special effects object cluster ("fish school") to the target position of the target object ("anchor's face").

[0073] In this embodiment, by processing the live data, directional information representing the movement direction of the special effects object cluster is obtained. In the subsequent rendering process of the special effects object cluster, the running direction of the special effects object cluster can be controlled based on the directional information in the rendering information, so that the special effects object cluster interacts with the content in the live screen, thereby improving the visual expressiveness and interactive effect of the target special effects.

[0074] Step S203: Based on the rendering information, obtain the initial spatial position of the target effect within the live stream frame. The initial spatial position represents the position of the effect object cluster within the live stream frame.

[0075] Step S204: Based on the initial spatial position, determine the cluster motion trajectory. The cluster motion trajectory represents the motion trajectory of the special effects object cluster when it starts moving from the initial spatial position.

[0076] For example, after obtaining the rendering information, the position of the special effects object cluster in the live broadcast screen can be determined based on the position information contained in the rendering information, i.e., the initial spatial position. For example, the initial spatial position refers to the information of a region, which can be understood as a rough estimate of the location of the special effects object cluster. Then, the initial spatial position is used as the starting point of the movement of the special effects object cluster, and the movement trajectory of the special effects object cluster is calculated, i.e., the cluster movement trajectory. Then, the movement of the special effects object cluster is controlled based on the cluster movement trajectory, thereby realizing the synchronous rendering of the target special effects on the terminal device side, so that when different terminal devices enter the target live broadcast room at different times, the special effects object clusters corresponding to the target special effects are displayed synchronously.

[0077] In one possible implementation, such as Figure 8 As shown, the specific implementation of step S204 includes:

[0078] Step S2041: Obtain the first preset rule, which represents the complete motion trajectory of the special effects object cluster within the first duration.

[0079] Step S2042: Determine the cluster movement trajectory based on the initial spatial position and the first preset rule.

[0080] For example, the first preset rule is information used to characterize the complete motion trajectory of the cluster of special effects objects corresponding to the target special effect. Specifically, the first preset rule is used to characterize the set of at least one motion trajectory corresponding to the cluster of special effects objects. The complete motion trajectory of the cluster of special effects objects can be a curve or a set of curves. Figure 9 A schematic diagram of a complete motion trajectory corresponding to a first preset rule provided in an embodiment of this disclosure, as shown below. Figure 9 As shown, the cluster of special effects objects is a "school of fish." The cluster moves along a complete motion trajectory from the left side of the screen (point A) towards the center (point B). After reaching the center, it disperses in all directions (three-dimensional directions) (points C, D, and E). The complete motion trajectory is formed by the set of curves L1 (passing through A, B, C), L2 (passing through A, B, D), and L3 (passing through A, B, E). Curves L1, L2, and L3 overlap between points A and B and separate after point B. The first preset rule describes the information of this complete motion trajectory; the specific implementation can be achieved through preset configuration information, which will not be elaborated here.

[0081] Furthermore, after obtaining the first preset rule corresponding to the target effect, it is combined with the initial spatial position obtained in the previous steps (e.g., Figure 9 Point F in the equation generates a motion trajectory representing the motion from the initial spatial position, i.e., a cluster motion trajectory. In other words, the motion trajectory after point F in the complete motion trajectory described above includes a set of curves consisting of curve L3 passing through (F, B, C), curve L4 passing through (F, B, D), and curve L5 passing through (F, B, E).

[0082] Step S205: Dynamically render the cluster of special effects objects within the live broadcast frame based on the cluster's motion trajectory.

[0083] In one possible implementation, such as Figure 10 As shown, the specific implementation of step S205 includes:

[0084] Step S2051: Obtain the current spatial coordinates of the special effects objects in the special effects object cluster.

[0085] Step S2052: Based on the current spatial coordinates, determine the first motion vector corresponding to the special effects object. The first motion vector represents the direction and / or speed of the special effects object moving from the current spatial coordinates along the cluster motion trajectory.

[0086] Step S2053: Determine the target space coordinates of the special effects object based on the first motion vector corresponding to the special effects object.

[0087] Step S2054: Render the corresponding special effect object at the target space coordinates corresponding to the special effect object, and update the current space coordinates of the special effect object based on the target space coordinates.

[0088] Step S2055: If the special effect object is within the display area, return to step S2051; otherwise, end the loop.

[0089] For example, after obtaining the cluster motion trajectory representing the remaining motion trajectory of the special effects object cluster, the movement of each special effects object in the cluster is controlled based on this cluster motion trajectory. Specifically, firstly, the current spatial coordinates of each special effects object in the cluster are obtained. The initial current spatial coordinates of each special effects object are obtained randomly through initialization based on the initial spatial position obtained in the above steps. For example, in a possible implementation, the current spatial coordinates of the special effects object are the coordinates within the spatial region corresponding to the initial spatial position. Then, based on the current spatial coordinates and the cluster motion trajectory, a motion vector representing the direction and / or velocity of the special effects object moving from the current spatial coordinates along the cluster motion trajectory is obtained, i.e., the first motion vector. In one possible implementation, the first motion vector is a velocity vector, representing the velocity value and velocity direction of the special effects object. Based on the first motion vector and a preset refresh time, the corresponding displacement vector is obtained. Then, the current spatial coordinates of the corresponding special effects object are updated (vector addition) based on the displacement vectors of each special effects object to obtain the target spatial coordinates of each special effects object, and the current spatial coordinates of the special effects object are updated with the target spatial coordinates. The specific implementation method for generating the displacement vector based on the velocity vector and time is existing technology known to those skilled in the art and will not be elaborated further. Further, exemplarily, the velocity of the cluster's motion trajectory can be determined based on the distance between the current spatial coordinates and the object's position. The object's position can be obtained based on live data; the specific implementation method has been described in previous embodiments and will not be repeated here.

[0090] Next, based on the target space coordinates of each special effect object, the corresponding special effect object is rendered, and the special effect object with the original current space coordinates is erased, thereby realizing the visual movement of the special effect object. Then, it is determined whether the special effect object is still displayed in the preset display area of ​​the live screen. If the display area still includes at least one special effect object, it means that the display process of the target special effect has not ended, and the process returns to step S2051; otherwise, the loop ends, and the display process of the target special effect ends.

[0091] In another possible implementation, such as Figure 11 As shown, the specific implementation of step S205 includes:

[0092] Step S2051: Obtain the current spatial coordinates of the special effects objects in the special effects object cluster.

[0093] Step S2052: Based on the current spatial coordinates, determine the first motion vector corresponding to the special effects object. The first motion vector represents the direction and / or speed of the special effects object moving from the current spatial coordinates along the cluster motion trajectory.

[0094] Step S2053A: Determine the second motion vector corresponding to the special effect object based on the current spatial coordinates of the special effect object and the second preset rule. The second preset rule represents the distance relationship and / or velocity relationship between the special effect object and its corresponding adjacent special effect objects, and the second motion vector represents the direction and / or velocity of the special effect object when it moves based on the second preset rule.

[0095] Step S2053B: Generate the target motion vector corresponding to the special effects object based on the first motion vector and the second motion vector corresponding to the special effects object.

[0096] Step S2054: Render the corresponding special effect object at the target space coordinates corresponding to the special effect object, and update the current space coordinates of the special effect object based on the target space coordinates.

[0097] Step S2055: If the special effect object is within the display area, return to step S2051; otherwise, end the loop.

[0098] Example action, in this embodiment, in Figure 11Based on the illustrated embodiment, after step S2052, a step is added to determine the second motion vector corresponding to the special effects object according to a second preset rule. The second preset rule characterizes the distance and / or velocity relationship between the special effects object and its corresponding adjacent special effects objects, and the second motion vector characterizes the direction and / or velocity of the special effects object when it moves according to the second preset rule. Since the special effects object cluster includes multiple special effects objects, in order to simulate the movement characteristics of a realistic biological cluster to the greatest extent, the positional and velocity relationships between each special effects object are restricted by the second preset rule. More specifically, the second preset rule may be, for example, that the distance between the target special effects object and its adjacent special effects objects is greater than 0, and the angle between the motion velocity vectors of the target special effects object and its adjacent special effects objects is greater than a preset angle, etc. By restricting the special effects objects through the second preset rule, a corresponding second motion vector is generated. Then, the first motion vector and the second motion vector are weighted and summed to obtain the target motion vector. The target spatial coordinates are determined based on the target motion vector and rendered. The specific implementation process has been described in previous embodiments and will not be repeated here.

[0099] In this embodiment, by calculating the first motion vector and the second motion vector corresponding to the special effect object respectively, the special effect object moves along the direction of the special effect object cluster while maintaining the motion state between the special effect objects, avoiding problems such as texture overlap and occlusion, and ensuring the visual effect of the target special effect.

[0100] Optionally, after step S201, the method further includes:

[0101] Step S206: Initialize the target effect and obtain the initial spatial position corresponding to the target effect.

[0102] Following step S203, the method further includes:

[0103] Step S207: Compare the difference between the initial spatial position and the initialized spatial position corresponding to the rendering information;

[0104] Step S208: If the difference is less than the first preset value, then render the target effect in the live broadcast frame based on the initialized spatial position; if the difference is not less than the first preset value, then execute step S204.

[0105] For example, after receiving live data containing the target special effect, the terminal device calls the corresponding special effect component to initialize the target special effect and obtains the initial spatial position corresponding to the target special effect. The initial spatial position is the initial position of the special effect object cluster after the target special effect is initialized, and the initial spatial position is a preset value corresponding to the target special effect.

[0106] Furthermore, after obtaining the initial spatial position corresponding to the rendering information, the initialized spatial position is compared with the initial spatial position to obtain the difference between the initial spatial position and the initialized spatial position. If the difference is small, i.e. less than the first preset value, in order to ensure the continuity of the target effect, the subsequent step of determining and dynamically rendering the cluster of effect objects in the live broadcast based on the cluster motion trajectory (remaining motion trajectory) is no longer executed. Instead, the target effect is rendered directly from the initialized spatial position based on the complete motion trajectory. Conversely, if the difference is not less than the first preset value, step S204 is executed. Since calculating the cluster motion trajectory and rendering the target effect based on the cluster motion trajectory can cause a certain degree of stuttering, in this embodiment, by comparing the initial spatial position and the initialized spatial position, and provided that the difference between the initial spatial position and the initialized spatial position is less than the first preset value, the target effect is displayed directly using the initialized spatial position based on the complete motion trajectory, thereby further improving the playback smoothness of the target effect.

[0107] Corresponding to the special effects display method in the above embodiments, Figure 12 This is a structural block diagram of a special effects display device provided in an embodiment of this disclosure. For ease of explanation, only the parts relevant to the embodiments of this disclosure are shown.

[0108] Reference Figure 12 Special effects display device 3 includes:

[0109] The acquisition module 31 is used to acquire the live data of the target live room. The live data is used to display the live screen containing the target special effects. The target special effects include a cluster of special effect objects moving within the first duration. The cluster of special effect objects includes at least two special effect objects.

[0110] The generation module 32 is used to generate rendering information of the target effect based on the live data. The rendering information represents the first rendering position corresponding to the effect object.

[0111] Rendering module 33 is used to render the target effect within the live broadcast frame based on the rendering information of the target effect.

[0112] In one embodiment of this disclosure, the generation module 32 is specifically used to: determine progress information based on live streaming data, wherein the progress information is used to characterize the current playback progress of the target special effect played in the target live streaming room; and generate rendering information of the target special effect based on the progress information.

[0113] In one embodiment of this disclosure, when the generation module 32 determines the progress information based on the live streaming data, it is specifically used to: parse the live streaming data to obtain the supplementary enhancement information corresponding to the current live streaming frame; and obtain the progress information based on the supplementary enhancement information corresponding to the current live streaming frame.

[0114] In one embodiment of this disclosure, the rendering information includes location information, which represents the spatial location of the special effects object cluster within the live broadcast frame;

[0115] When generating rendering information for the target effect based on the progress information, the generation module 32 is specifically used for: determining the playback timestamp of the target effect based on the progress information; obtaining the target video frame based on the playback timestamp, wherein the target video frame is the video frame corresponding to the playback timestamp in the pre-recorded video corresponding to the target effect; and generating rendering information based on the target video frame.

[0116] In one embodiment of this disclosure, the rendering information further includes at least one of the following: angle information and size information; wherein the angle information represents the display angle of at least one special effects object; and the size information represents the texture size of at least one special effects object.

[0117] In one embodiment of this disclosure, the rendering information includes orientation information, which represents the movement direction of the special effects object cluster. The generation module 32 is further configured to: determine the target position corresponding to the target object based on the live broadcast data; and generate orientation information based on the target position.

[0118] In one embodiment of this disclosure, when rendering the target effect in the live broadcast frame according to the rendering information of the target effect, the rendering module 33 is specifically used to: obtain the initial spatial position of the target effect in the live broadcast frame according to the rendering information, the initial spatial position representing the position of the cluster of effect objects in the live broadcast frame; determine the cluster motion trajectory based on the initial spatial position, the cluster motion trajectory representing the motion trajectory of the cluster of effect objects when it starts moving from the initial spatial position; and dynamically render the cluster of effect objects in the live broadcast frame based on the cluster motion trajectory.

[0119] In one embodiment of this disclosure, when the rendering module 33 determines the motion trajectory of the cluster based on the initial spatial position, it is specifically used to: obtain a first preset rule, the first preset rule representing the complete motion trajectory of the special effects object cluster within a first time period; and determine the motion trajectory of the cluster according to the initial spatial position and the first preset rule.

[0120] In one embodiment of this disclosure, when rendering module 33 dynamically renders a cluster of special effects objects within a live broadcast frame based on a cluster motion trajectory, it is specifically configured to: repeatedly execute the following steps to reach a preset condition: obtain the current spatial coordinates of the special effects objects in the cluster; determine the first motion vector corresponding to the special effects object, wherein the first motion vector represents the direction and / or speed of the special effects object moving from the current spatial coordinates along the cluster motion trajectory; determine the target spatial coordinates corresponding to the special effects object based on the first motion vector corresponding to the special effects object; render the corresponding special effects object at the target spatial coordinates corresponding to the special effects object, and update the current spatial coordinates of the special effects object based on the target spatial coordinates.

[0121] In one embodiment of this disclosure, after determining the first motion vector corresponding to the special effects object, the rendering module 33 is further configured to: determine the second motion vector corresponding to the special effects object according to the current spatial coordinates of the special effects object and a second preset rule, wherein the second preset rule characterizes the distance relationship and / or velocity relationship between the special effects object and its corresponding adjacent special effects object, and the second motion vector characterizes the direction and / or velocity of the special effects object when it moves according to the second preset rule; generate the target motion vector corresponding to the special effects object according to the first motion vector and the second motion vector corresponding to the special effects object; when the rendering module 33 determines the target spatial coordinates corresponding to the special effects object according to the first motion vector corresponding to the special effects object, it is specifically configured to: determine the target spatial coordinates corresponding to the special effects object according to the target motion vector corresponding to the special effects object.

[0122] In one embodiment of this disclosure, before rendering the target effect in the live broadcast frame according to the rendering information of the target effect, the rendering module 33 is further configured to: initialize the target effect and obtain the initial spatial position corresponding to the target effect; the rendering module 33 is specifically configured to: compare the difference between the first rendering position corresponding to the rendering information and the initial spatial position; if the difference is less than a first preset value, then render the target effect in the live broadcast frame based on the initial spatial position.

[0123] The acquisition module 31, generation module 32, and rendering module 33 are connected sequentially. The special effects display device 3 provided in this embodiment can execute the technical solution of the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0124] Figure 13 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure, such as... Figure 13 As shown, the electronic device 4 includes:

[0125] Processor 41, and memory 42 communicatively connected to processor 41;

[0126] Memory 42 stores instructions executed by the computer;

[0127] The processor 41 executes computer execution instructions stored in the memory 42 to achieve, for example, Figures 2-11 The special effects display method in the illustrated embodiment.

[0128] Optionally, the processor 41 and the memory 42 are connected via a bus 43.

[0129] For relevant instructions, please refer to the corresponding text. Figures 2-11 The relevant descriptions and effects of the steps in the corresponding embodiments are understood, and will not be elaborated on here.

[0130] refer to Figure 14 The diagram illustrates a structural schematic of an electronic device 900 suitable for implementing embodiments of the present disclosure. The electronic device 900 can be a terminal device or a server. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, personal digital assistants (PDAs), portable Android devices (PADs), portable media players (PMPs), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 14 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0131] like Figure 14 As shown, the electronic device 900 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 into a random access memory (RAM) 903. The RAM 903 also stores various programs and data required for the operation of the electronic device 900. The processing unit 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0132] Typically, the following devices can be connected to I / O interface 905: input devices 906 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 907 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 908 including, for example, magnetic tapes, hard disks, etc.; and communication devices 909. Communication device 909 allows electronic device 900 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 14 An electronic device 900 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0133] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 909, or installed from a storage device 908, or installed from a ROM 902. When the computer program is executed by a processing device 901, it performs the functions defined in the methods of embodiments of this disclosure.

[0134] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0135] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0136] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods shown in the above embodiments.

[0137] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0138] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0139] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0140] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0141] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0142] In a first aspect, according to one or more embodiments of this disclosure, a special effects display method is provided, comprising:

[0143] Acquire live data from the target live stream, the live data being used to display a live stream containing a target special effect, the target special effect comprising a cluster of special effect objects moving within a first duration, the cluster of special effect objects including at least two special effect objects; based on the live data, generate rendering information for the target special effect, the rendering information representing a first rendering position corresponding to the special effect object; render the target special effect within the live stream according to the rendering information for the target special effect.

[0144] According to one or more embodiments of this disclosure, generating rendering information for the target special effect based on the live streaming data includes: determining progress information based on the live streaming data, the progress information being used to characterize the current playback progress of the target special effect in the target live streaming room; and generating rendering information for the target special effect based on the progress information.

[0145] According to one or more embodiments of this disclosure, determining the progress information based on the live streaming data includes: parsing the live streaming data to obtain supplementary enhancement information corresponding to the current live streaming frame; and obtaining the progress information based on the supplementary enhancement information corresponding to the current live streaming frame.

[0146] According to one or more embodiments of this disclosure, the rendering information includes location information, which characterizes the spatial location of the special effects object cluster within the live stream frame; generating the rendering information of the target special effects based on the progress information includes: determining the playback timestamp of the target special effects based on the progress information; obtaining a target video frame based on the playback timestamp, wherein the target video frame is the video frame corresponding to the playback timestamp in the pre-recorded video corresponding to the target special effects; and generating the rendering information based on the target video frame.

[0147] According to one or more embodiments of this disclosure, the rendering information further includes at least one of the following: angle information and size information; wherein the angle information represents the display angle of at least one special effects object; and the size information represents the texture size of at least one special effects object.

[0148] According to one or more embodiments of this disclosure, the rendering information includes orientation information, which characterizes the movement direction of the special effects object cluster. The method further includes: determining the target position corresponding to the target object based on the live broadcast data; and generating the orientation information based on the target position.

[0149] According to one or more embodiments of this disclosure, rendering the target special effect within the live stream based on the rendering information of the target special effect includes: obtaining the initial spatial position of the target special effect within the live stream based on the rendering information, the initial spatial position representing the position of the cluster of special effect objects within the live stream; determining the cluster motion trajectory based on the initial spatial position, the cluster motion trajectory representing the motion trajectory of the cluster of special effect objects starting from the initial spatial position; and dynamically rendering the cluster of special effect objects within the live stream based on the cluster motion trajectory.

[0150] According to one or more embodiments of this disclosure, determining the cluster motion trajectory based on the initial spatial position includes: obtaining a first preset rule, the first preset rule representing the complete motion trajectory of the special effects object cluster within a first duration; and determining the cluster motion trajectory based on the initial spatial position and the first preset rule.

[0151] According to one or more embodiments of this disclosure, dynamically rendering the cluster of special effects objects within a live stream based on the cluster's motion trajectory includes: repeatedly executing the following steps to reach a preset condition: obtaining the current spatial coordinates of the special effects objects in the cluster; determining a first motion vector corresponding to the special effects object, wherein the first motion vector represents the direction and / or speed of the special effects object moving from the current spatial coordinates along the cluster's motion trajectory; determining the target spatial coordinates corresponding to the special effects object based on the first motion vector corresponding to the special effects object; rendering the corresponding special effects object at the target spatial coordinates corresponding to the special effects object, and updating the current spatial coordinates of the special effects object based on the target spatial coordinates.

[0152] According to one or more embodiments of this disclosure, after determining the first motion vector corresponding to the special effects object, the method further includes: determining the second motion vector corresponding to the special effects object based on the current spatial coordinates of the special effects object and a second preset rule, wherein the second preset rule characterizes the distance relationship and / or velocity relationship between the special effects object and its corresponding adjacent special effects object, and the second motion vector characterizes the direction and / or velocity of the special effects object when it moves based on the second preset rule; generating a target motion vector corresponding to the special effects object based on the first motion vector and the second motion vector corresponding to the special effects object; and determining the target spatial coordinates corresponding to the special effects object based on the first motion vector corresponding to the special effects object includes: determining the target spatial coordinates corresponding to the special effects object based on the target motion vector corresponding to the special effects object.

[0153] According to one or more embodiments of this disclosure, before rendering the target effect in the live stream based on the rendering information of the target effect, the method further includes: initializing the target effect to obtain an initial spatial position corresponding to the target effect; rendering the target effect in the live stream based on the rendering information of the target effect includes: comparing the difference between a first rendering position corresponding to the rendering information and the initial spatial position; if the difference is less than a first preset value, then rendering the target effect in the live stream based on the initial spatial position.

[0154] Secondly, according to one or more embodiments of this disclosure, a special effects display device is provided, comprising:

[0155] The acquisition module is used to acquire the live data of the target live room. The live data is used to display the live screen containing the target special effects. The target special effects include a cluster of special effects objects moving within a first duration. The cluster of special effects objects includes at least two special effects objects.

[0156] A generation module is used to generate rendering information for the target special effect based on the live streaming data, wherein the rendering information represents the first rendering position corresponding to the special effect object;

[0157] The rendering module is used to render the target effect within the live broadcast frame based on the rendering information of the target effect.

[0158] According to one or more embodiments of this disclosure, the generation module is specifically configured to: determine progress information based on the live streaming data, the progress information being used to characterize the current playback progress of the target special effect in the target live streaming room; and generate rendering information for the target special effect based on the progress information.

[0159] According to one or more embodiments of this disclosure, when the generation module determines the progress information based on the live streaming data, it is specifically used to: parse the live streaming data to obtain supplementary enhancement information corresponding to the current live streaming frame; and obtain the progress information based on the supplementary enhancement information corresponding to the current live streaming frame.

[0160] According to one or more embodiments of this disclosure, the rendering information includes location information, which characterizes the spatial location of the special effects object cluster within the live stream frame; when the generation module generates the rendering information of the target special effects based on the progress information, it is specifically used to: determine the playback timestamp of the target special effects based on the progress information; obtain a target video frame based on the playback timestamp, wherein the target video frame is the video frame corresponding to the playback timestamp in the pre-recorded video corresponding to the target special effects; and generate the rendering information based on the target video frame.

[0161] According to one or more embodiments of this disclosure, the rendering information further includes at least one of the following: angle information and size information; wherein the angle information represents the display angle of at least one special effects object; and the size information represents the texture size of at least one special effects object.

[0162] According to one or more embodiments of this disclosure, the rendering information includes orientation information, which represents the movement direction of the special effects object cluster. The generation module is further configured to: determine the target position corresponding to the target object based on the live broadcast data; and generate the orientation information based on the target position.

[0163] According to one or more embodiments of this disclosure, when the rendering module renders the target special effect in the live broadcast frame based on the rendering information of the target special effect, it is specifically configured to: obtain the initial spatial position of the target special effect in the live broadcast frame based on the rendering information, wherein the initial spatial position represents the position of the special effect object cluster in the live broadcast frame; determine the cluster motion trajectory based on the initial spatial position, wherein the cluster motion trajectory represents the motion trajectory of the special effect object cluster when it starts moving from the initial spatial position; and dynamically render the special effect object cluster in the live broadcast frame based on the cluster motion trajectory.

[0164] According to one or more embodiments of this disclosure, when the rendering module determines the cluster motion trajectory based on the initial spatial position, it is specifically configured to: obtain a first preset rule, wherein the first preset rule characterizes the complete motion trajectory of the special effects object cluster within the first duration; and determine the cluster motion trajectory according to the initial spatial position and the first preset rule.

[0165] According to one or more embodiments of this disclosure, when the rendering module dynamically renders the cluster of special effects objects within a live broadcast based on the cluster motion trajectory, it is specifically configured to: repeatedly execute the following steps to reach a preset condition: obtain the current spatial coordinates of the special effects objects in the cluster; determine a first motion vector corresponding to the special effects object, wherein the first motion vector represents the direction and / or speed of the special effects object moving from the current spatial coordinates along the cluster motion trajectory; determine the target spatial coordinates corresponding to the special effects object based on the first motion vector corresponding to the special effects object; render the corresponding special effects object at the target spatial coordinates corresponding to the special effects object, and update the current spatial coordinates of the special effects object based on the target spatial coordinates.

[0166] According to one or more embodiments of this disclosure, after determining the first motion vector corresponding to the special effects object, the rendering module is further configured to: determine the second motion vector corresponding to the special effects object based on the current spatial coordinates of the special effects object and a second preset rule, wherein the second preset rule characterizes the distance relationship and / or velocity relationship between the special effects object and its corresponding adjacent special effects object, and the second motion vector characterizes the direction and / or velocity of the special effects object when it moves based on the second preset rule; generate a target motion vector corresponding to the special effects object based on the first motion vector and the second motion vector corresponding to the special effects object; when the rendering module determines the target spatial coordinates corresponding to the special effects object based on the first motion vector corresponding to the special effects object, it is specifically configured to: determine the target spatial coordinates corresponding to the special effects object based on the target motion vector corresponding to the special effects object.

[0167] According to one or more embodiments of this disclosure, before rendering the target effect in the live broadcast frame based on the rendering information of the target effect, the rendering module is further configured to: initialize the target effect and obtain the initial spatial position corresponding to the target effect; the rendering module is specifically configured to: compare the difference between the first rendering position corresponding to the rendering information and the initial spatial position; if the difference is less than a first preset value, then render the target effect in the live broadcast frame based on the initial spatial position.

[0168] Thirdly, according to one or more embodiments of the present disclosure, an electronic device is provided, including: a processor, and a memory communicatively connected to the processor;

[0169] The memory stores computer-executed instructions;

[0170] The processor executes computer execution instructions stored in the memory to implement the special effects display method described in the first aspect and various possible designs of the first aspect.

[0171] Fourthly, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, wherein computer-executable instructions are stored therein, and when a processor executes the computer-executable instructions, the special effects display method described in the first aspect and various possible designs of the first aspect is implemented.

[0172] Fifthly, embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the special effects display method described in the first aspect and various possible designs of the first aspect.

[0173] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0174] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0175] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method for displaying special effects, characterized in that, include: Acquire live streaming data of the target live streaming room. The live streaming data is used to display the live streaming screen containing the target special effect. The target special effect includes a cluster of special effect objects moving within a first duration. The cluster of special effect objects includes at least two special effect objects. The live streaming data includes progress information, which is used to characterize the current playback progress of the target special effect in the target live streaming room. Based on the live stream data, determine the progress information; Based on the progress information, the rendering information for the target effect is generated; The rendering information represents the first rendering position corresponding to the special effects object; The target effect is rendered within the live stream frame based on the rendering information of the target effect.

2. The method according to claim 1, characterized in that, The live stream data also includes: supplementary enhancement information; the step of determining progress information based on the live stream data includes: Analyze the live data to obtain supplementary enhancement information corresponding to the current live frame; The progress information is obtained based on the supplementary enhancement information corresponding to the current live frame.

3. The method according to claim 1, characterized in that, The rendering information includes location information, which represents the spatial location of the special effects object cluster within the live broadcast frame. The step of generating rendering information for the target effect based on the progress information includes: Based on the progress information, determine the playback timestamp of the target special effect; Based on the playback timestamp, the target video frame is obtained. The target video frame is the video frame corresponding to the playback timestamp in the pre-recorded video corresponding to the target effect. The rendering information is generated based on the target video frame.

4. The method according to claim 3, characterized in that, The rendering information also includes at least one of the following: angle information and size information; The angle information represents the display angle of at least one special effects object; The size information represents the texture size of at least one special effects object.

5. The method according to claim 1, characterized in that, The rendering information includes orientation information, which represents the movement direction of the special effects object cluster. The method further includes: Based on the live broadcast data, determine the target location corresponding to the target object; The orientation information is generated based on the target location.

6. The method according to claim 1, characterized in that, Rendering the target effect within the live stream frame based on the rendering information of the target effect includes: Based on the rendering information, the initial spatial position of the target effect within the live stream is obtained, and the initial spatial position represents the position of the cluster of effect objects within the live stream. Based on the initial spatial position, the cluster motion trajectory is determined, and the cluster motion trajectory represents the motion trajectory of the special effects object cluster when it starts moving from the initial spatial position; The cluster of special effects objects is dynamically rendered within the live stream based on the cluster's motion trajectory.

7. The method according to claim 6, characterized in that, Determining the cluster's movement trajectory based on the initial spatial position includes: Obtain a first preset rule, which represents the complete motion trajectory of the special effects object cluster within the first duration; The movement trajectory of the cluster is determined based on the initial spatial location and the first preset rule.

8. The method according to claim 6, characterized in that, The process of dynamically rendering the cluster of special effects objects within the live stream based on the cluster's motion trajectory includes: Repeat the following steps until the preset conditions are met: Obtain the current spatial coordinates of the special effects objects in the special effects object cluster; Based on the current spatial coordinates, a first motion vector corresponding to the special effects object is determined. The first motion vector represents the direction and / or speed of the special effects object moving from the current spatial coordinates along the cluster motion trajectory. Based on the first motion vector corresponding to the special effects object, determine the target space coordinates corresponding to the special effects object; Render the corresponding special effect object at the target space coordinates corresponding to the special effect object, and update the current space coordinates of the special effect object based on the target space coordinates.

9. The method according to claim 8, characterized in that, After determining the first motion vector corresponding to the special effects object, the process also includes: Based on the current spatial coordinates of the special effect object and the second preset rule, a second motion vector corresponding to the special effect object is determined, wherein the second preset rule represents the distance relationship and / or velocity relationship between the special effect object and its corresponding adjacent special effect objects, and the second motion vector represents the direction and / or velocity of the special effect object when it moves based on the second preset rule; Based on the first motion vector and the second motion vector corresponding to the special effects object, generate the target motion vector corresponding to the special effects object; Determining the target space coordinates corresponding to the special effects object based on the first motion vector corresponding to the special effects object includes: The target spatial coordinates of the special effects object are determined based on the target motion vector corresponding to the special effects object.

10. The method according to any one of claims 1-9, characterized in that, Before rendering the target effect within the live stream based on the rendering information of the target effect, the method further includes: Initialize the target effect to obtain the initial spatial position corresponding to the target effect; Rendering the target effect within the live stream frame based on the rendering information of the target effect includes: Compare the difference between the first rendering position corresponding to the rendering information and the initial spatial position; If the difference is less than the first preset value, then the target effect is rendered within the live broadcast frame based on the initial spatial position.

11. A special effects display device, characterized in that, include: The acquisition module is used to acquire live data of the target live room. The live data is used to display the live screen containing the target special effect. The target special effect includes a cluster of special effect objects moving within a first duration. The cluster of special effect objects includes at least two special effect objects. The live data includes progress information, which is used to characterize the current playback progress of the target special effect in the target live room. The generation module is used to determine progress information based on the live streaming data; and to generate rendering information for the target special effect based on the progress information, wherein the rendering information represents the first rendering position corresponding to the special effect object. The rendering module is used to render the target effect within the live broadcast frame based on the rendering information of the target effect.

12. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the special effects display method as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the special effects display method as described in any one of claims 1 to 10.

14. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the special effects display method according to any one of claims 1 to 10.

Images