Data interaction method, device, computer equipment, storage medium, program product

Abstract

The present application relates to a data interaction method, device, computer equipment, storage medium and computer program product. The method includes: obtaining multimedia data, special effect configuration information and particle resources associated with the multimedia data; if a particle special effect event is triggered in the playback scene for the multimedia data, a particle node is generated according to the particle resource; according to the binding method in the special effect configuration information, the target node corresponding to the scene element to be implemented with the particle special effect in the playback scene is determined, and the particle node is bound to the target node; rendering is performed based on the bound particle node, so as to display the particle special effect in the display method indicated by the special effect configuration information during the presentation of the multimedia data. The use of this method can display the particle special effect in the playback scene, thereby enriching the diversity of special effect styles in the playback scene.

Description

Technical Field

[0001] The present application relates to the field of multimedia information technology, and in particular to a data interaction method, device, computer equipment, storage medium and computer program product. Background Art

[0002] With the development of video playback technology, users' viewing atmosphere has received more and more attention. For this reason, most related video players provide a bullet screen function, which can give users a sense of real-time interaction and enhance the viewing atmosphere of users. In order to further enhance the viewing interest of users, the related technology also proposes a way to display special effect bullet screen during video playback.

[0003] However, the special effects barrage in the related art are mostly realized based on pre-designed animations or image sequences, resulting in a single form of the displayed special effects barrage. Summary of the invention

[0004] Based on this, it is necessary to provide a data interaction method, device, computer equipment, computer-readable storage medium and computer program product that can enrich the diversity of special effect styles in the playback scene in response to the above technical problems.

[0005] On the one hand, the present application provides a data interaction method, comprising:

[0006] Acquire multimedia data, and acquire special effect configuration information and particle resources associated with the multimedia data;

[0007] If a particle special effect event is triggered in the playback scene of the multimedia data, a particle node is generated according to the particle resource;

[0008] According to the binding method in the special effect configuration information, determine the target node corresponding to the scene element on which the particle special effect is to be implemented in the playback scene, and bind the particle node to the target node;

[0009] Rendering is performed based on the bound particle nodes, so as to display the particle special effects in the display mode indicated by the special effect configuration information during the presentation of the multimedia data.

[0010] On the other hand, the present application also provides a data interaction device, including:

[0011] An acquisition module, used to acquire multimedia data, and acquire special effect configuration information and particle resources associated with the multimedia data;

[0012] A generation module, configured to generate a particle node according to the particle resource if a particle special effect event is triggered in a playback scene for the multimedia data;

[0013] A determination module, used to determine the target node corresponding to the scene element on which the particle special effect is to be implemented in the playback scene according to the binding method in the special effect configuration information, and bind the particle node to the target node;

[0014] A rendering module is used to perform rendering based on the bound particle nodes, so as to display the particle special effects in a display mode indicated by the special effect configuration information during the presentation of the multimedia data.

[0015] On the other hand, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0016] Acquire multimedia data, and acquire special effect configuration information and particle resources associated with the multimedia data;

[0017] If a particle special effect event is triggered in the playback scene of the multimedia data, a particle node is generated according to the particle resource;

[0018] According to the binding method in the special effect configuration information, determine the target node corresponding to the scene element on which the particle special effect is to be implemented in the playback scene, and bind the particle node to the target node;

[0019] Rendering is performed based on the bound particle nodes, so as to display the particle special effects in the display mode indicated by the special effect configuration information during the presentation of the multimedia data.

[0020] On the other hand, the present application also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0021] Acquire multimedia data, and acquire special effect configuration information and particle resources associated with the multimedia data;

[0022] If a particle special effect event is triggered in the playback scene of the multimedia data, a particle node is generated according to the particle resource;

[0023] According to the binding method in the special effect configuration information, determine the target node corresponding to the scene element on which the particle special effect is to be implemented in the playback scene, and bind the particle node to the target node;

[0024] Rendering is performed based on the bound particle nodes, so as to display the particle special effects in the display mode indicated by the special effect configuration information during the presentation of the multimedia data.

[0025] On the other hand, the present application also provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0026] Acquire multimedia data, and acquire special effect configuration information and particle resources associated with the multimedia data;

[0027] If a particle special effect event is triggered in the playback scene of the multimedia data, a particle node is generated according to the particle resource;

[0028] According to the binding method in the special effect configuration information, determine the target node corresponding to the scene element on which the particle special effect is to be implemented in the playback scene, and bind the particle node to the target node;

[0029] Rendering is performed based on the bound particle nodes, so as to display the particle special effects in the display mode indicated by the special effect configuration information during the presentation of the multimedia data.

[0030] The above-mentioned data interaction method, device, computer equipment, storage medium and computer program product can generate the required particle nodes according to the particle resources associated with the multimedia data when a particle special effect event is triggered in the playback scene of multimedia data. Since the properties of the particles in the particle resources have been configured, there is no need to reconfigure them. In this way, the generation efficiency of the particle nodes in the playback scene can be improved; then, according to the binding method configured in the special effect configuration information associated with the media data, the target node corresponding to the scene element to be implemented with the particle special effect in the playback scene is quickly and accurately determined, and the particle node is bound to the target node; finally, rendering is performed based on the bound particle node to display the particle special effect in the presentation process of the multimedia data according to the display method indicated by the special effect configuration information, so as to realize the display of the particle special effect in the playback scene, thereby enriching the diversity of special effect styles in the playback scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0032] Figure 1 An application environment diagram of a data interaction method in an embodiment;

[0033] Figure 2 A schematic diagram of a flow chart of a data interaction method in an embodiment;

[0034] Figure 3A A schematic diagram of a configuration interface for special effect configuration information in an embodiment;

[0035] Figure 3B An example diagram of different forms of particle special effects in an embodiment;

[0036] Figure 4 A schematic diagram of a flow chart of a method for determining a target node for binding a particle node in one embodiment;

[0037] Figure 5 is a schematic diagram of multiple layers included in a playback scene in one embodiment;

[0038] Figure 6 A schematic diagram of a custom trajectory in one embodiment;

[0039] Figure 7 A schematic diagram of a flow chart of a binding method for binding a particle node to a target node in one embodiment;

[0040] Figure 8 An example diagram of a conversion method of location area information in an embodiment;

[0041] Fig. 9 This is an example diagram of the life cycle of a bullet screen and the corresponding loading process in one embodiment;

[0042] Fig. 10A This is an example diagram of a particle special effect bound in a bullet screen in an embodiment;

[0043] Fig. 10B This is an example diagram of a particle binding special effect in another embodiment;

[0044] Fig.11 This is an example diagram of the display effect of the attribute animation corresponding to the particles in one embodiment;

[0045] Fig.12 A schematic diagram of the overall process of loading particle effects in one embodiment;

[0046] Fig.13 A schematic diagram of a method for implementing attribute animation in one embodiment;

[0047] Fig.14 is a structural block diagram of an application processing device in one embodiment;

[0048] Fig.15 The internal structure diagram of a computer device that executes a data interaction method in one embodiment. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0050] The data interaction method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store multimedia data, trajectory data, barrage data, particle resources, special effect configuration information, etc. that the server 104 needs to process. The data storage system can be set up separately, can be integrated on the server 104, or can be placed on the cloud or other network servers. An application client for playing multimedia data is deployed in the terminal 102, and the application client can be a video playback client, an audio playback client, a news information reading client, a browser client, an information stream client, an instant messaging client, a live broadcast client, etc. In response to the playback operation for the specified multimedia data, the terminal first pulls the specified multimedia data from the server, and at the same time pulls the special effect configuration information and particle resources associated with the multimedia data. Then, during the process of playing the multimedia data, if the terminal detects that a particle special effect event is triggered in the playback scene of the multimedia data, the terminal generates a particle node according to the particle resource of the multimedia data, and determines the target node corresponding to the scene element to be implemented with the particle special effect in the current playback scene from the playback scene according to the binding method in the special effect configuration information. After determining the exact position of the particle node in the target node, the particle node is bound to the target node and rendered. After the particle node is rendered, the particle special effect is displayed in the presentation process of the multimedia data according to the display method indicated by the special effect configuration information. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. Server 104 can be implemented by an independent server or a server cluster composed of multiple servers, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDN, Content Delivery Network), as well as big data and artificial intelligence platforms.

[0051] In some embodiments, Figure 2 As shown, a data interaction method is provided. In this embodiment, the method is applied to Figure 1The terminal in the example is used for explanation. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps 202 to 206. Among them:

[0052] Step 202: Acquire multimedia data, and special effect configuration information and particle resources associated with the multimedia data.

[0053] In actual implementation, the terminal is deployed with an application client for playing multimedia data, and the application client may be a video playing client, an audio playing client, etc. The multimedia data played by the terminal through the application client may be audio, video, and other multimedia content obtained from various sources (such as files, network streams, databases, etc.). In the process of playing multimedia data based on the application client, the terminal may also display various types of special effects set for the multimedia data, such as particle effects, lighting effects, shadow effects, material effects, etc. The relevant information of the special effects may be stored in the special effect configuration information associated with the multimedia data.

[0054] The special effect configuration information associated with multimedia data is explained. The special effect configuration information associated with multimedia data is usually stored in a certain format (such as JSON, XML or a custom data structure). The terminal can obtain the special effect configuration information associated with multimedia data by reading a file or from a database. The special effect configuration information is used to record the configuration information of various display style parameters configured for the current multimedia data. In the case where the special effect configuration information includes display style parameters related to particle special effects, the terminal can obtain particle resources related to particle special effects according to the display style parameters.

[0055] To explain particle resources, in actual implementation, the properties of a particle style in a certain form can usually be stored as a particle resource. Particle resources are related configuration files used to describe particle style parameters, and the file can also store particle image information at the same time. That is, particle resources are a file that records the parameter values ​​of particle style parameters in specified particle effects, and particle style parameters correspond to the properties of the particles themselves. If the property of the particle itself is the angle "angle", there is also a particle style parameter in the particle resource that indicates the angle "angle". In actual applications, particle resources can exist in the form of plist files. Particle resources are reusable as a separate file, that is, other scenes can also directly use the particle resources to achieve a certain preset particle style. Particle resources are compatible with the particle system in the Unreal Engine that supports playback scenes. The particle configuration attributes included in the particle resource may include at least: particle map, the style of displaying a single particle, usually in png format; the maximum number of particles, the maximum number of particles that can exist at the same time in a single particle emitter, when the number of particles reaches this value, the particle emission will stop; duration, the active duration of the particle emitter; emission frequency, the number of particles that the particle emitter can emit per second; the survival time of a single particle and its range of variation; particle size and its range of variation; particle spin angle and its range of variation, etc. In practical applications, the association between multimedia data and particle resources can be established in the special effects configuration information. In addition, a brief description of the particle system is given. The particle system represents the technology of simulating some specific fuzzy phenomena in three-dimensional computer graphics, and these phenomena are difficult to achieve realistic physical motion laws using other traditional rendering technologies. Phenomena often simulated by particle systems include fire, explosion, smoke, water flow, sparks, fallen leaves, clouds, fog, snow, dust, meteor trails, or abstract visual effects such as luminous trails, etc.

[0056] For example, Figure 3A As shown, Figure 3A A schematic diagram of the configuration interface of the special effect configuration information provided for the embodiment of the present application. Shown in the figure are display style parameters, the video ID is the identifier of the multimedia data in the playback scene, the special effect name is the name of the particle special effect, fireworks special effect, etc., selecting a particle resource is to select the particle resource plist file corresponding to the special effect name to establish an association between the particle resource and the multimedia data indicated by the video ID, and the coordinate parameters in the display style parameters are used to configure the location area information of the particle node. The custom parameters in the display style parameters can modify the initial values ​​of the particle style parameters in the particle resource to obtain the particle style parameters suitable for the current playback scene, thereby realizing the reuse of particle resources.

[0057] The acquisition method of special effect configuration information and particle resources is described. In response to the playback operation of the specified multimedia data, the terminal first performs the loading operation of the multimedia data. When the loading starts, the terminal sends a Figure 1 The server shown in the figure requests the special effect configuration information and particle resources associated with the multimedia data, receives the special effect configuration information and particle resources sent by the server, and saves them in the local storage space. It should be noted that the terminal can download the above resources immediately after receiving the response operation from the server to prevent the particle special effects from being delayed on the screen due to the time-consuming resource download.

[0058] Step 204: If a particle special effect event is triggered in the playback scene of multimedia data, a particle node is generated according to the particle resource.

[0059] In actual implementation, a particle effect event refers to the performance of a particle effect that is triggered when certain specific conditions or operations occur. These operations can be when the user performs a preset interactive operation or when the playback progress of the multimedia data reaches a preset time point, which is the time point when the particle effect begins to take effect. Among them, the user performs a preset interactive operation, which can be when the user interacts with the playback interface of the multimedia data, such as clicking, dragging, sliding, etc., which can trigger a particle effect event. These events can be used to enhance the user experience, such as displaying particle effects when the mouse hovers, releasing particles when clicking a button, and other state changes. Or the terminal detects that the playback progress of the multimedia data reaches the start time point when the particle effect set in the special effect configuration information takes effect, and it can be determined that the playback scene has triggered a particle effect event. The terminal detects that a particle effect event is triggered in the playback scene, and can generate a particle node based on the particle resource associated with the multimedia data.

[0060] In some embodiments, the special effect configuration information includes a special effect effective time period, and the terminal can determine that a particle special effect event is triggered for the playback scene of the multimedia data in the following manner: when the playback progress of the multimedia data is within the special effect effective time period, it is determined that a particle special effect event is triggered for the playback scene of the multimedia data; or, if a preset interactive operation is detected in the playback scene of the multimedia data, it is determined that a particle special effect event is triggered for the playback scene of the multimedia data.

[0061] In actual implementation, in addition to judging whether a particle special effect event is triggered by judging whether the playback progress of multimedia data is in the special effect effective time period, it is also possible to judge whether a particle special effect event is triggered by an interactive operation set in the playback scene of the multimedia data. Among them, whether the playback progress is in the special effect effective time period actually refers to whether the playback progress of the multimedia data hits the special effect effective time period, and the special effect effective time period can be determined by configuring the start time and end time in the special effect configuration information. The preset interactive operation can be a click operation on a specified function item (such as a button, icon, etc.). When the terminal detects a trigger operation for a preset interactive operation, it can be determined that a particle special effect event is triggered in the playback scene.

[0062] In this embodiment, by providing a variety of triggering methods for particle special effect events, it is possible to provide users with a variety of interactive methods, which helps to improve the user stickiness of live broadcast platforms and video websites and attract more users to participate in the interaction.

[0063] In actual implementation, the multimedia data playback scene can be realized based on the Unreal Engine. The node is a specific instance of the node class provided by the Unreal Engine, and the particle node is the node corresponding to the particle in the particle system provided by the Unreal Engine. At the technical level, by generating particle nodes supported by the Unreal Engine, the characteristics of displaying particle effects during the playback of multimedia data are realized. The terminal reads the attribute information of the particles configured in the particle resource, instantiates the node in the Unreal Engine, and obtains the corresponding particle node.

[0064] Step 206: determine the target node corresponding to the scene element on which the particle special effect is to be implemented in the playback scene according to the binding method in the special effect configuration information, and bind the particle node to the target node.

[0065] In actual implementation, the particle special effects can be bound to the scene elements in the playback scene. After binding the scene elements, the particle special effects can be displayed in the corresponding position and area of ​​the scene elements. The scene elements in the playback scene can be window elements, track elements, bullet screen elements, etc. The terminal can read the corresponding binding method from the special effect configuration information, and then determine the target scene element to be implemented in the playback scene according to the binding method. And determine the target node corresponding to the target scene element in the node tree provided by the Unreal Engine for the playback scene. Finally, the particle node is bound to the target node by setting the position attribute of the particle node so that the position of the particle node is consistent with that of the target node, that is, when binding the particle node, it is necessary to ensure that the position, size and other attributes of the particle node and the target node are consistent to achieve the correct visual effect. Among them, in the process of binding the particle node to the target node, it is necessary to perform coordinate conversion on the position area information of the particle node to obtain the relative position area information with the target node as a reference, so as to display the particle node at the target position of the target node. And by modifying the parent node attribute of the particle node, the parent node attribute of the particle node points to the target node, thereby realizing the binding of the particle.

[0066] Step 208 , rendering is performed based on the bound particle nodes, so as to display the particle special effects in a display manner indicated by the special effect configuration information during the presentation of the multimedia data.

[0067] In actual implementation, during the playback of multimedia data, the terminal renders the bound particle nodes through the particle rendering capability provided by the particle system of the Unreal Engine, so as to display the particle special effects in the playback interface according to the display mode indicated by the special effect configuration information. The display style parameters recorded in the special effect configuration information determine the display mode of the particle node in the playback scene, and can indicate that the particle special effects are displayed at the target position or associated target area in the scene element. The rendering process for the particle node at least includes coloring, transparency processing, size adjustment and other operations on the particle node through the particle style parameters in the particle resource, that is, the attribute information of the particle itself, such as color, transparency, size, etc.

[0068] For example, Figure 3B The basis shown Figure 3A The particle special effects of the bound barrage rendered by the display style parameters set by the special effect configuration information in and the particle style parameters in the particle resource. In the figure, a is the firefly barrage, b is the fire barrage, c is the accelerating sound wave barrage, d is the blood recovery barrage, e is the snowing barrage, f is the ribbon barrage, g is the raining barrage, and h is the trajectory fire barrage when the trajectory is bound.

[0069] In the above data interaction method, through the particle system of Unreal Engine, particles are combined with the playback scene to achieve a rich variety of special effects. In the application process, you only need to change the particle configuration to easily achieve various unique particle effects. Based on the use of the particle system of Unreal Engine, you can bring more realistic and vivid visual effects to the playback scene, such as flames, water flows, etc. At the same time, you can also add unique bullet screen styles such as smear and glow to the bullet screen in the playback scene to enhance the viewing experience. By supporting the binding of particle special effects such as bullet screen, trajectory, window, etc., a variety of different visual effects and interactive experiences can be created.

[0070] In some embodiments, generating a particle node based on a particle resource includes: determining a prefabricated body with a particle component and instantiating the prefabricated body as a general node; loading effect attribute information in special effect configuration information and applying the effect attribute information to the general node to obtain a particle node.

[0071] In actual implementation, the particle system built based on the particle component in the Unreal Engine includes multiple particle nodes, and the nodes are usually generated by instantiating the prefab. Among them, the prefab is used to store some reusable scene objects, which can contain nodes, components, and data on components. The instances generated by the prefab can inherit the data of the template and have their own customized data modifications. Based on this, the specific process of generating particle nodes is as follows: Determine the prefab with the embedded particle component: During the development process, prefabs with particle components can be designed and made in advance. These prefabs usually contain the basic properties and behaviors of the particles themselves, such as the position, size, color, life cycle, etc. of the particles. Instantiate the prefab as a general node: At runtime, instantiate the prefabricated particle component as a general node as needed. This usually involves creating a new node and copying the properties and behaviors of the prefab to the node. Load the display style parameters in the special effect configuration information: The special effect configuration information usually contains various display style attributes about the particle special effects, such as the binding method of the particle special effects, the location area information, and the number, color, speed, size, etc. of particles adapted to the current playback scene. After instantiating the general node, load these display style parameters. Apply the actual value of the display style parameter to the general node to achieve the particle special effects adapted to the special effect configuration information. After the above steps, the general node is adjusted to a particle node based on the description of the display style parameters. The adjusted particle node can be displayed in the style described by the display style parameters in the special effect configuration information to achieve the corresponding particle special effects.

[0072] In this embodiment, the particle nodes are generated by instantiating the preforms. Due to the high reusability of the preforms, the cost of constructing the particle nodes can be effectively reduced, and the generation efficiency of the particle nodes can be improved.

[0073] In some embodiments, Figure 4 As shown, according to the binding method in the special effect configuration information, the target node corresponding to the scene element to be implemented with the particle special effect in the playback scene is determined, including steps 402 to 406. Among them:

[0074] Step 402: Obtain a node tree provided by the Unreal Engine to support the playback scene, where the node tree consists of a plurality of nodes having a parent-child relationship.

[0075] In actual implementation, Unreal Engine provides the concept of node tree, based on which various elements in the scene supported by Unreal Engine can be constructed and organized. The terminal can obtain a node tree adapted to the current playback scene by calling the relevant application program interface provided by Unreal Engine. Multiple nodes in the node tree have a parent-child relationship. In the playback scene based on Unreal Engine, each scene element has a corresponding node in the node tree.

[0076] Step 404: determine the scene elements in the playback scene on which the particle special effects are to be implemented according to the binding method in the special effects configuration information.

[0077] In actual implementation, the terminal reads the binding method for the particle node in the special effect configuration information, and determines the scene element to be implemented in the playback scene for the particle special effect. As mentioned above, the binding method includes at least window binding, track binding and bullet binding, wherein, when the binding method is window binding, the scene element to be implemented in the playback scene for the particle special effect is the window, when the binding method is track binding, the scene element to be implemented in the playback scene for the particle special effect is the track, and when the binding method is bullet binding, the scene element to be implemented in the playback scene for the particle special effect is the bullet.

[0078] Step 406: Search the node corresponding to the scene element in the node tree as the target node.

[0079] In actual implementation, after determining the scene elements for the particle effects to be implemented, the terminal traverses the node tree corresponding to the playback scene, and determines the node corresponding to the scene elements for the particle effects to be implemented as the target node for binding the particle node. In this way, in the process of playing multimedia data, the particle effects described by the particle node are displayed on the scene elements indicated by the target node. Among them, in the process of traversing the node tree, it involves recursive traversal of the node tree, checking the attributes and types of each node to determine whether they meet the requirements in the special effect configuration information. Once the target node is determined, the position area information of the particle node configured by the special effect configuration information can be further converted to obtain the relative position area information relative to the target node, and then the parent node attribute of the particle node is modified to point to the target node, thereby realizing the binding of the particle.

[0080] In this embodiment, the target node is determined by traversing the node tree, which can improve the search efficiency of the target node.

[0081] The binding method of the particle node is explained, and the binding method includes at least one of the following: window binding, track binding, and barrage binding. Among them, window binding refers to binding the particle node to the scene element used to indicate the window in the playback scene, track binding refers to binding the particle node to the scene element used to indicate the track in the playback scene; barrage binding refers to binding the particle node to the scene element used to indicate the barrage in the playback scene. Among them, track binding can also be divided according to the track type, and the track type can at least include the barrage running track and the custom track. Next, each binding method is explained in detail.

[0082] With respect to window binding, in some embodiments, when the binding method is window binding, the method for determining the scene element for implementing the particle effects in the playback scene is as follows: when the binding method is window binding, determine the first layer for rendering the multimedia data; use the second layer above the first layer as the scene element for implementing the particle effects in the playback scene; the second layer is a transparent layer.

[0083] In actual implementation, when the playback scene of multimedia data is realized based on the Unreal Engine, the Unreal Engine provides at least two layers for the playback scene: a first layer for rendering multimedia data and a second layer for displaying other data except multimedia data. The other data here may include bullet screens in the playback scene, various special effects, etc. In order not to affect the playback of multimedia data in the first layer, the second layer is usually covered on the first layer in the form of a transparent layer. Generally, the size of the second layer is the same as that of the first layer, and it can adapt to different terminal screens. When the binding method is window binding, the terminal can use the second layer as a scene element for the particle effects to be implemented in the playback scene, that is, the particle effects are displayed on the second layer.

[0084] For example, Figure 5 As shown, the application client for playing multimedia data provided by the terminal includes a first layer for playing multimedia data, and a second layer for displaying other data except the multimedia data.

[0085] In this embodiment, when the binding method of the particle special effect is window binding, the second layer different from the first layer provided by the Unreal Engine for the playback scene is used as the scene element for the particle special effect to be implemented. This can simplify the binding operation of the particle special effect and improve the binding efficiency while ensuring that the multimedia information playback process is not affected.

[0086] With respect to track binding, in some embodiments, when the binding method is track binding, the method for determining the scene element for particle effects to be implemented in the playback scene is as follows: when the binding method is track binding, determine the track type of the track to be bound; if the track type is a bullet screen running track, use the bullet screen running track as the scene element for particle effects to be implemented in the playback scene.

[0087] In actual implementation, when the binding method is track binding, the scene element that matches the track type can also be determined according to the different track types. The track type can at least include a bullet screen running track and a custom track. The bullet screen running track in the playback scene usually refers to the area from the rightmost to the leftmost part of the playback interface. Therefore, when the track type is a bullet screen running track, the terminal can directly use the bullet screen running track as the scene element for the particle special effects to be implemented in the playback scene. When the track type is a custom track, the terminal first generates a new track in the playback scene based on the path information of the custom track, and then uses the new track as the scene element for the particle special effects to be implemented. After the particle node is bound to the track, the effect displayed in the playback interface can be that the particle special effects are displayed at the head of the corresponding track, that is, the particles move on the bullet screen running track.

[0088] In this embodiment, when the trajectory type is determined to be a bullet screen running trajectory, the existing trajectory can be directly used as the trajectory of the particle special effect to be implemented without reading other trajectory information again, which can improve the determination rate of the trajectory of the particle special effect to be implemented.

[0089] In some embodiments, when the binding method is track binding, if the track type is a custom track, the method of determining the scene element for implementing the particle special effects in the playback scene may also include: if the track type is a custom track, obtaining the path information of the custom track from the special effects configuration information; parsing the path information to obtain track coordinate information suitable for the playback scene; generating a custom track based on the track coordinate information, and using the custom track as the scene element for implementing the particle special effects in the playback scene.

[0090] In actual implementation, when the trajectory type is a custom trajectory, the terminal reads the path information of the custom trajectory, wherein the path information may be an SVG path, and the definition form of the SVG path is as follows: <svg> <path d=""×××，××，×××”">< / path> < / svg> , where the path element is used to describe the path. The terminal parses the path information to obtain the trajectory coordinate information adapted to the playback scene.

[0091] For example, Figure 6 As shown, Figure 6Number 1 in the figure shows a custom SVG path, and the display effect of the SVG path is a heart-shaped track. The terminal parses the path element in the SVG path to obtain a heart-shaped track as shown in number 2 in the figure. When the particle effects are displayed in the playback scene, the corresponding particle effects will be displayed along the heart-shaped track.

[0092] In this embodiment, the custom trajectory is used as a scene element for the particle special effects to be implemented in the playback scene, which can enrich the expression form of the particle special effects from the overall running trajectory level.

[0093] With respect to the barrage binding, in some embodiments, when the binding method is barrage binding, the method for determining the scene element for implementing the particle special effects in the playback scene is as follows: when the binding method is barrage binding, obtain the barrage within the time period when the special effects are effective; use at least a part of the barrage within the time period when the special effects are effective as the scene element for implementing the particle special effects in the playback scene.

[0094] In actual implementation, the special effect effective time period refers to the time interval from the first time point when the particle special effect event is triggered to display the particle special effect to the second time point when the particle special effect is hidden. The terminal determines the time period when the special effect is effective from the special effect configuration information, such as Figure 3A The start time attribute and the end time attribute shown in are the attributes used to determine the time period for the special effect to take effect. Then, all bullet comments for the current multimedia data within the time period for the special effect to take effect are obtained from the server or database. And according to the screening method in the special effect configuration information, at least a part of the bullet comments are screened from all the bullet comments, and the screened bullet comments are used as the scene elements for the particle special effects to be implemented in the playback scene. For example, the bullet comments can be screened according to the content, sender, timestamp and other attributes.

[0095] In this embodiment, when the binding method is barrage binding, the special effect effective time period is used as the constraint information for filtering at least a part of the barrage, which can reduce the time for the terminal to pull the barrage from the server and the consumption of computing resources, thereby improving the efficiency of barrage pulling and further improving the efficiency of barrage screening.

[0096] After determining the target node corresponding to the particle node, the terminal binds the particle node to the target node. The specific implementation process of binding the particle node to the target node is described below. Figure 7 As shown, binding a particle node to a target node includes steps 702 to 706. Among them:

[0097] Step 702, determining the position area information of the particle node from the special effect configuration information, where the position area information includes the position information of the particle emitter and the area information of the particle emission.

[0098] In actual implementation, after determining the target node for binding the particle node, in order to ensure the accuracy of the position of the particle node relative to the target node during the rendering process of the particle node, that is, to display the particle node at the correct position of the target node, the terminal can read the preset position area information of the particle node from the special effect configuration information. Among them, the position area information includes the position information of the particle emitter and the area information of the particle emission. Usually, when presetting the position area information of the particle node, since the size of the bound target node is uncertain, the area position information is set in the form of percentage coordinates, that is, the area determined by the two coordinate points (0,0) and (1,1) is used as the representation area of ​​the target node. Based on this, in the form of percentage coordinates, the position of the particle emitter can be set to (0.5,1) and the coordinates of the particle emission area can be set to (0.5,0) and so on.

[0099] Step 704 , based on the position information of the scene element on which the particle special effect is to be implemented, coordinate conversion is performed on the position area information of the particle node to obtain relative position area information.

[0100] In actual implementation, the terminal determines the position information of the scene element to be implemented with the particle special effect. If the scene element to be implemented with the particle special effect is a window for playing multimedia data, the position information corresponding to the window can be determined according to the resolution of the terminal screen, and the position area information of the particle node configured in the special effect configuration information is converted by coordinates to obtain the relative position area information relative to the scene element. Among them, the coordinate conversion can be achieved by operations such as translation, rotation or scaling to ensure that the particle node can be correctly bound to the scene element.

[0101] For example, Figure 8 As shown, the position conversion method configured in the special effect configuration information is conversion based on percentage coordinates, and the scene element to be implemented with the particle special effect is the window, that is, the window node is the parent node of the particle node. Figure 8 The position area information of the particle node in the special effect configuration information shown by number 1, wherein the position information of the particle emitter configured in the special effect configuration information is the center point (0.5, 0.5, 0, 0), the first two elements are the percentage coordinates relative to the parent node, and the last two elements are the offset values ​​in the X direction and the Y direction. The change in the area information of the particle node configured in the special effect configuration information relative to the parent node is (0.2, 0.1), and the change in the area information indicates the change in the particle emitter in the X direction and the Y direction. The change in the positive and negative directions of the X axis is 0.2, and the change in the positive and negative directions of the Y axis is 0.1. Therefore, the emission area information of the particles that can be determined is the area information determined by the four points [(0.3, 0.4), (0.7, 0.4), (0.7, 0.6), (0.3, 0.6)]. Figure 8 Number 2 shows the position area information of the particle node under the actual playback screen. The resolution of the playback screen is 1920*1080. After percentage coordinate conversion, the relative position information of the particle emitter in the particle node is (960, 540, 0, 0). The change in the area information of the particle node relative to the parent node is (384, 118). The relative position area information is the area information determined by the following four points [(576, 432), (1344, 432), (1344, 658), (576, 648)].

[0102] Step 706: bind the particle node to the target node based on the bound particle node and the relative position area information.

[0103] In actual implementation, before rendering the particle node, the terminal first determines the relative position area information of the particle node relative to the target node, and accurately determines the display position of the particle node in the child target node. Then, by modifying the parent node attribute of the particle node, the parent node attribute is modified to the target node, and the binding between the particle node and the target node is completed. In this way, it can ensure that the terminal renders the particle node at the accurate position of the target node. Figure 8 As shown, the particle node is bound to the center point (960, 540) of the target node.

[0104] In this embodiment, the position area information of the particle node in the special effect configuration information is converted into relative position area information adapted to the target node by means of coordinate conversion, which can improve the position accuracy of the particle node in the target node.

[0105] In some embodiments, the scene elements include barrages, and based on the position information of the scene elements on which the particle special effects are to be implemented, the position area information of the particle nodes is coordinate-converted to obtain relative position area information, including: determining the barrage area associated with the barrage based on the width and height information of the barrage in a rendered state; and performing a position conversion on the position area information of the particle nodes relative to the barrage area to obtain relative position area information.

[0106] In actual implementation, according to the life cycle of the barrage and the loading process of the barrage, it can be known that the barrage node corresponding to the barrage in the Unreal Engine does not include the width and height information of the barrage during the creation stage. Only after rendering one frame can its width and height be obtained. When determining the relative position area information of the particle node, the width and height of the barrage are needed as the basis. Therefore, when the scene element to be implemented with the particle special effects is determined to be the barrage, the terminal can only obtain the width and height information of the barrage when the barrage state is rendered.

[0107] In order to facilitate the understanding of the status of the bullet screen, the life cycle of the bullet screen is explained below. Fig. 9 , Fig. 9 The figure shows the life cycle of the bullet screen and the corresponding loading process. As shown in the figure, the entire life cycle of the bullet screen can be divided into the following stages: data sending, creating bullet screen nodes, initializing the bullet screen position, preparing to display on the screen, starting movement, and destroying the bullet screen. In addition, the bullet screen can be divided into four parts: waiting state Waiting, waiting for rendering state WaitForRender, rendering state Render, and destroying state Destory, depending on whether the bullet screen is rendered and displayed on the screen.

[0108] Specifically, in the data sending stage, when the terminal receives the barrage fragment data sent by the server, it will immediately convert the data into the barrage data class in the playback scene and save it in the memory. In the stage of creating a barrage node, the terminal pre-buries various types of barrage prefabricated bodies. When creating a barrage node, the barrage prefabricated body is obtained and instantiated as a barrage node. The playback node corresponding to the player in the playback scene continuously calls back the playback progress to the barrage component. When the playback progress reaches the time point when the specified barrage needs to be on the screen, the barrage data is retrieved. When the barrage is on the screen, the terminal first selects an idle track and designates the track as the track where the barrage needs to be on the screen. Then, the corresponding type of prefabricated body is instantiated as a barrage node through the barrage assembly class. At the same time, the data of the barrage node will be initialized, including filling text, adding images, drawing backgrounds, etc., and the barrage status will be changed to Waiting. When the position of the bullet screen is initialized, the status of the bullet screen is Waiting. Due to the characteristics of the Unreal Engine, the width and height attributes of the node need to be rendered before they can be obtained, so the terminal will place the bullet screen node outside the screen for rendering in advance, and change the status of the bullet screen to WaitingForRender. When the bullet screen is ready to be displayed, the status of the bullet screen is WaitingForRender. In order to prevent the bullet screen from overlapping with other bullet screens, the terminal waits to ensure that the current bullet screen will not overlap with the previous bullet screen, then puts the current bullet screen on the track and changes the status of the bullet screen to Render.

[0109] At the beginning of the movement of the bullet screen, the bullet screen state is Render state. The bullet screen component associated with the bullet screen saves the time the bullet screen has been running, which increases every frame. After calculating the speed of the bullet screen, the distance the bullet screen has run can be calculated by (speed * time), and the coordinate position of the bullet screen at this time can be inferred. This calculation logic is calculated every frame, and the position of the bullet screen is constantly set to achieve the effect of movement.

[0110] In the bullet discussion destruction phase, when the bullet discussion has survived for longer than the life time, the bullet discussion node will be destroyed and the bullet discussion status will be changed to Destory.

[0111] Based on the above characteristics of the bullet screen, when the particle node is bound to the bullet screen, it is necessary to ensure that the status of the bullet screen is Render before binding. When the bullet screen status changes from WaitForRender to Render, the correctness of the width and height of the bullet screen can be guaranteed. Therefore, when the bullet screen status is Render, the position of the particle node relative to the bullet screen node can be accurately calculated.

[0112] In actual implementation, when the binding method is bullet screen binding, the terminal determines that the scene element of the particle special effect to be implemented is the bullet screen. At the same time, the relative position area information of the particles in the binding area is determined by the binding area of ​​the particle special effect configured in the special effect configuration information in the bullet screen. If the binding area is at least a part of the bullet screen area to which the bullet screen belongs, the terminal can determine the relative position area information of the particle node in the bullet screen according to the percentage calculation method based on the position area information of the particle node configured in the special effect configuration information.

[0113] For example, Fig. 10A As shown in the figure, numbers ab show in the display schematic diagram of at least a part of the area of ​​the particle node bound to the barrage, and number a shows the partial area of ​​the bound barrage area starting from the starting position; the starting position of the bound barrage area shown by number b, that is, only the position information, there is no area, therefore, it is expressed as emitting particles from a point; number c shows the entire area of ​​the bound barrage area, that is, the particle emitter can emit particles in the entire area.

[0114] In this embodiment, by determining at least a portion of the barrage area as the binding position of the particle node in the barrage node, the display style of the particle special effects based on the existing barrage can be enriched. At the same time, because only the coordinate conversion based on the granularity of the barrage area needs to be performed, the efficiency of determining the particle position can be improved.

[0115] In some embodiments, the scene elements include barrages, and based on the position information of the scene elements on which the particle special effects are to be implemented, the position area information of the particle nodes is coordinate-converted to obtain relative position area information, including: determining a specified character from the barrage in a rendered state, and determining the coordinate information of the specified character relative to the barrage; based on the coordinate information, performing a position conversion on the position area information of the particle nodes to obtain relative position area information.

[0116] In actual implementation, when the binding method is bullet screen binding, if the set binding area is determined based on the bullet screen content, that is, the binding position of the particle node in the target node is the character at the specified position in the bullet screen. The terminal first detects whether the bullet screen of the particle node to be bound is in the rendered state. When the bullet screen is in the rendered state, the terminal can determine the height and width of the bullet screen, and determine the coordinate information of the specified character in the bullet screen relative to the bullet screen, and perform position conversion on the position area information recorded in the special effect configuration information, and convert the position area information into the relative position area information in the coordinate system of the bullet screen.

[0117] For example, Fig. 10B As shown, the designated character of the bullet comment to be bound is set as the first character of the bullet comment, and the area corresponding to the first character of the corresponding bullet comment is used as the actual area for determining the relative position area information of the particle node, and the particle emitter emits particles within the area.

[0118] In this embodiment, by determining the relative position area information of the particle node based on the coordinate conversion of the specified characters in the bullet screen, the display style of the particle special effects can be further enriched.

[0119] In some embodiments, the method further includes: obtaining animation attribute information in the special effect configuration information for describing the target animation executed by the particles; during the particle special effect display process, controlling the movement of each particle frame by frame according to the animation attribute information to achieve the target animation.

[0120] In actual implementation, in order to meet more usage scenarios and achieve more effects, the configuration of attribute animation is supported in the particle configuration. That is, the animation attribute information can also be set for the particles in the particle special effects in the current playback scene. Among them, the configuration of attribute animation is essentially to configure a structured data, which can be the data of the Json structure. The structured data mainly contains the total number of frames (totalFrame) of each loop and a limited number of attribute change structures (frameChanges). Each structure has five fields: the start frame number (startFrame), the interval frame number (frameInterval), the change attribute (attribute), the change type (changeType) and the change amount (value). Among them, the start frame number startFrame indicates the frame in which a loop starts to take effect, the interval frame number frameInterval indicates that this change is changed every few frames, the change attribute attribute corresponds to the attribute name in the particle configuration, the change type changeType indicates whether this change is increased, decreased or directly replaced on the original basis, and the change amount value indicates the value of each change. The specific meaning of the update function related to the animation property can be that within an execution cycle, starting from the start frame number, for the changed properties of the particles in the target frame determined based on the interval frame number, the adjustment operation indicated by the change amount is performed according to the change type to obtain the animation performed by the particle. From the special effect configuration information, the preset animation properties for executing particle animation are extracted, such as the angle, speed, acceleration, life cycle, color change, size change, particle map, etc. of the particle. And the target particle update function is set to update the state of the particle in each frame. In each frame of the Unreal Engine, the target particle update function is called. This function can update the state of each particle according to the animation property information. For example, the new position of the particle can be calculated based on its speed and current position. At the same time, the speed of the particle can also be adjusted according to other properties (such as acceleration). Similarly, the life cycle, color change, size change, etc. of the particle can also be processed. For example, as the life cycle of the particle decreases, you can gradually change its color or size.

[0121] For example, taking the implementation of the animation property of particle spin as an example, the Json structure is shown in the following fragment 1:

[0122] {

[0123] "totalFrame": 60,

[0124] "frameChanges": [

[0125] {

[0126] "attribute": "angle",

[0127] "value": "6",

[0128] "changeType":"plus",

[0129] "startFrame": 0,

[0130] "frameInterval": "1"

[0131] } ]

[0133] }

[0134] The above structure indicates that the particle attribute animation has 60 frames in one cycle (totalFrame is 60), starting from frame 0 (startFrame is 0), and each frame (frameInterval is 1) will add 6 to the "angle" attribute on the original basis. Through the above animation attributes, the particle spin effect is achieved.

[0135] Continuing with the example of particle map replacement, the corresponding Json structure is shown in the following fragment 2:

[0136] {

[0137] "totalFrame'": 360,

[0138] "frameChanges": [

[0139] {

[0140] "attribute": "textureImageData" ,

[0141] "value":" Figure 1 ",

[0142] "changeType" :"replace",

[0143] "startFrame":0,

[0144] "frameInterval": "Infinity"

[0145] }.

[0146] {

[0147] "attribute": "textureImageData" ,

[0148] "value":" Figure 2 ",

[0149] "changeType":"replace" ,

[0150] "startFrame": 180,

[0151] "frameInterval": "Infinity"

[0152] }

[0153] }

[0154] }

[0155] The above attribute animation data shows that the animation corresponds to a cycle of 360 frames. At frame 0, the particle map is converted to a map. Figure 1 The style shown, at frame 180, the texture is converted to a texture Figure 2 Style, the specific display effect is as follows Fig.11 As shown in (a) and (b), the particle texture will be converted to a gold coin style at the 0th frame, and the texture will be converted to a normal sprite style at the 180th frame.

[0156] In this embodiment, the richness of particle special effects is improved and the application scenarios of particle special effects are expanded by configuring attribute animation.

[0157] To explain the data interaction method provided by the present application in detail, an embodiment is used for explanation below. In this embodiment, the multimedia information is video information, and the particle special effects are applied to the playback scene of the video information. In this playback scene, the bullet screen particle special effects are displayed. This method is implemented based on the particle system of the game engine, and by combining particles with bullet screen scenes, a rich and diverse bullet screen gameplay is achieved.

[0158] In the related art, when displaying barrage, terminal text, pictures, animations, etc. are used to display barrage, such as by adding frame animations on the left and right sides of the barrage. In addition, playing special effects on the window is also achieved by looping the video frame animation. In the related art, whether it is binding barrage or binding window special effects, it is a preset, fixed visual effect, which is achieved through pre-designed animations or image sequences. Randomness and dynamic changes cannot be achieved, and there is a lack of realism and vividness. In addition, designers need to design multiple animations for different screen sizes and different scenes, resulting in poor reusability of special effects; at the same time, since ordinary barrage mainly relies on preset animations or image sequences, it is impossible to adjust its special effect parameters or interact with it in real time, resulting in poor interactivity.

[0159] Based on this, the data interaction method in the embodiment of the present application introduces the particle system of the Unreal Engine (such as a game engine) into the video playback scene, combines the real particle effect with the bullet screen, and generates a visual effect with randomness and dynamic changes. Since the particle system in the true sense is based on dynamic simulation, it has the characteristics of variable properties, dynamic simulation, randomness and controllability, so it can generate more realistic and vivid visual effects. At the same time, the particle effect is highly reusable, and only some parameters need to be replaced to meet the use of various scenes. And the same material can support different forms of expression, and binding has different effects on different targets.

[0160] In actual implementation, the method for realizing the embodiment of the present application can generally be divided into four steps: particle resource design, management station configuration, server-side distribution, and terminal display. Among them, particle resource design is essentially to configure the properties of the particles to obtain the particle resources mentioned above. The management station configuration is essentially to configure the association between particle special effects and video information, that is, to obtain the special effects configuration information mentioned above. The server-side distribution is essentially to request relevant barrage and special effects information from the server in the playback scene; the terminal display is essentially to display barrage and particle special effects in the process of playing video information, that is, when the time point when the user watches the video reaches the management station configuration interval, the configured particle special effects will be displayed at the corresponding position in the configuration.

[0161] Regarding particle resources, you need to configure the properties of particles before using them. Usually, you can store the properties of a particle style in a certain form as a particle resource (plist) file, and then you can directly read the file to implement a preset particle style. Common particle properties include but are not limited to the following information: particle map, the style of a single particle display, usually a png format image; the maximum number of particles, the maximum number of particles that can exist at the same time in a single particle emitter, when the number of particles reaches this value, the particle emission will stop; duration, the active duration of the particle emitter; emission frequency, the number of particles that a particle emitter can emit per second; the survival time and range of a single particle; particle size and range; particle spin angle and range. The above properties are particle resource configurations and can be stored in a plist file as a reusable particle resource.

[0162] During the configuration of the management console, you can also set attribute configurations that are adapted to the playback scene. The relevant attribute configurations may include:

[0163] 1) Binding method: set the binding window, track, bullet screen, etc.

[0164] 2) Position area attribute: bind the particle emitter to the specified position of the bullet screen and window, as well as the area information of particle emission.

[0165] 3) Whether the particle follows: If you choose to follow, the particle will move relative to the bullet screen. If you choose not to follow, the particle will only move under its own influence after being emitted, thus achieving the effect of trailing shadows.

[0166] 4) Attribute animation: You can animate at least one specified attribute of a particle.

[0167] 5) Custom SVG path: You can customize the trajectory of the emitted particles, so that you can achieve some particle effects running around the bullet screen.

[0168] The overall process of particle effect loading is described below. Fig.12 As shown, the overall process includes a video loading stage and a video playing stage.

[0169] The execution process in the video loading stage is as follows: the terminal that plays the video data starts to load the video data and requests the special effect configuration information associated with the video data from the server. The server sends a special effect list, and the terminal saves the special effect configuration information. The special effect configuration information includes the start time and end time of the special effect, which constitute the special effect effective time period; then the terminal performs pre-download of particle special effect resources. In addition, the terminal can download related resources immediately after receiving the response operation from the server to prevent the delay in the screen due to the time-consuming resource download.

[0170] The execution process in the video playback stage is as follows: the terminal (based on the game engine) determines whether the playback progress of the video hits the special effect effective time period in the special effect configuration information. If the playback progress hits the special effect effective time period, it indicates that the relevant particle special effects need to be displayed. At this time, the terminal instantiates the particle node and applies the particle configuration, that is, the attribute configuration of the particle node is performed according to the particle resource configuration in the special effect configuration information to obtain a particle node adapted to the video data. Finally, bind the target node, that is, bind the particle node to the target node. The target node is the node of the particle special effect to be implemented and is the parent node of the particle node. Render the particle nodes bound to the target node to display the particle effect during the playback of the video data. If the playback progress does not hit the special effect effective time period, remove all particle nodes generated by the configuration.

[0171] When the playback progress hits the time period when the special effects are effective, it indicates that the particle special effects need to be displayed at this time. At this time, the following logical processing begins: First, create a particle node. Whether it is bound to a bullet screen, a window, or a track, particle resources need to be loaded. In actual implementation, the terminal will pre-embed a prefab with a particle component, and the terminal will instantiate the prefab as a Node node in the game engine. Secondly, configure the application. The particle attribute configuration resource is a plist file, which represents a certain style of particle effect attribute configuration. First, load the particle attribute configuration resource into memory, and then apply the configuration attribute to the particle Node. In addition, you can also customize the configuration according to the actual application. Finally, bind the particle node to the target node.

[0172] It should be noted that before rendering the particle node corresponding to the video data, the position area information of the particle node relative to the target node (i.e., the relative position area information in the previous text) is first determined, and then the parent node of the particle node is modified to be the target node, and the particle node is rendered at the target position of the target node. The determination of the target position relative to the target node is related to the binding method of the particle node, and different binding methods can determine different target nodes.

[0173] The following is an explanation of the binding process based on different binding methods. In actual implementation, the binding methods include at least one of the following: window binding, track binding, or bullet screen binding. Window binding is to bind a window or track. At this time, you only need to find the corresponding window node or track node from the node tree provided by the game engine for the playback scene, determine the corresponding position through coordinate conversion, and then add the particle node to the target node.

[0174] Specifically, when the binding mode is window binding, the corresponding window node is searched from the node tree provided by the game engine for the playback scene. The position area information of the particle node configured in the special effect configuration information is converted according to a preset conversion mode to obtain the relative position area information of the particle node in the window node, wherein the conversion mode can be percentage coordinates, that is, the percentage coordinates of the area and position of the particle node are converted into relative coordinates in the window.

[0175] When the binding method is track binding, the particle node is bound to the corresponding track node. The track of the particle node to be bound can be an existing track in the playback scene, such as the bullet screen running track. At this time, you only need to obtain the corresponding track node, and then bind the particle Node to the head of the track Node. The particle Node moves in the bullet screen running track. If the track of the particle node to be bound is a custom SVG path, you need to read the SVG path first, convert it into coordinate information that is compatible with the logical processing process of the game engine, build a corresponding new track node, and bind the particle Node to the track Node.

[0176] In the case where the binding mode is bullet screen binding, as described above, after the terminal determines that the bullet screen is in the effective time period of the particle special effect, when it determines that the rendering state of the bullet screen is rendered, it performs coordinate conversion on the position area information of the particle node according to the width and height information of the bullet screen to obtain the relative position area information relative to the bullet screen. Based on the bullet screen in the rendered state, the correctness of the width and height of the bullet screen can be guaranteed, thereby ensuring the accuracy of the position information of the particle node.

[0177] After determining the target node corresponding to the particle node in the playback scene, the terminal determines the position and emission area of ​​the particle node in the target node according to the conversion method for the position area information configured in the special effect configuration information. Among them, for the position and emission area of ​​the particle Node, the specific rules are as follows: if the conversion method configured in the special effect configuration information is percentage coordinates, the position area information of the particle node is converted into specific relative coordinates (i.e., relative position area information) relative to the barrage through the width and height attributes of the barrage; if the conversion method configured in the special effect configuration information is the entire barrage, it means that the width and height of the emission area of ​​the particle node are the width and height of the entire barrage; if the conversion method configured in the special effect configuration information is based on the characters at the specified position in the barrage, first determine the coordinates of the characters at the specified position relative to the barrage, and then perform coordinate conversion on the position area information of the particle node to obtain the relative position area information of the characters relative to the specified position.

[0178] In actual implementation, you can also configure attribute animation for particles in the particle effects in the playback scene. As shown above, the configuration information of attribute animation is actually a json structure of data, which includes the total number of frames in each loop (totalFrame) and at least one attribute change structure (frameChanges). As mentioned above, each attribute change structure frameChanges includes at least five fields: start frame startFrame, interval frame frameInterval, change attribute attribute, change type changeType and change value.

[0179] In actual implementation, if the special effects configuration information associated with the playback scene is set with attribute animation for particles, the Unreal Engine that supports the playback scene will bind a component for processing attribute animation, ParticleAttributeAnimationComponent, in the particle node. This component is adapted to the specific Unreal Engine and can be used to save the relevant data of the attribute animation and the number of frames currently played in the attribute animation. In actual applications, a class dedicated to processing particle attribute animation can be set for the playback scene. This class can accept callbacks for each frame and execute the following for all particle components carrying ParticleAttrubuteAnimationComponent: Fig.13 The logic shown is used to implement the property animation function.

[0180] Among them, combined Fig.13 , explain the specific implementation steps of attribute animation. In each frame callback process, execute step 1) to determine whether the current frame number is greater than the starting frame number in the attribute change structure. If so, execute step 2) to continue to determine whether the current frame number is a multiple of the interval frame number in the attribute change structure. If so, execute step 3) to perform operations on the attributes in the particle node that are adapted to the change attributes in the attribute change structure, that is, modify the change attribute attribute set by the attribute change structure frameChanges according to the change type changeType, and modify the value to the change value. Then, execute step 4) to enter the frame callback process of the next frame of the current frame number, that is, add 1 to the current frame number, and take the remainder of totalFrame in the attribute change structure, that is, enter the next frame callback. Among them, if the execution results of step 2) and step 3) are no, directly enter the next frame callback.

[0181] For example, according to the property animation structure shown in the above-mentioned fragment 1, the effect of particle spinning can be achieved. In the property animation described in it, one cycle has 60 frames (totalFrame). Starting from the 0th frame, each frame will add 6 (value is 6) to the original "angle" property of the particle according to the change type (changeType is plus). According to the property animation structure shown in the above-mentioned fragment 2, the property animation achieved is: one cycle has 360 frames, and at the 0th frame, the particle map is converted to a map. Figure 1 The style shown, at frame 180, the texture is converted to a texture Figure 2 style.

[0182] By applying the embodiments of the present application, the following beneficial effects can be achieved:

[0183] 1) Rich visual experience: Since the real particle system has the characteristics of dynamic simulation, randomness and controllability, it can generate more realistic and vivid visual effects. Applying it to the barrage display in the playback scene can make the barrage in the playback scene have richer visual effects and improve the audience's viewing experience.

[0184] 2) Reduce labor costs: Particle resource plist files can be reused, eliminating the need to repeatedly design resources.

[0185] 3) Innovative interactive methods: The bullet screen with real particle system can provide more interactive methods for viewers. For example, viewers can trigger special particle effects by sending specific instructions or expressions. This will help improve the user stickiness of live broadcast platforms and video websites and attract more users to participate in the interaction.

[0186] 4) Improve community atmosphere: The bullet screen of the real particle system can create a more vivid and interesting interactive atmosphere, which helps to improve the community atmosphere and increase user participation and stickiness. At the same time, it can also provide users with more opportunities to show their personality and talents and cultivate a sense of belonging.

[0187] 5) Commercial value: Since the bullet screen of the real particle system has high commercial value, it can be used as an innovative marketing tool. For example, customized particle effects can be used to provide advertisers with a more attractive advertising display method, or unique particle effects can be provided to paying users as a member privilege, etc. This will help improve the profitability and market competitiveness of the product.

[0188] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0189] Based on the same inventive concept, the embodiment of the present application also provides a data interaction device for implementing the data interaction method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more data interaction device embodiments provided below can refer to the limitations on the data interaction method above, and will not be repeated here.

[0190] In an exemplary embodiment, Fig.14 As shown, a data interaction device 1400 is provided, including: an acquisition module 1410, a generation module 1420, a determination module 1430 and a rendering module 1440, wherein:

[0191] The acquisition module 1410 is used to acquire multimedia data, and special effect configuration information and particle resources associated with the multimedia data.

[0192] The generation module 1420 is used to generate a particle node according to the particle resource if a particle special effect event is triggered in the playback scene for the multimedia data.

[0193] The determination module 1430 is used to determine the target node corresponding to the scene element on which the particle special effect is to be implemented in the playback scene according to the binding method in the special effect configuration information, and bind the particle node to the target node.

[0194] The rendering module 1440 is used to perform rendering based on the bound particle nodes, so as to display the particle special effects according to the display mode indicated by the special effect configuration information during the presentation of the multimedia data.

[0195] In some embodiments, the generation module is also used to determine a prefabricated body with a particle component and instantiate the prefabricated body as a general node; load the effect attribute information in the special effect configuration information, and apply the effect attribute information to the general node to obtain a particle node.

[0196] In some embodiments, the determination module is also used to obtain a node tree provided by the Unreal Engine to support playback scenes, the node tree consisting of multiple nodes with parent-child relationships; determine the scene elements in the playback scene where the particle special effects are to be implemented according to the binding method in the special effects configuration information; and search the node corresponding to the scene element from the node tree as the target node.

[0197] In some embodiments, the determination module is also used to determine the first layer for rendering multimedia data when the binding method is window binding; the second layer above the first layer is used as a scene element for particle special effects to be implemented in the playback scene; and the second layer is a transparent layer.

[0198] In some embodiments, the determination module is further used to determine the trajectory type of the trajectory to be bound when the binding method is trajectory binding; if the trajectory type is a bullet screen running trajectory, the bullet screen running trajectory is used as a scene element for the particle special effects to be implemented in the playback scene.

[0199] In some embodiments, the determination module is also used to obtain path information of the custom trajectory from the special effect configuration information if the trajectory type is a custom trajectory; parse the path information to obtain trajectory coordinate information suitable for the playback scene; generate a custom trajectory based on the trajectory coordinate information, and use the custom trajectory as a scene element for the particle special effects to be implemented in the playback scene.

[0200] In some embodiments, the determination module is also used to obtain the barrage within the special effect effective time period when the binding method is barrage binding; and use at least a part of the barrage within the special effect effective time period as the scene element for the particle special effect to be implemented in the playback scene.

[0201] In some embodiments, the determination module is also used to determine the position area information of the particle node from the special effect configuration information, the position area information including the position information of the particle emitter and the area information of the particle emission; based on the position information of the scene elements on which the particle special effects are to be implemented, the position area information of the particle node is coordinate-converted to obtain the relative position area information; correspondingly, the rendering module is also used to render based on the bound particle nodes and the relative position area information.

[0202] In some embodiments, the scene elements include barrage, and the determination module is also used to determine the barrage area associated with the barrage based on the width and height information of the barrage in a rendered state; the position area information of the particle node is converted relative to the barrage area to obtain relative position area information.

[0203] In some embodiments, the scene elements include barrage, and the determination module is also used to determine the specified character from the barrage in the rendered state, and determine the coordinate information of the specified character relative to the barrage; based on the coordinate information, the position area information of the particle node is converted to obtain relative position area information.

[0204] In some embodiments, the rendering module is further used to obtain animation attribute information in the special effect configuration information for describing the target animation executed by the particles; during the particle special effect display process, the movement of each particle is controlled frame by frame according to the animation attribute information to achieve the target animation.

[0205] In some embodiments, the special effect configuration information includes a special effect effective time period, and the rendering module is also used to determine that a particle special effect event is triggered for the playback scene of the multimedia data when the playback progress of the multimedia data is within the special effect effective time period; or, when a preset interactive operation is detected in the playback scene of the multimedia data, it is determined that a particle special effect event is triggered for the playback scene of the multimedia data.

[0206] Each module in the above data interaction device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0207] In some embodiments, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Fig.15 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data interaction method is implemented.

[0208] Those skilled in the art will understand that Fig.15 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0209] In some embodiments, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0210] In some embodiments, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0211] In some embodiments, a computer program product is provided, including a computer program, which implements the steps in the above-mentioned method embodiments when executed by a processor.

[0212] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0213] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0214] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0215] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data interaction method, characterized in that: The method comprises: Acquire multimedia data, special effect configuration information and particle resources associated with the multimedia data, and establish an association relationship between the multimedia data and the particle resources in the special effect configuration information; If a particle special effect event is triggered in the playback scene for the multimedia data, a particle node is generated according to the particle resource, and the particle node is a node corresponding to a particle in a particle system provided in the Unreal Engine; the particle special effect is bound to a scene element in the playback scene, and after the scene element is bound, the particle special effect is displayed in a corresponding position and area of ​​the scene element; According to the binding method in the special effect configuration information, the target node corresponding to the scene element for implementing the particle special effect in the playback scene is determined, and the particle node is bound to the target node. The target node corresponding to the target scene element for implementing the particle special effect is determined in the node tree provided by the Unreal Engine for the playback scene. The particle node is bound to the target node by setting the position attribute of the particle node so that the position of the particle node is consistent with that of the target node. The scene elements in the playback scene include at least two of the window elements, the track elements, and the bullet screen elements. In the playback scene based on the Unreal Engine, each scene element has a corresponding node in the node tree. In the process of binding the particle node to the target node, the position of the particle node is adjusted based on the position information of the scene element for implementing the particle special effect. The coordinate transformation of the area information is performed to obtain the relative position area information with the target node as the reference, so as to display the particle node at the target position of the target node, and by modifying the parent node attribute of the particle node, the parent node attribute of the particle node is made to point to the target node, thereby realizing the binding of the particle, wherein the determination of the target position relative to the target node is related to the binding method of the particle node, and different binding methods determine different target nodes, and the binding methods include at least two of the following: window binding, track binding, and bullet screen binding, wherein window binding refers to binding the particle node to the scene element for indicating the window in the playback scene, and track binding refers to binding the particle node to the scene element for indicating the track in the playback scene; bullet screen binding refers to binding the particle node to the scene element for indicating the bullet screen in the playback scene; Rendering is performed based on the bound particle nodes, so as to display the particle special effects in the display mode indicated by the special effect configuration information during the presentation of the multimedia data.

2. The method according to claim 1, characterized in that The generating of particle nodes according to the particle resources comprises: Determine a prefab with a particle component embedded in the prefab, and instantiate the prefab as a general node; The effect attribute information in the special effect configuration information is loaded, and the effect attribute information is applied to the general node to obtain a particle node.

3. The method according to claim 1, characterized in that The step of determining the target node corresponding to the scene element on which the particle special effect is to be implemented in the playback scene according to the binding method in the special effect configuration information, and binding the particle node to the target node includes: Obtain a node tree provided by the Unreal Engine to support the playback scene, wherein the node tree is composed of a plurality of nodes having a parent-child relationship; Determining the scene elements for implementing the particle special effects in the playback scene according to the binding method in the special effects configuration information; A node corresponding to the scene element is searched from the node tree as a target node.

4. The method according to claim 3, characterized in that The step of determining the scene elements for implementing the particle special effects in the playback scene according to the binding mode in the special effects configuration information includes: In a case where the binding mode is window binding, determining a first layer for rendering the multimedia data; A second layer located above the first layer is used as a scene element for implementing particle special effects in the playback scene; the second layer is a transparent layer.

5. The method according to claim 3, characterized in that: The step of determining the scene elements for implementing the particle special effects in the playback scene according to the binding mode in the special effects configuration information includes: In the case where the binding mode is track binding, determining the track type of the track to be bound; If the trajectory type is a bullet screen running trajectory, the bullet screen running trajectory is used as a scene element for the particle special effects to be implemented in the playback scene.

6. The method according to claim 5, characterized in that The method further comprises: If the trajectory type is a custom trajectory, obtaining path information of the custom trajectory from the special effect configuration information; Parsing the path information to obtain trajectory coordinate information suitable for the playback scene; The custom trajectory is generated according to the trajectory coordinate information, and the custom trajectory is used as a scene element for the particle special effect to be implemented in the playback scene.

7. The method according to claim 3, characterized in that The step of determining the scene elements for implementing the particle special effects in the playback scene according to the binding mode in the special effects configuration information includes: In the case where the binding mode is barrage binding, obtaining the barrage within the special effect effective time period; At least a part of the bullet screen in the special effect effective time period is used as a scene element for the particle special effect to be implemented in the playback scene.

8. The method according to claim 1, characterized in that The method further comprises: Determine the position area information of the particle node from the special effect configuration information, wherein the position area information includes the position information of the particle emitter and the area information of the particle emission; Based on the position information of the scene element on which the particle special effect is to be implemented, coordinate conversion is performed on the position area information of the particle node to obtain relative position area information; The rendering based on the bound particle node includes: Rendering is performed based on the bound particle nodes and the relative position area information.

9. The method according to claim 8, characterized in that The scene element includes a bullet screen, and the coordinate conversion of the position area information of the particle node based on the position information of the scene element to be implemented with the particle special effect to obtain the relative position area information includes: Determine a bullet-screen area associated with the bullet-screen based on width and height information of the bullet-screen in a rendered state; The position area information of the particle node is converted relative to the bullet screen area to obtain relative position area information.

10. The method according to claim 8, characterized in that The scene element includes a bullet screen, and the coordinate conversion of the position area information of the particle node based on the position information of the scene element to be implemented with the particle special effect to obtain the relative position area information includes: Determine a designated character from the bullet-screen in a rendered state, and determine coordinate information of the designated character relative to the bullet-screen; Based on the coordinate information, the position area information of the particle node is converted to obtain relative position area information.

11. The method according to claim 1, characterized in that: The method further comprises: Obtaining animation attribute information used to describe the target animation executed by the particles in the special effect configuration information; During the display of the particle special effects, the movement of each particle is controlled frame by frame according to the animation attribute information to achieve the target animation.

12. The method according to any one of claims 1 to 11, characterized in that The special effect configuration information includes a special effect effective time period, and the method further includes: When the playback progress of the multimedia data is within the special effect effective time period, determining that a particle special effect event is triggered for the playback scene of the multimedia data; or, If a preset interactive operation is detected in the playback scene of the multimedia data, it is determined that a particle special effect event is triggered for the playback scene of the multimedia data.

13. A data interaction device, characterized in that: The device comprises: An acquisition module, used to acquire multimedia data, special effect configuration information and particle resources associated with the multimedia data, and establish an association relationship between the multimedia data and the particle resources in the special effect configuration information; A generation module, for generating a particle node according to the particle resource if a particle special effect event is triggered in the playback scene for the multimedia data, the particle node being a node corresponding to a particle in a particle system provided in the Unreal Engine, and binding the particle special effect to a scene element in the playback scene. After binding the scene element, the particle special effect is displayed in a corresponding position and area of ​​the scene element; A determination module is used to determine the target node corresponding to the scene element of the particle special effect to be implemented in the playback scene according to the binding method in the special effect configuration information, and bind the particle node to the target node, determine the target node corresponding to the target scene element of the particle special effect to be implemented in the node tree provided by the Unreal Engine for the playback scene, and bind the particle node to the target node by setting the position attribute of the particle node, so that the position of the particle node is consistent with that of the target node. The scene elements in the playback scene include at least two of the window elements, the track elements, and the bullet screen elements. In the playback scene based on the Unreal Engine, each scene element has a corresponding node in the node tree. In the process of binding the particle node to the target node, based on the position information of the scene element of the particle special effect to be implemented, the particle node is bound to the target node. The coordinate transformation is performed on the position area information of the point to obtain the relative position area information with the target node as the reference, so as to display the particle node at the target position of the target node, and the parent node attribute of the particle node is modified to point to the target node, thereby realizing the binding of the particle, wherein the determination of the target position relative to the target node is related to the binding mode of the particle node, different binding modes determine different target nodes, and the binding modes include at least two of the following: window binding, track binding, and bullet screen binding, wherein window binding refers to binding the particle node to the scene element for indicating the window in the playback scene, track binding refers to binding the particle node to the scene element for indicating the track in the playback scene; bullet screen binding refers to binding the particle node to the scene element for indicating the bullet screen in the playback scene; A rendering module is used to perform rendering based on the bound particle nodes, so as to display the particle special effects in a display mode indicated by the special effect configuration information during the presentation of the multimedia data.

14. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 12 are implemented.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

16. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

Images