Special effect resource rendering method, device, live broadcast system, equipment and storage medium

Abstract

The present application relates to the field of video and live broadcast technology, and provides a special effect resource rendering method, device, live broadcast system, electronic device and computer-readable storage medium. The present application can enrich the display mode and style of special effect resources on the terminal and improve its rendering display effect. The method includes: the terminal receives a special effect resource file, and then obtains the special effect resource data contained in the special effect resource file. If the special effect resource file also contains identification data for identifying that the special effect resource background can be added, the terminal obtains the corresponding background picture data; when rendering the special effect resource data, if the special effect resource data to be rendered is a special effect resource data corresponding to the background picture data, the terminal renders the corresponding background picture data and then renders the special effect resource data to present a special effect resource with the corresponding background picture as the background.

Description

Technical Field

[0001] The present application relates to the field of video and live broadcast technology, and in particular to a special effect resource rendering method, device, live broadcast system, electronic device and computer-readable storage medium. Background Art

[0002] With the development of video and network technology, applications such as live broadcast can add special effects such as live broadcast effects to provide users with live video playback, so as to enrich the information displayed in the live broadcast room. Among them, special effects resources are usually output after animation is made using relevant software and added to the rendering queue. For example, a special effects resource file in AVI or MP4 format can be output, and the special effects resource file will be used by the terminal.

[0003] In the current technology, after the terminal obtains the special effects resource file, it can parse and render it to the screen when the usage conditions are met. For example, when a specific user enters the live broadcast room, the terminal can render the corresponding live broadcast special effects resource on the live broadcast room interface. However, this technology is usually that the terminal parses the special effects resource data from the special effects resource file and then directly renders the special effects resource data. However, the special effects resource styles corresponding to the special effects resource files produced by the relevant software are relatively fixed and single, resulting in poor rendering effect of special effects resources by this technology. Summary of the invention

[0004] Based on this, it is necessary to provide a special effect resource rendering method, device, live broadcast system, electronic device and computer-readable storage medium to address the above technical problems.

[0005] In a first aspect, the present application provides a special effect resource rendering method, which is applied to a terminal. The method comprises:

[0006] Receive special effect resource files;

[0007] Obtaining special effect resource data contained in the special effect resource file;

[0008] If the special effect resource file also contains identification data for identifying that the special effect resource background can be added, then the corresponding background image data is obtained;

[0009] If the special effect resource data to be rendered is special effect resource data corresponding to background image data, the special effect resource data is rendered after rendering the background image data to present the special effect resource with the corresponding background image as the background.

[0010] In one embodiment, if the special effect resource file also contains identification data for identifying that the special effect resource background can be added, then the corresponding background image data is obtained, including: if the special effect resource file also contains identification data for identifying that the special effect resource background can be added, then the background addable layer number information is extracted from the identification data; according to the background addable layer number information, the background image data corresponding to each background level is obtained.

[0011] In one embodiment, if the special effect resource data to be rendered is special effect resource data corresponding to background image data, then the special effect resource data is rendered after rendering the background image data, including: if the special effect resource data to be rendered is special effect resource data corresponding to multiple layers of background image data, then the background level order is determined, and the background image data corresponding to each background level is rendered in sequence according to the background level order before rendering the special effect resource data.

[0012] In one embodiment, after obtaining the corresponding background image data, it also includes: when the corresponding background image data belongs to the target class background image data, matching the corresponding background image data with the special effect resource data contained in the special effect resource file in terms of frame number, so that the frame number of the corresponding background image data is the same as the frame number of the special effect resource data; rendering the special effect resource data after rendering the background image data includes: according to the frame index of the special effect resource data to be rendered, obtaining the corresponding target background image data from the corresponding background image data, and rendering the target background image data; after rendering the target background image data, obtaining the corresponding target special effect resource data from the special effect resource data contained in the special effect resource file according to the frame index of the special effect resource data to be rendered, and rendering the target special effect resource data.

[0013] In one embodiment, the corresponding background image data includes background image data corresponding to each background level; matching the corresponding background image data with the special effect resource data contained in the special effect resource file in terms of frame number includes: for the background image data of each background level, matching the background image data of the background level with the special effect resource data contained in the special effect resource file in terms of frame number, so that the frame number of the background image data of each background level is the same as the frame number of the special effect resource data; obtaining the corresponding target background image data from the corresponding background image data according to the frame index of the special effect resource data to be rendered, and rendering the target background image data includes: obtaining the target background image data corresponding to each background level from the background image data corresponding to each background level according to the frame index of the special effect resource data to be rendered, and rendering the target background image data corresponding to each background level in sequence according to the background level order.

[0014] In one embodiment, after receiving the special effect resource file, the method further includes: extracting description information of the special effect resource file; and determining, based on the description information, whether the special effect resource file contains identification data for identifying that a special effect resource background can be added.

[0015] In one embodiment, determining whether the special effect resource file contains identification data for identifying that a special effect resource background can be added based on the description information includes: obtaining a preset field in the description information for writing identification data by a special effect resource provider based on the description information; and determining whether the special effect resource file contains identification data for identifying that a special effect resource background can be added based on the identification data in the preset field in the description information.

[0016] In one embodiment, determining whether the special effect resource file contains identification data for identifying that a special effect resource background can be added based on the description information includes: determining a preset identification data keyword for matching the description information; matching the description information based on the preset identification data keyword; and determining whether the special effect resource file contains identification data for identifying that a special effect resource background can be added based on the matching result.

[0017] In a second aspect, the present application provides a live broadcast system. The system includes: a terminal and a live broadcast server; wherein the terminal is used to render live broadcast special effect resource data and corresponding background image data in a live broadcast special effect resource file sent by the live broadcast server according to the above method, and play the live broadcast special effect resource with the corresponding background image as the background in the live broadcast screen.

[0018] In a third aspect, the present application provides a special effect resource rendering device, which is applied to a terminal. The device includes:

[0019] A file receiving module, used for receiving special effect resource files;

[0020] A data acquisition module, used to acquire the special effect resource data contained in the special effect resource file;

[0021] A background acquisition module, used for acquiring corresponding background image data if the special effect resource file further includes identification data for identifying that the special effect resource background can be added;

[0022] The data rendering module is used to render the special effect resource data after rendering the background image data if the special effect resource data to be rendered is special effect resource data corresponding to background image data, so as to present the special effect resource with the corresponding background image as the background.

[0023] In a fourth aspect, the present application provides an electronic device. The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0024] Receive a special effect resource file; obtain the special effect resource data contained in the special effect resource file; if the special effect resource file also contains identification data for identifying that a special effect resource background can be added, obtain the corresponding background image data; if the special effect resource data to be rendered is special effect resource data corresponding to background image data, render the background image data and then render the special effect resource data to present a special effect resource with the corresponding background image as the background.

[0025] In a fifth aspect, the present application provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0026] Receive a special effect resource file; obtain the special effect resource data contained in the special effect resource file; if the special effect resource file also contains identification data for identifying that a special effect resource background can be added, obtain the corresponding background image data; if the special effect resource data to be rendered is special effect resource data corresponding to background image data, render the background image data and then render the special effect resource data to present a special effect resource with the corresponding background image as the background.

[0027] In the above-mentioned special effect resource rendering method, device, live broadcast system, equipment and medium, the terminal receives the special effect resource file, and then obtains the special effect resource data contained in the special effect resource file. If the special effect resource file also contains identification data for identifying that the special effect resource background can be added, the terminal obtains the corresponding background image data; when rendering the special effect resource data, if the special effect resource data to be rendered is special effect resource data corresponding to background image data, the terminal renders the corresponding background image data and then renders the special effect resource data to present the special effect resource with the corresponding background image as the background. In this solution, when rendering special effect resource data, the terminal does not directly render the special effect resource data parsed from the file, but first determines whether the file contains identification data for identifying the special effect resource background that can be added. If so, it is necessary to render the corresponding background image data first and then render the special effect resource data. In this way, the terminal can present the special effect resource with the corresponding background image as the background on the screen, and also realize the addition and flexible change of the background of the special effect resource on the terminal side, so that the terminal can dynamically change the background of the special effect resource according to actual business needs, enriching the display method and style of the special effect resource on the terminal, and improving the rendering and display effect of the special effect resource. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1This is an application scenario diagram of the special effect resource rendering method in the embodiment of the present application;

[0029] Figure 2 A flowchart of a special effect resource rendering method in an embodiment of the present application;

[0030] Figure 3 This is a flowchart of a special effect resource rendering method in an application example of this application;

[0031] Figure 4 This is a structural diagram of the live broadcast system in the embodiment of the present application;

[0032] Figure 5 This is a schematic diagram of the interface for playing live special effects resources in the application example of this application;

[0033] Figure 6 This is a structural block diagram of a special effect resource rendering device in an embodiment of the present application;

[0034] Figure 7 Schematic diagram of the internal structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0036] The special effect resource rendering method provided in the embodiment of the present application can be applied to Figure 1 In the application scenario shown, the application scenario may include a terminal 110 and a special effect resource provider 120, and the terminal 110 communicates with the special effect resource provider 120 through a network. Among them, the terminal 110 may be a device for rendering according to data information such as special effect resource files to present special effect resources and other related content, and the special effect resource provider 120 may be a device that provides data information such as special effect resource files to the terminal 110. The terminal 110 can be regarded as a demand end or a presentation end of special effect resources relative to the special effect resource provider 120. Specifically, the terminal 110 may be, but is not limited to, various personal computers, laptops, smart phones, and tablet computers, and the special effect resource provider 120 may be a terminal or a server, and the server may be implemented as an independent server or a server cluster composed of multiple servers.

[0037] In one embodiment, Figure 2 As shown, a special effect resource rendering method is provided, which can be Figure 1 The method may include the following steps:

[0038] Step S201, receiving a special effect resource file.

[0039] In this step, the terminal 110 can receive a special effect resource file from the special effect resource provider 120. Specifically, the special effect resource file can be a special effect resource file in MP4 format, which is recorded as an MP4 special effect resource file. In actual applications, the terminal 110 can request the corresponding special effect resource file from the special effect resource provider 120 when a special effect resource needs to be presented. The terminal 110 can send a special effect resource file acquisition request to the special effect resource provider 120. After receiving the request, the special effect resource provider 120 can send the corresponding MP4 special effect resource file to the terminal 110, so that the terminal 110 obtains the MP4 special effect resource file; in addition, the special effect resource provider 120 can also actively send a special effect resource file to the terminal 110 when it is necessary for the terminal 110 to present a certain special effect resource. After receiving the special effect resource file, the terminal 110 can render in a certain way according to the special effect resource data in the special effect resource file to present the corresponding special effect resource on the screen canvas. As an example, taking live broadcast special effects resources as an example, when a specific user enters the live broadcast room, the special effects resource provider 120 can send a live broadcast special effects resource file to the terminal 110. After receiving the live broadcast special effects resource file, the terminal 110 can render according to the live broadcast special effects resource data in the live broadcast special effects resource file to play on the screen canvas, such as live broadcast special effects resources that welcome the specific user to enter the live broadcast room.

[0040] Step S202, obtaining special effect resource data contained in the special effect resource file.

[0041] In this step, the terminal 110 can decapsulate the special effect resource file, extract the video track, and obtain the special effect resource data. The special effect resource data can be decoded by the terminal 110 into a frame queue, and the frame queue can contain each frame of special effect resource data that needs to be rendered by the terminal 110, and each frame of special effect resource data can correspond to its own frame index.

[0042] Step S203: If the special effect resource file also contains identification data for identifying that the special effect resource background can be added, then the corresponding background image data is obtained.

[0043] In this step, after the terminal 110 decapsulates the special effect resource file, if the special effect resource file extracted also contains identification data for identifying that the special effect resource background can be added, the terminal 110 can also obtain the corresponding background image data. Among them, the corresponding background image data can be background image data pre-stored on the terminal 110, or it can be background image data sent to the terminal 110 by the special effect resource provider 120. The background image data can be GIF, PNG and other types of background image data. The type and content of the background image data can be changed according to the actual business scenario requirements, that is, for the same special effect resource, the corresponding background image data obtained by the terminal 110 each time can be changed in type and content according to the actual business scenario requirements. The terminal 110 can obtain GIF background image data or PNG background image data, can obtain GIF background image data A or GIF background image data B, and can also obtain PNG background image data A or PNG background image data B.

[0044] Specifically, in addition to the special effect resource data, the special effect resource file sent by the special effect resource provider 120 may also include identification data that can be written into the special effect resource file by the special effect resource provider 120, and the identification data can be written into the special effect resource file in the form of json data. The identification data can be used by the special effect resource provider 120 to identify that the special effect resource corresponding to the special effect resource file has the attribute of adding a background. Specifically, for example, when the special effect resource provider 120 needs to identify that the special effect resource corresponding to the special effect resource file has the attribute of adding a background, the special effect resource provider 120 can write background1 as the identification data into the special effect resource file, and then send the special effect resource file containing the special effect resource data and the identification data to the terminal 110. Since the special effect resource provider 120 identifies that the background of the special effect resource is addable in the special effect resource file, the terminal 110 also needs to obtain the corresponding background image data. The background image data can be pre-stored by the terminal 110 or transmitted by the special effect resource provider 120. Therefore, when terminal 110 unpacks the special effect resource file, it can detect whether the special effect resource file contains identification data for identifying that the special effect resource background can be added. If it does, terminal 110 can obtain the corresponding background image data and cache it in the content for use in rendering.

[0045] Step S204 , if the special effect resource data to be rendered is special effect resource data corresponding to background image data, then the special effect resource data is rendered after the background image data is rendered to present a special effect resource with the corresponding background image as the background.

[0046] This step is the stage where the terminal 110 renders the special effect resource data. In this stage, the terminal 110 first determines whether the special effect resource data to be rendered is special effect resource data corresponding to background image data. If not, the terminal 110 can directly render the special effect resource data. If the special effect resource data to be rendered is special effect resource data corresponding to background image data, then the terminal 110 needs to render the special effect resource data and its corresponding scene image data in a certain order. The terminal 110 needs to first render the corresponding background image data in the above step S203, and then render the special effect resource data. Thus, the terminal 110 can present the special effect resource with the corresponding background image as the background on the screen canvas. In actual applications, if the terminal 1 10 has displayed a basic picture, such as a live broadcast picture, then in this step, on the upper layer of the live broadcast picture, first renders and displays the background picture corresponding to the live broadcast special effect resource, and then renders and displays the live broadcast special effect resource after the background picture is rendered. Therefore, in terms of the relative display levels of the three, the live broadcast picture is located in the lower layer, the live broadcast special effect resource is located in the upper layer, and the background picture is between the two, thereby achieving the effect of presenting the special effect resource with the corresponding background picture as the background on the screen canvas, and as mentioned above, for the same special effect resource, the corresponding background picture data obtained by the terminal 110 can be different, and can be specifically obtained according to the dynamic changes of the actual business scenario, and the background picture data corresponding to the scenario can be obtained, so as to support the terminal side to dynamically change the background of the special effect resource according to the business scenario.

[0047] In the special effect resource rendering method provided in this embodiment, the terminal 110 receives a special effect resource file, and then obtains the special effect resource data contained in the special effect resource file. If the special effect resource file also contains identification data for identifying that a special effect resource background can be added, the terminal 110 obtains the corresponding background image data. When rendering the special effect resource data, if the special effect resource data to be rendered is special effect resource data corresponding to background image data, the terminal 110 renders the corresponding background image data and then renders the special effect resource data to present the special effect resource with the corresponding background image as the background. In this solution, when rendering special effect resource data, terminal 110 does not directly render the special effect resource data parsed from the file, but first determines whether the file contains identification data for identifying the special effect resource background that can be added. If so, it is necessary to render the corresponding background image data first and then render the special effect resource data. In this way, terminal 110 can present the special effect resource with the corresponding background image as the background on the screen, and can also add and flexibly change the background of the special effect resource on the terminal side, so that terminal 110 can dynamically change the background of the special effect resource according to actual business needs, enriching the display method and style of the special effect resource on the terminal, and improving the rendering and display effect of the special effect resource.

[0048] In some embodiments, if the special effect resource file in the above step S203 also includes identification data for identifying that the special effect resource background can be added, obtaining the corresponding background image data may include:

[0049] If the special effect resource file also contains identification data for identifying that the special effect resource background can be added, then the number of background layers that can be added is extracted from the identification data; and the background image data corresponding to each background layer is obtained according to the number of background layers that can be added.

[0050] Specifically, as mentioned above, in addition to the special effect resource data, the special effect resource file sent by the special effect resource provider 120 may also include, for example, identification data for identifying that the special effect resource background can be added. In this embodiment, the identification data written by the special effect resource provider 120 may also be specifically used to carry the information of the number of background image layers that can be added to the special effect resource, i.e., the background addable layer information. Based on the background addable layer information, the terminal 110 may add multiple layers of background images to the special effect resource, further enrich the display mode and style of the special effect resource, and enhance its rendering display effect. Specifically, the special effect resource provider 120 may write background3 as identification data into the special effect resource file, wherein the background in background3 may be used to identify that the special effect resource background can be added, and the 3 therein may be used as the background addable layer information it carries, i.e., the number of background layers that can be added is 3. Thus, the terminal 110 obtains the background image data corresponding to each background level according to the background addable layer information.

[0051] Based on this, in one embodiment, if the special effect resource data to be rendered in the above step S204 is special effect resource data corresponding to background image data, rendering the special effect resource data after rendering the background image data further includes:

[0052] If the special effect resource data to be rendered is special effect resource data corresponding to multiple layers of background image data, the background level order is determined, and the background image data corresponding to each background level is rendered in sequence according to the background level order, followed by the rendering of the special effect resource data.

[0053] In this embodiment, the terminal 110 parses the identification data for identifying that the special effect resource background can be added from the special effect resource file and can extract the information on the number of background layers that can be added thereto, and then obtains the background image data corresponding to each background level. Based on this, in the rendering stage of the special effect resource data, if the special effect resource data to be rendered is special effect resource data corresponding to multiple layers of background image data, then the terminal 110, in addition to rendering the background first and then rendering the special effect resource, also needs to render in the order of the background levels when rendering the background so that the multiple layers of the background can be correctly rendered and displayed. Assume that the number of background layers that can be added is 2. After the terminal obtains the background image data corresponding to each of the two background layers, it obtains the background layer order of the two layers of background image data. The background layer order can be predetermined, that is, the layer order of the two layers of background image data can be determined after the production of the two layers of background image data is completed. For example, when background image data A is produced, it is marked as background image data belonging to the first background layer, and background image data B is marked as background image data belonging to the second background layer, and so on. Each background layer can include multiple background image data, and their types and contents can also be different. Terminal 110 can obtain background image data from the corresponding background layer according to the information on the number of background layers that can be added as background image data for adding the background to the special effect resource in the corresponding background layer, so as to enrich the display method and style of the special effect resource. Then, terminal 110 can render the background image data corresponding to the second background layer first, and then render the background image data corresponding to the first background layer in the order of background layers. After the background image data corresponding to the two background layers are rendered, the special effect resource data can be rendered. In this way, the effect of playing special effect resources with the corresponding multi-layer background images as the background can be achieved, so that the display method and style of the special effect resources are further enriched, and the display effect of the special effect resources is further improved.

[0054] In some embodiments, after obtaining the corresponding background image data in step S203, the following steps may also be included:

[0055] When the corresponding background image data belongs to the target class background image data, the corresponding background image data is matched with the special effect resource data contained in the special effect resource file in terms of frame number, so that the frame number of the corresponding background image data is the same as the frame number of the special effect resource data.

[0056] In this embodiment, the number of frames of the special effect resource data that the terminal 110 needs to render is usually multiple frames, and when the terminal 100 needs to add a background image to it, the number of frames of the corresponding background image data obtained may not match it, which will affect the display effect of the background image when the special effect resource is added. To ensure that the background image can be accurately added, after obtaining the corresponding background image data, the terminal 110 determines whether the background image data belongs to the target class background image data. The target class background image data can be a GIF format containing multiple frames of background image data. If the background image data belongs to the target class background image data, the terminal 110 can perform frame matching processing on it. , terminal 110 matches the frame number of the background image data with the special effect resource data contained in the special effect resource file, so that the frame number of the background image data is the same as the frame number of the special effect resource data. Specifically, when the frame number of the background image data is less than the frame number of the special effect resource data, the frame number of the background image data can be increased according to the background image data so that its frame number is the same as the frame number of the special effect resource data, and when the frame number of the background image data is greater than the frame number of the special effect resource data, the frame number of the background image data can be reduced according to the background image data so that its frame number is the same as the frame number of the special effect resource data. This ensures that before rendering each frame of the special effect resource data, the background image data of the corresponding frame can be obtained and rendered first.

[0057] Based on this, rendering the special effect resource data after rendering the background image data in step S204 further includes:

[0058] According to the frame index of the special effect resource data to be rendered, the corresponding target background image data is obtained from the corresponding background image data, and the target background image data is rendered; after rendering the target background image data, according to the frame index of the special effect resource data to be rendered, the corresponding target special effect resource data is obtained from the special effect resource data contained in the special effect resource file, and the target special effect resource data is rendered.

[0059] In this embodiment, as mentioned above, the number of frames of special effect resource data that the terminal 110 needs to render is usually multiple frames. In the rendering stage, the terminal 110 can obtain the frame index of the special effect resource data to be rendered. Since the number of frames of the background image data has been matched with the number of frames of the special effect resource data in the aforementioned embodiment, the terminal 110 can obtain the background image data corresponding to the frame index, that is, the target background image data, in the background image data with frame number matching based on the frame index of the special effect resource data to be rendered, and then render the target background image data. After rendering the target background image data, the terminal 110 can obtain the special effect resource data corresponding to the frame index, that is, the target special effect resource data, from the special effect resource data contained in the special effect resource file based on the frame index of the special effect resource data to be rendered, and then render the target special effect resource data. In this way, the terminal 110 can accurately and effectively add the corresponding background image to each frame of the special effect resource on the basis of frame number matching, so that the background addition effect of the special effect resource is more stable and reliable.

[0060] Furthermore, in some embodiments, the corresponding background image data may include the background image data corresponding to each background level. Thus, the above embodiment of matching the corresponding background image data with the special effect resource data contained in the special effect resource file in terms of frame number further includes: for the background image data of each background level, matching the background image data of the background level with the special effect resource data contained in the special effect resource file in terms of frame number, so that the frame number of the background image data of each background level is the same as the frame number of the special effect resource data. Specifically, when the corresponding background image data includes background image data corresponding to each background level, the terminal 110 performs frame matching processing on the background image data of each background level. For example, when the corresponding background image data includes GIF background image data A of the first background level and GIF background image data B of the second background level, the terminal 110 performs frame matching on the GIF background image data A of the first background level with the special effect resource data, and performs frame matching on the GIF background image data B of the second background level with the special effect resource data, so that the frame number of the GIF background image data A of the first background level is the same as the frame number of the special effect resource data, and the frame number of the GIF background image data B of the second background level is also the same as the frame number of the special effect resource data, so as to ensure that before rendering each frame of the special effect resource data in the case of multiple layers of background, the background image data of each layer of the corresponding frame can be obtained and rendered first.

[0061] Therefore, the above-mentioned embodiment further includes: obtaining the corresponding target background image data from the corresponding background image data according to the frame index of the special effect resource data to be rendered, and rendering the target background image data, according to the frame index of the special effect resource data to be rendered, from the background image data corresponding to each background level, obtaining the target background image data corresponding to each background level, and rendering the target background image data corresponding to each background level in sequence according to the background level order. Specifically, as mentioned above, in the case where the corresponding background image data includes the background image data corresponding to each background level, the terminal 110 will match the background image data of each background level with the special effect resource data in terms of the number of frames. Based on this, in the rendering stage, after obtaining the frame index of the special effect resource data to be rendered, the terminal 110 can obtain the background image data corresponding to the frame index at each background level, i.e., the target background image data, in the background image data that has been matched in terms of the number of frames according to the frame index, and obtain the target background image data corresponding to each background level. The target background image data of the frame has a corresponding background level sequence, so the terminal 110 renders the target background image data corresponding to each background level in sequence according to the background level sequence. For example, for the first frame, the target background image data corresponding to the second background level can be rendered first, and then the target background image data corresponding to the first background level can be rendered. After the target background image data corresponding to the two background levels are rendered, the special effect resource data of the first frame is rendered, and the rendering of the second and subsequent frames is performed in sequence. In this way, the terminal 110 can accurately and effectively add corresponding background images of each background level to each frame of the special effect resource on the basis of matching the number of frames at each background level, so as to achieve the effect of playing special effect resources with the corresponding multi-layer background images as the background, making the playback effect more stable and reliable.

[0062] In some embodiments, after receiving the special effect resource file in step S201, the method further includes the following steps:

[0063] Extract description information of the special effect resource file; and determine, based on the description information, whether the special effect resource file contains identification data for identifying that the special effect resource background can be added.

[0064] In this embodiment, the special effect resource provider 120 can write the identification data used to identify that the special effect resource background can be added into the description information of the special effect resource file and send it to the terminal 110. After receiving the special effect resource file, the terminal 110 decapsulates it, extracts the special effect resource data and description information, and determines whether the description information of the special effect resource file contains the identification data used to identify that the special effect resource background can be added. Figure 3The MP4 special effect resource file provided by the special effect resource provider 120 may include special effect resource data and description information. After receiving the MP4 special effect resource file, the terminal 110 extracts the special effect resource data and description information. If the description information includes identification data for identifying that the special effect resource background can be added, the terminal 110 can obtain the corresponding background image data passed in and cache it. When rendering each frame, the special effect resource data to be rendered, i.e., the target special effect resource data and the target background image data, are obtained according to the frame index, and the target background image data is first rendered and then the target special effect resource data is rendered. Specifically, the terminal 110 decapsulates the MP4 special effect resource file, extracts the special effect resource data and description information. On the one hand, the terminal 110 can decode to obtain a frame queue containing multiple frames of special effect resource data. On the other hand, the terminal 110 can determine whether the extracted description information includes identification data for identifying that the special effect resource background can be added. If so, the terminal 110 obtains the corresponding background image data and caches it in the memory. During the rendering stage, terminal 110 obtains the frame index of the special effect resource data to be rendered. If the special effect resource data to be rendered has corresponding background image data, terminal 110 obtains and renders the target background image data according to the frame index, and then obtains and renders the target special effect resource data from the frame queue according to the frame index, thereby presenting a special effect resource with the corresponding background image as the background.

[0065] Further, in some of the embodiments, determining whether the special effect resource file contains identification data for identifying that the special effect resource background can be added according to the description information in the above embodiments may include:

[0066] According to the description information, a preset field in the description information for writing identification data by the special effect resource provider is obtained; according to the identification data of the preset field in the description information, it is determined whether the special effect resource file contains identification data for identifying that the special effect resource background can be added.

[0067] In this embodiment, the terminal 110 can obtain the preset field such as metadata field in the description information for the special effect resource provider 120 to write identification data by parsing. The special effect resource provider 120 can write identification data in the preset field of the description information. The identification data can be identification data such as background for identifying that the special effect resource background can be added, or identification data such as background3 for indicating that the special effect resource background can be added and carries the information of the number of layers that can be added to the background, or identification data such as background0 for identifying that the special effect resource background cannot be added, thereby further improving its flexibility in setting the background addable attribute of the special effect resource. Therefore, after the terminal 110 parses and obtains the preset field in the description information for the special effect resource provider 120 to write identification data, it can extract the identification data of the preset field, thereby determining whether the special effect resource file contains identification data for identifying that the special effect resource background is variable, and can further facilitate the extraction of information such as the number of layers that can be added to the background.

[0068] Further, in some other embodiments, the foregoing embodiment determines whether the special effect resource file contains identification data for identifying that the special effect resource background can be added according to the description information, which may include:

[0069] Determine a preset identification data keyword for matching in the description information; match in the description information according to the preset identification data keyword; determine whether the special effect resource file contains identification data that can be added to the special effect resource background according to the matching result.

[0070] In this embodiment, the terminal 110 can determine whether the special effect resource file contains identification data for identifying that the special effect resource background can be added by keyword matching. The special effect resource provider 120 can use the character string containing the preset keyword as the identification data for identifying that the special effect resource background can be added according to the predetermined rule. The character string of the preset keyword can be used as the preset identification data keyword, such as background can be used as the preset identification data keyword. The special effect resource provider 120 can directly write background as the identification data for identifying that the special effect resource background can be added into the description information of the special effect resource file. In the embodiment, the preset identification data keyword can be sent in advance by the special effect resource provider 120 to the terminal 110 for storage. Therefore, after obtaining the special effect resource file, the terminal 110 can match the description information of the special effect resource file according to the preset identification data keyword determined for matching in the description information. If the matching result indicates that the description information contains the preset identification data keyword, the terminal 110 can determine that the special effect resource file contains identification data for identifying that the special effect resource background can be added. Therefore, the terminal 110 can more easily and quickly determine whether the special effect resource file contains identification data for identifying that the special effect resource background is variable.

[0071] In one embodiment, Figure 4 As shown, a live broadcast system is provided, which may include a terminal and a live broadcast server. The terminal may communicate with the live broadcast server via the Internet. In this embodiment, the live broadcast server may send a live broadcast special effect resource file to the terminal, and may also send corresponding background image data. The terminal of the live broadcast system may render the live broadcast special effect resource data and the corresponding background image data in the live broadcast special effect resource file sent by the live broadcast server according to the special effect resource rendering method described in any of the above embodiments, so as to play the live broadcast special effect resource with the corresponding background image as the background in the live broadcast screen. As an application example, Figure 5 As shown, the terminal of the live broadcast system can play the live broadcast special effect resource 520 with the corresponding background image 530 as the background in the live broadcast screen 510. In terms of the relative hierarchy of the three, the live broadcast screen is located in the lower layer, the live broadcast special effect resource is located in the upper layer, and the background image is between the two, so as to achieve the effect of presenting the live broadcast special effect resource with the corresponding background image as the background on the screen canvas, and for the same live broadcast special effect resource, the background image added by the terminal can be different, so that the terminal can dynamically change the background of the live broadcast special effect resource according to the business scenario needs. Therefore, this embodiment can apply the special effect resource rendering method provided by this application to the live broadcast system, so that in the live broadcast scenario, the terminal side can add the background of the live broadcast special effect resource, and can support dynamic changes according to the live broadcast business scenario, enriching the display method and style of the live broadcast special effect resource on the terminal, and improving the rendering and display effect of the live broadcast special effect resource.

[0072] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0073] Based on the same inventive concept, the embodiment of the present application also provides a special effect resource rendering device for implementing the special effect resource rendering method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more special effect resource rendering device embodiments provided below can refer to the limitations of the special effect resource rendering method above, and will not be repeated here.

[0074] In one embodiment, Figure 6 As shown, a special effect resource rendering device is provided, which can be applied to a terminal. The device 600 may include:

[0075] The file receiving module 601 is used to receive special effect resource files;

[0076] The data acquisition module 602 is used to acquire the special effect resource data contained in the special effect resource file;

[0077] Background acquisition module 603, used for acquiring corresponding background image data if the special effect resource file further includes identification data for identifying that the special effect resource background can be added;

[0078] The data rendering module 604 is used to render the special effect resource data after rendering the background image data if the special effect resource data to be rendered is special effect resource data corresponding to background image data, so as to present the special effect resource with the corresponding background image as the background.

[0079] In one embodiment, the background acquisition module 603 is used to extract the background addable layer information from the identification data if the special effect resource file also contains identification data for identifying that the special effect resource background can be added; and obtain the background image data corresponding to each background level according to the background addable layer information.

[0080] In one embodiment, the data rendering module 604 is used to determine the background level order if the special effect resource data to be rendered is special effect resource data corresponding to multiple layers of background image data, and render the special effect resource data after rendering the background image data corresponding to each background level in sequence according to the background level order.

[0081] In one embodiment, the background acquisition module 603 can also be used to match the frame number of the corresponding background image data with the special effect resource data contained in the special effect resource file when the corresponding background image data belongs to the target type background image data, so that the frame number of the corresponding background image data is the same as the frame number of the special effect resource data; the data rendering module 604 is used to obtain the corresponding target background image data from the corresponding background image data according to the frame index of the special effect resource data to be rendered, and render the target background image data; after rendering the target background image data, obtain the corresponding target special effect resource data from the special effect resource data contained in the special effect resource file according to the frame index of the special effect resource data to be rendered, and render the target special effect resource data.

[0082] In one embodiment, the corresponding background image data includes background image data corresponding to each background level; the background acquisition module 603 can also be used to match the background image data of each background level with the special effect resource data contained in the special effect resource file in terms of the number of frames, so that the number of frames of the background image data of each background level is the same as the number of frames of the special effect resource data; the data rendering module 604 is used to obtain the target background image data corresponding to each background level from the background image data corresponding to each background level according to the frame index of the special effect resource data to be rendered, and render the target background image data corresponding to each background level in sequence according to the background level order.

[0083] In one embodiment, the device 600 may further include: an identification determination module, configured to extract description information of the special effect resource file; and determine, based on the description information, whether the special effect resource file contains identification data for identifying that a special effect resource background can be added.

[0084] In one embodiment, the identification determination module can be used to obtain a preset field in the description information for writing identification data by the special effect resource provider based on the description information; and determine whether the special effect resource file contains identification data for identifying that the special effect resource background can be added based on the identification data of the preset field in the description information.

[0085] In one embodiment, the identification determination module can be used to determine a preset identification data keyword for matching in the description information; match the description information according to the preset identification data keyword; and determine whether the special effect resource file contains identification data that can be added to the special effect resource background based on the matching result.

[0086] Each module in the above-mentioned special effect resource rendering device can be implemented in whole or in part by software, hardware or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the electronic device in the form of hardware, or can be stored in the memory in the electronic device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0087] In one embodiment, an electronic device is provided. The electronic device may be a terminal, and its internal structure diagram may be as follows: Figure 7 As shown. The electronic device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a special effect resource rendering method is implemented. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the housing of the electronic device, or an external keyboard, touchpad or mouse, etc.

[0088] Those skilled in the art will understand that Figure 7 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0089] In one embodiment, an electronic device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0090] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0091] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0092] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0093] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A special effect resource rendering method, characterized in that: Applied to a terminal, the method comprises: Receive special effect resource files; Obtaining special effect resource data contained in the special effect resource file; If the special effect resource file also contains identification data for identifying that the special effect resource background can be added, the corresponding background image data is obtained; when the corresponding background image data belongs to the target class background image data, the corresponding background image data is matched with the special effect resource data contained in the special effect resource file in terms of frame number, so that the frame number of the corresponding background image data is the same as the frame number of the special effect resource data; If the special effect resource data to be rendered is special effect resource data corresponding to background image data, then the special effect resource data is rendered after rendering the background image data to present the special effect resource with the corresponding background image as the background; including: according to the frame index of the special effect resource data to be rendered, the corresponding target background image data is obtained from the corresponding background image data, and the target background image data is rendered; after rendering the target background image data, according to the frame index of the special effect resource data to be rendered, the corresponding target special effect resource data is obtained from the special effect resource data contained in the special effect resource file, and the target special effect resource data is rendered.

2. The method according to claim 1, characterized in that: If the special effect resource file further includes identification data for identifying that the special effect resource background can be added, obtaining the corresponding background image data includes: If the special effect resource file further includes identification data for identifying that the special effect resource background can be added, extracting the information of the number of layers that can be added to the background from the identification data; According to the information on the number of layers that can be added to the background, background image data corresponding to each background layer is obtained.

3. The method according to claim 2, characterized in that If the special effect resource data to be rendered is special effect resource data corresponding to background image data, then rendering the special effect resource data after rendering the background image data includes: If the special effect resource data to be rendered is special effect resource data corresponding to multiple layers of background image data, the background level sequence is determined, and the background image data corresponding to each background level is sequentially rendered according to the background level sequence, and then the special effect resource data is rendered.

4. The method according to claim 1, characterized in that: The corresponding background image data includes background image data corresponding to each background level; The matching of the corresponding background image data with the special effect resource data contained in the special effect resource file by frame number includes: For the background image data of each background level, matching the background image data of the background level with the special effect resource data contained in the special effect resource file by frame number, so that the frame number of the background image data of each background level is the same as the frame number of the special effect resource data; The step of acquiring corresponding target background image data from the corresponding background image data according to the frame index of the special effect resource data to be rendered, and rendering the target background image data comprises: According to the frame index of the special effect resource data to be rendered, the target background image data corresponding to each background level is obtained from the background image data corresponding to each background level, and the target background image data corresponding to each background level is rendered in sequence according to the background level order.

5. The method according to any one of claims 1 to 4, characterized in that: After receiving the special effect resource file, the method further includes: Extracting description information of the special effect resource file; According to the description information, it is determined whether the special effect resource file contains identification data for identifying that a special effect resource background can be added.

6. The method according to claim 5, characterized in that The step of determining, according to the description information, whether the special effect resource file contains identification data for identifying that a special effect resource background can be added includes: According to the description information, obtaining a preset field in the description information for the special effect resource provider to write identification data; According to the identification data of the preset field in the description information, it is determined whether the special effect resource file contains identification data for identifying that the special effect resource background can be added.

7. The method according to claim 5, characterized in that The step of determining, according to the description information, whether the special effect resource file contains identification data for identifying that a special effect resource background can be added includes: Determining a preset identification data keyword for matching in the description information; Matching the description information according to the preset identification data keyword; It is determined according to the matching result whether the special effect resource file contains identification data for identifying that the special effect resource background can be added.

8. A special effect resource rendering device, characterized in that: Applied to a terminal, the device comprises: A file receiving module, used for receiving special effect resource files; A data acquisition module, used to acquire the special effect resource data contained in the special effect resource file; A background acquisition module, for acquiring corresponding background image data if the special effect resource file also contains identification data for identifying that the special effect resource background can be added; when the corresponding background image data belongs to the target class background image data, matching the corresponding background image data with the special effect resource data contained in the special effect resource file by frame number, so that the frame number of the corresponding background image data is the same as the frame number of the special effect resource data; A data rendering module is used to render the special effect resource data after rendering the background image data if the special effect resource data to be rendered is special effect resource data corresponding to background image data, so as to present the special effect resource with the corresponding background image as the background; including: according to the frame index of the special effect resource data to be rendered, obtaining the corresponding target background image data from the corresponding background image data, and rendering the target background image data; after rendering the target background image data, according to the frame index of the special effect resource data to be rendered, obtaining the corresponding target special effect resource data from the special effect resource data contained in the special effect resource file, and rendering the target special effect resource data.

9. A live broadcast system, characterized in that: The system comprises: a terminal and a live broadcast server; wherein, The terminal is used to render the live special effects resource data and the corresponding background image data in the live special effects resource file sent by the live broadcast server according to the method described in any one of claims 1 to 7, and play the live special effects resource with the corresponding background image as the background in the live broadcast screen.

10. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images