Special effect resource rendering method and device, live broadcast system, equipment and storage medium

Abstract

The invention relates to the technical field of videos and live broadcast, and provides a special effect resource rendering method and device, a live broadcast system, electronic equipment and a computer readable storage medium. The method can enrich the display modes and styles of the special effect resources on the terminal, and improves the rendering display effect. The method comprises the following steps: receiving a special effect resource file by a terminal, extracting description information of the special effect resource file to determine whether the description information contains identification data used for identifying that a special effect resource background can be added, and then obtaining special effect resource data contained in the special effect resource file, if the special effect resource file further comprises identification data used for identifying that a special effect resource background can be added, the terminal obtains corresponding background picture data; and when the special effect resource data is rendered, if the to-be-rendered special effect resource data is the special effect resource data corresponding to the background picture data, the terminal renders the background picture data and then renders the special effect resource data according to the frame index so as to present the special effect resource taking the corresponding background picture as the background.

Description

[0001] This application is a divisional application of the application with the application number 202211098151.6, the application date of September 8, 2022, and the invention title of "Special Effect Resource Rendering Method, Device, Live Streaming System, Equipment and Storage Medium". Technical Field

[0002] This application relates to the field of video and live streaming technologies, and particularly to a special effect resource rendering method, device, live streaming system, electronic device and computer-readable storage medium. Background Art

[0003] With the development of video and network technologies, applications such as live streaming can, on the basis of providing live video playback for users, also add the display of special effect resources such as live streaming special effects, so as to make the information displayed in the live streaming room more abundant. Among them, special effect resources are usually output after making animations with relevant software and adding them to the rendering queue. For example, a special effect resource file in AVI or MP4 format can be output, and this special effect resource file will be used by the terminal.

[0004] In current technologies, after obtaining the special effect resource file, the terminal can parse and render it onto the screen when the usage conditions are met. For example, when a specific user enters the live streaming room, the terminal can render the corresponding live streaming special effect resource on the live streaming room interface. However, this technology usually directly renders the special effect resource data after parsing the special effect resource data from the special effect resource file. However, the special effect resource styles corresponding to the special effect resource files made by relevant software are relatively fixed and single, resulting in poor rendering effects of this technology for special effect resources. Summary of the Invention

[0005] Based on this, it is necessary to provide a special effect resource rendering method, device, live streaming system, electronic device and computer-readable storage medium for the above technical problems.

[0006] In a first aspect, this application provides a special effect resource rendering method. It is applied to a terminal. The method includes:

[0007] Receiving a special effect resource file and extracting the description information of the special effect resource file; according to the description information, determining whether the special effect resource file contains identification data for indicating that a background of the special effect resource can be added;

[0008] Obtaining the special effect resource data included in the special effect resource file;

[0009] If the special effect resource file also contains identification data for indicating that a background of the special effect resource can be added, obtaining the corresponding background image data;

[0010] If the special effect resource data to be rendered is special effect resource data corresponding to background image data, then according to the frame index of the special effect resource data to be rendered, first render the corresponding background image data and then render the special effect resource data, so as to present the special effect resource with the corresponding background image as the background.

[0011] In one embodiment, if the special effect resource file further includes identification data for indicating that a background can be added to the special effect resource, obtaining the corresponding background image data includes: if the special effect resource file further includes identification data for indicating that a background can be added to the special effect resource, extracting the information on the number of background layers that can be added from the identification data; and obtaining the background image data corresponding to each background level according to the information on the number of background layers that can be added.

[0012] In one embodiment, the special effect resource rendering method further includes: if the special effect resource data to be rendered is special effect resource data corresponding to multiple layers of background image data, determining the background level order, and rendering the background image data corresponding to each background level in sequence according to the background level order and then rendering the special effect resource data.

[0013] In one embodiment, after obtaining the corresponding background image data, it further includes: when the corresponding background image data belongs to the target type of background image data, performing frame matching on the corresponding background image data and the special effect resource data included in the special effect resource file, so that the number of frames of the corresponding background image data is the same as the number of frames of the special effect resource data;

[0014] The step of first rendering the corresponding background image data and then rendering the special effect resource data according to the frame index of the special effect resource data to be rendered includes: obtaining the corresponding target background image data from the corresponding background image data according to the frame index of the special effect resource data to be rendered, and rendering the target background image data; after rendering the target background image data, obtaining the corresponding target special effect resource data from the special effect resource data included in the special effect resource file according to the frame index of the special effect resource data to be rendered, and rendering the target special effect resource data.

[0015] In one embodiment, the corresponding background picture data includes background picture data corresponding to each background level; the step of matching the number of frames of the corresponding background picture data with the special effect resource data included in the special effect resource file includes: for the background picture data of each background level, matching the number of frames of the background picture data of this background level with the special effect resource data included in the special effect resource file, so that the number of frames of the background picture data of each background level is the same as the number of frames of the special effect resource data; the step of obtaining corresponding target background picture data from the corresponding background picture data according to the frame index of the special effect resource data to be rendered and rendering the target background picture data includes: according to the frame index of the special effect resource data to be rendered, obtaining the target background picture data corresponding to each background level from the background picture data corresponding to each background level, and sequentially rendering the target background picture data corresponding to each background level in the order of the background levels.

[0016] In one embodiment, after receiving the special effect resource file, the method further includes: extracting the description information of the special effect resource file; and determining whether the special effect resource file contains identification data for identifying that a background of the special effect resource can be added according to the description information.

[0017] In one embodiment, the step of determining whether the special effect resource file contains identification data for identifying that a background of the special effect resource can be added according to the description information includes: obtaining a preset field in the description information for the special effect resource provider to write identification data according to the description information; and determining whether the special effect resource file contains identification data for identifying that a background of the special effect resource can be added according to the identification data in the preset field of the description information.

[0018] In one embodiment, the step of determining whether the special effect resource file contains identification data for identifying that a background of the special effect resource can be added according to the description information includes: determining a preset identification data keyword for matching in the description information; performing a match in the description information according to the preset identification data keyword; and determining whether the special effect resource file contains identification data for identifying that a background of the special effect resource can be added according to the match result.

[0019] In a second aspect, the present application provides a live broadcast system. The system includes: a terminal and a live broadcast server; wherein, the terminal is configured to render the live broadcast special effect resource data and the corresponding background picture data in the live broadcast special effect resource file sent by the live broadcast server according to the method described above, and play the live broadcast special effect resource with the corresponding background picture as the background in the live broadcast screen.

[0020] In a third aspect, the present application provides a special effect resource rendering device. The device is applied to a terminal. The device includes:

[0021] A file receiving module, configured to receive a special effect resource file, extract description information of the special effect resource file; and determine whether the special effect resource file contains identification data for indicating that a background of the special effect resource can be added according to the description information;

[0022] A data acquisition module, configured to acquire special effect resource data included in the special effect resource file;

[0023] A background acquisition module, configured to acquire corresponding background image data if the special effect resource file further contains identification data for indicating that a background of the special effect resource can be added;

[0024] A data rendering module, configured to, if the special effect resource data to be rendered is special effect resource data corresponding to background image data, first render the corresponding background image data according to a frame index of the special effect resource data to be rendered, and then render the special effect resource data, so as to present a special effect resource with the corresponding background image as the background.

[0025] In a fourth aspect, the present application provides an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0026] Receive a special effect resource file, extract description information of the special effect resource file; determine whether the special effect resource file contains identification data for indicating that a background of the special effect resource can be added according to the description information; acquire special effect resource data included in the special effect resource file; if the special effect resource file further contains identification data for indicating that a background of the special effect resource can be added, acquire corresponding background image data; if the special effect resource data to be rendered is special effect resource data corresponding to background image data, first render the corresponding background image data according to a frame index of the special effect resource data to be rendered, and then render the special effect resource data, so as to present a special effect resource with the corresponding background image as the background.

[0027] In a fifth aspect, the present application provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0028] Receive the special effect resource file, and extract the description information of the special effect resource file; according to the description information, determine whether the special effect resource file contains identification data for indicating that the background of the special effect resource can be added; obtain the special effect resource data contained in the special effect resource file; if the special effect resource file also contains identification data for indicating that the background of the special effect resource can be added, then obtain the corresponding background image data; if the special effect resource data to be rendered is the special effect resource data corresponding to the background image data, then according to the frame index of the special effect resource data to be rendered, first render the corresponding background image data and then render the special effect resource data, so as to present the special effect resource with the corresponding background image as the background.

[0029] In the above special effect resource rendering method, device, live broadcast system, device and medium, the terminal receives the special effect resource file, and then obtains the special effect resource data contained in the special effect resource file. If the special effect resource file also contains identification data for indicating that the background of the special effect resource can be added, the terminal obtains the corresponding background image data; when rendering the special effect resource data, if the special effect resource data to be rendered is the special effect resource data corresponding to the background image data, the terminal renders the corresponding background image data and then renders the special effect resource data, so as to present the special effect resource with the corresponding background image as the background. In this solution, when the terminal renders the special effect resource data, it does not directly render the special effect resource data parsed from the file, but first determines whether the file also contains identification data for indicating that the background of the special effect resource can be added. If so, it is necessary to first render the corresponding background image data and then render the special effect resource data. Thus, the terminal can present the special effect resource with the corresponding background image as the background on the screen, and can also realize the addition and flexible change of the background of the special effect resource on the terminal side, enabling the terminal to dynamically change the background of the special effect resource according to actual business needs, enriching the display methods and styles of the special effect resource on the terminal, and improving the rendering and display effect of the special effect resource. Description of the Drawings

[0030] Figure 1 It is an application scenario diagram of the special effect resource rendering method in an embodiment of the present application;

[0031] Figure 2 It is a flow schematic diagram of the special effect resource rendering method in an embodiment of the present application;

[0032] Figure 3 It is a flow schematic diagram of the special effect resource rendering method in an application example of the present application;

[0033] Figure 4 It is a structural schematic diagram of the live broadcast system in an embodiment of the present application;

[0034] Figure 5 It is an interface schematic diagram of playing the live broadcast special effect resource in an application example of the present application;

[0035] Figure 6 It is a structural block diagram of a special effect resource rendering device in an embodiment of the present application;

[0036] Figure 7 It is an internal structure diagram of an electronic device in an embodiment of the present application. Detailed implementation manners

[0037] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0038] The special effect resource rendering method provided by the embodiments of the present application can be applied to Figure 1 the application scenario shown in the figure. This application scenario may include a terminal 110 and a special effect resource providing end 120. The terminal 110 communicates with the special effect resource providing end 120 through a network. Among them, the terminal 110 may be a device for rendering and presenting special effect resources and other related content according to data information such as special effect resource files. The special effect resource providing end 120 may be a device that provides data information such as special effect resource files to the terminal 110. The terminal 110 can be regarded as a demand end or a presentation end of special effect resources relative to the special effect resource providing end 120. Specifically, the terminal 110 may be, but is not limited to, various personal computers, laptop computers, smart phones, and tablet computers. The special effect resource providing end 120 may be a terminal or a server. The server may be implemented by an independent server or a server cluster composed of multiple servers.

[0039] In one embodiment, as Figure 2 shown in the figure, a special effect resource rendering method is provided. This method may be executed by the terminal 110 shown in Figure 1 the figure, and this method may include the following steps:

[0040] Step S201, receiving a special effect resource file.

[0041] In this step, the terminal 110 can receive a special effect resource file from the special effect resource provider 120. Specifically, the special effect resource file can be a special effect resource file in MP4 format, denoted as the MP4 special effect resource file. In practical applications, when the terminal 110 needs to present a special effect resource, it can request the corresponding special effect resource file from the special effect resource provider 120. The terminal 110 can send a special effect resource file acquisition request to the special effect resource provider 120. After receiving the request, the special effect resource provider 120 can send the corresponding MP4 special effect resource file to the terminal 110, and thus the terminal 110 obtains the MP4 special effect resource file. Additionally, the special effect resource provider 120 can also actively send a special effect resource file to the terminal 110 when it needs the terminal 110 to present a certain special effect resource. After receiving the special effect resource file, the terminal 110 can render it in a certain manner according to the special effect resource data in the special effect resource file to present the corresponding special effect resource on the screen canvas. As an example, taking the live special effect resource as an example, when a specific user enters the live broadcast room, the special effect resource provider 120 can send a live special effect resource file to the terminal 110. After receiving the live special effect resource file, the terminal 110 can render it according to the live special effect resource data in the live special effect resource file to play the live special effect resource such as welcoming the specific user into the live broadcast room on the screen canvas.

[0042] Step S202: Obtain the special effect resource data included in the special effect resource file.

[0043] In this step, the terminal 110 can unpack the special effect resource file, extract the video track, and obtain the special effect resource data. The special effect resource data can be decoded by the terminal 110 into a frame queue, and the frame queue can contain the special effect resource data of each frame that needs to be rendered by the terminal 110. Each frame of special effect resource data can correspond to its own frame index.

[0044] Step S203: If the special effect resource file also includes identification data for indicating that the background of the special effect resource can be added, obtain the corresponding background image data.

[0045] In this step, after the terminal 110 unpacks the special effect resource file, if it extracts identification data contained in the special effect resource file for identifying that a background can be added to the special effect resource, the terminal 110 can also obtain corresponding background image data. The corresponding background image data can be background image data pre-stored on the terminal 110, or can be background image data sent by the special effect resource provider 120 to the terminal 110. The background image data can be background image data of types such as GIF and PNG. The type and content of the background image data can be changed according to the requirements of the actual business scenario. That is, for the same special effect resource, the corresponding background image data obtained by the terminal 110 each time can be changed in terms of type and content according to the requirements of the actual business scenario. The terminal 110 can obtain GIF background image data or can obtain PNG background image data. It can obtain GIF background image data A or can obtain GIF background image data B, and can also obtain PNG background image data A or PNG background image data B.

[0046] Specifically, in addition to the special effect resource data, the special effect resource file sent by the special effect resource provider 120 can also contain identification data that can be written into the special effect resource file by the special effect resource provider 120, for example. The identification data can be written into the special effect resource file in the form of json data. The identification data can be used by the special effect resource provider 120 to identify that the special effect resource corresponding to the special effect resource file has the attribute that a background can be added. Specifically, in a case where, for example, the special effect resource provider 120 needs to identify that the special effect resource corresponding to the special effect resource file has the attribute that a background can be added, the special effect resource provider 120 can write background1 as the identification data into the special effect resource file, and then send the special effect resource file containing the special effect resource data and the identification data to the terminal 110. Since the special effect resource provider 120 identifies in the special effect resource file that the background of the special effect resource is addable, the terminal 110 accordingly also needs to obtain the corresponding background image data, which can be pre-stored on the terminal 110 or can be passed in by the special effect resource provider 120. Thus, when the terminal 110 unpacks the special effect resource file, it can detect whether the special effect resource file contains identification data for identifying that the background of the special effect resource is addable. If it contains, the terminal 110 can obtain the corresponding background image data and cache it in the content for use when rendering.

[0047] Step S204, if the special effect resource data to be rendered is special effect resource data corresponding to background image data, then render the background image data and then render the special effect resource data to present the special effect resource with the corresponding background image as the background.

[0048] This step is the stage where the terminal 110 renders the special effect resource data. In this stage, the terminal 110 first determines whether the special effect resource data to be rendered is the special effect resource data corresponding to the background image data. If not, the terminal 110 can directly render the special effect resource data. If the special effect resource data to be rendered is the special effect resource data corresponding to the background image data, then the terminal 110 needs to render the special effect resource data and its corresponding background image data in a certain order. The terminal 110 needs to first render the corresponding background image data in the above step S203, and then render the special effect resource data. Thus, the terminal 110 can present the special effect resource with the corresponding background image as the background on the screen canvas. In practical applications, if a basic screen, such as a live broadcast screen, is already displayed on the terminal 110, then this step is on the upper layer of the live broadcast screen. First, the background image corresponding to the live broadcast special effect resource is rendered and displayed, and then the live broadcast special effect resource is rendered and displayed after the background image is rendered. Therefore, in terms of the relative display hierarchy of the three, the live broadcast screen is located on the lower layer, the live broadcast special effect resource is located on the upper layer, and the background image is between the two. Thus, the effect of presenting the special effect resource with the corresponding background image as the background on the screen canvas is achieved. And as mentioned before, for the same special effect resource, the corresponding background image data obtained by the terminal 110 can be different, and can be dynamically changed according to the actual business scenario, obtaining the background image data corresponding to the scenario, supporting the terminal side to dynamically change the background of the special effect resource according to the business scenario.

[0049] In the special effect resource rendering method provided in this embodiment, the terminal 110 receives the special effect resource file, and then obtains the special effect resource data included in the special effect resource file. If the special effect resource file also includes the identification data for identifying that the background of the special effect resource can be added, then the terminal 110 obtains the corresponding background image data; when rendering the special effect resource data, if the special effect resource data to be rendered is the special effect resource data corresponding to the background image data, then the terminal 110 renders the corresponding background image data and then renders the special effect resource data to present the special effect resource with the corresponding background image as the background. In this solution, when the terminal 110 renders the special effect resource data, it does not directly render the special effect resource data parsed from the file, but first determines whether the file also includes the identification data for identifying that the background of the special effect resource can be added. If so, it is necessary to first render the corresponding background image data and then render the special effect resource data. Thus, the terminal 110 can present the special effect resource with the corresponding background image as the background on the screen, and also realizes the addition and flexible change of the background of the special effect resource on the terminal side, enabling the terminal 110 to dynamically change the background of the special effect resource according to the actual business requirements, enriching the display methods and styles of the special effect resource on the terminal, and improving the rendering and display effect of the special effect resource.

[0050] In some embodiments, if the special effect resource file further includes identification data for indicating that a background can be added to the special effect resource, obtaining the corresponding background image data in step S203 may include:

[0051] If the special effect resource file further includes identification data for indicating that a background can be added to the special effect resource, extract the background addable layer information from the identification data; according to the background addable layer information, obtain the background image data corresponding to each background level.

[0052] Specifically, as described above, in addition to the special effect resource data in the special effect resource file sent by the special effect resource provider 120, for example, the special effect resource provider 120 can also write identification data for indicating that a background can be added to the special effect resource. In this embodiment, the identification data written by the special effect resource provider 120 can also be specifically used to carry information about the number of background pictures that can be added to the special effect resource, that is, the background addable layer information. Based on this background addable layer information, the terminal 110 can add multiple layers of background pictures to the special effect resource, further enriching the display method and style of the special effect resource and improving its rendering display effect. Specifically, the special effect resource provider 120 can write background3 as identification data into the special effect resource file. Among them, background in background3 can be used to indicate that a background can be added to the special effect resource, and 3 in it can be used as the carried background addable layer information, that is, the number of addable background layers is 3. Thus, the terminal 110 obtains the background image data corresponding to each background level according to this background addable layer information.

[0053] Based on this, in one embodiment, if the special effect resource data to be rendered is special effect resource data corresponding to background image data, rendering the background image data and then rendering the special effect resource data in step S204 further includes:

[0054] If the special effect resource data to be rendered is special effect resource data corresponding to multiple layers of background image data, determine the background level order, and render the background image data corresponding to each background level in sequence according to the background level order and then render the special effect resource data.

[0055] In this embodiment, the terminal 110 parses the identification data for adding the background of the special effect resource from the special effect resource file and can extract the information on the number of background layers that can be added therefrom. Then, based on this, the terminal obtains the background image data corresponding to each background level. Based on this, in the rendering stage of the special effect resource data, if the special effect resource data to be rendered is the special effect resource data corresponding to multiple layers of background image data, then in addition to rendering the background first and then the special effect resource in sequence, when rendering the background, the terminal 110 also needs to render in the order of the background levels so that the multiple layers of background can be correctly rendered and displayed. Suppose the number of background layers that can be added is 2. After the terminal obtains the background image data corresponding to the two background levels, it obtains the background level order of the two layers of background image data. This background level order can be determined in advance, that is, the level order can be determined after the two layers of background image data are produced. For example, when the background image data A is produced, it is marked as the background image data of the first background level, and the background image data B is marked as the background image data of the second background level, and so on. The background image data of each background level can include multiple, and their types and contents can also be different. The terminal 110 can obtain the background image data from the corresponding background level according to the information on the number of background layers that can be added as the background image data for adding the background to the special effect resource in the corresponding background level to enrich the display methods and styles of the special effect resource. Then, the terminal 110 can render the background image data corresponding to the second background level first in the order of the background levels, and then render the background image data corresponding to the first background level. After the background image data corresponding to the two background levels are rendered, the special effect resource data is rendered. Thus, the effect of playing the special effect resource with the corresponding multiple layers of background images as the background can be achieved, so that the display methods and styles of the special effect resource are further enriched, and the display effect of the special effect resource is further improved.

[0056] In some embodiments, after obtaining the corresponding background image data in step S203, the following steps may further be included:

[0057] When the corresponding background image data belongs to the target type of background image data, the corresponding background image data is frame-matched with the special effect resource data included in the special effect resource file so that the number of frames of the corresponding background image data is the same as the number of frames of the special effect resource data.

[0058] In this embodiment, the number of frames of the special effect resource data that the terminal 110 needs to render is usually multiple frames. When the terminal 100 needs to add a background image, the number of frames of the corresponding background image data obtained may not match it, which will affect the display effect of the background image added to the special effect resource. To ensure that the background image can be accurately added, after obtaining the corresponding background image data, the terminal 110 determines whether the background image data belongs to the target type of background image data. The target type of background image data can be GIF format background image data containing multiple frames. If the background image data belongs to the target type of background image data, the terminal 110 can perform frame number matching processing on it. The terminal 110 matches the frame number of the background image data with the special effect resource data included in the special effect resource file to make the number of frames of the background image data the same as that of the special effect resource data. Specifically, when the number of frames of the background image data is less than that of the special effect resource data, the number of frames can be increased according to the background image data to make it the same as that of the special effect resource data. When the number of frames of the background image data is greater than that of the special effect resource data, the number of frames can be decreased according to the background image data to make it the same as that of the special effect resource data. Thus, it can be ensured that the background image data of the corresponding frame can be obtained and rendered first before rendering each frame of the special effect resource data.

[0059] Based on this, rendering the special effect resource data after rendering the background image data in step S204 further includes:

[0060] According to the frame index of the special effect resource data to be rendered, obtain the corresponding target background image data from the corresponding background image data, and render the target background image data; after rendering the target background image data, according to the frame index of the special effect resource data to be rendered, obtain the corresponding target special effect resource data from the special effect resource data included in the special effect resource file, and render the target special effect resource data.

[0061] In this embodiment, as described above, the number of frames of the special effect resource data that the terminal 110 needs to render is usually multiple frames. In the rendering stage, the terminal 110 can obtain the frame index of the special effect resource data to be rendered. Since the number of frames of the background image data has been matched with the number of frames of the special effect resource data in the foregoing embodiment, the terminal 110 can obtain the background image data corresponding to this frame index, that is, the target background image data, from the background image data with matched number of frames according to the frame index of the special effect resource data to be rendered, and then render the target background image data. After rendering the target background image data, the terminal 110 can then obtain the special effect resource data corresponding to this frame index, that is, the target special effect resource data, from the special effect resource data included in the special effect resource file according to the frame index of the special effect resource data to be rendered, and then render the target special effect resource data. Thus, the terminal 110 can accurately and effectively add the corresponding background image to each frame of the special effect resource on the basis of frame number matching, making the background addition effect of the special effect resource more stable and reliable.

[0062] Further, in some embodiments, the corresponding background image data may include the background image data corresponding to each background level. Thus, the above-mentioned matching of the corresponding background image data with the special effect resource data included in the special effect resource file further includes: for the background image data of each background level, matching the background image data of this background level with the special effect resource data included in the special effect resource file, so that the number of frames of the background image data of each background level is the same as the number of frames of the special effect resource data. Specifically, when the corresponding background image data includes the background image data corresponding to each background level, the terminal 110 performs frame number matching processing on the background image data of each background level. For example, if the corresponding background image data includes the GIF background image data A of the first background level and the GIF background image data B of the second background level, the terminal 110 matches the GIF background image data A of the first background level with the special effect resource data, and matches the GIF background image data B of the second background level with the special effect resource data, so that the number of frames of the GIF background image data A of the first background level is the same as the number of frames of the special effect resource data, and the number of frames of the GIF background image data B of the second background level is also the same as the number of frames of the special effect resource data, to ensure that the background image data of each layer corresponding to the corresponding frame can be obtained and rendered first before rendering each frame of the special effect resource data in the case of multiple layers of backgrounds.

[0063] Accordingly, obtaining corresponding target background image data from the corresponding background image data according to the frame index of the special effect resource data in the above embodiments and rendering the target background image data further includes: obtaining the target background image data corresponding to each background level from the background image data corresponding to each background level according to the frame index of the special effect resource data to be rendered, and sequentially rendering the target background image data corresponding to each background level in the order of the background levels. Specifically, as described above, when the corresponding background image data includes the background image data corresponding to each background level, the terminal 110 will perform frame matching on the background image data of each background level with the special effect resource data respectively. Based on this, in the rendering stage, after obtaining the frame index of the special effect resource data to be rendered, the terminal 110 can obtain the background image data corresponding to the frame index in each background level in the background image data after frame matching, that is, the target background image data, and obtain the target background image data corresponding to each background level. Since the target background image data of each frame has the corresponding background level order, the terminal 110 sequentially renders the target background image data corresponding to each background level in the order of the background levels. For example, for the first frame, the terminal 110 can first render the target background image data corresponding to the second background level, and then render the target background image data corresponding to the first background level. After the target background image data corresponding to these two background levels are rendered, the special effect resource data of the first frame is rendered, and then the rendering of the second and subsequent frames is performed in sequence. Thus, the terminal 110 can accurately and effectively add the background images of the corresponding background levels to each frame of the special effect resource on the basis of performing frame matching on each background level, achieving the effect of playing the special effect resource with the corresponding multi-layer background images as the background, and making the playing effect more stable and reliable.

[0064] In some embodiments, after receiving the special effect resource file in step S201, the above method further includes the following steps:

[0065] Extracting the description information of the special effect resource file; and determining whether the special effect resource file contains identification data for identifying that the background of the special effect resource can be added according to the description information.

[0066] In this embodiment, the special effect resource provider 120 can write the identification data for identifying that the background of the special effect resource can be added into the description information of the special effect resource file and send it to the terminal 110. After receiving the special effect resource file, the terminal 110 unpacks it, extracts the special effect resource data and the description information, and determines whether the description information of the special effect resource file contains the identification data for identifying that the background of the special effect resource can be added. Specifically, in combination with Figure 3, the MP4 special effect resource file provided by the special effect resource provider 120 may include special effect resource data and description information. After receiving the MP4 special effect resource file, the terminal 110 extracts the special effect resource data and description information. If the description information contains identification data that can be used to identify the background of the special effect resource that can be added, the terminal 110 can obtain the corresponding background image data passed in and cache it. When rendering each frame, according to the frame index, the special effect resource data to be rendered, that is, the target special effect resource data and the target background image data, are obtained. First, the target background image data is rendered, and then the target special effect resource data is rendered. Specifically, the terminal 110 unpacks the MP4 special effect resource file and extracts the special effect resource data and description information. On the one hand, the terminal 110 can decode to obtain a frame queue containing multiple frames of special effect resource data. On the other hand, the terminal 110 can determine whether the extracted description information contains identification data that can be used to identify the background of the special effect resource that can be added. If so, the terminal 110 obtains the corresponding background image data and caches it in the memory. In the rendering stage, the terminal 110 obtains the frame index of the special effect resource data to be rendered. If the special effect resource data to be rendered has corresponding background image data, the terminal 110 obtains and renders the target background image data according to the frame index, and then can obtain the target special effect resource data from the frame queue according to the frame index and render it, so as to present the special effect resource with the corresponding background image as the background.

[0067] Further, in some embodiments, determining whether the special effect resource file contains identification data that can be used to identify the background of the special effect resource that can be added according to the description information in the above embodiments may include:

[0068] According to the description information, obtain the preset field in the description information for the special effect resource provider to write the identification data; according to the identification data in the preset field in the description information, determine whether the special effect resource file contains identification data that can be used to identify the background of the special effect resource that can be added.

[0069] In this embodiment, the terminal 110 can obtain, through parsing, a preset field in the description information for the special effect resource provider 120 to write identification data, such as the metadata (meta - data) field. The special effect resource provider 120 can write identification data in the preset field of the description information. The identification data can be identification data for indicating that the background of the special effect resource can be added, such as "background", or identification data for indicating that the background of the special effect resource can be added and carrying information about the number of layers that the background can be added, such as "background3", or identification data for indicating that the background of the special effect resource cannot be added, such as "background0", further improving the flexibility of setting the background addable attribute of the special effect resource. Thus, after the terminal 110 parses and obtains the preset field in the description information for the special effect resource provider 120 to write identification data, it can extract the identification data of the preset field, thereby determining whether the special effect resource file contains identification data for indicating that the background of the special effect resource is variable, and can further facilitate the extraction of information such as the number of layers that the background can be added from it.

[0070] Further, in some other embodiments, determining whether the special effect resource file contains identification data for indicating that the background of the special effect resource can be added according to the description information in the foregoing embodiments may include:

[0071] Determining a preset identification data keyword for matching in the description information; performing a match in the description information according to the preset identification data keyword; and determining whether the special effect resource file contains identification data for indicating that the background of the special effect resource can be added according to the match result.

[0072] In this embodiment, the terminal 110 can determine whether the special effect resource file contains identification data for indicating that the background of the special effect resource can be added by keyword matching. The special effect resource provider 120 can use a string containing a preset keyword as the identification data for indicating that the background of the special effect resource can be added according to a predetermined rule. The string of the preset keyword can be used as the keyword of the preset identification data. For example, "background" can be used as the keyword of the preset identification data. The special effect resource provider 120 can directly write "background" as the identification data for indicating that the background of the special effect resource can be added into the description information of the special effect resource file. The keyword of the preset identification data can be pre-sent by the special effect resource provider 120 to the terminal 110 for storage. Thus, after obtaining the special effect resource file, the terminal 110 can match in the description information of the special effect resource file according to the determined keyword of the preset identification data for matching in the description information. If the matching result indicates that the description information contains the keyword of the preset identification data, the terminal 110 can determine that the special effect resource file contains the identification data for indicating that the background of the special effect resource can be added. Thus, the terminal 110 can relatively simply and quickly determine whether the special effect resource file contains the identification data for indicating that the background of the special effect resource can be changed.

[0073] In one embodiment, as Figure 4 shown, a live broadcast system is provided. The system can include a terminal and a live broadcast server. Among them, the terminal can communicate with the live broadcast server through the Internet. In this embodiment, the live broadcast server can send a live broadcast special effect resource file to the terminal, and can also send corresponding background image data. Among them, the terminal of the live broadcast system can render the live broadcast special effect resource data and the corresponding background image data in the live broadcast special effect resource file sent by the live broadcast server according to the special effect resource rendering method described in any of the above embodiments, so as to play the live broadcast special effect resource with the corresponding background image as the background in the live broadcast screen. As an application example, as Figure 5 shown, the terminal of the live broadcast system can play the live broadcast special effect resource 520 with the corresponding background image 530 as the background in the live broadcast screen 510. From the relative hierarchy of the three, the live broadcast screen is located at the lower layer, the live broadcast special effect resource is located at the upper layer, and the background image is in between, achieving the effect of presenting the live broadcast special effect resource with the corresponding background image as the background on the screen canvas. And for the same live broadcast special effect resource, the background image added by the terminal can be different. Thus, the terminal can dynamically change the background of the live broadcast special effect resource according to the needs of the service scenario. Thus, this embodiment can apply the special effect resource rendering method provided by this application to the live broadcast system, so that in the live broadcast scenario, the terminal side can add the background of the live broadcast special effect resource, which can support dynamic changes according to the live broadcast service scenario, enrich the display methods and styles of the live broadcast special effect resource on the terminal, and improve the rendering and display effect of the live broadcast special effect resource.

[0074] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are sequentially shown according to the indications of the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same moment, but can be executed at different moments. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0075] Based on the same inventive concept, an embodiment of the present application further provides a special effect resource rendering device for implementing the special effect resource rendering method described above. The implementation solution provided by this device for solving problems is similar to the implementation solution described in the above method. Therefore, the specific limitations in one or more embodiments of the special effect resource rendering device provided below can refer to the limitations on the special effect resource rendering method in the above text, and will not be repeated here.

[0076] In one embodiment, as Figure 6 shown, a special effect resource rendering device is provided. This device can be applied to a terminal. The device 600 may include:

[0077] A file receiving module 601, configured to receive a special effect resource file;

[0078] A data acquisition module 602, configured to acquire the special effect resource data included in the special effect resource file;

[0079] A background acquisition module 603, configured to, if the special effect resource file further includes identification data for identifying that a background of the special effect resource can be added, acquire the corresponding background image data;

[0080] A data rendering module 604, configured to, if the special effect resource data to be rendered is special effect resource data corresponding to background image data, render the background image data and then render the special effect resource data to present a special effect resource with the corresponding background image as the background.

[0081] In one embodiment, the background acquisition module 603 is configured to, if the special effect resource file further includes identification data for identifying that a background of the special effect resource can be added, extract the background addable layer number information from the identification data; and acquire the background image data corresponding to each background level according to the background addable layer number information.

[0082] In one embodiment, the data rendering module 604 is configured to, if the special effect resource data to be rendered is special effect resource data corresponding to multiple layers of background image data, determine the background level order, and render the background image data corresponding to each background level in sequence according to the background level order, and then render the special effect resource data.

[0083] In one embodiment, the background acquisition module 603 may further be configured to, when the corresponding background image data belongs to the target class of background image data, perform frame number matching on the corresponding background image data and the special effect resource data included in the special effect resource file, so that the number of frames of the corresponding background image data is the same as the number of frames of the special effect resource data; the data rendering module 604 is configured to obtain the corresponding target background image data from the corresponding background image data according to the frame index of the special effect resource data to be rendered, and render the target background image data; after rendering the target background image data, obtain the corresponding target special effect resource data from the special effect resource data included in the special effect resource file according to the frame index of the special effect resource data to be rendered, and render the target special effect resource data.

[0084] In one embodiment, the corresponding background image data includes the background image data corresponding to each background level; the background acquisition module 603 may further be configured to, for the background image data of each background level, perform frame number matching on the background image data of this background level and the special effect resource data included in the special effect resource file, so that the number of frames of the background image data of each background level is the same as the number of frames of the special effect resource data; the data rendering module 604 is configured to obtain the target background image data corresponding to each background level from the background image data corresponding to each background level according to the frame index of the special effect resource data to be rendered, and render the target background image data corresponding to each background level in sequence according to the background level order.

[0085] In one embodiment, the apparatus 600 may further include: an identification determination module, configured to extract the description information of the special effect resource file; and determine whether the special effect resource file includes identification data for identifying that a special effect resource background can be added according to the description information.

[0086] In one embodiment, the identification determination module may be configured to obtain a preset field in the description information for the special effect resource provider to write identification data according to the description information; and determine whether the special effect resource file includes identification data for identifying that a special effect resource background can be added according to the identification data in the preset field of the description information.

[0087] In one embodiment, an identification determination module can be used to determine a preset identification data keyword for matching in the description information; perform matching in the description information according to the preset identification data keyword; and determine whether the special effect resource file contains identification data that can be added to identify the special effect resource background according to the matching result.

[0088] Each module in the above special effect resource rendering device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor of the electronic device in hardware form or independent of it, or stored in the memory of the electronic device in software form, so that the processor can call and execute the operations corresponding to each of the above modules.

[0089] In one embodiment, an electronic device is provided. The electronic device can be a terminal, and its internal structure diagram can be as Figure 7 shown. The electronic device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a special effect resource rendering method. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the electronic device, or an external keyboard, touchpad, or mouse, etc.

[0090] Those skilled in the art can understand that Figure 7 the structure shown in

[0091] is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the electronic device to which the solution of the present application is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0092] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps in the above method embodiments.

[0093] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tapes, floppy disks, flash memories, optical memories, high-density embedded non-volatile memories, resistive random access memories (ReRAMs), magnetoresistive random access memories (MRAMs), ferroelectric random access memories (FRAMs), phase change memories (PCMs), graphene memories, etc. Volatile memories can include random access memory (RAM) or external cache memories, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in this application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0094] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope described in this specification.

[0095] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A special effect resource rendering method, characterized in that Applied to a terminal, the method includes: Receiving a special effect resource file, extracting description information of the special effect resource file; according to the description information, determining whether the special effect resource file contains identification data for indicating that a background of the special effect resource can be added; Obtaining special effect resource data included in the special effect resource file; If the special effect resource file further contains identification data for indicating that a background of the special effect resource can be added, obtaining corresponding background image data; If the special effect resource data to be rendered is special effect resource data corresponding to background image data, rendering the corresponding background image data first according to a frame index of the special effect resource data to be rendered, and then rendering the special effect resource data, so as to present a special effect resource with the corresponding background image as the background.

2. The method according to claim 1, wherein The step of if the special effect resource file further contains identification data for indicating that a background of the special effect resource can be added, obtaining corresponding background image data includes: If the special effect resource file further contains identification data for indicating that a background of the special effect resource can be added, extracting background addable layer information from the identification data; According to the background addable layer information, obtaining background image data corresponding to each background level.

3. The method according to claim 2, wherein It further includes: If the special effect resource data to be rendered is special effect resource data corresponding to multiple layers of background image data, determining a background level order, and rendering the background image data corresponding to each background level in sequence according to the background level order, and then rendering the special effect resource data.

4. The method according to claim 1, characterized in that, After obtaining the corresponding background image data, it further includes: When the corresponding background image data belongs to target type background image data, performing frame number matching on the corresponding background image data and the special effect resource data included in the special effect resource file, so that the number of frames of the corresponding background image data is the same as the number of frames of the special effect resource data; The step of rendering the corresponding background image data first according to a frame index of the special effect resource data to be rendered, and then rendering the special effect resource data includes: According to the frame index of the special effect resource data to be rendered, obtaining corresponding target background image data from the corresponding background image data, and rendering the target background image data; After rendering the target background image data, according to the frame index of the special effect resource data to be rendered, obtaining corresponding target special effect resource data from the special effect resource data included in the special effect resource file, and rendering the target special effect resource data.

5. The method according to claim 4, wherein The corresponding background image data includes background image data corresponding to each background level; The step of performing frame number matching on the corresponding background image data and the special effect resource data included in the special effect resource file includes: For the background image data of each background level, performing frame number matching on the background image data of the background level and the special effect resource data included in the special effect resource file, so that the number of frames of the background image data of each background level is the same as the number of frames of the special effect resource data; The step of obtaining corresponding target background image data from the corresponding background image data according to the frame index of the special effect resource data to be rendered, and rendering the target background image data includes: According to the frame index of the special effect resource data to be rendered, obtain the target background image data corresponding to each background level from the background image data corresponding to each background level, and render the target background image data corresponding to each background level in sequence according to the background level order.

6. The method according to claim 1, wherein The determining whether the special effect resource file contains identification data for indicating that a background can be added to the special effect resource according to the description information includes: According to the description information, obtain the preset field in the description information for the special effect resource provider to write the identification data; According to the identification data in the preset field in the description information, determine whether the special effect resource file contains identification data for indicating that a background can be added to the special effect resource.

7. The method according to claim 1, characterized in that, The determining whether the special effect resource file contains identification data for indicating that a background can be added to the special effect resource according to the description information includes: Determine the preset identification data keyword for matching in the description information; Match according to the preset identification data keyword in the description information; According to the matching result, determine whether the special effect resource file contains identification data for indicating that a background can be added to the special effect resource.

8. An special effect resource rendering device, characterized in that, Applied to a terminal, the device includes: A file receiving module, configured to receive a special effect resource file, extract the description information of the special effect resource file; according to the description information, determine whether the special effect resource file contains identification data for indicating that a background can be added to the special effect resource; A data acquisition module, configured to acquire the special effect resource data included in the special effect resource file; A background acquisition module, configured to, if the special effect resource file further contains identification data for indicating that a background can be added to the special effect resource, acquire the corresponding background image data; A data rendering module, configured to, if the special effect resource data to be rendered is special effect resource data corresponding to background image data, according to the frame index of the special effect resource data to be rendered, first render the corresponding background image data and then render the special effect resource data, so as to present the special effect resource with the corresponding background image as the background.

9. A live broadcast system, characterized in that, The system includes: a terminal and a live broadcast server; wherein, The terminal is configured to render the live broadcast special effect resource data and the corresponding background image data in the live broadcast special effect resource file sent by the live broadcast server according to the method according to any one of claims 1 to 7, and play the live broadcast special effect resource with the corresponding background image as the background in the live broadcast screen.

10. An electronic device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images