Virtual gift special effect rendering method and device, live broadcast system, equipment and medium

Abstract

The invention relates to a virtual gift special effect rendering method and device, a live broadcast system, electronic equipment and a computer readable storage medium. The method comprises the following steps: acquiring a gift animation file of a virtual gift; wherein the gift animation file comprises gift animation frames which store animation contents and corresponding special effect information at the same time; decoding the gift animation file to obtain a plurality of gift animation frames; wherein the gift animation frame comprises an animation content area and a special effect information area; respectively performing rendering calculation on the animation content area and the special effect information area to obtain animation content and special effect information; rendering the virtual gift according to the special effect information and the animation content; according to the technical scheme, the animation content and the special effect information corresponding to the animation content are stored in the gift animation frame at the same time, so that the special effect information is prevented from being input from the outside during rendering, the rendering complexity is reduced, the rendering performance is enhanced, and the video live broadcast effect is improved.

Description

Technical Field

[0001] The present application relates to the field of live broadcast technology, and in particular to a method, device, live broadcast system, electronic device and computer-readable storage medium for rendering virtual gift special effects in video live broadcast. Background Art

[0002] With the development of network technology, live streaming has been used by most Internet users. With its intuitive, immediacy and interactivity in content and form, live streaming has played an important role in promoting flexible employment, promoting economic and social development, and enriching the spiritual and cultural life of the people. Anchors can better display their talents in live broadcasts, thereby helping more anchors realize their self-worth.

[0003] During live video broadcasts, viewers and hosts usually interact by giving each other virtual gifts. The special effects of exquisite virtual gifts can attract viewers, improve interaction in the live broadcast room, and increase the live broadcast room's activity. However, the more exquisite the special effects of virtual gifts, the more complex the rendering process, and the higher the hardware performance requirements of the user's terminal device.

[0004] In conventional technology, the rendering process of each frame of gift animation of virtual gifts requires real-time external input of special effect information when realizing special effects, that is, additional information content added to realize special effects. In the image rendering operation of the terminal device, the decoding and processing of these additional information contents consumes more hardware performance. For users with lower performance terminal devices, it is easy to cause problems such as virtual gift playback freezes, and even affect the smoothness of the live video, seriously affecting the live video effect. Summary of the invention

[0005] Based on this, it is necessary to provide a virtual gift special effects rendering method, device, live broadcast system, electronic device and computer-readable storage medium, which can reduce the performance consumed by virtual gift special effects rendering and improve the video live broadcast effect.

[0006] A virtual gift special effects rendering method, comprising:

[0007] Acquire a gift animation file of a virtual gift; wherein the gift animation file includes a gift animation frame that simultaneously stores animation content and corresponding special effect information;

[0008] Decoding the gift animation file to obtain a plurality of gift animation frames; wherein the gift animation frame includes an animation content area and a special effect information area;

[0009] Rendering and calculating the animation content area and the special effect information area respectively to obtain the animation content and the special effect information;

[0010] The virtual gift is rendered according to the special effect information and the animation content.

[0011] In one embodiment, the virtual gift special effects rendering method further includes:

[0012] Configure the animation content area and special effect information area;

[0013] The animation content of the virtual gift is saved in the animation content area;

[0014] The special effect information corresponding to the animation content is stored in the special effect information area;

[0015] The animation content area and the special effect information area are synthesized into a gift animation frame.

[0016] In one embodiment, configuring the animation content area and the special effect information area includes:

[0017] Configure the animation content area and special effect information area with the same resolution and size;

[0018] The layout of the animation content area and the special effect information area is determined according to the width and height of the domain resolution of the animation content area; if the width of the animation content area is greater than the height, the animation content area and the special effect information area are arranged up and down, otherwise they are arranged left and right.

[0019] In one embodiment, the special effect information corresponding to the animation content is stored in a special effect information area, including:

[0020] Obtain special effect information of each pixel of the animation content of the virtual gift;

[0021] Convert special effect information into RGB channel values ​​according to preset encoding rules;

[0022] The RGB channel values ​​are saved to the pixel points corresponding to the special effect information area.

[0023] In one embodiment, the special effect information includes: transparency of animation content, index of dynamic element and coordinate information of dynamic element.

[0024] In one embodiment, converting the special effect information into RGB channel values ​​according to a preset encoding rule includes:

[0025] Convert the transparency of the animation content and the index of the dynamic element into R channel values, and convert the coordinate information of the dynamic element into G channel values ​​and B channel values ​​respectively.

[0026] In one embodiment, the transparency of the animation content and the index of the dynamic element are converted into an R channel value, including:

[0027] Convert the index of the dynamic element to the last n bits of the R channel value;

[0028] The transparency of the animation content is calculated by a mapping function as a virtual transparency value, and the virtual transparency value is converted into the first m bits of the R channel value;

[0029] Among them, the total number of bits of the R channel value is n+m.

[0030] In one embodiment, obtaining a gift animation file of a virtual gift includes:

[0031] Load the gift animation file and configuration information of the virtual gift from the live broadcast server;

[0032] The decoding of the gift animation file to obtain a plurality of gift animation frames includes:

[0033] The gift animation file is decoded to obtain gift animation frames, and the animation content area and special effect information area of ​​each gift animation frame are determined according to the configuration information.

[0034] In one embodiment, the configuration information includes: the layout of the animation content area and the special effect information area and the number of digits n occupied by the index of the dynamic element.

[0035] In one embodiment, rendering and calculating the animation content area and the special effect information area to obtain the animation content and the special effect information respectively includes:

[0036] Rendering and calculating the animation content area of ​​the gift animation frame to obtain the animation content texture;

[0037] Rendering and calculating the special effect area of ​​the gift animation frame to obtain a special effect information texture;

[0038] The dynamic element texture array is obtained by rendering calculation according to the index of the dynamic element and its corresponding dynamic element image.

[0039] In one embodiment, rendering the virtual gift according to the special effect information and the animation content includes:

[0040] Sample the animation content texture to obtain the animation RGB channel value of each pixel;

[0041] Sample the special effect information texture to obtain the special effect RGB channel value of each pixel;

[0042] Obtaining the transparency of the animation content of each pixel and the RGBA value of the dynamic element according to the special effect RGB channel value and the dynamic element texture array;

[0043] The animation RGB channel value of each pixel, the transparency of the animation content and the RGBA value of the dynamic element are fused and rendered.

[0044] In one embodiment, obtaining the transparency of the animation content of each pixel and the RGBA value of the dynamic element according to the special effect RGB channel value and the dynamic element texture array includes:

[0045] Obtain a virtual transparency value according to the special effect RGB channel value, determine a mapping function according to the number of bits n occupied by the index of the dynamic element, and calculate the transparency of the animation content according to the mapping function and the virtual transparency value;

[0046] According to the index of the dynamic element and the coordinate information of the dynamic element, the corresponding coordinate point of the dynamic element texture in the dynamic element texture array is sampled to obtain the RGBA value of the dynamic element.

[0047] A virtual gift special effects rendering device, comprising:

[0048] An acquisition module, used to acquire a gift animation file of a virtual gift; wherein the gift animation file includes a gift animation frame that simultaneously stores animation content and corresponding special effect information;

[0049] A decoding module, used for decoding the gift animation file to obtain a plurality of gift animation frames; wherein the gift animation frame includes an animation content area and a special effect information area;

[0050] A calculation module, used for performing rendering calculations on the animation content area and the special effect information area respectively to obtain the animation content and the special effect information;

[0051] A rendering module is used to render the virtual gift according to the special effect information and animation content.

[0052] A live broadcast system, comprising: an anchor terminal, an audience terminal and a live broadcast server; wherein the anchor terminal and the audience terminal are respectively connected to the live broadcast server via a communication network;

[0053] The anchor terminal is used to access the anchor in the live broadcast room and collect the anchor's live video stream and upload it to the live broadcast server;

[0054] The live broadcast server is used to forward the live broadcast video stream of the anchor to the audience end, and store the gift animation file of the virtual gift;

[0055] The audience terminal is used for audience users accessing the live broadcast room, receiving the live video stream of the anchor for playback, and using the virtual gift special effect rendering method to play the gift animation file of the virtual gift.

[0056] An electronic device, comprising:

[0057] one or more processors;

[0058] Memory;

[0059] One or more applications, wherein the one or more applications are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs are configured to execute the steps of the virtual gift special effects rendering method.

[0060] A computer-readable storage medium stores at least one instruction, at least one program, a code set or an instruction set, wherein the at least one instruction, the at least one program, the code set or the instruction set is loaded by the processor and executes the steps of the virtual gift special effect rendering method.

[0061] The technical solution of the present application saves the animation content and its corresponding special effect information in the gift animation frame of the gift animation file at the same time. When rendering the virtual gift special effect, the gift animation file of the virtual gift is obtained, the gift animation file is decoded to obtain multiple gift animation frames, the animation content area and the special effect information area of ​​the gift animation frame are respectively rendered and calculated to obtain the animation content and the special effect information, and the virtual gift is rendered according to the special effect information and the animation content. This technical solution saves the animation content and its corresponding special effect information in the gift animation frame at the same time, thereby avoiding the need to input special effect information from the outside during rendering, reducing rendering complexity, enhancing rendering performance, and improving the video live broadcast effect.

[0062] Furthermore, the gift animation file of the virtual gift is produced in the video format, and the RGB channel value is used to represent the special effect information required for rendering, which also reduces the storage size and improves the transmission speed, while also reducing the transmission traffic cost.

[0063] Furthermore, by using RGB channel values ​​to store dynamic element indexes, coordinate information, and gift animation transparency, etc., it is possible to add a variety of special effect messages, thereby improving the live video effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 This is a schematic diagram of an example live broadcast service application scenario;

[0065] Figure 2 is a flow chart of a method for rendering special effects of virtual gifts in one embodiment;

[0066] Figure 3 This is a sample gift animation frame diagram;

[0067] Figure 4This is a schematic diagram of the layout of the animation content area and the special effect information area;

[0068] Figure 5 is a schematic diagram of the structure of a virtual gift special effect rendering device according to an embodiment;

[0069] Figure 6 This is a schematic diagram of the live broadcast system structure;

[0070] Figure 7 is a block diagram of an example electronic device. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0072] In the embodiments of the present application, the words "first", "second", etc. are used to distinguish between identical or similar items with substantially the same effects and functions. "at least one" means one or more, and "multiple" means two or more. For example, multiple objects refer to two or more objects. "Include" or "comprises" and similar words mean that the information appearing before "include" or "comprises" covers the information listed after "include" or "comprises" and its equivalents, and does not exclude other information. "And / or" mentioned in the embodiments of the present application indicates that three types of relationships may exist. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0073] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0074] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0075] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0076] The technical solution provided in the embodiments of the present application can be applied to Figure 1 In the application scenario of the method related to this application shown in FIG. Figure 1 This is a schematic diagram of an example of a live broadcast service application scenario. The live broadcast system may include a live broadcast server, an anchor terminal and an audience terminal. The anchor terminal and the audience terminal communicate with the live broadcast server through a communication network, so that the anchor at the anchor terminal and the audience users at the audience terminal can perform real-time network live broadcast. Among them, for the anchor terminal and the audience terminal, their terminal devices can be but are not limited to various personal computers, laptops, smart phones and tablets. The live broadcast server can be implemented by an independent server or a server cluster composed of multiple servers.

[0077] In order to achieve better live video streaming effects, the present application provides a high-performance virtual gift special effects rendering method. By simultaneously saving the animation content and its corresponding special effects information in the gift animation frame, it avoids inputting special effects information from the outside during rendering, reduces the rendering complexity, and can achieve smooth playback of exquisite virtual gift special effects, thereby improving the live video streaming effect.

[0078] In the following embodiments, Figure 1 The live broadcast server, anchor terminal and audience terminal of the live broadcast service application scenario shown in the embodiment are described; Figure 2 As shown, Figure 2 The flowchart of a virtual gift special effect rendering method according to an embodiment includes the following steps:

[0079] Step S10, obtaining a gift animation file of the virtual gift; wherein the gift animation file includes gift animation frames that simultaneously store animation content and its corresponding special effect information.

[0080] Specifically, when a viewer presents a virtual gift to the anchor, the viewer terminal may download a gift animation file of the virtual gift from the live broadcast server for rendering on the viewer terminal.

[0081] In one embodiment, when obtaining the gift animation file of the virtual gift in step S10, the gift animation file of the virtual gift and its configuration information can be loaded from the live broadcast server; wherein the configuration information is descriptive information for indicating the configuration status of the gift animation file.

[0082] Exemplarily, the gift animation file chooses video as the carrier mode, and the video formats include mp4, mov, flv, avi and other formats; using video mode, the volume is small, and the resource package occupies less memory space, which is conducive to the storage of resources, and the network transmission process is fast and the bandwidth is low, which can significantly reduce server storage costs and traffic costs, and the cost of live broadcast operation is lower.

[0083] Exemplarily, special effect information may refer to additional information content added to achieve special effects, such as the transparency of animation content, dynamic element coordinate information, etc.; the transparency effect is an indispensable part of virtual gift special effects, and dynamic elements refer to resources that are not stored in the resource package of the gift animation, but are dynamically selected for rendering according to actual needs when rendering the virtual gift; virtual gift special effects can also support dynamic element insertion, such as inserting pictures, text and other elements in a certain frame of animation.

[0084] The gift animation file includes gift animation frames that simultaneously store animation content and its corresponding special effect information.

[0085] In one embodiment, the gift animation frames in the gift animation file of the virtual gift may be synthesized by the following method, specifically including:

[0086] S101, configuring an animation content area and a special effect information area.

[0087] Specifically, the gift animation frame can be divided into two parts, one is the animation content area, and the other is the special effect information area; for example, Figure 3 As shown, Figure 3 This is a schematic diagram of an example gift animation frame. The left part of the gift animation frame is the animation content, and the right part is the special effect information. The special effect information stores information such as the transparency of the animation content.

[0088] In one embodiment, the method of configuring the animation content area and the special effect information area in step S101 may include the following steps:

[0089] Configure an animation content area and a special effect information area with the same resolution; determine the layout of the animation content area and the special effect information area according to the width and height of the domain resolution of the animation content area; if the width of the animation content area is greater than the height, the animation content area and the special effect information area are arranged in a top-to-bottom layout, otherwise they are arranged in a left-to-right layout.

[0090] Specifically, whether to adopt a left-right layout or a top-down layout for the gift animation frame can be determined according to the resolution aspect ratio of the animation content. If the width is greater than the height, a top-down layout is adopted. If the width is less than the height, a left-right layout is adopted. The above layout strategy can reduce the maximum edge resolution of the synthesized gift animation frame, which is more conducive to video decoding on the viewer side.

[0091] like Figure 4 As shown, Figure 4 This is a schematic diagram of the layout of the animation content area and the special effect information area. Assume that the resolution of the animation content of the gift animation is 1000×600, and the special effect information area is also 1000×600. If it is synthesized in a left-right layout, such as Figure 4 As shown in (a), the resolution of the synthesized gift animation frame is 2000×600, and the maximum edge resolution is 2000. If it is synthesized in a top-down layout, such as Figure 4 As shown in (b), the resolution of the synthesized gift animation frame is 1000×1200, and the maximum edge resolution is 1200. For the terminal device decoding process, 1000×1200 is more compatible than 2000×600, avoiding decoding failure when the terminal device's hardware decoding has restrictions on the resolution size (such as a maximum value of 1920).

[0092] S102: Save the animation content of the virtual gift in the animation content area.

[0093] Specifically, the RGB channel value of each pixel point of the animation content of the virtual gift is obtained, where RGB represents Red, Green, and Blue, and the RGB channel value is saved in the animation content area.

[0094] S103: Save the special effect information corresponding to the animation content in a special effect information area.

[0095] Specifically, the special effect information of each pixel point of the animation content is obtained, and then the special effect information is saved in the special effect information area corresponding to each pixel point of the animation content.

[0096] The method for storing the special effect information corresponding to the animation content in the special effect information area may include the following as an embodiment:

[0097] (1) Obtain special effect information of each pixel of the animation content of the virtual gift.

[0098] Specifically, the transparency of the animation content of each pixel point of the animation content, the index of the dynamic element, the coordinate information of the dynamic element and other special effect information are obtained.

[0099] (2) Convert the special effect information into RGB channel values ​​according to the preset encoding rules.

[0100] Specifically, the range of each of the three RGB channels is 0-255, and the binary representation is 0-11111111. Each channel value corresponds to an 8-bit binary number, and the three RGB channels correspond to a 24-bit binary number. Based on this, a coding rule can be preset to convert special effect information into RGB channel values, thereby saving special effect information in the gift animation frame.

[0101] In one embodiment, in order to fully utilize the bits of the three RGB channels to store special effect information, the method of converting the special effect information into RGB channel values ​​in the above embodiment may include the following steps:

[0102] (a) Convert the transparency of the animated content and the index of the dynamic element to R channel values.

[0103] Generally speaking, there are only a few dynamic elements in a virtual gift, which can be represented by a few digits. For example, the maximum number of dynamic elements is 8, which requires 2 to the cube, that is, binary 111, which requires three digits to represent, and the index can be 0-7.

[0104] In one embodiment, the above embodiment of converting the transparency of the animation content and the index of the dynamic element into an R channel value may include:

[0105] The index of the dynamic element is converted into the last n bits of the R channel value; the transparency of the animation content is calculated by a mapping function as a virtual transparency value, and the virtual transparency value is converted into the first m bits of the R channel value; wherein the total number of bits of the R channel value is n+m=8.

[0106] Specifically, the last few bits of the R channel are used to represent the index of the dynamic element, and the value range of n is (1, 3). When the number of dynamic elements does not exceed 2, the last bit of the R channel value can be used to represent it. When the number of dynamic elements is 3 or 4, the last 2 bits of the R channel value can be used to represent it. When the number of dynamic elements is 5 to 8, the last 3 bits of the R channel value can be used to represent it. By way of example, assuming there are 3 dynamic elements, the R channel value corresponding to the index of the first dynamic element is ××××××00, the R channel value corresponding to the index of the second dynamic element is ××××××01, and the R channel value corresponding to the index of the third dynamic element is ××××××10, where × represents a digit, and the index of the dynamic element is pre-configured.

[0107] Among them, since the human eye is not so sensitive to transparency, it does not need to be very precise. Therefore, in the R channel value, except for the last n bits used to represent the index of the dynamic element, the remaining digits are used to represent the transparency of the animation content.

[0108] Since the real transparency ranges from 0 to 255 and requires 8 bits of binary code to represent, at this time, when fewer code bits are used to represent, a virtual transparency value can be calculated through a mapping function and then converted when used.

[0109] For example, when the number of dynamic elements is 3, only the last 2 bits of the R channel value are needed to represent the index of the dynamic element. The first 6 bits can be used to represent the transparency of the animation content, which can range from 0 to 63. At this time, the virtual transparency value is alpha_virtual, and the real transparency is alpha_real, which ranges from 0 to 255. The conversion calculation can be performed through the mapping function, that is, alpha_real=alpha_virtual×4, or alpha_virtual=alpha_real / 4. For example, the real transparency of 165 and 166 are both represented by the virtual transparency value of 42. Although the above mapping conversion process will bring a certain loss of precision, since the human eye is less sensitive to transparency, the above difference will not affect the final rendering effect.

[0110] (b) Convert the coordinate information of the dynamic elements into G channel values ​​and B channel values ​​respectively.

[0111] Specifically, the x and y values ​​of the coordinates can be represented by the G channel value and the B channel value respectively. Since the range that the G and B channel values ​​can represent is 0-255, when representing the coordinate information of the dynamic element, the display area position and size can not exceed the range of 256×256; the dynamic element can be in the form of a picture, and some picture formats are transparent, such as png, and some are non-transparent, such as jpg. You can choose according to specific needs. If the dynamic element also needs to have transparency, use a picture format with transparency, and there is no specific restriction.

[0112] When representing the coordinate information of a dynamic element, the resolution of the width and height of the element size is 256×256. At this time, the G channel value can be used to represent the width of the dynamic element, and the B channel value can be used to represent the height of the dynamic element. For example, if the image size of a dynamic element is 100×100, the coordinate point of its lower left corner is (0, 0), and one of the coordinate points is (36, 24), then the G channel value of the corresponding pixel point in the RGB channel value is 36, and the B channel value is 24.

[0113] (3) The RGB channel values ​​are saved to the pixel points corresponding to the special effect information area.

[0114] Specifically, the special effect information is carried in a video format, and in each frame of the gift animation, the special effect information is saved through RGB channel values.

[0115] For example, in the virtual gift special effects file produced, there are 3 dynamic elements. The transparency of the animation content that needs to be displayed at a certain pixel point is 140, and the second dynamic element needs to be displayed, and its coordinate point is (52, 67), which is converted to the RGB channel value of (141, 52, 67), where the mapping relationship of the R channel value is to convert the real transparency 140 into a virtual transparency value. Since the 3 dynamic elements occupy the last 2 bits of the R channel value, the virtual transparency value is 141÷4=35, corresponding to the binary bit 100011; the second element is 01, so the value of R is 10001101, which is 141 in decimal.

[0116] S104, synthesizing the animation content area and the special effect information area into a gift animation frame.

[0117] Specifically, the animation content area and the special effect information area are combined into a gift animation frame. The special effect information corresponding to each frame of the gift animation is different. Combining these two parts into a gift animation frame can avoid the situation where the two parts have different download speeds due to network problems, and avoid the need to deal with various synchronization problems due to different decoding speeds, thereby achieving a better synchronization effect.

[0118] As in the schemes of the above-mentioned embodiments, the RGB channel value of each pixel is used to represent the transparency of the animation content, the index of the dynamic element, the coordinate information of the dynamic element and other special effect information. Therefore, in the rendering process, when the RGB channel value of the pixel is sampled, the required special effect information can be directly obtained through simple calculation; while conventionally, when rendering each frame of video image, special effect information is input from the outside in real time, it is necessary to increase the information transmission between the CPU (Central Processing Unit) and the GPU (Graphics Processing Unit), it is necessary to judge the coordinates of the dynamic elements, and to use if statements, and in the GPU shader, if statements consume a lot of performance, and the technical solution of the present application can significantly reduce the complexity and improve the rendering performance.

[0119] In one embodiment, after the gift animation file is produced according to the technical solution of the above embodiment, the gift animation file can be uploaded to the live broadcast server and relevant configuration information can be written. The configuration information may include the layout of the animation content area and the special effect information area, the index of dynamic elements, etc. For example, the layout of the animation content area and the special effect information area and the number of bits n occupied by the index of the dynamic element; the audience end can download the gift animation file of the virtual gift and its configuration information from the live broadcast server to the local for special effects rendering.

[0120] Step S20, decoding the gift animation file to obtain multiple gift animation frames; wherein the gift animation frame includes an animation content area and a special effect information area.

[0121] In this step, the audience can decode the gift animation file downloaded from the live broadcast server to obtain multiple gift animation frames.

[0122] In one embodiment, step S20, decoding the gift animation file to obtain a plurality of gift animation frames, includes:

[0123] The gift animation file is decoded to obtain gift animation frames, and the animation content area and special effect information area of ​​each gift animation frame are determined according to the configuration information.

[0124] Specifically, each of the multiple gift animation frames includes an animation content area and a gift special effect information area, and the positions of the animation content area and the special effect information area can be determined according to the configuration information.

[0125] Step S30 , performing rendering calculations on the animation content area and the special effect information area respectively to obtain the animation content and the special effect information.

[0126] For example, during the rendering process, the viewer can use Shader on the GPU to render and calculate animation content and special effect information. Shader is a script used for GPU rendering.

[0127] In one embodiment, the method of performing rendering calculations on the animation content area and the special effect information area to obtain the animation content and the special effect information in step S30 includes:

[0128] S301, performing rendering calculation on the animation content area of ​​the gift animation frame to obtain the animation content texture.

[0129] S302, performing rendering calculation on the special effect area of ​​the gift animation frame to obtain a special effect information texture.

[0130] S303, performing rendering calculation according to the index of the dynamic element and its corresponding dynamic element image to obtain a dynamic element texture array.

[0131] Exemplarily, the shader includes the following information: ① The index of the dynamic element, which can be obtained from the configuration information and then input into the shader; ② The animation content texture, which takes the animation content area in the decoded gift animation frame as input; ③ The special effect information texture, which takes the special effect information area in the decoded gift animation frame as input; ④ The dynamic element texture array, which can be represented by texture[8], saves the corresponding dynamic element image, the array subscript 0, that is, texture[0], is the corresponding first dynamic element, and the array subscript 1, that is, texture[1], is the corresponding second dynamic element.

[0132] Step S40: Rendering the virtual gift according to the special effect information and animation content.

[0133] In this step, the animation content and special effect information are calculated by rendering, and the special effect information and animation content are rendered on the audience side to realize the rendering process of the virtual gift.

[0134] In one embodiment, the method of rendering the virtual gift according to the special effect information and the animation content in step S40 may include the following steps:

[0135] S401, sampling the animation content texture to obtain the animation RGB channel value of each pixel.

[0136] S402, sampling the special effect information texture to obtain the special effect RGB channel value of each pixel.

[0137] S403, obtaining the transparency of the animation content of each pixel and the RGBA value of the dynamic element according to the special effect RGB channel value and the dynamic element texture array.

[0138] As an embodiment, the process of obtaining the transparency of the animation content of each pixel and the RGBA value of the dynamic element may include the following:

[0139] (a) obtaining a virtual transparency value according to the special effect RGB channel value, determining a mapping function according to the number of bits n occupied by the index of the dynamic element, and calculating the transparency of the animation content according to the mapping function and the virtual transparency value.

[0140] For example, assuming that the special effect RGB channel value is (35, 52, 67), where the R channel value is 10001101, the index of the dynamic element occupies n=2 bits, and the virtual transparency value is 35, the real transparency can be obtained by conversion calculation of the mapping function as 35×4=140.

[0141] (b) According to the index of the dynamic element and the coordinate information of the dynamic element, the corresponding coordinate point of the dynamic element texture in the dynamic element texture array is sampled to obtain the RGBA value of the dynamic element, where RGBA is a color space representing Red, Green, Blue and Alpha.

[0142] For example, the R channel value is 10001101, which means that the index of the dynamic element is the first dynamic element. Combined with the corresponding coordinate point (52, 67) of the dynamic element texture, the RGBA value of the dynamic element can be sampled.

[0143] S404: fusion rendering of the animation RGB channel value of each pixel, the transparency of the animation content and the RGBA value of the dynamic element.

[0144] Specifically, the special effects of the virtual gift are finally obtained by merging and rendering the animation RGB channel values ​​of each pixel of the gift animation frame, the transparency of the corresponding animation content, and the RGBA value of the dynamic element.

[0145] As in the schemes of the above-mentioned embodiments, by simultaneously saving the animation content and its corresponding special effect information in the gift animation frame, the special effect information is avoided from being input from the outside during rendering, thereby reducing the rendering complexity, enhancing the rendering performance, and improving the video live broadcast effect.

[0146] An embodiment of a virtual gift special effects rendering device is described below.

[0147] like Figure 5 As shown, Figure 5 The diagram is a schematic diagram of a virtual gift special effect rendering device according to an embodiment, including:

[0148] The acquisition module 10 is used to acquire the gift animation file of the virtual gift; wherein the gift animation file includes gift animation frames that simultaneously store the animation content and its corresponding special effect information.

[0149] The decoding module 20 is used to decode the gift animation file to obtain multiple gift animation frames; wherein the gift animation frame includes an animation content area and a special effect information area.

[0150] The calculation module 30 is used to perform rendering calculations on the animation content area and the special effect information area respectively to obtain the animation content and the special effect information.

[0151] The rendering module 40 is used to render the virtual gift according to the special effect information and the animation content.

[0152] The gift animation file of the virtual gift used by the acquisition module 10 can be prepared in advance and saved in the live broadcast server.

[0153] The gift animation frame can be divided into two parts, one is the animation content area, and the other is the special effect information area; the animation content area and the special effect information area are configured with the same resolution. For the gift animation frame, whether to use a left-right layout or a top-down layout can be determined according to the resolution aspect ratio of the animation content. If the width is greater than the height, a top-down layout is adopted; if the width is less than the height, a left-right layout is adopted.

[0154] Get the RGB channel value of each pixel of the animation content of the virtual gift, get the special effect information of each pixel of the animation content, and then save it in the special effect information area corresponding to each pixel of the animation content. You can get the special effect information such as the transparency of the animation content of each pixel of the animation content, the index of the dynamic element, and the coordinate information of the dynamic element. The range of each channel of the three RGB channels is 0 to 255, and the binary representation is 0 to 11111111. Each channel value corresponds to an 8-bit binary, and the three RGB channels are a 24-bit binary; based on this, you can preset the encoding rules to convert the special effect information into RGB channel values, so as to save the special effect information in the gift animation frame.

[0155] The special effect information is carried in video format. In each frame of the gift animation, the special effect information is saved through the RGB channel value. The animation content area and the special effect information area are combined into a gift animation frame. The special effect information corresponding to each frame of the gift animation is different. Combining these two parts into a gift animation frame can avoid the situation where the download speed of the two parts is different due to network problems, and avoid the need to deal with various synchronization problems due to different decoding speeds, thereby achieving a better synchronization effect.

[0156] For the decoding module 20, the audience terminal can decode the gift animation file downloaded from the live broadcast server to obtain multiple gift animation frames. Specifically, each of the multiple gift animation frames includes an animation content area and a gift special effect information area, and the positions of the animation content area and the special effect information area can be determined according to the configuration information.

[0157] For the calculation module 30, during the rendering process, the audience end can use Shader on the GPU to render and calculate the animation content and special effect information. Shader is a script used for GPU rendering. The method for rendering and calculating the animation content area and the special effect information area to obtain the animation content and special effect information can be as follows: the animation content area of ​​the gift animation frame is rendered and calculated to obtain the animation content texture; the special effect area of ​​the gift animation frame is rendered and calculated to obtain the special effect information texture; the dynamic element texture array is rendered and calculated according to the index of the dynamic element and its corresponding dynamic element picture.

[0158] For the rendering module 40, the rendering can be used to calculate the animation content and special effect information, and the special effect information and animation content can be rendered on the audience side to realize the rendering process of the virtual gift; the rendering method of the virtual gift can include the following: sampling the animation content texture to obtain the animation RGB channel value of each pixel; sampling the special effect information texture to obtain the special effect RGB channel value of each pixel; obtaining the virtual transparency value according to the special effect RGB channel value, determining the mapping function according to the number of bits n occupied by the index of the dynamic element, calculating the transparency of the animation content according to the mapping function and the virtual transparency value, sampling the corresponding coordinate points of the dynamic element texture in the dynamic element texture array according to the index of the dynamic element and the coordinate information of the dynamic element to obtain the RGBA value of the dynamic element; by fusing and rendering the animation RGB channel values ​​of each pixel of the gift animation frame, the corresponding transparency of the animation content and the RGBA value of the dynamic element, finally obtaining the special effect of the virtual gift.

[0159] The virtual gift special effects rendering device of this embodiment can execute a virtual gift special effects rendering method provided by the embodiment of the present application. The implementation principle is similar. The actions performed by each module in the virtual gift special effects rendering device in each embodiment of the present application correspond to the steps in the virtual gift special effects rendering method in each embodiment of the present application. For the detailed functional description of each module of the virtual gift special effects rendering device, please refer to the description in the corresponding virtual gift special effects rendering method shown in the previous text, which will not be repeated here.

[0160] An embodiment of the live broadcast system is described below.

[0161] The live broadcast system of this embodiment refers to Figure 6 As shown, Figure 6This is a schematic diagram of the structure of an example live broadcast system. The system framework may include an anchor terminal, a live broadcast server and an audience terminal. The audience terminal and the anchor terminal may access the live broadcast platform through a client installed on an electronic device. For example, the anchor terminal and the audience terminal may be computer devices such as a PDA, a smart phone, a tablet computer, a desktop computer or a laptop computer, etc., but this is not limited. They may also be software modules of an application. The live broadcast server may be implemented using an independent server or a server cluster consisting of multiple servers.

[0162] As in the above-mentioned live broadcast system, the anchor end and the audience end are respectively connected to the live broadcast server through the communication network; the anchor end is used to access the anchor in the live broadcast room and collect the anchor's live video stream and upload it to the live broadcast server; the live broadcast server is used to forward the anchor's live video stream to the audience end, and store the gift animation file of the virtual gift; the audience end is used for the audience users who access the live broadcast room, receive the anchor's live video stream for playback, and use the virtual gift special effects rendering method of any of the aforementioned embodiments to play the gift animation file of the virtual gift.

[0163] For example, in a live broadcast scenario, a gift animation file of a virtual gift can be first produced and saved to a live broadcast server. When a viewer on the audience side presents a virtual gift, the audience side downloads the corresponding gift animation file from the live broadcast server, decodes it to obtain the animation content and special effects information, and finally completes the rendering process of the virtual gift locally on the audience side.

[0164] In the live broadcast system of this embodiment, the live broadcast server saves the animation content and its corresponding special effect information in the gift animation frame of the gift animation file at the same time. When rendering the virtual gift special effect, the audience downloads the gift animation file of the virtual gift, decodes the gift animation file to obtain multiple gift animation frames, renders and calculates the animation content area and the special effect information area of ​​the gift animation frame respectively to obtain the animation content and special effect information, and renders the virtual gift according to the special effect information and animation content. By saving the animation content and its corresponding special effect information in the gift animation frame at the same time, the special effect information is avoided from being input from the outside during rendering, the rendering complexity is reduced, the rendering performance is enhanced, and the video live broadcast effect is improved. Further, by using the video format to make the gift animation file of the virtual gift, using the RGB channel value to represent the special effect information required for rendering, the storage size is reduced and the transmission speed is increased, and the transmission traffic cost can also be reduced. Further, by using the RGB channel value to store dynamic element indexes, coordinate information, and gift animation transparency, etc., it is possible to add a variety of special effect messages, thereby improving the video live broadcast effect.

[0165] Embodiments of electronic devices and computer-readable storage media are described below.

[0166] The present application provides a technical solution for an electronic device to implement functions related to a virtual gift special effect rendering method. The electronic device includes: one or more processors; a memory; one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured for use in the virtual gift special effect rendering method of any embodiment.

[0167] The electronic device may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The electronic device may also be a mobile or stationary server.

[0168] like Figure 7 As shown, Figure 7It is a block diagram of an example electronic device, which includes a processor, a memory, a communication interface, a display screen and an input device connected by a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner, and the communication interface communicates with a database that stores data via one or more networks. Examples of these networks include a public switched telephone network (PSTN, Public Switched Telephone Network), a local area network (LAN, Local Area Network), a wide area network (WAN, Wide Area Network), a personal area network (PAN, Personal Area Network) or a combination of communication networks such as the Internet. The access device may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE802.11 wireless local area network (WLAN) wireless interface, a Wi-MAX (World Interoperability for Microwave Access) interface, an Ethernet interface, a USB (Universal Serial Bus) interface, a cellular network interface, a Bluetooth interface, and a near field communication (NFC). When the computer program is executed by a processor, a method for obtaining a steady-state value of a delay estimation is implemented. The display screen of the electronic device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device may be a touch layer covered on the display screen, or a key, a trackball or a touch pad provided on the housing of the electronic device, or an external keyboard, touch pad or mouse, etc.

[0169] Those skilled in the art will understand that Figure 7 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0170] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0171] The present application provides a technical solution of a computer-readable storage medium to implement the related functions of a virtual gift special effect rendering method and a video encoding method. The computer-readable storage medium stores at least one instruction, at least one program, a code set or an instruction set, and at least one instruction, at least one program, a code set or an instruction set is loaded by a processor and executes the virtual gift special effect rendering method of any embodiment. In an exemplary embodiment, the computer-readable storage medium may be a non-temporary computer-readable storage medium including instructions, such as a memory including instructions. For example, the non-temporary computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a tape, a floppy disk, and an optical data storage device, etc.

[0172] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0173] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0174] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A virtual gift special effects rendering method, characterized in that: include: Acquire a gift animation file of a virtual gift; wherein the gift animation file includes a gift animation frame that simultaneously stores animation content and corresponding special effect information; Decoding the gift animation file to obtain a plurality of gift animation frames; wherein the gift animation frame includes an animation content area and a special effect information area; Rendering and calculating the animation content area and the special effect information area respectively to obtain the animation content and the special effect information; The virtual gift is rendered according to the special effect information and the animation content.

2. The virtual gift special effects rendering method according to claim 1, characterized in that: Also includes: Configure the animation content area and special effect information area; The animation content of the virtual gift is saved in the animation content area; The special effect information corresponding to the animation content is stored in the special effect information area; The animation content area and the special effect information area are synthesized into a gift animation frame.

3. The virtual gift special effects rendering method according to claim 2, characterized in that: Configure the animation content area and special effect information area, including: Configure the animation content area and special effect information area with the same resolution and size; The layout of the animation content area and the special effect information area is determined according to the width and height of the domain resolution of the animation content area; if the width of the animation content area is greater than the height, the animation content area and the special effect information area are arranged up and down, otherwise they are arranged left and right.

4. The virtual gift special effects rendering method according to claim 3, characterized in that: The special effect information corresponding to the animation content is stored in the special effect information area, including: Obtain special effect information of each pixel of the animation content of the virtual gift; Convert special effect information into RGB channel values ​​according to preset encoding rules; The RGB channel values ​​are saved to the pixel points corresponding to the special effect information area.

5. The virtual gift special effects rendering method according to claim 4, characterized in that: The special effect information includes: transparency of animation content, index of dynamic elements and coordinate information of dynamic elements.

6. The virtual gift special effects rendering method according to claim 5, characterized in that: The special effect information is converted into RGB channel values ​​according to the preset encoding rules, including: Convert the transparency of the animation content and the index of the dynamic element into R channel values, and convert the coordinate information of the dynamic element into G channel values ​​and B channel values ​​respectively.

7. The virtual gift special effects rendering method according to claim 6, characterized in that: Convert the transparency of animated content and the index of dynamic elements to R channel values, including: Convert the index of the dynamic element to the last n bits of the R channel value; The transparency of the animation content is calculated by a mapping function as a virtual transparency value, and the virtual transparency value is converted into the first m bits of the R channel value; Among them, the total number of bits of the R channel value is n+m.

8. The virtual gift special effects rendering method according to claim 7, characterized in that: Get the gift animation file of the virtual gift, including: Load the gift animation file and configuration information of the virtual gift from the live broadcast server; The decoding of the gift animation file to obtain a plurality of gift animation frames includes: The gift animation file is decoded to obtain gift animation frames, and the animation content area and special effect information area of ​​each gift animation frame are determined according to the configuration information.

9. The virtual gift special effects rendering method according to claim 8, characterized in that: The configuration information includes: the layout mode of the animation content area and the special effect information area and the number of digits n occupied by the index of the dynamic element.

10. The virtual gift special effects rendering method according to claim 9, characterized in that: Rendering and calculating the animation content area and the special effect information area respectively to obtain the animation content and the special effect information includes: Rendering and calculating the animation content area of ​​the gift animation frame to obtain the animation content texture; Rendering and calculating the special effect area of ​​the gift animation frame to obtain a special effect information texture; The dynamic element texture array is obtained by rendering calculation according to the index of the dynamic element and its corresponding dynamic element image.

11. The virtual gift special effects rendering method according to claim 10, characterized in that: Rendering the virtual gift according to the special effect information and the animation content includes: Sample the animation content texture to obtain the animation RGB channel value of each pixel; Sample the special effect information texture to obtain the special effect RGB channel value of each pixel; Obtaining the transparency of the animation content of each pixel and the RGBA value of the dynamic element according to the special effect RGB channel value and the dynamic element texture array; The animation RGB channel value of each pixel, the transparency of the animation content and the RGBA value of the dynamic element are fused and rendered.

12. The virtual gift special effects rendering method according to claim 11, characterized in that: Obtaining the transparency of the animation content of each pixel and the RGBA value of the dynamic element according to the special effect RGB channel value and the dynamic element texture array, including: Obtain a virtual transparency value according to the special effect RGB channel value, determine a mapping function according to the number of bits n occupied by the index of the dynamic element, and calculate the transparency of the animation content according to the mapping function and the virtual transparency value; According to the index of the dynamic element and the coordinate information of the dynamic element, the corresponding coordinate point of the dynamic element texture in the dynamic element texture array is sampled to obtain the RGBA value of the dynamic element.

13. A virtual gift special effects rendering device, characterized in that: include: An acquisition module, used to acquire a gift animation file of a virtual gift; wherein the gift animation file includes a gift animation frame that simultaneously stores animation content and corresponding special effect information; A decoding module, used for decoding the gift animation file to obtain a plurality of gift animation frames; wherein the gift animation frame includes an animation content area and a special effect information area; A calculation module, used for performing rendering calculations on the animation content area and the special effect information area respectively to obtain animation content and special effect information; A rendering module is used to render the virtual gift according to the special effect information and animation content.

14. A live broadcast system, characterized in that: include: An anchor terminal, an audience terminal and a live broadcast server; wherein the anchor terminal and the audience terminal are respectively connected to the live broadcast server via a communication network; The anchor terminal is used to access the anchor in the live broadcast room and collect the anchor's live video stream and upload it to the live broadcast server; The live broadcast server is used to forward the live broadcast video stream of the anchor to the audience end, and store the gift animation file of the virtual gift; The audience terminal is used for audience users accessing the live broadcast room, receiving the live video stream of the anchor for playback, and using the virtual gift special effect rendering method described in any one of claims 1-12 to play the gift animation file of the virtual gift.

15. An electronic device, characterized in that: The electronic device comprises: one or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs are configured to execute the steps of the virtual gift special effect rendering method described in any one of claims 1-12.

16. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded by the processor and execute the steps of the virtual gift special effects rendering method described in any one of claims 1-12.

Images