Rendering and processing methods, terminals, servers, and storage media for motion graphics videos

Abstract

This application discloses a rendering method, terminal, server, and storage medium for motion-effect videos. The method includes: obtaining blending elements required for rendering the motion-effect video; obtaining insertion elements corresponding to the blending elements and associated with the live stream screen; blending and rendering the blending elements with the corresponding insertion elements to obtain a blended image; and rendering the blended image at the corresponding position of the current source video frame according to the first position data corresponding to the current source video frame of the motion-effect video, until all source video frames of the motion-effect video are rendered. Through the above method, this application can improve the entertainment and interactivity during live streaming.

Description

Technical Field

[0001] This application relates to the field of live streaming technology, and in particular to rendering methods, terminals, servers and storage media for motion-effect videos. Background Technology

[0002] With the development of internet and communication technologies, society has entered an era of intelligent interconnection. Interacting, entertaining, and working on the internet is becoming increasingly common. Among these, live streaming technology is particularly prevalent. People can watch or participate in live streams anytime, anywhere through smart devices, greatly enriching their lives and broadening their horizons.

[0003] During a live stream, users can send virtual gifts to the streamer, thereby increasing the interaction between the audience and the streamer. However, the virtual gifts sent by each user have the same effect, which fails to enhance the entertainment and interactivity of the live stream. Summary of the Invention

[0004] The main technical problem addressed in this application is to provide a rendering and processing method, terminal, server, and storage medium for motion-effect videos, which can improve the entertainment and interactivity during live streaming.

[0005] To solve the above-mentioned technical problems, one technical solution adopted in this application is: to provide a rendering method for a virtual gift animation video, the method comprising: obtaining a blending element required for rendering the animation video; obtaining an insertion element corresponding to the blending element and associated with the live broadcast screen of the live broadcast room; blending the blending element and the corresponding insertion element to obtain a blended image; and rendering the blended image at the corresponding position of the current source video frame according to the first position data corresponding to the current source video frame of the animation video, until all source video frames of the animation video are rendered.

[0006] To solve the above-mentioned technical problems, another technical solution adopted in this application is: providing a rendering method for a virtual gift animation video, the method comprising: receiving a gift-giving request sent by a client terminal that sends a virtual gift; responding to the gift-giving request, sending a gift-giving information notification to all client terminals in the live broadcast room, so that all client terminals in the live broadcast room can use the gift-giving information notification to obtain the blending elements required for rendering the animation video, obtain the insertion elements corresponding to the blending elements and associated with the live broadcast screen of the live broadcast room, and blend the blending elements with the corresponding insertion elements to obtain a blended image, and then, when rendering the animation video, rendering the blended image at the corresponding position of the current source video frame according to the first position data corresponding to the current source video frame of the animation video, until all source video frames of the animation video are rendered.

[0007] To solve the above-mentioned technical problems, another technical solution adopted in this application is: to provide an electronic terminal, which includes a processor, a memory and a communication circuit; the memory and the communication circuit are coupled to the processor, the memory stores a computer program, and the processor is able to execute the computer program to implement the rendering processing method of the virtual gift animation video provided in this application as described above.

[0008] To solve the above-mentioned technical problems, another technical solution adopted in this application is: to provide a server, which includes a processor, a memory and a communication circuit; the memory and the communication circuit are coupled to the processor, the memory stores a computer program, and the processor is able to execute the computer program to implement the rendering processing method of the virtual gift animation video provided in this application as described above.

[0009] To solve the above-mentioned technical problems, another technical solution adopted in this application is to provide a computer-readable storage medium that stores a computer program that can be executed by a processor to implement the rendering processing method for the motion effect video of virtual gifts provided in this application as described above.

[0010] The beneficial effects of this application are as follows: Unlike existing technologies, this method obtains the blending elements required for rendering animated videos, acquires the insertion elements corresponding to the blending elements and associated with the live stream screen of the live room, and performs blending rendering on the insertion elements corresponding to the blending elements to obtain a blended image. Then, when rendering the animated video, the blended image is rendered at the corresponding position of the current source video frame according to the first position data corresponding to the current source video frame of the animated video, until all source video frames of the animated video are rendered. This can effectively and efficiently realize the dynamic insertion of insertion elements associated with the live stream screen of the live room during the rendering of source video frames, enhance the display effect of animated videos when users send virtual gifts, and ensure that the animated video effects of virtual gifts presented in different live stream screens of different live rooms are different, realizing personalized video gift effects, which is conducive to increasing the entertainment and interactivity of the live stream and improving user stickiness. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of the system composition of the live streaming system used in the first embodiment of the rendering and processing method for the animation video of virtual gifts in this application;

[0012] Figure 2 This is a flowchart illustrating the first embodiment of the rendering process method for the motion effect video of virtual gifts in this application;

[0013] Figure 3 This is a schematic diagram of the source images for the first embodiment of the rendering processing method for the motion effect video of the virtual gift in this application;

[0014] Figure 4 This is a schematic diagram of the output video frames of the first embodiment of the rendering processing method for the motion effect video of the virtual gift in this application;

[0015] Figure 5 This is a timing diagram of the first embodiment of the rendering processing method for the motion effect video of virtual gifts in this application;

[0016] Figure 6 This is a schematic diagram of the current live stream screen and the processed image in the first embodiment of the rendering and processing method for the animation video of the virtual gift in this application;

[0017] Figure 7 This is a schematic diagram of the first embodiment of the rendering processing method for the animation video of the virtual gift in this application, which shows how a mixed texture image is obtained by rendering a first texture image and a second texture image together.

[0018] Figure 8 This is a flowchart illustrating the second embodiment of the rendering process method for the motion effect video of virtual gifts in this application;

[0019] Figure 9 This is a schematic block diagram of the circuit structure of the electronic terminal embodiment of this application;

[0020] Figure 10 This is a schematic block diagram of the circuit structure of the server embodiment of this application;

[0021] Figure 11 This is a schematic block diagram of the circuit structure of the computer-readable storage medium of this application. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0023] During live streaming, users can express their support for the streamer by sending virtual gifts. As live streaming interaction continues to develop, the effects of virtual gifts are becoming increasingly diverse, including not only static gift images but also virtual gifts with animated videos. When a user sends a virtual gift with an animated video to a streamer, the corresponding animated video will be played on the live stream screen, making the live stream more visually appealing. This not only encourages users to send gifts to the streamer, thereby increasing the streamer's popularity, but also enhances the viewing experience and interactivity of the live stream.

[0024] After long-term research, the inventors discovered that during live streaming interactions, when users send virtual gifts with animated videos to the streamer, the effect displayed on the live stream screen is always the same. In other words, regardless of which user sends the gift in which streamer's live stream, the animated video displayed on the live stream screen is identical. It does not perform any personalized processing based on the information of the gift-giver and the streamer or the current live stream screen. To a certain extent, this reduces users' interest in sending gifts and fails to further enhance the entertainment and interactivity of the live stream.

[0025] like Figure 1 As shown in the embodiment of the virtual gift animation video rendering method of this application, the virtual gift animation video rendering method can be applied to a live streaming system 1. Specifically, the live streaming system 1 may include a server 10, a broadcaster terminal 20, a viewer terminal 30, and a configuration terminal 40. The broadcaster terminal 20, viewer terminal 30, and configuration terminal 40 can be electronic terminals. Specifically, the broadcaster terminal 20 and viewer terminal 30 are electronic terminals with corresponding client terminal programs installed, i.e., client terminals. The electronic terminal can be a mobile terminal, a computer, a server, or other terminals, etc. The mobile terminal can be a mobile phone, a laptop, a tablet computer, a smart wearable device, etc., and the computer can be a desktop computer, etc.

[0026] Server 10 can pull live data streams from broadcaster terminal 20, process the acquired live data streams accordingly, and then push them to viewer terminal 30. Viewer terminal 30 can then watch the live stream of the broadcaster or guest. Live data stream mixing can occur at least among the live server 10, broadcaster terminal 20, and viewer terminal 30. Video and audio co-streams are possible between broadcaster terminals 20 and between broadcaster terminal 20 and viewer terminal 30. In video co-streams, the co-hosts can push live data streams, including video streams, to the live server 10, which then pushes the corresponding live data to the co-hosts and viewer terminal 30. Broadcaster terminal 20 and viewer terminal 30 can then display the corresponding live stream in the live stream room.

[0027] Server 10 can include various services, such as live streaming service, streaming media service and gift service. Live streaming service is often used for broadcasting by anchors and multi-person live streaming. Streaming media service is often used for processing video stream distribution and mixing. Gift service is specifically used for processing gift-giving related business. These services can be deployed independently on server 10 to complete their respective tasks, or they can be deployed on different interconnected servers 10 to form a server cluster.

[0028] Of course, the terms "anchor terminal 20" and "viewer terminal 30" are relative. The terminal currently engaged in live streaming is the anchor terminal 20, while the terminal watching the live stream is the viewer terminal 30. The configuration terminal 40 is used to configure client terminals and servers, such as configuring the interface layout, gift effects, animations, and other functions displayed on the client terminal. The configuration terminal 40 can send completed projects to the server 10, which then sends them to the client terminals for backend configuration.

[0029] like Figure 2 As shown, the first embodiment of the rendering method for the virtual gift animation video of this application can use a client terminal as the execution subject. This embodiment may include: S100: Obtaining the blending elements required for rendering the animation video. S200: Obtaining the insertion elements corresponding to the blending elements and associated with the live broadcast screen of the live broadcast room. S300: Blending and rendering the blending elements with the corresponding insertion elements to obtain a blended image. S400: When rendering the animation video, rendering the blended image at the corresponding position of the current source video frame according to the first position data corresponding to the current source video frame of the animation video, until all source video frames of the animation video are rendered.

[0030] In the rendering method of the virtual gift animation video of this application, the blending elements required for rendering the animation video are obtained, and the insertion elements corresponding to the blending elements and associated with the live broadcast screen of the live broadcast room are obtained. The insertion elements corresponding to the blending elements are blended and rendered to obtain a blended image. Then, when rendering the animation video, the blended image is rendered at the corresponding position of the current source video frame according to the first position data corresponding to the current source video frame of the animation video, until all source video frames of the animation video are rendered. This can effectively and efficiently realize the dynamic insertion of the insertion elements associated with the live broadcast screen of the live broadcast room when the source video frames are rendered, which enhances the display effect of the animation video when the user gives virtual gifts. It can realize that the animation video effect of virtual gifts presented in the live broadcast screen of different live broadcast rooms is different, realize personalized video gift effects, which is conducive to increasing the entertainment and interactivity of the live broadcast process and improving user stickiness.

[0031] The method described in this embodiment can be applied to scenarios where a client terminal processes a motion effect resource file corresponding to a virtual gift after receiving it via server 10 and configuring it via configuration terminal 40.

[0032] Before sending the motion effect resource file to the client terminal via server 10, it is necessary to generate the motion effect resource file through configuration terminal 40, which may include the following steps:

[0033] S010: Obtain the first position data of the blending element in each frame of the source image, so as to serve as the position data of the blending element in the source video frame corresponding to the source video and the source image.

[0034] After designing material resources using AE software, the terminal 40 can read the material resources through the corresponding software. The material resources can be used to describe the insertion of corresponding elements into the source video, including the identification information, position and shape of the insertion elements, so that the client terminal can know what elements to extract and what shape to render them in at what position on the source video frame.

[0035] After reading the source materials, they are analyzed. By analyzing each image in the source materials, the position of each blending element in that image can be calculated, obtaining the first position data of each blending element. This allows us to obtain the first position data of each blending element in all source images for the project. The first position data of the blending elements in the corresponding source images is used to reflect / map the position data of the corresponding inserted elements in the source video frames, enabling the client terminal to know where the extracted inserted elements should be rendered in the source video frames.

[0036] To obtain the first position data of the blended elements in each source image, a first transformation matrix can be calculated to map the relative changes of the blended elements between the first frame and subsequent frames. For example, given the frame rate (fps) and duration of the entire animation project rendered in the source video, the process can be divided into fps*duration frames, meaning the source images can have fps*duration frames. The positions of the blended elements on these frames need to be obtained. Specifically, by analyzing these frames, a corresponding first transformation matrix can be generated. For example, a transformation matrix can be created for each frame to describe the position changes of the blended elements in different frames.

[0037] The first transformation matrix of each subsequent frame's source image maps the relative changes of the blending elements of that subsequent frame's source image relative to the same blending elements of the first frame's source image. For example, if the total number of source images is 10 frames, then a first transformation matrix is ​​created for frames 2-10 respectively. The first transformation matrix of the source image in different frames describes the relative changes of the blending elements of that source image relative to the blending elements of the first frame's source image.

[0038] For the specific process of creating the first transformation matrix, please refer to the following steps included in step S010:

[0039] S011: Obtain the change data of the blending elements of each subsequent frame material image relative to the blending elements of the first frame material image in the translation, rotation, and scaling dimensions.

[0040] Subsequent frame footage images are those that are relative to the first frame footage image within the same project; that is, footage images that follow the first frame footage image are called subsequent frame footage images.

[0041] Transformation data can refer to the translation, rotation, and scaling of blended elements across different frames of source images. This transformation data can be determined or input during the design of the source material, specifying the translation, rotation, and scaling dimensions of the blended elements in each frame, or it can be derived from the analysis and calculation of all frames of the designed source material.

[0042] For example, at 0s, the anchor point of blend element 1 is (0,0), its position is (225,730), its rotation angle is 0, and its scale is 60%. At 2s, the anchor point of blend element 1 is (0,0), its position is (352,730), its rotation angle is 0, and its scale is 60%.

[0043] The anchor point can be selected as the origin of blend element 1, which is the top-left corner of the layer range of blend element 1. In this case, the anchor point is (0,0). Positions of (225,730) and (352,730) represent the anchor point's position relative to the origin of the source image, i.e., relative to the top-left corner of the source image. Position reflects the translation relationship of the blend element, rotation angle reflects the rotation relationship, and scaling ratio reflects the scaling relationship. This allows us to calculate the changes in translation, rotation, and scaling between 2s and 0s, thus obtaining the change data of 2s relative to 0s.

[0044] In summary, by calculating the changes in subsequent frame images relative to the first frame image in three dimensions, the corresponding change data can be obtained, which is then used to construct the corresponding first transformation matrix.

[0045] S012: Calculate the first transformation matrix corresponding to each subsequent frame image using the change data corresponding to each subsequent frame image.

[0046] After obtaining the change data of each subsequent frame image, the corresponding first transformation matrix can be calculated using the change data of each subsequent frame image. After calculating the first transformation matrix corresponding to each subsequent frame image, the first position data corresponding to each subsequent frame image can be calculated using the first position data of the first frame image, as detailed in S022 below.

[0047] S013: Obtain the first position data of the blending element in the first frame of the material image, and transform the first position data of the blending element in the first frame of the material image by the first transformation matrix corresponding to each subsequent frame of the material image to obtain the first position data of the blending element in each subsequent frame of the material image.

[0048] The first position data of the blending elements in the first frame of the source image is obtained, which is also the starting position data for the animation process of the blending elements. The first position data of the first frame of the source image is then transformed using the first transformation matrix of each subsequent frame of the source image to obtain the first position data of the same blending element in each subsequent frame of the source image. In this way, the first position data of the blending elements in each subsequent frame of the source image can be quickly calculated using the first transformation matrix; that is, the position of the source image is converted into position data that the computer can read and process.

[0049] Specifically, the process of calculating the first position data corresponding to each subsequent frame image using the first position data corresponding to the first frame image can be found in the following steps included in S013:

[0050] S0131: Determine the first coordinate data corresponding to the vertex coordinates of the blending element in the first frame of the source image, so as to obtain the first position data of the blending element in the first frame of the source image.

[0051] For example, based on the width and height of the blending elements in the first frame of the source image, and their positions (e.g., anchor point positions), the extent of the layer can be determined: top-left vertex, bottom-left vertex, top-right vertex, and bottom-right vertex. The coordinates of each vertex can then be calculated, yielding the first coordinate data. The first position data can be represented by the first coordinate data. Figure 3 Taking the blending element 1 shown in Figure A as an example, the coordinates of the four vertices of the blending element 1 in the first frame of the image can be determined using the above method.

[0052] S0132: Multiply the first coordinate data by the first transformation matrix corresponding to each subsequent frame material image to obtain the transformation coordinate data of the blending element in each subsequent frame material image.

[0053] Combining the first transformation matrix calculated in the above steps, the first coordinate data is multiplied by the first transformation matrix corresponding to the subsequent frame's material image to calculate the transformed coordinate data of that subsequent frame's material image. Specifically, the four vertices calculated above are multiplied by the first transformation matrix to obtain the new four vertex positions of the blending element of the corresponding frame's material image. However, the positions of these four vertices are not necessarily the vertex positions of the blending element's layer range in that frame's material image.

[0054] For example, Figure 3As shown in Figure B, blending element 1 in a subsequent frame has been moved and rotated relative to blending element 1 in the first frame. The square-shaped blending element 1 in the first frame appears as a rhombus-like shape in the subsequent frame, but their layer extents differ. In the first frame, the layer extent of blending element 1 is defined by the four vertices of the square, while in the subsequent frame, it is defined by the dashed box shown in the figure. Therefore, the layer extent of the blending element in that frame needs to be determined based on the new positions of the four vertices.

[0055] S0133: Use the transformed coordinate data of the blending element to determine the second coordinate data corresponding to the vertex coordinates of each vertex of the blending element in each subsequent frame image, so as to obtain the first position data of the blending element in each subsequent frame image.

[0056] Specifically, the maximum and minimum values ​​of the blending elements in the frame's source image can be determined from the transformed coordinate data obtained in the above steps, both on the horizontal and vertical axes. The layer extent of the blending elements is then determined using these maximum and minimum values.

[0057] For example, in the new vertex positions corresponding to the transformed coordinate data, the maximum and minimum values ​​of the x and y axes are determined, resulting in four values: (minX, minY, maxX, maxY). The new four vertex positions of the layer range of the blending element in the subsequent frame's image are then determined: top left: (minX, minY); bottom left: (minX, maxY); top right: (maxX, minY); bottom right: (maxX, maxY), which are the second coordinate data. The first position data of the blending element in the subsequent frame's image can be represented by the second coordinate data.

[0058] After obtaining the first position data of the blending elements in each frame of the source image, the first position data is saved and recorded as the display position of the blending element in the corresponding source video frame of that frame. This process is repeated to calculate the four vertex positions of all blending elements in each frame of the source image and store them in a dictionary.

[0059] S020: Send the source video data and first location data to the server, which then forwards them to the client terminal.

[0060] After obtaining the first position data, the source video data and the first position data can be output. Server 10 can forward the received data to the client terminal for rendering. After receiving the source video data and the first position data from server 10, the client terminal obtains the insertion elements corresponding to the blending elements and renders the insertion elements into the corresponding source video frames according to the corresponding first position data when rendering the source video in the live broadcast. Before outputting the source video data and the first position data, the output video frame and the second position data can be obtained. Specifically, the output canvas corresponding to the output video frame can be divided into a first area and a second area with intervals. Optionally, the number of output video frames is the same as the number of source video frames, that is, the output video frames and source video frames correspond one-to-one. The first area of ​​the output video frame displays the corresponding source video frame, and the second area of ​​the output video frame displays the blending elements to be inserted into the source video frame. The function of the second area is actually to store the blending elements corresponding to the source video frame so that the client terminal can specifically identify each blending element.

[0061] When determining the output video frames, such as Figure 4 As shown, the source video frame can be displayed in the first area of ​​the output canvas. Each blending element from the source video frame's corresponding source image is sequentially copied to the second area of ​​the output canvas to form the output video frame. The second position data of each blending element in the second area of ​​the output canvas is recorded. Specifically, the output video frame and the second position data of the inserted layer in the second area can be obtained through corresponding plugins / extensions in After Effects software. The second position data can be expressed using the coordinate data of the inserted layer in the output video frame. Description information is generated by combining the first position data of the blending elements in the source video frame and the second position data in the output video frame. Specifically, the data in the description information can be saved as a JSON array, which can contain one or more blending elements. Each blending element has width, height, index information, type, etc. The description information can also include the first and second position data saved as a JSON array. This JSON array can be named "datas" for faster retrieval of the first and second position data. Finally, all the output video frames are converted sequentially into the output video to obtain the video data of the output video, i.e., the motion effect resource file. Specifically, the motion effect resource file can be an MP4 resource file.

[0062] The rendering method described in this embodiment can be applied to scenarios such as live streaming. Specifically, it can be used for scenarios where users send virtual gifts to the streamer who is currently live streaming.

[0063] The following is a detailed description of this embodiment, which may include the following steps:

[0064] S100: Get the blending elements needed to render the motion graphics video.

[0065] The animated video can be a video corresponding to a virtual gift. When a user watching the live stream sends a virtual gift to the streamer through the viewer terminal 30, both the viewer terminal 30 and the streamer terminal 20 will play the animated video corresponding to the virtual gift.

[0066] Blend elements are elements used to insert interpolated elements into animated video during the rendering process. Interpolated elements are elements that correspond to blend elements and are associated with the live stream screen.

[0067] During the rendering process of animated videos, since there is a one-to-one correspondence between blended elements and inserted elements, the inserted element can be inserted into the animated video through the blended element corresponding to the inserted element. This allows the animated video to present different display effects based on different live broadcast screens during playback, thereby achieving personalized video gift effects. This helps to increase the entertainment and interactivity of the live broadcast and improve user engagement.

[0068] In one implementation, the following steps may be included before S100:

[0069] S110: Receive gift-giving notifications from the server.

[0070] Specifically, gift-giving notifications can be received through a gift service. These notifications are generated by the gift service after receiving a gift request from the giver. For example, Figure 5 As shown, when a user in the live stream sends a gift to the streamer, they can send a gift request to the gift service. After receiving the gift request and processing it, if the gift is successfully sent, the gift service can broadcast the gift information to all client terminals in the current live stream.

[0071] S120: Parse the gift-giving notification, obtain the gift's identity identifier, and use the gift's identity identifier to obtain the corresponding animation resource file.

[0072] Gift notifications can include a gift identifier and a gift type identifier. Specifically, the gift identifier can be used to obtain the corresponding animation resource file, and the gift type identifier can be used to determine the type of virtual gift given by the user.

[0073] During the process of obtaining the corresponding resource file using the gift identity, the gift acquisition path corresponding to the gift identity can be found in the preset gift configuration list. The animation resource file corresponding to the gift acquisition path is downloaded from the content delivery network in server 10. The client terminal can save the downloaded animation resource file locally. In this way, when the same animation video needs to be used again, it does not need to be downloaded again, which helps to save loading time and improve processing efficiency. That is to say, when the user of the client terminal gives the same virtual gift, there is no need to download the animation resource file corresponding to the virtual gift repeatedly. The animation resource file can be obtained directly from the local machine.

[0074] S130: Parse the motion effect resource file to obtain the output video file and description information.

[0075] The motion effect resource file can include an output video file and descriptive information. The output video file can be a video file composed of all output video frames arranged in sequence. Specifically, the output video frame can include a first region, a second region, and a third region set at intervals. The first region is used to set the image of the current source video frame, the second region is used to set the images of each blending element corresponding to the current source video frame at intervals, and the third region is used to set the image of the control element corresponding to the current source video frame. The length of the current output video frame can be twice the length of the current source video frame, and the length and width of the first region correspond to the length and width of the current source video frame. Therefore, by parsing the motion effect resource file, the output video file and descriptive information can be obtained, facilitating the acquisition of the current source video frame based on the output video file and the extraction of the corresponding blending elements based on the descriptive information.

[0076] S200: Get the inserted element that corresponds to the blend element and is associated with the live screen of the live room.

[0077] Inserted elements are elements that need to be inserted into the animated video during the rendering process and are associated with the live stream frame. During the rendering process, corresponding inserted elements and blending elements can be inserted into the source video frames of the animated video. This allows the inserted elements associated with the current live stream frame to be displayed simultaneously when the animated video is played, enabling the animated video to present different effects based on different live stream frames, thus enhancing the entertainment and interactivity of the live stream.

[0078] In one implementation, S200 may include the following steps:

[0079] S210: Obtain the inserted image obtained from a screenshot of the live stream and use it as an inserted element.

[0080] Screenshots can be images captured from the current live stream in the streaming room through a streaming media service. Inserted images can be images obtained from screenshots and inserted as elements into the animated video.

[0081] In one implementation, the steps preceding S210 may include:

[0082] S211: Obtain the latest screenshot of the live stream from the server.

[0083] The process of capturing screenshots can be achieved through streaming media services. Specifically, the main task of streaming media services is to periodically capture and process the video stream of the live broadcast room. Generally speaking, streaming media services can capture one frame of the image from the video stream every 3-10 seconds and save it in the content delivery network of server 10 as a screenshot.

[0084] For instructions on how to obtain a screenshot, please refer to the following steps included in S211:

[0085] S2111: Receive the screenshot address sent by the live streaming server. The screenshot address is the address recorded by the live streaming server after capturing the latest screenshot of the live stream. The screenshot is then retrieved according to the screenshot address.

[0086] The client terminal can obtain the corresponding screenshot by receiving the screenshot address corresponding to the screenshot obtained after the streaming media service takes a screenshot of the live broadcast. Specifically, the screenshot address can be the address recorded by the streaming media service after the latest screenshot of the live broadcast is captured. Since the streaming media service periodically captures and processes the video stream of the live broadcast, and after capturing a screenshot, it can save the screenshot in the content delivery network and record the corresponding screenshot address. Specifically, the most recent screenshot address will overwrite the previous screenshot address, so that the address obtained by the client terminal from the streaming media service each time is the address of the most recent screenshot.

[0087] S212: Obtain the inserted image based on the screenshot.

[0088] After obtaining a screenshot based on the screenshot address, an inserted image can be obtained based on the screenshot. Specifically, the inserted image can include the screenshot and a processed image generated by processing the screenshot.

[0089] For instructions on how to obtain the inserted image based on a screenshot, please refer to the following steps included in S212:

[0090] S2121: Obtain a screenshot and perform corresponding image processing on the screenshot to obtain a processed image.

[0091] Inserted images can include screenshots and processed images. The processed image can be obtained by further processing the screenshot. By using both the screenshot and the processed image as the inserted image, both the screenshot and the processed image can be inserted into the video frames of the motion effect video during the rendering process. Thus, when playing the motion effect video, the screenshot can be displayed first, and then transformed into the processed image. This allows viewers to see both the unprocessed screenshot and the processed image simultaneously. Different processed images can be obtained based on different screenshots, and thus different effects can be presented in the motion effect video based on different screenshots, which helps to improve the personalization of the live broadcast.

[0092] In one implementation, S2121 may include the following steps:

[0093] S21211: Save the screenshot obtained according to the screenshot address locally, and record the corresponding first local address.

[0094] The first local address can be the address pointing to the screenshot. After the client terminal obtains the screenshot according to the screenshot address obtained from the streaming media service, the screenshot can be saved locally, and the first local address corresponding to the screenshot can be recorded. When inserting the screenshot into the motion effect video, the screenshot can be quickly obtained based on the first local address, which helps to improve the efficiency of the rendering process.

[0095] S21212: Use a preset processing filter to perform image processing on the screenshot to generate a processed image, save the processed image locally, and record the corresponding second local address.

[0096] The preset processing filter can be a pre-set processing filter in the client terminal. After the client terminal acquires a screenshot, it performs image processing on the screenshot that matches the preset processing filter, thereby obtaining a processed image. This processed image is also saved locally, along with a corresponding second local address. Specifically, the second local address can be a address pointing to the processed image. Retrieving the processed image quickly based on the second local address improves the efficiency of the rendering process. For example, such as... Figure 6As shown, if the current live stream is a live streamer's room and the streamer's face is present in the live stream frame, when a viewer sends a cartoon face gift, a screenshot can be captured. Then, a preset cartoon face filter can be used to process the screenshot to obtain a cartoon face image. During the rendering process of the video frames of the animation video corresponding to the cartoon face gift, both the screenshot and the cartoon face image can be inserted as interpolated elements. Thus, during the playback of the animation video, the screenshot can be displayed first, and then transition to the cartoon face image. This allows the cartoon face gift to present different effects based on the screenshot of the current live stream frame, adding more fun to the gift-giving process for users.

[0097] S21213: The first index information and the first local address corresponding to the screenshot, as well as the second index information and the second local address corresponding to the processed image, are stored together in a Map data structure to form a Map structure array.

[0098] The Map structure array can be obtained from the pre-set business side of the client terminal and is used to associate and store index information and the second local address of the corresponding inserted element. Specifically, the Key in the Map structure array can be used to store index information, and the Value in the Map structure array can be used to store the second local address of the inserted element. The Map structure array carrying the corresponding index information is determined using the index information of the mixed element corresponding to the current source video frame, and then the second local address of the inserted element is obtained from the Map structure array.

[0099] S220: Receive index information from the server.

[0100] The index information can be the index information in the description information. Since the index information can be associated with the inserted element and the mixed element, that is, the corresponding inserted element can be determined by receiving the index information of the determined mixed element.

[0101] S230: Obtain the local address pointed to by the index information in the Map structure array according to the index information, and obtain the matching insertion image according to the local address pointed to by the index information.

[0102] Specifically, if the index information is the first index information, then the first local address pointed to by the first index information is obtained from the Map structure array, and a screenshot is obtained according to the first local address as an insertion element. If the index information is the second index information, then the second local address pointed to by the second index information is obtained from the Map structure array, and a processed image is obtained according to the second local address as an insertion element.

[0103] S300: Blend the blending element with the corresponding inserting element to obtain a blended image.

[0104] After obtaining the blending element and the corresponding insert element associated with the screenshot of the live stream, the corresponding insert element and blending element can be blended and rendered to obtain a blended image. This blended image can then be rendered in the source video frame of the motion video, thereby enabling the simultaneous display of the insert element associated with the screenshot of the live stream when the virtual gift is given, while playing the motion video of the virtual gift, thus achieving a personalized motion video of the virtual gift.

[0105] In one implementation, S300 may include the following steps:

[0106] S310: Get the first texture image of the blending element.

[0107] After extracting the blending elements from the output video frames of the motion effect resource file, in order to facilitate the blending rendering of the blending elements and the inserted elements, it is necessary to render the blending elements and the inserted elements separately to obtain their respective texture images. Then, the respective texture images are blended and rendered to obtain a blended texture image. Specifically, the first texture image can be an image obtained by rendering using the pixel data of the blending elements.

[0108] For instructions on how to obtain the first texture image of the blending elements, please refer to the following steps included in S310:

[0109] S311: Obtain the first texture image of the blending elements from the current output video frame using the description information.

[0110] The description information can be generated by combining the first position data of the blended elements in the source video frame and the second position data in the output video frame. Specifically, after receiving the motion effect resource file, the client terminal parses the motion effect resource file and extracts the description information from the header files of the output video file and the motion effect resource file.

[0111] For instructions on how to obtain the first texture image of the blending elements from the current output video frame using the description information, please refer to the following steps included in S311:

[0112] S3111: Extract the second position data of the blending element corresponding to the corresponding position of the current output video frame from the description information.

[0113] Since the current output video frame is set with the image of the current source video frame and the image of the blending element at intervals, and the blending element, the current source video frame and the current output video frame are in one-to-one correspondence, in the process of obtaining the pixel data of the blending element, the current output video frame corresponding to the current source video frame can be determined, and then the second position data of the blending element in the current output video frame can be obtained. Then, the pixel data of the blending element can be obtained using the second position data.

[0114] S3112: Extract pixel data of the blending elements in the current output video frame using the second position data.

[0115] The pixel data of the blending element can be the pixel value corresponding to each pixel in the blending element, specifically, it can be the RGB value. The second position data can represent the position of the blending element in the current output video frame. Since the pixel values ​​of the blending element only have two colors, black and white, and the pixel value corresponding to black is 0 in hexadecimal and the pixel value corresponding to white is 255 in hexadecimal, the blending element stores only 0 and 255 pixel values. Therefore, the RGB values ​​of each pixel of the blending element's image can be determined by the second position data of the blending element in the current output video frame.

[0116] S3113: Generate the first texture image of the blended elements using the pixel data of the blended elements.

[0117] In creating the first texture image, RGB data can be obtained by acquiring the blending element, and then rendered onto the first blank texture image to obtain the first texture image. The first blank texture image can be a blank texture image used to render the first texture image; specifically, it can be obtained by rendering the blending element onto the first blank texture image. During the creation of the first blank texture image, a preset plugin tool can be used to create the first blank texture image based on the width and height of the blending element.

[0118] S320: Get the second texture image of the inserted element.

[0119] After obtaining the insert element corresponding to the blending element and associated with the live stream frame, the insert element can be rendered on the second texture image. This facilitates the blending rendering of the first texture image of the blending element corresponding to the second texture image to obtain a blended texture image, which in turn facilitates the blending rendering of the blended texture image in the source video frame of the motion effect video. Specifically, the second texture image can be an image obtained by rendering using the pixel data of the insert element.

[0120] For instructions on how to obtain the second texture image of the blended elements, please refer to the following steps included in S320:

[0121] S321: Extract the index information corresponding to the mixed elements from the description information.

[0122] The description information may include the index information corresponding to the mixed element. Since the index information can associate the corresponding mixed element and the inserted element, the inserted element corresponding to the mixed element can be determined by extracting the index information corresponding to the mixed element from the description information.

[0123] S322: Use index information to retrieve the pixel data of the matching insertion element in the Map structure array.

[0124] By utilizing the index information, we can obtain the second local address that points to the inserted element stored locally from the Map structure array. We can then retrieve the pixel data of the corresponding inserted element according to the address pointed to by the second local address, which makes it easier to render the second texture image using the pixel data of the inserted element.

[0125] S323: Generate a second texture image of the inserted element using the pixel data of the inserted element.

[0126] In creating the second texture image, the pixel data of the inserted element can be obtained and rendered onto a second blank texture image. The second blank texture image can be a blank texture image used to render the second texture image; specifically, the second texture image can be obtained by rendering the pixel data of the inserted element onto the second blank texture image. During the creation of the second blank texture image, a preset plugin can be used to create the second blank texture image based on the width and height of the inserted element.

[0127] S330: Render the first texture image and the second texture image together to obtain a blended texture image of the blended image.

[0128] After obtaining the first texture image rendered using the pixel data of the blending element and the second texture image rendered using the pixel data of the inserted element, the first and second texture images can be blended to obtain a blended texture image. After obtaining the blended texture image, the blending element and the inserted element can be inserted into the source video frame of the motion effect video by blending the blended texture image with the corresponding current source video frame.

[0129] In one implementation, S330 may include the following steps:

[0130] S331: Process the second texture image according to the shape of the first texture image, so that the shape of the processed second texture image matches the shape of the first texture image, thereby obtaining a mixed texture image.

[0131] like Figure 7 As shown, Figure 7 In this context, 'a' can represent the first texture image. Figure 7 In this context, 'b' can represent the second texture image. Figure 7 In this context, 'c' can represent a blended texture image. During the process of blending and rendering a first texture image and a second texture image, the second texture image is processed using the first texture image to create an image whose shape matches that of the first texture image, thus becoming the blended texture image.

[0132] S400: When rendering motion video, the blended image is rendered at the corresponding position of the current source video frame according to the first position data corresponding to the current source video frame of the motion video, until all source video frames of the motion video are rendered.

[0133] During the rendering of motion effect videos, the mixed texture images corresponding to the obtained mixed images can be rendered at the corresponding positions of the current source video frames according to their respective first position data. After rendering all the source video frames in the motion effect video, it is possible to simultaneously display the screenshots and processed images corresponding to the motion effect video when playing the motion effect video, thereby realizing the personalized display of the motion effect video.

[0134] In one implementation, S400 may include the following steps:

[0135] S410: Render the blended texture image at the corresponding position of the third texture image of the current source video frame according to the first position data corresponding to the current source video frame.

[0136] The first position data indicates the position in the current source video frame where the inserted element needs to be inserted. The third texture image can be a texture image obtained by rendering the current source video frame. By rendering the blended texture image onto the third texture image according to the first position data, the inserted element can be inserted into the current source video frame.

[0137] For how to render the blended texture image at the corresponding position of the third texture image of the current source video frame according to the first position data corresponding to the current source video frame, the following steps included in S410 can be referenced:

[0138] S411: Obtain the first texture image and the third texture image from the motion effect resource file.

[0139] The first texture image can be an image rendered based on the pixel data of the acquired blending elements in the output video frame of the motion effect resource file, and the third texture image can be an image rendered using the pixel data of the source video frame in the motion effect resource file.

[0140] For instructions on how to obtain the third texture image, please refer to the following steps included in S411:

[0141] S4111: Obtain the third texture image of the current source video frame from the current output video frame of the output video file.

[0142] Since the current source video frame is set in the first region of the current output video frame, the corresponding current source video frame can be obtained from the output video frame, and then the current source video frame can be rendered to obtain the third texture image.

[0143] In one implementation, S4111 may include the following steps:

[0144] S41111: Obtain the pixel data of the current source video frame from the current output video frame of the output video file.

[0145] Since the image of the current source video frame is set in the first region of the corresponding current output video frame, when obtaining the pixel data of the current source video frame, the pixel values ​​of each pixel of the image of the current source video frame can be obtained in the corresponding current output video frame.

[0146] S41112: Generate a third texture image of the current source video frame using the pixel data of the current source video frame.

[0147] In creating a third texture image, the pixel data of the current source video frame can be obtained and rendered onto a third blank texture image. The third blank texture image can be a blank texture image used for rendering the third texture image; specifically, it can be obtained by rendering the pixel data of the current source video frame onto a third blank texture image. During the creation of the third texture image, a preset plugin tool can be used to create a third blank texture image based on the width and height of the current source video frame. Specifically, the width of the third blank texture image can be half the width of the current output video frame, and the height of the third blank texture image can be the same as the height of the current output video frame.

[0148] After obtaining the third texture image, the blended texture image can be rendered at the position corresponding to the first position data in the third texture image, thereby inserting the blended element and the inserted element into the current source video frame.

[0149] During the rendering process of each source video frame of the motion effect video, the blending element and the insertion element corresponding to the current source video frame are obtained. Then, the first texture image corresponding to the blending element and the second texture image corresponding to the insertion element are blended and rendered to obtain a blended texture image. Then, the blended texture image is rendered in the third texture image corresponding to the current source video frame according to the first position data, until the rendering of all source video frames in the motion effect video is completed.

[0150] In summary, in the first embodiment of the rendering method for the virtual gift animation video of this application, the mixing elements required for rendering the animation video are obtained, and the insertion elements corresponding to the mixing elements and associated with the live broadcast screen of the live broadcast room are obtained. The insertion elements corresponding to the mixing elements are mixed and rendered to obtain a mixed image. Then, when rendering the animation video, the mixed image is rendered at the corresponding position of the current source video frame according to the first position data corresponding to the current source video frame of the animation video, until all source video frames of the animation video are rendered. This can effectively and efficiently realize the dynamic insertion of the insertion elements associated with the live broadcast screen of the live broadcast room when the source video frames are rendered, increase the display effect of the animation video during the process of users giving virtual gifts, and realize that the animation video effect of virtual gifts presented in the live broadcast screen of different live broadcast rooms is different, realizing personalized video gift effects, which is conducive to increasing the entertainment and interactivity during the live broadcast and improving user stickiness.

[0151] The second embodiment of the rendering method for the animation video of virtual gifts in this application can use server 10 as the execution subject, and implement the technical solution described in the first embodiment of the rendering method for the animation video of virtual gifts in this application from the perspective of server 10.

[0152] like Figure 8 As shown, this embodiment may include the following steps:

[0153] M100: Receives gift requests from client terminals that send virtual gifts.

[0154] A gift-giving request can be a request sent by a client terminal to the gift service when sending a virtual gift to a streamer.

[0155] In one implementation, the steps preceding M100 may include:

[0156] M110: In response to a gift request, capture a screenshot of the live stream.

[0157] After receiving a gift request from a client terminal, the gift service can process the request. If the gift is successfully sent, it can send a request to the streaming media service to obtain a real-time screenshot of the live stream. Specifically, after a successful gift gift, the service can also determine whether the virtual gift requires a screenshot by checking the gift type identifier carried in the gift request. For example, a cartoon face gift requires a screenshot to generate a cartoon face image. If the gift type identifier of the virtual gift indicates that a screenshot is required, a screenshot request is sent to the streaming media service.

[0158] For instructions on how to capture a screenshot of a live stream, please refer to the following steps included in the M110 tutorial:

[0159] M111: Captures and saves screenshots of the live stream at preset time intervals, and records the address of the latest captured screenshot.

[0160] Streaming services can periodically capture and process the video stream from a live broadcast. Generally, they can capture one frame from the video stream every 3-10 seconds, save it as a screenshot, and record the URL of the captured frame. Specifically, the URL of the most recent screenshot will overwrite the URL of the previous one, ensuring that the latest screenshot is always captured.

[0161] M120: Based on the gift-giving request and the address of the screenshot acquisition, a gift-giving information notification is generated so that all client terminals in the live broadcast room can obtain the inserted image obtained from the screenshot of the live broadcast and use it as an insert element.

[0162] After obtaining a screenshot through the streaming media service, a gift-giving notification can be generated based on the gift-giving request and the address where the screenshot was obtained. This notification enables all client terminals in the live stream to obtain the inserted image, which is based on the screenshot of the live stream and used as an insert element.

[0163] In one implementation, M120 may include the following steps:

[0164] M121: Performs a validity check on the screenshot.

[0165] After obtaining screenshots, it's necessary to verify their legality to prevent them from violating regulations and affecting the generated motion graphics video. Specifically, the legality verification process can include checking for violations, such as explicit content, black screens, and whether the screenshot has passed review.

[0166] M122: If the verification passes, a gift-giving notification will be generated based on the gift-giving request and the URL of the screenshot.

[0167] If the legitimacy verification passes, a gift notification can be generated based on the gift request and the URL where the screenshot was obtained.

[0168] M200: In response to a gift-giving request, a gift-giving information notification is sent to all client terminals in the live broadcast room, enabling all client terminals in the live broadcast room to obtain the blending elements required for rendering the motion effect video using the gift-giving information notification, obtain the insertion elements corresponding to the blending elements and associated with the live broadcast screen of the live broadcast room, and blend the blending elements with the corresponding insertion elements to obtain a blended image. Then, when rendering the motion effect video, the blended image is rendered at the corresponding position of the current source video frame according to the first position data corresponding to the current source video frame of the motion effect video, until all source video frames of the motion effect video are rendered.

[0169] Specifically, the gift-giving notification can be broadcast to all client terminals in the live stream. Upon receiving the notification, each client terminal can use it to obtain the blending elements needed for rendering the animated video, acquire the corresponding insertion elements associated with the live stream frame, and then blend the blending elements with the corresponding insertion elements to obtain a mixed image. During the rendering of the animated video, the mixed image is rendered at the corresponding position of the current source video frame according to the first position data corresponding to the current source video frame, until all source video frames of the animated video are rendered. Specific operation steps can be found in the description of the first embodiment of the virtual gift animated video rendering processing method of this application, and will not be repeated here.

[0170] like Figure 9 As shown, the electronic terminal 100 described in the embodiments of this application can be the client terminal described above. The electronic terminal 100 includes a processor 110, a memory 120, and a communication circuit. The memory 120 and the communication circuit are coupled to the processor 110.

[0171] The memory 120 is used to store computer programs and may be RAM (Read-Only Memory), ROM (Random Access Memory), or other types of storage devices. Specifically, the memory may include one or more computer-readable storage media, which may be non-transitory. The memory may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory is used to store at least one line of program code.

[0172] Processor 110 is used to control the operation of electronic terminal 100. Processor 110 can also be referred to as CPU (Central Processing Unit). Processor 110 may be an integrated circuit chip with signal processing capabilities. Processor 110 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. The general-purpose processor can be a microprocessor, or processor 110 can be any conventional processor.

[0173] The processor 110 is used to execute the computer program stored in the memory 120 to implement the rendering processing method described in the embodiment of the virtual gift animation video rendering processing method of this application.

[0174] In some embodiments, the electronic terminal 100 may further include a peripheral device interface 130 and at least one peripheral device. The processor 110, memory 120, and peripheral device interface 130 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 130 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of a radio frequency circuit 140, a display screen 150, an audio circuit 160, and a power supply 170.

[0175] Peripheral device interface 130 can be used to connect at least one I / O (Input / output) related peripheral device to processor 110 and memory 120. In some embodiments, processor 110, memory 120 and peripheral device interface 130 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 110, memory 120 and peripheral device interface 130 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0176] The radio frequency (RF) circuit 140 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 140 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 140 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 140 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 140 can communicate with other terminals via at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 140 may also include circuitry related to NFC (Near Field Communication), which is not limited in this application.

[0177] Display screen 150 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 150 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 110 for processing. In this case, display screen 150 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one display screen 150, located on the front panel of electronic terminal 100; in other embodiments, there may be at least two display screens, respectively located on different surfaces of electronic terminal 100 or in a folded design; in still other embodiments, display screen 150 may be a flexible display screen, located on a curved or folded surface of electronic terminal 100. Furthermore, display screen 150 may be configured as a non-rectangular irregular shape, i.e., a non-rectangular screen. Display screen 150 may be made of materials such as LCD (Liquid Crystal Display) or OLED (Organic Light-Emitting Diode).

[0178] The audio circuit 160 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to the processor 110 for processing, or input to the radio frequency circuit 140 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each positioned at a different location on the electronic terminal 100. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert electrical signals from the processor 110 or the radio frequency circuit 140 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 160 may also include a headphone jack.

[0179] Power supply 170 is used to power the various components in electronic terminal 100. Power supply 170 can be alternating current, direct current, a disposable battery, or a rechargeable battery. When power supply 170 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, while a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0180] For a detailed description of the functions and execution processes of each functional module or component in the electronic terminal embodiment of this application, please refer to the description in the above embodiment of the rendering processing method for the animation video of virtual gifts in this application, which will not be repeated here.

[0181] In the several embodiments provided in this application, it should be understood that the disclosed electronic terminal 100 and background processing method can be implemented in other ways. For example, the embodiments of the electronic terminal 100 described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0182] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0183] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0184] like Figure 10 As shown, the server 200 described in the server embodiment of this application can be the server 10 described above. The server 200 includes a processor 210, a memory 220, and a communication circuit. The memory 220 and the communication circuit are coupled to the processor 210.

[0185] In some embodiments, server 200 may further include: peripheral device interface 230, radio frequency circuit 240, display screen 250, audio circuit 260, and power supply 270. Specifically, the circuit structure of server 200 is the same as that of electronic terminal 100 in the above-described electronic terminal embodiments of this application, and therefore will not be described again.

[0186] See Figure 11 If the integrated units described above are implemented as software functional units and sold or used as independent products, they can be stored in computer-readable storage medium 300. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions / computer programs to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this invention. The aforementioned storage medium includes various media such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks, as well as electronic terminals such as computers, mobile phones, laptops, tablets, and cameras that have the aforementioned storage media.

[0187] The execution process of program data in a computer-readable storage medium can be described in the above embodiment of the rendering processing method for the animation video of virtual gifts in this application, and will not be repeated here.

[0188] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A rendering process method of a virtual gift motion effect video, characterized in that, include: Obtain the blending elements required to render the animated video; Obtain the insertion element that corresponds to the mixed element and is associated with the live broadcast screen of the live broadcast room; The blending element is blended with the corresponding inserting element and rendered to obtain a blended image; When rendering the motion effect video, the mixed image is rendered at the corresponding position of the current source video frame according to the first position data corresponding to the current source video frame of the motion effect video, until all source video frames of the motion effect video are rendered. The step of obtaining the inserted element corresponding to the mixed element and associated with the live broadcast screen includes: Obtain the inserted image obtained from a screenshot of the live stream and use it as the inserted element; Prior to obtaining the inserted image obtained from a screenshot of the live stream and used as the inserted element, the process includes: Obtain the latest screenshot of the live stream from the server; The inserted image is obtained based on the screenshot. The step of obtaining the screenshot of the live stream obtained by the server from the latest capture of the live stream room includes: Receive the screenshot address sent by the server, where the screenshot address is the address recorded by the server after the latest screenshot of the live broadcast is obtained, and obtain the screenshot according to the screenshot address; The inserted image includes the screenshot and a processed image generated by processing the screenshot; obtaining the inserted image based on the screenshot includes: Obtain the screenshot, and further perform corresponding image processing on the screenshot to obtain the processed image; Specifically, when playing the animated video, the screenshot is first displayed, and then the screenshot is transformed into the processed image.

2. The rendering processing method according to claim 1, characterized in that: The process of obtaining the screenshot and further processing the screenshot to obtain the processed image includes: The screenshot obtained according to the screenshot address is saved locally, and the corresponding first local address is recorded; The screenshot is processed using a preset processing filter to generate the processed image, which is then saved locally, and the corresponding second local address is recorded.

3. The rendering processing method according to claim 2, characterized in that: The process of obtaining the screenshot and performing corresponding image processing on the screenshot to generate the processed image further includes: The first index information and the first local address corresponding to the screenshot, as well as the second index information and the second local address corresponding to the processed image, are associated and stored in a Map data structure to form a Map structure array.

4. The rendering processing method according to claim 3, characterized in that: The step of obtaining the inserted image obtained from a screenshot of the live stream and used as the inserted element includes: Receive index information from the server; The local address pointed to by the index information is obtained from the Map structure array according to the index information, and the matching insertion image is obtained from the local address pointed to by the index information; wherein, if the index information is the first index information, the first local address pointed to by the first index information is obtained from the Map structure array, and the screenshot is obtained from the first local address as the insertion element; if the index information is the second index information, the second local address pointed to by the second index information is obtained from the Map structure array, and the processed image is obtained from the second local address as the insertion element.

5. The rendering processing method according to any one of claims 1-4, characterized in that: The process of obtaining the blending elements required for rendering the motion video includes: Obtain the first texture image of the blended element; The step of obtaining the inserted element corresponding to the mixed element and associated with the live broadcast screen of the live broadcast room includes: Obtain the second texture image of the inserted element; The step of blending the blending element with the corresponding inserted element to obtain a blended image includes: The first texture image and the second texture image are blended and rendered to obtain a blended texture image of the blended image; The step of rendering the mixed image at the corresponding position of the current source video frame according to the first position data corresponding to the current source video frame of the motion effect video includes: The hybrid texture image is rendered at the corresponding position of the third texture image of the current source video frame according to the first position data corresponding to the current source video frame.

6. The rendering processing method according to claim 5, characterized in that: Before obtaining the blending elements required to render the motion video, the process includes: Receive gift notifications from the server; The gift-giving notification is parsed to obtain the gift's identity identifier, and the corresponding animation resource file is obtained using the gift's identity identifier; The step of obtaining the first texture image of the blended elements includes: Obtain the first texture image and the third texture image from the motion effect resource file.

7. The rendering processing method according to claim 6, characterized in that: The step of obtaining the first texture image and the third texture image from the motion effect resource file includes: The motion effect resource file is parsed to obtain the output video file and description information; The third texture image of the current source video frame is obtained from the current output video frame of the output video file; wherein the current output video frame is provided with the image of the current source video frame and the image of the blending element at intervals; The first texture image of the blending element is obtained from the current output video frame using the description information.

8. The rendering processing method according to claim 7, characterized in that: Extracting the third texture image of the current source video frame from the current output video frame of the output video file includes: Obtain the pixel data of the current source video frame from the current output video frame of the output video file; The third texture image of the current source video frame is generated using the pixel data of the current source video frame.

9. The rendering processing method according to claim 7, characterized in that: The step of extracting the first texture image of the blending elements from the current output video frame using the description information includes: Extract the second position data of the blending element corresponding to the corresponding position of the current output video frame from the description information; The pixel data of the blending element is extracted from the current output video frame using the second location data; The first texture image of the blending element is generated using the pixel data of the blending element.

10. The rendering processing method according to claim 7, characterized in that: The step of obtaining the second texture image of the inserted element includes: Extract the index information corresponding to the mixed element from the description information; The pixel data of the matching inserted element is obtained from the Map structure array using the index information; The second texture image of the inserted element is generated using the pixel data of the inserted element.

11. The rendering processing method according to claim 5, characterized in that: The step of blending and rendering the first texture image and the second texture image to obtain the blended texture image includes: The second texture image is processed according to the shape of the first texture image, so that the shape of the processed second texture image matches the shape of the first texture image, thereby obtaining the hybrid texture image.

12. A rendering process method of a virtual gift motion effect video, characterized in that, include: The client terminal that receives the virtual gift sends a gift request; In response to the gift-giving request, a gift-giving information notification is sent to all client terminals in the live broadcast room, so that all client terminals in the live broadcast room can use the gift-giving information notification to obtain the blending elements required for rendering the motion effect video, obtain the insertion elements corresponding to the blending elements and associated with the live broadcast screen of the live broadcast room, and blend the blending elements with the corresponding insertion elements to obtain a blended image. Then, when rendering the motion effect video, the blended image is rendered at the corresponding position of the current source video frame according to the first position data corresponding to the current source video frame of the motion effect video, until all source video frames of the motion effect video are rendered. The step of obtaining the inserted element corresponding to the mixed element and associated with the live broadcast screen of the live broadcast room includes: Obtain the inserted image obtained from a screenshot of the live stream and use it as the inserted element; Prior to obtaining the inserted image obtained from a screenshot of the live stream and used as the inserted element, the process includes: Obtain the latest screenshot of the live stream from the server; The inserted image is obtained based on the screenshot. The step of obtaining the screenshot of the live stream obtained by the server from the latest capture of the live stream room includes: Receive the screenshot address sent by the server, where the screenshot address is the address recorded by the server after the latest screenshot of the live broadcast is obtained, and obtain the screenshot according to the screenshot address; The inserted image includes the screenshot and a processed image generated by processing the screenshot; obtaining the inserted image based on the screenshot includes: Obtain the screenshot, and further perform corresponding image processing on the screenshot to obtain the processed image; Specifically, when playing the animated video, the screenshot is first displayed, and then the screenshot is transformed into the processed image.

13. The rendering processing method according to claim 12, characterized in that: Before sending the gift-giving notification to all client terminals in the live stream, the following steps are included: In response to the gift-giving request, a screenshot of the live stream is captured in the live stream room. The gift-giving information notification is generated based on the gift-giving request and the address where the screenshot was obtained, so that all client terminals in the live broadcast room can obtain the inserted image obtained from the screenshot of the live broadcast screen and used as the inserted element.

14. The rendering processing method according to claim 13, characterized in that: The process of capturing a screenshot of the live stream from the live stream room includes: The live stream screen of the live stream room is captured at preset time intervals to obtain screenshots and saved. The acquisition address of the latest captured screenshot is also recorded.

15. The rendering processing method according to claim 14, characterized in that: The process of generating the gift-giving notification based on the gift-giving request and the URL of the screenshot includes: The screenshots are validated for legality. If the verification passes, the gift-giving information notification is generated based on the gift-giving request and the address where the screenshot was obtained.

16. An electronic terminal, characterized in that It includes a processor, a memory, and a communication circuit; the memory and the communication circuit are coupled to the processor, the memory stores a computer program, and the processor is capable of executing the computer program to implement the rendering processing method as described in any one of claims 1-11.

17. A server, characterized in that, It includes a processor, a memory, and a communication circuit; the memory and the communication circuit are coupled to the processor, the memory stores a computer program, and the processor is capable of executing the computer program to implement the rendering processing method as described in any one of claims 12-15.

18. A computer-readable storage medium, characterized in that, The system contains a computer program that can be executed by a processor to implement the rendering processing method as described in any one of claims 1-15.

Images