Transparent special effect playing method, device, equipment and medium thereof

Abstract

The application discloses a transparent special effect playing method and device, equipment and medium, and the method comprises the following steps: in response to a display instruction of a target gift special effect in a live broadcast room, a hardware decoding interface is called to decode the gift special effect video to obtain a decoded image frame contained in the gift special effect video; the original image size and the layout image size of the current decoded image frame are checked; if the original image size and the layout image size are different, the height and width trimming factors are adjusted based on the original image size and the layout image size; the color texture image and the transparent texture image contained in the current decoded image frame are determined according to the height and width trimming factors, so that the color texture image and the transparent texture image are trimmed from the current decoded image frame; and the color texture image and the transparent texture image are merged to generate a current transparent special effect image frame of the target gift special effect for display. The application effectively guarantees the display effect of the transparent gift special effect in the interface and improves the live broadcast visual effect.

Description

Technical Field

[0001] This application relates to the field of online live streaming, and in particular to a method for playing transparent special effects. It also relates to the corresponding apparatus, equipment, and non-volatile storage medium. Background Technology

[0002] In existing live streaming rooms, viewers can interact with the streamer by sending virtual gifts. Some virtual gifts will display corresponding animation effects on the live streaming interface after being sent. Rich and dazzling gift animation effects can effectively enhance the atmosphere of the live streaming room. Common gift animation effect technologies are GIF, APNG, or Lottie. Among them, GIF has the advantages of high compatibility and low storage space consumption, but it only supports a maximum of 256 colors. Gift animation effects with a limited number of colors affect the user's viewing experience. In contrast, APNG supports a wider and clearer color range than GIF, but its poor compatibility limits the development and design of gift animation effects. While Lottie has a high vector animation compression rate, it does not support complex particle effects and cannot display exquisite gift animation effects. Furthermore, it requires a lot of performance for processing, affecting the stability of the live streaming application. When gift animation effects are displayed, it can easily cause the live streaming interface to lag, affecting the user's live streaming experience.

[0003] In view of the problems existing in the virtual gift animation effects of current live streaming platforms, the applicant has made corresponding explorations in order to solve these problems. Summary of the Invention

[0004] The purpose of this application is to provide a method for playing transparent special effects to meet user needs, and it also relates to the corresponding apparatus, device, non-volatile storage medium and computer program product.

[0005] To achieve the objectives of this application, the following technical solution is adopted:

[0006] A method for playing transparent special effects, proposed for the purposes of this application, includes the following steps:

[0007] In response to the display command applied to the target gift effect in the live broadcast room, the hardware decoding interface is called to decode the gift effect video and obtain the decoded image frames contained in the gift effect video. Each decoded image frame contains a color texture image and a transparency texture image.

[0008] Check whether the original image size and the layout image size of the current decoded image frame are the same. If they are not the same, adjust the height clipping factor and the width clipping factor based on the size ratio between the original image size and the layout image size.

[0009] Based on the height clipping factor and the width clipping factor, determine the respective texture image regions of the color texture image and the transparent texture image contained in the current decoded image frame, so as to clip the color texture image and the transparent texture image from the current decoded image frame;

[0010] The color texture image and the transparent texture image are merged to generate the current transparent effect image frame of the target gift effect, and the current transparent effect frame image is output and displayed on the live broadcast interface.

[0011] In a further embodiment, the step of calling the hardware decoding interface to decode the gift effect video and obtain the decoded image frames contained in the gift effect video includes the following steps performed by the client:

[0012] Responding to the gift selection and gifting event of the virtual gift control in the live broadcast room interface, determining the target gift effect corresponding to the event, and generating a display instruction containing the gift icon of the target gift effect;

[0013] Check if there is a target gift effect video corresponding to the target gift identifier in the gift effect library. If it exists, retrieve the target gift video from the gift effect library. If it does not exist, retrieve the target gift effect video from the server.

[0014] The hardware decoding interface is invoked to decode the target gift effect video, obtain multiple decoded image frames contained in the target gift effect video, and sort the decoded image frames according to the timeline of the target gift effect video.

[0015] In a further embodiment, the step of calling the hardware decoding interface to decode the gift effect video and obtain the decoded image frames contained in the gift effect video includes the following steps performed by the client:

[0016] Call the hardware decoding interface to decode the gift effect video and obtain the decoded image frames;

[0017] When the decoded image frame is a luminance-chrominance image frame, the luminance component and chrominance component contained therein are extracted from the luminance-chrominance image frame based on the original image size of the luminance-chrominance image frame.

[0018] According to the image conversion algorithm, based on the luminance component and chrominance component, the luminance and chrominance image frame is converted into a decoded image frame containing a color texture image and a transparency texture image.

[0019] In a further embodiment, the step of verifying whether the original image size and the layout image size of the current decoded image frame are the same, and if they are not the same, adjusting the height clipping factor and the width clipping factor based on the size ratio between the original image size and the layout image size, includes the following steps performed by the client:

[0020] Obtain the original image size and layout image size of the current decoded image frame, wherein the original image size includes the original image width and the original image height, and the layout image size includes the layout image width and the layout image height;

[0021] Determine whether the height of the original image is equal to the height of the layout image. If they are not equal, adjust the height clipping factor based on the height ratio of the original image height to the layout image height. If they are equal, adjust the height clipping factor based on a preset value.

[0022] Determine whether the width of the original image is equal to the width of the layout image. If they are not equal, adjust the width cropping factor based on the width ratio of the original image width to the layout image width. If they are equal, adjust the width cropping factor based on a preset value.

[0023] In a further embodiment, the step of determining the respective texture image regions of the color texture image and the transparent texture image contained in the current decoded image frame based on the height clipping factor and the width clipping factor, so as to clip the color texture image and the transparent texture image from the current decoded image frame, includes the following steps performed by the client:

[0024] The color texture algorithm is invoked to determine the color texture image region from the current decoded image frame based on the texture coordinates of the current decoded image frame, the height clipping factor, and the width clipping factor.

[0025] The transparent texture algorithm is invoked to determine the transparent texture image region from the current decoded image frame based on the texture coordinates of the current decoded image frame, the height clipping factor, and the width clipping factor.

[0026] The color texture image is cropped from the color texture image region of the current decoded image frame, and the transparent texture image is cropped from the transparent texture image region of the current decoded image frame.

[0027] In a further embodiment, the step of merging the color texture image and the transparent texture image to generate the current transparent effect image frame of the target gift effect, and outputting and displaying the current transparent effect frame image to the live broadcast interface, includes the following steps performed by the client:

[0028] The color texture image and the transparent texture image are merged to generate a transparent effect image frame for the target gift effect;

[0029] Determine the display position of the target gift effect in the current live stream interface;

[0030] Adjust the image size of the transparent effect image frame according to the current interface size of the live broadcast room, and output the transparent effect image frame to the gift display position for display.

[0031] In a further embodiment, after the step of merging the color texture image and the transparent texture image to generate the current transparent effect image frame of the target gift effect, and outputting and displaying the current transparent effect frame image to the live broadcast interface, the following steps are included, which are performed by the client:

[0032] Obtain the next decoded image frame contained in the gift effect video, and based on the height cropping factor and width cropping factor adjusted according to the first decoded image frame, crop out the color texture image and transparent texture image contained in the next decoded image frame;

[0033] The color texture image and the transparent texture image are merged to generate the next transparent effect image frame of the target gift effect. After the transparent effect image frame corresponding to the first decoded image frame is displayed, the next transparent effect image frame is displayed.

[0034] Transparent effect image frames of each decoded image frame contained in the gift effect video are generated in sequence and displayed.

[0035] A transparent special effects playback device proposed for the purposes of this application includes:

[0036] The decoding image acquisition module is used to respond to the display command applied to the target gift effect in the live broadcast room, call the hardware decoding interface to decode the gift effect video, and obtain the decoded image frames contained in the gift effect video. Each decoded image frame contains a color texture image and a transparent texture image.

[0037] The image size verification module is used to verify whether the original image size and the layout image size of the current decoded image frame are the same. If they are not the same, the height clipping factor and the width clipping factor are adjusted based on the size ratio between the original image size and the layout image size.

[0038] The texture image cropping module is used to determine the texture image regions of the color texture image and the transparent texture image contained in the current decoded image frame according to the height cropping factor and the width cropping factor, so as to crop the color texture image and the transparent texture image from the current decoded image frame;

[0039] The special effects image display module is used to merge the color texture image and the transparent texture image to generate the current transparent special effects image frame of the target gift special effects, and output the current transparent special effects frame image to the live broadcast interface.

[0040] In a further embodiment, the decoded image acquisition module includes:

[0041] The Gift Identifier Determination Submodule is used to respond to the gift selection and gifting event of the virtual gift control in the live broadcast room interface, determine the target gift effect corresponding to the event, and generate a display instruction containing the gift identifier of the target gift effect.

[0042] The special effects video acquisition submodule is used to query whether there is a target gift special effects video corresponding to the target gift identifier in the gift special effects library. If it exists, the target gift video is retrieved from the gift special effects library; if it does not exist, the target gift special effects video is retrieved from the server.

[0043] The image frame decoding submodule is used to call the hardware decoding interface to decode the target gift effect video, obtain multiple decoded image frames contained in the target gift effect video, and sort the decoded image frames according to the timeline of the target gift effect video.

[0044] In a preferred embodiment, the decoded image acquisition module further includes:

[0045] The image frame acquisition submodule is used to call the hardware decoding interface to decode the gift effect video and obtain the decoded image frames;

[0046] The component extraction submodule is used to extract the luminance and chrominance components contained in the luminance and chrominance image frame based on the original image size of the luminance and chrominance image frame when the decoded image frame is a luminance and chrominance image frame.

[0047] The image frame conversion submodule is used to convert the luminance and chrominance image frame into a decoded image frame containing a color texture image and a transparency texture image based on the luminance component and chrominance component according to the image conversion algorithm.

[0048] In a further embodiment, the texture image cropping module includes:

[0049] The image size acquisition submodule is used to acquire the original image size and layout image size of the current decoded image frame, wherein the original image size includes the original image width and the original image height, and the layout image size includes the layout image width and the layout image height;

[0050] The height factor adjustment submodule is used to determine whether the height of the original image is equal to the height of the layout image. If they are not equal, the height clipping factor is adjusted based on the height ratio of the original image height to the layout image height. If they are equal, the height clipping factor is adjusted based on a preset value.

[0051] The width factor adjustment submodule is used to determine whether the width of the original image is equal to the width of the layout image. If they are not equal, the width cropping factor is adjusted based on the width ratio of the original image width to the layout image width. If they are equal, the width cropping factor is adjusted based on a preset value.

[0052] In a further embodiment, the texture image cropping module includes:

[0053] The color region determination submodule is used to call the color texture algorithm to determine the color texture image region of the color texture image from the current decoded image frame based on the texture coordinates of the current decoded image frame, the height clipping factor, and the width clipping factor.

[0054] The transparent region determination submodule is used to call the transparent texture algorithm to determine the transparent texture image region from the current decoded image frame based on the texture coordinates of the current decoded image frame, the height clipping factor, and the width clipping factor.

[0055] The texture image cropping submodule is used to crop the color texture image from the color texture image region of the current decoded image frame and to crop the transparent texture image from the transparent texture image region of the current decoded image frame.

[0056] In a further embodiment, the special effects image display module includes:

[0057] The texture image merging submodule is used to merge the color texture image and the transparent texture image to generate a transparent effect image frame for the target gift effect;

[0058] The display position determination submodule is used to determine the display position of the target gift effect in the current live broadcast interface.

[0059] The transparent image output submodule is used to adjust the image size of the transparent effect image frame according to the current interface size of the live broadcast room, and output the transparent effect image frame to the gift display position for display.

[0060] To address the aforementioned technical problems, this application also provides a computer device, including a memory and a processor. The memory stores computer-readable instructions, which, when executed by the processor, cause the processor to perform the steps of the aforementioned transparent special effects playback method.

[0061] To address the aforementioned technical problems, this application also provides a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the aforementioned transparent special effects playback method.

[0062] To address the aforementioned technical problems, this application also provides a computer program product, including a computer program and computer instructions. When the computer program and computer instructions are executed by a processor, the processor performs the steps of the aforementioned transparent special effects playback method.

[0063] Compared with existing technologies, the advantages of this application are as follows:

[0064] This application utilizes hardware-decoded video processing to display gift effects in live streams. It obtains decoded image frames by hardware decoding the virtual gift effect video, then crops them. Following the timeline of the gift image video, it sequentially crops the color texture images and transparent texture images contained within the decoded image frames, merges them to generate transparent effect image frames for display. The effective display of each transparent effect image frame on the live stream interface presents the transparent gift effect of the virtual gift. Compared to traditional gift animation effects technologies such as GIF, APNG, or Lottie, hardware-decoded video processing of gift effect videos has a higher compression rate, saving device storage space. Furthermore, hardware-based decoding and playback of gift effects offers higher processing efficiency, improving the stability of the live stream application. It also provides fast and smooth playback of gift effects for users in the live stream, effectively enhancing the live stream atmosphere.

[0065] Secondly, this application dynamically adjusts the cropping factor to control the texture image cropping of the decoded image frame, solving the cropping deviation caused by the hardware decoding aligning the resolution of each image frame in the gift effect video at a resolution multiple of 16 during decoding. The cropping factor is dynamically adjusted according to the size ratio of the original image size and the layout image size after filling of the decoded image frame, controlling the texture area that needs to be cropped in the decoded image frame, so as to crop out the complete color texture image and transparent texture image from the decoded image frame, so that the final gift effect does not have green edges or black shadow residue and other effect loss phenomena, ensuring the display effect of the gift effect in the live broadcast interface. Attached Figure Description

[0066] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0067] Figure 1 A typical network deployment architecture diagram related to the implementation of the technical solution of this application;

[0068] Figure 2 This is a flowchart illustrating a typical embodiment of the transparent special effects playback method of this application;

[0069] Figure 3 This is a schematic diagram of a decoded image frame in this application that has color texture images and transparent texture images and has not been padded with size.

[0070] Figure 4 This is a schematic diagram of the size-filled decoded image frame with color texture image and transparent texture image in this application, and the decoded image frame shown is a decoded image frame that has undergone traditional half-cropping.

[0071] Figure 5 This is a schematic diagram of the size-filled decoded video frame with color texture image and transparent texture image in this application, and the decoded image frame shown is a decoded image frame cropped based on height cropping factor and width cropping factor;

[0072] Figure 6 This is a schematic diagram illustrating how, in this application, color texture images and transparent texture images are cropped from a decoded image frame based on a height cropping factor and a width cropping factor, and how the color texture images and transparent texture images are merged to generate a transparent effect image frame.

[0073] Figure 7 This is a schematic diagram of the live streaming interface that outputs and displays transparent special effects image frames in this application;

[0074] Figure 8 A schematic diagram of the live stream interface for displaying the virtual gift selection control in this application;

[0075] Figure 9 This is a flowchart illustrating the specific implementation of the method for triggering the display command and obtaining the gift effect video in this application;

[0076] Figure 10 This is a schematic diagram of the luminance and chromaticity image frame from which grayscale components are extracted in this application;

[0077] Figure 11 This is a schematic diagram of the luminance and chrominance image frame from which chrominance components are extracted in this application;

[0078] Figure 12 This is a schematic flowchart illustrating the specific implementation method of converting a luminance-chrominance image frame into a decoded image frame with color texture and transparency texture in this application.

[0079] Figure 13 This is a flowchart illustrating a specific implementation method for detecting the image size and layout image size of a decoded image frame in this application to adjust the height and width cropping factors.

[0080] Figure 14 This is a flowchart illustrating a specific implementation method for cropping color texture images and transparent texture images from decoded image frames based on height and width cropping factors in this application.

[0081] Figure 15This is a flowchart illustrating the specific implementation method of merging color texture images and transparent texture images to generate transparent effect image frames for output display in this application.

[0082] Figure 16 This is a flowchart illustrating the specific implementation method of cropping other decoded image frames based on the first decoded image frame of a gift effect video in this application, by adjusting the height and width cropping factors.

[0083] Figure 17 This is a schematic block diagram of a typical embodiment of the transparent special effects playback device of this application;

[0084] Figure 18 This is a basic structural block diagram of a computer device according to an embodiment of this application. Detailed Implementation

[0085] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0086] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0087] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0088] Those skilled in the art will understand that the terms "client," "terminal," and "terminal device" as used herein include both devices that receive wireless signals, devices that only possess wireless signal receiver capabilities without transmission capabilities, and devices with receiving and transmitting hardware, devices that have receiving and transmitting hardware capable of bidirectional communication over a bidirectional communication link. Such devices may include: cellular or other communication devices such as personal computers or tablets, having single-line displays, multi-line displays, or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service) that can combine voice, data processing, fax, and / or data communication capabilities; PDA (Personal Digital Assistant) that may include a radio frequency receiver, pager, internet / intranet access, web browser, notepad, calendar, and / or GPS (Global Positioning System) receiver; and traditional laptops and / or handheld computers or other devices that have and / or include radio frequency receivers. As used herein, "client," "terminal," and "terminal device" can be portable, transportable, installed in a means of transportation (air, sea, and / or land), or suitable and / or configured to operate locally and / or in a distributed manner, operating in any other location on Earth and / or in space. "Client," "terminal," and "terminal device" as used herein can also be a communication terminal, an internet access terminal, or a music / video playback terminal, such as a PDA, a MID (Mobile Internet Device), and / or a mobile phone with music / video playback capabilities, or a smart TV, set-top box, etc.

[0089] The hardware referred to by the names "server," "client," and "work node" in this application is essentially an electronic device with the equivalent capabilities of a personal computer. It is a hardware device with the necessary components revealed by the von Neumann architecture, such as a central processing unit (including an arithmetic logic unit and a control unit), memory, input devices, and output devices. The computer program is stored in its memory, and the central processing unit loads the program stored in the secondary storage into the main memory to run it, executes the instructions in the program, and interacts with the input and output devices to complete specific functions.

[0090] It should be noted that the concept of "server" used in this application can also be extended to the case of server clusters. Based on the network deployment principles understood by those skilled in the art, the servers should be logically divided. Physically, these servers can be independent of each other but accessible through interfaces, or they can be integrated into a single physical computer or a computer cluster. Those skilled in the art should understand this flexibility and should not use it to constrain the implementation of the network deployment method in this application.

[0091] Please see Figure 1 The hardware infrastructure required for implementing the technical solutions of this application can be deployed according to the architecture shown in the figure. The server 80 mentioned in this application is deployed in the cloud and acts as an online server. It can further connect to relevant data servers and other servers providing related support, thereby forming a logically related service cluster to provide services to relevant terminal devices such as the smartphone 81 and personal computer 82 shown in the figure, or third-party servers (not shown). Both the smartphone and personal computer can access the Internet through known network access methods and establish a data communication link with the cloud server 80 to run terminal applications related to the services provided by the server.

[0092] For servers, the application is usually built as a service process, with corresponding program interfaces exposed for remote calls by applications running on various terminal devices. The relevant technical solutions in this application that are suitable for running on servers can be implemented in servers in this way.

[0093] The application mentioned refers to an application running on a server or terminal device. This application implements the relevant technical solutions of this application in a programmed manner. Its program code can be stored in a non-volatile storage medium that can be recognized by a computer in the form of computer-executable instructions, and is loaded into memory by the central processing unit for execution. The relevant device of this application is constructed by the operation of the application on the computer.

[0094] For servers, the application is usually built as a service process, with corresponding program interfaces exposed for remote calls by applications running on various terminal devices. The relevant technical solutions in this application that are suitable for running on servers can be implemented in servers in this way.

[0095] Those skilled in the art will understand that although the various methods in this application are described based on the same concept and thus present commonality among them, they can be performed independently unless otherwise specified. Similarly, the various embodiments disclosed in this application are all based on the same inventive concept; therefore, concepts expressed in the same way, as well as concepts that are appropriately changed for convenience but are expressed differently, should be understood equivalently.

[0096] Please see Figure 2 The present application discloses a method for playing transparent special effects, which, in a typical embodiment, includes the following steps:

[0097] Step S11: In response to the display command applied to the target gift effect in the live stream, the hardware decoding interface is called to decode the gift effect video, obtaining the decoded image frames contained in the gift effect video. Each decoded image frame contains a color texture image and a transparency texture image.

[0098] The display instructions are triggered by viewers in the live stream or pushed by the live stream platform's business server. If the user currently on the client is a viewer who is sending virtual gifts to the streamer in the live stream, the client determines the target gift effect corresponding to the event by responding to the gift selection and gifting event of the virtual gift control in the live stream interface, and generates a display instruction containing the gift identifier of the target gift effect. If the user currently on the client is a streamer or a viewer in the live stream, the display instructions are broadcast and pushed by the live stream platform's business server to the clients of each viewer and streamer.

[0099] The display instruction includes a gift identifier corresponding to the target gift effect, so that the client can query the gift effect video corresponding to the gift identifier of the target gift effect from the gift effect library. If the gift effect video exists in the gift effect library, the client will retrieve the target gift video from the gift effect library. If it does not exist, the client will push a video retrieval instruction containing the gift identifier to the server to retrieve the gift effect video pushed by the server.

[0100] The gift effect video contains multiple decoded image frames ordered along a timeline. The gift effect video is decoded via the hardware decoding interface to obtain each decoded image frame. Specifically, when not decoded, the gift effect video contains image frames encoded using RGBA or YUV color encoding. In RGBA color encoding, RGB refers to the three primary optical colors, which are the three indivisible basic colors: red (R), green (G), and blue (B). RGB can form any color. In image display, an image can be decomposed into countless pixels, which can be represented using RGB. In RGBA, A represents transparency (Alpha). In YUV color encoding, Y refers to the grayscale component or luminance, and UV refers to the chrominance component. Chromaticity defines the chroma and saturation of a color. YUV color encoding can be performed using 4:4:4, 4:2:2, or 4:2:0 sampling formats.

[0101] The decoded image frames are generally encoded using RGBA color encoding, and each decoded image frame contains a color texture image and a transparency texture image. If the gift effect video encodes each image frame using YUV color encoding, then each image frame is converted from YUV color encoding to RGBA color encoding format, so that each decoded image frame obtained from the gift effect video contains the color texture image and transparency texture image, for subsequent cropping and merging. Specifically, when the decoded image frame is a luminance and chrominance image frame encoded using YUV color encoding, then based on the original image size of the luminance and chrominance image frame, the luminance component and chrominance component contained in it are extracted from the luminance and chrominance image frame. According to the image conversion algorithm, based on the luminance component and chrominance component, the luminance and chrominance image frame is converted into the decoded image frame containing the color texture image and transparency texture image.

[0102] The aforementioned hardware decoding interface refers to the interface provided by the method for decoding the gift effect video based on device hardware (CPU or GPU). Correspondingly, the gift effect video is generally an MP4 format video. It can be understood that the hardware decoding generally refers to MP4 hardware decoding. Compared to gift animation effect technologies such as GIF, APNG, or Lottie, the gift effect video corresponding to MP4 hardware decoding has a higher compression rate, saving device storage space. Furthermore, decoding and playing gift effects using device hardware has higher processing efficiency. However, MP4 hardware decoding typically discards the alpha transparency channel in the gift effect video. Therefore, each decoded image frame in the gift effect video has the aforementioned color texture image and transparency texture image. The color texture image displays the RGB channels in the RGBA encoding information, and the transparency texture image displays the alpha channel in the RGBA encoding information. Subsequently, the color texture image and transparency texture image will be cropped and merged to generate the corresponding effect image frame for output to display the corresponding gift effect. For details, please refer to [link to documentation]. Figure 3 , Figure 3 The decoded image frame having the color texture image (RGB) and the transparency texture image (Alpha) without size padding.

[0103] Please refer to Figure 3 and Figure 4 When the MP4 hardware decodes the gift effect video, it will perform resolution alignment on each original image frame contained in the video at a resolution multiple of 16. This resolution alignment typically involves padding the original image frames so that the image size of the decoded image frame is padded from the original image size to the layout image size. Specifically, Figure 3 The original image frame shown is 2000*900 pixels in size. After resolution-aligned padding, the original image frame shown is... Figure 3 The original image frame shown will be converted to Figure 4 The decoded image frame shown is padded with 8 pixels in both width and height, and is 2008*908 pixels. MP4 hardware decoding typically halves the decoded image frame. Figure 4The decoded image frame shown will be cropped in half at point 401 (dashed line). It can be seen that the two images after cropping lose some of their color texture image (RGB) and transparency texture image (Alpha). This results in shadows and green edges appearing in the merged effect image frame, affecting the display of the gift effect in the live stream interface. To crop out the complete color texture image (RGB) and transparency texture image (Alpha) from the decoded image frame, the cropping factor will be adjusted to correct the cropping deviation and ensure the display effect of the gift effect. Subsequent steps will describe in detail the cropping factor and the specific implementation of how to crop the color texture image and transparency texture image from the decoded image frame; this step will not be elaborated upon here.

[0104] Step S12: Verify whether the original image size and the layout image size of the current decoded image frame are the same. If they are not the same, adjust the height clipping factor and the width clipping factor based on the size ratio between the original image size and the layout image size.

[0105] The currently decoded image frame refers to the decoded image frame that is currently being cropped and merged by the current thread.

[0106] Please refer to Figure 3 and Figure 4 The original image size refers to the total image size of the color texture image and the transparency texture image contained in the decoded image frame, such as... Figure 3 The image size of the image frame shown is the original image size. Correspondingly, the layout image size refers to the image size of the decoded image frame after being decoded and filled through the hardware decoding interface, such as... Figure 4 The image size of the decoded image frame is the size of the padded layout image.

[0107] The height clipping factor is a factor used to determine the height of the texture image regions of the color texture image and the transparent texture image in the decoded image frame. It adjusts the height of the texture image regions during the clipping process of the color texture image and the transparent texture image. Correspondingly, the width clipping factor is a factor used to determine the width of the texture image regions of the color texture image and the transparent texture image in the decoded image frame. It adjusts the width of the texture image regions during the clipping process of the color texture image and the transparent texture image. By adjusting the height clipping factor and the width clipping factor, the width and height of the texture image regions of the color texture image and the transparent texture image in the decoded image frame are accurately determined, thereby cropping the complete color texture image and transparent texture image from the decoded image frame.

[0108] Regarding the verification of whether the original image size and the layout image size of the currently decoded image frame are the same, specifically, the image height and image width of both the original image size and the layout image size will be verified to be the same. For the image height verification, if they are the same, the height clipping factor will be adjusted based on a preset value, which is generally set to 1. If they are not the same, the height clipping factor will be adjusted based on the height ratio of the two image heights, where the image height of the original image size will be used as the numerator, and the image height of the layout image size will be used as the denominator. For the image width verification, if they are the same, the width clipping factor will be adjusted based on a preset value, which is generally set to 1. If they are not the same, the width clipping factor will be adjusted based on the width ratio of the two image widths, where the image width of the original image size will be used as the numerator, and the image width of the layout image size will be used as the denominator.

[0109] Step S13: Based on the height clipping factor and the width clipping factor, determine the respective texture image regions of the color texture image and the transparent texture image contained in the current decoded image frame, so as to clip the color texture image and the transparent texture image from the current decoded image frame.

[0110] After adjusting the height and width clipping factors by verifying the original image size and layout image size of the current decoded image frame, the texture image regions of the color texture image and the transparent texture image in the current decoded image frame are determined based on the height and width clipping factors.

[0111] When determining the texture image region of the color texture image in the current decoded image frame based on the height and width clipping factors, a color texture algorithm is invoked. Based on the texture coordinates of the current decoded image frame, the height and width clipping factors, the color texture image region of the color texture image is determined from the current decoded image frame. The texture coordinates are generally determined based on the layout image size of the current decoded image frame, and are generally divided into horizontal and vertical texture coordinates. The values ​​of the horizontal and vertical texture coordinates range from [0,1]. Figure 5 As shown, Figure 5 The texture coordinates of the decoded image frame shown will be determined based on the shown planar coordinate axis 501; the algorithm expression of the color texture algorithm is as follows:

[0112] (texCord.x*factorX*0.5,texCord.y*factorY)

[0113] Wherein, “texCord.x” is the horizontal texture coordinate, “texCord.y” is the vertical texture coordinate, “factorX” is the width clipping factor, and “factorY” is the height clipping factor.

[0114] Accordingly, when determining the texture image region of the transparent texture image in the current decoded image frame based on the height clipping factor and the width clipping factor, the transparent texture algorithm will be invoked to determine the transparent texture image region of the transparent texture image from the current decoded image frame based on the texture coordinates of the current decoded image frame, the height clipping factor, and the width clipping factor; the algorithm expression of the transparent texture algorithm is as follows:

[0115] (texCord.x*factorX*0.5+factorX*0.5,texCord.y*factorY)

[0116] The texture image regions of the color texture image and the transparency texture image contained in the current decoded image frame are determined, and the color texture image and the transparency texture image are cropped from their respective texture image regions. Please refer to [reference needed]. Figure 5 and Figure 6 ,from Figure 5 The texture image region 502 of the color texture image shown is cropped out as follows: Figure 6 The color texture image 601 shown corresponds to the color texture image 601 shown. Figure 5 The transparent texture image shown is cropped from the texture image region 503. Figure 6 The transparent texture image 602 shown; Figures 3 to 5 and Figure 6 The color texture image shown differs from the transparent texture image in display because... Figures 3 to 5 The color and transparency texture images shown are schematic diagrams rather than actual rendered images, in order to improve... Figures 3 to 5 Its visibility makes it easy to understand, and Figure 6 The color texture image and transparency texture image shown are actual rendered images, for use in... Figure 7 The text describes how gift effects are displayed on the live stream interface.

[0117] Step S14: Merge the color texture image and the transparent texture image to generate the current transparent effect image frame of the target gift effect, and output and display the current transparent effect frame image to the live broadcast interface.

[0118] After acquiring the color texture image and the transparent texture image, they are merged to generate a transparent effect image frame for the target gift effect. The display position of the target gift effect within the current live stream interface is determined, and the image size of the transparent effect image frame is adjusted according to the current live stream interface size. The transparent effect image frame is then output to the gift display position for display. Specifically, as follows... Figure 7 As shown, Figure 7 The gift effect 701 shown is... Figure 6 The display style of the transparent effect image frame generated by merging the color texture image 601 and the transparent texture image 602 in the live broadcast interface.

[0119] When merging the color texture image and the transparent texture image, the RGBA data of the color texture image and the transparent texture image are merged to generate a corresponding transparent effect image frame. For example, when the RGBA data of the color texture image is (Ra, Ga, Ba, Alphaa) and the RGBA data of the transparent texture image is (Rb, Gb, Bb, Alphab), the merging expression of the transparent effect image frame generated by merging the two is as follows:

[0120] Rc = Ra * Alphaa + Rb * Alphab

[0121] Gc = Ga * Alphaa + Gb * Alphab

[0122] Bc = Ba * Alphaa + Bb * Alphab

[0123] The resulting transparent effect image frame will consist of triplet data and RGB data, with the RGB data of the transparent effect image frame being (Rc, Gc, Bc).

[0124] Similarly, the client will generate transparent effect image frames in sequence according to the timeline of the gift image video to display each decoded image frame contained in the gift effect video, so that the target gift effect of the virtual gift given by the viewer to the streamer is fully displayed in the live broadcast interface.

[0125] As can be seen from the typical implementation of this method, this method is based on hardware decoding video processing to display gift effects in the live broadcast room. It obtains decoded image frames by hardware decoding the gift effect video of virtual gifts, and then crops them. According to the timeline of the gift image video, the color texture images and transparent texture images contained in the decoded image frames are cropped in sequence and merged to generate transparent effect image frames for display. The effective display of each transparent effect image frame in the live broadcast room interface presents the transparent gift effect of virtual gifts. Compared with traditional gift animation effect technologies such as GIF, Apng, or Lottie, the gift effect video based on hardware decoding video processing has a higher compression rate, which can save device storage space. Moreover, the decoding and playback of gift effects by device hardware has higher processing efficiency, improves the running stability of the live broadcast application, and provides fast and smooth playback of gift effects for users in the live broadcast room to watch, effectively enhancing the live broadcast atmosphere.

[0126] Secondly, this method dynamically adjusts the cropping factor to control the texture image cropping of the decoded image frame, solving the cropping deviation caused by the hardware decoding process aligning the resolution of each image frame in the gift effect video at a resolution multiple of 16. The cropping factor is dynamically adjusted according to the size ratio of the original image size and the layout image size after filling of the decoded image frame, controlling the texture area that needs to be cropped in the decoded image frame, so as to crop out the complete color texture image and transparent texture image from the decoded image frame, so that the final gift effect does not have green edges or black shadow residues, ensuring the display effect of the gift effect in the live broadcast interface.

[0127] The above typical embodiments and their variations fully disclose the implementation scheme of the transparent special effects playback method of this application. However, various variations of the method can still be derived by changing and expanding some technical means. Other embodiments are briefly described below:

[0128] In one embodiment, please refer to Figure 8 and 9 The step of calling the hardware decoding interface to decode the gift effect video and obtain the decoded image frames contained in the gift effect video includes the following steps performed by the client:

[0129] Step S111: In response to the gift selection and gifting event acting on the virtual gift control in the live stream interface, determine the target gift effect corresponding to the event, and generate a display instruction containing the gift icon of the target gift effect.

[0130] Please refer to Figure 8 , Figure 8The virtual gift control 801 shown displays multiple virtual gifts. The virtual gift control 801 is displayed in the live broadcast room interface. Users in the live broadcast room can select virtual gifts to send to the host user in the live broadcast room through the virtual gift control 801. For example, when a user selects gift B through the virtual gift control 801, the gift selection and gifting event will be triggered for gift B. The client will respond to the virtual gift selection and gifting event and generate the display instruction containing the gift identifier of the target gift effect of gift B.

[0131] Step S112: Query the gift effects library to see if a target gift effect video corresponding to the target gift identifier exists. If it exists, retrieve the target gift video from the gift effects library; otherwise, retrieve the target gift effect video from the server.

[0132] After the client responds to the display command, it will query the gift effects library to see if there is a target gift effect video corresponding to the target gift identifier contained in the display command. The gift effects library stores multiple mapping relationship data consisting of gift identifiers and gift effect videos.

[0133] If a target gift effect video corresponding to the target gift identifier can be found in the gift effect library, then the target gift effect video is obtained. If the target gift effect video does not exist in the gift effect library, the client will push a gift effect retrieval instruction containing the target gift identifier to the server, so that the server can respond to the gift effect retrieval instruction, retrieve the target gift effect video corresponding to the target gift identifier from the constantly updated remote gift effect library, and push it to the client.

[0134] Step S113: Call the hardware decoding interface to decode the target gift effect video, obtain multiple decoded image frames contained in the target gift effect video, and sort the decoded image frames according to the timeline of the target gift effect video:

[0135] After obtaining the target gift effect video, the client will call the hardware interface to decode the target gift effect video to obtain multiple decoded image frames contained in the target gift effect video. The client will sort the decoded image frames according to the timeline of the target gift effect video so that they can be cropped and merged in sequence to generate transparent effect image frames corresponding to each decoded image frame for output.

[0136] In this embodiment, users can interact with the streamer through a virtual gift gifting service. The client locally manages and stores the gift effect videos of each virtual gift. For non-existent gift effect videos, they can be obtained from a remote server for updating. Furthermore, the gift effect videos are decoded through a hardware decoding interface to obtain each decoded image frame for cropping and merging.

[0137] In one embodiment, please refer to Figures 10 to 12 The step of calling the hardware decoding interface to decode the gift effect video and obtain the decoded image frames contained in the gift effect video includes the following steps performed by the client:

[0138] Step S111': Call the hardware decoding interface to decode the gift effect video and obtain the decoded image frames:

[0139] After acquiring the gift effect video, the hardware decoding interface will be called to decode the gift effect video and obtain the decoded image frames contained in the gift effect video.

[0140] Step S112': When the decoded image frame is a luminance / chrominance image frame, then based on the original image size of the luminance / chrominance image frame, extract the luminance component and chrominance component contained therein from the luminance / chrominance image frame:

[0141] The aforementioned luminance and chrominance image refers to an image based on YUV color encoding. In YUV color encoding, Y refers to the grayscale component or luminance, and UV refers to the chrominance component. Chromaticity defines the chromaticity and saturation of a color, and the YUV color encoding can be performed using a 4:4:4, 4:2:2, or 4:2:0 sampling format.

[0142] For information on extracting the luminance and chrominance components from the luminance and chrominance image frame, please refer to [link / reference needed]. Figure 10 and Figure 11 , Figure 10 and Figure 11 The corresponding luminance and chrominance image sampling format is 4:2:0. Figure 10 The image size of the luminance and chrominance image frame 1001 shown is the original image size. Figure 11 The image size of the luminance and chrominance image frame 1101 shown is one-quarter of the original image size, so that... Figure 10 For example, each pixel of the luminance / chrominance image frame 1001 will be scanned line by line. Taking the scanning window 1002 as an example, the luminance / chrominance image frame 1001 will be scanned and extracted from top to bottom line by line to extract the luminance component in the luminance / chrominance image frame 1001. Correspondingly, the luminance component will also be extracted. Figure 11The luminance and chrominance image frame 1101 shown is scanned line by line to extract the chrominance components contained in the luminance and chrominance image frame 1101 shown.

[0143] Step S113': According to the image conversion algorithm, based on the luminance component and chrominance component, the luminance and chrominance image frame is converted into a decoded image frame containing a color texture image and a transparency texture image.

[0144] After extracting the luminance and chrominance components contained in the luminance and chrominance image, the luminance and chrominance image frame is converted into a decoded image frame containing a color texture image and a transparency texture image based on the luminance and chrominance components according to the image conversion algorithm.

[0145] The algorithmic expression for the image conversion algorithm is as follows:

[0146]

[0147] Wherein, Y is the luminance component, and UV is the chrominance component. The decoded image frame containing the color texture image and the transparent texture image is generally encoded using RGBA color encoding. RGB refers to the three primary optical colors, which are the three basic colors that cannot be further decomposed in color. The three primary optical colors are red (R), green (G), and blue (B), which are R, G, and B in the algorithm expression shown.

[0148] In this embodiment, by converting the YUV color-encoded image frames in the gift effect video into RGB-encoded image frames, the subsequent adaptive cropping and merging to generate effect image frames can be compatible with YUV color-encoded image frames, thus improving the versatility of the adaptive cropping and merging function.

[0149] In one embodiment, please refer to the figure. Figure 3 , Figure 4 and Figure 13 The step of verifying whether the original image size and the layout image size of the current decoded image frame are the same, and if they are not the same, adjusting the height clipping factor and the width clipping factor based on the size ratio between the original image size and the layout image size, includes the following steps performed by the client:

[0150] Step S121: Obtain the original image size and layout image size of the current decoded image frame, wherein the original image size includes the original image width and the original image height, and the layout image size includes the layout image width and the layout image height.

[0151] Please refer to Figure 3 and Figure 4The original image size refers to the total image size of the color texture image and the transparency texture image contained in the decoded image frame, such as... Figure 3 The image size of the image frame shown is the original image size. Correspondingly, the layout image size refers to the image size of the decoded image frame after being decoded and filled through the hardware decoding interface, such as... Figure 4 The image size of the decoded image frame is the size of the padded layout image.

[0152] Step S122: Determine whether the height of the original image is equal to the height of the layout image. If they are not equal, adjust the height clipping factor based on the height ratio of the original image height to the layout image height. If they are equal, adjust the height clipping factor based on a preset value.

[0153] The height of the original image is checked to see if it is equal to the height of the layout image. The height cropping factor is then adjusted accordingly. If they are equal, the height cropping factor is adjusted based on a preset value, such as the preset value, which is typically set to 1. If the height of the original image is not equal to the height of the layout image, the height cropping factor is adjusted based on the height ratio between the original image height and the layout image height, where the original image height is the numerator and the layout image height is the denominator, and the ratio between the two is used as the height cropping factor.

[0154] Step S123: Determine whether the width of the original image is equal to the width of the layout image. If they are not equal, adjust the width cropping factor based on the width ratio of the original image width to the layout image width. If they are equal, adjust the width cropping factor based on a preset value.

[0155] The original image width and the layout image width are checked to determine if they are equal. If they are equal, the width cropping factor is adjusted based on a preset value, such as the preset value, which is typically set to 1. If the original image width and the layout image width are not equal, the width cropping factor is adjusted based on the width ratio between the original image width and the layout image width, where the original image width is the numerator and the layout image width is the denominator, and the ratio between them is used as the width cropping factor.

[0156] In this embodiment, the cropping factor is adjusted based on the original image size and the filled layout image size of the decoded image frame. The cropping factor is adjusted in two dimensions: height and width. When cropping the decoded image frame according to the cropping factor, the complete color texture image and transparent texture image can be cropped from the decoded image frame, thereby ensuring the display effect of the gift effect.

[0157] In one embodiment, please refer to Figure 5 , Figure 6 and Figure 14 The step of determining the respective texture image regions of the color texture image and the transparency texture image contained in the current decoded image frame based on the height clipping factor and the width clipping factor, so as to clip the color texture image and the transparency texture image from the current decoded image frame, includes the following steps performed by the client:

[0158] Step S131: Invoke the color texture algorithm to determine the color texture image region from the current decoded image frame based on the texture coordinates of the current decoded image frame, the height clipping factor, and the width clipping factor.

[0159] The algorithm expression for the color texture algorithm is as follows:

[0160] (texCord.x*factorX*0.5,texCord.y*factorY)

[0161] Wherein, “texCord.x” is the horizontal texture coordinate, “texCord.y” is the vertical texture coordinate, “factorX” is the width clipping factor, and “factorY” is the height clipping factor.

[0162] Step S132: Invoke the transparent texture algorithm to determine the transparent texture image region from the current decoded image frame based on the texture coordinates of the current decoded image frame, the height clipping factor, and the width clipping factor.

[0163] The algorithm expression for the transparent texture algorithm is as follows:

[0164] (texCord.x*factorX*0.5+factorX*0.5,texCord.y*factorY)

[0165] Step S133: Cropping the color texture image from the color texture image region of the current decoded image frame, and cropping the transparent texture image from the transparent texture image region of the current decoded image frame:

[0166] Please refer to Figure 5 and Figure 6The textural image regions of the color texture image and the transparency texture image contained in the current decoded image frame are determined, and the color texture image and the transparency texture image are cropped from their respective texture image regions. For example, from... Figure 5 The texture image region 502 of the color texture image shown is cropped out as follows: Figure 6 The color texture image 601 shown corresponds to the color texture image 601 shown. Figure 5 The transparent texture image shown is cropped from the texture image region 503. Figure 6 The transparent texture image 602 shown.

[0167] In this embodiment, the decoded image frame is cropped based on the cropping factor adjusted according to the original size and layout size of the decoded image frame, so as to crop out the complete color texture image and transparent texture image from the decoded image frame, ensuring that the generated special effects image frame has the color texture image and transparent texture image without loss.

[0168] In one embodiment, please refer to Figure 6 , Figure 7 and Figure 15 The step of merging the color texture image and the transparent texture image to generate the current transparent effect image frame of the target gift effect, and outputting and displaying the current transparent effect frame image to the live broadcast interface, includes the following steps performed by the client:

[0169] Step S141: Merge the color texture image and the transparent texture image to generate a transparent effect image frame for the target gift effect:

[0170] Please refer to Figure 6 ,Will Figure 6 The color texture image 601 shown is merged with the transparent texture image 602 shown to generate the transparent effect image frame 603 shown.

[0171] Step S142: Determine the display position of the target gift effect in the current live stream interface:

[0172] The display position of gift effects in the live stream interface is the same as the display position of the transparent effect image frame in the live stream interface.

[0173] Step S143: Adjust the image size of the transparent effect image frame according to the current interface size of the live stream, and output the transparent effect image frame to the gift display position for display.

[0174] Before the transparent special effects image frame is output to the gift display position, the image size of the transparent special effects image frame will be adjusted proportionally according to the current interface size of the live broadcast room to prevent the transparent special effects image frame from obscuring too much interface area and affecting the user's interface operation.

[0175] Please refer to Figure 6 and Figure 7 , Figure 7 The transparent effect image frame 701 displayed in the live stream interface is... Figure 6 The transparent effect image frame 603 shown.

[0176] In this embodiment, a transparent effect image frame for gift effects is generated by merging color texture images and transparent texture images. The size of the transparent effect image frame is adjusted based on the size of the live broadcast interface to adapt to the user's operation, and then output to the gift display position for display of gift effects.

[0177] In one embodiment, please refer to Figure 16 After the steps of merging the color texture image and the transparent texture image to generate the current transparent effect image frame of the target gift effect, and outputting and displaying the current transparent effect frame image to the live broadcast interface, the following steps are performed by the client:

[0178] Step S15: Obtain the next decoded image frame contained in the gift effect video. Based on the height and width cropping factors adjusted according to the first decoded image frame, crop out the color texture image and transparency texture image contained in the next decoded image frame:

[0179] Since the original image size of each decoded image frame in the gift effect video is generally the same, after adjusting the height and width cropping factors according to the first decoded image frame ranked first in the gift effect video, when cropping the next decoded image frame and subsequent decoded image frames of the first decoded image frame, the color texture image and transparent texture image can be cropped based on the height and width cropping factors.

[0180] Step S16: Merge the color texture image and the transparent texture image to generate the next transparent effect image frame of the target gift effect. After the transparent effect image frame corresponding to the first decoded image frame is displayed, display the next transparent effect image frame:

[0181] After cropping the color texture image and transparent texture image contained in the next decoded image frame based on the height cropping factor and width cropping factor adjusted according to the first decoded image frame, the color texture image and transparent texture image are merged to generate the next transparent effect image frame, so that the next transparent effect image frame is displayed after the transparent effect image frame corresponding to the first decoded image frame of the gift effect video is displayed.

[0182] Step S17: Generate and display the transparent effect image frames of each decoded image frame contained in the gift effect video in sequence.

[0183] Based on the timeline of the gift effect video, transparent effect image frames corresponding to each decoded image frame are generated sequentially, and these transparent effect image frames are output sequentially to be displayed on the live broadcast interface, so as to fully output the gift effect corresponding to the gift effect video to be displayed on the live broadcast interface.

[0184] In this embodiment, for gift effect videos with all decoded image frames having the same image size, after adjusting the cropping factor based on the first decoded image frame, the same cropping factor can be used to crop other decoded image frames to improve the generation efficiency of transparent effect image frames and quickly output the gift effect to the live broadcast interface for display.

[0185] Furthermore, by functionalizing the various steps in the methods disclosed in the above embodiments, a transparent special effects playback device of this application can be constructed. Following this approach, please refer to... Figure 17 In one typical embodiment, the device includes: a decoded image acquisition module 11, used to respond to a display command applied to a target gift effect in the live broadcast room, call a hardware decoding interface to decode the gift effect video, and obtain decoded image frames contained in the gift effect video, each decoded image frame containing a color texture image and a transparent texture image; an image size verification module 12, used to verify whether the original image size and the layout image size of the current decoded image frame are the same, and if they are not the same, adjust the height clipping factor and the width clipping factor based on the size ratio between the original image size and the layout image size; a texture image clipping module 13, used to determine the texture image regions of the color texture image and the transparent texture image contained in the current decoded image frame according to the height clipping factor and the width clipping factor, so as to clip the color texture image and the transparent texture image from the current decoded image frame; and an effect image display module 14, used to merge the color texture image and the transparent texture image to generate the current transparent effect image frame of the target gift effect, and output and display the current transparent effect image frame to the live broadcast room interface.

[0186] In one embodiment, the decoded image acquisition module 11 includes: a gift identifier determination submodule, used to respond to a gift selection and gifting event acting on a virtual gift control in the live streaming room interface, determine the target gift effect corresponding to the event, and generate a display instruction containing a gift identifier of the target gift effect; an effect video acquisition submodule, used to query whether there is a target gift effect video corresponding to the target gift identifier in the gift effect library, and if it exists, obtain the target gift video from the gift effect library, and if it does not exist, obtain the target gift effect video from the server; and an image frame decoding submodule, used to call the hardware decoding interface to decode the target gift effect video, obtain multiple decoded image frames contained in the target gift effect video, and sort the decoded image frames according to the timeline of the target gift effect video.

[0187] In another embodiment, the decoded image acquisition module 12 further includes: an image frame acquisition submodule, used to call the hardware decoding interface to decode the gift effect video and acquire a decoded image frame; a component extraction submodule, used to extract the luminance component and chrominance component contained in the luminance and chrominance image frame based on the original image size of the luminance and chrominance image frame when the decoded image frame is a luminance and chrominance image frame; and an image frame conversion submodule, used to convert the luminance and chrominance image frame into a decoded image frame containing a color texture image and a transparency texture image based on the luminance component and chrominance component according to an image conversion algorithm.

[0188] In one embodiment, the texture image cropping module 12 includes: an image size acquisition submodule, used to acquire the original image size and layout image size of the current decoded image frame, wherein the original image size includes the original image width and the original image height, and the layout image size includes the layout image width and the layout image height; a height factor adjustment submodule, used to determine whether the original image height is equal to the layout image height, and if not, adjust the height cropping factor based on the height ratio of the original image height to the layout image height, and if they are equal, adjust the height cropping factor based on a preset value; and a width factor adjustment submodule, used to determine whether the original image width is equal to the layout image width, and if not, adjust the width cropping factor based on the width ratio of the original image width to the layout image width, and if they are equal, adjust the width cropping factor based on a preset value.

[0189] In one embodiment, the texture image cropping module 13 includes: a color region determination submodule, used to call a color texture algorithm to determine a color texture image region of a color texture image from the current decoded image frame based on the texture coordinates of the current decoded image frame, the height cropping factor, and the width cropping factor; a transparent region determination submodule, used to call a transparent texture algorithm to determine a transparent texture image region of a transparent texture image from the current decoded image frame based on the texture coordinates of the current decoded image frame, the height cropping factor, and the width cropping factor; and a texture image cropping submodule, used to crop the color texture image from the color texture image region of the current decoded image frame and to crop the transparent texture image from the transparent texture image region of the current decoded image frame.

[0190] In one embodiment, the special effects image display module 14 includes: a texture image merging submodule, used to merge the color texture image and the transparent texture image to generate a transparent special effects image frame of the target gift special effects; a display position determination submodule, used to determine the gift display position of the target gift special effects in the current live broadcast interface; and a transparent image output submodule, used to adjust the image size of the transparent special effects image frame according to the interface size of the current live broadcast interface, and output the transparent special effects image frame to the gift display position for display.

[0191] To address the aforementioned technical problems, this application also provides a computer device for running a computer program implemented according to the transparency effect playback method. Please refer to the following for details. Figure 18 , Figure 18 This is a basic structural block diagram of the computer device in this embodiment.

[0192] like Figure 18 The diagram shows the internal structure of a computer device. This computer device includes a processor, non-volatile storage medium, memory, and a network interface connected via a system bus. The non-volatile storage medium stores the operating system, a database, and computer-readable instructions. The database may store control information sequences. When the computer-readable instructions are executed by the processor, they enable the processor to implement a transparent special effects playback method. The processor provides computing and control capabilities, supporting the operation of the entire computer device. The memory stores computer-readable instructions, which, when executed by the processor, enable the processor to implement a transparent special effects playback method. The network interface of the computer device is used for communication with a terminal. Those skilled in the art will understand that… Figure 10The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0193] In this embodiment, the processor executes the specific functions of each module / submodule in the transparent effects playback device of this application, and the memory stores the program code and various types of data required to execute the above modules. The network interface is used for data transmission between the user terminal and the server. In this embodiment, the memory stores the program code and data required to execute all modules / submodules in the transparent effects playback device, and the server can call the server's program code and data to execute the functions of all submodules.

[0194] This application also provides a non-volatile storage medium in which the transparent special effects playback method is written as a computer program and stored in the storage medium in the form of computer-readable instructions. When the computer-readable instructions are executed by one or more processors, it means that the program is running in the computer, thereby causing one or more processors to perform the steps of the transparent special effects playback method of any of the above embodiments.

[0195] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0196] In summary, this application effectively ensures the display effect of transparent gift effects in the interface, enhancing the live streaming experience.

[0197] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0198] Those skilled in the art will understand that the steps, measures, and solutions in the various operations, methods, and processes discussed in this application can be alternated, modified, combined, or deleted. Furthermore, other steps, measures, and solutions in the various operations, methods, and processes discussed in this application can also be alternated, modified, rearranged, decomposed, combined, or deleted. Furthermore, steps, measures, and solutions in the prior art that are similar to those disclosed in this application can also be alternated, modified, rearranged, decomposed, combined, or deleted.

[0199] The above description is only a partial embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for playing transparent special effects, characterized in that, Includes the following steps: In response to a display command applied to a target gift effect in the live stream, the system calls a hardware decoding interface to decode the gift effect video, obtaining decoded image frames contained in the gift effect video. Each decoded image frame contains a color texture image and a transparency texture image. This includes: calling the hardware decoding interface to decode the gift effect video to obtain decoded image frames; when the decoded image frame is a luminance / chrominance image frame encoded in YUV format, extracting the luminance and chrominance components contained in the luminance / chrominance image frame based on its original image size; and converting the luminance / chrominance image frame into a decoded image frame encoded in RGB format containing a color texture image and a transparency texture image based on the luminance and chrominance components according to an image conversion algorithm. Verify whether the original image size and the layout image size of the current decoded image frame are the same. If they are not the same, adjust the height clipping factor and the width clipping factor based on the size ratio between the original image size and the layout image size. The original image size includes the original image width and the original image height, and the layout image size includes the layout image width and the layout image height. The layout image size refers to the image size of the decoded image frame after being decoded and filled through the hardware decoding interface. Based on the height and width clipping factors, the texture image regions of the color texture image and the transparent texture image contained in the current decoded image frame are determined, so as to crop the color texture image and the transparent texture image from the current decoded image frame. This includes: calling a color texture algorithm to determine the color texture image region of the color texture image from the current decoded image frame based on the texture coordinates of the current decoded image frame, the height clipping factor, and the width clipping factor; calling a transparent texture algorithm to determine the transparent texture image region of the transparent texture image from the current decoded image frame based on the texture coordinates of the current decoded image frame, the height clipping factor, and the width clipping factor; cropping the color texture image from the color texture image region of the current decoded image frame; and cropping the transparent texture image from the transparent texture image region of the current decoded image frame. The color texture image and the transparent texture image are merged to generate the current transparent effect image frame of the target gift effect, and the current transparent effect frame image is output and displayed on the live broadcast interface.

2. The method according to claim 1, characterized in that, The step of calling the hardware decoding interface to decode the gift effect video and obtain the decoded image frames contained in the gift effect video includes the following steps performed by the client: Responding to the gift selection and gifting event of the virtual gift control in the live broadcast room interface, determining the target gift effect corresponding to the event, and generating a display instruction containing the gift icon of the target gift effect; Check if there is a target gift effect video corresponding to the target gift identifier in the gift effect library. If it exists, retrieve the target gift video from the gift effect library. If it does not exist, retrieve the target gift effect video from the server. The hardware decoding interface is invoked to decode the target gift effect video, obtain multiple decoded image frames contained in the target gift effect video, and sort the decoded image frames according to the timeline of the target gift effect video.

3. The method according to claim 1, characterized in that, The step of verifying whether the original image size and the layout image size of the current decoded image frame are the same, and if they are not the same, adjusting the height clipping factor and the width clipping factor based on the size ratio between the original image size and the layout image size, includes the following steps performed by the client: Obtain the original image size and layout image size of the current decoded image frame; Determine whether the height of the original image is equal to the height of the layout image. If they are not equal, adjust the height clipping factor based on the height ratio of the original image height to the layout image height. If they are equal, adjust the height clipping factor based on a preset value. Determine whether the width of the original image is equal to the width of the layout image. If they are not equal, adjust the width cropping factor based on the width ratio of the original image width to the layout image width. If they are equal, adjust the width cropping factor based on a preset value.

4. The method according to claim 1, characterized in that, The step of merging the color texture image and the transparent texture image to generate the current transparent effect image frame of the target gift effect, and outputting and displaying the current transparent effect frame image to the live broadcast interface, includes the following steps performed by the client: The color texture image and the transparent texture image are merged to generate a transparent effect image frame for the target gift effect; Determine the display position of the target gift effect in the current live stream interface; Adjust the image size of the transparent effect image frame according to the current interface size of the live broadcast room, and output the transparent effect image frame to the gift display position for display.

5. The method according to any one of claims 1 to 4, characterized in that, After the steps of merging the color texture image and the transparent texture image to generate the current transparent effect image frame of the target gift effect, and outputting and displaying the current transparent effect frame image to the live broadcast interface, the following steps are performed by the client: Obtain the next decoded image frame contained in the gift effect video, and based on the height cropping factor and width cropping factor adjusted according to the first decoded image frame, crop out the color texture image and transparent texture image contained in the next decoded image frame; The color texture image and the transparent texture image are merged to generate the next transparent effect image frame of the target gift effect. After the transparent effect image frame corresponding to the first decoded image frame is displayed, the next transparent effect image frame is displayed. Transparent effect image frames of each decoded image frame contained in the gift effect video are generated in sequence and displayed.

6. A transparent special effects playback device, characterized in that, include: The image acquisition module is used to respond to the display command applied to the target gift effect in the live broadcast room, call the hardware decoding interface to decode the gift effect video, and obtain the decoded image frames contained in the gift effect video. Each decoded image frame contains a color texture image and a transparency texture image. The module includes: calling the hardware decoding interface to decode the gift effect video to obtain the decoded image frames; when the decoded image frame is a luminance and chrominance image frame encoded in YUV format, extracting the luminance component and chrominance component contained in the luminance and chrominance image frame based on the original image size of the luminance and chrominance image frame; and converting the luminance and chrominance image frame into a decoded image frame encoded in RGB format containing the color texture image and the transparency texture image according to the image conversion algorithm. The image size verification module is used to verify whether the original image size and the layout image size of the current decoded image frame are the same. If they are not the same, the height clipping factor and the width clipping factor are adjusted based on the size ratio between the original image size and the layout image size. The original image size includes the original image width and the original image height, and the layout image size includes the layout image width and the layout image height. The layout image size refers to the image size of the decoded image frame after being decoded and filled through the hardware decoding interface. The texture image cropping module is used to determine the respective texture image regions of the color texture image and the transparent texture image contained in the current decoded image frame according to the height cropping factor and the width cropping factor, so as to crop the color texture image and the transparent texture image from the current decoded image frame. This includes: calling a color texture algorithm to determine the color texture image region of the color texture image from the current decoded image frame according to the texture coordinates of the current decoded image frame, the height cropping factor, and the width cropping factor; calling a transparent texture algorithm to determine the transparent texture image region of the transparent texture image from the current decoded image frame according to the texture coordinates of the current decoded image frame, the height cropping factor, and the width cropping factor; cropping the color texture image from the color texture image region of the current decoded image frame; and cropping the transparent texture image from the transparent texture image region of the current decoded image frame. The special effects image display module is used to merge the color texture image and the transparent texture image to generate the current transparent special effects image frame of the target gift special effects, and output the current transparent special effects frame image to the live broadcast interface.

7. The transparent special effects playback device according to claim 6, characterized in that, The decoded image acquisition module further includes: The Gift Identifier Determination Submodule is used to respond to the gift selection and gifting event of the virtual gift control in the live broadcast room interface, determine the target gift effect corresponding to the event, and generate a display instruction containing the gift identifier of the target gift effect. The special effects video acquisition submodule is used to query whether there is a target gift special effects video corresponding to the target gift identifier in the gift special effects library. If it exists, the target gift video is retrieved from the gift special effects library; if it does not exist, the target gift special effects video is retrieved from the server. The image frame decoding submodule is used to call the hardware decoding interface to decode the target gift effect video, obtain multiple decoded image frames contained in the target gift effect video, and sort the decoded image frames according to the timeline of the target gift effect video.

8. The transparent special effects playback device according to claim 6 or 7, characterized in that, The texture image cropping module includes: The image size acquisition submodule is used to obtain the original image size and layout image size of the current decoded image frame; The height factor adjustment submodule is used to determine whether the height of the original image is equal to the height of the layout image. If they are not equal, the height clipping factor is adjusted based on the height ratio of the original image height to the layout image height. If they are equal, the height clipping factor is adjusted based on a preset value. The width factor adjustment submodule is used to determine whether the width of the original image is equal to the width of the layout image. If they are not equal, the width cropping factor is adjusted based on the width ratio of the original image width to the layout image width. If they are equal, the width cropping factor is adjusted based on a preset value.

9. An electronic device comprising a central processing unit and a memory, characterized in that, The central processing unit is used to invoke and run a computer program stored in the memory to perform the steps of the method as described in any one of claims 1 to 5.

10. A non-volatile storage medium, characterized in that, It stores, in the form of computer-readable instructions, a computer program implemented according to any one of claims 1 to 5, which, when invoked by a computer, performs the steps included in the method.

Images