Virtual interactive gift generation method and device, computer device, and storage medium

Abstract

The present disclosure provides a virtual interactive gift generation method and device, computer equipment and storage medium, belonging to the technical field of network. The virtual interactive gift generation method provided by the present disclosure only needs to automatically obtain a gift material with a transparent background by an intelligent agent, and the intelligent agent automatically calculates the rendering anchor followed by the material according to the material type of the material subject in the generated gift material, thereby generating a virtual interactive gift. In the generation process, no manual intervention is required, only a brief expectation of the appearance of the virtual interactive gift is needed, and the virtual interactive gift capable of interacting with the object can be automatically generated, which greatly reduces the cost and the production time of the gift.

Description

Technical Field

[0001] This disclosure relates to the field of network technology, and in particular to a method, apparatus, computer device, and storage medium for generating virtual interactive gifts. Background Technology

[0002] With the development of internet technology, major online platforms have begun to provide users with virtual interactive gifts. These virtual interactive gifts can change according to the recipient's actions after being given. For example, if a streamer is given a pair of glasses, the glasses will move with the streamer's head when the streamer moves, showing a wearing effect. However, these virtual interactive gifts require designers to create 3D models of the gift materials in advance and then perform animation rendering based on the models. This process is time-consuming and costly. Therefore, there is an urgent need for a low-cost method to generate virtual interactive gifts that saves on pre-production time. Summary of the Invention

[0003] This disclosure provides a method, apparatus, computer device, and storage medium for generating virtual interactive gifts, which reduces costs and shortens the production time of virtual interactive gifts. The technical solution of this disclosure is as follows: According to one aspect of the present disclosure, a method for generating virtual interactive gifts is provided, the method comprising: Obtain the input prompt word, which indicates the presentation form of the virtual interactive gift to be generated; The prompt words are processed by an intelligent agent to obtain gift materials. The intelligent agent is used to schedule various stages in the generation process of virtual interactive gifts. The agent obtains the material type of the main material in the gift material, and determines the rendering anchor point of the virtual interactive gift on the target object based on the material type. The intelligent agent generates a virtual interactive gift based on the gift material and the rendering anchor point.

[0004] According to one aspect of the present disclosure, a virtual interactive gift generation device is provided, comprising: The prompt word acquisition module is configured to acquire the input prompt words, which indicate the presentation form of the virtual interactive gift to be generated; The material generation module is configured to process the prompt words through an intelligent agent to obtain gift materials. The intelligent agent is used to schedule various stages in the generation process of virtual interactive gifts. Anchor point determination module is configured to obtain the material type of the main material in the gift material through the intelligent agent, and determine the rendering anchor point of the virtual interactive gift on the target object based on the material type; The gift generation module is configured to generate virtual interactive gifts based on the gift materials and the rendering anchor points.

[0005] In some embodiments, the material generation module includes a material generation unit and a transparency processing unit. The material generation unit is configured to process the prompt words by calling a large model through the intelligent agent to obtain initial gift materials; The transparency processing unit is configured to process the background area of ​​the initial gift material to make it transparent through the intelligent agent, thereby obtaining the gift material.

[0006] In some embodiments, the transparency processing unit is configured to perform transparency processing on the solid color background area in the initial gift material through the agent, and to perform layered transparency processing on the non-solid color background area in the initial gift material by calling a large model through the agent, thereby obtaining the gift material.

[0007] In some embodiments, the anchor point determination module is configured to perform at least one of the following: The intelligent agent invokes a semantic recognition model to perform semantic recognition on the prompt word, thereby obtaining the first material type corresponding to the prompt word; The intelligent agent calls the semantic recognition model to perform semantic recognition on the prompt word, obtains the semantic recognition result, and based on the gift tier information, calls the large model to expand the semantic recognition result to obtain the second material type corresponding to the prompt word; The intelligent agent invokes a large model to parse the content of the gift material, thereby obtaining the third material type of the main material in the gift material.

[0008] In some embodiments, the anchor point determination module is configured to query the following strategy corresponding to the material type based on the material type through the intelligent agent; The agent determines the model part corresponding to the gift material on the target model according to the following strategy, and outputs the coordinates of the model part as the rendering anchor point of the virtual interactive gift on the target object.

[0009] According to one aspect of the embodiments of this disclosure, an interactive method based on virtual interactive gifts is provided, the method comprising: Provides a page for creating virtual interactive gifts; In response to the prompt words entered on the creation page, the prompt words are processed by an intelligent agent to obtain gift materials. The gift materials have a transparent background. The material type of the main material in the gift materials is obtained. Based on the material type, the rendering anchor point of the virtual interactive gift on the target object is determined. Based on the gift materials and the rendering anchor point, the virtual interactive gift is generated. The prompt words indicate the appearance of the virtual interactive gift to be generated. In response to the completion of the virtual interactive gift generation, a gifting operation is initiated in the live broadcast room for the virtual interactive gift.

[0010] According to one aspect of the embodiments of this disclosure, an interactive device based on virtual interactive gifts is provided, the device comprising: The page provides modules that are configured to provide a creation page for virtual interactive gifts; The gift generation module is configured to respond to a prompt word entered on the creation page, process the prompt word through an intelligent agent to obtain gift material, the gift material has a transparent background, obtain the material type of the main material in the gift material, determine the rendering anchor point of the virtual interactive gift on the target object based on the material type, and generate the virtual interactive gift based on the gift material and the rendering anchor point, the prompt word indicating the appearance of the virtual interactive gift to be generated; The gift-giving initiation module is configured to initiate a gift-giving operation for the virtual interactive gift within the live stream in response to the completion of the virtual interactive gift generation.

[0011] According to another aspect of the embodiments of the present disclosure, a computer device is provided, the computer device comprising: One or more processors; Memory used to store the executable program code of the processor; The processor is configured to execute the program code to implement the aforementioned virtual interactive gift generation method or interactive method based on virtual interactive gifts.

[0012] According to another aspect of the present disclosure, a computer-readable storage medium is provided, wherein when program code in the computer-readable storage medium is executed by a processor of a computer device, the computer device is enabled to perform the above-described virtual interactive gift generation method or an interaction method based on virtual interactive gifts.

[0013] According to another aspect of the present disclosure, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the above-described virtual interactive gift generation method or an interaction method based on virtual interactive gifts.

[0014] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0015] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0016] Figure 1 This is a schematic diagram illustrating the implementation environment of a virtual interactive gift generation method according to an exemplary embodiment.

[0017] Figure 2 This is a flowchart illustrating a method for generating virtual interactive gifts according to an exemplary embodiment.

[0018] Figure 3 This is a flowchart illustrating a method for generating virtual interactive gifts according to an exemplary embodiment.

[0019] Figure 4 This is a schematic diagram illustrating a creation page according to an exemplary embodiment.

[0020] Figure 5 This is a block diagram illustrating a virtual interactive gift generation device according to an exemplary embodiment.

[0021] Figure 6 This is a block diagram illustrating an interactive device based on a virtual interactive gift, according to an exemplary embodiment.

[0022] Figure 7 This is a block diagram illustrating a terminal according to an exemplary embodiment.

[0023] Figure 8 This is a block diagram illustrating a server according to an exemplary embodiment. Detailed Implementation

[0024] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0025] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0026] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in this disclosure are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0027] Figure 1 This is a schematic diagram illustrating an implementation environment for a virtual interactive gift generation method according to an exemplary embodiment. See also: Figure 1 The implementation environment specifically includes: terminal 101 and server 102. Terminal 101 and server 102 can be connected directly or indirectly through wired or wireless communication, which is not limited herein.

[0028] Terminal 101 is one implementation of a computer device, and can be at least one of the following: a smartphone, smartwatch, desktop computer, laptop, MP3 player, MP4 player, and laptop computer. Terminal 101 runs an application that supports the generation of virtual interactive gifts. Users can log in to this application through Terminal 101 to access the services provided by the application. For example, users can generate virtual interactive gifts through this application on Terminal 101; of course, this application can be a social application or a multimedia resource playback application, etc.

[0029] Terminal 101 generally refers to one of a plurality of terminals; this embodiment uses terminal 101 as an example. Those skilled in the art will understand that the number of terminals can be more or less. For example, there may be several terminals, or dozens or hundreds of terminals, or even more. This disclosure does not limit the number of terminals or the type of device.

[0030] Server 102 is at least one of a single server, multiple servers, a cloud computing platform, and a virtualization center. Server 102 provides backend services for applications that support the generation of virtual interactive gifts. Server 102 can connect to terminal 101 via a wireless or wired network. Server 102 can provide the terminal with resource publishing, browsing, and database services, and of course, it can also provide other types of video services. In some embodiments, the number of servers may be more or fewer, and this disclosure does not limit this. Of course, server 102 may also include other functional servers to provide more comprehensive and diversified services.

[0031] Figure 2 This is a flowchart illustrating a method for generating virtual interactive gifts according to an exemplary embodiment, see below. Figure 2 Taking the server as the executing entity as an example, the method includes the following steps: In step 201, the input prompt word is obtained, which indicates the presentation form of the virtual interactive gift to be generated.

[0032] The generation process of this virtual interactive gift can be initiated by the user on the client side. That is, the client can provide a creation landing page, offering at least one creation category, such as a virtual interactive gift creation page, a video gift creation page, etc. In response to a trigger operation on the virtual interactive gift creation page, the page is displayed, and the user can input original ideas or preset words on it. Specifically, the user can input prompt words that indicate the appearance of the virtual interactive gift to be generated, thus instructing the automated generation of the virtual interactive gift. In some embodiments, the above generation process can also be tied to corresponding resource consumption, such as paying a certain fee to support the gift generation process.

[0033] The virtual interactive gift disclosed herein refers to a virtual gift that can interact with the host in a video stream in real time. It is a gift that combines the virtual and real worlds. The host's actions or postures will cause the virtual interactive gift to change accordingly. It is different from video gifts, which are special effects videos that will be played at a preset position in the live stream after being sent. The difference between the two is that video gifts can be used without secondary processing, while virtual interactive gifts require A to process them according to the host's shape, display position, etc. after they are sent.

[0034] In step 202, the prompt words are processed by an intelligent agent to obtain gift materials. The intelligent agent is used to schedule various stages in the generation process of virtual interactive gifts.

[0035] The prompts can indicate what kind of virtual interactive gift the user wants to generate. Therefore, by processing the prompts, gift materials that reflect the user's intention to generate the gift can be obtained.

[0036] In step 203, the agent obtains the material type of the main material in the gift material, and determines the rendering anchor point of the virtual interactive gift on the target object based on the material type.

[0037] After the gift material is generated, the intelligent agent can automatically obtain the material type of the main material in the gift material, so as to determine the rendering anchor point of the virtual interactive gift to be generated on the target object for different material types. That is, it should render each part of the virtual interactive gift to which position on the target object.

[0038] In step 204, a virtual interactive gift is generated by an intelligent agent based on the gift material and rendering anchor point.

[0039] Virtual interactive gifts include gift materials and rendering anchors, which can be packaged into a gift configuration file and provided to the client in the form of a gift configuration file. After obtaining the gift configuration file corresponding to the virtual interactive gift, the client can parse the gift configuration file to obtain the gift materials and rendering anchors, and render the gift materials at the corresponding positions of the anchor objects in the live broadcast room according to the rendering anchors, thereby forming the interactive effect of the virtual interactive gift.

[0040] It should be noted that the above-mentioned gift generation process can be triggered by technicians when listing gifts, or by users in the live stream based on their own gifting needs. The generated virtual interactive gifts can be displayed in the user's own gift panel, waiting for the user to give them, or the gifting operation can be automatically initiated after generation. This disclosure does not limit this. For example, in the case of automatically initiating the gifting operation after generation, the user can enter a prompt on the above-mentioned creation page to trigger the generation of the virtual interactive gift. During the generation process, a "Generating" page can be displayed. Optionally, the generation progress can be displayed on the "Generating" page based on different generation stages in the background, such as "Material Generation in Progress" or "Anchor Point Rendering in Progress". After generation is completed, the generated virtual interactive gift can be displayed, and the gifting operation in the live stream can be automatically executed. Alternatively, after providing a preview, the corresponding gifting operation or storage can be executed or the gift can be stored in the gift panel based on the user's confirmation or storage operation.

[0041] The virtual interactive gift generation method provided in this embodiment only requires an intelligent agent to automatically obtain gift materials with a transparent background. The intelligent agent then calculates the rendering anchor points that the material follows by combining the material type of the main body of the generated gift material, thereby generating a virtual interactive gift. In the generation process, no human intervention is required. Only a brief expectation of the appearance of the virtual interactive gift needs to be provided to automatically generate a virtual interactive gift that can interact with the object, which greatly reduces the cost and the production time of the gift.

[0042] Figure 3 This is a flowchart illustrating a method for generating virtual interactive gifts according to an exemplary embodiment, see below. Figure 3 Taking the interaction between the terminal and the server as an example, the method includes the following steps: In step 301, the terminal provides a page for creating virtual interactive gifts.

[0043] In this embodiment, the creation page for the virtual interactive gift can provide a prompt input area and a gift type setting option. The prompt input area is used to input a description of the virtual interactive gift to be generated by the audience in the live stream. This description can be in natural language. The creation page can also provide multiple prompt tags for the audience to choose from, thereby lowering the creation threshold and providing creative references. The gift type setting option is used to set the gift type, which can refer to the category of the virtual interactive gift, such as headwear, clothing, or ambient interactive items.

[0044] See Figure 4 ,Should Figure 4 The diagram shows a creation page where the user can input custom prompts in the prompt input area 401. The creation page also provides multiple gift type settings options, such as ambient interaction type option 402, headwear gift option 403, full-body clothing option 404, and live stream decoration option 405. Furthermore, the prompt input area 401 can also include multiple prompt labels 406 for the user to select during creation. After selecting the above gift type settings option and inputting the prompt, the user can trigger the generation process of the virtual interactive gift by triggering the creation option 407.

[0045] In step 302, in response to an input operation on the creation page of the terminal, the terminal obtains a prompt word.

[0046] The input operation can refer to the object inputting prompt words through text input or voice input, or it can refer to the object inputting corresponding prompt words by selecting prompt word tags provided on the creation page. This embodiment does not limit the specific input operation.

[0047] In step 303, in response to the prompt word entered on the creation page, the server uses an agent to call a large model to process the prompt word and obtain initial gift materials. The agent is used to schedule various stages in the generation process of virtual interactive gifts.

[0048] The generation of this virtual interactive gift involves several steps, including material generation based on prompts (step 303), background transparency processing (step 304 below), material type identification, and rendering anchor point determination. The intelligent agent can execute corresponding processes by calling relevant functional modules at each stage of the generation process, thus achieving fully automated virtual interactive gift generation. It is understandable that this intelligent agent can be obtained through the unified arrangement of multiple functional modules.

[0049] In step 304, the intelligent agent makes the background area of ​​the initial gift material transparent to obtain the gift material.

[0050] In step 304 above, the pure background asset generated by AIGC is made transparent to generate a transparent asset with an alpha channel. This transparency process can adopt a dual-path design: the main path performs real-time algorithm matting on the pure background areas of the initial gift material, while the auxiliary path calls a large AI model to perform layered processing on complex background areas, i.e., non-pure background areas. In other words, in step 304 above, the agent performs matting on the pure background areas of the initial gift material, and the agent calls a large model to perform layered transparency processing on the non-pure background areas of the initial gift material, thus obtaining the gift material.

[0051] In some embodiments, before performing transparency processing, the agent first determines the background complexity of the initial gift material. If the background color concentration of the initial gift material meets a preset condition, it is determined that the initial gift material is suitable for the main path; if the background of the initial gift material contains multiple colors, textures, shadows, or is similar to the main color of the material, it is determined that the initial gift material is suitable for the secondary path. Thus, the agent can prioritize the main path with faster processing speed in solid color background scenes and use the secondary path with higher processing accuracy in non-solid color background scenes.

[0052] The following explains the processing method for the main path mentioned above. The steps involved in using an agent to perform image cutout of the solid-color background area in the initial gift material include the following stages: In the first stage, the agent determines the background key color corresponding to the solid color background. If the upstream module provides a key color parameter, the key color parameter is taken as the background key color; if no key color parameter is provided, sampling is performed from the edge region of the initial gift material, the color cluster with the highest occurrence frequency is determined based on the color histogram of the edge region, and the color median of the color cluster is taken as the background key color.

[0053] In the second stage, the gift material is converted from the RGB frame to the YUV space to decouple luminance and chrominance. For any pixel , its luminance component can be expressed as:

[0054] On this basis, the background matching degree of the pixel belonging to the background region is determined according to the proximity between the chrominance component of the pixel and the chrominance direction of the background key color. A higher background matching degree indicates that the pixel is closer to the solid color background; a lower background matching degree indicates that the pixel is more likely to belong to the material main body.

[0055] In the third stage, the Alpha transparency value is generated based on the background matching degree. Optionally, let the background matching degree of the pixel be M, and the transparency mapping parameter be and , then the Alpha transparency value of the pixel can be expressed as:

[0056] where clip means limiting the calculation result to the range from 0 to 1, Alpha=0 means the pixel is set to be completely transparent, Alpha=1 means the pixel is set to be completely opaque, and 0<Alpha<1 means the pixel is located in the edge transition region between the background and the material main body.

[0057] In the fourth stage, color overflow suppression is performed on the edge of the material main body. Since the solid color background may produce residual green, blue or other background colors at the edge of the material main body, the agent determines the channel to be suppressed based on the main color channel corresponding to the background key color, performs color offset on this channel, and maintains the original light and dark relationship of the material main body through the luminance ratio. Thereby, background color pollution at the edge of the main body can be reduced.

[0058] In the fifth stage, a gift material is generated based on the corrected foreground color and the Alpha transparency value. For an initial gift material in the form of an image, an RGBA image with a transparency channel is output; for an initial gift material in the form of a video, the above processing is performed on each video frame frame by frame, and a video material including an RGB foreground and an Alpha grayscale mask is output. Since the main path mainly includes color statistics, chrominance conversion and pixel-wise transparency calculation, it can be executed on a CPU and is suitable for solid color background materials requiring high processing speed.

[0059] The processing method of the above auxiliary path is explained below. First, the agent preprocesses the initial gift material to obtain the image size and scales it to the input size required by the model. Second, the agent calls a visual understanding model or a text-image model to perform content recognition and description on the initial gift material, generating text prompts to guide image layering. Third, the agent inputs the preprocessed image and the text prompts into a fine-tuned image layering model (Qwen Image-layer). This model performs layer-level parsing and separation of different visual elements in the initial gift material, obtaining independent layers corresponding to the main body of the material and their transparency information. Finally, the agent generates gift material with a transparent background based on the main body layer and transparency information.

[0060] For image matting, this disclosure employs a finely tuned Image Layer model. The core capability of this model is not simply binary separation of the foreground and background layers, but rather the ability to decompose multiple visual elements in an image into independent layers. Furthermore, when separating a layer, it can appropriately fill in occluded or removed areas based on the image context, thereby forming other layers with continuous content and complete structure.

[0061] After fine-tuning, this embodiment of the disclosure applies the image layering capability to complex background matting processing of generative streaming materials. The image layering model includes an image encoding module, a text encoding module, a denoising generation module, and an image decoding module. The image encoding module encodes the initial gift material into latent space features, the text encoding module encodes the text prompt information into conditional features, the denoising generation module generates layering results based on the latent space features and conditional features, and the image decoding module decodes the layering results into a foreground layer and a transparency layer. Through the above auxiliary paths, even if the background of the initial gift material contains desktop elements, shadows, gradients, textures, or other non-solid color content, a gift material with a transparent background can still be obtained.

[0062] The architecture that performs transparency processing through the above dual-path approach uses the main path to process solid-color backgrounds to ensure real-time transparency processing, and the secondary path to process complex backgrounds, which can ensure both processing effectiveness and performance.

[0063] In step 305, the server obtains the material type of the main material in the gift material through the intelligent agent.

[0064] Step 305 above can be implemented using an intelligent agent to determine the material type of the main material. This process can be achieved in any of the following ways: In one possible approach, an agent invokes a semantic recognition model to perform semantic recognition on the prompt words, thereby obtaining the first material type corresponding to the prompt words.

[0065] When the user explicitly describes the type of material, the agent can directly perform semantic recognition based on the prompt word to obtain the corresponding material type. For example, if the prompt word is "hat," the agent can directly perform semantic recognition through a semantic recognition model and extract the category tag "hat."

[0066] In one possible approach, an agent invokes a semantic recognition model to perform semantic recognition on the prompt words, obtaining the semantic recognition result. Based on the gift tier information, a larger model is invoked to expand the semantic recognition result, resulting in the second material type corresponding to the prompt words. The gift tier information can be matched with the generation capability of the large model used. Higher-tier gifts can be expanded using a larger model with stronger generation capabilities, while lower-tier gifts can be expanded using a larger model with relatively weaker generation capabilities. In other words, the gift tier is positively correlated with the generation capability of the large model used.

[0067] When the prompt word entered by the user cannot extract the material type, the agent can call a large model to expand it to complete the material form description and embed the material type in the expanded result. For example, if the prompt word is "dragon and phoenix", and the agent cannot determine the material type by semantic recognition based on the semantic recognition model, the large model can predict it, for example, predict that it is clothing with dragon and phoenix patterns, and add clothing as the material type in the expanded result, thus realizing type-aware pre-reasoning.

[0068] Both of the above implementation methods can be performed after the input prompt words, meaning the material type is determined before the gift material is generated. In some embodiments, image recognition can also be used as a fallback solution to ensure the material type is obtained. That is, in one possible approach, an agent calls a large model to perform content parsing on the gift material to obtain the third material type of the main material in the gift material. When the gift material has no semantic prior, a large model (e.g., a multimodal visual understanding model) is called to perform content parsing on the generated gift material to identify the material type of the main material, such as jewelry, clothing, props, etc.

[0069] The semantic recognition path intervenes during the user's creative input stage: the results of the two paths are uniformly output as structured resource metadata tags, which are used by the downstream rendering engine to call the skeletal tracking anchor points for special effects overlay.

[0070] In step 306, the server uses an agent to query the following strategy corresponding to the material type based on the material type.

[0071] The following strategy is used to determine which body parts of the target object and which points within those parts the gift material should follow.

[0072] In this embodiment, a material type-anchor point mapping table can be predefined to indicate which material types should be applicable to which follow strategy, thereby enabling the determination of the anchor point to be followed based on the follow strategy. For example, hats correspond to heads, clothing to bodies, gloves to hands, and so on. After determining the above material types, a query can be performed based on this mapping table to obtain the follow strategy corresponding to the material type.

[0073] In step 307, the server uses an agent to determine the model part corresponding to the gift material on the target model according to the following strategy, and outputs the coordinates of the model part as the rendering anchor point of the virtual interactive gift on the target object.

[0074] The following strategy is used to determine which body parts of the target object and which points within those parts the gift material should follow. Accordingly, to determine the coordinates of the points that the current gift material should follow, a target model can be used as a reference for determining the coordinates. The target model is a model corresponding to the body parts in the following strategy, and does not necessarily have to be the entire body.

[0075] The specific process of step 307 is explained below. Taking headwear as an example, the resource anchor point calculation module uses a four-stage pipeline to automatically calculate and render anchor point coordinates and wearing width. Based on the resource type, the anchor point coordinates and wearing parameters are automatically calculated through a multi-stage image analysis algorithm; this process includes the following four stages: In the first stage, the alpha channel of the gift material is binarized to obtain a binary image. Different values ​​in this binary image are used to distinguish between the opaque main body area and the transparent background. Then, all independent color blocks are marked in the binary image, for example, using the 8-connectivity rule. Next, based on the material type, connected components are filtered in the binary image to retain multiple connected components; that is, points outside the retained connected components are deleted. During the filtering process, the filtering conditions for connected components can be determined based on the material type, and filtering is performed based on these conditions to obtain multiple connected components. Different filtering conditions are related to the material type of the gift material. For example, for a headband, the filtering conditions could be the largest area and an area percentage ≥ 15%. That is, for this type of gift material, filtering can retain the connected component with the largest area and the secondary connected components with an area percentage ≥ 15%.

[0076] To ensure that subsequent operations are based on a relatively regular source material, after obtaining the above-mentioned multiple connected components, spatial cleaning can be performed on the retained connected components. For example, morphological opening and closing operations can be performed on the connected components to smooth the contours and connect adjacent disconnected foreground regions without changing their size.

[0077] The second stage involves detecting "necks" on the main width curve of the processed binary image; that is, identifying the supporting portion of the main body of the material on the main width curve. For example, for gift materials like headbands, the detection condition can be set to a local minimum row where the width abruptly drops below 25% of the maximum value and has a continuous thick structure above. After identifying the core portion of the main body of the material, the remaining parts can be cut off to avoid interfering with the positioning of the supporting area. For example, for the headband mentioned above, after identifying the supporting portion, the hanging decorations (such as tassels or pendants) below it can be cut off. For other material types, corresponding detection conditions can be configured to detect the supporting portion.

[0078] The main body width curve is used to reflect the variation of the main body row number and the width of the main body in that row. It is obtained through the following process: scan the binary image along the vertical direction of the image, calculate the total horizontal width of the foreground gift pixels in that row, and draw the "row number - width of the gift in that row" one by one to obtain a one-dimensional variation curve, which is the main body width curve.

[0079] The third stage involves extracting support bands from the identified support portions. This process searches for the lowest continuous broadband interval within the vertical range of the main body of the support portion and extracts its bottom sub-band as the support band. Then, the number of significant continuous segments is counted row by row within the support band. Based on the number of significant continuous segments, it is determined whether the support is a single-support type (e.g., a hat) or a dual-support type (e.g., a wreath / headband). In some embodiments, the support band can also be determined based on a distance transformation. The distance transformation refers to calculating the shortest distance from each foreground pixel to the nearest background (black) pixel. The shorter the shortest distance, the greater the probability that it belongs to the support band. Based on the above distance transformation and a preset distance, pixels whose shortest distance falls within the preset distance range can be selected as the support band.

[0080] In the fourth stage, anchor point coordinates are calculated based on the determined material type. For single-unit continuous material types, the X-axis coordinate of the anchor point can be taken as the average of the midline of the support band. For double-sided separated material types, the X-axis coordinate of the anchor point can be taken as the average of the midpoints of the inner edges of the left and right intervals in the support band. Then, symmetry is fine-tuned by maximizing the left and right mirror overlap, and the Y-axis coordinate of the anchor point can be taken as the average Y-axis of the support band. The wearing width can be taken as the width of the bounding box of the largest connected component, and a confidence score with three weights including subject sharpness, width stability, and symmetry is output. If it is lower than the threshold, it is automatically marked for manual review.

[0081] The above four-stage processing structure can be applied to various material types. For newly added material types, the logic of the second to fourth stages can be customized based on the newly added material type, and routes can be registered to achieve pluggable expansion.

[0082] In step 308, the server generates a virtual interactive gift based on the gift material and rendering anchor point through the intelligent agent.

[0083] Effects configuration generation module: Assembles the calculation results into standardized rendering configuration files.

[0084] In some embodiments, if the live stream audience enters the creation page through a gift creation portal provided within the live stream, for example, through... Figure 4 Upon entering the creation page, the gift-giving process can be automatically initiated upon the completion of gift generation, eliminating the need for the user to initiate the gift-giving operation again, thus avoiding cumbersome steps and improving human-computer interaction efficiency. Correspondingly, in response to the completion of the virtual interactive gift generation, the terminal initiates a gift-giving operation for the virtual interactive gift within the live stream. After the gift-giving operation is initiated within the live stream, the terminal can directly render the gift material at the corresponding display position of the streamer in the live stream based on the rendering anchor point in the virtual interactive gift.

[0085] The virtual interactive gift generation method provided in this embodiment only requires an intelligent agent to automatically obtain gift materials with a transparent background. The intelligent agent then calculates the rendering anchor points that the material follows by combining the material type of the main body of the generated gift material, thereby generating a virtual interactive gift. In the generation process, no human intervention is required. Only a brief expectation of the appearance of the virtual interactive gift needs to be provided to automatically generate a virtual interactive gift that can interact with the object, which greatly reduces the cost and the production time of the gift.

[0086] For the platform, this disclosed embodiment significantly reduces the production cost and time of virtual interactive gifts, rapidly enriches the gift library, and enhances the ecological supply in live streaming scenarios. For live streamers, since the aforementioned virtual interactive gifts can be customized based on prompts, the live stream can offer more personalized and exclusive virtual interactive gifts, enhancing the live stream's unique character and improving the interactive atmosphere and effect. For live stream users, the virtual interactive gifts are more diverse and innovative in style, with more varied interactive gameplay, significantly improving the viewing experience and participation experience of gift-giving.

[0087] Figure 5 This is a block diagram illustrating a virtual interactive gift generation device according to an exemplary embodiment. See also Figure 5 The device includes: The prompt word acquisition module 501 is configured to acquire the input prompt word, which indicates the presentation form of the virtual interactive gift to be generated; The material generation module 502 is configured to process the prompt words based on a large model to obtain gift materials, wherein the gift materials have a transparent background; Anchor point determination module 503 is configured to determine the rendering anchor point of the virtual interactive gift on the target object based on the material main body type in the gift material; The gift generation module 504 is configured to generate a virtual interactive gift based on the gift material and the rendering anchor point.

[0088] In some embodiments, the material generation module includes a model processing unit and a transparency processing unit. The model processing unit is configured to input the prompt words into the large model to obtain initial gift materials; The transparency processing unit is configured to process the background area of ​​the initial gift material to make it transparent, thereby obtaining the gift material.

[0089] In some embodiments, the transparency processing unit is configured to perform transparency processing on the solid color background area in the initial gift material through the agent, and to perform layered transparency processing on the non-solid color background area in the initial gift material by calling a large model through the agent, thereby obtaining the gift material.

[0090] In some embodiments, the anchor point determination module includes a type identification unit and an anchor point determination unit. A type recognition unit is configured to recognize the gift material to obtain the material type in the gift material; Anchor point determination unit is configured to determine the rendering anchor point of the virtual interactive gift on the target object according to the following strategy corresponding to the material type, wherein the following strategy indicates the body part that the material follows.

[0091] In some embodiments, the type identification unit is configured to perform at least one of the following: Semantic recognition is performed on the prompt words to obtain the first material type corresponding to the prompt words; The prompt word is semantically recognized, and the semantic recognition result is expanded based on the gift tier information to obtain the second material type corresponding to the prompt word; By analyzing the content of the gift materials using a large model, the material type of the main material in the gift materials can be obtained.

[0092] In some embodiments, the anchor point determination unit is configured to query the following strategy corresponding to the material type based on the material type; Based on the following strategy, the model part corresponding to the gift material on the target model is determined, and the coordinates of the model part are output as the rendering anchor point of the virtual interactive gift on the target object.

[0093] It should be noted that the virtual interactive gift generation device provided in the above embodiments is only illustrated by the division of the above functional units when generating virtual interactive gifts. In practical applications, the above functions can be assigned to different functional units as needed, that is, the internal structure of the computer device can be divided into different functional units to complete all or part of the functions described above. In addition, the virtual interactive gift generation device and the virtual interactive gift generation method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0094] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0095] Figure 6 This is a block diagram illustrating an interactive device based on a virtual interactive gift, according to an exemplary embodiment. See also... Figure 6 The device includes: The page provides module 601, which is configured as a creation page for providing virtual interactive gifts; The gift generation module 602 is configured to respond to a prompt word entered on the creation page, process the prompt word through an intelligent agent to obtain gift material, the gift material having a transparent background, obtain the material type of the main material in the gift material, determine the rendering anchor point of the virtual interactive gift on the target object based on the material type, and generate the virtual interactive gift based on the gift material and the rendering anchor point, the prompt word indicating the appearance of the virtual interactive gift to be generated; The gift initiation module 603 is configured to initiate a gifting operation for the virtual interactive gift in the live broadcast room in response to the completion of the virtual interactive gift generation.

[0096] In some embodiments, the gift generation module 602 described above is the aforementioned Figure 5 The virtual interactive gift generation device provided in the illustrated embodiment includes all the functional modules included in the virtual interactive gift generation device, which will not be described in detail here.

[0097] It should be noted that the interactive device based on virtual interactive gifts provided in the above embodiments is only illustrated by the division of the above functional units when interacting with virtual interactive gifts. In practical applications, the above functions can be assigned to different functional units as needed, that is, the internal structure of the computer device can be divided into different functional units to complete all or part of the functions described above. In addition, the interactive device based on virtual interactive gifts provided in the above embodiments and the interactive method embodiments based on virtual interactive gifts belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0098] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0099] Computer equipment can be implemented as a terminal. Figure 7 This is a block diagram illustrating a terminal 700 according to an exemplary embodiment. Figure 7 A structural block diagram of a terminal 700 provided in an exemplary embodiment of this disclosure is shown. The terminal 700 may be a smartphone, tablet computer, MP3 player (Moving Picture Experts Group Audio Layer III), MP4 player (Moving Picture Experts Group Audio Layer IV), laptop computer, or desktop computer. The terminal 700 may also be referred to as a user device, portable terminal, laptop terminal, desktop terminal, or other names.

[0100] Typically, terminal 700 includes a processor 701 and a memory 702.

[0101] Processor 701 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 701 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 701 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 701 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the screen. In some embodiments, processor 701 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0102] The memory 702 may include one or more computer-readable storage media, which may be non-transitory. The memory 702 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 702 are used to store at least one computer program, which is executed by the processor 701 to implement the method provided in the method embodiments of this application.

[0103] In some embodiments, the terminal 700 may also optionally include a peripheral device interface 703 and at least one peripheral device. The processor 701, memory 702, and peripheral device interface 703 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 703 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency circuit 704, a display screen 705, a camera assembly 706, an audio circuit 707, and a power supply 708.

[0104] Peripheral device interface 703 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 701 and memory 702. In some embodiments, processor 701, memory 702 and peripheral device interface 703 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 701, memory 702 and peripheral device interface 703 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0105] The radio frequency (RF) circuit 704 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 704 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 704 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. In some embodiments, the RF circuit 704 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 704 can communicate with other terminals through at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 704 may also include circuitry related to NFC (Near Field Communication), which is not limited in this application.

[0106] Display screen 705 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 705 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 701 for processing. In this case, display screen 705 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one display screen 705, disposed on the front panel of terminal 700; in other embodiments, there may be at least two display screens, disposed on different surfaces of terminal 700 or in a folded design; in other embodiments, display screen 705 may be a flexible display screen, disposed on a curved or folded surface of terminal 700. Furthermore, display screen 705 may be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. Display screen 705 may be made of materials such as LCD (Liquid Crystal Display) or OLED (Organic Light-Emitting Diode).

[0107] The camera assembly 706 is used to acquire images or videos. In some embodiments, the camera assembly 706 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the terminal, and the rear-facing camera is located on the back of the terminal. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, VR (Virtual Reality) shooting, or other fusion shooting functions. In some embodiments, the camera assembly 706 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash is a combination of a warm-light flash and a cool-light flash, which can be used for light compensation at different color temperatures.

[0108] The audio circuit 707 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to the processor 701 for processing, or input to the radio frequency circuit 704 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each located at a different part of the terminal 700. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert the electrical signals from the processor 701 or the radio frequency circuit 704 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 707 may also include a headphone jack.

[0109] Power supply 708 is used to power the various components in terminal 700. Power supply 708 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 708 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, while a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0110] Those skilled in the art will understand that Figure 7 The structure shown does not constitute a limitation on terminal 700, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0111] Figure 8 This is a schematic diagram of a server structure provided in an embodiment of this application. The server 800 can vary significantly depending on its configuration or accuracy. It may include one or more Central Processing Units (CPUs) 801 and one or more memories 802. The memories 802 store at least one computer program, which is loaded and executed by the processor 801 to implement the methods provided in the various method embodiments described above. Of course, the server may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The server may also include other components for implementing device functions, which will not be elaborated upon here.

[0112] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory including instructions, which can be executed by a processor of a computer device, such as a terminal, to perform the method provided in the above embodiments. Optionally, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0113] This disclosure also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the methods provided in the above embodiments.

[0114] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0115] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for generating virtual interactive gifts, characterized in that, The method includes: Obtain the input prompt word, which indicates the presentation form of the virtual interactive gift to be generated; The prompt words are processed by an intelligent agent to obtain gift materials. The intelligent agent is used to schedule various stages in the generation process of virtual interactive gifts. The agent obtains the material type of the main material in the gift material, and determines the rendering anchor point of the virtual interactive gift on the target object based on the material type. The intelligent agent generates a virtual interactive gift based on the gift material and the rendering anchor point.

2. The virtual interactive gift generation method according to claim 1, characterized in that, The process of processing the prompt words using a large model through the intelligent agent to obtain gift materials includes: The intelligent agent calls a large model to process the prompt words and obtain initial gift materials; The gift material is obtained by making the background area of ​​the initial gift material transparent using the intelligent agent.

3. The virtual interactive gift generation method according to claim 2, characterized in that, The process of making the background area of ​​the initial gift material transparent using the intelligent agent to obtain the gift material includes: The agent performs transparency processing on the solid color background area in the initial gift material, and the agent calls the large model to perform layered transparency processing on the non-solid color background area in the initial gift material, thus obtaining the gift material.

4. The virtual interactive gift generation method according to claim 1, characterized in that, The material types obtained by the intelligent agent for the main body of the gift material include: The intelligent agent invokes a semantic recognition model to perform semantic recognition on the prompt word, thereby obtaining the first material type corresponding to the prompt word; The intelligent agent calls the semantic recognition model to perform semantic recognition on the prompt word, obtains the semantic recognition result, and based on the gift grade information, calls the large model to expand the semantic recognition result to obtain the second material type corresponding to the prompt word; The intelligent agent invokes a large model to parse the content of the gift material, thereby obtaining the third material type of the main material in the gift material.

5. The virtual interactive gift generation method according to claim 1, characterized in that, The step of determining the rendering anchor point of the virtual interactive gift on the target object based on the material type includes: The intelligent agent queries the following strategy corresponding to the material type based on the material type. The agent determines the model part corresponding to the gift material on the target model according to the following strategy, and outputs the coordinates of the model part as the rendering anchor point of the virtual interactive gift on the target object.

6. An interactive method based on virtual interactive gifts, characterized in that, include: Provides a page for creating virtual interactive gifts; In response to the prompt words entered on the creation page, the prompt words are processed by an intelligent agent to obtain gift materials. The gift materials have a transparent background. The material type of the main material in the gift materials is obtained. Based on the material type, the rendering anchor point of the virtual interactive gift on the target object is determined. Based on the gift materials and the rendering anchor point, the virtual interactive gift is generated. The prompt words indicate the appearance of the virtual interactive gift to be generated. In response to the completion of the virtual interactive gift generation, a gifting operation is initiated in the live broadcast room for the virtual interactive gift.

7. A virtual interactive gift generation device, characterized in that, include: The prompt word acquisition module is configured to acquire the input prompt words, which indicate the presentation form of the virtual interactive gift to be generated; The material generation module is configured to process the prompt words through an intelligent agent to obtain gift materials, wherein the gift materials have a transparent background; Anchor point determination module is configured to obtain the material type of the main material in the gift material through the intelligent agent, and determine the rendering anchor point of the virtual interactive gift on the target object based on the material type; The gift generation module is configured to generate virtual interactive gifts based on the gift materials and the rendering anchor points.

8. An interactive device based on virtual interactive gifts, characterized in that, include: The page provides modules that are configured to provide a creation page for virtual interactive gifts; The gift generation module is configured to respond to a prompt word entered on the creation page, process the prompt word through an intelligent agent to obtain gift material, the gift material has a transparent background, obtain the material type of the main material in the gift material, determine the rendering anchor point of the virtual interactive gift on the target object based on the material type, and generate the virtual interactive gift based on the gift material and the rendering anchor point, the prompt word indicating the appearance of the virtual interactive gift to be generated; The gift-giving initiation module is configured to initiate a gift-giving operation for the virtual interactive gift within the live stream in response to the completion of the virtual interactive gift generation.

9. A computer device, characterized in that, The computer device includes: One or more processors; Memory used to store the executable program code of the processor; The processor is configured to execute the program code to implement the virtual interactive gift generation method as described in any one of claims 1 to 5 or the interactive method based on virtual interactive gifts as described in claim 6.

10. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of a computer device, the computer device is enabled to perform the virtual interactive gift generation method as described in any one of claims 1 to 5 or the interactive method based on virtual interactive gifts as described in claim 6.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the virtual interactive gift generation method according to any one of claims 1 to 5 or the interactive method based on virtual interactive gifts according to claim 6.

Images