Live interactive method and device, computer readable storage medium

Abstract

This disclosure relates to a live streaming interactive method and apparatus, and a computer-readable storage medium. The live streaming interactive method includes: displaying a live stream screen, wherein the live stream screen includes identifiers of multiple users displayed at different locations; and, in response to a target user among the multiple users obtaining a virtual resource, displaying the changing effects of a virtual object corresponding to the virtual resource moving from a starting position to a target position, wherein the target position corresponds to the identifier of the target user.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to live interactive methods and apparatus, and computer-readable storage media. Background Technology

[0002] With the rapid development of internet technology, people are using an increasing number of applications in their daily lives. Live interactive applications, through real-time interaction, greatly enrich users' social experiences and entertainment, enabling users to communicate and interact more closely in virtual space.

[0003] When watching a live interactive interface with multiple participants (such as a multi-person chat room), viewers can not only watch the performances or discussions of the participants in real time, but also send virtual gifts to them. Summary of the Invention

[0004] According to some embodiments of this disclosure, a live interactive method is provided, comprising: displaying a live screen, wherein the live screen includes identifiers of multiple users displayed at different locations; and, in response to a target user among the multiple users obtaining virtual resources, displaying the changing effect of a virtual object corresponding to the virtual resources moving from a starting position to a target position, wherein the target position corresponds to the identifier of the target user.

[0005] According to some other embodiments of this disclosure, a live interactive device is provided, including: a screen display module configured to display a live screen, wherein the live screen includes identifiers of multiple users displayed at different locations; and an effect display module configured to, in response to a target user among the multiple users obtaining virtual resources, display the change effect of a virtual object corresponding to the virtual resources moving from a starting position to a target position, wherein the target position corresponds to the identifier of the target user.

[0006] According to other embodiments of the present disclosure, an electronic device is provided, including: a memory; and a processor coupled to the memory, the processor being configured to execute a live interactive method according to some embodiments of the present disclosure based on instructions stored in the memory.

[0007] According to other embodiments of the present disclosure, a computer-readable storage medium is provided having computer program instructions stored thereon, which, when executed by a processor, implement a live interactive method according to some embodiments of the present disclosure.

[0008] According to other embodiments of this disclosure, a computer program product is provided, including computer program instructions that, when executed by a processor, implement a live interactive method according to some embodiments of this disclosure.

[0009] Other features, aspects, and advantages of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0010] Embodiments of this disclosure are described below with reference to the accompanying drawings. It should be understood that the drawings described below are merely illustrative of some embodiments of this disclosure and are not intended to limit the scope of this disclosure. In the drawings:

[0011] Figure 1 Flowcharts showing some embodiments of the live interactive method disclosed herein;

[0012] Figure 2 A schematic diagram showing a live broadcast screen according to some embodiments of the present disclosure;

[0013] Figures 3A-3C A schematic diagram showing a live broadcast screen according to some embodiments of the present disclosure;

[0014] Figures 4A-4B Schematic diagrams showing live broadcast screens according to other embodiments of the present disclosure;

[0015] Figures 5A-5C A schematic diagram illustrating the movement trajectory of a virtual object according to some embodiments of the present disclosure;

[0016] Figure 6 Schematic diagrams illustrating material extensions according to some embodiments of the present disclosure;

[0017] Figure 7 A schematic diagram illustrating the generation of prompt text according to some embodiments of the present disclosure is shown;

[0018] Figure 8 A schematic diagram showing original video frames according to some embodiments of the present disclosure;

[0019] Figure 9 This diagram illustrates the generation of virtual objects according to some embodiments of the present disclosure;

[0020] Figure 10 A schematic diagram of a virtual object according to some embodiments of the present disclosure is shown;

[0021] Figure 11 A schematic diagram illustrating the effects of variations generated according to some embodiments of the present disclosure is provided.

[0022] Figure 12 Schematic diagrams illustrating the effects of variations in production according to other embodiments of this disclosure;

[0023] Figure 13 A schematic diagram illustrating the generation of orbital motion animations according to some embodiments of the present disclosure is shown;

[0024] Figure 14 A block diagram of a live interactive device according to some embodiments of the present disclosure is shown;

[0025] Figure 15 A block diagram of an electronic device according to some embodiments of the present disclosure is shown;

[0026] Figure 16 Block diagrams of electronic devices according to other embodiments of the present disclosure are shown.

[0027] It should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not necessarily drawn to actual scale. The same or similar reference numerals are used in the various drawings to denote the same or similar parts. Therefore, once an item is defined in one drawing, it may not be discussed further in subsequent drawings. Detailed Implementation

[0028] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. It should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein.

[0029] It should be understood that the various steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect. Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of components and steps set forth in these embodiments should be interpreted as merely exemplary and do not limit the scope of this disclosure.

[0030] As used in this disclosure, the term "comprising" and its variations are open-ended terms that include at least the following elements / features but do not exclude other elements / features, i.e., "including but not limited to". The term "based on" means "at least partially based on".

[0031] It should be noted that the concepts of "first," "second," etc., used in this disclosure are used only to distinguish different devices, modules, or units, and are not intended to define the order of functions performed by these devices, modules, or units or their interdependencies. Unless otherwise specified, the concepts of "first," "second," etc., are not intended to imply that the objects described herein must be in a given temporal, spatial, rank, or any other given order.

[0032] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0033] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0034] The user information disclosed herein is all information and data authorized by the user or fully authorized by all parties. The collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals shall be provided for users to choose to authorize or refuse.

[0035] The embodiments of this disclosure are described in detail below with reference to the accompanying drawings; however, this disclosure is not limited to these specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. Furthermore, in one or more embodiments, specific features, structures, or characteristics can be combined in any suitable manner that will be apparent to those skilled in the art from this disclosure.

[0036] In related technologies, viewers can send virtual gifts to other users in a live chat interface (e.g., a multi-user chat room). However, the display of virtual gifts is relatively simple, and the effects corresponding to the virtual resources are only displayed in a fixed area of ​​the screen, making it difficult for viewers and other users to intuitively perceive who received the virtual gift.

[0037] This disclosure provides a live interactive method and apparatus, as well as a computer-readable storage medium, which allows viewers to intuitively perceive which user has received virtual resources.

[0038] Figure 1 This is a flowchart illustrating some embodiments of the live interactive method disclosed herein.

[0039] like Figure 1 As shown, the live streaming interaction method includes steps S1 to S2. This embodiment of the live streaming interaction method can be executed on the client side or partially on the server side. The client can be, for example, the client of multiple live streaming users participating in the live streaming interaction or the client of a viewing user. The client can be application software or hardware; for example, the client can be a live streaming application, a hardware device in a user device, etc.

[0040] In step S1, a live stream is displayed, which includes identifiers of multiple users displayed in different locations.

[0041] Figure 2 A schematic diagram illustrating a live broadcast screen according to some embodiments of the present disclosure is shown. Figure 2As shown, a user's identifier can be their avatar or nickname, etc. Around each user's identifier, the number of virtual gifts that user has received can also be displayed. The identifiers for the streamer and co-streaming users can be displayed in the same or different ways on the live stream screen. For example... Figure 2 The identifier for the live streamer is larger and centered, thus distinguishing it from the identifier for other live streamers.

[0042] In step S2, in response to the target user among multiple users obtaining virtual resources, the changes in the virtual object corresponding to the virtual resource as it moves from the starting position to the target position are displayed, wherein the target position corresponds to the identifier of the target user.

[0043] The target users are, for example, users participating in a live chat or the streamer. Virtual resources are digital items that viewers send to users participating in a live chat or the streamer, often presented as gifts or props. When a virtual resource is sent, it triggers the display of an associated virtual object, i.e., a gift effect. For example, on a live streaming platform, after a viewer sends a virtual sports car as a gift, an animation of the sports car can appear on the screen, which is the virtual object corresponding to the virtual sports car gift. Sound effects can also be played simultaneously with the visual effects.

[0044] The starting position is, for example, a specific location within the live stream or the edge of the live stream. The target position is, for example, the area where the target user's identifier is displayed or the area surrounding that area. The virtual object itself changes as it moves from the starting position to the target position. The virtual object displayed at the starting position and the virtual object displayed at the target position may be different.

[0045] Figures 3A-3C A schematic diagram showing a live broadcast screen according to some embodiments of the present disclosure.

[0046] The following is combined Figures 3A-3C This section describes variations of virtual objects according to some embodiments of the present disclosure.

[0047] The virtual object includes, for example, a first object and a second object. The display shows the changes in the virtual object corresponding to the virtual resource as it moves from the starting position to the target position, including: displaying the effect of the first object and the second object separating and the second object moving to the target position as the virtual object moves from the starting position to the target position.

[0048] like Figure 3AAs shown, when a viewer sends the virtual resource "Heart Bunny" to User 1, the animations of the first object "Rabbit" and the second object "Heart" are first displayed at the starting position of the live stream screen. For example, at the starting position, the "Rabbit" dances while the "Heart" flies around. After the animation at the starting position is complete, the "Rabbit" and "Heart" separate, and the "Heart" moves along the trajectory of the dotted arrow to the target position, such as around User 1's avatar. After the "Heart" moves to the target position, an animation of the "Heart" near the target position is displayed, such as the "Heart" rotating around User 1's avatar. Furthermore, after the first and second objects separate, an animation of the first object leaving the screen can be displayed. Alternatively, after the first and second objects separate, not only can an animation of the second object moving to the target position be displayed, but also animations of the first object remaining at the separation position, returning to the starting position, or moving to other positions on the screen.

[0049] By having two objects appear at the starting position and then separate, the viewer's attention is effectively guided. First, the viewer's gaze is drawn to the first and second objects in the overall scene. Then, as the second object moves, the viewer's eye naturally follows, focusing on the user icon that received the gift. This method seamlessly connects the effects at the starting and ending positions, adding dynamism and variation to the overall presentation, making the animation more vivid and engaging.

[0050] The virtual object includes, for example, a first object and a second object. The display shows the changes in the virtual object corresponding to the virtual resource as it moves from the starting position to the target position, including: displaying the effect of the first object transforming into the second object as the virtual object moves from the starting position to the target position, wherein the first object and the second object are different.

[0051] For example, a beam of light is displayed at the starting position, transforming into a halo as it moves to the target position. This change in form from the first object to the second, combined with the movement from the starting position to the target position, achieves a smooth and natural visual transition, adding richer dynamic variation. Simultaneously, this method smoothly and coherently guides the viewer's eye to the identifier of the user who received the gift.

[0052] It can also be like Figure 3C As shown, the first virtual object is displayed first. Then, the animation of the second object appearing is displayed, resulting in the following: Figure 3A The scene shown depicts the first object gradually disappearing, while the second object moves to the target location, as shown in the image. Figure 3B As shown.

[0053] In some embodiments, displaying the changes of a virtual object corresponding to a virtual resource as it moves from a starting position to a target position includes displaying the deformation and / or scaling effects of the virtual object as it moves from the starting position to the target position.

[0054] For example, a heart is displayed at the starting position, and as the heart moves to the target position, the heart is enlarged and displayed.

[0055] Figures 4A-4B A schematic diagram showing a live broadcast screen according to other embodiments of the present disclosure.

[0056] The following is combined Figures 4A-4B The present disclosure describes variations of virtual objects according to other embodiments.

[0057] The virtual objects include, for example, a first object and a second object. The display shows the changes in the virtual objects corresponding to the virtual resources as they move from the starting position to the target position, including: displaying the effect of the first object moving out of the live broadcast screen from the starting position; and displaying the effect of the second object moving from the entry position into the live broadcast screen to the target position.

[0058] For example, the first object is a "rocket," and the second object is a "satellite." Starting from the initial position, the animation of the first object, the "rocket," launching is displayed. For example... Figure 4A As shown, for example, the live stream screen moves upwards along the dotted line from the starting position. Then, as... Figure 4B As shown, a "satellite" flies in from off-screen and moves along the dotted line to the target location, near the avatar of the user who received the gift. After reaching the target location, an animation of the "satellite" circling the user's avatar can also be displayed. In this way, the transition from displaying the first object in the initial position to the second object moving to the target position is natural, making the overall effect smoother and creating different layers.

[0059] The following describes a situation where multiple users received virtual resources within a short period of time.

[0060] Taking the example of a first user and a second user receiving virtual resources, if the time difference between the first user and the second user receiving the same type of virtual resource is less than a threshold, the effects of the first user and the second user receiving the virtual resource can be displayed together.

[0061] In some embodiments, the virtual object includes, for example, a first object, a second object, and a third object. Displaying the changes in the virtual object corresponding to the virtual resource as it moves from a starting position to a target position includes: displaying the separation of the first object and the second object, and the separation of the first object and the third object, during the movement of the virtual object from the starting position to the target position; displaying the effect of the second object moving to the target position of the first user; and displaying the effect of the third object moving to the target position of the second user.

[0062] For example, if the first object is a "rabbit" and the second and third objects are both "hearts", then it can be displayed that the "rabbit" sends out multiple "hearts", which are moved to the first user and the second user respectively.

[0063] In some embodiments, the virtual object includes, for example, a first object, a second object, and a third object. Displaying the changes of the virtual object corresponding to the virtual resource as it moves from the starting position to the target position includes: displaying the effect of the first object transforming into the second object and the third object as the virtual object moves from the starting position to the target position.

[0064] For example, a portion of the first object transforms into the second object, and another portion of the first object transforms into the third object. The effect of moving the second object to the first user's target location is displayed, as is the effect of moving the third object to the second user's target location.

[0065] In some embodiments, the virtual objects include, for example, a first object, a second object, and a third object. Displaying the changes in the virtual objects corresponding to the virtual resources as they move from a starting position to a target position includes: displaying the effect of the first object moving out of the live stream from its starting position; displaying the effect of the second object moving from its entry position into the live stream to a first user target position; and displaying the effect of the third object moving from its entry position into the live stream to a second user target position.

[0066] For example, the first object is a "rocket," and the second and third objects are "satellites." Starting from the initial position, the animation of the first object, the "rocket," launching is displayed. For example... Figure 4A As shown, the "rocket" moves upwards along the dotted line from its starting position out of the live stream. Then, two "satellites" fly in from off-screen and move to the vicinity of the first and second users' avatars, respectively.

[0067] According to some embodiments of this disclosure, visual linkage between the effects at the starting position and the effects at the target position is achieved by displaying the changes in the virtual object corresponding to the virtual resource as it moves from the starting position to the target position.

[0068] Figures 5A-5CA schematic diagram illustrating the movement trajectory of a virtual object according to some embodiments of the present disclosure is shown.

[0069] The following is combined Figures 5A-5C This section introduces the location and movement trajectory of virtual objects in the live broadcast.

[0070] In some embodiments, the live streaming interaction method further includes determining a designated position outside the live streaming screen based on the exit position of the first object moving out of the live streaming screen from the starting position; determining a connecting line from the designated position outside the live streaming screen to the target position; and determining an entry position based on the intersection of the connecting line and the edge of the screen.

[0071] like Figure 5A As shown, the black triangle represents the exit position of the first object as it moves out of the live stream from its starting position. The designated position (shown as a black circle) can be a position above the exit position, such as centered at the exit position. Then, the line connecting the designated position and the user identifier is determined; the intersection of this line and the screen is the entry position of the second object. The second object moves along this line within the live stream.

[0072] When different target users acquire virtual resources, the starting position of the virtual object can be the same. For example... Figure 5B As shown, for different target users, the starting position is the position indicated by the white circle in the live stream. Or, for example... Figure 5C As shown, for different target users, the starting position is the position indicated by the white circle at the edge of the live broadcast screen, indicating that the virtual object enters from the edge of the screen or outside the screen.

[0073] According to some embodiments of this disclosure, existing materials can be extended to generate various animation effects for virtual objects, which will be described below.

[0074] In some embodiments, a first prompt text is determined based on the virtual resource; based on the first prompt text, a first machine learning model is used to perform style transfer processing on the material corresponding to the virtual resource to obtain a virtual object.

[0075] For example, a virtual resource corresponds to a video or image clip. Based on this clip, a first prompt text is generated. This first prompt text is used as a prompt word for a first machine learning model. The first machine learning model then performs style transfer processing on the clip corresponding to the virtual resource to obtain a virtual object. The first machine learning model may include a style transfer model. The virtual object and the clip are not exactly the same, but their styles are similar.

[0076] In some embodiments, determining a first prompt text based on a virtual resource includes: acquiring a video corresponding to the virtual resource; selecting a video frame from the video; and generating the first prompt text based on the semantic information of the video frame.

[0077] For example, the virtual resource is a rocket. The corresponding video for the virtual resource is a cartoon of a rocket launch. Multiple video frames are extracted from the video, and these frames all contain a rocket; these rockets are then used as source material. Based on the semantic information of the rocket and the video frames, a graph-to-text model is used to generate the initial prompt text.

[0078] In some embodiments, based on the information of the live streaming room corresponding to the live streaming screen, a second machine learning model is used to perform style transfer processing on the materials corresponding to the virtual resources to obtain virtual objects.

[0079] The information corresponding to the live stream includes the live stream name, the scene of the live stream, information about the users in the live stream (such as user avatars and nicknames), and information about the interactions between users and viewers. In other words, even for the same virtual resources, the virtual objects can differ for different live streams and different target users. By generating virtual objects based on the information of the live stream, it is possible to adapt to different live stream scenarios, adding more elements and variations to the live stream's effects.

[0080] In some embodiments, based on the information of the live broadcast room corresponding to the live broadcast screen and the virtual object, a third machine learning model is used to generate the change effect of the virtual object moving from the starting position to the target position.

[0081] For example, virtual objects can be derived from the original materials and live stream information, and variations can be derived from the virtual objects and live stream information. In other words, even for the same virtual resources, the variations of virtual objects can differ for different live streams and different target users.

[0082] In some embodiments, the virtual object includes a first object and a second object. The live interactive method further includes: determining a second prompt text based on virtual resources; and generating a second object associated with the first object using a fourth machine learning model based on the first object and the second prompt text. For example, the second object is derived from the first object. The first object and the second object can cooperate to form a hierarchical animation effect.

[0083] According to some embodiments of this disclosure, by utilizing content generation technology and based on materials corresponding to virtual resources, more animation effects can be developed, reducing design costs.

[0084] The following describes a method for expanding materials to obtain the effect of changing virtual objects according to some embodiments of this disclosure.

[0085] Figure 6 A schematic diagram illustrating material extensions according to some embodiments of the present disclosure is shown.

[0086] like Figure 6As shown, the first step is to obtain video footage of the gift. Multiple frames are extracted from the video footage, and object recognition and action recognition models are used to identify and understand the objects and behaviors in the video, generating preliminary content copy. Keywords related to the video's mood can also be added.

[0087] Based on the initial content copy, multiple sets of creative text describing the changing effects of virtual objects are generated, along with prompts related to the virtual objects within the creative text. During this process, pre-trained models can be introduced to help generate richer and more coherent associative prompts. Then, suitable prompts are selected for later use.

[0088] Elements needed for reference in the video are extracted using a segmentation model and cropped into standard reference images. After extracting the reference images, these images are used as input for generating subsequent image materials. An image cue adapter and control network are employed for style referencing and style transfer. The resulting virtual objects are then matched to the overall style or the style of some objects in the original video material.

[0089] Based on the selected prompts, a text-to-image model is invoked, along with reference material styles, to generate static images that meet the conditions. Similarly, based on the selected animated prompts, a text-to-video model is invoked, along with reference material styles, to generate animated images that meet the conditions. Alternatively, an image-to-video model can be further applied to convert static images into animated images. Ultimately, a series of static images or animated images related to the virtual resources are obtained.

[0090] Figure 7 A schematic diagram illustrating the generation of prompt text according to some embodiments of the present disclosure is shown.

[0091] like Figure 7 As shown, a language model is used to understand the video, chat scenarios, and effects, resulting in understood text. Based on this understood text, brainstorming is used to generate creative text describing the effects related to the content. Combining this with supporting materials, the creative text is further refined, adding details to obtain prompt words.

[0092] Figure 8 A schematic diagram of original video frames according to some embodiments of the present disclosure is shown.

[0093] The elements that need to be referenced are, for example, the original video frames. Figure 8 The generated cue words include positive and / or negative cue words, such as: "A bottle of orange juice, the orange juice is bright and clear, the transparent bottle has a reflective surface, the glass is smooth, high resolution, no background, no shadows, isolated object."

[0094] The relationships between virtual objects and elements in the original material include, for example, similarity, complementarity, contrast, continuation, symbolism, gradation, simplification, reproduction, reversal, magnification, reference, and sequence.

[0095] Figure 9 A schematic diagram illustrating the generation of virtual objects according to some embodiments of the present disclosure is shown.

[0096] like Figure 9 As shown, firstly, the frame with the most content in the video is captured and cropped as input for object and action recognition. The model extracts the elements that need reference from the video and crops them into standard reference images. After the reference elements are extracted, the reference images are used as input for generating subsequent image materials, and a plugin is used for style reference and style transfer. Combined with the selected prompts, a text-to-image model is invoked, along with the style of the reference materials, to generate static images that meet the conditions. The resulting virtual objects are then matched to the overall style or the style of some objects in the original video content.

[0097] Figure 10 A schematic diagram of a virtual object according to some embodiments of the present disclosure is shown.

[0098] like Figure 10 As shown, from Figure 8 The bottle from which the extraction was made (i.e. Figure 10 The bottle on the left, after the above process, expands to obtain the following: Figure 10 The bottle to the right of the middle arrow.

[0099] After obtaining the static image, you also need to generate animation effects for the image. Animation effects can be categorized into several types.

[0100] In one scenario, the change effect of a virtual object is an animation derived from the deformation or scaling of an existing object. For example, a star might produce an animation effect. Or, a positioning animation (the effect of reaching a target position) might be a further animation effect of an extended element (e.g., a heart might produce a zoom-in animation after reaching the microphone position).

[0101] Figure 11 A schematic diagram illustrating the effects of generating variations according to some embodiments of the present disclosure is shown.

[0102] like Figure 11 As shown, the effect in this case is generated using a graph-to-video approach. First, the static image undergoes sparse control preprocessing, then is processed by a control network before being passed to the sampler. Next, the static image is encoded, and N copies of the image are made, where N = n * number of frames, and n is the duration. The latent space based on the static image is sent to the sampler. By using a graph-to-video approach, the image style is used as the overall input, eliminating the need for additional style transfer; the result is optimized by changing the style model.

[0103] Another scenario involves the virtual object's transformation effect being a new animation created by the creator, encompassing not only existing objects but also derived ones. For example, a beam of light reaching a microphone might create a halo, the actual shape of which differs significantly from the light beam itself.

[0104] Figure 12 A schematic diagram illustrating the effects of variations generated according to other embodiments of this disclosure is shown.

[0105] like Figure 12 As shown, this approach primarily generates animated GIFs from text, which is not limited by static materials and produces animated effects that more closely resemble the descriptions in the prompt text. For example, by using text-to-video generation, an image cue adapter can be employed for style transfer, and the results can be optimized by combining style model changes.

[0106] Figure 13 A schematic diagram illustrating the generation of orbital motion animations according to some embodiments of the present disclosure is shown.

[0107] The effects of virtual object changes can be pre-generated. Then, after a target user in the live stream receives a gift, the animation of the virtual object moving from its initial position to the target position is rendered. Below, we combine... Figure 13 This paper introduces the methods for generating orbital motion animations.

[0108] like Figure 13 As shown, the generated motion effect resource files are first stored. Based on the vertex positions of the live stream room, the preset positions of the chat room scene area, and the target user's avatar display area receiving the virtual resources, the path animation of the virtual object is rendered, ensuring that the virtual object eventually reaches the target microphone position along the motion track. The coordinates of the path and the relationships between motion effect resources can be configured based on the motion effect resource files. During the rendering process, the vertex and texture coordinates of each motion effect resource are calculated, and texture mapping is applied to the track area. Various motion trajectory algorithms can be configured within a specific area.

[0109] Because the extended effects were prepared in advance, when the target user receives the gift, only relatively simple rendering and position calculation are needed to generate the trajectory animation. This allows for a rapid response to user actions, reducing the delay from gift triggering to animation display, enabling viewers to see the animation effect immediately and improving the real-time performance and smoothness of the entire live stream. Furthermore, the deep integration and rendering of the extended effects with the animation path creates a better sense of visual synergy.

[0110] A live interactive method according to some embodiments of the present disclosure includes: displaying a live screen, wherein the live screen includes identifiers of multiple users displayed at different locations; and, in response to a target user among the multiple users obtaining virtual resources, displaying the changing effects of a virtual object corresponding to the virtual resources moving from a starting position to a target position, wherein the target position corresponds to the identifier of the target user.

[0111] In other words, the virtual object not only changes itself but also moves from its starting position to its target position. This approach adds dynamism and variability to the overall presentation, making the animation more vivid and engaging, while also seamlessly connecting the starting and target positions. Compared to playing virtual resources only in a fixed area of ​​the screen, this disclosed solution guides the viewer's attention to the user's identifier by combining it with the user's location, allowing them to intuitively understand who received the virtual resource and enhancing interactivity and visual impact.

[0112] Figure 14 A block diagram of a live interactive device according to some embodiments of the present disclosure is shown.

[0113] like Figure 14 As shown, the live interactive device 1 includes: a screen display module 11, configured to display a live screen, wherein the live screen includes identifiers of multiple users displayed at different locations; and an effect display module 12, configured to, in response to a target user among the multiple users obtaining virtual resources, display the changing effect of a virtual object corresponding to the virtual resource moving from a starting position to a target position, wherein the target position corresponds to the identifier of the target user.

[0114] The screen display module 11 of the live interactive device 1 can be used to perform... Figure 1 Step S1. The effect display module 12 of the live interactive device 1 can be used to perform... Figure 1 Step S2.

[0115] In some embodiments, the live interactive device 1 further includes a processing module configured to: determine a first prompt text based on virtual resources; and perform style transfer processing on the material corresponding to the virtual resources using a first machine learning model based on the first prompt text to obtain a virtual object.

[0116] In some embodiments, the live streaming interactive device 1 further includes a processing module configured to: perform style transfer processing on the material corresponding to the virtual resource based on the information of the live streaming room corresponding to the live streaming screen, using a second machine learning model, to obtain a virtual object.

[0117] In some embodiments, the live interactive device 1 further includes a generation module configured to generate, based on the information of the live room corresponding to the live screen and the virtual object, a third machine learning model to generate the change effect of the virtual object moving from the starting position to the target position.

[0118] In some embodiments, the live interactive device 1 further includes: a generation module configured to: determine a second prompt text based on virtual resources; and generate a second object associated with the first object using a fourth machine learning model based on the first object and the second prompt text.

[0119] In some embodiments, the live interactive device 1 further includes: a determining module configured to determine a designated position outside the live screen based on the exit position of the first object moving out of the live screen from the starting position; determine a connecting line from the designated position to the target position; and determine an entry position based on the intersection of the connecting line and the edge of the live screen.

[0120] Figure 15 A block diagram of an electronic device according to some embodiments of the present disclosure is shown.

[0121] Memory 21 is used to store one or more computer-readable instructions. Memory 21 may include any combination of various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory, including but not limited to random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), read-only memory (ROM), and flash memory. Memory 21 may, for example, store operating systems, application programs, bootloaders, databases, and other programs, as well as various application programs and various data.

[0122] The processor 22 is configured to execute computer-readable instructions to implement the song selection method of any of the foregoing embodiments or the method of any of the foregoing embodiments. Specific implementations of each step of the method can be found in the above embodiments, and repeated details will not be elaborated here.

[0123] Processor 22 can be configured to execute Figure 1 The processor 22 can be various processing devices, such as a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The central processing unit (CPU) can be based on x86 or ARM architectures, etc.

[0124] The processor 22 and the memory 21 can communicate with each other directly or indirectly. For example, the processor 22 and the memory 21 can communicate via a network. The network can include a wireless network, a wired network, and / or any combination of wireless and wired networks. The processor 22 and the memory 21 can also communicate with each other via a system bus, which is not a limitation of this disclosure. The components of the electronic device 2 shown in Figure 15 are merely exemplary and not restrictive. The electronic device 2 may have other components as needed for actual applications. The processor 22 can control other components in the electronic device 2 to perform desired functions.

[0125] Electronic device 2 can be implemented by software, firmware and / or hardware, and can be integrated into a device with the relevant application installed.

[0126] Figure 16 Block diagrams of electronic devices according to other embodiments of the present disclosure are shown.

[0127] Figure 16 The electronic device 3 shown can be a computer system with a dedicated hardware structure, which can perform corresponding functions when the relevant application is installed.

[0128] Electronic devices include, but are not limited to, mobile terminals such as smartphones, laptops, personal digital assistants (PDAs), tablet computers (PCs), PMPs (portable multimedia players), in-vehicle terminals (such as in-vehicle navigation terminals), wearable devices, and fixed terminals such as digital televisions and desktop computers.

[0129] like Figure 16 As shown, the Central Processing Unit (CPU) 31 performs various processes based on a program stored in the Read-Only Memory (ROM) 32 or a program loaded from the storage section 38 into the Random Access Memory (RAM) 33. The RAM 33 stores data required as needed when the CPU 31 performs various processes, etc. The CPU is merely exemplary; it could also be other types of processors, such as the various processors described above. The ROM 32, RAM 33, and storage section 38 can be various forms of computer-readable storage media. It should be noted that although... Figure 16 The diagram shows ROM 32, RAM 33 and storage section 38, but one or more of them may be combined or located in the same or different memory or storage modules.

[0130] CPU 31, ROM 32 and RAM 33 are interconnected via bus 34. Input / output interface 35 is also connected to bus 34.

[0131] The following components are connected to the input / output interface 35: input section 36, such as a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output section 37, including displays such as cathode ray tube (CRT), liquid crystal display (LCD), speakers, vibrators, etc.; storage section 38, including hard disk, magnetic tape, etc.; and communication section 39, including network interface cards such as LAN cards, modems, etc. Communication section 39 allows communication processing via a network such as the Internet. It is easy to understand that, although... Figure 16 The portion of the electronic device 3 shown communicates via bus 34, but it may also communicate via a network or other means, wherein the network may include a wireless network, a wired network, and / or any combination of wireless and wired networks.

[0132] As needed, drive 310 is also connected to input / output interface 35. Removable media 311, such as disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on drive 310 as needed, so that computer programs read from them can be installed into storage section 38 as needed.

[0133] When the above series of processes are implemented through software, the program constituting the software can be installed from a network such as the Internet or a storage medium such as removable medium 311.

[0134] According to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product that, when run on a computer, causes the computer to perform the methods described in any of the foregoing embodiments. The computer program product includes computer instructions carried on a computer-readable medium, containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer instructions can be downloaded and installed from a network via communication section 39, or installed from storage section 38, or installed from ROM 32. When the computer program is executed by CPU 31, the methods of embodiments of this disclosure are performed.

[0135] It should be noted that, in the context of this disclosure, a computer-readable medium can be a tangible medium that may contain or store programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0136] A computer-readable medium may be a computer-readable storage medium, a computer-readable signal medium, or any combination thereof.

[0137] Computer-readable storage media include, but are not limited to, systems, apparatuses, or devices that are electrical, magnetic, optical, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. Computer instructions are stored on the computer-readable storage medium that, when executed by a processor, implement the methods described in any of the foregoing embodiments.

[0138] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0139] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0140] In some embodiments, a computer program is also provided, comprising: instructions that, when executed by a processor, cause the processor to perform the methods described in any of the foregoing embodiments. For example, the instructions may be embodied in computer program code.

[0141] In embodiments of this disclosure, computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0142] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0143] The functions described above can be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary hardware logic components that can be used include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0144] While specific embodiments of this disclosure have been described in detail by way of example, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of this disclosure. Those skilled in the art should understand that modifications can be made to the above embodiments without departing from the scope and spirit of this disclosure. The scope of this disclosure is defined by the appended claims.

Claims

1. A live streaming interactive method, comprising: The live stream is displayed, including identifiers of multiple users displayed in different locations. In response to a target user among the plurality of users obtaining a virtual resource, the changes in the virtual object corresponding to the virtual resource as it moves from the starting position to the target position are displayed, wherein the target position corresponds to the identifier of the target user.

2. The live streaming interaction method according to claim 1, wherein, The virtual object includes a first object and a second object. The display of the changes in the virtual object corresponding to the virtual resource as it moves from the starting position to the target position includes: During the process of the virtual object moving from the starting position to the target position, the effect of the first object transforming into the second object is displayed, wherein the first object and the second object are different.

3. The live streaming interaction method according to claim 1, wherein, The virtual object includes a first object and a second object. The display of the changes in the virtual object corresponding to the virtual resource as it moves from the starting position to the target position includes: During the process of the virtual object moving from the starting position to the target position, the effect of the first object separating from the second object and the second object moving to the target position is displayed.

4. The live streaming interaction method according to claim 1 further includes: Based on the virtual resource, determine the first prompt text; Based on the first prompt text, the first machine learning model is used to perform style transfer processing on the material corresponding to the virtual resource to obtain the virtual object.

5. The live streaming interaction method according to claim 4, wherein, Based on the virtual resource, the first prompt text is determined, including: Obtain the video corresponding to the virtual resource; Select video frames from the video; The first prompt text is generated based on the semantic information of the video frame.

6. The live interactive method according to claim 1 further includes: Based on the information of the live streaming room corresponding to the live streaming screen, the second machine learning model is used to perform style transfer processing on the material corresponding to the virtual resource to obtain the virtual object.

7. The live interactive method according to claim 1 further includes: Based on the information of the live broadcast room corresponding to the live broadcast screen and the virtual object, a third machine learning model is used to generate the change effect of the virtual object as it moves from the starting position to the target position.

8. The live streaming interaction method according to claim 1, wherein, The virtual object includes a first object and a second object, and the live streaming interaction method further includes: Based on the virtual resource, determine the second prompt text; Based on the first object and the second prompt text, a second object associated with the first object is generated using a fourth machine learning model.

9. The live streaming interaction method according to claim 1, wherein, The changes in the virtual object corresponding to the virtual resource as it moves from the starting position to the target position include: As the virtual object moves from the starting position to the target position, the effects of deformation and / or scaling of the virtual object are displayed.

10. The live streaming interaction method according to claim 1, wherein, The virtual object includes a first object and a second object. The display of the changes in the virtual object corresponding to the virtual resource as it moves from the starting position to the target position includes: This displays the effect of the first object moving out of the live stream from the starting position; This displays the effect of the second object moving from its entry position into the live stream to its target position.

11. The live interactive method according to claim 10, further comprising: Based on the exit position of the first object moving out of the live broadcast screen from the starting position, a designated position outside the live broadcast screen is determined; Determine the connecting line from the specified location to the target location; The entry position is determined based on the intersection of the connecting line and the edge of the live stream screen.

12. A live streaming interactive device, comprising: The screen display module is configured to display a live stream, wherein the live stream includes identifiers of multiple users displayed in different locations; The effect display module is configured to, in response to a target user among the plurality of users obtaining a virtual resource, display the change effect of a virtual object corresponding to the virtual resource as it moves from a starting position to a target position, wherein the target position corresponds to the identifier of the target user.

13. An electronic device, comprising: Memory; as well as A processor coupled to the memory, the processor being configured to execute the live interactive method according to any one of claims 1 to 11 based on instructions stored in the memory.

14. A computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the live interactive method according to any one of claims 1 to 11.

15. A computer program product comprising computer program instructions that, when executed by a processor, implement the live interactive method according to any one of claims 1 to 11.

Images