Virtual item processing method, device and system in live broadcast process

Abstract

The invention provides a virtual item processing method and device in a live broadcast process, electronic equipment, a computer readable medium and a program product. A specific embodiment of the invention comprises the following steps: determining respective picture areas of a plurality of live broadcast objects in a live broadcast picture; determining a target position of a presentation operation for the virtual item in the live broadcast picture; and establishing an association relationship between the virtual article and the live broadcast object in the picture area where the target position is located. The invention provides a virtual article presentation mode based on matching between the position and the area, and the convenience and the accuracy of the virtual article presentation process are improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, electronic device, computer-readable medium, and program product for processing virtual items during live streaming. Background Technology

[0002] Current live streaming platforms generally support the "joint live streaming" function, where multiple streamers appear together in the same video stream and interact. In existing technology, when viewers send gifts to streamers in a "joint live stream," they typically need to click on a gift panel, manually select the target streamer, and complete the gifting process. This operation is cumbersome and lacks integration with the live stream feed.

[0003] This section is intended to provide background or context for the embodiments of this application as set forth in the claims. The description herein is not to be construed as prior art simply because it is included in this section. Summary of the Invention

[0004] This application provides a method, apparatus, electronic device, computer-readable storage medium, and program product for processing virtual items during live streaming, which can solve the problem of cumbersome operation in the process of gifting virtual items and improve the convenience and accuracy of the gifting process.

[0005] One aspect of this application provides a method for handling virtual items during a live stream, comprising: determining the respective screen areas of multiple live stream objects in the live stream; determining the target location of a virtual item gifting operation in the live stream; and establishing an association between the virtual item and the live stream object in the screen area where the target location is located.

[0006] Another aspect of this application provides a virtual item processing apparatus during a live stream, comprising: an area determination unit configured to determine the respective screen areas of multiple live stream objects in the live stream screen; a location determination unit configured to determine the target location of a virtual item gifting operation in the live stream screen; and a relationship establishment unit configured to establish an association relationship between the virtual item and the live stream object in the screen area where the target location is located.

[0007] Another aspect of this application provides a live streaming interactive system, comprising: a server for sending a video stream representing a live stream to a client; and a client for: displaying the live stream; determining the respective screen areas of multiple live objects in the live stream; determining the target location of a virtual item gifting operation in the live stream; and establishing an association between the virtual item and the live object in the screen area where the target location is located.

[0008] In another aspect of this application, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the virtual item processing method in the live streaming process as described above.

[0009] Another aspect of this application provides a computer-readable storage medium having computer program instructions stored thereon, which can be executed by a processor to implement the virtual item processing method during the live streaming process as shown above.

[0010] Another aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the virtual item processing method during the live streaming process as described above.

[0011] The solution provided in this application provides a virtual item gifting method based on the matching between location and region by determining the respective screen areas of multiple live objects in the live broadcast screen; determining the target position of the virtual item gifting operation in the live broadcast screen; and establishing the association between the virtual item and the live object in the screen area where the target position is located. This improves the convenience and accuracy of the virtual item gifting process.

[0012] Furthermore, by analyzing the layout description data in the video stream corresponding to the live broadcast and combining it with the normalization of coordinate data, the screen area of ​​each of the multiple live broadcast objects can be determined in real time, improving the accuracy of the screen area and the efficiency of the determination process; multiple area determination methods are provided under weak network conditions to ensure the integrity and effectiveness of the virtual item gifting process; within a preset time period, virtual items given based on gifting operations can be withdrawn, which can avoid the loss of virtual items due to accidental operation and improve the user's sense of security and user experience. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0015] Figure 1 A flowchart illustrating a method for handling virtual items during a live stream, provided as an embodiment of this application;

[0016] Figure 2 This is a schematic diagram of the live broadcast screen of the "joint live broadcast" in this embodiment;

[0017] Figure 3 This is a schematic diagram of a virtual item recall scenario in this embodiment;

[0018] Figure 4 A flowchart illustrating a method for handling virtual items during a live stream, provided as another embodiment of this application;

[0019] Figure 5 A schematic diagram of the structure of a virtual item processing device during live streaming provided in an embodiment of this application;

[0020] Figure 6 This is a schematic diagram of the structure of a live interactive system provided in an embodiment of this application;

[0021] Figure 7 An interactive timing diagram of a live streaming interactive system provided in an embodiment of this application;

[0022] Figure 8 A schematic diagram of the structure of a device suitable for implementing the solutions in the embodiments of this application.

[0023] The same or similar reference numerals in the accompanying drawings represent the same or similar parts. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] In a typical configuration of this application, the terminal and the service network devices each include one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0026] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0027] Computer-readable media include permanent and non-permanent, removable and non-removable media, which can store information by any method or technology. Information can be computer program instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, read-only optical disc (CD-ROM), digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0028] This application provides a method for handling virtual items during live streaming. By determining the respective screen areas of multiple live streaming objects in the live stream; determining the target location of the virtual item gifting operation in the live stream; and establishing the association between the virtual item and the live streaming object in the screen area where the target location is located, a virtual item gifting method based on the matching between location and area is provided, which improves the convenience and accuracy of the virtual item gifting process.

[0029] In practical scenarios, the execution entity of this method can be a user device, a device formed by integrating user devices and network devices through a network, or an application running on the aforementioned devices. The user devices include, but are not limited to, various terminal devices such as computers, mobile phones, tablets, smartwatches, and wristbands. The network devices include, but are not limited to, network hosts, single network servers, multiple network server sets, or cloud computing-based computer sets. Here, the cloud consists of a large number of hosts or network servers based on cloud computing. Cloud computing is a type of distributed computing, consisting of a virtual computer composed of a group of loosely coupled computer sets.

[0030] Figure 1 This application illustrates a processing flow 100 for handling virtual items during a live stream, as provided in an embodiment of this application. The processing flow 100 iteratively executes a filtering operation through the following steps until a preset termination condition is met:

[0031] Step 101: Determine the respective screen areas of multiple live subjects in the live stream.

[0032] In this embodiment, the live stream includes multiple live subjects. Each live subject's screen area can be of the same size or different sizes. For example, the live stream may be a "joint live stream," where multiple live subjects appear together in the same video stream and interact by splicing their images together. (Continue to refer to...) Figure 2 The diagram shows a schematic of the live broadcast screen 200 of the "joint live broadcast". The live broadcast screen 200 includes the screen area 201 of anchor A, the screen area 202 of anchor B, the screen area 203 of anchor C and the screen area 204 of anchor D.

[0033] As an example, based on the splicing characteristics and visual feature analysis of live stream footage, the precise determination of the respective screen regions of multiple live stream objects is achieved through a logical process of "preprocessing - segmentation structure recognition - subject recognition - dynamic verification". The specific steps are as follows:

[0034] Step 1: Live stream preprocessing.

[0035] First, the acquired live video stream undergoes frame synchronization processing to extract continuous and stable video frames as the basis for analysis (avoiding misjudgment of regions due to frame jitter). Second, standardized preprocessing operations are performed: adaptive median filtering is used to reduce noise that may exist in the live video (such as pixel distortion caused by transmission stuttering), preserving the edge and detail features of the image. At the same time, the video frames are uniformly converted to a fixed resolution (such as 1920×1080) to eliminate the impact of differences in output resolution between different live streaming devices on subsequent analysis. Finally, gamma correction (used to perform non-linear calculations or inverse calculations on the luminance or tristimulus values ​​of light in a film or imaging system) is used to adjust the brightness and contrast of the image, ensuring the segmentation boundaries in the image.

[0036] Step Two: Live Stream Segmentation Structure Recognition. The essence of joint live stream image stitching is the visual representation of "physical partitions" or "logical partitions." The core of this step is to identify the partitioning rules:

[0037] Boundary Feature Detection: Edge detection algorithms (such as the Canny operator) are used to scan the screen to identify continuous straight line boundaries with a certain length and width (in hard-slicing scenarios, such as the black borders and white dividing lines of a grid-based split screen). The boundary lines are fitted using the Hough line transform to determine whether the lines are horizontal or vertical (consistent with typical split-screen layout characteristics), and the distribution density and spacing of the lines are statistically analyzed. If there are two sets of mutually perpendicular and evenly distributed boundary lines in the screen, it is determined to be a grid-type splicing (such as a 2×2 split screen or a 3×1 vertical screen splicing), and the preliminary boundary coordinates of each partition are determined based on the intersection points of the lines.

[0038] Non-boundary splicing scene processing: For soft splicing scenes without obvious physical boundaries (such as free splicing through natural partitioning of image content), region clustering analysis is adopted: Based on the color and texture features of pixels, the K-Means clustering algorithm is used to divide the image into several visually similar regions. Combined with the characteristic that "the background style of each live broadcast area is relatively uniform" in the live broadcast image, clustering regions whose area meets the display requirements of the live broadcast object are selected (excluding decorative areas that are too small and global background areas that are too large).

[0039] Stitching type verification: By analyzing the area ratio and position distribution of each preliminary partition (such as a picture-in-picture scene where the main screen occupies more than the sub-screen), the stitching type (grid split screen, picture-in-picture, free stitching, etc.) is determined, and the boundary coordinates of each preliminary partition (such as the pixel coordinates of the upper left and lower right corners) are recorded.

[0040] To verify the accuracy of the partition identification results in step two, the main body of the live broadcast object within the partition can be located in step three.

[0041] The initial zoning only represents the physical division of the image; further steps are needed to determine whether each zoning includes the live-streaming subject. For each initial zoning, a deep learning-based object detection model (focusing on core features of the live-streaming subject such as the human body and face) is used to detect the presence of the live-streaming subject within the zoning. Faces are detected first (facial features are unique and highly recognizable). If a face is detected, the human body contour is determined using the face as the core, combined with human pose detection (such as key point detection). If no clear face is detected (e.g., the live-streaming subject is sideways or shot from a distance), the complete human body region is located using a human detection model.

[0042] Step four: Dynamic adjustment and verification of the screen area. During live streaming, there are dynamic scenarios such as the movement of the live subject and minor adjustments to the screen layout (e.g., slight changes in split-screen ratio). Real-time tracking and verification are necessary to ensure the accuracy of the screen area.

[0043] Real-time tracking: Target tracking algorithms (such as KCF correlation filtering algorithm) are used to continuously track the main features (face, human body key points) of each live object to determine the presence of live objects in the screen area.

[0044] Splicing structure verification: Repeat the boundary detection and cluster analysis in step two at fixed intervals (e.g., 10 frames) to verify whether the overall splicing structure has changed (e.g., switching from 2×2 split screen to 1×3 split screen). If a structural change is detected, the initial partitions are re-divided and the screen areas of all objects are updated.

[0045] Validation: The accuracy of the region is verified through two dimensions: first, the spatial dimension, to ensure that the screen areas of each object do not overlap (the core feature of joint live streaming is that the screen areas of each live streaming object are independent); second, the content dimension, to ensure that the main features of each region remain stable (excluding false detections caused by screen flickering or temporary obstruction). If overlap or unstable features occur, the main object positioning and region adjustment are re-executed.

[0046] For special scenarios such as irregular free splicing, the fixed grid assumption must be abandoned, and the area should be divided based on regional connectivity and subject distribution. If the subject features in a certain area are not related to other areas, and the color and texture features of that area are self-contained, it is determined as an independent live streaming object's screen area. The coordinates of the irregularly shaped screen area are determined by fitting the area boundary using the convex hull algorithm.

[0047] As another example, using "signal source association analysis - scene feature anchoring - dynamic collaborative verification" as the core logic, this method achieves accurate positioning of multiple live streaming object screen areas by mining the association information in the live streaming signal link and the scene stability characteristics of the live streaming objects. The specific steps are as follows:

[0048] Step 1: Analyze the correlation information of the live signal source, prioritizing the mining of native correlation information in the signal link to provide a basic framework for regional division.

[0049] Metadata parsing for streaming: Extracting metadata information of the joint live stream from the streaming server of the live streaming platform, including the independent streaming identifier, streaming resolution, bitrate, frame synchronization timestamp of each live stream object, as well as the platform's preset screen composition rules (such as the officially configured split-screen layout template identifier and the preset proportion of each live stream object's screen). For example, if the platform is configured with a "3 live stream objects vertical screen splicing" template, the preset position (such as the left, center, and right regions) and initial resolution ratio of each live stream object's screen in the composite video frame can be directly obtained, and the boundary range of each live stream object's screen can be initially determined (based on the coordinates calculated from the total resolution after composition).

[0050] Multi-stream synchronization feature extraction: Since joint live streaming needs to ensure the synchronization of frames of multiple live streaming objects, the timestamps of each live streaming object have a fixed correlation. By analyzing the intra-frame features of different regions in the synthesized video frame (such as pixel update frequency and encoding compression traces), matching the encoding features of the corresponding independent streams (such as the compression algorithm identifier of a specific live streaming object), a one-to-one association between the synthesized frame region and the independent stream is established, correcting possible layout deviations in the metadata (such as frame offset caused by the delay of a certain live streaming object's stream).

[0051] Initial region framework construction: Combining the layout parameters of metadata with the matching results of encoding features, the initial region boundaries (top left and bottom right coordinates) of each live object screen are marked in the pixel coordinate system of the synthesized video frame, forming a "signal source-region" association mapping table, which provides a foundation for subsequent refinement.

[0052] Step two, scene feature matching and region refinement. Based on the initial region framework, the stability features of each live streaming object's live streaming scene (such as fixed background and unique identifier) ​​are used for precise matching to refine and verify the region boundaries, avoiding region offset caused by signal metadata errors.

[0053] During the pre-live broadcast warm-up phase (or the first frame), scene features of each individual live stream subject's feed are extracted to establish a dedicated feature library. Features include: a color histogram of a fixed background (such as the color distribution of a blue background wall behind the live stream subject), key anchor points within the scene (such as the microphone position on the live stream subject's table, or the signage behind them), and the common posture outlines of the live stream subject (such as the human proportions when seated). The feature library is updated in real time to adapt to slight changes in the scene (such as adjustments to lighting brightness).

[0054] For each initial region constructed in the first step, the image features within that region are extracted and compared with the scene feature database of each live streaming object for similarity matching (e.g., calculating color histogram similarity using Bach distance, and matching key anchor points using the SIFT (Scale-Invariant Feature Transform) algorithm). If the similarity between the features of a certain region and the feature database of live streaming object A exceeds a preset threshold (e.g., 90%), then that region is confirmed as the image range of live streaming object A, and the initial region boundary is corrected based on the matched key anchor point positions (e.g., if the actual anchor point position is 5 pixels to the left of the initial region boundary, then the left boundary is simultaneously shifted 5 pixels to the left).

[0055] For the matched region, a contour smoothing algorithm is used to process the boundary details. For hard splicing scenarios (such as those with black dividing lines), the pixel position of the dividing line is accurately located by detecting the pixel grayscale value of the dividing line (usually pure black or pure white), and the final region boundary is determined based on the inside of the dividing line. For soft splicing scenarios (without obvious dividing lines), the boundary is determined based on the abrupt change point of the scene features (such as the intersection of the background color of live streaming object A and the background color of live streaming object B) and the pixel gradient change rate, ensuring that the region does not contain scene elements of adjacent live streaming objects.

[0056] In some optional implementations of this embodiment, the execution entity can perform step 101 as follows: parse the layout description data in the video stream corresponding to the live screen to determine the respective screen areas of multiple live objects in the live screen.

[0057] The aforementioned execution entity uses the SEI (Supplemental Enhancement Information) parsing module to parse the layout description data carried in the video stream corresponding to the live broadcast in real time, and obtains the layout structure of the screen area of ​​each of the multiple live broadcast objects, such as coordinates, width, height, proportion, and layer, thereby determining the screen area of ​​each of the multiple live broadcast objects in the live broadcast.

[0058] After determining the respective screen areas of multiple live streaming objects, the correspondence between live streaming objects and screen areas can be converted into the following two-dimensional lookup table structure and cached in the interaction engine:

[0059] Mapping table M = {screen area coordinate range → live stream object ID (Identity Document)}

[0060] For example, the SEI parsing module reads the payload from the SEI data and extracts the layout description structure: [

[0062] {"id":"A","x":0.0,"y":0.0,"w":0.5,"h":0.5,"z":1},

[0063] {"id":"B","x":0.5,"y":0.0,"w":0.5,"h":0.5,"z":1} ]

[0065] The id is a string representing the live object ID corresponding to the screen area; x and y are floats representing the coordinates of the top-left corner of the screen area; w and h are floats representing the width and height ratios of the screen area; and z is an int representing the layer of the screen area.

[0066] In this implementation, the screen area of ​​each live streaming object is accurately located directly through the layout description data, which greatly reduces the calculation cost and the misjudgment rate. It can quickly adapt to multiple layout live streaming scenarios and ensure the real-time and accuracy of live streaming interaction.

[0067] In some optional implementations of this embodiment, the execution entity can determine the respective screen areas of multiple live objects in the live broadcast frame in the following ways:

[0068] The first step is to parse the layout description data in the video stream corresponding to the live broadcast to obtain the coordinates of the screen area corresponding to each of the multiple live broadcast objects.

[0069] Layout description data is structured data embedded in video frames, used to define the layout rules of the live broadcast screen. It includes core information such as the coordinates, quantity, arrangement, and size ratio of the live broadcast object screen area, providing accurate data support for user terminal rendering screen and locating virtual gift placement.

[0070] In this implementation, the aforementioned execution entity can use a parsing module to parse the layout description data carried in the video stream corresponding to the live broadcast in real time, and obtain the layout structure such as the starting coordinates, width, height, and proportion of the screen area of ​​each of the multiple live broadcast objects, thereby determining the screen area coordinates of each of the multiple live broadcast objects in the live broadcast.

[0071] The second step is to normalize the coordinates of the screen area to determine the screen area of ​​each of the multiple live objects in the live broadcast.

[0072] Coordinate normalization is a process that maps the pixel coordinates of different devices to a unified range of (0~1) based on the total width and height of the live screen, in order to eliminate resolution differences.

[0073] The layout description data in the video stream is analyzed, and the original pixel coordinates of each live object's screen area (such as X1, Y1 at the top left corner and X2, Y2 at the bottom right corner) are determined by the width, height, and starting point coordinates. Based on the total width W and total height H of the live screen, normalized coordinates are calculated: X1'=X1 / W, Y1'=Y1 / H, X2'=X2 / W, Y2'=Y2 / H, mapping the coordinates to the (0~1) range. After normalization, regardless of the terminal resolution (such as 720P for mobile phones and 1080P for computers), the screen areas of each object can be proportionally restored, achieving cross-platform coordinate unification and ensuring accurate adaptation of virtual item effects, targeted interactions, and other functions to different devices.

[0074] In this implementation, based on the rapid and accurate acquisition of regional coordinates through layout data, normalization processing is used to eliminate the resolution differences between different devices, thereby achieving cross-platform coordinate unification. This provides precise location support for functions such as virtual item special effects delivery and targeted interaction, and can further ensure the consistency and accuracy of interaction.

[0075] In some optional implementations of this embodiment, the execution entity can perform step 101 as follows:

[0076] In response to the current network condition being weak, at least one of the following methods is used to determine the respective screen areas of multiple live subjects in the live stream:

[0077] Method 1: Determine the respective screen areas of multiple live objects in the live stream based on the screen area in the previous frame of the live stream.

[0078] The SEI information used to carry layout data in the video stream is easily lost in weak network conditions. In this case, instead of waiting for new data, the cached coordinates of the previous frame's area are directly reused. This mechanism prioritizes low latency and ensures the continuity of area data through extremely simple logic.

[0079] For example, in a 2×2 split-screen live stream, if a weak network causes the current frame's SEI transmission to fail, the system directly calls the screen area coordinates of the four live objects cached in the previous frame (such as the normalized coordinates of the top left corner (0,0) and the bottom right corner (0.5,0.5)) to ensure that interactions such as virtual item placement are not interrupted. This is suitable for scenarios where the layout of the live objects does not change for a short period of time.

[0080] Method 2: Determine the respective screen areas of multiple live objects in the live stream based on the changing trends of the screen areas in the multiple frames of the live stream up to the present.

[0081] When the SEI (Sequence Frame Indicator) is periodically lost or delayed, the coordinates of the image region from the most recent N frames (usually 5-8 frames, balancing accuracy and computational load) are extracted. The changing patterns are analyzed using linear fitting or trend interpolation to predict the coordinates of the current frame. This mechanism is suitable for scenarios where the frame area of ​​a live stream object moves slowly and is adjusted gradually, offering higher accuracy than simply reusing the previous frame.

[0082] For example, in a split-screen live stream of a live subject, the screen area of ​​a certain live subject slowly moves from the left side of the live screen to the center. The system extracts the coordinates of the screen area of ​​the last 5 frames, calculates the change trend of the X-axis increasing by 0.05 per frame, and predicts the coordinates of the screen area of ​​the current frame based on this. This avoids the lag in area positioning caused by missing data and is suitable for dynamic scenarios where the area box of the live subject moves slightly and is adjusted slightly.

[0083] Method 3: In response to network transmission latency fluctuations exceeding a preset fluctuation threshold, the respective screen areas of multiple live objects in the live broadcast are determined based on the screen area corresponding to the current locked time period.

[0084] A preset network latency fluctuation threshold (e.g., 50ms) is set. When a sudden increase or decrease in latency exceeding the threshold is detected, it is determined to be regional jitter, triggering a locking period (e.g., 0.3 seconds). Based on the stable coordinates of the last frame before the fluctuation, the region remains unchanged during the locking period. This mechanism avoids interaction misalignment caused by frequent switching through "short-term locking".

[0085] For example, if the latency suddenly spikes from 100ms to 300ms and then drops back under weak network conditions, exceeding the threshold, the system immediately locks the area coordinates of the current live stream object for 0.3 seconds. During this period, even if subsequent frames send offset coordinates, the system will not update them to ensure the accuracy of the virtual item gifting process and avoid accidental gifting.

[0086] This implementation provides three methods for determining the image area in weak network environments. These three methods are adapted to different weak network scenarios and can quickly output stable area coordinates, avoiding positioning lag or drift.

[0087] Step 102: Determine the target location of the virtual item gifting operation in the live stream.

[0088] In this embodiment, virtual items refer to non-physical assets developed and designed by the live streaming platform and existing in digital form. They are given specific usage and interactive attributes through platform rules. They are not only the core interactive medium for users to convey emotions and show support to the live streaming audience, but also have multiple functions such as creating atmosphere and highlighting identity. Based on the differences in the quantity, presentation form and function of virtual resources that can be exchanged for virtual items, they can be divided into basic companion virtual items such as flowers and hearts, customized virtual items such as exclusive items for the live streaming audience and user-defined virtual items, and special effect virtual items such as virtual balloons and lollipops.

[0089] The gifting of virtual items is a digital act in live streaming where users select a target virtual item based on platform rules and complete the delivery of the item to the live stream through specified interactive actions. The core is to connect the user's payment / rights consumption with the live stream participant's benefits / interactive feedback, thereby realizing emotional transmission and scene atmosphere creation.

[0090] For example, a user can complete the gifting of a virtual item with two clicks. The first click is a click (selection) action targeting the virtual item, and the second click is a click (selection) action targeting the live stream subject. The executing entity can determine the click position (target position) of the second click action within the live stream screen.

[0091] In some optional implementations of this embodiment, the gifting operation is a drag-and-drop operation. As an intuitive and precise way to gift virtual items in live streaming scenarios, drag-and-drop is implemented through a closed loop of "selection-drag-delivery," adapting to multi-device interaction habits. Specifically, in the pre-operation, the user clicks the virtual item entry in the live stream room, bringing up the virtual item panel. Draggable virtual items are displayed as icons in the panel, labeled with their names and quantities. Simultaneously, each area of ​​the live stream screen will display a slight highlight, clearly indicating the delivery range.

[0092] During the drag-and-drop operation, the user long-presses the target virtual item icon to trigger dragging (long-press with finger on mobile devices and long-press with mouse on PC). Preset animation effects can be set during dragging, such as the icon becoming a semi-transparent preview state and following the operation cursor / finger movement; when the preview state enters the screen area of ​​a live object, the screen area is highlighted and a "can be gifted" prompt is displayed, and releasing the finger / mouse completes the delivery.

[0093] In weak network environments, preloading drag-and-drop preview resources prevents lag; if the drag is released to an area other than the live stream object, a pop-up window will prompt "Please drag to the live stream object area" and cancel the operation; mobile devices support scaling the preview icon during dragging to improve accuracy.

[0094] In this implementation, the aforementioned execution entity can perform step 102 as follows: determine the target position of the release point of the drag-and-drop operation in the live broadcast screen.

[0095] The system tracks the coordinates of the drag-and-drop preview icon in real time and correlates them with the normalized coordinate system of the live stream. At the moment of release during the drag-and-drop operation, the system accurately captures the real-time coordinates of the center of the preview icon, corrects for deviations using the normalized coordinate system, and finally determines that the center coordinates are the position of the release point in the live stream.

[0096] In this implementation, users can gift virtual items by dragging and dropping, which further improves the convenience of the gifting process and enhances the interactivity between users and the live stream.

[0097] Step 103: Establish the association between the virtual item and the live-streaming object in the screen area where the target location is located.

[0098] In this embodiment, firstly, the screen area where the target location is located is determined from multiple screen areas that correspond one-to-one with multiple live streaming objects; then, the association relationship between the virtual item and the live streaming object in the screen area where the target location is located is established.

[0099] For multiple screen regions in a live broadcast, the coordinate range of the screen region is matched with the coordinates of the target location. That is, it is determined whether the coordinates of the target location are included in the coordinate range of the screen region. In response, the screen region is determined as the screen region where the target location is located.

[0100] For example, for a target location (x_t, y_t), find the record in the mapping table M that satisfies (x_t, y_t) belongs to region R_i:

[0101] if (x_t ∈ [x_i, x_i+w_i]) && (y_t ∈ [y_i, y_i+h_i]):

[0102] target_id = id_i

[0103] Then, identify the live stream object within the screen area where the target location is located, and store the virtual item ID, live stream object ID, and gift timestamp in the database, while simultaneously generating associated credentials.

[0104] In some optional implementations of this embodiment, the execution entity can perform step 103 as follows:

[0105] The first step is to determine the matching area corresponding to the screen area of ​​each of the multiple live streaming objects based on the amount of virtual resources exchanged for virtual items.

[0106] The number of virtual resources is negatively correlated with the size of the matching region.

[0107] The matching area corresponding to the screen area must at least partially overlap with the screen area; for example, the matching area corresponding to the screen area contains the screen area. The larger the number of virtual resources corresponding to a virtual item, the smaller the size of the matching area; the smaller the number of virtual resources corresponding to a virtual item, the larger the size of the matching area.

[0108] For example, for items with a large number of virtual resources, the matching area of ​​the screen region is set to be smaller than the screen region and contained within the screen region; for items with a small number of virtual resources, the screen region is set to be larger than the matching area corresponding to the screen region and contained within the screen region.

[0109] The second step is to establish a relationship between the virtual item and the live stream object corresponding to the target matching area, in response to the target location being within the target matching area.

[0110] For each matching region corresponding to multiple screen areas in the live broadcast, the coordinate range of the matching region is matched with the coordinates of the target position. That is, it is determined whether the coordinates of the target position are included in the coordinate range of the matching region. In response, the matching region is determined as the target matching region.

[0111] In response to the target location being within the target matching area, the system confirms that the user has gifted the virtual item to the live stream object in the screen area corresponding to the target matching area, and establishes an association between the virtual item and the live stream object corresponding to the target matching area.

[0112] In this implementation, matching areas corresponding to the screen areas of multiple live streaming objects are determined based on the number of virtual resources exchanged for virtual items. Larger matching areas are set for items with smaller numbers of virtual resources to further improve the efficiency and convenience of the gifting process; smaller matching areas are set for items with larger numbers of virtual resources to ensure the accuracy of the identified live streaming objects while maintaining the convenience of gifting.

[0113] In some optional implementations of this embodiment, the execution entity may also perform the following operation: display a preset movement animation of the virtual item from the target location to a specified location in the screen area corresponding to the target matching area.

[0114] When the virtual item has entered the matching area at the target location, the aforementioned execution entity can predict that the target of the virtual item is the live stream object in the screen area corresponding to the matching area. The user can then release the drag operation, and the aforementioned execution entity can display a preset movement animation of the virtual item from the target location to the specified position in the screen area corresponding to the target matching area.

[0115] It is understandable that in some situations, users may pass through the matching area that is not the target live streaming object during the drag-and-drop operation. In this case, users only need to continue the drag-and-drop operation without releasing the virtual item.

[0116] The designated location, such as the center of the area, displays a "magnetic" effect of virtual items moving from the target location to the designated location based on a preset movement animation. Furthermore, when a virtual item icon enters the matching area, the corresponding screen area is highlighted, for example, the screen area is slightly enlarged (approximately 3%–5%), and light effects or pulsating halos are generated at the edges.

[0117] In this implementation, a preset movement animation of the virtual item from the target location to a specified position in the screen area corresponding to the target matching area is used to achieve a "magnetic attraction" effect of the virtual item in the live broadcast screen, which can further improve the user experience and viewing effect.

[0118] In some optional implementations of this embodiment, the execution entity may also perform the following operations:

[0119] The first step is to retain the amount of virtual resources in the account of the user who performed the gifting operation that were used to exchange for virtual items, within a preset time period starting from the completion time of the gifting operation.

[0120] The duration of the preset time period can be flexibly set according to the actual situation, for example, 5 seconds.

[0121] Taking drag-and-drop operation as an example, after a user releases a virtual item, the aforementioned executing entity does not immediately deduct the amount of virtual resources used to exchange for virtual items from the account of the user who performed the gifting operation, but instead retains them for a preset time period (the duration of the preset time period).

[0122] The second step involves canceling the association between the virtual item and the live-streamed object in the screen area where the target location is located, based on the revocation operation for the virtual item.

[0123] In response to receiving a revocation operation for a virtual item within a preset time period, the association between the virtual item and the live-streamed object in the screen area where the target location is located is revoked.

[0124] Continue to refer to Figure 3The diagram illustrates a scenario 300 where a virtual item is withdrawn. In the screen area corresponding to the live stream object, a virtual button 301 representing the withdrawal intention can be displayed, showing the remaining time for the withdrawal operation.

[0125] In this implementation, the right to withdraw virtual resources is reserved for a preset time, supporting withdrawal and revocation of association. This can correct operations such as accidental gifting, reduce unnecessary losses for users, and improve the user experience. At the same time, it can protect users' rights and interests in virtual resources, enhance users' trust in the platform, and reduce the risk of disputes.

[0126] In some optional implementations of this embodiment, the execution entity may also perform the following operations:

[0127] The first step is to determine the movement trajectory of the virtual gift in the live stream based on the gifting action. The second step is to display the preset gifting animation based on the movement trajectory.

[0128] As an example, the aforementioned execution entity automatically identifies the pixel position of the button of the currently clicked virtual item, takes the geometric center of the button as the starting point of the animation, ensures that the starting point is completely aligned with the user's operation position, avoids the feeling of "clicking the edge of the button but starting from the center", and improves the smoothness of operation.

[0129] Then, the normalized coordinates of the screen area of ​​the live object corresponding to the virtual item are obtained in real time, converted into pixel coordinates of the current device, and the geometric center of the area is taken as the endpoint. Even if the screen area of ​​the live object is slightly adjusted, the endpoint will be updated synchronously to ensure the "precise landing area" of the virtual item.

[0130] Then, the trajectory is calculated using a second-order Bézier curve. The curvature parameters are preset according to the type of virtual object. Lightweight virtual objects (such as flowers) have a gentle arc to simulate natural falling; high-order virtual objects (such as rockets) have a steep arc and are accompanied by an increase in speed to enhance the sense of impact. The trajectory is smooth and jagged throughout.

[0131] Upon reaching the destination, particle effects matching the virtual items are triggered. For example, the heart-shaped virtual item blooms with pink light spots and scatters in all directions, while the rocket-shaped virtual item produces a golden spark trail before exploding and dissipating. The effects last for 0.8-1.2 seconds, and the brightness is adapted to the brightness of the video screen to avoid overexposure.

[0132] When the special effect is triggered, a semi-transparent, high-brightness mask with 50% transparency is superimposed on the target area. The shape of the mask perfectly matches the frame of the live streaming object area. It fades away after 0.3 seconds, allowing users to intuitively perceive that "the virtual item has been delivered to the target".

[0133] The animation layer and video layer are rendered separately to avoid animation calculations consuming video decoding resources; the video player rendering callbacks are used to composite the video in real time at the OpenGL layer, achieving a high frame rate of 60 frames per second, so that the animation and video will not appear even under weak network conditions.

[0134] In this implementation, the use of preset gift animations helps to further enhance the user experience and interactivity with the live stream.

[0135] Figure 4 This application illustrates another processing flow 400 of a method for handling virtual items during a live stream, as provided in an embodiment of this application. Processing flow 400 includes the following steps:

[0136] Step 401: Parse the layout description data in the video stream corresponding to the live broadcast to obtain the coordinates of the screen area corresponding to each of the multiple live broadcast objects.

[0137] Step 402: Normalize the screen area coordinates to determine the screen area of ​​each of the multiple live objects in the live screen.

[0138] Step 403: Determine the target location in the live stream screen where the virtual item gifting operation will be released.

[0139] Step 404: Establish the association between the virtual item and the live-streaming object in the screen area where the target location is located.

[0140] Step 405: In response to the fact that the current time is within a preset time period starting from the completion time of the gifting operation, retain the amount of virtual resources in the account of the user who performed the gifting operation that were used to exchange for virtual items.

[0141] Step 406: Based on the withdrawal operation for the virtual item, cancel the association between the virtual item and the live broadcast object in the screen area where the target location is located.

[0142] As can be seen from this embodiment, with Figure 1 Compared with the corresponding embodiments, the process of determining the specific screen area and withdrawing the virtual item in the live broadcast process in this embodiment further improves the flexibility, convenience and operability of the item gifting process.

[0143] Furthermore, this application also provides a virtual item processing device during live streaming, the structure of which is as follows: Figure 5 As shown.

[0144] A virtual item processing device 500 for live streaming includes: a region determination unit 501 configured to determine the respective screen regions of multiple live streaming objects in the live streaming screen; a position determination unit 502 configured to determine the target position of a virtual item gifting operation in the live streaming screen; and a relationship establishment unit 503 configured to establish an association relationship between the virtual item and the live streaming object in the screen region where the target position is located.

[0145] In some optional implementations of this embodiment, the region determination unit 501 is further configured to: parse the layout description data in the video stream corresponding to the live broadcast image, and determine the respective image regions of multiple live broadcast objects in the live broadcast image.

[0146] In some optional implementations of this embodiment, the region determination unit 501 is further configured to: parse the layout description data in the video stream corresponding to the live screen to obtain the screen region coordinates corresponding to each of the multiple live objects; normalize the screen region coordinates to determine the screen region of each of the multiple live objects in the live screen.

[0147] In some optional implementations of this embodiment, the region determination unit 501 is further configured to: in response to the current network state being a weak network state, determine the respective screen regions of multiple live objects in the live broadcast frame using at least one of the following methods: determine the respective screen regions of multiple live objects in the live broadcast frame based on the screen regions in the previous frame of the live broadcast frame; determine the respective screen regions of multiple live objects in the live broadcast frame based on the changing trends of the screen regions in the multiple frames of the live broadcast frame up to the current time; in response to the network transmission delay fluctuation exceeding a preset fluctuation threshold, determine the respective screen regions of multiple live objects in the live broadcast frame based on the screen regions corresponding to the locked time period at the current moment.

[0148] In some optional implementations of this embodiment, the gifting operation is a drag-and-drop operation, and the position determination unit 502 is further configured to: determine the target position of the release point of the drag-and-drop operation in the live broadcast screen.

[0149] In some optional implementations of this embodiment, the relationship establishment unit 503 is further configured to: determine the matching area corresponding to the screen area of ​​each of the multiple live objects according to the number of virtual resources of the virtual items, wherein the number of virtual resources is negatively correlated with the size of the matching area; and establish an association relationship between the virtual items and the live objects corresponding to the target matching area in response to the target location being in the target matching area.

[0150] In some optional implementations of this embodiment, the above-mentioned device further includes: a display unit (not shown in the figure) configured to display a preset movement animation of a virtual item from a target location to a specified position in the screen area corresponding to the target matching area.

[0151] In some optional implementations of this embodiment, the above apparatus further includes: a resource retention unit (not shown in the figure), configured to retain the amount of virtual resources in the account of the user who performed the gifting operation for exchanging virtual items within a preset time period starting from the completion time of the gifting operation; and an item withdrawal unit (not shown in the figure), configured to cancel the association between the virtual item and the live broadcast object in the screen area where the target location is located according to the withdrawal operation for the virtual item.

[0152] In some optional implementations of this embodiment, the above-mentioned apparatus further includes: a trajectory determination unit (not shown in the figure) configured to determine the movement trajectory of the virtual gift in the live broadcast screen according to the gifting operation; and a display unit further configured to display a preset gifting animation of the virtual gift according to the movement trajectory.

[0153] In the virtual item processing device during live streaming provided in this application embodiment, the area determination unit determines the respective screen areas of multiple live streaming objects in the live streaming screen; the position determination unit determines the target position of the virtual item gifting operation in the live streaming screen; and the relationship establishment unit establishes the association relationship between the virtual item and the live streaming object in the screen area where the target position is located, thereby providing a virtual item gifting method based on the matching between position and area, which improves the convenience and accuracy of the virtual item gifting process.

[0154] Furthermore, this application also provides a live streaming interactive system, the structure of which is as follows: Figure 6 As shown.

[0155] A live streaming interactive system 600 includes: a server 601 for sending a video stream representing a live stream to a client; and a client 602 for: displaying the live stream; determining the respective screen areas of multiple live objects in the live stream; determining the target location of a virtual item gifting operation in the live stream; and establishing a relationship between the virtual item and the live object in the screen area where the target location is located.

[0156] In some optional implementations of this embodiment, the client 602 is further used to parse the layout description data in the video stream corresponding to the live broadcast screen to determine the respective screen areas of multiple live broadcast objects in the live broadcast screen.

[0157] In some optional implementations of this embodiment, the client 602 is further configured to: parse the layout description data in the video stream corresponding to the live broadcast to obtain the screen area coordinates corresponding to each of the multiple live broadcast objects; normalize the screen area coordinates to determine the screen area of ​​each of the multiple live broadcast objects in the live broadcast.

[0158] In some optional implementations of this embodiment, the client 602 is further configured to: in response to the current network state being a weak network state, determine the respective screen areas of multiple live objects in the live broadcast frame using at least one of the following methods: determine the respective screen areas of multiple live objects in the live broadcast frame based on the screen areas in the previous frame of the live broadcast frame; determine the respective screen areas of multiple live objects in the live broadcast frame based on the changing trends of the screen areas in the multiple frames of the live broadcast frame up to the current time; in response to the network transmission delay fluctuation exceeding a preset fluctuation threshold, determine the respective screen areas of multiple live objects in the live broadcast frame based on the screen area corresponding to the locked time period at the current moment.

[0159] In some optional implementations of this embodiment, the gifting operation is a drag-and-drop operation, and the client 602 is further used to: determine the target position of the release point of the drag-and-drop operation in the live broadcast screen.

[0160] In some optional implementations of this embodiment, the client 602 is further configured to: determine the matching area corresponding to the screen area of ​​each of the multiple live objects according to the number of virtual resources of the virtual items being replaced, wherein the number of virtual resources is negatively correlated with the size of the matching area; and establish an association between the virtual items and the live objects corresponding to the target matching area in response to the target location being in the target matching area.

[0161] In some optional implementations of this embodiment, the client 602 is also used to display a preset movement animation of the virtual item from the target location to a specified location in the screen area corresponding to the target matching area.

[0162] In some optional implementations of this embodiment, the client 602 is further configured to, in response to the current time being within a preset time period starting from the completion time of the gifting operation, retain the amount of virtual resources in the account of the user who performed the gifting operation that are used to exchange for virtual items; and, according to the withdrawal operation for the virtual items, cancel the association between the virtual items and the live broadcast object in the screen area where the target location is located.

[0163] In some optional implementations of this embodiment, the client 602 is further configured to: determine the movement trajectory of the virtual gift in the live broadcast screen based on the gifting operation; and display a preset gifting animation of the virtual gift based on the movement trajectory.

[0164] Continue to refer to Figure 7 The diagram illustrates the interaction sequence of a live streaming interactive system. The system specifically includes a streaming media server 701, a user terminal 702, and a service server 703. The streaming media server 701 and service server 703 correspond to the aforementioned server-side components, and the user terminal 702 corresponds to the aforementioned client-side component.

[0165] 1. The streaming media server pushes live video streams to user terminals.

[0166] 2. After receiving the video stream, the user terminal parses the video stream and its layout description data, renders and displays the complete live broadcast screen (including the screen area of ​​each live object).

[0167] 3. The user terminal initiates a virtual item configuration request to the business server.

[0168] 4. The business server returns data such as the list of virtual items, the quantity of virtual resources, and icons. The terminal loads and displays the virtual item panel.

[0169] 5. When a user performs a drag-and-drop operation on a virtual item on the terminal, the terminal tracks the drag trajectory in real time and determines the release point.

[0170] 6. Report a gift request to the business server, including core data such as user account ID, virtual item ID, target live stream object screen area identifier, and release point coordinates.

[0171] 7. After receiving the request, the business server verifies the user account's virtual resource balance and the validity of the operation, and completes the deduction and settlement of virtual resources (such as accumulating the corresponding revenue for the live streaming object).

[0172] 8. The user terminal establishes the association between the virtual item and the target live streaming object, and generates an operation result response (including success flag and animation trigger parameters).

[0173] 9. The user terminal triggers the flight animation module to perform trajectory calculation and special effects rendering, simultaneously compositing and displaying the animation layer and video layer locally. In this embodiment, the client determines the respective screen areas of multiple live objects in the live stream; determines the target location of the virtual item gifting operation in the live stream; and establishes the association between the virtual item and the live object in the screen area where the target location is located. This provides a virtual item gifting method based on location and area matching, improving the convenience and accuracy of the virtual item gifting process.

[0174] Based on the same inventive concept, this application also provides an electronic device. The method corresponding to the electronic device can be the virtual item processing method in the live streaming process of the aforementioned embodiments, and its problem-solving principle is similar to that method. The electronic device provided in this application includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the methods and / or technical solutions of the various embodiments of this application.

[0175] The aforementioned electronic devices can be user devices, or devices composed of user devices and network devices integrated through a network, or applications running on such devices. The user devices include, but are not limited to, various terminal devices such as computers, mobile phones, tablets, smartwatches, and wristbands. The network devices include, but are not limited to, network hosts, single network servers, multiple network server sets, or cloud computing-based computer sets, and can be used to implement some processing functions when setting an alarm clock. Here, the cloud consists of a large number of hosts or network servers based on cloud computing. Cloud computing is a type of distributed computing, consisting of a virtual computer composed of a group of loosely coupled computer sets.

[0176] Figure 8 The diagram illustrates the structure of an apparatus suitable for implementing the methods and / or technical solutions in the embodiments of this application. The apparatus 800 includes a Central Processing Unit (CPU) 801, which can perform various appropriate actions and processes based on a program stored in a Read Only Memory (ROM) 802 or a program loaded from a storage portion 808 into a Random Access Memory (RAM) 803. The RAM 803 also stores various programs and data required for system operation. The CPU 801, ROM 802, and RAM 803 are interconnected via a bus 804. An Input / Output (I / O) interface 805 is also connected to the bus 804.

[0177] The following components are connected to I / O interface 805: an input section 806 including a keyboard, mouse, touchscreen, microphone, infrared sensor, etc.; an output section 807 including a cathode ray tube (CRT), liquid crystal display (LCD), LED display, OLED display, etc., and speakers, etc.; a storage section 808 including one or more computer-readable media such as hard disk, optical disk, magnetic disk, semiconductor memory, etc.; and a communication section 809 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 809 performs communication processing via a network such as the Internet.

[0178] In particular, the methods and / or embodiments in this application can be implemented as computer software programs. For example, the embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. When the computer program is executed by a central processing unit (CPU) 801, it performs the functions defined in the methods of this application.

[0179] Another embodiment of this application provides a computer-readable storage medium having computer program instructions stored thereon, which can be executed by a processor to implement the methods and / or technical solutions of any one or more embodiments of this application.

[0180] Specifically, this embodiment may employ any combination of one or more computer-readable media. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0181] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including—but not limited to—electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0182] The program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0183] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0184] The flowcharts or block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-specific system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0185] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0186] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or page components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units through some interfaces, and may be electrical, mechanical, or other forms.

[0187] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0188] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0189] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0190] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

[0191] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a device claim may also be implemented by a single unit or device through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any specific order.

Claims

1. A method for handling virtual items during live streaming, wherein, The method includes: Determine the respective screen areas of multiple live subjects in the live stream; Determine the target location of the virtual item gifting operation within the live stream screen; Establish the association between the virtual item and the live stream object in the screen area where the target location is located.

2. The method according to claim 1, wherein, Determining the respective screen areas of multiple live subjects in the live stream includes: The layout description data in the video stream corresponding to the live broadcast is analyzed to determine the respective screen areas of the multiple live broadcast objects in the live broadcast.

3. The method according to claim 2, wherein, The step of parsing the layout description data in the video stream corresponding to the live stream to determine the respective screen areas of multiple live objects in the live stream includes: The layout description data in the video stream corresponding to the live broadcast image is parsed to obtain the screen area coordinates corresponding to each of the multiple live broadcast objects. The screen area coordinates are normalized to determine the respective screen areas of the multiple live objects in the live screen.

4. The method according to claim 1, wherein, Determining the respective screen areas of multiple live subjects in the live stream includes: In response to the current network being in a weak network state, the respective screen areas of the multiple live streaming objects in the live streaming screen are determined using at least one of the following methods: The respective screen areas of the multiple live objects in the live frame are determined based on the screen area in the previous frame of the live frame. Based on the changing trends of the screen regions in the multiple frames of the live stream up to the current time, the screen regions of each of the multiple live stream objects in the live stream are determined. In response to network transmission latency fluctuations exceeding a preset fluctuation threshold, the respective screen areas of multiple live streaming objects in the live streaming screen are determined based on the screen area corresponding to the locked time period at the current moment.

5. The method according to claim 1, wherein, The gifting operation is a drag-and-drop operation, and Determining the target location of the virtual item gifting operation within the live stream screen includes: Determine the target location of the release point of the drag-and-drop operation in the live stream screen.

6. The method according to claim 1, wherein, The process of establishing the association between the virtual item and the live-streamed object in the screen area where the target location is located includes: Based on the number of virtual resources used to replace the virtual items, a matching region is determined for each of the screen areas of the multiple live streaming objects, wherein the number of virtual resources is negatively correlated with the size of the matching region; In response to the target location being within the target matching area, an association is established between the virtual item and the live stream object corresponding to the target matching area.

7. The method according to claim 1, wherein, Also includes: A preset movement animation is displayed, showing the virtual item moving from the target location to a specified location in the screen area corresponding to the target matching area.

8. The method according to claim 1, wherein, Also includes: In response to the fact that the current time is within a preset time period starting from the completion time of the gifting operation, the amount of virtual resources in the account of the user who performed the gifting operation used to exchange for the virtual items is retained; Based on the revocation operation for the virtual item, the association between the virtual item and the live stream object in the screen area where the target location is located is revoked.

9. The method according to any one of claims 1-8, wherein, Also includes: The movement trajectory of the virtual gift in the live broadcast screen is determined based on the gifting operation; The virtual gift is presented with a preset gift-giving animation based on the movement trajectory.

10. A virtual item processing device during live streaming, wherein, The device includes: The region determination unit is configured to determine the respective screen regions of multiple live objects in the live stream; The location determination unit is configured to determine the target location of the virtual item gifting operation in the live broadcast screen; The relationship establishment unit is configured to establish an association between the virtual item and the live streaming object in the screen area where the target location is located.

11. A live streaming interactive system, wherein, The system includes: The server-side component is used to send the video stream representing the live broadcast to the client. The client is configured to: display the live stream; determine the respective screen areas of multiple live stream objects in the live stream; determine the target location of the virtual item gifting operation in the live stream; and establish an association between the virtual item and the live stream object in the screen area where the target location is located.

12. An electronic device, the electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 9.

13. A computer-readable medium having stored thereon computer program instructions that can be executed by a processor to implement the method as described in any one of claims 1 to 9.

14. A computer program product comprising a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 9.

Images