Video stream output method and apparatus

Abstract

Embodiments of the present application provide a video stream output method and device, computer equipment, computer readable storage medium and computer program product, belonging to the technical field of video. The method comprises: obtaining event data corresponding to an interactive live process and real-time media frame data of at least two virtual roles; generating interactive semantic state information representing the interactive process based on the event data; generating role image data of each virtual role based on the real-time media frame data; determining a target position of each virtual role in a pre-configured stage space coordinate system based on the interactive semantic state information, determining display parameters corresponding to the target position; the stage space coordinate system corresponds to a display plane with a space transformation parameter; performing spatial reconstruction on the role image data based on the target position, the display parameters and the space transformation parameter; and outputting a synthesized video stream of the spatial reconstruction result. The embodiments of the present application can improve the spatial expression capability and the confrontation performance of the virtual anchor interactive scene.

Description

Technical Field

[0001] This application relates to the field of video technology, and in particular to a video stream output method, apparatus, computer device, computer-readable storage medium, and computer program product. Background Technology

[0002] With the development of live streaming interaction and virtual character (such as virtual anchors / digital humans) technology, live streaming platforms have seen a large number of real-time playback scenarios featuring content presented by virtual characters. In interactive (PK) live streams and other competitive interactions, it is usually necessary to composite the real-time footage of at least two virtual characters into the same output video stream so that viewers can intuitively perceive the process of the competition and the changes in the scene on the same display plane. The current common practice is to process live event data (such as gifts, interactions, and interaction timelines) and real-time video frames, extract the character images, and overlay them onto the background for output. However, as the interaction progresses, the positions and displays of the characters on the unified stage are relatively fixed. If the virtual characters of both sides are displayed in a fixed split screen or a simple side-by-side manner, the viewing experience is poor.

[0003] It should be noted that the above content is not necessarily prior art, nor is it intended to limit the scope of patent protection of this application. Summary of the Invention

[0004] This application provides a video stream output method, apparatus, computer device, computer-readable storage medium, and computer program product to solve or alleviate one or more of the technical problems mentioned above.

[0005] One aspect of this application provides a video stream output method, the method comprising:

[0006] Acquire event data corresponding to the interactive live broadcast process, as well as real-time media frame data of at least two virtual characters; Based on the event data, interactive semantic state information representing the interactive process is generated; Generate character image data for each virtual character based on the real-time media frame data; Based on the interactive semantic state information, the target position of each virtual character in the pre-configured stage space coordinate system is determined, and the display parameters corresponding to the target position are determined; the stage space coordinate system and the display plane have corresponding spatial transformation parameters; Based on the target position, the display parameters, and the spatial transformation parameters, spatial reconstruction is performed on the character image data; The synthesized video stream is the result of the output spatial reconstruction.

[0007] Optionally, acquiring real-time media frame data of at least two virtual characters includes: Acquire at least two video frames; Add a frame timestamp to each of the video frames in the aforementioned stream; Based on the timestamp, the timestamps of each event in the event data are aligned with those of each video frame.

[0008] Optionally, the generation of interactive semantic state information includes: The interaction state is obtained by updating the interaction state based on the event data; Calculate the advantage value based on the event data; The output includes at least the interaction semantic state information of the interaction state and the advantage value.

[0009] Optionally, it also includes the operation of generating the spatial transformation parameters: Generate mapping parameters from stage coordinates to display pixel coordinates; Generate scaling parameters related to depth Z so that different depths correspond to different display sizes.

[0010] Optionally, determining the target station location includes: The interactive semantic state information is input into the station mapping model or station mapping function to obtain the target station. Output the coordinates of the target position in the stage space coordinate system.

[0011] Optionally, the station mapping model or station mapping function includes station rules that switch according to the interaction state, and the station rules include at least: In the initialization state, at least two virtual characters are set to symmetrical initial positions; During periods of stability or stalemate, the X-axis position and / or Z-axis depth of the leading and lagging players are adjusted differentially based on their advantage values. During the climax, the leading team's position is changed to the center or foreground area of ​​the stage, while the lagging team's position is changed to the edge or background area of ​​the stage.

[0012] Optionally, the display parameters include at least one of translation parameters, scaling parameters, and / or hierarchy parameters, and the display parameters are determined based on the interactive semantic state information.

[0013] Optionally, character image data for each virtual character is generated based on the real-time media frame data, including: Perform keying or segmentation processing on the real-time media frame data, or extract the region image containing the character; Output character image data containing alpha channels, or output character region image data for rendering.

[0014] Optionally, the method further includes: For each of the character image data streams, extract character semantic metadata for stage reconstruction; The semantic metadata of the character is associated with the corresponding image data of the character.

[0015] Optionally, determining the display parameters includes: Based on the interactive semantic state information, the display parameters are determined, and the character display size reference parameters are determined based on the character semantic metadata. The character's orientation correction parameters and / or perspective compensation parameters are determined based on the character's semantic metadata.

[0016] Optionally, the spatial reconstruction includes: The character image data is either horizontally flipped or left unchanged based on the orientation correction parameters. The character image data is scaled based on the display size reference parameter and the scaling parameter. When the perspective compensation parameters meet the preset conditions, vertical compression compensation is applied to the background characters.

[0017] Optionally, the spatial reconstruction further includes: Establish a collision detection zone for each virtual character; Detect whether the collision detection areas of different virtual characters intersect; In the case of intersection, the target position and / or display parameters of at least one virtual character should be adjusted to avoid collisions. The collision detection area is a bounding box, which includes at least a center point and half-width and half-height parameters.

[0018] Optionally, the output synthesized video stream includes: Based on the depth Z of the target station or the level parameter, sort the virtual characters by level; The occlusion relationship is determined based on the sorting results, and the rendering order is generated.

[0019] Optionally, before outputting the synthesized video stream, the method further includes: Read stage configuration data; Load stage background resources and initialize the rendering pipeline; The stage configuration data includes at least stage space coordinate system parameters and / or the space transformation parameters.

[0020] Optionally, it also includes: The target shot is determined based on the interactive semantic state information; Virtual camera parameters are generated based on the target lens; Update the spatial transformation parameters and / or the display parameters based on the virtual camera parameters; Based on the updated spatial transformation parameters and / or display parameters, the synthesized video stream is output.

[0021] Optionally, determining the target shot based on the interactive semantic state information includes: Obtain audience attention information; When the audience attention information indicates that the audience should focus on one side of the interaction, the close-up shot of the one side of the interaction is determined as the target shot; If the audience attention information does not indicate unilateral attention, the target shot is determined from a preset shot set based on the interaction state.

[0022] Optionally, generating virtual camera parameters based on the target lens includes: Determine the camera position, gaze point, field of view, and / or aspect ratio in the virtual camera parameters; Apply interpolation update rules to the camera position and / or the gaze point.

[0023] Optionally, it also includes camera switching control operations: In the absence of a manual camera switching instruction, the target camera is determined based on a preset director decision tree; If the time since the last shot change is less than a preset duration threshold, the current shot remains unchanged.

[0024] Another aspect of this application provides a video stream output device, the device comprising: The acquisition module is used to acquire event data corresponding to the interactive live broadcast process and real-time media frame data of at least two virtual characters; The first generation module is used to generate interactive semantic state information representing the interactive process based on the event data; The second generation module is used to generate character image data for each virtual character based on the real-time media frame data. The determination module is used to determine the target position of each virtual character in a pre-configured stage space coordinate system based on the interactive semantic state information, and to determine the display parameters corresponding to the target position; the stage space coordinate system and the display plane have spatial transformation parameters corresponding to each other; The reconstruction module is used to perform spatial reconstruction on the character image data based on the target position, the display parameters, and the spatial transformation parameters. The output module is used to output the synthesized video stream of the spatial reconstruction results.

[0025] Another aspect of this application provides a computer device, including: At least one processor; and A memory that is communicatively connected to the at least one processor; Wherein: the memory stores instructions that can be executed by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method as described above.

[0026] Another aspect of this application provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the method described above.

[0027] Another aspect of this application provides a computer program product including a computer program that, when executed by a processor, implements the method described above.

[0028] The embodiments of this application employing the above technical solution may include the following advantages: First, event data corresponding to the interactive live broadcast process and real-time media frame data of at least two virtual characters are acquired. Based on the event data, interactive semantic state information characterizing the interactive confrontation process is generated, enabling subsequent stage layout and screen presentation to be dynamically driven by the semantics of the confrontation process, avoiding reliance solely on fixed positions or static synthesis unrelated to the confrontation. Further, based on the interactive semantic state information, the target position of each virtual character in a pre-configured stage space coordinate system is determined, and the display parameters corresponding to the target position are determined. Simultaneously, the spatial transformation parameters corresponding to the stage space coordinate system and the display plane are used to map the target position and display parameters to the presentation result on the display plane in a deterministic manner. Subsequently, based on the target position, display parameters, and spatial transformation parameters, spatial reconstruction is performed on the character image data, so that the position, scale, and hierarchy of each character in the same output screen are updated according to changes in the interactive semantic state. Therefore, this technical solution can establish a processing link in interactive live streaming scenarios, namely event data → interactive semantic state → target position / display parameters → spatial reconstruction → synthetic video stream output. This enables the spatial relationships in the virtual stage to change dynamically with the interaction process, and intuitively maps the confrontation state into changes in spatial layout. This enhances the spatial expression and confrontation performance of the virtual anchor interactive scene, avoids the problem of monotonous stage performance caused by traditional fixed positions, and improves the viewing experience. Attached Figure Description

[0029] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0030] Figure 1This diagram schematically illustrates the operating environment of the video stream output method according to Embodiment 1 of this application; Figure 2 A flowchart illustrating a video stream output method according to Embodiment 1 of this application is shown schematically. Figure 3 Schematic illustration Figure 2 Flowchart of the sub-steps in step S200; Figure 4 Schematic illustration Figure 2 Flowchart of the sub-steps in step S202; Figure 5 Schematic illustration Figure 4 Flowchart of the sub-steps in step S402; Figure 6 Schematic illustration Figure 2 Flowchart of the sub-steps in step S204; Figure 7 The illustration schematically shows a new flow in the video stream output method according to Embodiment 1 of this application; Figure 8 Schematic illustration Figure 7 Flowchart of the sub-steps in step S700; Figure 9 Schematic illustration Figure 7 Flowchart of another sub-step in step S700; Figure 10 The illustration schematically shows a new flow in the video stream output method according to Embodiment 1 of this application; Figure 11 The illustration schematically shows a new flow in the video stream output method according to Embodiment 1 of this application; Figure 12 Schematic illustration Figure 2 Flowchart of the sub-steps in step S206; Figure 13 Schematic illustration Figure 12 Flowchart of the sub-steps in step S1200; Figure 14 Schematic illustration Figure 2 Flowchart of the sub-steps in step S206; Figure 15 Schematic illustration Figure 2 Flowchart of the sub-steps in step S208; Figure 16 Schematic illustration Figure 2 Flowchart of another sub-step in step S208; Figure 17 Schematic illustration Figure 2 Flowchart of another sub-step in step S208; Figure 18Schematic illustration Figure 2 Flowchart of the sub-steps in step S210; Figure 19 Schematic illustration Figure 2 Flowchart of another sub-step in step S210; Figure 20 The illustration schematically shows a new flow in the video stream output method according to Embodiment 1 of this application; Figure 21 The illustration schematically shows a new flow in the video stream output method according to Embodiment 1 of this application; Figure 22 Schematic illustration Figure 21 Flowchart of the sub-steps in step S2100; Figure 23 Schematic illustration Figure 21 Flowchart of the sub-steps in step S2102; Figure 24 The illustration schematically shows a new flow in the video stream output method according to Embodiment 1 of this application; Figure 25 The diagram illustrates an application example of the video stream output method according to Embodiment 1 of this application. Figure 26 The diagram illustrates an application example of the video stream output method according to Embodiment 1 of this application. Figure 27 A block diagram of a video stream output device according to Embodiment 2 of this application is schematically shown; and Figure 28 A schematic diagram of the hardware architecture of a computer device according to Embodiment 3 of this application is shown. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0032] It should be noted that the descriptions involving "first," "second," etc., in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0033] It should be noted that, in any stage of this application involving the collection, storage, use, transmission, and processing of data, each stage strictly adheres to the laws, regulations, industry standards, and regulatory requirements of the data source, usage location, and relevant countries and regions to ensure the legality and compliance of data activities. In the collection stage, the purpose, method, and scope of collection are clearly communicated to the data subject in a prominent manner. Collection is conducted only after obtaining the data subject's legal authorization, ensuring that the collection process follows the "minimum necessary" principle and does not exceed the scope of data collection. In the storage stage, storage periods are limited, and data is promptly deleted or anonymized / encrypted after the storage purpose is achieved. In the usage stage, a strict data security protection mechanism is implemented, using field-level desensitization technology and processing the original data according to preset desensitization rules. For different types of data, multiple desensitization strategies, such as data generalization, data anonymization, and data encryption, are employed to effectively mitigate the risk of sensitive information leakage and ensure that all data used is securely processed and desensitized, comprehensively protecting the rights and interests of data subjects and data security. In the transmission and processing stages, the confidentiality and security of data are ensured during transmission and processing.

[0034] In the description of this application, it should be understood that the numerical labels before the steps do not indicate the order of the steps, but are only used to facilitate the description of this application and to distinguish each step, and therefore should not be construed as a limitation of this application.

[0035] First, a definition of the terminology used in this application is provided: Live streaming: refers to live broadcasting in real time, which involves transmitting audio and video signals via the Internet or other networks within a certain period of time for users to watch online.

[0036] Virtual characters: refers to virtual figures generated by computers that can exhibit behavior and appearance in a digital environment.

[0037] Centroid location: refers to the location of the center point of an object, figure, or system.

[0038] Bounding box: refers to the smallest bounding box of an object in space, that is, the object is completely wrapped by a rectangle or cube to define its position and the space it occupies.

[0039] Alpha blending: refers to the process of calculating and layering the transparency of an image or object to obtain the final composite color and transparency effect.

[0040] Secondly, to facilitate understanding of the technical solutions provided in the embodiments of this application by those skilled in the art, the relevant technologies are described below: With the development of the live streaming industry, the following problems exist in multi-person interaction or virtual anchor interaction scenarios in virtual environments: First, the positions of virtual characters on the stage are mostly preset or statically configured, making it difficult to dynamically adjust them according to changes in their state during the interaction process. This results in the stage space relationships failing to reflect the progress of the confrontation and changes in advantages.

[0041] Second, when extracting or separating virtual characters, only the image data of the characters is usually obtained, lacking semantic descriptions related to spatial layout such as the character's orientation and size, which leads to a lack of effective calculation basis for subsequent stage layout and perspective adjustment.

[0042] Third, in virtual anchor interaction or multi-person interaction systems, the switching of viewing perspectives usually relies on manual operation or simple rules, making it difficult to dynamically adjust according to the status of the interaction process and the audience's focus, resulting in a mismatch between the stage presentation and the audience experience.

[0043] To address this, this application provides a video stream output technology solution. This solution introduces a stage layout mechanism based on the semantic state of the interaction process within a unified virtual stage. According to different stages or results of the interaction, the positions, depths, and relative relationships of multiple virtual characters in the stage coordinate system are dynamically calculated and updated. This allows the spatial relationships of virtual characters to change in real time with the interaction process, mapping the adversarial state to spatial layout changes. This overcomes the problem of monotonous stage performance caused by fixed positions in the background technology, enhancing the spatial expression and adversarial performance of the virtual anchor's interactive scenes. During the character stream construction stage, not only is character image data extracted, but also character semantic metadata such as orientation and size related to the character is generated simultaneously. This character semantic metadata is used as input parameters for the unified virtual stage layout calculation. By introducing character semantic metadata, the unified virtual stage has a clear spatial calculation basis when recombining multiple characters, overcoming the lack of layout semantic support in the background technology and improving the stability, spatial consistency, and visual rationality of multi-character simultaneous synthesis. By introducing an automatic directing mechanism that combines interaction state information and audience focus targets, the observation angle and camera parameters of the virtual stage are dynamically adjusted according to the current interaction stage, character spatial relationships, or audience selection. The aforementioned automatic broadcasting mechanism ensures that the viewing angle aligns with the interactive process and the audience's focus, overcoming the rigidity of perspective control in background technologies, improving the continuity and visual appeal of the presentation, and enhancing the immersive viewing experience in virtual anchor interactive scenarios. See below for details.

[0044] Finally, for ease of understanding, an exemplary operating environment is provided below.

[0045] like Figure 1 As shown in the diagram, the operating environment includes: service platform 2, broadcaster terminals (4A, 4B, ..., 4M), and viewer terminals (6A, 6B, ..., 6N). In a live broadcast scenario, the broadcaster terminals (4A, 4B, ..., 4M) log in to service platform 2 and push live broadcast data to the viewer terminals (6A, 6B, ..., 6N) in real time through service platform 2.

[0046] Service platform 2 can provide live streaming services, which can be a single server, a server cluster, or a cloud computing service center.

[0047] The broadcast terminals (4A, 4B, ..., 4M) are used to generate live streaming data in real time and to push the live streaming data. The live streaming data may include audio data or video data. The broadcast terminals can be electronic devices such as smartphones or tablets. Alternatively, the broadcast terminals can be virtual computing instances within service platform 2.

[0048] Viewer terminals (6A, 6B, ..., 6N) can be configured to receive live data from the broadcaster terminal in real time. Viewer terminals (6A, 6B, ..., 6N) can be any type of computing device, such as smartphones, tablets, laptops, smart TVs, in-vehicle terminals, etc. Viewer terminals (6A, 6B, ..., 6N) can have a built-in browser or dedicated program to receive the live data and output content to the user. The content may include video, audio, comments, text data, and / or the like.

[0049] The audience terminals (6A, 6B, ..., 6N) may include a player. The player outputs (e.g., displays, presents) content to the user. This content may include video, audio, comments, text data, and / or the like. The audience terminals (6A, 6B, ..., 6N) may include an interface that may include an input element (touchscreen). For example, the input element may be configured to receive user instructions that cause the audience terminals (6A, 6B, ..., 6N) to perform various operations, such as sending bullet comments, entering comments, sending gifts, etc.

[0050] The broadcast terminals (4A, 4B, ..., 4M), viewer terminals (6A, 6B, ..., 6N), and service platform 2 can be connected via a network. The network may include various network devices, such as routers, switches, multiplexers, hubs, modems, bridges, repeaters, firewalls, and / or proxy devices. The network may include physical links, such as coaxial cable links, twisted-pair cable links, fiber optic links, and combinations thereof and / or the like. The network may include wireless links, such as cellular links, satellite links, Wi-Fi links, and / or the like.

[0051] It should be noted that the number of broadcast terminals and viewer terminals shown in the diagram is merely illustrative and is not intended to limit the scope of patent protection of this application. In practice, any number of broadcast terminals and viewer terminals may be used.

[0052] The technical solution of this application will be described below through multiple embodiments, using service platform 2 as the implementing entity. It should be understood that these embodiments can be implemented in many different forms and should not be construed as being limited to the embodiments described herein.

[0053] Example 1 Figure 2 A flowchart illustrating a video stream output method according to Embodiment 1 of this application is shown schematically.

[0054] like Figure 2 As shown, the video stream output method may include steps S200~S210, wherein: Step S200: Obtain event data corresponding to the interactive live broadcast process and real-time media frame data of at least two virtual characters; Step S202: Generate interactive semantic state information representing the interactive process based on the event data; Step S204: Generate character image data for each virtual character based on the real-time media frame data; Step S206: Based on the interactive semantic state information, determine the target position of each virtual character in the pre-configured stage space coordinate system, and determine the display parameters corresponding to the target position; the stage space coordinate system and the display plane have corresponding spatial transformation parameters; Step S208: Based on the target position, the display parameters, and the spatial transformation parameters, perform spatial reconstruction on the character image data; Step S210: Output the synthesized video stream of the spatial reconstruction result.

[0055] The video stream output method provided in this embodiment first acquires event data corresponding to the interactive live broadcast process and real-time media frame data of at least two virtual characters. Based on the event data, it generates interactive semantic state information representing the interactive confrontation process, enabling subsequent stage layout and screen presentation to be dynamically driven by the semantics of the confrontation process, avoiding reliance solely on fixed positions or static synthesis unrelated to the confrontation. Further, based on the interactive semantic state information, it determines the target position of each virtual character in a pre-configured stage space coordinate system and determines the display parameters corresponding to the target position. Simultaneously, it utilizes the spatial transformation parameters corresponding to the stage space coordinate system and the display plane to map the target position and display parameters to a presentation result on the display plane in a deterministic manner. Subsequently, based on the target position, display parameters, and spatial transformation parameters, it performs spatial reconstruction of the character image data, updating the position, scale, and hierarchy of each character in the same output screen as the interactive semantic state changes. Therefore, this technical solution can establish a processing link in interactive live streaming scenarios, namely event data → interactive semantic state → target position / display parameters → spatial reconstruction → synthetic video stream output. This enables the spatial relationships in the virtual stage to change dynamically with the interaction process, and intuitively maps the adversarial state into changes in spatial layout. This improves the spatial expression and adversarial performance of the virtual anchor interactive scene, avoids the problem of monotonous stage performance caused by traditional fixed positions, and improves the user experience.

[0056] The following combination Figure 2 The steps in steps S200 to S210, as well as other optional steps, are described in detail.

[0057] Step S200 Acquire event data corresponding to the interactive live broadcast process, as well as real-time media frame data of at least two virtual characters.

[0058] Interactive (e.g., PK) live streaming can be a competitive interactive activity occurring on a live streaming platform, such as PK live streams between virtual streamers, multi-player competitive interactive live streams, or other live streams with real-time competitive relationships. Event data during interactive live streaming can include a set of data describing interactive behaviors or changes in interactive states during the interactive process, such as gift data, interaction data, and interaction control data. Gift data can be used to represent the degree of support viewers have for different virtual characters, such as gift value, number of gifts, and gift sending time; interaction data can be used to reflect the degree of viewer participation, such as the number of likes, comments, or bullet comments; and interaction control data can be used to characterize changes in the interactive phase, such as interaction start events, phase transition events, or interaction end events.

[0059] Real-time media frame data for virtual characters can be video frame data collected and uploaded to the live streaming system by different virtual anchor devices during the live broadcast. For example, real-time media frame data can originate from camera data captured by the anchor device, virtual avatar rendering data, or other video data that can represent the visual appearance of the virtual character. Real-time media frame data can exist in the form of continuous video frames, each frame containing image pixel information and corresponding time information, thereby reflecting the changes in the virtual character's actions or postures at different points in time.

[0060] The following will provide further illustrative examples of specific methods for acquiring real-time media frame data through concrete embodiments.

[0061] In optional embodiments, such as Figure 3 As shown, step S200 may include: Step S300: Acquire at least two video frames.

[0062] Step S302: Add a frame timestamp to each of the video frames.

[0063] Step S304: Based on the timestamp, align the timestamps of each event in the event data with those of each video frame.

[0064] In practical applications, interactive live streaming generates numerous interactive events, such as gift-giving or "like" events. If there is a lack of a unified time reference between event data and video frames, the timing of event effects appearing in the video may deviate. For example, gift effects might appear earlier or later, affecting the real-time performance and viewing experience of the live stream. Therefore, this embodiment maps event data to corresponding video frames to determine the trigger time position of the event in the video frame, thereby performing event-related image processing on the corresponding video frame. In some embodiments, buffer queues can be established for both video frames and event data, and the event data can be sorted chronologically before gradually matching events with video frames. This method can improve matching accuracy even when there is a time difference between event data and video data. In some embodiments, a time window matching strategy can also be used, setting a corresponding time window range for each video frame. When an event timestamp falls within this time window, the event is associated with that video frame. This method maintains good stability even with network latency or time fluctuations.

[0065] In this embodiment, by aligning the event data with the video frames using timestamps, the event trigger time can be kept consistent with the video frame time, thereby ensuring that the interactive effect is presented at the correct video frame time.

[0066] Step S202 Based on the event data, interactive semantic state information representing the interactive process is generated.

[0067] After acquiring event data corresponding to the interactive live stream process, statistical analysis and semantic mapping can be performed on the event data to generate interactive semantic state information that reflects the progress of the interaction. Interactive semantic state information can be used to characterize semantic information such as the current interaction stage, interaction intensity, or interaction trend. Interactive semantic state information can include interaction stage information, interaction intensity information, win / loss trend information, or interaction popularity information. For example, interaction stage information can be used to indicate the current stage of the interaction, such as the preparation stage, the formal interaction stage, or the end stage; interaction intensity information can be used to indicate the degree of competition between the two sides, such as a normal state, an intense state, or a climax state; win / loss trend information can be used to indicate the current trend of score changes for both sides, such as a leading state, a catching-up state, or a overtaking state.

[0068] The following will provide further illustrative examples of specific methods for generating interactive semantic state information through more concrete embodiments.

[0069] In optional embodiments, such as Figure 4 As shown, step S202 may include: Step S400: Update the interactive state machine based on the event data to obtain the interactive state.

[0070] Step S402: Calculate the advantage value based on the event data.

[0071] Step S404: Output interactive semantic state information that includes at least the interactive state and the advantage value.

[0072] An interactive state machine can be used to describe the state change relationships between different stages during an interactive live stream. In some embodiments, the system can maintain a multi-state machine model for unified management of the interactive process. The interactive state machine can include multiple state nodes, such as: standby state, initialization state, stable period, tug-of-war period, climax period, settlement period, and end state. Real-time input of event data (such as gift value, gift-giving frequency, etc.) can trigger transitions between these states. For example, when the system does not detect an interactive start event, the interactive state machine can be in a standby state. When an interactive start event is detected, such as an interactive start command or the entry of both virtual characters into the interactive scene, the state machine can be switched to the initialization state, indicating that the interaction has started and both virtual characters have entered the stage area. After the interaction begins, the system can continuously collect gift data received by both virtual characters and dynamically update the interactive state based on changes in gift quantity. For example, when the difference in gift quantity between the two sides is small and the difference percentage is less than a preset threshold (e.g., 15%), the interactive state can be determined as a stable period, indicating that the competition between the two sides is relatively even. When the system detects that the gift amounts of both parties alternate in leading, and the magnitude of a single reversal exceeds a preset threshold (e.g., 10%), the interaction state machine can be updated to the tug-of-war phase, which indicates that the two parties are in a relatively intense competitive state. When one party's gift amount leads by a large margin, such as more than 30%, and this lead lasts for more than a preset time (e.g., 10 seconds), the interaction state machine can be updated to the climax phase, indicating that the interaction has entered a relatively intense or critical stage. When the interaction is nearing its end, such as when the interaction countdown is less than a preset time (e.g., 30 seconds), the interaction state machine can be updated to the settlement phase. When the interaction end condition is met, such as when the interaction countdown ends or the system receives an interaction end event, the interaction state machine can be updated to the end state, indicating that the interaction result has been determined.

[0073] In practical applications, after obtaining the interaction status, an advantage score can be further calculated based on event data. This advantage score can be used to quantify the relative advantage of different virtual characters in the current interaction. For example, the gift values ​​obtained by the first and second virtual characters during the current interaction can be statistically analyzed, for example, denoted as GiftValue_A and GiftValue_B respectively. Furthermore, the advantage score (Advantage_Score_A) can be calculated using the following formula: Advantage_Score_A=(GiftValue_A-GiftValue_B) / (GiftValue_A+GiftValue_B) The above calculation formula shows that the advantage value ranges from -1 to 1. When the advantage value is close to +1, it indicates that the first virtual character has a significant advantage. When the advantage value is close to -1, it indicates that the second virtual character has a significant advantage. When the advantage value is close to 0, it indicates that the gift amounts for both sides are roughly equal, and the competition is relatively even. This calculation formula transforms event data into a numerical indicator that directly reflects the interactive competitive situation.

[0074] In this embodiment, the interaction state is obtained by updating the interaction state machine based on event data, and the advantage value is calculated based on the event data. Then, the interaction semantic state information containing the interaction state and the advantage value is output. In this way, event data can be transformed into semantic information that can characterize the progress of interaction and competitive relationship, thereby providing a more accurate basis for subsequent virtual character positioning adjustment, stage space reconstruction and video image generation, and improving the dynamic performance of live broadcast.

[0075] In an optional embodiment, the generation of interactive semantic state information may further include: Leader indicator, tempo tag, and / or remaining time.

[0076] In practical applications, key semantic information during the interaction process can be further extracted based on the interaction state and advantage value, outputting structured interaction semantic state information. For example, the following output data structure can be constructed: { "pk_state": "CLIMAX", "advantage_score": 0.42, "leader": "host_A", "rhythm": "burst", "time_remaining": 45 } The leader identifier can be determined based on the advantage value. For example, when the advantage value Advantage_Score_A is greater than 0, the first virtual role (host_A) can be identified as the leader. When the advantage value is less than 0, the second virtual role (host_B) can be identified as the leader. The rhythm label can be determined based on the current state of the interaction state machine. For example, when the interaction state is in a stable phase, rhythm labels such as "stable" or "even" can be generated. When the interaction state is in a tug-of-war phase, rhythm labels such as "tug-of-war" or "confrontation" can be generated. When the interaction state enters a climax phase, rhythm labels such as "outburst" or "climax" can be generated. The remaining time (time_remaining) can be calculated based on the interaction's countdown information. For example, in an interactive live streaming scenario, the system can maintain an interaction timer. When the interaction starts, the timer counts down according to a preset duration (e.g., 180 seconds). The system can read the current value of the timer in real time as the remaining time and write this remaining time into the interaction semantic state information.

[0077] In this embodiment, by further determining the leading player identifier, rhythm tag, and remaining time, more key semantic elements reflecting the interactive progress and competitive situation can be added to the interactive semantic state information. This allows the interactive semantic state information to not only describe the competitive relationship between the two parties but also depict the interactive rhythm and time process. Therefore, when subsequently adjusting the virtual character positions, arranging the stage space, or generating video footage, more detailed control over the visual presentation can be achieved based on richer semantic information. This results in the generated video footage more accurately reflecting the interactive progress, further enhancing the expressiveness and viewing experience of the live stream content.

[0078] In optional embodiments, such as Figure 5 As shown, step S402, "calculating the advantage value based on the event data," may include: Step S500: Statistical analysis of gift value and / or gift-giving frequency based on a sliding time window.

[0079] Step S502: Update the advantage value and / or the rhythm label based on the statistical results.

[0080] For example, real-time data streams, including gift data streams, audience interaction data, and PK timeline information, can be accessed from the live streaming platform interface. The accessed data is then uniformly formatted to include fields such as timestamp, streamer ID, gift value, and gift-giving frequency. A data buffer queue can be established for the accessed data, and the data entering the queue can be continuously statistically analyzed based on a sliding time window. For instance, within a preset time window (e.g., 5 seconds or 10 seconds), the total value of gifts received by both streamers, the number of gifts sent, or the change in gift-giving frequency can be statistically analyzed. Based on the statistical results, the difference or ratio between the two streamers can be calculated to obtain an advantage value reflecting the real-time advantage of both sides. In some embodiments, the gift inflow rate can be further calculated, and the first derivative of this rate can be calculated to determine whether the gift inflow intensity is rapidly increasing, thereby accurately identifying whether the interaction has entered a "burst phase." The variance of gift quantity changes within multiple consecutive time windows (e.g., three consecutive time windows) can also be detected. When a large fluctuation in gift quantity is detected within multiple time windows, a "tug-of-war" characteristic of alternating leadership in gift quantity between the two streamers can be identified. Based on the above statistical results, corresponding rhythm tags can be output, such as smooth, accelerated, explosive, and decay.

[0081] In this embodiment, by statistically analyzing gift value and / or gift-giving frequency based on a sliding time window, and updating the advantage value and / or rhythm label based on the statistical results, dynamic updates can be made in conjunction with the changing trends of interaction data over time, more accurately reflecting the competitive situation and changes in the rhythm of interaction between the two parties during the interaction process.

[0082] Step S204 Based on the real-time media frame data, character image data for each virtual character is generated.

[0083] After acquiring real-time media frame data from at least two virtual characters, image parsing processing can be performed on the real-time media frame data to obtain the character image data corresponding to each virtual character at the current time. The character image data can be used to characterize the visual state of each virtual character at the current moment. In some embodiments, character identification information can be added to the character image data to identify the virtual character to which the current character image belongs, such as anchor A or anchor B. By attaching character identification information to the character image data, the images of different virtual characters can be processed independently based on the character identification information, such as setting different positions, scaling ratios, or visual effects.

[0084] The following will provide further illustrative examples of specific methods for generating character image data for virtual characters through more concrete embodiments.

[0085] In optional embodiments, such as Figure 6As shown, step S204, "generating character image data for each virtual character based on the real-time media frame data," may include: Step S600: Perform keying or segmentation processing on the real-time media frame data, or extract the region image containing the character.

[0086] Step S602: Output character image data containing the alpha channel, or output character area image data for rendering.

[0087] In practical applications, image segmentation can be performed on the anchor or virtual characters in real-time media frames to obtain independently renderable character images. For example, a deep learning-based real-time keying model can be used to segment the character region in a video frame, generating RGBA image data including the alpha channel. During processing, the character's edge regions can be refined, for example, by controlling the edge softening radius to be less than 2 pixels and smoothing the transparency transition areas to avoid jagged or abrupt boundaries. In some embodiments, the region image containing the character's main body can also be extracted, for example, by detecting the character's bounding box or semantic segmentation region, cropping the image region containing the character, and outputting the corresponding character region image data. Thus, the main character can be separated from the original live video frame, allowing the character image to participate as an independent layer in subsequent image compositing or rendering processes.

[0088] In this embodiment, by performing keying or segmentation processing on real-time media frame data and outputting character image data or character region image data containing alpha channels, the main character can be separated from the original video background, so that the character image can be independently positioned, scaled, or overlaid during subsequent stage layout adjustments or video frame generation.

[0089] In optional embodiments, such as Figure 7 As shown, the method may further include: Step S700: For each channel of character image data, extract character semantic metadata for stage reconstruction.

[0090] Step S702: Associate the character semantic metadata with the corresponding character image data.

[0091] In practical applications, character semantic metadata can be used to represent structured information about the physical features and spatial posture of virtual characters. Character semantic metadata can be extracted based on deep learning algorithms or image recognition techniques. For example, a human keypoint detection model can be used to obtain the coordinates of key points of a virtual character and calculate its orientation angle. Alternatively, object detection algorithms can be used to determine the bounding rectangle and centroid position of the character's pixel region. After extracting the character semantic metadata, it can be associated with the corresponding character image data. For example, a corresponding data record can be created for each stream of character image data, storing both the character image data and the corresponding semantic fields, such as orientation parameters, size parameters, and region coordinates. By establishing the association between character semantic metadata and the corresponding character image data, the character image data and its semantic information can form a unified data unit, serving as input data in the subsequent stage reconstruction process. Therefore, when compositing multiple characters on the same stage, the spatial relationships between the characters can be calculated based on the character semantic metadata, avoiding problems such as imbalanced proportions, unreasonable occlusion relationships, or inconsistent character orientations that may occur when relying solely on image overlay in the subsequent virtual stage compositing process.

[0092] In this embodiment, by extracting character semantic metadata for stage reassembly from each character image data and associating the character semantic metadata with the corresponding character image data, the character images can carry structured information describing the character's spatial attributes before entering the stage reassembly process. This provides a clear data foundation for the subsequent virtual stage layout and improves the stability and spatial consistency of image synthesis in multi-character scenes.

[0093] In optional embodiments, such as Figure 8 As shown, step S700, "For each channel of character image data, extract character semantic metadata for stage reconstruction," may include: Step S800: Determine the bounding rectangle of the character pixel region.

[0094] Step S802: Determine the position of the character's center of mass.

[0095] Step S804: Determine the character height ratio, where the height ratio is the ratio of the height of the outer rectangle to the height of the video frame.

[0096] The bounding rectangle of a character's pixel region can be the smallest rectangular boundary surrounding that region, used to describe the character's spatial occupancy within a video frame. In some embodiments, the boundary extrema of the character's pixel distribution (i.e., the maximum and minimum values ​​of pixels in the horizontal and vertical directions) can be identified by scanning the alpha channel of real-time media frame data, thereby obtaining the bounding rectangle that represents the character's physical outline. The character's centroid position can be the geometric center coordinates (Centroid_X, Centroid_Y) of the character's pixel region, used to determine the character's alignment, rotation center, or the starting point of the animation trajectory. In some embodiments, the character's centroid position can be determined by calculating the mean coordinates of the character's pixels. Height percentage can describe the character's vertical proportion in the stage space, guiding stage scaling, depth sorting, and visual focus effects. By using height percentage as a spatial reference parameter, reasonable proportions and distinct layers can be ensured when compositing multiple characters, while also facilitating automatic adjustment of the character's foreground or background position, improving visual harmony. For example, when the height percentage of character A is 0.8 and the height percentage of character B is 0.5, it indicates that the shooting distance of anchor B is relatively far. At this point, the scaling ratio can be automatically adjusted based on the height ratio to ensure that the two characters maintain proportional height when displayed on the same stage. Precise collision detection can be performed using the bounding rectangle and centroid position. For example, when two characters move closer together due to stage reconstruction, the system can calculate whether the two bounding rectangles overlap and, combined with changes in the centroid distance, automatically trigger displacement compensation logic to prevent visual clipping or unreasonable occlusion between characters.

[0097] In this embodiment, by determining the bounding rectangle, centroid position, and height proportion of each character, the spatial characteristics of each character in the video frame can be determined, thereby providing accurate and stable spatial reference parameters for spatial reconstruction and multi-character compositing, ensuring the consistency of character position, size, and proportion, and improving the visual rationality and stability of the compositing effect.

[0098] In optional embodiments, such as Figure 9 As shown, step S700, "For each channel of character image data, extract character semantic metadata for stage reconstruction," may further include: Step S900: Calculate the character's orientation angle based on the human body key point detection results.

[0099] Step S902: Map the orientation angle to an orientation category to obtain orientation information.

[0100] In practical applications, the coordinates of key points on the character's shoulders (such as the left shoulder point (Shoulder_L) and the right shoulder point (Shoulder_R)) can be obtained first. Then, the character's orientation angle (Orientation_Angle) can be obtained based on the angle between the line connecting the left and right shoulder points and the camera. Specifically, it can be calculated using the following formula: Orientation_Angle=arctan2(Shoulder_R.x-Shoulder_L.x,Shoulder_R.y-Shoulder_L.y) For example, the orientation angle can be further mapped to categories such as front (e.g., |Orientation_Angle|<15°), left (e.g., |Orientation_Angle|>15°), and right (e.g., |Orientation_Angle|<-15°) according to a preset threshold, thereby generating specific orientation information based on the orientation angle.

[0101] In this embodiment, by calculating the character's orientation angle and mapping it to an orientation category, a precise directional reference can be provided for the spatial reconstruction of characters and the synthesis of multiple characters, ensuring that the characters' orientations on the virtual stage are reasonable and consistent, and further enhancing the overall visual coordination and the realism of the stage performance.

[0102] Step S206 Based on the interactive semantic state information, the target position of each virtual character in the pre-configured stage space coordinate system is determined, and the display parameters corresponding to the target position are determined; the stage space coordinate system and the display plane have spatial transformation parameters.

[0103] A stage space coordinate system can be a three-dimensional coordinate system used to describe the structure of a virtual stage space. This can be achieved by constructing a three-dimensional coordinate system with the stage center as the reference point, using coordinate axes to describe positions within the stage space. Within this three-dimensional coordinate system, multiple available standing positions can be pre-configured for different virtual characters, each corresponding to a spatial coordinate location. For example, multiple positions such as left-side, right-side, or center-side can be set in the stage space coordinate system to adjust the position of the virtual character under different interactive states.

[0104] The target position can be the desired location of a virtual character within the stage space coordinate system, determined based on the semantic state information of the interaction. For example, in a normal interaction, different virtual characters can be positioned on opposite sides of the stage; when the interaction enters an intense or climax phase, the virtual characters can be moved to the center of the stage or closer together to each other, thereby intensifying the competitive atmosphere. After determining the target position, the corresponding display parameters can be further determined. Display parameters describe the visual presentation of the virtual character in the final display screen. Display parameters can include information such as image scaling ratio, display hierarchy order, or screen position parameters. By setting different display parameters, the size or hierarchy of the virtual character in the screen can be changed, resulting in richer visual effects. For example, when a virtual character is in a leading position, its display ratio or display hierarchy can be appropriately increased to highlight its dominant position in the current interaction.

[0105] Spatial transformation parameters can be used to describe the mapping relationship from the stage space coordinate system to the display plane. For example, spatial transformation parameters can include projection parameters, viewpoint parameters, or scaling parameters. By performing spatial transformation processing on the stage space coordinates, the position of a virtual character in the three-dimensional stage space can be converted into its display position on the two-dimensional display plane.

[0106] In this embodiment, the target position of each virtual character is determined in a pre-constructed stage space coordinate system based on interactive semantic state information, and the corresponding display parameters are determined according to the target position, so as to realize the dynamic layout of each virtual character in the video screen.

[0107] In optional embodiments, such as Figure 10 As shown, the method also includes the operation of pre-configuring the stage space coordinate system: Step S1000: Define the horizontal X-axis, vertical Y-axis, and depth Z-axis.

[0108] Step S1002: Set area anchor points, which include at least the stage center point and the initial positions of at least two virtual characters.

[0109] The horizontal X-axis, vertical Y-axis, and depth Z-axis can be used to represent the character's horizontal position, vertical height, and front / back depth, respectively. In practical applications, area anchor points can include the stage center point (Center), the initial position of character A (Pos_A_Init), the initial position of character B (Pos_B_Init), the foreground focus area (Focus_Front), and the background retreat area (Back_Retreat), etc. Each anchor point can be represented using three-dimensional coordinates in a coordinate system, used for character spatial reconstruction, dynamic movement, and display parameter generation. By pre-setting area anchor points, it can be ensured that the character is distributed on the stage according to the preset position, orientation, and depth.

[0110] In this embodiment, by establishing a stage space coordinate system and setting key area anchor points, a specific and calculable spatial layout basis can be provided for a multi-role virtual stage, thereby ensuring the consistency, spatial rationality, and visual harmony of the character display.

[0111] In optional embodiments, such as Figure 11 As shown, the method further includes the operation of generating the spatial transformation parameters: Step S1100: Generate mapping parameters from stage coordinates to display pixel coordinates.

[0112] Step S1102: Generate scaling parameters related to depth Z so that different depths correspond to different display sizes.

[0113] In practical applications, stage coordinates can be used to describe the position of a character in a virtual stage space. Stage coordinates can include horizontal coordinates (Stage_X), vertical coordinates (Stage_Y), and depth coordinates (Stage_Z). Display pixel coordinates can be used to describe the character's display position in the video frame. They can use the pixel plane of the video frame as the coordinate system, for example, with the top left corner of the video frame as the origin, the horizontal axis as the X-axis, and the vertical axis as the Y-axis. To establish the positional correspondence between the stage space and the display screen, mapping parameters from stage coordinates to display pixel coordinates can be generated. For example, according to preset perspective mapping rules, the horizontal coordinate (Stage_X) in stage coordinates can be mapped to screen pixel coordinates (Screen_X), and the vertical coordinate (Stage_Y) in stage coordinates can be mapped to screen pixel coordinates (Screen_Y). For example, Screen_X and Screen_Y can be determined according to the following formulas: Screen_X = (Stage_X 0.5 + 0.5) Screen Width Screen_Y = (1.0 Stage_Y) Screen_Height Here, Screen_Width represents the width of the video frame in pixels, and Screen_Height represents the height of the video frame in pixels. Using this mapping method, standardized position coordinates in the stage space can be converted into actual pixel positions in the video frame, thus determining the character's display position on the screen.

[0114] Furthermore, to reflect the depth relationship of the character in stage space, scaling parameters related to depth Z can be generated. For example, the corresponding scaling factor Scale_Factor can be determined based on the depth coordinate Stage_Z, where Scale_Factor can be determined according to the following formula: Scale_Factor = 0.5 + 0.5 Stage_Z Scaling factors can be used to scale the display size of characters in a video frame, allowing characters at different depths to appear at different sizes. For example, a larger Stage_Z value indicates a larger Scale_Factor, resulting in a larger display size for the character. Conversely, a smaller Stage_Z value indicates a smaller Scale_Factor, resulting in a smaller display size for the character.

[0115] In this embodiment, by generating mapping parameters from stage coordinates to display pixel coordinates and combining them with scaling parameters related to depth Z, the display position and size of the character in the video frame can be determined, so that characters in different stage positions can present a display effect that conforms to spatial perspective, thereby improving the spatial consistency and visual realism of the virtual stage characters.

[0116] In optional embodiments, such as Figure 12 As shown, step S206, "determining the target position of each virtual character in the pre-configured stage space coordinate system based on the interactive semantic state information, and determining the display parameters corresponding to the target position," may include: Step S1200: Input the interactive semantic state information into the station mapping model or station mapping function to obtain the target station.

[0117] Step S1202: Output the coordinates of the target position in the stage space coordinate system.

[0118] In practical applications, to dynamically determine a character's position on a virtual stage based on interactive semantic state information, a position mapping model or function can be constructed to calculate the character's target position based on the input interactive semantic state information. The target position can represent the character's three-dimensional position in the stage space coordinate system, including, for example, the horizontal coordinate x, the vertical coordinate y, and the depth coordinate z. For example, the position mapping function can be defined as: f(pk_state, advantage_score) → (x, y, z) Here, `pk_state` represents the current interaction state, such as the confrontation phase, the display phase, or the settlement phase, while `advantage_score` represents the current character's advantage over the opponent. By inputting the above interaction semantic state information into the positioning mapping function, the corresponding target positioning coordinates can be obtained, thereby determining the character's target position in the stage space.

[0119] In this embodiment, by inputting interactive semantic state information into the position mapping model or position mapping function to calculate the target position and outputting the coordinates of the target position in the stage space coordinate system, the spatial layout of the character on the stage can be dynamically adjusted according to the semantic state during the interaction process, thereby realizing the intelligent adjustment of the character's position in the virtual stage and improving the dynamism and spatial performance of the screen presentation in the interactive scene.

[0120] In optional embodiments, such as Figure 13 As shown, the station mapping model or station mapping function in step S1200 includes station rules that switch according to the interaction state, and the station rules include at least: Step S1300: In the initialization state, set at least two virtual characters to symmetrical initial positions.

[0121] Step S1302: During the stable or tug-of-war period, make differentiated adjustments to the X-axis position and / or Z-axis depth of the leading and lagging sides based on the advantage value.

[0122] Step S1304: During the climax, update the position of the leading team to the center of the stage or the foreground area, and update the position of the lagging team to the edge of the stage or the background area.

[0123] For example, the positioning mapping model can dynamically switch different positioning rules based on the interactive semantic state, thereby realizing the dynamic layout adjustment of virtual characters in the stage space. The interactive semantic state can include different stages such as the initialization state, the stable period, the tug-of-war period, and the climax period. In the initialization state, symmetrical initial positions can be set for at least two virtual characters participating in the interaction. For example, the initial position of the first virtual character can be set to Pos_A = (-0.4, 0.5, 0.5), and the initial position of the second virtual character can be set to Pos_B = (+0.4, 0.5, 0.5). This allows both characters to maintain a symmetrical and consistent initial layout on the stage, thus forming a stable adversarial scene structure. In the stable period, the depth position of the characters can be fine-tuned based on the advantage value of both sides. For example, when the absolute value of the advantage value is less than a preset threshold (e.g., 0.15), it can be considered that both sides are in a relatively balanced state. At this time, the depth can be fine-tuned while keeping the horizontal position basically unchanged. Specifically, the depth offset Z_offset = advantage_score can be calculated based on the advantage value advantage_score. The depth coordinates of the leading player are adjusted to Pos_Leader.z = Pos_Leader.z + Z_offset, while the depth coordinates of the lagging player are adjusted to Pos_Follower.z = Pos_Follower.z. Z_offset. This creates a subtle visual variation in depth, reflecting the real-time advantage difference between the two sides. During periods of intense competition, the character's lateral and depth positions can be dynamically adjusted based on the advantage score. For example, the lateral offset X_offset = advantage_score can be calculated based on the advantage score. The value is set to 0.2, and the horizontal coordinate of the first virtual character is adjusted to Pos_A.x = -0.4 + X_offset, while the horizontal coordinate of the second virtual character is adjusted to Pos_B.x = +0.4. X_offset. Additionally, the depth difference between the two sides can be amplified based on the absolute value of the advantage value; for example, the depth coordinates of the leading side can be set to Z_Leader = 0.5 + |advantage_score|. 0.3, set the depth coordinates of the lagging side to Z_Follower = 0.5. |advantage_score| 0.2. This allows for dynamic changes in the horizontal and vertical positions of both characters on the stage, further enhancing the spatial tension during the confrontation. During the climax, character positioning can be more significantly adjusted based on the advantageous direction. For example, the horizontal coordinate of the leading character can be set to Pos_Leader.x = advantage_score. The value is increased by 0.1, causing the character to gradually move closer to the center of the stage, and its depth coordinate is set to Pos_Leader.z = 0.8, thus moving the character to the foreground area in the frame. Simultaneously, the horizontal coordinate of the lagging character can be set to Pos_Follower.x = advantage_score The character's height is adjusted to 0.6, causing it to gradually move towards the edge of the stage, and its depth coordinate is set to Pos_Follower.z = 0.3, thus placing it in the background. Furthermore, the vertical coordinate of the lagging character can be adjusted to Pos_Follower.y = 0.4, creating a slight downward tilt effect in the frame. This creates a clear spatial contrast during the climax, further enhancing the dramatic tension in the scene.

[0124] In this embodiment, by setting position rules that switch according to the interactive state in the position mapping model, and making differentiated adjustments to the position of the characters at different stages based on the advantage value, the virtual characters can present a dynamically changing spatial layout during the interaction process, thereby enhancing the visual expressiveness of the confrontation process and further improving the visual experience of the audience during the viewing process.

[0125] In an optional embodiment, the display parameters include at least one of translation parameters, scaling parameters, and / or hierarchy parameters, and the display parameters are determined based on the interactive semantic state information.

[0126] In practical applications, translation parameters determine a character's display position on the screen, scaling parameters determine a character's display size, and layer parameters determine a character's foreground and background occlusion. These display parameters can be dynamically determined based on interactive semantic state information. For example, the depth position of a character in the stage space can be adjusted based on the current interaction stage or the relative strengths of both sides, and corresponding display scaling parameters can be generated based on this depth position. This allows characters with different height proportions to maintain a reasonable visual proportion when displayed. Layer parameters can also be determined based on a character's depth value. For example, characters with larger depth values ​​can be set to a higher rendering layer, allowing them to be rendered first on the screen, while characters with smaller depth values ​​can be set to a lower rendering layer, thus creating an occlusion relationship between foreground and background. Furthermore, the translation parameters can be adjusted in conjunction with interactive semantic state information. For instance, when the dominant side is in a climax, the character can be appropriately translated towards the center of the stage to strengthen the visual focus on the screen.

[0127] In this embodiment, by dynamically determining display parameters including translation parameters, scaling parameters, and hierarchy parameters based on interactive semantic state information, the position, size, and hierarchy of virtual characters in the screen can be dynamically changed with the interactive state while ensuring the reasonable display ratio of the characters. This enhances the spatial representation effect in the interactive scene and improves the overall visual presentation effect.

[0128] In optional embodiments, such as Figure 14 As shown, step S206 may further include: Step S1400: Based on the display parameters determined by the interactive semantic state information, the character display size reference parameters are determined based on the character semantic metadata.

[0129] Step S1402: Determine the character orientation correction parameters and / or perspective compensation parameters based on the character semantic metadata.

[0130] In practical applications, character semantic metadata can be used to describe the basic attribute information of virtual characters, such as the character's height percentage in the original footage, default orientation, and posture characteristics. Through this character semantic metadata, the display method of the virtual character in the stage space can be adaptively adjusted, thereby achieving metadata-driven spatial reconstruction. For example, the default orientation information of the character (e.g., facing left or right) can be determined based on the character semantic metadata, and then the character's orientation correction parameters can be determined according to the required orientation.

[0131] In this embodiment, by determining the baseline parameters for the character's display size based on the character's semantic metadata, and further determining the character's orientation correction parameters and perspective compensation parameters, the display method of the character in the stage space can be adaptively adjusted while maintaining the consistency of the character's basic visual features. This achieves metadata-driven spatial reconstruction and enhances the spatial performance and visual realism of the virtual character in interactive scenes.

[0132] Step S208 Based on the target position, the display parameters, and the spatial transformation parameters, spatial reconstruction is performed on the character image data.

[0133] Spatial reconstruction refers to the process of recalculating and adjusting the position and visual effect of character image data on the display plane based on the positional relationships in the virtual stage space. By performing spatial reconstruction, the positional relationships of each virtual character in the final display screen can be kept consistent with the target positions in the stage space coordinate system, realizing the presentation of the preset stage space structure in the two-dimensional display screen, thereby improving the expressiveness and overall visual effect of the live broadcast.

[0134] The following will provide further illustrative examples of the specific methods for reconstructing the execution space through more concrete embodiments.

[0135] In optional embodiments, such as Figure 15 As shown, step S208, "based on the target position, the display parameters, and the spatial transformation parameters, perform spatial reconstruction on the character image data," may include: Step S1500: Update the real-time station position based on the target station position.

[0136] Step S1502: Apply interpolation update rules to the real-time station position to transition the current station position to the target station position.

[0137] In practical applications, character positions can be updated based on target positions to generate corresponding real-time positions. For example, the updated position of the character in the current frame can be determined based on the coordinates of the target position and the current position. In some implementations, the target position can be used as a reference position for character position updates, and the character can gradually approach the target position by updating frame by frame. Furthermore, interpolation update rules can be applied to the real-time positions to achieve a smooth transition from the current position to the target position. For example, exponential decay interpolation can be used to update the character position, and the update rule can be expressed by the following formula: current_pos = current_pos 0.85 + target_pos 0.15 Here, `current_pos` represents the character's current position coordinates in the current frame, and `target_pos` represents the character's target position coordinates. Using the above update rules, the current position can be adjusted towards the target position proportionally in each frame, gradually approaching the target location. Because the current position is partially updated in each frame, the problem of the character appearing to jump abruptly in the frame can be effectively avoided.

[0138] In some embodiments, the interpolation calculations described above can be performed frame-by-frame according to the video frame update frequency, for example, performing a position update operation once in each frame rendering cycle. By continuously iterating the above update rules, the character's position can gradually approach the target position in a relatively short time. For example, the character's position can reach approximately 95% of the target position within about 200ms, thereby achieving relatively fast position adjustment while ensuring a smooth transition.

[0139] In this embodiment, by generating real-time positions based on target positions and applying interpolation update rules to them, a smooth transition can be achieved when the character's position changes, thereby avoiding abrupt changes in the character's position on the screen and improving the continuity of movement and visual performance of the virtual character in interactive scenes.

[0140] In optional embodiments, such as Figure 16 As shown, step S208, "performing spatial reconstruction of the character image data based on the target position, the display parameters, and the spatial transformation parameters," may further include: Step S1600: Perform a horizontal flip or leave the character image data unchanged based on the orientation correction parameters.

[0141] Step S1602: Perform scaling transformation on the character image data based on the display size reference parameter and the scaling parameter.

[0142] Step S1604: When the perspective compensation parameters meet the preset conditions, apply vertical compression compensation to the background character.

[0143] In practical applications, character semantic metadata can include a `height_ratio` parameter representing the character's height proportion within the original footage. This parameter reflects the character's basic height ratio in the frame. By using this `height_ratio` as the base scaling parameter `Base_Scale`, the character's basic display size in the video frame can be determined. Furthermore, the character's display size can be adaptively adjusted by incorporating the character's depth value (Z) in the stage space. For example, the character's scaling parameters can be determined using the following formula: Display_Scale = Base_Scale (0.5 + 0.5 Z) Here, Display_Scale represents the scaling ratio of the character's display in the video frame, Base_Scale represents the character's base display size, and Z represents the character's depth value in the stage space. Furthermore, scaling transformations can be performed on the character image data based on the display size baseline parameter and scaling parameters to adjust the character's size after spatial reconstruction. When it is detected that the character's current facing direction is inconsistent with the target standing position direction, the character image can be corrected for orientation. For example, when the character defaults to facing right and its target standing position is on the right side of the stage, the character image can be horizontally flipped to face the center of the stage. When the character defaults to facing left and its target standing position is on the left side of the stage, the original orientation of the character image can be maintained. When the character's depth value Z is less than a preset threshold (e.g., 0.3), it can be determined that the character is in the background area of ​​the stage. In this case, the character's display height can be slightly compressed according to the following formula: Display_Height = Display_Height (0.9 + 0.1 Z) This method allows for a certain degree of vertical compression of background characters, thereby reducing visual distortion caused by perspective and making the spatial proportions of the characters in the picture more natural.

[0144] In this embodiment, by sequentially performing image transformation operations such as orientation correction, size scaling, and perspective compensation on the character image data, the presentation effect of the virtual character in the video screen can be kept consistent with the standing relationship in the stage space, thereby improving the spatial performance effect of the virtual character in the interactive scene and enhancing the visual consistency of the overall picture.

[0145] In optional embodiments, such as Figure 17 As shown, step S208, "performing spatial reconstruction of the character image data based on the target position, the display parameters, and the spatial transformation parameters," may further include: Step S1700: Establish a collision detection area for each virtual character.

[0146] Step S1702: Detect whether the collision detection areas of different virtual characters intersect.

[0147] Step S1704: In the case of intersection, adjust the target position and / or display parameters of at least one virtual character to avoid collisions. The collision detection area is a bounding box, which includes at least a center point and half-width and half-height parameters.

[0148] In practical applications, after determining the initial positions of each virtual character, a spatial occupancy analysis can be performed on each virtual character. Specifically, the actual pixel area of ​​the character can be calculated based on the rendering result of the character in the current frame on the screen, and a corresponding outline polygon can be generated based on the character's outline to describe the actual area occupied by the character on the screen. On this basis, a corresponding bounding box can be generated based on the outline range of the character, and the center point of the bounding box and the half-width and half-height parameters can be used as descriptive parameters for the collision detection area, thereby establishing the collision detection area for each virtual character. By detecting whether the bounding boxes of different characters intersect, it can be determined whether there is a spatial conflict between the characters. When a collision or potential overlap is detected, the target position and / or display parameters of at least one character can be dynamically adjusted for avoidance. For example, the character position can be translated along the X-axis to maintain a minimum safe distance between characters. At the same time, the Z-axis depth or Y-axis height can be adjusted appropriately to enhance the sense of spatial separation. The avoidance adjustment can be combined with the character's semantic metadata and current display parameters, such as character size, orientation, display scaling ratio, etc., to ensure that the character display remains natural and consistent with the stage layout after the avoidance operation.

[0149] In some embodiments, the presence of a character at the edge of the screen can be detected by the character's outline. For example, if the character's outline is less than a preset percentage (e.g., 5% of the screen width or height) from the screen boundary, the character is deemed to pose a risk of edge occlusion. When the character approaches the screen edge, its position can be slightly adjusted inwards to prevent the character from being cropped from the screen.

[0150] In this embodiment, by establishing a collision detection area for each virtual character and making avoidance adjustments when necessary, the overlapping or occlusion of characters on the stage display can be effectively avoided.

[0151] Step S210 The output is a synthesized video stream of the spatial reconstruction result.

[0152] The spatial reconstruction result can include a set of character image data after spatial positioning adjustments. This set of character image data can include the target display position of each virtual character in the display plane, the corresponding image data, and parameter information related to the display effect. Based on the spatial reconstruction result, the final display form of each virtual character in the current video frame can be determined. In some embodiments, the spatial reconstruction result corresponding to each virtual character can be drawn onto the screen area according to the screen size of the display plane, thereby obtaining a complete video frame. After generating consecutive video frames, the video frames can be encapsulated to form transmittable video stream data. The composite video stream can be sent to a live distribution node, which further distributes the composite video stream to multiple viewing terminals, allowing multiple users to watch the live broadcast simultaneously.

[0153] The following will provide further illustrative examples of how to output a synthesized video stream through more specific embodiments.

[0154] In optional embodiments, such as Figure 18 As shown, step S210 may include: Step S1800: Sort each virtual character by level based on the depth Z of the target station or the level parameter.

[0155] Step S1802: Determine the occlusion relationship based on the sorting results and generate the rendering order.

[0156] In practical applications, to achieve reasonable visual overlay on a virtual stage, virtual characters can be sorted according to their depth Z and layer parameters in the 3D stage space. For example, characters with a depth less than 0.5 are assigned to the background character layer, and characters with a depth greater than or equal to 0.5 are assigned to the foreground character layer, so that the distance relationship can be correctly reflected during rendering. Special effects elements (such as particle effects, lighting effects, etc.) can also be assigned to different priority effect layers to ensure that visual effects are not obscured or misaligned by characters. The occlusion relationship between characters can be determined based on the sorting results, and the final rendering order can be generated. During actual rendering, each layer can be processed sequentially in the rendering pipeline, starting from the furthest background layer and rendering layer by layer to the foreground character layer and high-priority effect layers, ensuring that elements in each layer are correctly overlaid according to depth and priority, thereby avoiding visual overlap or occlusion errors.

[0157] In this embodiment, by sorting and generating the rendering order based on the target position depth and layer parameters, it can be ensured that the display layers of virtual characters, special effects and backgrounds on the stage are reasonable and the occlusion relationship is clear, thereby improving the visual consistency and stage performance of the final composite image, while ensuring the continuity and immersion of the user's viewing experience.

[0158] In optional embodiments, such as Figure 19 As shown, step S210 may further include: Step S1900: Perform layer-by-layer compositing in the order of at least background layer, background character layer, and foreground character layer.

[0159] Step S1902: Perform layer-by-layer alpha blending based on the transparency channel to generate composite image data.

[0160] In practical applications, to ensure visual consistency and spatial stability of the final output image, elements in the virtual stage can be rendered and composited in layers. For example, each layer can be processed sequentially according to a preset rendering hierarchy, such as starting with the background image or video at the bottom, then rendering the background character layer, effects layer A, foreground character layer, and effects layer B in sequence, until the entire image is complete. Layered rendering allows for independent processing of elements of different depths and priorities, preventing foreground elements from obscuring or misaligning the background and background characters, while ensuring the scalability of effects and character interactions and the consistency of visual effects. Furthermore, by processing the transparency channel information of each layer through layer-by-layer alpha blending, the overlay and semi-transparent effects of image elements can be achieved. For example, when rendering the background character layer, the RGBA data of the character image can be alpha-blended with the background layer to maintain the smoothness of the character's edge transparency transition. When rendering the foreground character or high-priority effects layer, using transparency channels for blending ensures the correct overlay of effects such as particles and lighting with the character and background, avoiding compositing distortion, color overflow, or occlusion errors.

[0161] In this embodiment, by compositing layer by layer in hierarchical order and combining alpha blending with a transparency channel, a stable, clear and consistent final image output of multiple characters and special effects scenes in a virtual stage can be achieved, avoiding image distortion or inconsistency.

[0162] In optional embodiments, such as Figure 20 As shown, before outputting the synthesized video stream, the method may further include: Step S2000: Read stage configuration data.

[0163] Step S2002: Load the stage background resources and initialize the rendering pipeline.

[0164] The stage configuration data includes at least stage space coordinate system parameters and / or the space transformation parameters.

[0165] Stage configuration data may include stage space coordinate system parameters, area anchor point positions, and spatial transformation parameters, which can be used for subsequent character positioning calculations, spatial reconstruction, and rendering operations. Loaded background resources may include static images, video frames, or 3D scene data. In this embodiment, by reading the stage configuration, loading background resources, and initializing the rendering pipeline, it can be ensured that the output composite video stream has a stable image structure and correct spatial mapping relationships.

[0166] The following will provide further illustrative examples of the specific methods for automatically switching lenses through more concrete embodiments.

[0167] In optional embodiments, such as Figure 21As shown, the video stream output method may further include: Step S2100: Determine the target shot based on the interactive semantic state information.

[0168] Step S2102: Generate virtual camera parameters based on the target lens.

[0169] Step S2104: Update the spatial transformation parameters and / or the display parameters based on the virtual camera parameters.

[0170] Step S2106: Output the synthesized video stream based on the updated spatial transformation parameters and / or display parameters.

[0171] For example, a suitable target shot can be selected from a predefined shot state library based on real-time interactive semantic state information. The shot state library may include shot ID, shot name, field of view, focus object, and applicable scene. For instance, a wide-angle shot CAM_01 for two people on stage is suitable for the initial or stable period, covering the X-axis [-0.8, 0.8] and Z-axis [0.2, 1.0] and focusing on both characters. A medium shot CAM_02 for the dominant party is suitable for the climax period, focusing on the leading character. Close-ups of party A (CAM_03) and party B (CAM_04) can be used to focus on characters specified by the audience. A panoramic stage shot CAM_05 can be used to display the entire stage during the settlement period.

[0172] In practical applications, after acquiring the semantic state information of the interaction, the target shot can be determined based on the current interaction stage and the leading party's information, and corresponding virtual camera parameters can be generated based on the target shot. For example, when the interaction enters a stage where the leading party continues to lead, a medium shot targeting the leading party can be selected as the target shot, and corresponding virtual camera parameters can be generated. These virtual camera parameters can include parameters such as camera position=[cam_x,cam_y,cam_z], camera focus look_at=[target_x,target_y,target_z], field of view (fov), and aspect ratio.

[0173] In some embodiments, lens motion control can be performed based on virtual camera parameters. For example, when it is necessary to zoom in on the scene, the camera depth coordinates cam_z=lerp(current_z,target_z,t) can be adjusted by an interpolation function to achieve a frame-by-frame zoom-in effect; when it is necessary to follow the character's movement laterally, smooth translation can be achieved by cam_x=lerp(current_x,target_x,t); when it is necessary to highlight the leading character, the camera focus can be gradually moved towards the position of the leading character, for example, look_at=lerp(current_focus,leader_position,t) to achieve a focus following effect.

[0174] In some embodiments, transition effects can also be applied during shot switching, such as performing a fade-in / fade-out transparency transition of about 300ms between different shots, superimposing a directional motion blur effect when performing a fast push-in or pull-out shot, or applying a slight Gaussian blur (e.g., radius 1-2px) to non-focus characters to enhance the sense of depth of the image and the prominence of the subject.

[0175] In this embodiment, by dynamically selecting the target lens, the automatic switching of the view can be achieved, which solves the problems of single lens perspective and lack of directing ability in traditional systems, realizes automatic focusing of the focus character, and improves the viewing experience and the professionalism of the video.

[0176] In optional embodiments, such as Figure 22 As shown, step S2100, "determining the target shot based on the interactive semantic state information," may include: Step S2200: Obtain audience attention information.

[0177] Step S2202: If the audience attention information indicates that the audience is paying attention to one side of the interaction, determine the close-up shot of the one side of the interaction as the target shot.

[0178] Step S2204: In the case where the audience attention information does not indicate unilateral attention, the target shot is determined from the preset shot set based on the interaction state.

[0179] In practical applications, audience attention information can include data that complies with data compliance guidelines (audience preference tags, historical viewing behavior, or real-time interaction choices, etc.), such as audience markings like "only watch A," "only watch B," or options to follow a particular side in an interactive questionnaire. When an audience marking "only watch A" is detected, CAM_03 can be set as the target shot; when an audience marking "only watch B" is detected, CAM_04 can be set as the target shot. In the absence of a single-sided attention instruction, for example, when the interaction is in the initial or stable phase, it can be determined that the current stage is a normal confrontation, and a wide-angle shot of both characters on stage (such as CAM_01) can be set as the target shot to show the initial confrontation between the two sides. When the interaction enters a tug-of-war phase, further judgment can be made by combining rhythm tags: if the current rhythm characteristic is "explosion," it indicates a sudden surge in interaction intensity, and the shot can be cut to a medium shot of the dominant side (such as CAM_02) to enhance the visual tension; if the rhythm does not explode, the wide-angle shot (such as CAM_01) can be maintained. When the interaction reaches its climax, if one side is identified as having a significant advantage, the target camera can be locked onto a medium shot of the advantageous side (e.g., CAM_02), using a magnified visual language to highlight the leader's performance. When the interaction enters the settlement phase, the target camera can be switched to a panoramic view of the stage (e.g., CAM_05), presenting the final score and overall stage effects with the widest possible field of view.

[0180] In this embodiment, by dynamically selecting the target shot by combining audience attention information and interaction status, personalized content display can be achieved, and manual intervention in switching shots can be reduced.

[0181] In optional embodiments, such as Figure 23 As shown, step S2102, "generating virtual camera parameters based on the target lens," may include: Step S2300: Determine the camera position, gaze point, field of view, and / or aspect ratio in the virtual camera parameters.

[0182] Step S2302: Apply interpolation update rules to the camera position and / or the gaze point.

[0183] In practical applications, when switching target shots based on changes in interactive semantic state (e.g., transitioning from a stable period to a climax), directly replacing the current virtual camera parameters with those of the new shot would result in abrupt, hard cuts in the live stream, leading to poor viewing quality. Therefore, after obtaining the camera position and gaze point (starting point) of the current frame and the camera position and gaze point (ending point) of the new target shot, interpolation update rules can be applied to these coordinate parameters. For example, a nonlinear interpolation algorithm with easing characteristics (such as an Ease-in-out function) can be used. Thus, during the transition period of shot switching, the spatial displacement of the camera and the shift of the gaze point can exhibit a dynamic trajectory of "slow start—intermediate acceleration—smooth stop."

[0184] In this embodiment, by applying interpolation update rules to the camera position and / or gaze point during shot switching, the jump and disjointed feeling that may be caused by automatic shot scheduling is avoided, thereby improving the smoothness of the shot movement and the viewing experience of the audience.

[0185] In optional embodiments, such as Figure 24 As shown, the method may further include lens switching control operations: Step S2400: In the absence of a manual camera switching instruction, the target camera is determined based on a preset director decision tree.

[0186] Step S2402: If the time since the last shot switch is less than a preset duration threshold, keep the current shot unchanged.

[0187] For example, during the interaction, the interaction data (such as the gift value) may change drastically multiple times in a very short period of time (such as within 1 second), and the system may quickly switch back and forth between "medium shot of the dominant party" and "two-person panoramic shot". In order to avoid "flickering broadcast" caused by data jitter as much as possible, the shot switch can be triggered only if the time since the last shot switch exceeds a preset change threshold.

[0188] In this embodiment, by controlling the camera switching based on time thresholds and data change amplitude, the real-time response of the director's decision-making can be ensured while minimizing frequent camera switching caused by fluctuations in interactive data. This improves the viewer's visual continuity and viewing experience, and reduces viewing discomfort caused by camera shake.

[0189] In some embodiments, the stage coordinate system can be extended to a multi-person layout structure to enable the display of three or more characters in the same scene. Specifically, the original linear positioning can be extended to a circular or polygonal layout. For example, multiple positioning nodes distributed at equal angles can be generated in the stage coordinate system according to the number of characters, and each character can be mapped to the corresponding node position. After determining the target positioning of each character, collision detection, gaze orientation adjustment, and perspective scaling calculation can be performed. This allows for the stable display of multiple characters within the same stage space, avoiding character occlusion or positioning chaos, thereby expanding the system's application capabilities in scenarios such as multi-person interactive live streaming.

[0190] In some embodiments, the stage coordinate system can be mapped to world coordinates in the 3D engine to construct a 3D stage scene. Specifically, the position parameters of characters on the stage can be directly mapped to spatial coordinates in the 3D engine, and the rendering capabilities provided by the 3D engine can be used to uniformly process characters, backgrounds, and lighting. For example, spatial movement of characters and camera following can be achieved on the 3D stage using 3D cameras, lighting models, and 3D stage resources. This can enhance the spatial hierarchy and visual presentation of the image, making the live broadcast more immersive and expressive.

[0191] In some embodiments, artificial intelligence models can be used to perform sentiment analysis on live stream comments, and the analysis results can be used as supplementary information to the interactive semantic state information. Specifically, real-time comment text can be categorized or scored for sentiment, such as determining whether the overall sentiment of the comments is supportive of one side, encouraging interaction, or expressing neutral emotions. The sentiment score can then be integrated with interaction metrics such as gift data to update the interactive semantic state information. This allows the system to not only make camera and positioning decisions based on gift or points data, but also adjust its display strategy based on audience emotional feedback, thereby further enhancing the interactivity of the visuals and audience engagement.

[0192] To make this application easier to understand, the following is combined with... Figures 25-26 Two exemplary applications are provided.

[0193] Example Application 1: S100 loads the stage configuration and initializes the system environment, including: reading stage coordinate system parameters, loading background resources (such as background images, videos, or 3D scenes), and initializing the rendering pipeline for subsequent image compositing.

[0194] S102, Establish a data connection. Specifically, connect to the interface provided by the live streaming platform, subscribe to the PK event stream, and obtain basic information about the participating streamers A and B, such as user IDs, avatar resources, and live stream addresses.

[0195] S104, start the various functional sub-modules. Specifically, start the PK semantic parsing module, the character metadata extraction module, the camera decision module, and the rendering output module.

[0196] S106, Acquire raw video frames. Specifically, acquire real-time video frames from broadcaster A and broadcaster B respectively, and align the two video frames according to their timestamps to ensure that the two video streams are on the same time reference when the images are subsequently composited.

[0197] S108, perform PK semantic parsing. Specifically, fetch the latest gift data and update the PK state machine, calculate the current advantage score based on the gift increment or score change, determine the current dominant player, and output the PK semantic state information: {pk_state, advantage_score, leader}.

[0198] S110, Extract character metadata. Specifically, perform keying on the video frames of anchors A and B, separate the character area from the video background, and detect the character's orientation, size, and position in the frame, thereby outputting the character image and corresponding metadata: {rgba_image, metadata}.

[0199] S112, Perform intelligent positioning calculation. Specifically, determine the target positions of both characters based on the PK semantic state, and smoothly transition the position changes through an interpolation function. At the same time, perform collision detection and avoidance processing between characters, and output the target positions and display scales of both characters: {Pos_A, Pos_B, Scale_A, Scale_B}.

[0200] S114, Perform spatial reconstruction. Specifically, correct the character's orientation based on the character's metadata, and calculate the perspective scaling ratio based on the stage depth relationship, thereby generating a mapping relationship from stage coordinates to screen coordinates.

[0201] S116, Execute shot decision. Specifically, determine the target shot state based on the PK state, the current dominant party, and audience preference information, and calculate virtual camera parameters accordingly, such as camera position, field of view, and focus position, thereby outputting the virtual camera parameters: {cam_position, cam_fov, look_at}.

[0202] S118, Execute rendering compositing. Specifically, sort the character layers according to their Z-depth, perform alpha blending layer by layer in depth order, crop the image according to the virtual camera's field of view, and overlay special effects elements and interface components.

[0203] S120 performs video encoding and output. Specifically, it encodes the composited footage and pushes the encoded live stream to the content delivery network for real-time playback by viewers.

[0204] In this exemplary application, by sequentially executing processing steps such as stage configuration loading, data connection establishment, video frame acquisition, PK semantic parsing, character metadata extraction, intelligent positioning calculation, spatial reconstruction, camera decision-making, and rendering output, the spatial relationships within the virtual stage can dynamically change with the interaction process, intuitively mapping the adversarial state into changes in spatial layout. This enhances the spatial expression and adversarial performance of the virtual anchor's interactive scenes, avoids the monotonous stage presentation caused by traditional fixed positions, and improves the user experience.

[0205] Example Application 2: Background: Streamer A (female virtual character "A") and streamer B (male virtual character "B") are having a live PK battle. The battle lasts for 5 minutes, and the stage background is a sci-fi style virtual city. The viewer "User 001" chooses to watch both sides at the same time.

[0206] S20: T=0s~10s——PK initialization phase.

[0207] 1. The output of the PK semantic parsing module is as follows: { "pk_state":"INIT"; "advantage_score": 0.0; "leader": null; "time_remaining":300 } 2. The stage space coordinate system is set as follows: Initial position of station A: (-0.4, 0.5, 0.5).

[0208] Initial position of station B: (+0.4, 0.5, 0.5).

[0209] 3. Character metadata extraction: A: Facing forward (2.1°), height percentage 0.68, centroid (320, 480).

[0210] B: Facing forward (-1.3°), height percentage 0.75, center of mass (960, 460).

[0211] 4. Intelligent station positioning module: Both sides are positioned on the left and right sides of the stage, at the same depth (Z=0.5).

[0212] Display size (1080p resolution) is calculated based on metadata: A: 1080 0.68 1.0 = 734px.

[0213] B: 1080 0.75 1.0 = 810px.

[0214] 5. Camera Decision: Select the CAM_01 wide-angle lens for two people on the same stage.

[0215] Virtual camera position: (0, 0.5, -1.5).

[0216] Field of view: 50°, covering both sides.

[0217] 6. Rendering output: The scene depicts a sci-fi city background with person A (on the left) and person B (on the right), standing symmetrically.

[0218] The countdown timer at the top of the screen reads: "05:00".

[0219] S21: T=10s~90s——Stability period.

[0220] 7. The PK semantic parsing process is as follows: Gift data continued to flow in, with both sides alternating in slight lead.

[0221] The advantage_Score fluctuates in the range of [-0.12, +0.10].

[0222] Status determination: STABLE stationary period.

[0223] 8. Intelligent station positioning module: Triggering fine-tuning rules: The current advantage score is +0.08, indicating that A is slightly ahead.

[0224] Fine-tune depth Z_offset=0.08 0.1 = 0.008.

[0225] Depth Z of A: 0.5 + 0.008 = 0.508 Depth Z of point B: 0.5 - 0.008 = 0.492 Visual effect: A is slightly forward, but the difference is not obvious.

[0226] 9. Shot Decisions: Maintain the CAM_01 wide-angle lens for both people on the same stage, and keep the camera position unchanged.

[0227] 10. Special Effects System: A pale blue halo is generated under the feet of person A, and its intensity varies with the advantage_score.

[0228] S22: T=90s~150s——the seesaw period.

[0229] 11. PK semantic parsing: A change in rhythm was detected: a certain fan, B, began to fight back, and the rate of gift inflow suddenly increased.

[0230] At T=95s, the advantage_score reversed from +0.10 to -0.18.

[0231] State transition: CONTEST tug-of-war period.

[0232] Rhythm tag: Burst.

[0233] 12. Intelligent station positioning module: Triggering the tug-of-war period rule: advantage_score=-0.18 (B is in the lead).

[0234] A new location: X = -0.4 + (-0.18) 0.2 = -0.436 Z = 0.5 - 0.18 0.2 = 0.464 New location for B: X = +0.4 - (-0.18) 0.2 = +0.436 Z = 0.5 + 0.18 0.3 = 0.554 Smooth transition: Position adjustment completed in 1.2 seconds.

[0235] Visual effect: Person B clearly moves forward and slightly moves closer to the center, while Person A moves backward.

[0236] 13. Shot Decision: The rhythm tag rhythm="burst" was detected and the pk state pk_state=CONTEST.

[0237] Decision: Maintain the dual-person wide-angle lens CAM_01.

[0238] However, a slight zoom-in was performed, adjusting the virtual camera parameters "position": [cam_x, cam_y, cam_z], changing cam_z from -1.5 to -1.3.

[0239] Effect: The image is slightly zoomed in to focus on the dynamics of both parties.

[0240] 14. Special Effects System: Red energy particles appear around a certain B, and their density increases with the dominance value.

[0241] The intensity of the halo around A decreases.

[0242] 15. At T=120s, the gift quantity reverses again: advantage_score returned to +0.22 (A overtook it).

[0243] Automatic positioning adjustment: Person A moves forward, Person B moves backward.

[0244] The process is smooth and without abrupt changes.

[0245] S23: T=150s~240s——Climax period 16. PK semantic parsing: At 155 seconds, fan A started showering gifts.

[0246] advantage_score quickly climbed to +0.45.

[0247] The lead lasted for more than 10 seconds.

[0248] State transition: CLIMAX Climax.

[0249] 17. Intelligent Positioning Module: Triggering Climax Period Rules: Party A (the dominant party): X=0.45 0.1 = 0.045 (moving towards the center of the stage) Z=0.8 (Foreground) Y=0.5 (Maintain) Party B (the disadvantaged party): X=-0.45 0.6 = -0.27 (moving backward to the left) Z=0.3 (background) Y=0.4 (Longitudinal subsidence) Scale adaptation: Display size of component A = Original size of component A (0.5+0.5 0.8) = Original size of A 0.9 Display size of component B = Original size of component B (0.5+0.5 0.3) = Original size of B 0.65 18. Details of spatial reconstruction: Metadata for a certain element A shows its orientation as "front," matching the foreground position, so there's no need to flip it.

[0250] Metadata for a certain B shows its orientation as "15° to the right" and its position as left rear, according to the system.

[0251] It needs to be horizontally mirrored so that it faces the center of the stage.

[0252] Perspective distortion compensation: At Z=0.3, the vertical compression coefficient of point B is 0.9 + 0.1. 0.3 = 0.93 19. Shot Decisions: Decision tree output: Dominant side, mid-range camera CAM_02.

[0253] Virtual camera parameter adjustment: position:(0.02,0.55,-1.0) / / Close to a certain A look_at:(0.045,0.5,0.8) / / Focus is locked on a specific A fov: 40° / / Field of view angle, narrowing the field of view Transition: 300ms fade in / fade out 20. Rendering effect: Main subject of the image: A occupies the center-right of the screen, taking up approximately 55% of the screen size.

[0254] B is located to the left rear, its size reduced to about 25%, and slightly blurred (blur radius 1.5px).

[0255] Special effect: A high-density blue particle stream surrounds object A, while particles around object B are sparse and have a dull color.

[0256] UI: The top of the screen displays "A is leading by 45%", and the progress bar updates dynamically.

[0257] S24: T=240s~270s——Settlement period 21. PK semantic parsing: PK remaining time time_remaining<30s.

[0258] State transition: SETTLE settlement period.

[0259] Ultimately, advantage_score = +0.38 (A is in the lead).

[0260] 22. Intelligent Station Positioning Module: The positions of both sides began to slowly return to symmetry: The location of target A is (-0.3, 0.5, 0.6).

[0261] The location of target B is (+0.3, 0.5, 0.6).

[0262] Smooth transition duration: 8 seconds.

[0263] 23. Shot Decisions: Switch to the stage panoramic camera CAM_05.

[0264] Adjust virtual camera parameters: position:(0,0.6,-2.0) / / Pull away.

[0265] look_at:(0,0.5,0.5) / / Overview of the whole.

[0266] fov:60° Transition effect: 500ms smooth zoom.

[0267] 24. Rendering effect: The screen presents the complete stage view, with the background cityscape fully unfolded.

[0268] Both characters are visible simultaneously and are similar in size.

[0269] A countdown effect appears in the center of the screen: "10...9...8...".

[0270] S25: T=270s~300s——End Phase 25. PK semantic parsing: State transition: END (End state).

[0271] Final result: A wins.

[0272] 26. Shot Decisions: Execution result display logic: Quickly switch to a close-up shot of A (CAM_03).

[0273] Lasts for 2 seconds.

[0274] Switch back to the panoramic stage shot CAM_05.

[0275] 27. Special Effects System: A victory effect is triggered around A: golden ribbons and firework particles.

[0276] Character B's transparency has dropped to 70%, indicating a "lost" state.

[0277] 28. Rendering Output: The results panel in the center of the screen reads: "Congratulations to A for winning!"

[0278] The background music switched to a victory theme.

[0279] Both characters perform preset actions (A cheers, B bows).

[0280] 29. System Reset: Five seconds later, the character smoothly exits the scene.

[0281] Stage space coordinate system reset.

[0282] The PK semantic state machine returns to the IDLE standby state.

[0283] Waiting for the next round of competition or the end of the live stream.

[0284] Example 2 Figure 27 The diagram schematically illustrates a video stream output device according to Embodiment 2 of this application. This device can be divided into one or more program modules. One or more program modules are stored in a storage medium and executed by one or more processors to complete the embodiment of this application. The program module referred to in this embodiment is a series of computer program instruction segments capable of performing a specific function. The following description will specifically introduce the function of each program module in this embodiment. For example... Figure 27 As shown, the device 1600 may include: an acquisition module 1610, a first generation module 1620, a second generation module 1630, a determination module 1640, a reconstruction module 1650, and an output module 1660, wherein: The acquisition module 1610 is used to acquire event data corresponding to the interactive live broadcast process and real-time media frame data of at least two virtual characters; The first generation module 1620 is used to generate interactive semantic state information representing the interactive process based on the event data; The second generation module 1630 is used to generate character image data of each virtual character based on the real-time media frame data; The determining module 1640 is used to determine the target position of each virtual character in a pre-configured stage space coordinate system based on the interactive semantic state information, and to determine the display parameters corresponding to the target position; the stage space coordinate system and the display plane have spatial transformation parameters corresponding to each other; Reconstruction module 1650 is used to perform spatial reconstruction on the character image data based on the target position, the display parameters, and the spatial transformation parameters; Output module 1660 is used to output the synthesized video stream of the spatial reconstruction result.

[0285] In an optional embodiment, acquiring real-time media frame data of at least two virtual characters includes: Acquire at least two video frames; Add a frame timestamp to each of the video frames in the aforementioned stream; Based on the timestamp, the timestamps of each event in the event data are aligned with those of each video frame.

[0286] In an optional embodiment, the generation of interactive semantic state information includes: The interaction state is obtained by updating the interaction state based on the event data; Calculate the advantage value based on the event data; The output includes at least the interaction semantic state information of the interaction state and the advantage value.

[0287] In an optional embodiment, the generation of interactive semantic state information further includes: Leader indicator, tempo tag, and / or remaining time.

[0288] In an optional embodiment, calculating the advantage value based on the event data includes: Statistics on gift value and / or gift-giving frequency are based on a sliding time window; The dominance value and / or the rhythm label are updated based on the statistical results.

[0289] In an optional embodiment, the device 1600 further includes a configuration module for pre-configuring the operation of the stage space coordinate system: Define the horizontal X-axis, the vertical Y-axis, and the depth Z-axis; Set regional anchor points, which include at least the center point of the stage and the initial positions of at least two virtual characters.

[0290] In an optional embodiment, the device 1600 is further configured to generate the spatial transformation parameters: Generate mapping parameters from stage coordinates to display pixel coordinates; Generate scaling parameters related to depth Z so that different depths correspond to different display sizes.

[0291] In an optional embodiment, determining the target station location includes: The interactive semantic state information is input into the station mapping model or station mapping function to obtain the target station. Output the coordinates of the target position in the stage space coordinate system.

[0292] In an optional embodiment, the station mapping model or station mapping function includes station rules that switch according to interaction state, and the station rules include at least: In the initialization state, at least two virtual characters are set to symmetrical initial positions; During periods of stability or stalemate, the X-axis position and / or Z-axis depth of the leading and lagging players are adjusted differentially based on their advantage values. During the climax, the leading team's position is changed to the center or foreground area of ​​the stage, while the lagging team's position is changed to the edge or background area of ​​the stage.

[0293] In an optional embodiment, the display parameters include at least one of translation parameters, scaling parameters, and / or hierarchy parameters, and the display parameters are determined based on the interactive semantic state information.

[0294] In an optional embodiment, the spatial reconstruction includes: Real-time station position is obtained based on target station position updates; An interpolation update rule is applied to the real-time station position to transition the current station position to the target station position.

[0295] In an optional embodiment, generating character image data for each virtual character based on the real-time media frame data includes: Perform keying or segmentation processing on the real-time media frame data, or extract the region image containing the character; Output character image data containing alpha channels, or output character region image data for rendering.

[0296] In an optional embodiment, the device 1600 further includes an association module for: For each of the character image data streams, extract character semantic metadata for stage reconstruction; The semantic metadata of the character is associated with the corresponding image data of the character.

[0297] In an optional embodiment, the extraction of role semantic metadata includes: Determine the bounding rectangle of the character's pixel region; Determine the position of the character's center of gravity; Determine the character height ratio, which is the ratio of the height of the outer rectangle to the height of the video frame.

[0298] In an optional embodiment, the extraction of role semantic metadata further includes: Calculate the character's orientation angle based on the results of human keypoint detection; The orientation angle is mapped to an orientation category to obtain orientation information.

[0299] In an optional embodiment, determining the display parameters includes: Based on the interactive semantic state information, the display parameters are determined, and the character display size reference parameters are determined based on the character semantic metadata. The character's orientation correction parameters and / or perspective compensation parameters are determined based on the character's semantic metadata.

[0300] In an optional embodiment, the spatial reconstruction includes: The character image data is either horizontally flipped or left unchanged based on the orientation correction parameters. The character image data is scaled based on the display size reference parameter and the scaling parameter. When the perspective compensation parameters meet the preset conditions, vertical compression compensation is applied to the background characters.

[0301] In an optional embodiment, the spatial reconstruction further includes: Establish a collision detection zone for each virtual character; Detect whether the collision detection areas of different virtual characters intersect; In the case of intersection, the target position and / or display parameters of at least one virtual character should be adjusted to avoid collisions. The collision detection area is a bounding box, which includes at least a center point and half-width and half-height parameters.

[0302] In an optional embodiment, the output synthesized video stream includes: Based on the depth Z of the target station or the level parameter, sort the virtual characters by level; The occlusion relationship is determined based on the sorting results, and the rendering order is generated.

[0303] In an optional embodiment, the output synthesized video stream further includes: Perform layer-by-layer compositing in the order of at least background layer, background character layer, and foreground character layer; Layer-by-layer alpha blending is performed based on the transparency channel to generate composite image data.

[0304] In an optional embodiment, before outputting the synthesized video stream, the apparatus 1600 further includes an initialization module for: Read stage configuration data; Load stage background resources and initialize the rendering pipeline; The stage configuration data includes at least stage space coordinate system parameters and / or the space transformation parameters.

[0305] In an optional embodiment, the device 1600 is further configured to: The target shot is determined based on the interactive semantic state information; Virtual camera parameters are generated based on the target lens; Update the spatial transformation parameters and / or the display parameters based on the virtual camera parameters; Based on the updated spatial transformation parameters and / or display parameters, the synthesized video stream is output.

[0306] In an optional embodiment, determining the target shot based on the interactive semantic state information includes: Obtain audience attention information; When the audience attention information indicates that the audience should focus on one side of the interaction, the close-up shot of the one side of the interaction is determined as the target shot; If the audience attention information does not indicate unilateral attention, the target shot is determined from a preset shot set based on the interaction state.

[0307] In an optional embodiment, generating virtual camera parameters based on the target lens includes: Determine the camera position, gaze point, field of view, and / or aspect ratio in the virtual camera parameters; Apply interpolation update rules to the camera position and / or the gaze point.

[0308] In an optional embodiment, the device 1600 is also used for lens switching control operations: In the absence of a manual camera switching instruction, the target camera is determined based on a preset director decision tree; If the time since the last shot change is less than a preset duration threshold, the current shot remains unchanged.

[0309] Example 3 Figure 28 This illustration schematically shows a hardware architecture diagram of a computer device 10000 suitable for implementing a video stream output method according to Embodiment 3 of this application. In some embodiments, the computer device 10000 may be a terminal device such as a smartphone, wearable device, tablet computer, personal computer, in-vehicle terminal, game console, virtual device, workbench, digital assistant, set-top box, robot, etc. In other embodiments, the computer device 10000 may be a rack server, blade server, tower server, or cabinet server (including standalone servers or server clusters composed of multiple servers), etc. Figure 28 As shown, the computer device 10000 includes, but is not limited to: a memory 10010, a processor 10020, and a network interface 10030 that can communicate and be linked with each other via a system bus. Wherein: The memory 10010 includes at least one type of computer-readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 10010 may be an internal storage module of a computer device 10000, such as the hard disk or memory of the computer device 10000. In other embodiments, the memory 10010 may also be an external storage device of the computer device 10000, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 10000. Of course, the memory 10010 may also include both the internal storage module and the external storage device of the computer device 10000. In this embodiment, the memory 10010 is typically used to store the operating system and various application software installed on the computer device 10000, such as the program code of the method described in the foregoing embodiment. Furthermore, the memory 10010 can also be used to temporarily store various types of data that have been output or will be output.

[0310] In some embodiments, processor 10020 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other chip. Processor 10020 is typically used to control the overall operation of computer device 10000, such as performing control and processing related to data interaction or communication with computer device 10000. In this embodiment, processor 10020 is used to run program code stored in memory 10010 or process data.

[0311] Network interface 10030 may include a wireless network interface or a wired network interface, which is typically used to establish a communication link between computer device 10000 and other computer devices. For example, network interface 10030 is used to connect computer device 10000 to an external terminal via a network, establishing a data transmission channel and communication link between computer device 10000 and the external terminal. The network may be an intranet, the Internet, Global System for Mobile Communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, or other wireless or wired networks.

[0312] It should be pointed out that, Figure 28 Only computer devices with components 10010-10030 are shown; however, it should be understood that it is not required to implement all of the shown components, and more or fewer components may be implemented instead.

[0313] In this embodiment, the video stream output method stored in memory 10010 can also be divided into one or more program modules and executed by one or more processors (such as processor 10020) to complete the embodiment of this application.

[0314] Example 4 This application also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the steps of the method described in the foregoing embodiments.

[0315] In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the computer-readable storage medium may be an internal storage unit of a computer device, such as the hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device. Of course, the computer-readable storage medium may include both the internal storage unit and the external storage device of the computer device. In this embodiment, the computer-readable storage medium is typically used to store the operating system and various application software installed on the computer device, such as the program code of the method described in the foregoing embodiments. In addition, the computer-readable storage medium can also be used to temporarily store various types of data that have been output or will be output.

[0316] Example 5 This application also provides a computer program product, including a computer program that, when executed by a processor, implements the methods described in the above embodiments.

[0317] Obviously, those skilled in the art should understand that the modules or steps of the embodiments of this application described above can be implemented using general-purpose computer devices. They can be centralized on a single computer device or distributed across a network of multiple computer devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computer device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the embodiments of this application are not limited to any particular combination of hardware and software.

[0318] It should be noted that the above are merely preferred embodiments of this application and do not limit the scope of patent protection of this application. Any equivalent structural or procedural changes made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of this application.

Claims

1. A video stream output method, characterized in that, The method includes: Acquire event data corresponding to the interactive live broadcast process, as well as real-time media frame data of at least two virtual characters; Based on the event data, interactive semantic state information representing the interactive process is generated; Generate character image data for each virtual character based on the real-time media frame data; Based on the interactive semantic state information, the target position of each virtual character in the pre-configured stage space coordinate system is determined, and the display parameters corresponding to the target position are determined; the stage space coordinate system and the display plane have corresponding spatial transformation parameters; Based on the target position, the display parameters, and the spatial transformation parameters, spatial reconstruction is performed on the character image data; The synthesized video stream is the result of the output spatial reconstruction.

2. The method according to claim 1, characterized in that, The acquisition of real-time media frame data from at least two virtual characters includes: Acquire at least two video frames; Add a frame timestamp to each of the video frames in the aforementioned stream; Based on the timestamp, the timestamps of each event in the event data are aligned with those of each video frame.

3. The method according to claim 1, characterized in that, The generated interactive semantic state information includes: The interaction state is obtained by updating the interaction state based on the event data; Calculate the advantage value based on the event data; The output includes at least the interaction semantic state information of the interaction state and the advantage value.

4. The method according to claim 1, characterized in that, It also includes the operation of generating the spatial transformation parameters: Generate mapping parameters from stage coordinates to display pixel coordinates; Generate scaling parameters related to depth Z so that different depths correspond to different display sizes.

5. The method according to claim 1, characterized in that, The determination of the target station location includes: The interactive semantic state information is input into the station mapping model or station mapping function to obtain the target station. Output the coordinates of the target position in the stage space coordinate system.

6. The method according to claim 5, characterized in that, The station mapping model or station mapping function includes station rules that switch according to the interaction state, and the station rules include at least: In the initialization state, at least two virtual characters are set to symmetrical initial positions; During periods of stability or stalemate, the X-axis position and / or Z-axis depth of the leading and lagging players are adjusted differentially based on their advantage values. During the climax, the leading team's position is changed to the center or foreground area of ​​the stage, while the lagging team's position is changed to the edge or background area of ​​the stage.

7. The method according to claim 1, characterized in that, The display parameters include at least one of translation parameters, scaling parameters, and / or hierarchy parameters, and the display parameters are determined based on the interactive semantic state information.

8. The method according to claim 1, characterized in that, Based on the real-time media frame data, character image data for each virtual character is generated, including: Perform keying or segmentation processing on the real-time media frame data, or extract the region image containing the character; Output character image data containing alpha channels, or output character region image data for rendering.

9. The method according to claim 1 or 8, characterized in that, The method further includes: For each of the character image data streams, extract character semantic metadata for stage reconstruction; The semantic metadata of the character is associated with the corresponding image data of the character.

10. The method according to claim 9, characterized in that, The determination of display parameters includes: Based on the interactive semantic state information, the display parameters are determined, and the character display size reference parameters are determined based on the character semantic metadata. The character's orientation correction parameters and / or perspective compensation parameters are determined based on the character's semantic metadata.

11. The method according to claim 10, characterized in that, The spatial reconstruction includes: The character image data is either horizontally flipped or left unchanged based on the orientation correction parameters. The character image data is scaled based on the display size reference parameters and scaling parameters. When the perspective compensation parameters meet the preset conditions, vertical compression compensation is applied to the background characters.

12. The method according to claim 1, characterized in that, The spatial reconstruction also includes: Establish a collision detection zone for each virtual character; Detect whether the collision detection areas of different virtual characters intersect; In the case of intersection, the target position and / or display parameters of at least one virtual character should be adjusted to avoid collisions. The collision detection area is a bounding box, which includes at least a center point and half-width and half-height parameters.

13. The method according to claim 1, characterized in that, The output synthesized video stream includes: Based on the depth Z or level parameter of the target station, the virtual characters are sorted hierarchically. The occlusion relationship is determined based on the sorting results, and the rendering order is generated.

14. The method according to claim 1, characterized in that, Before outputting the synthesized video stream, the following is also included: Read stage configuration data; Load stage background resources and initialize the rendering pipeline; The stage configuration data includes at least stage space coordinate system parameters and / or the space transformation parameters.

15. The method according to claim 1, characterized in that, Also includes: The target shot is determined based on the interactive semantic state information; Virtual camera parameters are generated based on the target lens; Update the spatial transformation parameters and / or the display parameters based on the virtual camera parameters; Based on the updated spatial transformation parameters and / or display parameters, the synthesized video stream is output.

16. The method according to claim 15, characterized in that, Determining the target shot based on the interactive semantic state information includes: Obtain audience attention information; When the audience attention information indicates that the audience should focus on one side of the interaction, the close-up shot of the one side of the interaction is determined as the target shot; If the audience attention information does not indicate unilateral attention, the target shot is determined from a preset shot set based on the interaction state.

17. The method according to claim 15, characterized in that, The virtual camera parameters generated based on the target lens include: Determine the camera position, gaze point, field of view, and / or aspect ratio in the virtual camera parameters; Apply interpolation update rules to the camera position and / or the gaze point.

18. The method according to claim 15, characterized in that, It also includes camera switching control operations: In the absence of a manual camera switching instruction, the target camera is determined based on a preset director decision tree; If the time since the last shot change is less than a preset duration threshold, the current shot remains unchanged.

19. A video stream output device, characterized in that, The device includes: The acquisition module is used to acquire event data corresponding to the interactive live broadcast process and real-time media frame data of at least two virtual characters; The first generation module is used to generate interactive semantic state information representing the interaction process based on the event data; The second generation module is used to generate character image data for each virtual character based on the real-time media frame data. The determination module is used to determine the target position of each virtual character in a pre-configured stage space coordinate system based on the interactive semantic state information, and to determine the display parameters corresponding to the target position; the stage space coordinate system and the display plane have spatial transformation parameters corresponding to each other; The reconstruction module is used to perform spatial reconstruction on the character image data based on the target position, the display parameters, and the spatial transformation parameters. The output module is used to output the synthesized video stream of the spatial reconstruction results.

20. A computer device, characterized in that, include: At least one processor; and A memory communicatively connected to the at least one processor; wherein: The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 18.

21. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the method as described in any one of claims 1 to 18.

22. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 18.

Images