Digital human live broadcast system and method based on WebRTC

Abstract

The invention provides a WebRTC-based digital human live broadcast system and method, and the system comprises an interaction terminal layer which is used for collecting interaction data inputted by a user based on a WebRTC client, and transmitting the interaction data to a core service layer based on a WebRTC protocol. And the core service layer comprises a semantic understanding module, a voice synthesis module, a digital human driving module and an audio and video synchronization module, and is used for generating digital human audio and video streams with synchronized audio and picture according to the interaction data. And the platform adaptation layer is used for converting the digital human audio and video stream into a predefined streaming media format, and pushing the digital human audio and video stream after format conversion to a target live broadcast platform. According to the invention, low-delay transmission of real-time interaction data is realized through protocol adaptation, and the integration of the system and the compatibility with a live broadcast platform are improved.

Description

Technical Field

[0001] This invention relates to the field of digital human live streaming technology, specifically to a digital human live streaming system and method based on WebRTC. Background Technology

[0002] With the rapid development of artificial intelligence technology, digital human technology has been gradually applied to various fields such as live streaming, customer service, and content creation. Especially in the live streaming industry, combining digital human avatars with real-time interactive capabilities to create virtual anchors capable of intelligent dialogue with viewers has become an important trend for enhancing user engagement and content appeal. This "real-time interaction + human-like presentation" model places higher demands on the response speed, platform compatibility, and technological integration of live streaming systems.

[0003] Currently, digital human live streaming solutions in the industry typically rely on a combination of a series of independent technical components. For example, they use traditional application programming interfaces (APIs) to call large language models, speech synthesis modules, and digital human driving engines to achieve the corresponding functions. At the communication level, server-based relay solutions or simple push streaming protocols are used to push the live stream.

[0004] While the aforementioned technical solutions can achieve the basic functions of digital human live streaming to a certain extent, they typically suffer from the following drawbacks in live streaming scenarios with extremely high real-time requirements: First, the various functional modules (such as LLM, TTS, and Avatar) are mostly independent services, connected through serial API calls. This results in long data transmission links and multiple processing steps, leading to a high overall latency from user inquiry to digital human response, making it difficult to meet the "second-level" or even "millisecond-level" real-time interaction requirements in live streaming scenarios. Second, existing systems are mostly designed for general live streaming platforms (such as mainstream short video platforms), lacking dedicated adaptation for specific target live streaming platforms with specific audio and video encoding formats, streaming protocols, and authentication mechanisms. This can easily lead to problems such as streaming failure, video stuttering, or audio-visual asynchrony, resulting in poor stability of the digital human live streaming system. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a WebRTC-based digital human live streaming system and method, aiming to solve the technical problems of high real-time interaction latency and poor platform compatibility in existing digital human live streaming systems.

[0006] This invention discloses a WebRTC-based digital human live streaming system, characterized in that it includes an interactive terminal layer, a core service layer, and a platform adaptation layer that are sequentially connected in communication. The interactive terminal layer is configured to: collect user-input interactive data based on the WebRTC client, and transmit the interactive data to the core service layer based on the WebRTC protocol. The interactive data includes voice data and text data. The core service layer includes a semantic understanding module, a speech synthesis module, a digital human driving module, and an audio-video synchronization module; The semantic understanding module is configured to receive interaction data and perform intent recognition on the interaction data based on a pre-trained large language model to generate digital human response text. The speech synthesis module is configured to convert the digital human's response text into an audio stream; The digital human driving module is configured to: acquire the prosodic features of the audio stream and drive the preset digital human image model to generate the video stream corresponding to the audio stream; The audio-video synchronization module is configured to: perform timing calibration on the audio and video streams, and output a synchronized digital human audio-video stream; The platform adaptation layer is configured to convert the digital human audio and video stream into a predefined streaming media format and push the converted digital human audio and video stream to the target live streaming platform, with the streaming media format being compatible with the target live streaming platform.

[0007] Preferably, the interactive terminal layer executes the transmission of interactive data to the core service layer based on the WebRTC protocol, including: Encrypt the interactive data; Establish a secure communication channel with the core service layer based on the WebRTC protocol; Based on a secure communication channel, encrypted interactive data is transmitted to the core service layer.

[0008] Preferably, the semantic understanding module is configured with a live streaming scenario corpus; The semantic understanding module is also configured to perform contextual filtering and optimization of the digital human's response text based on a live streaming scenario corpus.

[0009] Preferably, the prosodic features of the audio stream include speech rate features, intonation features, and pause prosodic features; The digital human driving module executes the preset digital human avatar model to generate the video stream corresponding to the audio stream, including: The speech rate features are mapped to the lip-opening rhythm of the digital human avatar model; Map intonation features to dynamic changes in facial expressions of a digital human avatar model; The pause prosody features are mapped to pause actions in the digital human avatar model, including blinking or nodding. The audio stream is generated based on the rhythm of lip opening and closing, dynamic changes in facial expressions, and pauses. The video stream is encoded in H.264 format.

[0010] Preferably, the platform adaptation layer includes a protocol adaptation module, an authentication module, and a streaming control unit; The protocol adaptation module is configured to convert the digital human audio and video streams into a preset push streaming format based on RTMP, which is compatible with the target live streaming platform. The authentication and authorization module is configured to integrate the authentication interface of the target live streaming platform and perform identity verification and push streaming permission application for the live streaming account; The streaming control unit is configured to: monitor the network bandwidth of the target live streaming platform in real time, adjust the bitrate adaptively based on the network bandwidth, and initiate a reconnection mechanism when the network is interrupted.

[0011] Preferably, the streaming control unit is configured to: perform adaptive bitrate adjustment when the network bandwidth is in the range of 300kbps to 2Mbps; and the timeout time for the interrupted stream reconnection mechanism is less than or equal to a preset timeout threshold.

[0012] This invention discloses a WebRTC-based digital human live streaming method, applied to the aforementioned WebRTC-based digital human live streaming system. The WebRTC-based digital human live streaming method includes: Collect user-input interaction data based on a WebRTC client; A pre-trained large language model is used to perform intent recognition on the interactive data to generate digital human response text, and an audio stream is generated based on the digital human response text; The digital human avatar model generates a video stream based on the audio stream; Timing calibration is performed on the audio and video streams to obtain a synchronized audio-visual digital human audio-visual stream; Perform protocol conversion on the digital human audio and video stream, and push the processed digital human audio and video stream to the target live streaming platform.

[0013] Preferably, the interaction data collected from user input via a WebRTC client includes: The WebRTC interface is used to receive user voice input and / or text input in order to obtain interactive data. The interactive data is encrypted using a preset protocol, which includes the SRTP protocol. The encrypted interactive data is buffered to eliminate transmission delays caused by network jitter.

[0014] Preferably, a pre-trained large language model is used to perform intent recognition on the interaction data to generate digital human response text, including: The pre-trained large language model is invoked to perform intent recognition and semantic parsing on the interactive data to obtain the initial text response. The large language model is based on the Transformer architecture. Based on a pre-configured corpus of live streaming scenarios, the initial text response is filtered and optimized for context to obtain the digital human's response text.

[0015] Preferably, the WebRTC-based digital human live streaming method further includes: Real-time monitoring of network bandwidth during live streaming on the target platform; Reduce the streaming bitrate when the network bandwidth is lower than the preset bandwidth threshold; In the event of a streaming interruption, a reconnection mechanism will be activated, and streaming will resume upon successful reconnection.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention deeply integrates the WebRTC protocol into the communication link between the interactive terminal layer and the core service layer, achieving low-latency transmission and encrypted protection of interactive data. This reduces link latency from the data acquisition end to the service processing end, significantly lowering the overall waiting time from user inquiry to digital human response, thus meeting the core requirements of real-time interaction in live streaming scenarios. Simultaneously, through optimization of the live streaming corpus in the semantic understanding module, accurate mapping of audio prosodic features in the digital human-driven module, and dedicated protocol conversion and streaming control for the target live streaming platform in the platform adaptation layer, it effectively solves problems such as high module connection latency, weak scenario adaptability, and poor platform compatibility in existing systems. Attached Figure Description

[0017] Figure 1 A schematic diagram of the structure of the WebRTC-based digital human live streaming system provided by the present invention; Figure 2 This is a flowchart illustrating the WebRTC-based digital human live streaming method provided by the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] The present invention will now be described in further detail with reference to the accompanying drawings.

[0020] This invention provides a WebRTC-based digital human live streaming system, such as... Figure 1 As shown, the digital human live streaming system comprises an interactive terminal layer, a core service layer, and a platform adaptation layer. The interactive terminal layer is configured to collect user-input interactive data using a WebRTC client and transmit the interactive data to the core service layer based on the WebRTC protocol.

[0021] In practical applications, the interactive terminal layer is typically deployed within the user's browser or mobile application. The WebRTC client captures the user's voice and text data in real time as interactive data by calling the device's microphone, keyboard, and other input interfaces. For example, when a user says "Can the digital human introduce today's live broadcast topic?" on the live broadcast interface, the WebRTC client encodes this voice data and encrypts it using the SRTP protocol to avoid the risk of data leakage during transmission. Subsequently, the WebRTC client completes NAT traversal based on the WebRTC protocol, establishes a secure communication channel with the signaling server of the core service layer, and transmits the encrypted interactive data to the semantic understanding module. The end-to-end latency of the entire data transmission link can be controlled within 150ms, far lower than the latency level of traditional HTTP interface calls.

[0022] The core service layer, serving as the system's processing hub, includes a semantic understanding module, a speech synthesis module, a digital human driving module, and an audio-video synchronization module. These modules work collaboratively in a pipeline manner. Specifically, after receiving encrypted interactive data, the semantic understanding module calls a pre-trained large language model based on the Transformer architecture to perform intent recognition and generate the digital human's response text.

[0023] For example, the model will first perform word segmentation and semantic analysis on the text after speech-to-text transcription, identify the user's core need as "getting an introduction to the live broadcast topic", and then generate an initial reply text: "Today's live broadcast topic is 'Application Practice of WebRTC Technology in Digital Human Live Broadcast'. Next, I will explain it from the aspects of system architecture and real-time interaction principles."

[0024] In this embodiment of the invention, the semantic understanding module is configured with a live streaming scenario corpus, and the semantic understanding module performs scenario-based filtering and optimization on the digital human's reply text based on the live streaming scenario corpus.

[0025] For example, the semantic understanding module will search the built-in live streaming scenario corpus, which contains commonly used live streaming phrases, industry terminology standards, and sensitive word filtering rules, to optimize the initial response—for example, changing "explain in detail" to "break it down in detail for everyone" to better fit the live streaming scenario, and filtering out any potentially colloquial or redundant expressions to ensure that the response is both professional and in line with the live streaming context.

[0026] After receiving the optimized response text, the speech synthesis module uses TTS technology to convert the digital human's response text into an audio stream. Next, the digital human driving module acquires the prosodic features of the audio stream and drives a pre-defined digital human avatar model to generate a corresponding video stream. Specifically, during the conversion process, the digital human driving module extracts the prosodic features of the audio stream, including speech rate features, intonation features, and pause prosodic features. The digital human driving module maps speech rate features to the lip-opening rhythm of the digital human avatar model, intonation features to the dynamic changes in facial expressions of the digital human avatar model, and pause prosodic features to the pause actions of the digital human avatar model, including blinking or nodding. Then, based on the lip-opening rhythm, the dynamic changes in facial expressions, and the pause actions, a corresponding video stream is generated. The video stream is encoded in H.264 format.

[0027] For example, when the word "detailed breakdown" is spoken at a rate of 4 syllables per second in the audio stream, the digital human driving module maps this speech rate feature to a mouth-opening rhythm of 2 times per second in the digital human avatar model, corresponding to the pronunciation action of "detailed-breakdown". If the intonation of the "application practice" section shows an upward trend, the module will trigger dynamic changes in the digital human's facial expressions, such as slightly raising the eyebrows and slightly raising the corners of the mouth, to enhance the digital human's emotional expression. When a 0.5-second pause rhythmic feature appears in the audio stream, the module will drive the digital human to perform blinking or nodding actions, simulating the natural pausing habits of a real anchor. When generating the video stream, the digital human driving module, based on a preset 3D digital human avatar model, injects the motion parameters corresponding to the rhythmic features into the model in real time through skeletal binding and keyframe animation technology, generating a video stream encoded in H.264 format. The frame rate of this video stream is maintained at 30fps to ensure smooth picture quality.

[0028] In this embodiment of the invention, the audio-video synchronization module performs timing calibration on both the audio stream and the video stream after receiving them. Specifically, the audio-video synchronization module extracts the timestamp sequence of the audio stream and the frame timestamp sequence of the video stream, calculates the time difference between the two using a dynamic time warping algorithm, and if it finds that the audio stream is 10ms ahead of the video stream, it will perform frame insertion processing on the video stream or perform microsecond-level buffering delay on the audio stream, ultimately outputting a digital human audio-video stream with an audio-visual synchronization error of less than 5ms, avoiding the problem of "lip movements not matching the sound" during live streaming.

[0029] In this embodiment of the invention, the platform adaptation layer, acting as a bridge connecting the core service layer and the target live streaming platform, performs protocol conversion on the digital human audio and video stream, converts the digital human audio and video stream into a predefined streaming media format, and pushes the converted digital human audio and video stream to the target live streaming platform.

[0030] For example, the protocol adaptation module of the platform adaptation layer pre-stores the streaming protocols and encoding format requirements of mainstream live streaming platforms. For instance, for the RTMP streaming protocol supported by a target live streaming platform, the module will convert the WebRTC native RTP encapsulation format to FLV encapsulation format and adjust the audio encoding from OPUS to AAC to ensure that the audio and video streams comply with the platform's access standards.

[0031] In this embodiment of the invention, the platform adaptation layer includes a protocol adaptation module, an authentication module, and a streaming control unit. The protocol adaptation module is configured to convert the digital human audio and video streams into a preset streaming format based on RTMP, the preset streaming format being compatible with the target live streaming platform. The authentication module is configured to integrate the authentication interface of the target live streaming platform and perform identity verification and streaming permission application for the live streaming account. The streaming control unit is configured to monitor the network bandwidth of the target live streaming platform in real time, perform adaptive bitrate adjustment based on network bandwidth, and initiate a reconnection mechanism when the network is interrupted.

[0032] Specifically, the protocol adaptation module converts the digital human's audio and video streams into RTMP streams compatible with the target live streaming platform; the authentication module automatically calls the OAuth interface of the target live streaming platform to obtain the push stream address; the push stream control unit pushes the RTMP stream to the target live streaming platform and monitors the network status. When the network packet loss rate is greater than 5%, the bitrate is automatically reduced from 2Mbps to 1Mbps to ensure uninterrupted live streaming.

[0033] For example, when the target platform requires authentication based on the OAuth2.0 protocol, the authentication module will send a request containing account information to the platform's authentication server and obtain a token valid for 24 hours before the push process can be started.

[0034] For example, the streaming control unit monitors the network bandwidth of the target live streaming platform in real time. If the current bandwidth is detected to be 800kbps, which is lower than the preset bandwidth threshold of 1Mbps, the streaming bitrate will be automatically reduced from 2Mbps to 1Mbps, while maintaining the video resolution at 720P to avoid screen stuttering due to insufficient bandwidth. If a network interruption occurs, the streaming control unit will immediately activate the reconnection mechanism and attempt to re-establish the streaming connection within 3 seconds. After a successful reconnection, the audio and video streams will continue to be pushed without losing the previous live streaming progress, ensuring the continuity of the live streaming process.

[0035] like Figure 2 As shown, the WebRTC-based digital human live streaming method of this invention is applied to the above-mentioned WebRTC-based digital human live streaming system. The WebRTC-based digital human live streaming method includes the following steps.

[0036] S1. Collect user-input interaction data based on a WebRTC client.

[0037] S2. Use a pre-trained large language model to perform intent recognition on the interaction data to generate digital human response text, and generate an audio stream based on the digital human response text.

[0038] S3 drives the digital human avatar model to generate a video stream based on the audio stream.

[0039] S4. Perform timing calibration on the audio and video streams to obtain a synchronized digital human audio and video stream.

[0040] S5. Perform protocol conversion on the digital human audio and video stream, and push the processed digital human audio and video stream to the target live streaming platform.

[0041] In this embodiment of the invention, a WebRTC client is deployed to receive user voice input and / or text input via the WebRTC interface to obtain interactive data. The interactive data is encrypted using a preset protocol, including the SRTP protocol. Then, the encrypted interactive data is buffered to eliminate transmission delays caused by network jitter.

[0042] Furthermore, a pre-trained large language model is invoked to perform intent recognition and semantic parsing on the interaction data to obtain an initial text response. This large language model is based on the Transformer architecture. Then, based on a pre-configured live-streaming corpus, the initial text response undergoes context filtering and context optimization to obtain the digital human's response text.

[0043] For example, if a user asks, "What is the topic of today's live stream?", the large language model outputs, "Today's live stream topic is 'Digital Human Technology Popularization,' and we will introduce the principles of WebRTC communication next!" The TTS engine then converts the response text into an OPUS-encoded audio stream with a cheerful tone. This audio stream is bound to a pre-trained digital human avatar model, and lip-syncing algorithms are used to match the pronunciation lip movements to generate a video stream.

[0044] Furthermore, by performing timing calibration on the audio and video streams, a synchronized digital human audio and video stream is obtained. Finally, a protocol conversion is performed on the digital human audio and video stream, and the processed digital human audio and video stream is pushed to the target live streaming platform.

[0045] In this embodiment of the invention, the network bandwidth of the target live streaming platform is monitored in real time. If the network bandwidth is lower than a preset bandwidth threshold, the streaming bitrate is reduced. In the event of a streaming interruption, a reconnection mechanism is initiated, and streaming resumes after a successful reconnection.

[0046] For example, when the streaming control unit detects that the real-time network bandwidth of the target live streaming platform drops to 600kbps, it will automatically trigger the bitrate adjustment logic: first, the video encoding bitrate will be reduced from 1.5Mbps to 800kbps, while maintaining the video resolution at 720P and the frame rate at 30fps to avoid excessive degradation of image clarity; if the bandwidth continues to drop below 400kbps, the resolution will be further adjusted to 480P to ensure stable transmission of audio and video streams. When the network connection is suddenly interrupted (e.g., an RTMP streaming link timeout is detected), the streaming control unit will immediately initiate the reconnection mechanism: first, the current streaming task will be paused and the timestamp and bitrate parameters of the last frame will be saved; second, a reconnection request will be sent to the streaming server of the target platform at 500ms intervals, with a maximum of 5 attempts; if the connection is successfully established on the third attempt, the streaming will resume based on the saved timestamp, ensuring the continuity of the digital human live stream content. Users will only experience a brief pause of less than 2 seconds, without any interruption in the live stream content or the need to re-enter the live stream room.

[0047] As described above, this invention provides a WebRTC-based digital human live streaming system and method. The system comprises an interactive terminal layer, a core service layer, and a platform adaptation layer connected sequentially. It utilizes the WebRTC protocol to achieve low-latency acquisition and encrypted transmission of user interaction data. The core service layer generates audio and video streams suitable for live streaming scenarios through pipelined processing of semantic understanding, speech synthesis, digital human driving, and audio-video synchronization. The platform adaptation layer completes protocol conversion and streaming adaptation, ensuring the real-time performance and stability of the digital human live stream. This system and method solve the problems of high interactive latency, audio-visual asynchrony, and difficulty in cross-platform adaptation in traditional digital human live streaming. Through WebRTC's real-time communication capabilities and modular collaborative architecture, it achieves natural interaction between users and digital humans. It can be widely applied to e-commerce live streaming, knowledge popularization, corporate training, and other scenarios, providing an efficient and feasible technical solution for the large-scale deployment of digital human live streaming. Furthermore, the system supports parallel deployment of multiple instances and can dynamically expand computing resources according to the scale of the live stream.

[0048] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A WebRTC-based digital human live streaming system, characterized in that, This includes the interactive terminal layer, the core service layer, and the platform adaptation layer, which are connected sequentially. The interactive terminal layer is configured to: collect user-input interactive data based on a WebRTC client, and transmit the interactive data to the core service layer based on the WebRTC protocol; the interactive data includes voice data and text data. The core service layer includes a semantic understanding module, a speech synthesis module, a digital human driving module, and an audio-video synchronization module; The semantic understanding module is configured to: receive the interaction data and perform intent recognition on the interaction data based on a pre-trained large language model to generate digital human response text; The speech synthesis module is configured to convert the digital human's reply text into an audio stream; The digital human driving module is configured to: acquire the prosodic features of the audio stream and drive a preset digital human image model to generate a video stream corresponding to the audio stream; The audio-video synchronization module is configured to: perform timing calibration on the audio stream and the video stream, and output a synchronized digital human audio-video stream; The platform adaptation layer is configured to convert the digital human audio and video stream into a predefined streaming media format, and push the converted digital human audio and video stream to the target live streaming platform, wherein the streaming media format is compatible with the target live streaming platform.

2. The digital human live streaming system according to claim 1, characterized in that, The interactive terminal layer executes the WebRTC-based transmission of interactive data to the core service layer, including: The interactive data is encrypted. Establish a secure communication channel with the core service layer based on the WebRTC protocol; Based on the secure communication channel, the encrypted interactive data is transmitted to the core service layer.

3. The digital human live streaming system according to claim 1, characterized in that, The semantic understanding module is configured with a live streaming scenario corpus; The semantic understanding module is also configured to perform contextual filtering and optimization of the digital human's response text based on the live streaming scenario corpus.

4. The digital human live streaming system according to claim 1, characterized in that, The prosodic features of the audio stream include speech rate features, intonation features, and pause prosodic features; The digital human driving module executes a preset digital human avatar model to generate a video stream corresponding to the audio stream, including: The speech rate features are mapped to the lip-opening rhythm of the digital human image model; The intonation features are mapped to the dynamic changes of facial expressions in the digital human avatar model; The pause rhythm features are mapped to pause actions of the digital human image model, including blinking or nodding. The audio stream is generated based on the lip-opening rhythm, the dynamic changes in facial expressions, and the pauses. The video stream is encoded in H.264 format.

5. The digital human live streaming system according to claim 1, characterized in that, The platform adaptation layer includes a protocol adaptation module, an authentication and authorization module, and a streaming control unit. The protocol adaptation module is configured to convert the digital human audio and video stream into a preset streaming format based on RTMP, wherein the preset streaming format is compatible with the target live streaming platform; The authentication and authorization module is configured to: integrate the authentication interface of the target live streaming platform, and verify the identity of the live streaming account and apply for push streaming permissions; The streaming control unit is configured to: monitor the network bandwidth of the target live streaming platform in real time, and adjust the bitrate adaptively based on the network bandwidth, and initiate a reconnection mechanism when the network is interrupted.

6. The digital human live streaming system according to claim 5, characterized in that, The streaming control unit is configured to: perform adaptive bitrate adjustment when the network bandwidth is in the range of 300kbps to 2Mbps; and the timeout time for the interrupted stream reconnection mechanism is less than or equal to a preset timeout threshold.

7. A WebRTC-based digital human live streaming method, applied to a WebRTC-based digital human live streaming system as described in any one of claims 1-8, characterized in that, include: Collect user-input interaction data based on a WebRTC client; The interaction data is used to perform intent recognition using a pre-trained large language model to generate digital human response text, and an audio stream is generated based on the digital human response text. The digital human avatar model generates a video stream based on the audio stream; Timing calibration is performed on the audio stream and the video stream to obtain a synchronized digital human audio and video stream. The protocol conversion is performed on the digital human audio and video stream, and the processed digital human audio and video stream is pushed to the target live streaming platform.

8. The digital human live streaming method according to claim 7, characterized in that, The interaction data collected from user input based on the WebRTC client includes: The user's voice input and / or text input are received via the WebRTC interface to obtain the interactive data. The interactive data is encrypted using a preset protocol, which includes the SRTP protocol. The encrypted interactive data is buffered to eliminate transmission delays caused by network jitter.

9. The digital human live streaming method according to claim 7, characterized in that, The step of using a pre-trained large language model to perform intent recognition on the interaction data to generate digital human response text includes: The pre-trained large language model is invoked to perform intent recognition and semantic parsing on the interaction data to obtain an initial text response. The large language model is based on the Transformer architecture. Based on a pre-configured corpus of live streaming scenarios, the initial text response is subjected to context filtering and context optimization to obtain the digital human's response text.

10. The digital human live streaming method according to claim 7, characterized in that, Also includes: Real-time monitoring of network bandwidth during live streaming on the target platform; When the network bandwidth is lower than a preset bandwidth threshold, reduce the push stream bitrate; In the event of a streaming interruption, a reconnection mechanism will be activated, and streaming will resume upon successful reconnection.

Images