Live broadcast interaction system and device based on multi-mode intelligent agent
By constructing a live interactive system with multimodal intelligent agents, multimodal data is collected and processed in real time to identify user intentions and emotions and dynamically generate interactive content. This solves the problems of single information and slow response in existing technologies and achieves a live interactive experience with high concurrency and strong robustness.
Patent Information
- Application Number
- CN202511341715.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-19
- Publication Date
- 2026-01-02
AI Technical Summary
Existing live streaming interaction methods rely on text-based bullet comments or tipping functions, which have a single information dimension, cannot fully capture users' multimodal information, and lack intelligent processing capabilities, resulting in increased system response time and failing to meet the real-time requirements of live streaming scenarios.
A live interactive system based on multimodal intelligent agents is constructed, including modules for multimodal data acquisition and processing, information fusion, intent and emotion recognition, interactive content generation, and distributed processing. A distributed real-time processing architecture is adopted to collect and process multimodal interaction data in real time, dynamically generate multimodal interactive content, and optimize the model through a feedback mechanism.
It achieves high concurrency and robust live interactive performance, can identify user intent and emotions in real time, dynamically generate multimodal interactive content, and improve the intelligence and real-time nature of user experience.
Smart Images

Figure CN121262431A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of live interactive multimodal data processing technology, specifically a live interactive system and device based on multimodal intelligent agents. Background Technology
[0002] With the rapid development of the live streaming industry, users' demands for interactive live streaming experiences are constantly increasing. In recent years, the development of artificial intelligence and multimodal perception technology has enabled live streaming interactions to achieve real-time fusion and understanding of multi-source data such as voice, images, and text through intelligent agents, thereby achieving a more natural and immersive interactive experience. How to build a live streaming interactive system that can support high concurrency, strong robustness, and intelligent response based on multimodal intelligent agents has become an important direction for the development of the live streaming industry.
[0003] Chinese invention application CN120128735A discloses a digital human live-streaming interaction method based on artificial intelligence. The method includes: Step S1: Collecting multimodal data from the live-streaming environment, and performing noise suppression, format conversion, and normalization on the collected data to provide basic data and preprocessing conditions for subsequent data compression, task scheduling, and latency optimization; Step S2: Based on the data collected and processed in Step S1, calculating the probability of occurrence of each data symbol, calculating data entropy, and adjusting the adaptive compression factor in conjunction with the real-time network status to provide data transmission conditions for downstream task scheduling; Step S3: Based on the compressed data obtained in Step S2. This method employs an adaptive data compression and transmission optimization scheme to achieve real-time data compression and transmission, resulting in a significant reduction in network latency. Compared to existing schemes with fixed compression parameters, this invention solves the problem of unstable data latency in high-concurrency scenarios.
[0004] Existing technologies suffer from the following problems: traditional live streaming interaction methods mainly rely on text-based bullet comments or tipping functions, resulting in a relatively singular information dimension and an inability to fully capture the user's true intentions and emotions in multimodal information such as voice and facial expressions; they lack intelligent multimodal information processing capabilities and personalized interactive experiences; and when handling complex multimodal tasks, they increase system response time, failing to meet the millisecond-level real-time requirements of live streaming scenarios; therefore, there is an urgent need for a live streaming interaction system and device based on multimodal intelligent agents to solve the problems existing in the current technologies. Summary of the Invention
[0005] To address the aforementioned technical problems, the present invention aims to provide a live interactive system based on a multimodal intelligent agent, which includes the following modules: Multimodal data acquisition and processing module; real-time acquisition of multimodal interaction data in live interactive scenarios, and preprocessing of multimodal interaction data; Multimodal fusion module: Constructs a multimodal information fusion model, performs multimodal information fusion on preprocessed multimodal interaction data, and obtains multimodal fused information; Intent and sentiment recognition module: Construct a user intent-sentiment integrated analysis model to identify the user's specific intent and sentiment tendency based on the fused multimodal information; Interactive content generation module: Based on the user's specific intentions and emotional tendencies, and according to the user's historical multimodal interaction data, dynamically generate multimodal interactive content; Distributed processing module: Constructs multimodal tasks and adopts a distributed real-time processing architecture to perform distributed deployment and parallel computing of multimodal tasks; Feedback and Optimization Module: Establish a multimodal interaction feedback mechanism to collect user feedback data on generated multimodal interactive content in real time, and continuously optimize the multimodal information fusion model and multimodal interactive content.
[0006] In a preferred embodiment, the multimodal interaction data specifically includes: voice interaction data, text interaction data, behavioral interaction data, and emoji interaction data.
[0007] In a preferred embodiment, the specific process of fusing multimodal information on the preprocessed multimodal interaction data to obtain multimodal fused information includes: The preprocessed multimodal interaction data is processed using a Transformer-based cross-modal attention mechanism to extract multimodal feature vectors. These multimodal feature vectors specifically include: speech-to-text feature vectors, text semantic encoding feature vectors, behavioral feature vectors, and emoji semantic sentiment mapping vectors. A multimodal information fusion model is then constructed. The multimodal feature vectors are input into the encoder layer of the multimodal information fusion model. The encoder layer captures the dependencies between the various multimodal feature vectors through a self-attention mechanism. The multimodal information fusion model aligns the multimodal feature vectors through a feature alignment layer. Furthermore, the multimodal information fusion model achieves interaction and association between the multimodal feature vectors through a cross-attention mechanism layer. Further, the multimodal information fusion model dynamically adjusts the weight ratio of each modality feature in the fusion process through a modality weight adaptive learning layer. Finally, the multimodal information fusion model outputs a unified feature representation after fusion through a decoder layer, thus achieving multimodal information fusion.
[0008] In a preferred embodiment, the specific process of identifying the user's specific intent and emotional inclination includes: A user intent-sentiment integrated analysis model is constructed, which specifically includes, but is not limited to, an intent analysis layer and a sentiment analysis layer. The fused multimodal information is input into the user intent-sentiment integrated analysis model. The intent analysis layer extracts semantic keywords from the fused multimodal information and combines them with a pre-defined live streaming scenario intent knowledge base to identify the user's specific intent. The sentiment analysis layer extracts sentiment features from the fused multimodal information, focusing on capturing sentiment words, intonation change mapping features, and emoji semantic sentiment mapping vectors. Sentiment analysis is performed using a sentiment dictionary to obtain the sentiment analysis results. These results are then mapped to positive, negative, and neutral sentiment dimensions, and the probability distribution of each sentiment dimension is output to obtain the user's sentiment tendency.
[0009] In a preferred embodiment, the specific process of dynamically generating multimodal interactive content includes: The system acquires historical multimodal interaction data and user profiles, constructs personalized preference vectors based on these profiles, and builds user preference models based on these vectors. It inputs specific user intentions and emotional tendencies into these models and introduces dynamic emotion-driven control factors. Combined with a live streaming context memory management module, a content generation engine dynamically generates multimodal interactive content. This multimodal interactive content specifically includes text, voice, emoticons, and interactive actions, which are output in real-time to the live streaming interactive scenario in a multimodal format.
[0010] In a preferred embodiment, the live streaming context memory management module specifically includes: a short-term memory unit, a long-term memory unit, a memory retrieval unit, a memory update unit, and a memory compression unit.
[0011] In a preferred embodiment, the multimodal task specifically includes: a multimodal information fusion task, a task of identifying user specific intentions and user emotional tendencies, and a multimodal interactive content generation task; The modules mentioned above are connected via wired and / or wireless means to enable data transmission between them.
[0012] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention collects multimodal interaction data in real-time during live interactive scenarios and preprocesses the multimodal interaction data; further, it constructs a multimodal information fusion model to fuse the preprocessed multimodal interaction data to obtain multimodal fused information; further, it constructs a user intent-emotion comprehensive analysis model to identify the user's specific intent and emotional tendency based on the fused multimodal information; further, it dynamically generates multimodal interactive content based on the user's specific intent and emotional tendency, and according to the user's historical multimodal interaction data; further, it constructs multimodal tasks and adopts a distributed real-time processing architecture to perform distributed deployment and parallel computing of multimodal tasks; and further, it establishes a multimodal interaction feedback mechanism to collect user feedback data on the generated multimodal interactive content in real time and continuously optimize the multimodal information fusion model and multimodal interactive content. Attached Figure Description
[0013] Figure 1 This is a schematic diagram showing the connections between various modules of a live interactive system based on a multimodal intelligent agent, according to an embodiment of this application. Figure 2 This is a schematic diagram of an electronic device according to an embodiment of this application; Figure 3 This is a schematic diagram of the storage medium according to an embodiment of this application. Detailed Implementation
[0014] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, features described in some examples may be combined in other examples.
[0015] It should be noted that, unless otherwise defined, the technical or scientific terms used in one or more embodiments of the present invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in one or more embodiments of the present invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.
[0016] Example 1 Please see Figure 1 As shown in the figure, this application provides a live interactive system based on a multimodal intelligent agent. The system includes the following modules: a multimodal data acquisition and processing module, a multimodal fusion module, an intent and emotion recognition module, an interactive content generation module, a distributed processing module, and a feedback and optimization module. The modules described above are connected via wired and / or wireless means to enable data transmission between them. Multimodal data acquisition and processing module; real-time acquisition of multimodal interaction data in live interactive scenarios, and preprocessing of multimodal interaction data; Based on the above embodiments, the multimodal interaction data specifically includes: voice interaction data, text interaction data, behavioral interaction data, and emoji interaction data; It should be noted that the voice interaction data specifically includes, but is not limited to: audio information input by the host and the audience through the microphone of the live streaming device (such as a mobile phone, computer, tablet, etc.) during the live broadcast, and the audio information specifically includes, but is not limited to: voice questions, voice comments, etc. Text interaction data includes, but is not limited to: text information entered by viewers in the live stream's bullet screen and comment sections, as well as text information such as announcements and interactive topics posted by the streamer; Behavioral interaction data includes, but is not limited to: viewers' actions on the live stream interface, such as liking, following, sharing, gifting, and clicking on products; it also includes the streamer's live stream behavior data, such as camera switching and body movement recognition results. The emoji interaction data includes, but is not limited to: various static or dynamic emoji resources sent by viewers or broadcasters during the interaction process, as well as their corresponding sending time, recipients, and other related information; The preprocessing of multimodal interaction data specifically includes: denoising, segmenting, and extracting features from voice interaction data to convert it into recognizable voice-text feature vectors; cleaning text interaction data to remove invalid characters and duplicate content, and performing word segmentation, part-of-speech tagging, and semantic encoding; standardizing behavioral interaction data to unify data format and timestamps, and extracting key behavioral features; and unifying the format and extracting features from emoji interaction data to establish a mapping relationship between emojis and semantic sentiment, facilitating subsequent multimodal data fusion and understanding. Multimodal fusion module: Constructs a multimodal information fusion model, performs multimodal information fusion on preprocessed multimodal interaction data, and obtains multimodal fused information; Based on the above embodiments, the specific process of fusing multimodal information on the preprocessed multimodal interaction data to obtain multimodal fused information includes: Multimodal feature vectors are extracted from the preprocessed multimodal interaction data using a Transformer-based cross-modal attention mechanism. These multimodal feature vectors specifically include: speech-to-text feature vectors, text semantic encoding feature vectors, behavioral feature vectors, and emoji semantic sentiment mapping vectors. A multimodal information fusion model is constructed, and the multimodal feature vectors are input into the encoder layer of the model. The encoder layer captures the dependencies between the various multimodal feature vectors through a self-attention mechanism. The multimodal information fusion model aligns the multimodal feature vectors through a feature alignment layer. Furthermore, the multimodal information fusion model achieves interaction and association between the multimodal feature vectors through a cross-attention mechanism layer. Further, the multimodal information fusion model dynamically adjusts the weight ratio of each modality feature in the fusion process through a modality weight adaptive learning layer. Finally, the multimodal information fusion model outputs a unified feature representation after fusion through a decoder layer, thus achieving multimodal information fusion. It should be noted that multimodal information fusion models specifically include: multimodal graphical neural networks (GNN), multimodal Transformer, Co-Attention, and LLM fusion models; Intent and sentiment recognition module: Construct a user intent-sentiment integrated analysis model to identify the user's specific intent and sentiment tendency based on the fused multimodal information; Based on the above embodiments, the specific process of identifying user intent and user sentiment includes: constructing a user intent-sentiment comprehensive analysis model, which specifically includes, but is not limited to, an intent analysis layer and a sentiment analysis layer; inputting the fused multimodal fusion information into the user intent-sentiment comprehensive analysis model, extracting semantic keywords from the fused multimodal fusion information through the intent analysis layer and combining them with a preset live streaming scenario intent knowledge base to identify the user's specific intent; the sentiment analysis layer extracts sentiment features from the fused multimodal fusion information, focusing on capturing sentiment words, intonation change mapping features, and emoticon semantic sentiment mapping vectors in the multimodal fusion information, and performing sentiment analysis with the help of a sentiment dictionary to obtain sentiment analysis results, mapping the sentiment analysis results to positive, negative, neutral, and other sentiment dimensions, and outputting the probability distribution of each sentiment dimension to obtain the user's sentiment tendency; It should be noted that identifying a user's specific intent includes, but is not limited to, inquiries, interactions, purchases, and complaints. The pre-defined live streaming scenario intent knowledge base is built by researchers in this technical field through continuous experimentation based on historical live streaming scenario analysis of user intent. For example, in e-commerce live streaming scenarios, the live streaming scenario intent knowledge base specifically includes: product interaction intent, order operation intent, after-sales consultation intent, and benefit participation intent, etc. Interactive content generation module: Based on the user's specific intentions and emotional tendencies, and according to the user's historical multimodal interaction data, dynamically generate multimodal interactive content; Based on the above embodiments, the specific process of dynamically generating multimodal interactive content includes: acquiring users' historical multimodal interaction data and user profiles; constructing personalized preference vectors based on user profiles; constructing user preference models based on users' historical multimodal interaction data and personalized preference vectors; inputting users' specific intentions and emotional tendencies into the user preference models; introducing dynamic emotion-driven control factors; and, in conjunction with the live streaming context memory management module, dynamically generating multimodal interactive content through a content generation engine. The multimodal interactive content specifically includes: text, voice, emoticons, and interactive actions output in real-time to the live streaming interactive scenario in a multimodal format. The live streaming context memory management module specifically includes: short-term memory unit, long-term memory unit, memory retrieval unit, memory update unit, and memory compression unit; It should be noted that the short-term memory unit is used to maintain the context information of the current live interaction session, supporting fast access and updates; the long-term memory unit is used to store historical live interaction records, supporting persistent storage and retrieval; the memory retrieval unit uses semantic retrieval and similarity matching algorithms to quickly retrieve relevant memories, supporting multimodal memory retrieval; the memory update unit dynamically updates memory content based on new interaction information, supporting incremental learning and knowledge updates; and the memory compression unit intelligently compresses and optimizes memory information, reducing storage overhead while maintaining information integrity. It should be noted that the role of the emotion-driven control factor is to ensure that the generated content not only conforms to the user's language... It meets the needs of users while maintaining a style and expression that matches their current emotional state; the content generation engine specifically includes: text generation module, audio generation module, etc. Distributed processing module: Constructs multimodal tasks and adopts a distributed real-time processing architecture to perform distributed deployment and parallel computing of multimodal tasks; Based on the above embodiments, the multimodal tasks specifically include: multimodal information fusion task, user specific intent and user emotional tendency identification task, and multimodal interactive content generation task; It should be noted that the purpose of adopting a distributed real-time processing architecture to perform distributed deployment and parallel computing of multimodal tasks is: In terms of performance, by breaking down tasks such as multimodal information fusion, identifying user specific intentions and user emotional tendencies, and generating multimodal interactive content into different computing nodes for parallel processing, the overall latency can be significantly reduced, ensuring real-time response capabilities in live interactive scenarios. In terms of scalability, the distributed architecture can dynamically schedule computing resources according to the multimodal task load, achieve horizontal scaling, adapt to the need to process massive user multimodal interaction data at the same time, and avoid single-point performance bottlenecks. In terms of fault tolerance and stability, distributed deployment supports task-level redundancy and failover mechanisms. When a node fails, other nodes can quickly take over the computation to ensure continuous operation. In terms of task decoupling, different multimodal subtasks run in independent services or containers, which can optimize the model structure, computing framework and storage method according to the characteristics of the task, thereby improving the flexibility and maintainability of the overall architecture. In terms of resource utilization, distributed parallel computing can achieve efficient scheduling of heterogeneous hardware such as CPU, GPU, and TPU, making the computing more suited to the characteristics of the task. For example, GPUs accelerate deep neural network inference, while CPUs handle preprocessing and rule calculation, thereby achieving the best performance-cost ratio. Therefore, adopting a distributed real-time processing architecture can comprehensively improve the real-time performance, scalability, stability, intelligence level, and real-time response capability of multimodal live interactive systems. Feedback and Optimization Module: Establish a multimodal interaction feedback mechanism to collect user feedback data on generated multimodal interactive content in real time, and continuously optimize the multimodal information fusion model and multimodal interactive content; Based on the above embodiments, the feedback data specifically includes: user viewing time, text interaction content, voice interaction content, emoticon usage content, user satisfaction, etc. It should be noted that a multimodal interaction feedback mechanism is established to collect user feedback data on the generated multimodal interactive content in real time. Based on the feedback data, the parameters of the multimodal information fusion model are fine-tuned through online learning or incremental training, so that the model can extract and fuse multimodal features more accurately in subsequent processing. Furthermore, based on a reinforcement learning framework, user satisfaction is used as a reward signal to dynamically adjust the multimodal interactive content, so that the output content continuously improves in terms of semantic relevance, emotional consistency, and personalized preference satisfaction.
[0017] Example 2 Please see Figure 2 As shown, according to another aspect of this application, an electronic device 500 is also provided. The electronic device 500 may include one or more processors and one or more memories. The memory stores computer-readable code, which, when executed by the one or more processors, can perform a live interactive system based on a multimodal intelligent agent as described above. A live interactive system based on a multimodal intelligent agent, according to an embodiment of this application, can also be used with the aid of... Figure 2 The architecture of the electronic device shown is used to implement this. For example... Figure 2 As shown, the electronic device 500 may include a bus 501, one or more CPUs 502, a read-only memory (ROM) 503, a random access memory (RAM) 504, a communication port 505 connected to a network, an input / output 506, a hard disk 507, etc. The storage device in the electronic device 500, such as the ROM 503 or the hard disk 507, may store a live interactive system based on a multimodal intelligent agent provided in this application. Furthermore, the electronic device 500 may also include a user interface 508. Of course, Figure 2 The architecture shown is merely exemplary; when implementing different devices, it can be omitted or added according to actual needs. Figure 2 One or more components in the illustrated electronic device; Example 3 Please see Figure 3The diagram illustrates a computer-readable storage medium 600 according to one embodiment of this application. The computer-readable storage medium 600 stores computer-readable instructions. When executed by a processor, the computer-readable instructions can perform a live interactive system based on a multimodal intelligent agent, as described above with reference to the accompanying drawings, according to an embodiment of this application. The storage medium 600 includes, but is not limited to, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.
[0018] Furthermore, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, this application provides a non-transitory machine-readable storage medium storing machine-readable instructions that can be executed by a processor to perform instructions corresponding to the method steps provided in this application. When this computer program is executed by a central processing unit (CPU), it performs the functions defined in the method of this application.
[0019] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
[0020] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A live interactive system based on multimodal intelligent agents, characterized in that, The system includes the following modules: Multimodal data acquisition and processing module; Real-time acquisition of multimodal interaction data in live streaming scenarios, and preprocessing of the multimodal interaction data; Multimodal fusion module; A multimodal information fusion model is constructed to fuse preprocessed multimodal interaction data to obtain multimodal fused information. Intent and sentiment recognition module: Construct a user intent-sentiment integrated analysis model to identify the user's specific intent and sentiment tendency based on the fused multimodal information; Interactive content generation module; Based on the user's specific intentions and emotional tendencies, and according to the user's historical multimodal interaction data, multimodal interactive content is dynamically generated. Distributed processing module: Constructs multimodal tasks and adopts a distributed real-time processing architecture to perform distributed deployment and parallel computing of multimodal tasks; Feedback and Optimization Module: Establish a multimodal interaction feedback mechanism to collect user feedback data on generated multimodal interactive content in real time, and continuously optimize the multimodal information fusion model and multimodal interactive content.
2. The live interactive system based on a multimodal intelligent agent according to claim 1, characterized in that, Multimodal interaction data specifically includes: voice interaction data, text interaction data, behavioral interaction data, and emoji interaction data.
3. The live interactive system based on a multimodal intelligent agent according to claim 1, characterized in that, The specific process of fusing multimodal information from preprocessed multimodal interaction data to obtain multimodal fused information includes: The preprocessed multimodal interaction data is processed using a Transformer-based cross-modal attention mechanism to extract multimodal feature vectors. These multimodal feature vectors specifically include: speech-to-text feature vectors, text semantic encoding feature vectors, behavioral feature vectors, and emoji semantic sentiment mapping vectors. A multimodal information fusion model is then constructed. The multimodal feature vectors are input into the encoder layer of the multimodal information fusion model. The encoder layer captures the dependencies between the various multimodal feature vectors through a self-attention mechanism. The multimodal information fusion model aligns the multimodal feature vectors through a feature alignment layer. Furthermore, the multimodal information fusion model achieves interaction and association between the multimodal feature vectors through a cross-attention mechanism layer. Further, the multimodal information fusion model dynamically adjusts the weight ratio of each modality feature in the fusion process through a modality weight adaptive learning layer. Finally, the multimodal information fusion model outputs a unified feature representation after fusion through a decoder layer, thus achieving multimodal information fusion.
4. A live interactive system based on a multimodal intelligent agent according to claim 1, characterized in that, The specific process of identifying a user's specific intent and emotional inclination includes: A user intent-sentiment integrated analysis model is constructed, which specifically includes, but is not limited to, an intent analysis layer and a sentiment analysis layer. The fused multimodal information is input into the user intent-sentiment integrated analysis model. The intent analysis layer extracts semantic keywords from the fused multimodal information and combines them with a pre-defined live streaming scenario intent knowledge base to identify the user's specific intent. The sentiment analysis layer extracts sentiment features from the fused multimodal information, focusing on capturing sentiment words, intonation change mapping features, and emoji semantic sentiment mapping vectors. Sentiment analysis is performed using a sentiment dictionary to obtain the sentiment analysis results. These results are then mapped to positive, negative, and neutral sentiment dimensions, and the probability distribution of each sentiment dimension is output to obtain the user's sentiment tendency.
5. A live interactive system based on a multimodal intelligent agent according to claim 1, characterized in that, The specific process of dynamically generating multimodal interactive content includes: The system acquires historical multimodal interaction data and user profiles, constructs personalized preference vectors based on these profiles, and builds user preference models based on these vectors. It inputs specific user intentions and emotional tendencies into these models and introduces dynamic emotion-driven control factors. Combined with a live streaming context memory management module, a content generation engine dynamically generates multimodal interactive content. This multimodal interactive content specifically includes text, voice, emoticons, and interactive actions, which are output in real-time to the live streaming interactive scenario in a multimodal format.
6. A live interactive system based on a multimodal intelligent agent according to claim 5, characterized in that, The live streaming context memory management module specifically includes: short-term memory unit, long-term memory unit, memory retrieval unit, memory update unit, and memory compression unit.
7. A live interactive system based on a multimodal intelligent agent according to claim 1, characterized in that, Multimodal tasks specifically include: multimodal information fusion tasks, tasks for identifying specific user intentions and user emotional tendencies, and multimodal interactive content generation tasks.
8. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that... When the processor executes the computer program, it implements a live interactive system based on a multimodal intelligent agent as described in any one of claims 1-7.
9. A computer-readable storage medium, characterized in that... The computer-readable storage medium stores a computer program, which, when executed, implements the live interactive system based on a multimodal intelligent agent as described in any one of claims 1-7.
Citation Information
Patent Citations
Digital human live broadcast interaction method based on artificial intelligence
CN120128735A
Cited By
Emotion analysis and strategy optimization system based on multi-modal acquisition and large language model
CN121479714A
