2594 results about "Digital human" patented technology

The invention provides an intelligent real-time interactive question-answering system based on a virtual digital human, and belongs to the technical field of voice signal processing and voice recognition, and the system comprises a data acquisition module which receives a voice or text interaction request input by a user, collects the expression dynamic parameter sequence and limb movement sequence data of the user in real time, and transmits the data to a user interaction module; obtaining a standardized voice feature vector and structured text data; the cross-modal fusion module is used for constructing an interactive feature matrix; the behavior decision module outputs a decision instruction set; the knowledge retrieval module is used for generating an answer text with emotional adaptability and voice features; and the voice generation module is used for generating a mouth shape animation key frame, a micro expression parameter sequence and a limb action track of the virtual digital human, generating a voice response in combination with the answer text and the voice characteristics, and pushing the voice response to the user terminal. According to the method, the interaction experience and adaptability of the virtual digital human are remarkably improved.

The invention discloses an intelligent multi-modal virtual digital human interaction system based on an AI language large model. The system comprises a high-authenticity face generation module; the high-authenticity face generation module uses an AdaAN network, based on adaptive feature fusion and voice driving and time sequence modeling of voice features, feature information related to voice is extracted, the extracted voice features are processed through a deep neural network, it is ensured that the voice and facial expressions are highly aligned in time and space, and the face recognition accuracy is improved. Collecting a bio-electricity signal, mapping the signal to facial muscle movement, generating a final facial expression, and interacting with a user; the system further comprises an intelligent interaction module, a training optimization and efficient generation module, an efficient integration module, a multi-modal data acquisition module, an AI large model core processing module, a digital human image generation and driving module, an interaction scene adaptation module and a feedback optimization module. The invention further discloses a multi-mode digital human interaction method which has wide application value.

The invention belongs to the technical field of virtual person generation, and particularly relates to a digital person video generation method based on a multi-modal large model, and the method comprises the following steps: 1, constructing a multi-modal data system; 2, multi-modal large model training and adaptation are carried out; 3, constructing a digital human three-dimensional model; step 4, performing semantic analysis and modal mapping; 5, generating a time sequence action and a mouth shape; step 6, building and rendering a virtual scene; step 7, audio and video synchronous rendering and synthesis; step 8, quality optimization and defect repair; and step 9, performing user interaction and iterative optimization. Through technical innovation and engineering, the core pain point in digital human video generation is solved, efficient, vivid and customizable content production capacity is provided for virtual anchors, intelligent customer service, enterprise training and other scenes, and the AI digital human technology is promoted to be applied to large-scale business from experiments.

The invention provides a lightweight digital human lesson preparation system based on an intelligent agent, and belongs to the field of intelligent teaching. Through collaborative operation of four core modules of knowledge graph construction and reasoning, multi-modal cognitive agent, lightweight digital human generation and intelligent teaching plan assistance, the problems of low efficiency of resource integration, teaching content homogenization, insufficient digital human interaction experience and the like in traditional lesson preparation are solved. The knowledge graph construction and reasoning module is used for constructing a structured knowledge graph and realizing knowledge point association mining and teaching logic reasoning; the multi-modal cognitive agent module is used for generating personalized explanation content according with a teaching target by fusing multi-modal courseware analysis, semantic understanding and lecture style dynamic adaptation functions; the lightweight digital human generation module is combined with model pruning and emotion modeling technologies to synchronously output natural voice and a high-simulation digital human image; the intelligent teaching plan auxiliary module helps the teacher to intelligently generate a teaching plan and a teaching outline according to the courseware content based on the knowledge graph.

The invention discloses a digital human emotion recognition and feedback system based on a large model, and relates to the technical field of large models.During operation of the system, multi-modal data of a driver are collected in real time through an embedded sensor network in various driving environments, and the emotion, driving behaviors and driving environment parameters of the driver are obtained; the method comprises the steps of constructing fatigue, anger and other multi-dimensional emotion features of a driver to form an emotion feature vector, analyzing an emotion change trend through threshold comparison, obtaining an evaluation state, dynamically generating a feedback strategy for optimizing a driving state based on an emotion state and a driving scene of the driver, and embedding a generative large model into a module. Diversified feedback responses including audio reminding, driving mode adjustment and entertainment content recommendation are automatically generated, feedback strength and form are adaptively adjusted through multi-dimensional parameters, a feedback generation effect is dynamically adjusted, and an emotion state analysis result and vehicle behavior monitoring data are integrated, so that a user experience is improved. And evaluating the potential dangerous state of the driver in real time.

The invention discloses a digital human real-time interaction system and method. The interaction system comprises terminal equipment and a cloud server. The terminal device receives voice input by a user and transmits the voice to the cloud server. The cloud server comprises a video acquisition and processing module, an intelligent question and answer and voice generation module, an audio processing module and an image rendering module. A video capture and processing module captures original video data indicative of an object of a digital figure and generates successive image frames. The intelligent question and answer and voice generation module converts the voice into a text; performing semantic analysis on the text, understanding the intention of the user, and generating a corresponding reply text; the reply text is also converted into a reply voice. The audio processing module processes the reply voice and the continuous image frames to generate first video data. And the image rendering module fuses the first video data with the virtual background, and performs real-time processing on the fused video to generate a response video. And the terminal equipment outputs the response video.

The invention provides a digital human interaction control method and device fusing emotion semantics and logical reasoning and a storage medium. The method comprises the following steps: analyzing multi-modal input data of a user, constructing emotion-semantics joint representation, and generating a logic decision path; and through a cognitive fusion module, emotion-semantic representation and a logic decision path are fused, and an interaction response adapting to emotion and logic consistency is generated. The system optimizes an emotion semantic model and a logical reasoning rule on line according to user feedback and interaction history, and real-time interaction of emotion dynamic and logical rules is achieved. The system can dynamically adjust the logic decision path based on the multi-mode emotional state of the user, and improves the naturalness and situation adaptability of interaction. A dynamic time warping algorithm and a factorization machine are introduced to process a multi-modal feature fusion problem, and the accuracy and robustness of emotional state recognition are improved. The online optimization mechanism enables the model and the rule to be evolved continuously, and reasoning errors are corrected automatically through user feedback, so that error circulation is avoided.

The invention discloses a real-time interactive digital human system supporting high concurrency and an implementation method thereof, and relates to the technical field of digital human interaction.The system comprises a model instance pool module used for loading the weight of a deep learning model to a shared memory area through the memory mapping technology; the multi-thread scheduling module is used for scheduling user requests by adopting a lock-free queue and a dynamic priority algorithm; the asynchronous pipeline processing module is composed of decoupled micro-services, and the modules are connected in series through asynchronous message queues; the client SDK is used for dynamically switching a rendering mode according to terminal hardware performance and network conditions; the elastic capacity expansion and contraction module is used for monitoring resource loads in real time and automatically adjusting the number of service instances; and the audio and video synchronization calibration module is used for ensuring that the synchronization error of the audio and video frames is lower than a preset threshold value through a timestamp alignment algorithm. According to the scheme, core challenges in a high-concurrency digital human interaction scene can be systematically solved, and innovative support is provided for large-scale real-time application.

The invention relates to the technical field of data processing, and discloses a training method and system for generating a 5D digital human based on AIGC, and a medium. The method comprises the following steps: preprocessing real person feature data to obtain multi-dimensional preprocessed data; performing multi-level processing on the multi-dimensional preprocessing data to obtain 5D digital human generation data; performing cross-modal coordination processing on the 5D digital human generation data through a progressive multi-task learning algorithm to obtain to-be-processed digital human data; performing dynamic knowledge integration processing on the to-be-processed digital human data to obtain adaptive digital human data; carrying out personalized fine tuning and style migration processing on the adaptive digital human data to obtain target digital human data; and performing real-time rendering and intelligent interaction processing on the target digital human data to obtain target 5D digital human data, and generating a digital human training scheme according to the target 5D digital human data. According to the method and the device, the training efficiency and accuracy of generating the 5D digital human based on the AIGC are improved.

Embodiments of the present invention provide a server, a display device, and a digital human processing method. The method comprises: receiving image data; inputting the image data into a shared encoder to obtain a first depth encoding feature; inputting the first depth encoding feature into a segmentation decoder to obtain a segmentation mask; inputting the segmentation mask and the depth encoding feature into a guided flow architecture module to obtain a refined feature; inputting the depth encoding feature and the refined feature into an image matting decoder to obtain a face image having undergone image matting; performing image quality enhancement processing on said face image to obtain a face enhanced image; determining face keypoint data; and generating standard data. According to the embodiments of the present invention, a segmentation mask is obtained by means of a segmentation decoder, and the segmentation mask provides, for an image matting decoder, semantic information obtained by means of segmentation, such that the accuracy of an image matting result is higher, the portrait edge segmentation is cleaner, the quality and precision of standard data are improved, and human assistance information is not needed.

The invention relates to a psychological counseling robot based on multi-modal data and a large vertical model. The robot is used for providing psychological counseling services for scenes such as families, schools and enterprises, and the front end depends on an intelligent robot terminal, adopts a main mode of virtual digital man-machine interaction and is assisted by multi-modal emotion recognition to enhance the psychological counseling effect; the rear end takes an intelligent agent as a control core and takes a psychological big model as a computing platform. The strong semantic understanding and dialogue interaction capability of the large model is utilized, and the multi-modal mode recognition of expressions and voices is combined, so that the psychological assessment and psychological healing functions are realized, and meanwhile, the knowledge supplementing and checking capability of an external knowledge base is realized; according to the method, a strategy of combining structured prompt with multi-modal emotion data enhancement is used, performance advantages of a large model are fully utilized by means of a retrieval enhancement generation and knowledge graph technology, meanwhile, occurrence of illusion problems is reduced, a digital human interaction mode is adopted, interpersonal barriers are reduced, and consultation quality is improved.

The embodiment of the invention provides a government affair digital human dynamic interaction method and system based on a multi-mode large model. The method is applied to the technical field of government affair intelligent services, and comprises the following steps: acquiring a policy announcement text, and performing cleaning and structuring processing to obtain a structured policy data set; and extracting old and new policy data, performing difference comparison, marking key change fields, and generating policy change data. Abstracting and element extraction are carried out on the change data to form structured semantic fragments, and the structured semantic fragments are incrementally embedded into the policy knowledge graph. And generating a question and answer pair sample based on the updated knowledge graph, and carrying out self-supervised fine tuning training on the multi-modal large model. A user inputs multi-modal data, and the model generates a government affair response and feeds back the government affair response. According to the scheme, the multi-modal large model can continuously keep the latest policy knowledge; the model is enabled to generate accurate government affair response with consistent context while understanding multi-modal input such as text, voice and image, and timeliness, accuracy and interactive experience of policy interpretation are improved.

A high-quality digital human avatar is quickly reconstructed from a monocular video and is rendered in real time, the method is widely applied to various fields, and an existing method based on a neural radiation field is high in training and rendering cost. According to the dynamic human body three-dimensional modeling method based on the three-dimensional Gaussian sputtering technology, three-dimensional Gaussian can be expanded to a dynamic human scene, and rapid training and real-time rendering can be achieved. Specifically, the three-dimensional Gaussian of the human body is initialized in a standard space, and the three-dimensional Gaussian is converted from the standard space to a posture space by utilizing linear hybrid skin (LBS) according to posture parameters, so that fine details of the human body are learned at very low calculation cost; a posture parameter refining module and an LBS weight optimization module are designed to improve the problem of inaccurate detection of posture parameters in a real data set, and multi-resolution hash coding is introduced to accelerate the model training speed.

The invention discloses an emotion-enhanced digital human driving and presenting system and method, and belongs to the technical field of computers.The system comprises an audio decoupling module, a potential diffusion model module, a human motion prior module, a training module and a rendered image generation module; predicting and reconstructing the facial expression and gesture action sequence to obtain a final action sequence; and generating a neural rendering image based on the final action sequence. According to the method, through voice input, emotion and style features in voice are captured, a cross attention feature fusion mechanism is introduced to enhance expression of emotion and personal style features, a voice-driven gesture generation task is expanded to a facial expression and gesture action synchronous generation task, the integrity of generated digital human actions is realized, and the user experience is improved. The emotional expressive force of facial expressions and gestures of the digital human is greatly improved.

Provided in the embodiments of the present application are a video generation method and interaction method based on a digital human, and a device, a storage medium and a program product. In the embodiments of the present application, text-to-speech processing is performed on the basis of voice features of a user and an emotion label, speech-to-expression processing is performed on the basis of a mapping relationship between the voice features of the user and expression coefficients, and a digital human model is rendered on the basis of speech signals and the expression coefficients, so as to obtain video data of the digital human model. Thus, voice features of a user are accurately simulated, so as to ensure that a speech output of a digital human sounds natural and is also highly personalized, thereby realizing personalized driving of the digital human, and improving the realism of the digital human in terms of voice and dynamic images. Thus, the user experience is improved, and the interactivity of the digital human and the authenticity and immersion are enhanced.

The invention relates to the field of knowledge graph construction, in particular to a policy knowledge graph construction method and system based on digital human interaction data analysis. The policy knowledge graph construction system based on digital human interaction data analysis comprises a knowledge graph preliminary construction module, a first graph updating module, a second graph updating module and a digital human interaction module. According to the method, the policy source webpage content change is monitored in real time, and the question information in the digital human interaction data is deeply mined, so that the full-life-cycle dynamic maintenance of the policy knowledge graph is realized; on one hand, a webpage policy updating event is accurately captured, and knowledge injection is automatically completed in combination with a semantic unit matching algorithm; and on the other hand, the conflict characteristics questioned by the user in the interaction data are extracted through dialogue semantic analysis, atlas correction is triggered after confidence assessment and multi-stage verification, it is ensured that the end-to-end timeliness from policy release to user perception is controlled within a reasonable period, and the policy response accuracy of the digital human service is remarkably improved.

The invention discloses a method and a system for generating a digital human video, and relates to the technical field of digital humans, and the method and the system for generating the digital human video can be used for obtaining user behavior data, preprocessing the user behavior data, generating a user feature vector, and improving the accuracy of personalized recognition of a user. And inputting the user feature vector into the adversarial network, generating an initial digital human image, analyzing real-time interaction data of the user, obtaining a user interaction matching degree index, realizing dynamic matching of the digital human image and the user emotion, and enhancing the interaction experience. A feedback mechanism is established through a fuzzy logic control algorithm, expression details, intonations and body languages of a digital person are adjusted, a personalized digital person image is generated, personalized dialogue content is generated through a natural language generation model, and a digital person video is generated in combination with distributed rendering, so that high-quality and personalized digital person video output is realized, and the user experience is improved. Therefore, the satisfaction degree and participation sense of the user are improved.

The invention relates to the field of multi-modal interaction analysis, in particular to a multi-modal interaction method and system of a digital human agent. The method comprises the following steps: acquiring a real-time face image and a voice signal input stream of an interactive user based on an intelligent agent; performing real-time micro-expression recognition and deep emotion analysis based on the real-time facial image to obtain real-time emotion features of the user; performing time sequence evolution analysis on the real-time emotion characteristics of the user, performing holographic user emotion deep mining, and constructing a user emotion holographic characteristic spectrum; carrying out adaptive acoustic gain processing on the voice signal input stream, and carrying out voice-emotion association analysis based on the user emotion holographic characteristic spectrum to generate a voice-emotion linkage mapping spectrum; and carrying out eyeball fixation point migration tracking based on the user emotion holographic feature map and the real-time face image, and generating a user interaction depth intention signal. Through the real-time deep semantic understanding and emotion perception ability, the intelligent agent interaction intelligence and response accuracy are improved.

The invention provides an AI-based virtual digital human interaction system, and relates to the technical field of data processing, and the system is used for obtaining a user feature data set by performing feature extraction on an interaction data set, calculating an audio change rate and speech speed stability, obtaining speech emotion data, calculating syntactic complexity and counting the number of emotion words. Calculating a facial expression symmetry degree; obtaining facial emotion data; calculating body stability and identifying a visual staying area; obtaining attention state data; adjusting tone and speed of the virtual digital person according to the virtual digital person text, optimizing sentence pattern complexity and sentiment words in the virtual digital person text, adjusting changes of eyes, eyebrows and mouth corners of the virtual digital person, and adjusting information display sequence and visual focus guidance. According to the method, the interaction efficiency of the virtual digital human is improved by analyzing the facial expressions and the body actions of the user.

The invention discloses an emotional understanding and feedback method for a full-duplex interactive question-answering digital person, and the method comprises the steps: S1, arousing interaction: enabling a digital person of a digital person system to enter an interaction state in a human body detection arousing or voice instruction arousing manner by a user; s2, state analysis: the digital human system judges the state of the user in real time by collecting image data and audio data of the user, and identifies and analyzes the direction, emotion and speech content of the user in real time; and S3, feedback output: the digital human system drives the digital human in voice, expression and body movement through a screen in real time according to the analysis result of the direction, emotion and speech content of the user, and feeds back the result to the user in a manner of adding auxiliary characters, images and videos. The method has the advantages that the emotional state of the user is accurately judged through an advanced algorithm, digital human expressions, body actions and voice feedback are driven in real time according to the emotion and speech content of the user, response is made in time, and a real-person-like real-time interaction scene is achieved.

The invention discloses an AI digital human interaction system based on emotion recognition, relates to the technical field of emotion calculation, and improves the accuracy of emotion recognition through multi-modal emotion data acquisition and processing of an emotion primary recognition module and a secondary recognition module. The system effectively distinguishes complex emotions, such as fatigue and depression, of the trainee in the virtual education task by calculating the first emotion coefficient and the second emotion coefficient; secondary emotion recognition is carried out, so that misjudgment is reduced, and the emotion state recognition accuracy is ensured; through an emotion feedback module, the system dynamically adjusts the expression, intonation and interaction frequency of the AI digital human according to a second emotion coefficient, so that the emotion feedback is more natural, and the emotion recognition sense of the student is enhanced; besides, the culture adaptation analysis module collects cross-culture emotion data based on the culture background of the trainee, calculates and evaluates a culture adaptation coefficient, and automatically adjusts an emotion recognition and feedback strategy, thereby improving the adaptability of the system in a multi-culture environment and the accuracy of emotion expression.

The invention relates to the technical field of computer vision and human-computer interaction, and discloses an AI digital human intelligent interaction method, system and equipment based on multiple modalities, and the method comprises the steps: pre-awakening a digital human when a human face is detected, and further thoroughly awakening the digital human based on recognized preset voice information or preset gesture information; voice and video information of a user in the interaction process is obtained, a keyword extraction result, a gesture recognition result and an emotional state tag are generated, a pre-constructed knowledge base is utilized to retrieve related information, a big language generation model module is combined to generate an answer text, and the answer text is input into a preset voice synthesis model to generate emotional voice output. And based on the current emotional state label of the user, driving the digital human animation to be output in an emotional manner. According to the method and the system, the digital human for understanding the emotion of the user, generating personalized answers, providing voices with rich emotions and displaying natural expressions and actions can be created, better interaction with the user can be realized, and more humanized and effective services can be provided.

The invention discloses an interactive personal health management system based on artificial intelligence and an operation method. Different from a part of existing modes, limited health data and brief basic information which are acquired by a single acquisition device and serve as a unique health information source of a user are not limited, but only serve as health basic information, and a concept and an implementation scheme of'health supplement information 'are provided. On the basis of obtaining the health basic information, through alternate or simultaneous use of multiple types of collection devices and multiple collection ways and modes, more health supplement information of the user is obtained, and a comprehensive AI health information source of the user is jointly constructed. The AI health information source is analyzed and calculated through an artificial intelligence health algorithm, once the AI health information source changes, the AI health assessment result is adjusted accordingly, an infinite loop mechanism is formed, the user health assessment result is promoted to be infinitely close to precision, and health information is interacted with the user in real time through the AI health digital human in a natural language.

The invention provides a virtual human design and application platform, method and device based on artificial intelligence, and relates to the technical field of virtual digital humans. The method comprises the steps of performing local anonymization on multi-modal input data on user equipment, encoding generated anonymized multi-modal features to obtain a multi-modal feature vector, and inputting the multi-modal feature vector into an emotion calculation model to obtain a user emotion intensity quantized value; inputting the multi-modal feature vector into a context sensing model, and generating a user intention vector after context correction in combination with a knowledge graph; generating an updated personality parameter matrix according to the user emotion intensity quantized value and the user intention vector; and outputting voice waveform data, facial muscle motion parameters and skeleton joint coordinate data based on the personality parameter matrix, and driving the virtual digital human three-dimensional model to perform real-time rendering. According to the scheme, the naturalness, emotional resonance and long-term user retention rate of virtual digital human interaction can be improved, and user privacy data security is protected.

The invention provides a digital human interaction method and system based on naked eye 3D visualization, and the method comprises the steps: obtaining query content provided by a user in a multi-modal manner, carrying out the semantic understanding and intention analysis of the query content through a natural language processing technology, and obtaining user intention data; performing matching search in a preset content database according to the user intention data, and if a highly related answer template or video clip exists, calling preset content and feeding back the preset content to the user; according to different answer sources, corresponding digital human video visualization schemes are adopted, a digital human driving engine is used for matching mouth shapes, expressions and actions, and matched digital human videos are synthesized; and displaying the digital human video through a transparent LCD screen.

The invention discloses a holographic display digital human speech recognition enhancement method based on multi-modal interactive learning, and the method comprises the following steps: S1, collecting and preprocessing data, and generating a multi-modal feature matrix; s2, inputting the multi-modal feature matrix into a self-adaptive multi-modal interaction kernel to generate multi-modal interaction features; s3, inputting the multi-modal interaction features into an emotion convolution block to generate semantic-emotion mapping features; s4, inputting the semantic-emotion mapping features into a cross attention generator, and outputting multi-modal fusion features; s5, inputting the multi-modal fusion features into a multi-modal fusion-decomposition two-way network, and generating multi-modal features after decoupling optimization; s6, inputting the multi-modal features into a holographic-speech symbiosis optimization module to generate speech recognition and holographic display data; and S7, performing parameter optimization by applying self-supervised learning and comparative learning strategies to realize self-adaptive adjustment. According to the invention, the enhancement of holographic display digital human speech recognition is realized by using a multi-mode interactive learning method.

The invention discloses an accurate international communication digital human real-time dialogue method fused with a multi-modal technology. The method comprises the following steps: S1, constructing a digital human image and tone; s2, propagation content generation and problem guidance; s3, semantic analysis and intention clarification based on the real-time voice dialogue; s4, geographic preference modeling and path planning; s5, cross-context propagation content generated based on retrieval enhancement is generated; s6, visually displaying the propagation content; and S7, carrying out digital human-driven multi-language propagation content real-time output and feedback closed-loop optimization. Through accurate utterance expression analysis, accurate international propagation problem recommendation is provided, cross-context propagation content generation based on semantic understanding is realized, digital people with voice features and visual images are constructed, real-time dialogue interaction of users is realized, and user experience is improved. The system can carry out geographic modeling according to the region where the accurate problem is located, language preference and propagation object culture characteristics, and differential propagation path planning is achieved.

The invention discloses a chronic disease follow-up visit and management platform driven by a large model, and relates to the technical field of medical information technology platforms. The system is applied to managing and tracking the follow-up visit process of chronic disease patients and providing health consultation and answers, and comprises a user interface module, a logic control module, a database module, a large model module and an auxiliary function module, and the user interface module is used for providing an interactive interface between a patient and a doctor. An intelligent and efficient system platform is constructed by integrating a large model technology, a voice recognition technology and a digital human technology, the system platform adopts a privatized large model, data security and privacy protection are ensured, medical institutions are allowed to autonomously optimize and control the application of the model, and on the whole, through an advanced technical means, the intelligent and efficient system platform has the advantages of being high in practicability and easy to popularize. Refining, individuation and high efficiency of chronic disease management are remarkably improved, and the purpose is to improve the health outcome and life quality of patients.

The invention provides an editable digital human modeling method and device based on neural texture and three-dimensional Gaussian, and the method comprises the steps: carrying out the modeling of the expression, posture and appearance of a digital human through a three-dimensional Gaussian splashing model according to the motion sequence video data of a target object, and obtaining a Gaussian model of the digital human; modeling the expression, posture and appearance of the digital human through the neural texture model to obtain a neural texture model of the digital human; pruning the Gaussian model, and performing hybrid rendering according to a rendered image output by the pruned Gaussian model and a rendered image output by the neural texture model of the digital human to obtain a hybrid rendered image; and optimizing a Gaussian model of the digital human based on the mixed rendered image and the action sequence video data to obtain an editable digital human model. According to the method, the storage overhead and the structural complexity of the digital human model are reduced, and the high-frequency details and editing efficiency of the skin surface of the digital human model are improved.

The invention discloses a multi-mode collaborative digital human interaction method and related equipment based on a thinking chain, and the method comprises the steps: receiving language, action and text signals inputted by a user through a multi-mode perception module, and generating a multi-mode thinking chain driven by the thinking chain in combination with a historical dialogue scene; the thinking chain dynamically binds time sequence nodes of voices, actions and expressions and defines semantic association of the time sequence nodes; emotional consistency, time sequence coherence and intention matching degree of multi-modal output are verified through a real-time cooperative verification mechanism, and local backtracking and dynamic correction of a thinking chain are triggered; and finally, driving the digital person to output collaborative voice, actions and expressions according to the optimized thinking chain. According to the method, a multi-modal interaction thinking chain is generated by adopting a large language model, semantic deep collaboration and dynamic adaptability of digital human output are realized through a sequential binding and real-time verification mechanism, the problems of multi-modal splitting, intention matching deviation and response stiffness in the traditional technology are solved, and the interaction simulation degree and scene robustness are remarkably improved.