2041 results about "Digital human" patented technology

The invention provides an intelligent real-time interactive question-answering system based on a virtual digital human, and belongs to the technical field of voice signal processing and voice recognition, and the system comprises a data acquisition module which receives a voice or text interaction request input by a user, collects the expression dynamic parameter sequence and limb movement sequence data of the user in real time, and transmits the data to a user interaction module; obtaining a standardized voice feature vector and structured text data; the cross-modal fusion module is used for constructing an interactive feature matrix; the behavior decision module outputs a decision instruction set; the knowledge retrieval module is used for generating an answer text with emotional adaptability and voice features; and the voice generation module is used for generating a mouth shape animation key frame, a micro expression parameter sequence and a limb action track of the virtual digital human, generating a voice response in combination with the answer text and the voice characteristics, and pushing the voice response to the user terminal. According to the method, the interaction experience and adaptability of the virtual digital human are remarkably improved.

The invention discloses an intelligent multi-modal virtual digital human interaction system based on an AI language large model. The system comprises a high-authenticity face generation module; the high-authenticity face generation module uses an AdaAN network, based on adaptive feature fusion and voice driving and time sequence modeling of voice features, feature information related to voice is extracted, the extracted voice features are processed through a deep neural network, it is ensured that the voice and facial expressions are highly aligned in time and space, and the face recognition accuracy is improved. Collecting a bio-electricity signal, mapping the signal to facial muscle movement, generating a final facial expression, and interacting with a user; the system further comprises an intelligent interaction module, a training optimization and efficient generation module, an efficient integration module, a multi-modal data acquisition module, an AI large model core processing module, a digital human image generation and driving module, an interaction scene adaptation module and a feedback optimization module. The invention further discloses a multi-mode digital human interaction method which has wide application value.

The invention belongs to the technical field of virtual person generation, and particularly relates to a digital person video generation method based on a multi-modal large model, and the method comprises the following steps: 1, constructing a multi-modal data system; 2, multi-modal large model training and adaptation are carried out; 3, constructing a digital human three-dimensional model; step 4, performing semantic analysis and modal mapping; 5, generating a time sequence action and a mouth shape; step 6, building and rendering a virtual scene; step 7, audio and video synchronous rendering and synthesis; step 8, quality optimization and defect repair; and step 9, performing user interaction and iterative optimization. Through technical innovation and engineering, the core pain point in digital human video generation is solved, efficient, vivid and customizable content production capacity is provided for virtual anchors, intelligent customer service, enterprise training and other scenes, and the AI digital human technology is promoted to be applied to large-scale business from experiments.

The invention provides a lightweight digital human lesson preparation system based on an intelligent agent, and belongs to the field of intelligent teaching. Through collaborative operation of four core modules of knowledge graph construction and reasoning, multi-modal cognitive agent, lightweight digital human generation and intelligent teaching plan assistance, the problems of low efficiency of resource integration, teaching content homogenization, insufficient digital human interaction experience and the like in traditional lesson preparation are solved. The knowledge graph construction and reasoning module is used for constructing a structured knowledge graph and realizing knowledge point association mining and teaching logic reasoning; the multi-modal cognitive agent module is used for generating personalized explanation content according with a teaching target by fusing multi-modal courseware analysis, semantic understanding and lecture style dynamic adaptation functions; the lightweight digital human generation module is combined with model pruning and emotion modeling technologies to synchronously output natural voice and a high-simulation digital human image; the intelligent teaching plan auxiliary module helps the teacher to intelligently generate a teaching plan and a teaching outline according to the courseware content based on the knowledge graph.

The invention provides a digital human interaction control method and device fusing emotion semantics and logical reasoning and a storage medium. The method comprises the following steps: analyzing multi-modal input data of a user, constructing emotion-semantics joint representation, and generating a logic decision path; and through a cognitive fusion module, emotion-semantic representation and a logic decision path are fused, and an interaction response adapting to emotion and logic consistency is generated. The system optimizes an emotion semantic model and a logical reasoning rule on line according to user feedback and interaction history, and real-time interaction of emotion dynamic and logical rules is achieved. The system can dynamically adjust the logic decision path based on the multi-mode emotional state of the user, and improves the naturalness and situation adaptability of interaction. A dynamic time warping algorithm and a factorization machine are introduced to process a multi-modal feature fusion problem, and the accuracy and robustness of emotional state recognition are improved. The online optimization mechanism enables the model and the rule to be evolved continuously, and reasoning errors are corrected automatically through user feedback, so that error circulation is avoided.

The invention discloses a real-time interactive digital human system supporting high concurrency and an implementation method thereof, and relates to the technical field of digital human interaction.The system comprises a model instance pool module used for loading the weight of a deep learning model to a shared memory area through the memory mapping technology; the multi-thread scheduling module is used for scheduling user requests by adopting a lock-free queue and a dynamic priority algorithm; the asynchronous pipeline processing module is composed of decoupled micro-services, and the modules are connected in series through asynchronous message queues; the client SDK is used for dynamically switching a rendering mode according to terminal hardware performance and network conditions; the elastic capacity expansion and contraction module is used for monitoring resource loads in real time and automatically adjusting the number of service instances; and the audio and video synchronization calibration module is used for ensuring that the synchronization error of the audio and video frames is lower than a preset threshold value through a timestamp alignment algorithm. According to the scheme, core challenges in a high-concurrency digital human interaction scene can be systematically solved, and innovative support is provided for large-scale real-time application.

The embodiment of the invention provides a government affair digital human dynamic interaction method and system based on a multi-mode large model. The method is applied to the technical field of government affair intelligent services, and comprises the following steps: acquiring a policy announcement text, and performing cleaning and structuring processing to obtain a structured policy data set; and extracting old and new policy data, performing difference comparison, marking key change fields, and generating policy change data. Abstracting and element extraction are carried out on the change data to form structured semantic fragments, and the structured semantic fragments are incrementally embedded into the policy knowledge graph. And generating a question and answer pair sample based on the updated knowledge graph, and carrying out self-supervised fine tuning training on the multi-modal large model. A user inputs multi-modal data, and the model generates a government affair response and feeds back the government affair response. According to the scheme, the multi-modal large model can continuously keep the latest policy knowledge; the model is enabled to generate accurate government affair response with consistent context while understanding multi-modal input such as text, voice and image, and timeliness, accuracy and interactive experience of policy interpretation are improved.

Provided in the embodiments of the present application are a video generation method and interaction method based on a digital human, and a device, a storage medium and a program product. In the embodiments of the present application, text-to-speech processing is performed on the basis of voice features of a user and an emotion label, speech-to-expression processing is performed on the basis of a mapping relationship between the voice features of the user and expression coefficients, and a digital human model is rendered on the basis of speech signals and the expression coefficients, so as to obtain video data of the digital human model. Thus, voice features of a user are accurately simulated, so as to ensure that a speech output of a digital human sounds natural and is also highly personalized, thereby realizing personalized driving of the digital human, and improving the realism of the digital human in terms of voice and dynamic images. Thus, the user experience is improved, and the interactivity of the digital human and the authenticity and immersion are enhanced.

The invention relates to the field of knowledge graph construction, in particular to a policy knowledge graph construction method and system based on digital human interaction data analysis. The policy knowledge graph construction system based on digital human interaction data analysis comprises a knowledge graph preliminary construction module, a first graph updating module, a second graph updating module and a digital human interaction module. According to the method, the policy source webpage content change is monitored in real time, and the question information in the digital human interaction data is deeply mined, so that the full-life-cycle dynamic maintenance of the policy knowledge graph is realized; on one hand, a webpage policy updating event is accurately captured, and knowledge injection is automatically completed in combination with a semantic unit matching algorithm; and on the other hand, the conflict characteristics questioned by the user in the interaction data are extracted through dialogue semantic analysis, atlas correction is triggered after confidence assessment and multi-stage verification, it is ensured that the end-to-end timeliness from policy release to user perception is controlled within a reasonable period, and the policy response accuracy of the digital human service is remarkably improved.

The invention relates to the field of multi-modal interaction analysis, in particular to a multi-modal interaction method and system of a digital human agent. The method comprises the following steps: acquiring a real-time face image and a voice signal input stream of an interactive user based on an intelligent agent; performing real-time micro-expression recognition and deep emotion analysis based on the real-time facial image to obtain real-time emotion features of the user; performing time sequence evolution analysis on the real-time emotion characteristics of the user, performing holographic user emotion deep mining, and constructing a user emotion holographic characteristic spectrum; carrying out adaptive acoustic gain processing on the voice signal input stream, and carrying out voice-emotion association analysis based on the user emotion holographic characteristic spectrum to generate a voice-emotion linkage mapping spectrum; and carrying out eyeball fixation point migration tracking based on the user emotion holographic feature map and the real-time face image, and generating a user interaction depth intention signal. Through the real-time deep semantic understanding and emotion perception ability, the intelligent agent interaction intelligence and response accuracy are improved.

The invention provides an AI-based virtual digital human interaction system, and relates to the technical field of data processing, and the system is used for obtaining a user feature data set by performing feature extraction on an interaction data set, calculating an audio change rate and speech speed stability, obtaining speech emotion data, calculating syntactic complexity and counting the number of emotion words. Calculating a facial expression symmetry degree; obtaining facial emotion data; calculating body stability and identifying a visual staying area; obtaining attention state data; adjusting tone and speed of the virtual digital person according to the virtual digital person text, optimizing sentence pattern complexity and sentiment words in the virtual digital person text, adjusting changes of eyes, eyebrows and mouth corners of the virtual digital person, and adjusting information display sequence and visual focus guidance. According to the method, the interaction efficiency of the virtual digital human is improved by analyzing the facial expressions and the body actions of the user.

The invention relates to the technical field of computer vision and human-computer interaction, and discloses an AI digital human intelligent interaction method, system and equipment based on multiple modalities, and the method comprises the steps: pre-awakening a digital human when a human face is detected, and further thoroughly awakening the digital human based on recognized preset voice information or preset gesture information; voice and video information of a user in the interaction process is obtained, a keyword extraction result, a gesture recognition result and an emotional state tag are generated, a pre-constructed knowledge base is utilized to retrieve related information, a big language generation model module is combined to generate an answer text, and the answer text is input into a preset voice synthesis model to generate emotional voice output. And based on the current emotional state label of the user, driving the digital human animation to be output in an emotional manner. According to the method and the system, the digital human for understanding the emotion of the user, generating personalized answers, providing voices with rich emotions and displaying natural expressions and actions can be created, better interaction with the user can be realized, and more humanized and effective services can be provided.

The invention discloses an interactive personal health management system based on artificial intelligence and an operation method. Different from a part of existing modes, limited health data and brief basic information which are acquired by a single acquisition device and serve as a unique health information source of a user are not limited, but only serve as health basic information, and a concept and an implementation scheme of'health supplement information 'are provided. On the basis of obtaining the health basic information, through alternate or simultaneous use of multiple types of collection devices and multiple collection ways and modes, more health supplement information of the user is obtained, and a comprehensive AI health information source of the user is jointly constructed. The AI health information source is analyzed and calculated through an artificial intelligence health algorithm, once the AI health information source changes, the AI health assessment result is adjusted accordingly, an infinite loop mechanism is formed, the user health assessment result is promoted to be infinitely close to precision, and health information is interacted with the user in real time through the AI health digital human in a natural language.

The invention provides a virtual human design and application platform, method and device based on artificial intelligence, and relates to the technical field of virtual digital humans. The method comprises the steps of performing local anonymization on multi-modal input data on user equipment, encoding generated anonymized multi-modal features to obtain a multi-modal feature vector, and inputting the multi-modal feature vector into an emotion calculation model to obtain a user emotion intensity quantized value; inputting the multi-modal feature vector into a context sensing model, and generating a user intention vector after context correction in combination with a knowledge graph; generating an updated personality parameter matrix according to the user emotion intensity quantized value and the user intention vector; and outputting voice waveform data, facial muscle motion parameters and skeleton joint coordinate data based on the personality parameter matrix, and driving the virtual digital human three-dimensional model to perform real-time rendering. According to the scheme, the naturalness, emotional resonance and long-term user retention rate of virtual digital human interaction can be improved, and user privacy data security is protected.

The invention provides a digital human interaction method and system based on naked eye 3D visualization, and the method comprises the steps: obtaining query content provided by a user in a multi-modal manner, carrying out the semantic understanding and intention analysis of the query content through a natural language processing technology, and obtaining user intention data; performing matching search in a preset content database according to the user intention data, and if a highly related answer template or video clip exists, calling preset content and feeding back the preset content to the user; according to different answer sources, corresponding digital human video visualization schemes are adopted, a digital human driving engine is used for matching mouth shapes, expressions and actions, and matched digital human videos are synthesized; and displaying the digital human video through a transparent LCD screen.

The invention discloses an accurate international communication digital human real-time dialogue method fused with a multi-modal technology. The method comprises the following steps: S1, constructing a digital human image and tone; s2, propagation content generation and problem guidance; s3, semantic analysis and intention clarification based on the real-time voice dialogue; s4, geographic preference modeling and path planning; s5, cross-context propagation content generated based on retrieval enhancement is generated; s6, visually displaying the propagation content; and S7, carrying out digital human-driven multi-language propagation content real-time output and feedback closed-loop optimization. Through accurate utterance expression analysis, accurate international propagation problem recommendation is provided, cross-context propagation content generation based on semantic understanding is realized, digital people with voice features and visual images are constructed, real-time dialogue interaction of users is realized, and user experience is improved. The system can carry out geographic modeling according to the region where the accurate problem is located, language preference and propagation object culture characteristics, and differential propagation path planning is achieved.

The invention provides an editable digital human modeling method and device based on neural texture and three-dimensional Gaussian, and the method comprises the steps: carrying out the modeling of the expression, posture and appearance of a digital human through a three-dimensional Gaussian splashing model according to the motion sequence video data of a target object, and obtaining a Gaussian model of the digital human; modeling the expression, posture and appearance of the digital human through the neural texture model to obtain a neural texture model of the digital human; pruning the Gaussian model, and performing hybrid rendering according to a rendered image output by the pruned Gaussian model and a rendered image output by the neural texture model of the digital human to obtain a hybrid rendered image; and optimizing a Gaussian model of the digital human based on the mixed rendered image and the action sequence video data to obtain an editable digital human model. According to the method, the storage overhead and the structural complexity of the digital human model are reduced, and the high-frequency details and editing efficiency of the skin surface of the digital human model are improved.

The invention discloses a multi-mode collaborative digital human interaction method and related equipment based on a thinking chain, and the method comprises the steps: receiving language, action and text signals inputted by a user through a multi-mode perception module, and generating a multi-mode thinking chain driven by the thinking chain in combination with a historical dialogue scene; the thinking chain dynamically binds time sequence nodes of voices, actions and expressions and defines semantic association of the time sequence nodes; emotional consistency, time sequence coherence and intention matching degree of multi-modal output are verified through a real-time cooperative verification mechanism, and local backtracking and dynamic correction of a thinking chain are triggered; and finally, driving the digital person to output collaborative voice, actions and expressions according to the optimized thinking chain. According to the method, a multi-modal interaction thinking chain is generated by adopting a large language model, semantic deep collaboration and dynamic adaptability of digital human output are realized through a sequential binding and real-time verification mechanism, the problems of multi-modal splitting, intention matching deviation and response stiffness in the traditional technology are solved, and the interaction simulation degree and scene robustness are remarkably improved.

The invention relates to the technical field of data processing, and discloses a multi-modal digital human generation method and system based on AIGC and a storage medium. The method comprises the steps of performing modeling and dynamic weight adjustment processing on multi-modal feature data to obtain dynamic personalized knowledge data; performing intelligent matching processing on the target scene information and the dynamic personalized knowledge data to obtain situational behavior instruction data; performing hierarchical action sequence generation, adaptive resolution optimization and speech synthesis processing on the initial digital human to obtain multi-modal digital human performance data; and behavior deviation identification is carried out on the multi-modal digital human performance data to obtain behavior deviation data and performance feedback data, and adaptive optimization processing is carried out on the multi-modal digital human performance data and the performance feedback data to obtain a target digital human. According to the invention, the efficiency and accuracy of multi-modal digital human generation based on AIGC are improved.

The invention discloses a personalized digital human generation method based on a single video, and relates to the field of virtual digital human modeling and driving, and the method aims at the video and voice data of a target person, through introducing a multi-modal alignment constrained voice driving synchronization mechanism and combining semantic understanding and an emotion label expression generation model, a personalized digital human model is generated. High synchronization, nature and vividness of digital human facial expressions and voice contents are realized. According to the method, a self-supervised style consistency constraint is added in model training to ensure that the style of a generated character image is stable, and a 3D semantic mask is added in an image fusion stage to improve the fusion precision and realistic effect of a synthetic facial expression and a reference face. According to the method, the digital human can be rapidly cloned and driven to synthesize the expression through the voice only through a single video sample, the generation process is efficient, and the obtained digital human video has excellent sense of reality and interactivity.

The invention relates to the technical field of artificial intelligence and virtual reality, and discloses a virtual digital human generation and interaction optimization system based on multi-modal data fusion, and the system comprises a multi-modal input fusion module which is used for collecting and fusing multi-modal data from audio, video and touch; the adaptive behavior adjustment module is used for adaptively adjusting the behavior and response strategy of the virtual digital human according to the user input and the interaction history; the real-time dynamic rendering module is used for generating and rendering an image of the virtual digital human and synchronizing actions, expressions and language output of the virtual digital human in real time; the personalized customization interface module is used for providing personalized customization functions of the appearance, the sound and the behavior of the virtual digital human for the user; and the system integration and optimization module is used for integrating the modules. By fusing data (such as audio, video and touch) of multiple modes, the system can comprehensively understand behaviors and emotional states of the user, interaction delay is reduced, and the user can experience a smoother interaction process.

The invention discloses a non-perpetual culture immersive interactive experience device and system based on a virtual digital human, and relates to the technical field of intangible cultural heritage intelligent interaction, and the device comprises a multi-mode sensing module which collects data such as user actions and voices through a depth camera and the like; the non-perpetual knowledge base module stores a non-perpetual knowledge graph and a case library; the digital human modeling engine fuses the inheritor characteristics and the user data to generate a virtual digital human; the immersion interaction engine constructs a cross-platform rendering environment to realize five-sense fusion experience; and the adaptive learning module analyzes the interaction sequence to optimize the digital human behavior, and all the modules cooperate to realize the intelligent interaction experience of non-genetic culture. Through the multi-modal perception module, the digital human modeling module and the like, the immersive interactive experience of the non-perpetual culture is realized, the user behavior can be accurately captured, the characteristic virtual digital human is generated, the five-sense fusion experience is provided, the content can be optimized based on the user interaction, the inheritance and propagation of the non-perpetual culture are promoted, and the participation degree and the sense of identity of the user are improved.

The invention relates to a multi-modal fusion real-time digital human driving method based on unified behavior vector mapping, and belongs to the technical field of human-computer interaction. The method comprises the following steps of: 1) extracting multi-modal features of voice, action and vision, and generating input features by adopting language feature-emotion decoupling, staged action modeling and macroscopic-micro expression flow analysis; (2) cross-modal collaborative alignment and conflict correction are carried out, and high-precision fusion is realized through knowledge graph constraint time sequence decomposition, particle-by-particle interactive fusion and a situation-driven modal dominant strategy; and 3) constructing a three-dimensional behavior vector space, mapping the multi-modal features to a unified coordinate by utilizing honeycomb grid projection, and driving the digital human to output in combination with a coordinate-action mapping table. According to the method, the problems of difficulty in multi-modal time sequence alignment, feature isomerism and out-of-control conflict are solved, real-time interaction of naturalization, emotional and scenarized is realized, and the expressive force and adaptability of the digital human are remarkably improved.

The invention provides a three-dimensional Gaussian digital human generation system and method and electronic equipment, and relates to the technical field of augmented reality, and the system comprises a data obtaining unit which is used for obtaining visual depth data, dynamic behavior data and environmental perception data, and generating point cloud data and multi-modal data according to the visual depth data, the dynamic behavior data and the environmental perception data; the rendering adjustment unit is used for monitoring the fixation area in real time and determining a rendering strategy; according to the real-time computing power parameter, determining a rendering specification; the driving unit is used for performing feature extraction on the multi-modal data to obtain multi-modal features and generating face driving parameters and limb driving parameters of the digital human model; and the modeling unit is used for generating a basic grid of the digital human model according to the point cloud data and the initial digital human model, and performing expression rendering and action rendering on the basic grid according to a rendering strategy and a rendering specification in combination with the face driving parameters and the limb driving parameters to obtain a rendered digital human model. The fluency, authenticity and naturalness of the display effect are improved.

The invention discloses a method for realizing synchronization of a character expression and a lip shape in a video through an emotion in sound and a cloned digital human system, and the method for realizing synchronization of the character expression and the lip shape in the video through the emotion in the sound comprises the steps: collecting audio information, and extracting multi-dimensional features of the sound; applying a pre-trained expression and lip shape generation model to generate corresponding expression parameters and lip shape parameters according to the multi-dimensional features of the sound; performing fusion processing on the expression parameters and the lip shape parameters, and generating a continuous animation sequence according to the fused parameters; and rendering the continuous animation sequence to generate video information with expressions, lip shapes and sound emotion synchronization. The high-precision expression and lip shape synchronization is realized, the natural fidelity of the cloned digital human is improved, and the application range and value of the cloned digital human are expanded.

The invention discloses a multi-mode driven virtual digital human face animation generation method and system, and relates to the field of computer graphics, and the method comprises the steps: obtaining voice input and text input, and extracting voice features and text features; dynamically fusing the two features through an attention fusion model to generate facial expression and head posture control parameters, and dynamically adjusting contribution weights of a voice mode and a text mode to the control parameters by adopting a driving strategy of differentiation of facial upper expression and facial lower expression; and performing local deformation on the virtual digital human face image based on the control parameter to generate an initial animation frame, and performing refining processing on the initial animation by using a generative adversarial network to obtain a refined face animation. By means of the technical scheme, the natural and vivid virtual human face animation matched with the voice content and the text semantics can be generated.

The invention provides a digital human generation method based on a multi-modal large model. The method comprises the following steps: constructing a digital human basic model; generating a structured training set; generating a question and answer model supporting multi-channel interaction; semantic answers of the user questions are output, text emotional tendencies of the semantic answers are extracted, and emotional intensity parameters are output; generating facial muscle movement track data, and performing real-time rendering on the digital human basic model according to the facial muscle movement track data to output a digital human three-dimensional image with emotion expression. According to the embodiment of the invention, cross-modal alignment is carried out on text, image and audio data, and a multi-modal large model containing visual, voice and knowledge models is optimized by using a joint training method, so that more natural and smoother multi-channel interaction experience is realized; in addition, by introducing an emotion recognition model and a face interaction model, the emotion tendency contained in the semantic answer can be captured and reflected more accurately, so that a digital human three-dimensional image with real emotion expression is output.

The invention discloses a real-time digital human generation system and method based on low-calculation-amount voice driving, and relates to the technical field of digital human interaction, and the system comprises an audio processing module which is configured to receive voice input in real time and extract an audio feature vector; the driving and rendering module is used for mapping the audio feature vectors into parameters representing mouth movement, generating a dynamic mouth image based on the preprocessed static face reference data and the parameters, and fusing the dynamic mouth image with the reference image; the synchronous control module is used for ensuring the synchronization of the audio features and the rendering video frames according to a timestamp alignment mechanism and a PID feedback control algorithm; and the dynamic scheduling module is used for monitoring a hardware resource load in real time and realizing dynamic allocation of computing resources through multi-thread parallel and task priority adjustment. According to the technical scheme provided by the invention, the breakthrough of low delay, high fidelity and low energy consumption of the digital human can be realized in mobile terminal, embedded and multi-platform scenes.

The invention discloses a controllable digital human real-time interaction system based on a multi-modal sensor, and relates to the technical field of digital humans, and the system comprises a multi-modal sensor data collection module which captures user interaction videos, collects interaction voice signals of a user, and collects the temperature of the user and the distance between the user and equipment; the multi-modal feature extraction module comprises a static feature extraction unit and a dynamic feature extraction unit; the multi-modal fusion module is used for analyzing the trustworthiness degree from the multi-modal feature extraction module, judging the importance of each information source in the current task, comprehensively considering and dynamically adjusting weight distribution, and fusing multi-modal feature data; the intention understanding module is used for understanding the interaction intention of the user by utilizing a large language model (LLM) based on the fused features and contexts; and the digital human interaction output module is used for generating an open domain text reply based on the interaction intention of the user and finally realizing digital human multi-mode fusion interaction output.

The invention relates to the technical field of electric digital data processing, in particular to an artificial intelligence algorithm-driven digital human language understanding and corresponding reaction interaction method. The method comprises the following steps: collecting historical language data, text data, action instruction data and physical environment information of a user to obtain structured training data; vectorizing the text data to obtain a text semantic vector; environment feature vectors are extracted from the physical environment information; splicing the text semantic vector and the environment feature vector to obtain an environment enhanced text vector; and constructing a pre-training language model based on the environment enhanced text vector, and performing secondary pre-training to generate an optimized language model parameter. According to the invention, by fusing the user language and the physical environment information and combining with the multi-module intelligent component, context perception, efficient understanding and multi-modal natural interaction of the digital human in a complex scene are realized, and the intelligence, adaptability and user experience of the system are remarkably improved.