32842 results about "Speech sound" patented technology

The invention provides an intelligent real-time interactive question-answering system based on a virtual digital human, and belongs to the technical field of voice signal processing and voice recognition, and the system comprises a data acquisition module which receives a voice or text interaction request input by a user, collects the expression dynamic parameter sequence and limb movement sequence data of the user in real time, and transmits the data to a user interaction module; obtaining a standardized voice feature vector and structured text data; the cross-modal fusion module is used for constructing an interactive feature matrix; the behavior decision module outputs a decision instruction set; the knowledge retrieval module is used for generating an answer text with emotional adaptability and voice features; and the voice generation module is used for generating a mouth shape animation key frame, a micro expression parameter sequence and a limb action track of the virtual digital human, generating a voice response in combination with the answer text and the voice characteristics, and pushing the voice response to the user terminal. According to the method, the interaction experience and adaptability of the virtual digital human are remarkably improved.

The invention discloses a Transform-based cross-modal fusion multi-modal emotion recognition method and device, which are used for solving the problems of modal isomerism, difficulty in time alignment and insufficient dynamic emotion modeling in a multi-modal emotion recognition task, and the method takes the accuracy and robustness of emotion recognition as performance evaluation indexes. Firstly, feature information of three modes of vision, voice and text is obtained, feature extraction is performed on each mode through a deep learning model, then features of different modes are fused by using a cross-mode Transform module, and a complex dependency relationship between the modes is dynamically modeled through a multi-head self-attention mechanism, so that more accurate emotion recognition is realized, and the emotion recognition efficiency is improved. And finally, performing emotion prediction on the fused features based on time sequence modeling and an emotion classification module. According to the method, the problems of modal isomerism, difficulty in time alignment and insufficient dynamic emotion modeling in multi-modal emotion recognition can be effectively solved.

The invention belongs to the technical field of artificial intelligence and big data, and discloses a customer data processing and insight system based on a big language model. The system is composed of a multi-source data access module, a data preprocessing and label fusion module, a large language model semantic understanding module, a knowledge enhancement and semantic linkage module, an insight generation and visualization module, an intelligent strategy output module and a feedback learning and self-optimization module. According to the method, multi-source heterogeneous data such as texts, voices and structured behaviors are integrated, and the deep semantic analysis capability of a large language model is combined, so that global modeling of customer behaviors and intentions is realized; a multi-modal synchronous acquisition and standardization mechanism eliminates data format barriers, and a dynamic label mechanism adapts to context changes, so that the system can capture deep semantic association in customer expression, and compared with a traditional keyword matching method, the semantic understanding accuracy is improved by more than 40%, and a more complete data base is provided for insight generation.

The invention discloses an online teaching interaction method based on a multi-modal knowledge graph, a medium and equipment, and the method comprises the steps: firstly collecting original teaching information, and carrying out the cross-modal semantic alignment processing, and obtaining structured teaching information; then constructing a multi-modal knowledge graph containing text concept entities, video key frame feature vectors and voice text transcription contents, and forming a concept-visual feature association edge, a concept-voice segment association edge and a cross-modal similarity association edge; converting the user interaction behavior data into a knowledge graph query vector, extracting a three-dimensional teaching situation sub-graph from the multi-modal knowledge graph, and generating progressive or comparative teaching content according to a user operation type; and finally, dynamically adjusting a teaching strategy by constructing a cognitive state tracking matrix, and generating a personalized learning navigation map. The problems of knowledge fragmentation and insufficient interaction intellectualization in virtual simulation teaching are solved, and the teaching effect of ideological and political education is improved.

The invention relates to the technical field of voice semantics, can be applied to business scenes of financial science and technology, medical health, poster design and the like, and discloses a visual sense generation method and device based on semantic association modeling, equipment and a medium. Generating a demand text containing theme and style parameters; semantic features in the demand text are extracted, semantic association weights are constructed, and element layout coordinates are optimized in combination with spatial distribution constraints; and encoding the layout information into a control matrix, fusing the control matrix with the initial noise, adjusting a noise reduction process through an encoding and decoding network, and generating target visual content highly matched with the semantic meaning of the user instruction. According to the method, the layout optimization function is constructed, the diffusion model is guided to focus the semantic salient region in space, language model output and the visual generation process are closely combined, structured response and space mapping of user semantic requirements are achieved, and the expression consistency and personalized adaptation capacity of visual content generation are improved.

The invention discloses an intelligent multi-modal virtual digital human interaction system based on an AI language large model. The system comprises a high-authenticity face generation module; the high-authenticity face generation module uses an AdaAN network, based on adaptive feature fusion and voice driving and time sequence modeling of voice features, feature information related to voice is extracted, the extracted voice features are processed through a deep neural network, it is ensured that the voice and facial expressions are highly aligned in time and space, and the face recognition accuracy is improved. Collecting a bio-electricity signal, mapping the signal to facial muscle movement, generating a final facial expression, and interacting with a user; the system further comprises an intelligent interaction module, a training optimization and efficient generation module, an efficient integration module, a multi-modal data acquisition module, an AI large model core processing module, a digital human image generation and driving module, an interaction scene adaptation module and a feedback optimization module. The invention further discloses a multi-mode digital human interaction method which has wide application value.

The invention relates to the technical field of intelligent conference processing, and relates to a conference summary processing method and system using AI, and the method comprises the steps: carrying out the real-time noise suppression of a collected conference audio stream and associated text data through a noise suppression algorithm, and carrying out the cross-modal alignment of the denoised data through a cross-modal alignment algorithm; a domain-specific attention head is inserted into an attention layer of the pre-trained Transform model, a domain-enhanced speech recognition model is constructed, and audio is converted into a text sequence with a speaker tag; adopting a heterogeneous graph neural network to construct a structured topic evolution graph; key decision nodes in the structured topic evolution graph are extracted based on a reinforcement learning strategy, and a final conference summary document is generated. In the decoding stage, the fusion proportion of the acoustic model and the language model is dynamically adjusted based on the real-time acoustic confidence coefficient, the recognition rate of the vocabularies in the professional field is increased, and the problems of frequent term transcription errors and poor semantic coherence in the professional conference are effectively solved.

The invention relates to a multi-microphone array beam forming signal enhancement method and device, and the method comprises the steps: obtaining a multi-channel time domain signal collected by a multi-microphone array, and carrying out the time-frequency conversion, and obtaining a plurality of time-frequency domain signals; acquiring a spatial distribution vector of the multi-microphone array, and performing deformation compensation processing on the complex number of time-frequency domain signals to obtain sound field consistency signals; performing sound source direction feature extraction on the sound field consistency signal, and performing complex time-space attention beam forming network processing to obtain a beam forming signal; performing beam direction and gain parameter optimization on the beam forming signal through a preset adversarial environment simulator to obtain a dynamic frequency band enhanced signal; and performing inverse time-frequency conversion on the dynamic frequency band enhanced signal, and performing low-delay optimization to obtain a target quality voice output signal. According to the invention, signal deformation compensation can be carried out more accurately.

The invention provides an intelligent sound box voice processing method and system based on artificial intelligence, and the method comprises the steps: obtaining audio data and mouth shape video data, carrying out the processing of the audio data and the mouth shape video data, and carrying out the multi-modal feature fusion, and obtaining a fusion feature; performing bimodal voice activity detection on the fusion features to obtain effective voice data; performing context sensing recognition of audio and video fusion on the effective voice data to obtain a first text; constructing a user feature model, and performing semantic understanding on the text based on the model to obtain an understanding result; performing intention recognition and slot filling based on the understanding result to obtain user intention and key information; generating a response strategy in combination with the user intention, the key information and the environment perception data; generating response voice according to the response strategy; and monitoring feedback information of the user to the response voice in real time, and updating the user feature model and the response strategy evaluation model based on feedback. According to the scheme, the voice can be recognized more accurately, and the safety and robustness of the system are enhanced.

The invention relates to the technical field of large language models, and discloses a voice data interaction feedback control processing method based on a large language model. The method comprises the following steps: acquiring an industrial voice instruction through an AMR main controller, and obtaining a standardized vector through industrial lexicon matching and intention classification; inputting an industrial large language model for reasoning processing, and generating an AMR execution scheme; carrying out distributed coordination and task allocation on the AMR cluster to form a control instruction sequence; and the motion controller executes monitoring, processes exceptions and outputs a feedback strategy. Accurate classification and semantic understanding of complex industrial instructions are achieved, the processing capacity of professional knowledge in the industrial field is improved, and meanwhile high-precision real-time state monitoring is achieved.

A system, method, and computer-program product includes identifying a plurality of code synthesis items for a target programming language, generating a code synthesis prompt based on a first sampling of the plurality of code synthesis items, synthesizing, via a large language model, a plurality of raw code segments using the code synthesis prompt, executing the plurality of raw code segments with a code interpreter associated with the target programming language, determining one or more valid code segments of the plurality of raw code segments that the code interpreter successfully executed, aggregating, via a second sampling, the one or more valid code segments into one or more validated code synthesis training samples, and training a code generation model using the one or more validated code synthesis training samples. User interfaces may be provided to allow target coding tasks to be specified via text or speech.

A real-time system for the multimodal generation of virtual reality scenes based on artificial intelligence for the creation of immersive three-dimensional environments from natural language narratives, consisting of: a speech capture module configured to continuously record a user's spoken narrative via one or more directional microphones, preprocesses the captured signal by noise reduction and temporal alignment, and outputs a digital speech stream; A speech-to-text processing unit that is operationally coupled to the speech capture module and configured for real-time speech recognition using a continuous neural transformer model. The unit is trained to transcribe natural language utterances into structured text data while maintaining contextual continuity throughout the evolving narrative. a semantic interpretation processing unit that is communicatively linked to the speech recognition unit and configured to perform natural language understanding techniques to extract contextual entities, spatial references, temporal relationships, and object attributes from the transcribed narrative; the engine includes a large language model that is fine-tuned for spatial reasoning tasks; a scene graph generation module configured to transform the interpreted semantic data into a structured, hierarchical representation that defines nodes for identified entities and edges for corresponding relationships, with each node associated with metadata describing geometry, position, orientation, texture, and linking attributes between objects; a multimodal image-language model processor coupled with the scene graph generation module, wherein the processor is configured to retrieve, adapt, or synthesize appropriate three-dimensional elements from a pre-trained visual-lexical embedding space and align these elements with their semantic and spatial definitions derived from the scene graph; a scene assembly and rendering controller configured to create a cohesive virtual scene from the aligned assets, perform real-time rendering using a GPU-accelerated ray tracing pipeline, and produce a stereoscopic visual output that corresponds to the evolving narrative; A head-mounted virtual reality visualization device connected to the rendering engine and configured to display the generated immersive environment to the user in real time. The device features motion sensors and inside-out tracking cameras to detect head and body movements, dynamically updating viewing angles and perspective within the rendered scene; and a bidirectional feedback module integrated into the head-mounted device and connected to the semantic interpretation processing unit; the module is configured to interpret corrective commands, gestures, or supplementary comments from the user to refine or modify specific scene elements without interrupting the real-time visualization; The system continuously updates the virtual scene as the narrative develops, ensuring temporal synchronization between speech input and rendered output below a defined latency threshold, thus enabling a natural, dialogic construction of complex three-dimensional virtual environments.

The invention discloses an automatic labeling and acquisition standardization method and system based on video content, and the method comprises the steps: decoding a video file into a processable video frame sequence, carrying out the standardization processing, unifying the format and the coding standard, and adjusting the resolution and the frame rate; performing content analysis on the standardized video frame sequence, extracting key frames, identifying objects, scenes and actions, and generating a video analysis result and a preliminary label; according to a video content analysis result, separating an audio stream from the video to carry out noise reduction processing and voice-to-text analysis, and combining the processed text with the video content analysis result to generate multi-modal comprehensive annotation information; based on the comprehensive labeling information, taking feedback of the user in video retrieval, label clicking and watching behaviors as an adjustment basis of label weights, and updating weight distribution of labels by analyzing user behavior data; labeling information and weight distribution are integrated into an index database, and indexes are updated in real time according to user behaviors and content changes.

The invention provides an intelligent mechanical arm control system scheme based on multi-model cooperation, and belongs to the field of automation and intelligent control. The system is divided into four parts of man-machine interaction, detection and perception, task decoupling planning and program generation, and robot execution operation. Man-machine interaction is carried out, a user inputs an instruction in a text or voice format, and instruction processing is carried out through voice recognition and a natural language translation model; detecting and sensing, and identifying and positioning a target captured by the camera by using a multi-modal large model; task decoupling planning and program generation: analyzing the user instruction and the perception information into an executable program by applying a task planning model and a program translation model; and checking and correcting the program by using the program checking model, and driving the mechanical arm to complete a task. According to the system scheme, variable environments can be dealt with, complex language instructions can be understood, intelligent control over the mechanical arm is achieved, and the multi-task execution capacity and the intelligent level of the mechanical arm are remarkably improved.

The invention relates to a multi-modal digital employee reception system and method based on edge calculation, and relates to the technical field of artificial intelligence, and the system comprises a multi-modal data collection module, an edge calculation module, a data fusion and decision module, a trigger activation module, an emotion recognition module, an interaction response module and a behavior prediction module. The method comprises the following steps: acquiring voice, image and action information of a user through a high-resolution camera, a microphone array and an infrared sensor; locally processing data by using an edge calculation module, and extracting voice, vision and thermal imaging features; a Transformer model is dynamically optimized through Test Time Training (TTT), multi-modal data are dynamically fused in combination with a multi-head attention mechanism, a comprehensive decision vector is generated, the system can accurately judge the emotional state of a user, user behaviors are predicted based on a fused Token sequence and historical records, personalized services are actively provided, and by dynamically adjusting a trigger threshold value and a weight distribution mechanism, the emotion state of the user can be accurately judged. The false triggering rate and the missed triggering rate are reduced, and the stability and the robustness of the system are enhanced.

The invention provides an AI-based marketing video auditing method, and relates to the technical field of AI marketing video auditing, and the method comprises the steps: obtaining a multi-modal data original structure set, and extracting image semantic features, voice expression features, text semantic features and scene label information, and obtaining an image semantic feature set, a visual rhythm feature set, a voice expression feature set, a voice and picture synchronous association vector structure, a text semantic feature set and a subtitle semantic and image main body linkage relation graph. By constructing an image semantic feature set, a voice expression feature set, a text semantic feature set and a visual rhythm feature set and fusing the image semantic feature set, the voice expression feature set, the text semantic feature set and the visual rhythm feature set into a multi-modal content fusion feature tensor, unified modeling of an AI marketing video at visual, auditory and semantic levels can be realized; and subsequent microscopic consistency detection, compliance knowledge graph and emotion semantic conflict identification are effectively performed, so that full-link risk perception and accurate auditing of video contents are realized.

The invention relates to dialect speech synthesis, in particular to a semi-supervised dialect emotion speech synthesis system based on hybrid experts, which comprises a text analysis module for preprocessing an input dialect text and generating a text representation vector through feature extraction and feature fusion; the mixed expert module is used for acquiring dialect acoustic features, rhythm features, emotional features and general acoustic features; the dynamic routing module is used for realizing intelligent cooperation among experts through a task-aware soft routing algorithm; the semi-supervised learning module is used for training supervised learning by using the marked dialect emotion voice data and training self-supervised learning by using the unmarked dialect emotion voice data; the acoustic parameter generation module is used for integrating the output of each expert to generate a complete acoustic parameter set; the neural vocoder is used for converting the acoustic parameter set into final dialect emotion voice; the method can effectively overcome the defect that the dialect emotional voice is difficult to accurately synthesize under the condition of lack of sample resources.

A method for early diagnosis of Parkinson's disease based on multimodal deep learning is provided. Audio-visual data of a to-be-diagnosed subject while performing a speech task is acquired. The audio-visual data are preprocessed to extract a plurality of audio segments and a plurality of video segments. A face image sequence is extracted from each of the plurality of video segments. A Mel-spectrogram of each of the plurality of audio segments is calculated. The face image sequence and the Mel-spectrogram are input into a multimodal deep learning model to output a classification result for Parkinson's disease early diagnosis of the to-be-diagnosed subject. A system for early diagnosis of Parkinson's disease based on multimodal deep learning is also provided.

The invention relates to the technical field of man-machine interaction, in particular to a robot behavior mode dynamic adjustment method based on multi-mode perception, which comprises the following steps: S1, collecting a visual image, a voice signal and an environment parameter of a target scene in real time; s2, extracting each modal feature; s3, dynamically allocating weights, and generating a fusion feature vector; s4, identifying a current scene type and a user attribute; s5, matching a corresponding interaction mode from a preset strategy library based on the scene type and the user attribute identified in the step S4; and S6, executing the matched interaction mode in the S5, and updating the weight distribution rule according to the feedback data. According to the method, vision, voice and environment characteristics are fused through a multi-mode perception technology, the behavior mode of the robot is dynamically adjusted based on a self-adaptive weighted fusion algorithm, and the interaction strategy is optimized in combination with user feedback, so that the interaction accuracy and the intelligent level of the robot under different scenes and user attributes are improved.

The invention relates to the technical field of voice processing, can be applied to business scenes of financial science and technology, medical health and the like, and discloses a voice intention recognition method, device and equipment and a medium, and the method comprises the steps: obtaining a to-be-processed voice signal, carrying out the voice activity detection processing of the voice signal, dividing the voice signal into a plurality of voice segments, analyzing semantic contents of the plurality of voice segments, determining semantic correlation information of each voice segment, analyzing sound source attributes of the plurality of voice segments, determining sound field type information of each voice segment, screening out a target voice segment from the plurality of voice segments according to the semantic correlation information and the sound field type information, and executing intention recognition processing based on the target voice segment to generate an intention recognition result. According to the invention, through a dual analysis mechanism of semantic correlation information and sound field type information, effective screening of voice segments is realized before voice recognition, non-target voice or interference segments are effectively prevented from being sent to an intention recognition model, and the accuracy of a recognition result is improved.

Techniques are disclosed for enabling creators to create multimodal chatbots that are based on or simulate / model characters. The characters may be from audiovisual (AV) media such as films and TV shows or real people. The application leverages a combination of Visual Interpretation AI, Retrieval-Augmented Generation (RAG), Low-Rank Adaptation (LoRA), and function calling to provide rich, interactive experiences. The inferencing performed by an instant multimodal chatbot utilizes base weights, character weights, relationship weights, experience weights as we as environmental inputs. An instant chatbot uses a number of AI models including a video to text model, an image to text model, a sensory to text model, a large language model, a text to video model, a text to image model and a text to voice model. A user can interact with the chatbot in a variety of ways including text, audio and video.

The invention discloses a mechanical arm control method, system and equipment for realizing a multi-modal universal operation task. The method comprises the following steps: acquiring multi-modal data of text, voice and vision; performing modal alignment on the multi-modal input data, encoding the multi-modal input data into a uniform input representation vector, and splicing the uniform input representation vector into a multi-modal instruction paragraph; retrieving a user information voice database based on voiceprint features of the voice instruction data to obtain task associated voice data; instruction information is input into the strategy large model to obtain an action plan; a post-processing task action planning input controller controls a mechanical arm, and a multi-mode input instruction is updated in real time to perceive the latest task state; and performing closed-loop control based on the initial target instruction information and the environment perception information until the task is completed. The task planning and error correction capacity of the mechanical arm is improved through closed-loop control and voice data retrieval based on the large strategy model, and the dynamic adaptive capacity, the autonomous error correction capacity and the multi-mode interaction capacity of the mechanical arm are improved.

The invention discloses a cockpit active recommendation system and method based on a knowledge graph and semantic reasoning, and relates to the technical field of intelligent cockpits. The system receives natural language voice input of a user, executes voice recognition and semantic analysis, extracts user intention, keywords and slot entities, generates structured semantic information, constructs or calls a knowledge graph structure with semantic relation edges in combination with environment context information, and obtains the knowledge graph structure with the semantic relation edges. Semantic path reasoning is carried out based on the path dependence weight and the semantic similarity, a semantic edge label guided graph attention mechanism is introduced to calculate a path consistency score, a candidate recommendation set is generated, the semantic fitting degree and the path score are fused to sort and output recommendation content, and the graph edge weight and the user portrait are updated based on user feedback. According to the method, semantic understanding precision, recommendation path interpretability and system adaptive capacity are improved, and the method is suitable for personalized voice recommendation, man-machine interaction and scene linkage control tasks in an intelligent cockpit.

The invention discloses a dual-prevention intelligent interaction system based on a large language model, and belongs to the technical field of artificial intelligence. The system comprises a big language base model, a knowledge base management module, an agent module, a multi-modal big model and an evaluation module, and the big language base model understands user intentions and processes natural language tasks and provides token processing capacity for other modules; the knowledge base management module is used for classified storage and management of professional knowledge, establishing indexes according to semantics and finally performing localized storage; the intelligent agent module is used for analyzing input of a user, and different workflows are packaged in different intelligent agents so as to provide personalized services for the user; the multi-modal large model module receives voice input by a user, converts the voice into characters, transmits description of information to the Agent, generates answers by analyzing semantics, converts the characters into the voice and sends the voice to the user; the evaluation module establishes an evaluation index system from the four aspects of accuracy, answer correlation, context correlation and context recall rate.

The invention provides a multi-modal intelligent question-answering and recommendation system supporting emotional speech output, and relates to the technical field of visual question-answering, and the system specifically comprises a multi-modal question-answering module, a multi-modal recommendation module, a speech recognition module, and a speech synthesis module. Wherein the multi-mode question answering module converts an image input by a user into a natural language form, and final question answering is completed in combination with a comet knowledge base COMET; the voice recognition module is used for recognizing voice data input into the multi-mode intelligent question answering and recommending system and processing the voice data into a text form; the multi-modal recommendation module identifies whether modal missing exists in input data or not, generates representation of the missing modal, obtains vector representation of the user and the article through a graph convolutional neural network and learning according to an interaction matrix of the user and the article, carries out reliability calculation, and finally sends a multi-modal recommendation result to the user and the article. Sending back to the multi-mode question and answer module; and the voice synthesis module outputs the voice to the user.

The invention discloses a large-model complaint intention recognition method based on sentiment analysis, and the method comprises the steps: obtaining call voice data of a customer, and converting the call voice data into text data; preprocessing the text data, and removing noise and marking components; performing emotion feature analysis on the text data, extracting an emotion feature value, and generating an emotion vector; the emotion vectors and the text data are input into a large model together, and the large model combines the emotion feature values and context semantics to generate intention feature vectors; constructing an emotion-intention state vector, inputting the emotion-intention state vector into an asynchronous dominant actor commentator algorithm model, and generating a corresponding complaint intention probability value; judging a complaint intention probability, and generating risk early warning; and preferentially distributing high-risk customers and responding to customer demands. Through combination of voice data preprocessing, text semantic feature extraction, emotion intensity quantitative analysis and a multi-modal fusion algorithm and efficient risk assessment based on an asynchronous dominant actor reviewer model, the early warning and response capabilities of customer complaint risks are significantly improved.

A neural text-to-speech system may be configured to emphasize words. Applying emphasis where appropriate enables the TTS system to better reproduce prosodic characteristics of human speech. Emphasis may make the resulting synthesized speech more understandable and engaging than synthesized speech lacking emphasis. Emphasis may be manually annotated to, and / or predicted from, a source text (e.g., a book). In some implementations, the system may use a generative model such as a variational autoencoder to generate word acoustic embeddings indicating how emphasis is to be reflected in the synthesized speech. A phoneme encoder of the TTS system may process phonemes to generate phoneme embeddings. A decoder may process the word acoustic embeddings and the phoneme embeddings to generate spectrogram data representing the synthesized speech.

The invention discloses a large-model real-time voice text intention recognition method and system based on text prediction, and the method comprises the steps: obtaining the real-time voice data of a user, carrying out the real-time voice recognition processing through a streaming voice recognition interface, and obtaining a part of transcriptional text; inputting the partial transcription text into a mask language model for text prediction, and generating a plurality of high-credibility complete sentence candidates; based on the complete sentence candidates, the complete sentence candidates are input into a large language model in parallel for intention recognition, a corresponding intention result is obtained, and a mapping relation between the candidate sentences and the intention recognition result is established; and obtaining a sentence completely expressed by the user, calculating the similarity between the complete actual sentence and a plurality of high-credibility complete sentence candidates through a multi-level text similarity algorithm, selecting the candidate sentence with the highest similarity score, and directly obtaining a corresponding final intention recognition result based on the mapping relationship. The objective of the invention is to solve the technical problem of high response delay of an existing voice intention recognition system.

Systems, methods, and computer program products are disclosed for generating a common record based on differing source inputs. Generating a common record based on differing source inputs may include receiving via at least one sensor first non-audible signals indicative of verbalization of an individual; interpreting first words of the individual at least in part using the first non-audible signals received from the at least one sensor; receiving second signals generated by a source other than the individual; performing speech recognition on the second signals to interpret second words from the source; and using the first words and the second words to generate a record.

The invention discloses a robot anthropomorphic interaction method based on multi-modal emotion recognition and customized portrait generation. The method comprises the following steps: S1, dynamically fusing multi-modal emotions; the method comprises the following steps: S1, synchronously acquiring voice, visual and text signals through a multi-source heterogeneous sensor, capturing a user voice stream by a high-fidelity microphone array, and extracting acoustic characteristics such as intonation and speed, S2, performing cross-modal reasoning; s3, synchronously generating contents; step S4: style migration; step S5, anthropomorphic voice and expression generation; according to the method, man-machine interaction emotion is analyzed and generated by utilizing a large language model and multi-modal information fusion, the singleness of interaction emotion and the deficiency of emotional sharing ability are avoided, a strong emotion interaction characteristic is achieved, the image of the robot is obtained through a generative technology and can be migrated to any image, the limitation that a specific image is independently made is broken through, and the interaction effect of the robot is improved. The advantage that one robot can be suitable for different scenes is achieved.