281 results about "Emotion classification" patented technology

This invention relates to a gait emotion recognition method, system, storage medium, and computer device based on spatiotemporal graph convolution. The method includes the following steps: S1, data augmentation by reversing the temporal direction of gait; S2, obtaining deep emotion features and prior emotion features respectively through a spatiotemporal graph convolutional network and prior feature statistical methods; S3, performing nonlinear mapping on the prior emotion features using a feature mapping layer; S4, inputting the fused features of the deep emotion features and prior emotion features into an emotion classifier to obtain the emotion category. The feature mapping layer of this invention achieves more effective feature fusion by performing nonlinear mapping on prior features; it also introduces causal temporal convolution to replace general temporal convolution, effectively extracting fine-grained temporal features by enhancing temporal correlation and cross-period feature fusion. Furthermore, a walking direction recognition auxiliary task is designed to accelerate the training and convergence speed of the model, enhancing the ability to extract temporal-dependent features and the performance of emotion recognition.

The invention discloses an emotion recognition method and system based on cross-modal alignment and matching enhancement. According to the method, firstly, feature extraction is carried out on text, audio and video modalities in a data set, and then a text and audio cross-modal emotion alignment module and a text and video cross-modal emotion alignment module are constructed respectively, so that cross-modal semantic alignment is realized. Constructing an emotion label matching module based on an alignment result, generating modal pairs with similar emotions but different labels by using a difficult negative sample mining strategy, and paying attention to cross-modal emotion consistency through a dichotomy task guide model; performing modal feature fusion on the three modals through a six-layer attention crossing mechanism, finally splicing feature vectors, inputting the spliced feature vectors into a long-sequence context fusion modeling module for deep modal fusion, and capturing cross-modal interaction information; and the fused features are sent to an emotion classification module, and a final emotion category recognition result is output.

The invention discloses a multi-modal emotion recognition method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring text, video and audio data of a user and respectively performing feature extraction to obtain text features, audio features and facial features; the three features are input into a pre-trained multi-modal emotion recognition model, the multi-modal emotion recognition model comprises a first fusion module, a second fusion module, a third fusion module, a fourth fusion module and a classification module, the audio features and the text features are fused through the first fusion module, and audio text features are obtained; fusing the facial features and the text features by using a second fusion module to obtain facial text features; performing feature enhancement on the text features by using a third fusion module to obtain enhanced text features; fusing the audio text features, the face text features and the enhanced text features by using a fourth fusion module to obtain multi-modal features; and classifying the multi-modal features by using a classification module to obtain a sentiment classification result of the user.

The invention discloses a multi-mode emotion continuous recognition method for medical treatment, belongs to the technical field of artificial intelligence and medical treatment information, and mainly aims to simulate the dynamic change process of emotion by establishing a Neural ODEs framework and overcome the static property and discreteness of emotion modeling in a traditional method. Through a causal inference technology, emotional features are separated from individual-independent physiological differences, and the generalization ability across individuals is improved. A self-supervised learning method is utilized, the synergistic effect between the EEG and the eye movement signal is improved through cross-modal contrast learning, and the emotion recognition precision is enhanced. The calculation complexity is reduced through a dynamic sparse attention mechanism, and meanwhile, focusing is performed on a key time slice in emotion recognition. Through multi-task joint learning, the model learns multiple tasks such as emotion intensity regression and tested identity recognition during emotion classification, and the personalized emotion recognition capability is improved.

The invention discloses a music stave sentiment classification method and system based on multi-level distillation, and belongs to the technical field of music information processing. The method comprises the steps of firstly collecting music stave data and converting the data into stave data vectors, then performing feature extraction by using a long short-term memory network, then constructing a teacher network and a student network for knowledge distillation, and realizing multi-level knowledge transmission through temperature scaling, KL divergence loss and mask feature distillation. And finally, training a lightweight classification model to complete sentiment classification. The knowledge distillation technology is creatively applied to staff sentiment classification, the classification accuracy is effectively improved through an online multi-level distillation mode, and the technical problems that a traditional method lacks semantic information and a self-supervised model is not suitable for sentiment tasks are solved. The method has the main advantages of high classification precision, light model weight, capability of effectively capturing music emotion features and the like.

The invention provides a multi-modal sentiment analysis method based on a diffusion model and self-paced learning. The method comprises the following steps: firstly, dividing a data set into a missing image modal data set and a complete modal data set according to image modal integrity; thirdly, constructing a feature alignment diffusion model, and performing image generation; training the diffusion model by adopting a self-paced learning strategy and a missing image data set; and based on the trained diffusion model, guiding a reverse process through text features to generate feature representation of the missing image. And carrying out weighted fusion on the generated image features and text features by using an attention mechanism, and dynamically adjusting contribution weights of all modalities to generate a complete multi-modal feature representation. And finally, integrating a missing modal completion result and the complete modal features to form a unified multi-modal representation, inputting the unified multi-modal representation into a multi-modal sentiment classification module, and outputting a sentiment classification result. According to the method, the problem of multi-modal sentiment analysis under random missing of image modals is effectively solved, and the generation quality and semantic consistency are improved.

The invention discloses a multi-modal sentiment analysis method and system based on context enhancement and cross attention, and aims to solve the problem of low sentiment analysis precision caused by insufficient multi-modal feature fusion and insufficient context information utilization in the existing multi-modal sentiment analysis scheme. The system comprises a feature extraction module, an intra-modal context enhancement (ICE) module, a modal alignment and dynamic gating (GCU) weighting module, a time sequence-modal cross attention fusion (TMA) module (cross-modal depth fusion module), and a shared representation generation and multi-task parallel decoding module. The text features are subjected to deep context coding through a BERT model, and the audio and visual features are processed through multi-scale convolution and time sequence Transform; the ICE module enhances audio and visual features by capturing time sequence dependence in a single mode; the GCU module generates a gating weight moment by moment based on a GRU network driven by a global context, and dynamically weights the three modal features; the TMA module realizes deep interaction and fusion in time and modal dimensions through asymmetric time sequence-modal cross attention to generate a shared representation; and the decoding module executes sentiment regression, sentiment classification and modal reconstruction tasks in parallel. Through the structure, the precision and robustness of sentiment analysis in a complex scene are effectively improved.

The invention provides an aspect-level sentiment classification method and system based on multi-modal alignment and reflection enhancement, and relates to the technical field of artificial intelligence, and the method comprises the steps: employing a pre-trained teacher model to enable an original image to be combined with a knowledge guide prompt to generate text description; splicing the description, the original text, the target aspect word and the structured reasoning prompt template into a multi-modal aligned enhanced input sequence, and inputting the multi-modal aligned enhanced input sequence into a pre-trained student model to obtain a predicted emotion tag; aiming at a sample with a prediction error, generating an reflection-correction inference chain by utilizing a teacher model, and constructing an reflection enhancement data set; and finally, combining the original data set and the reflection enhancement data set, and performing supervised fine tuning training on the student model. According to the method, the semantic alignment and self-error correction capability of the model is enhanced, so that the sentiment classification accuracy is improved.

The present invention provides a user habit-based application (app) recommendation method, a mobile terminal, and a storage medium. The method comprises the steps of: collecting user inherent attribute information, app interaction information, and location information, performing analysis and quantification, then performing marking, and establishing user interaction scenario information; collecting input information during app interaction, and recording the input information as interaction associated data on the basis of a corresponding time period and an app package name; processing the interaction associated data via a first neural network, and outputting emotional classification; processing the interaction scenario information via a second neural network, and outputting real label probability distribution information; matching reference interaction scenarios in a corpus according to the real label probability distribution information, and finding corresponding apps; and performing emotional classification on the reference interaction scenarios of the apps on the basis of an SVM algorithm, and recommending to a user the app conforming to an emotional classification tendency. In this way, a user is helped to screen for the app fitting the app interaction habit of the user, and the time cost of searching for the app by the user is reduced.

The invention belongs to the field of deep learning, and provides a cross-subject brain electrical emotion recognition method and system for space-time adaptive graph coding learning, and the method comprises the steps: extracting features from different frequency bands through a sliding window technology, and constructing a feature matrix covering channels and time dimensions; fusing space-time hybrid embedding, time embedding and space embedding, and converting the feature matrix into a high-dimensional semantic vector; calculating channel characteristic difference and time trend change, and generating a dynamic topology matrix adaptive to cross-tested individual difference; combining the dynamic topological matrix to calculate spatial cross-channel and time stride length attention in parallel, and fusing and strengthening key spatial-temporal features; and the decoder maps the coding features by means of a domain adversarial decoding module, reduces the distribution difference between a source domain and a target domain through cross-domain adversarial training, and outputs an emotion classification result. Through the method, the cross-subject generalization ability and the recognition accuracy of electroencephalogram emotion recognition are improved.

The invention discloses a Chinese language evaluation method and system based on character sentiment analysis, a medium and equipment, and the method comprises the steps: collecting sentiment labeling data, cultural background features and learning historical data of a user, and extracting sentiment primitive features and a sentiment combination mode through a multi-level sentiment classification model; quantizing the culture background features into culture cognition vectors and performing cross-modal alignment; utilizing a space-time attention mechanism to identify an emotion understanding breaking point and constructing a user cognition map; and finally generating a Chinese language ability report containing a multi-dimensional evaluation result. The method breaks through the quantitative limitation of traditional evaluation on emotion dimensions and cultural connotation, can accurately evaluate the dominant language ability of a learner by establishing a dynamic association model of emotion expression and cultural cognition, can deeply analyze the emotion understanding level and the cultural symbol mastering degree of the learner, and improves the evaluation accuracy of the learner. Accurate data support is provided for personalized language teaching, and scientificity and practicability of Chinese language ability evaluation are remarkably improved.

The invention belongs to the technical field of intelligent emotion recognition, and discloses an aspect-level multi-modal emotion analysis method, system and device and a medium, and the method comprises the steps: respectively obtaining image features and text features based on an image and a text, and selecting aspect word features through a pooling operation; modeling the image-text words and the image-aspect words through an image convolutional neural network to obtain image-text interaction features and image-aspect interaction features; image-text interaction features are used as query vectors, image-aspect interaction features are used as key vectors and value vectors, residual connection and normalization are introduced, interaction features are obtained and fused with aspect word features, and sentiment classification is performed based on the fused features. According to the method, from multi-modal feature extraction, interactive modeling to fusion optimization, a series of operations are progressive layer by layer, comprehensive, accurate and high-quality feature support is provided for sentiment classification, and finally, the accuracy and reliability of aspect-level multi-modal sentiment classification are effectively improved.

A system and method for real-time facial and sentiment detection using a computing system. The system includes a video input module that receives real-time video input from various sources such as webcams, security cameras, and smartphone cameras. The video frames are pre-processed by adjusting the resolution, converting color spaces, and isolating the foreground from the background. A facial detection module employs a convolutional neural network to identify and localize human facial regions within the video frames. Geometric and appearance features are extracted from the localized facial regions by a feature extraction module. A sentiment classification module classifies the extracted features to determine sentiments using a deep learning model. The system also includes a module for API integration, enabling third-party applications to utilize the sentiment recognition results.

System and method for aspect-level sentiment classification. The system includes a computing device, the computing device has a processer and a storage device storing computer executable code. The computer executable code is configured to: receive an aspect term-sentence pair; embed the aspect term-sentence pair; parse the sentence using multiple parsers to obtain dependency trees, and perform edge union to obtain a merged graph; combine the embedding and the merged graph to obtain a relation graph; perform a relation graph neural network on the relation graph; extract hidden representation of the aspect term from updated relation neural network; and classify the aspect term based on the extracted representation to obtain a predicted classification label of the aspect term. During training, the computer executable code is further configured to calculate a loss function based on the predicted label and the ground truth label, and adjust parameters of models.

The invention provides a sentiment classification method based on a visual language model and conditional reasoning, which comprises the following steps of: firstly, acquiring a text picture pair and labeling sentiment labels on the text picture pair to form a sentiment label set; then, a visual language model is used as a strategy model, general reasoning and conditional reasoning are carried out on the text picture pairs respectively, reasoning characterization, emotion prediction labels and conditional reasoning results are generated, and response samples are formed after the reasoning characterization, the emotion prediction labels and the conditional reasoning results are combined; and calculating a reward value and an advantage estimation value based on the response sample to optimize the strategy model, finally utilizing the optimized strategy model to carry out sentiment prediction on a new text picture pair, and outputting a final sentiment classification result. According to the method, the defect that a traditional multi-classification model is easily interfered by noise texts or complex visual contents is overcome, the classification precision is improved, a general reasoning process and a conditional reasoning process of a strategy model are recombined into a group of response samples, it is guaranteed that each group of response contains different classification labels, and the problem of advantage collapse is solved.

The invention provides a multi-modal dialogue emotion recognition method based on emotion memory enhancement, and relates to the technical field of emotion recognition, and the method comprises the steps: extracting the multi-modal features of a dialogue; respectively embedding speaker representation embedding vectors into the multi-modal features of the dialogue to obtain multi-modal sequence features; performing time sequence modeling on the multi-modal sequence features by adopting an extended long and short-term memory network to obtain a multi-modal hidden state sequence; inputting the multi-modal hidden state sequence into a preset memory module to obtain a multi-modal memory sequence; performing cross-modal alignment and fusion on the multi-modal memory sequence to obtain a cross-modal fusion feature sequence; and performing time sequence modeling on the cross-modal fusion feature sequence, and mapping an output sequence after secondary time sequence modeling to obtain a final sentiment classification result of the dialogue. The accuracy of emotion recognition in the prior art is effectively improved.

The invention discloses a speech emotion recognition method and system based on multi-modal feature fusion, and relates to the technical field of speech emotion recognition. The speech emotion recognition method and system based on multi-modal feature fusion comprises the following steps: S1, collecting a speech emotion data set, and carrying out label unified coding and normalization processing; s2, frame-level acoustics and construction of frequency spectrum, rhythm and sound quality emotion features are carried out; s3, a speech emotion representation generation method fusing multi-sub-mode depth coding and a gating cooperative attention mechanism; s4, performing random forest weight initialization and two-order variation grey wolf mapping evaluation; and S5, voice emotion recognition and operation feedback adaptive updating are carried out. According to the method, the feature selection efficiency and the emotion classification accuracy in voice emotion recognition are effectively improved, and the problems that existing voice emotion feature selection is single in stage and single in index, emotion retention and real-time performance are difficult to consider while dimension reduction is performed, and the overall performance is limited are solved.

The invention relates to a multi-modal sentiment analysis method based on text enhancement and modal completion perception fusion, and belongs to the field of natural language processing. The method comprises the following steps: 1, extracting text semantic features by utilizing a pre-training language model, and performing linear transformation on non-text features to form unified multi-modal input representation; 2, injecting a cross-modal enhancement module into the pre-training language model, fusing non-text information by taking a text as a core, and performing multi-modal input representation; 3, introducing a modal completion module, generating a missing modal completion representation through reconstruction loss and random modal discarding, and suppressing noise features in combination with a convolution gating structure; and guiding full-connection network learning modal weight to realize joint optimization of sentiment classification and regression. According to the method provided by the invention, on the premise of keeping the dominance of the text, through cross-modal interaction and dynamic weighting and in combination with a modal completion mechanism, the model has relatively strong emotion recognition capability.

The invention discloses an intelligent teaching assisting method and system integrated with the whole process and total elements of education and teaching, and relates to the field of education and teaching. According to the method, misunderstanding concept classification models are integrated, and a dynamic knowledge graph containing target subject knowledge is established; clustering knowledge concepts in the dynamic knowledge graph by adopting a Mapper algorithm of topological data analysis to obtain a course map; when the user completes interaction of the selected theme cluster, a cognitive state matrix is obtained by adopting a graph knowledge tracking model, an emotion category probability distribution vector is obtained by adopting an emotion classification model, and a learner state vector is obtained; a large language model optimized through process supervision and reinforcement learning is adopted as an inference engine; and outputting a targeted teaching strategy and teaching content by using an inference engine according to the learner state vector. According to the method, a personalized teaching environment which can perform smooth and dynamic natural language interaction and can accurately diagnose and effectively correct the specific deep-level cognition mistake of students in a target subject can be created.

The invention relates to the technical field of artificial intelligence and data processing, and discloses a game experience sentiment classification method based on Transform and stacked ensemble learning, which comprises the following steps: constructing a user game behavior map, and obtaining a game situation and user behavior information by adopting map neural network coding; based on a game context awareness neural collaborative network, fusing the information with the original text content, and outputting initial emotion probability distribution; outputting a time sequence correction result and predicting uncertainty by combining a plurality of probability distributions and time sequence characteristics through a time sequence evolution perception integration mechanism; triggering a personalized processing flow according to a comparison result of the predicted uncertainty and a preset threshold value; and finally, generating a final sentiment classification result, and feeding back and updating the user game behavior map to form a closed loop. According to the method, game context and time sequence dynamics are combined, the consistency of sentiment classification results is improved, the adaptability of the method in processing complex samples is enhanced, and self-adaptive optimization of the model is achieved.

The invention discloses a multi-modal sentiment analysis method combining dynamic sentiment knowledge and a sparse attention mechanism. The method comprises the following steps: acquiring and preprocessing text, image and audio multi-modal training samples; extracting preliminary features of each modal by using the pre-training model; emotion feature expression is enhanced through an emotion related feature extraction module; the enhanced features are input into a Transform Encoder, and an emotion knowledge vector is dynamically generated; guiding multi-modal feature semantic alignment by using the vector; reserving the first k maximum values of the attention score by adopting a sparse attention mechanism, and realizing effective fusion of modal features; and finally, outputting an emotion analysis result through the emotion classifier. According to the method, a multi-task joint loss function is designed, and end-to-end optimization is carried out in combination with classification loss, alignment loss and comparison loss. According to the method, the problems of difficulty in modal alignment, much fusion redundant information and insufficient cross-domain adaptability in multi-modal sentiment analysis are effectively solved, and the accuracy and generalization ability of sentiment analysis are remarkably improved.

The invention discloses an emotion analysis method based on a multi-modal large model. The method comprises the following steps: S1, extracting multi-modal emotion features; respectively designing special emotional feature extractors for three modes of facial expression, voice and text; s2, carrying out cross-modal emotion alignment and fusion; mapping the emotion features of different modes to a unified emotion semantic space; s3, an emotion inconsistency detection mechanism; the method is specially used for detecting the emotion inconsistency phenomenon between different modes. S4, fine-grained sentiment classification is carried out; the sentiment classifier comprises three sub-tasks of basic sentiment classification, complex sentiment recognition and sentiment intensity regression; s5, a model fine tuning training strategy; and optimizing the performance of the model by adopting a multi-stage fine-tuning training strategy. According to the method, deep fusion and accurate analysis of facial expressions, voice acoustic features and text semantic information are realized, so that a complex emotional state and an emotional inconsistency phenomenon are effectively recognized.

The invention discloses a multi-modal sentiment classification method based on multi-view interaction representation, and relates to the technical field of multi-modal information processing, and the method comprises the steps: collecting and preprocessing the voice, facial expression and text data of a user, and generating a multi-modal data packet; based on the multi-modal data packet, performing feature extraction by adopting a hierarchical attention mechanism to obtain an aligned voice expression text feature sequence, and generating a multi-modal feature flow; performing sentiment classification on the sentiment benchmark result by adopting a time sequence gating network to obtain a service state vector and a conflict view vector; according to the multi-view fusion emotion feature sequence, a time sequence gating network and a multi-mode recognition model are adopted for recognition, and a matched emotion classification result and a natural language reply are generated in combination with the service state. According to the method, sentiment classification is carried out on the sentiment benchmark result by adopting the time sequence gating network, the service state vector and the conflict view vector are obtained, and synchronous description of the sentiment state and the context is realized.

The embodiment of the invention provides a voice emotion recognition method and device, electronic equipment, a readable storage medium and a computer program product, and relates to the technical field of computers. The method comprises the following steps: framing a voice signal to obtain a plurality of audio frames; determining a frame-level amplitude representation value of the audio frame; based on the frame-level amplitude representation values of the plurality of audio frames, amplitude change relative values of adjacent audio frames are determined to construct an inter-frame amplitude feature sequence, and the inter-frame amplitude feature sequence comprises a plurality of amplitude change relative values arranged according to the time sequence of the audio frames; and inputting the audio inter-frame amplitude feature sequence into a sentiment classification model, and recognizing a sentiment classification result of the voice signal. According to the method, the amplitude change relative values of the adjacent audio frames are determined based on the frame-level amplitude representation values of the multiple audio frames, the amplitude change relative values of the adjacent audio frames are dimensionless relative variations, and the method has high robustness and high sensitivity to different recording environments and tiny emotion changes.

The invention provides a natural speech emotion synthesis and recognition method and system fused with deep learning, and relates to the technical field of speech processing, and the method comprises the steps: obtaining to-be-processed speech data and corresponding text content, extracting acoustic feature representation and semantic feature representation, building edge connection between a time sequence frame node of an acoustic feature and a semantic unit node of a semantic feature, constructing a bidirectional emotion association graph, calculating an edge weight, and propagating and updating based on graph convolution operation to obtain fusion feature representation; inputting the fusion features into an emotion classifier to obtain an emotion state identifier, calculating an emotion target area mask according to edge connection weight distribution, and generating an emotion regulation and control parameter; and performing speech synthesis based on the emotion regulation and control parameters and performing consistency verification to obtain synthesized speech and emotion deviation feedback information. According to the method, deep fusion of acoustics and semantics is realized through the bidirectional emotion association graph, and the emotion recognition accuracy and the emotion expressive force of speech synthesis are improved.

The invention relates to a multi-modal sentiment analysis method based on a large language model and quantum computing, and belongs to the technical field of multi-modal feature alignment and fusion. The multi-modal sentiment analysis method comprises the steps of constructing a multi-modal sentiment analysis data set, extracting text features and multi-granularity image sentiment description features by using the large language model and a target detection technology, carrying out multi-level fusion on the image emotion description and the original text to obtain cross-modal potential emotion association information; designing a cross-modal feature fusion mechanism based on a parameterized quantum circuit, and performing efficient fusion on multi-modal features by using quantum superposition and quantum entanglement features to generate unified emotion feature representation; performing sentiment classification based on the sentiment feature representation, and outputting a sentiment analysis result; according to the method, cross-modal potential emotion association can be effectively captured, the fusion efficiency and expression ability of multi-modal features are improved, the method is remarkably superior to the prior art in emotion analysis tasks, and the robustness and accuracy are high.

The invention discloses a multi-modal sentiment analysis method based on global-local multi-level fusion, and the method comprises the steps: S1, defining a multi-modal sentiment analysis task, and carrying out the single-modal feature coding, so as to accurately recognize and classify the sentiment state through the fusion of the information of two modes of text and image; s2, obtaining enhanced modal representation based on global-local multi-level fusion; s3, label comparison learning training and data comparison learning training are carried out; s4, performing joint optimization of the target based on the joint loss function so as to improve the overall performance of the model; and outputting a joint optimization result to an emotion classification result. According to the method, dual targets of fine-grained emotion information extraction and multi-modal information fusion are effectively realized.

The invention relates to the technical field of data processing, in particular to a multi-modal feature alignment semantic fusion method based on deep learning, which comprises the following steps of: acquiring a multi-modal data fragment through an event triggering acquisition mechanism, generating a cross-modal time sequence association confidence coefficient matrix by adopting a multi-granularity time sequence modeling network, and performing semantic fusion on the cross-modal time sequence association confidence coefficient matrix; a bidirectional iteration alignment module is used for realizing fine alignment of feature sequences, adaptive semantic fusion is performed through a dynamic cross-modal Transform architecture, emotional state probability distribution is output in combination with an online emotion classification and delay prediction mechanism, and finally a system parameter optimization closed loop is constructed. According to the method, the complex interaction problem of the multi-modal data in the aspects of feature distribution heterogeneity, time sequence asynchronism and semantic gap is effectively solved, the emotion calculation precision is improved, and the response delay of the mental health service is reduced.

The invention provides a few-lead physiological signal emotion recognition method based on prototype feature learning, which comprises the following key steps: carrying out band-pass filtering and wavelet denoising processing on an original EEG signal and an original ECG signal, extracting a differential entropy DE feature from the EEG signal, and extracting a heart rate variability HRV feature from the ECG signal; based on training set sample feature data, performing time window alignment and splicing processing on the DE features of the extracted EEG signals and the HRV features of the extracted ECG signals, and performing feature fusion by using a multi-head self-attention mechanism; learning prototype features of each emotion category; training set sample feature data is used as input of paired learning, and the similarity between samples is learned by defining a loss function; paired learning is carried out on test set sample feature data, the similarity between sample features is calculated, and sentiment classification is achieved. According to the method, high accuracy is achieved on a few-lead physiological signal emotion recognition task, and the method is remarkably superior to a traditional machine learning method and a deep learning method.

The invention provides an emotional voice interaction method and device based on a humanoid robot, and relates to the field of robot interaction. The method comprises the following steps: acquiring voice data of a target user through a humanoid robot, and extracting acoustic features and semantic features in the voice data; inputting the acoustic features and the semantic features into a cross-modal fusion emotion classification model, and outputting an emotion recognition result corresponding to the target user; determining an emotion recognition misjudgment risk, and correcting an emotion recognition result based on the emotion recognition misjudgment risk; inputting the corrected emotion recognition result and the semantic feature into an emotion voice understanding interaction model, and outputting an emotion voice interaction result; and controlling the humanoid robot to execute a corresponding multi-modal interaction response based on the emotion voice interaction result. The problem that the voice interaction experience is greatly reduced due to the fact that the voice interaction content output by an existing humanoid robot is difficult to fit the real emotional state of the elderly user in multiple aspects such as text expression and voice emotion is solved.