708 results about "Emotion classification" patented technology

The invention discloses a Transform-based cross-modal fusion multi-modal emotion recognition method and device, which are used for solving the problems of modal isomerism, difficulty in time alignment and insufficient dynamic emotion modeling in a multi-modal emotion recognition task, and the method takes the accuracy and robustness of emotion recognition as performance evaluation indexes. Firstly, feature information of three modes of vision, voice and text is obtained, feature extraction is performed on each mode through a deep learning model, then features of different modes are fused by using a cross-mode Transform module, and a complex dependency relationship between the modes is dynamically modeled through a multi-head self-attention mechanism, so that more accurate emotion recognition is realized, and the emotion recognition efficiency is improved. And finally, performing emotion prediction on the fused features based on time sequence modeling and an emotion classification module. According to the method, the problems of modal isomerism, difficulty in time alignment and insufficient dynamic emotion modeling in multi-modal emotion recognition can be effectively solved.

This invention relates to a gait emotion recognition method, system, storage medium, and computer device based on spatiotemporal graph convolution. The method includes the following steps: S1, data augmentation by reversing the temporal direction of gait; S2, obtaining deep emotion features and prior emotion features respectively through a spatiotemporal graph convolutional network and prior feature statistical methods; S3, performing nonlinear mapping on the prior emotion features using a feature mapping layer; S4, inputting the fused features of the deep emotion features and prior emotion features into an emotion classifier to obtain the emotion category. The feature mapping layer of this invention achieves more effective feature fusion by performing nonlinear mapping on prior features; it also introduces causal temporal convolution to replace general temporal convolution, effectively extracting fine-grained temporal features by enhancing temporal correlation and cross-period feature fusion. Furthermore, a walking direction recognition auxiliary task is designed to accelerate the training and convergence speed of the model, enhancing the ability to extract temporal-dependent features and the performance of emotion recognition.

The invention belongs to but not limited to the technical field of emotion recognition, and discloses a multi-modal classroom emotion recognition method and system based on modal adaptive learning, and the method comprises the steps: building a multi-modal data collection system, collecting the multi-modal data, such as texts, audios and videos, in a classroom teaching process in real time, and carrying out the real-time collection of the multi-modal data; and preprocessing and feature extraction are carried out on data of different modes. For emotion expression feature differences of different classroom scenes, a multi-modal data fusion method based on adaptive modal scoring (AMS) is provided, and a time sequence modeling and emotion classification model is constructed. The multi-modal fusion and time sequence modeling mechanism simulates the multi-dimensional perception of the teacher on the emotional states of the students in the class, and enhances the interpretability of the emotion recognition system. According to the system, weighted fusion is carried out on data features from texts, audios and videos through fusion of multi-modal data, and complete emotion feature representation is formed.

A system for sentiment analysis in language texts based on deep learning, consisting of: a preprocessing unit configured to receive and normalize raw text input from various sources, with the unit performing tokenization, punctuation removal, text stemming, and standardization to prepare text data for analysis; an embedding unit that is operationally connected to the preprocessing unit and is configured to convert normalized text data into numerical representations, wherein the embedding unit also contains pre-trained language embeddings to capture semantic relationships in the text and dynamically selects embeddings based on the source and linguistic context of the text data; a sentiment analysis neural network (SANNet) operationally connected to the embedding unit, comprising layers of a convolutional neural network (CNN) for local pattern recognition and layers of a recurrent neural network (RNN) configured to capture sequential dependencies within text data, with the SANNet also including an attention mechanism to dynamically weight word relevance based on contextual meaning, and the SANNet being trained to interpret and classify complex sentiment patterns in the text; A context module integrated into the neural network for mood analysis. The module includes a bidirectional transformer layer for the simultaneous processing of forward and backward word dependencies, enabling the system to capture complex language structures, including sarcasm, irony, and mixed emotions, and thus perform an accurate mood classification; an output interpretation unit that is operationally connected to the neural network for sentiment analysis and is configured to convert probability-based classification results into human-readable sentiment categories with associated confidence levels, wherein the output interpretation unit also provides interface compatibility for external applications, thus enabling direct integration of sentiment classifications; A continuous training module connected to the neural network for sentiment analysis. The module is configured to use real-time user feedback and sentiment accuracy metrics to retrain the SANNet, thus ensuring adaptive learning for evolving language patterns and domain-specific contexts; a GPU / TPU-optimized processing card that is operationally connected to the neural network for sentiment analysis and is configured to accelerate deep learning calculations within the system, thereby enabling high-throughput, real-time sentiment analysis.

The invention discloses a multi-modal sentiment analysis and interactive adjustment method and system based on a large model, and relates to the technical field of artificial intelligence and natural language processing. Comprising the steps of 1, collecting dialogue video data and preprocessing the video data, 2, respectively carrying out image feature extraction, voice feature extraction and text feature extraction according to video frames, audios and texts, 3, carrying out interactive fusion on every two modal features through a cross attention mechanism, and step 4, generating a corresponding dialogue strategy according to the emotion label in the recognition result and the requirement of appointed emotion adjustment, and carrying out interaction according to the dialogue strategy, so as to obtain the emotion recognition result of the multi-layer perceptron after the feature after the interaction fusion is subjected to average pooling processing and then is input into the multi-layer perceptron to carry out emotion classification, and obtaining a classification corresponding to the highest probability as the emotion label recognition result. Dynamically adjusting a dialogue strategy according to the change of the emotion label; the method is suitable for enhancing the emotion understanding and response ability of the intelligent dialogue system and improving the user experience.

Methods and apparatus are described for evaluating dubbing of media content. Emotions are identified based on combinations of attributes determined for segments of a source language audio and a dubbed audio. The emotions may be compared to determine emotional prosody transfer between the source audio and dubbed audio. Based on the comparison, a notification is generated indicating whether an emotion classification associated with the source audio matches an emotion classification associated with the dubbed audio.

The invention discloses a feature attention and bilinear gating fused speech emotion recognition method and device, and the method comprises the following steps: 1, collecting an audio file, obtaining corresponding label information, generating audio waveform and time frequency representation data through preprocessing, and constructing an audio waveform mask and a time frequency mask to mark an effective information region; 2, constructing a dual-path feature extraction module which comprises a time-frequency feature extraction module and a pre-training acoustic feature coding module; wherein the time-frequency feature extraction module models emotion correlation through local convolution and a multi-dimensional attention mechanism, and performs global time sequence modeling based on a bidirectional gated loop network; the pre-training acoustic feature coding module extracts high-level speech representation with high expression ability for emotion distinguishing by using a pre-training model; and step 3, constructing a feature fusion module and an emotion classification module, and combining with a dual-path feature extraction module to form a speech emotion recognition model.

The invention discloses an emotion recognition method based on EEG space-time frequency feature fusion, which comprises the following steps: S1, carrying out a series of preprocessing steps on an original EEG signal, including baseline correction, standardization and time window addition; s2, performing form conversion on the electroencephalogram signals processed in the S1 to obtain spatial information and time-frequency information of the electroencephalogram signals; s3, multi-aspect feature extraction is carried out; s4, fusing the extracted emotion features, and performing multi-feature emotion recognition; s5, further extracting time features of the electroencephalogram signals by using BILSTM added with a channel attention mechanism, and performing deep fusion of the features; and S6, obtaining an emotion classification result. According to the emotion recognition method based on EEG space-time-frequency feature fusion, feature information of the time dimension, the frequency dimension and the space dimension of the EEG signals is fully considered, the highest recognition accuracy is achieved on the awakening dimension and the titer dimension, the emotion recognition accuracy and robustness are improved, and the effectiveness of the method is verified.

The invention provides a text user sentiment analysis method and system based on multiple modes and AI, and relates to the technical field of text analysis, and the method comprises the steps: obtaining and preprocessing text, image and audio information of a user, and extracting semantic, visual and acoustic feature vectors; multi-modal features are fused through a cross-modal attention mechanism; utilizing a graph neural network to construct a user emotion social graph to calculate emotion propagation intensity; and finally, obtaining an analysis result containing emotion category and intensity through an emotion classifier. The emotion state of the user can be comprehensively captured, the emotion analysis accuracy is improved, and complex emotion expression is effectively recognized.

The invention relates to the technical field of robot perception, in particular to an intelligent accompanying human-type robot based on multi-mode emotion interaction and a perception method, and the method comprises the following steps: collecting facial expression and body posture data of a user through a visual perception unit, collecting intonation, speed, volume and keyword data through a voice perception unit, and carrying out the recognition of the speech, speed, volume and keyword data; data preprocessing and feature extraction are carried out; secondly, carrying out time sequence modeling on the extracted visual and voice features based on a recurrent neural network, and generating visual and voice time sequence feature vectors; the method comprises the steps that firstly, time sequence features are extracted, then, a multi-mode fusion model based on an attention mechanism is adopted, the time sequence features are fused into a comprehensive emotion feature vector used for emotion state classification, and finally, personalized interaction content such as voice, expressions and limb actions matched with the emotion state of a user is generated according to an emotion classification result. According to the method, the accuracy of robot emotion perception and the naturalness of man-machine interaction are remarkably improved, and the method is suitable for multi-scene application of education, medical treatment, old-age care and the like.

The invention relates to the technical field of language translation, in particular to a translation precision optimization method and system based on artificial intelligence, and the method comprises the steps: obtaining a source language text, carrying out the phrasing operation on the source language text to obtain a source language sentence sequence of the source text, carrying out the word segmentation operation on each source language sentence in the source language sentence sequence, and carrying out the word segmentation operation on each source language sentence; obtaining a word sequence of the source language sentence; and performing sentiment classification on each source language sentence in the source language sentence sequence based on a pre-trained first sentiment classification model. The sentiment classification is performed on the source language sentences and the target language sentences on the basis of the first sentiment classification model and the second sentiment classification model, and the similarity between the sentences is calculated, so that the sentences after translation are kept consistent in sentiment, the problem of sentiment deviation generated in a machine translation process is favorably avoided, and the translation efficiency is improved. Especially for some texts with strong emotion colors, it is ensured that translated emotion information can be correctly conveyed.

The invention discloses a multi-modal emotion analysis method and device, and relates to the technical field of emotion analysis. The method comprises the following steps: preprocessing each modal emotion of multi-modal emotion data, performing dimension alignment, and determining a target text feature, an aligned video feature and an aligned audio feature; a single-mode encoder is adopted to encode the aligned video features and the aligned audio features, and target video features and target audio features are correspondingly obtained; performing cross-membrane attention mechanism operation on the basis of the target text features, the target audio features and the target video features through a cross-modal fusion network and a cross-membrane conversion network, inputting the target text features, the target audio features and the target video features into a Transform network, performing self-attention operation, performing feature fusion, and determining emotion fusion features; and performing sentiment classification on the sentiment fusion features by adopting a sentiment classification layer, and outputting a sentiment analysis result. Based on the scheme, information loss during modal interaction is reduced, and the reliability of multi-modal emotion analysis is improved.

The invention discloses an emotion recognition method and system based on cross-modal alignment and matching enhancement. According to the method, firstly, feature extraction is carried out on text, audio and video modalities in a data set, and then a text and audio cross-modal emotion alignment module and a text and video cross-modal emotion alignment module are constructed respectively, so that cross-modal semantic alignment is realized. Constructing an emotion label matching module based on an alignment result, generating modal pairs with similar emotions but different labels by using a difficult negative sample mining strategy, and paying attention to cross-modal emotion consistency through a dichotomy task guide model; performing modal feature fusion on the three modals through a six-layer attention crossing mechanism, finally splicing feature vectors, inputting the spliced feature vectors into a long-sequence context fusion modeling module for deep modal fusion, and capturing cross-modal interaction information; and the fused features are sent to an emotion classification module, and a final emotion category recognition result is output.

The invention belongs to the technical field of natural language processing, and particularly relates to a text sentiment classification method integrated with a novel adversarial strategy and enhanced feature extraction, which comprises the following steps of: carrying out preprocessing and word segmentation operation on a text; constructing a model; the embedding learning layer obtains word embedding vectors of a text through BERT and BiLSTM; the enhanced feature extraction layer performs weighted fusion on a local attention mechanism and a TextCNN model; the classification layer performs emotion polarity classification according to the extracted feature vectors to obtain a final classification result; and carrying out data enhancement by adopting a large-batch adversarial training strategy. According to the method, a local attention mechanism and TextCNN weighting are combined, and emotion features of different granularities in a text are fully mined. The local attention mechanism can identify key information in a text and endow the key information with a higher weight, and the TextCNN can capture local features and a context dependency relationship, so that compared with an existing method, the method has more advantages in the aspect of feature extraction, and text representation with more semantic representativeness is generated.

The invention discloses a video multi-mode sentiment analysis method and device based on multi-layer perceptron fusion, and relates to the technical field of sentiment analysis, and the method comprises the steps: S1, extracting text features, image features and audio features in a video; extracting time sequence information in the image features and the audio features to obtain time sequence image features and time sequence audio features; s2, constructing a video multi-modal sentiment analysis model comprising a multi-modal feature capture module, a multi-layer perceptron fusion module and a sentiment classifier, and constructing a loss function according to modal similarity and modal heterogeneity between modals; s3, training the model; and S4, inputting the text features, the time sequence image features and the time sequence audio features into the trained model to obtain emotion polarity probability distribution. According to the method, similarity loss and heterogeneity loss are constructed, and sequence, channel and modal dimension fusion is carried out by using a multi-layer perceptron, so that the calculation complexity and memory consumption are reduced, and the integrity and discrimination capability of the multi-modal emotion features are improved.

The invention relates to a multi-modal emotion recognition method based on heart and brain coupling and a graph neural network, and belongs to the field of artificial intelligence. Comprising the steps of data preprocessing, graph representation construction, multi-view graph convolutional network construction, fusion graph network construction and cross-domain joint optimization and sentiment classification. The method has the advantages that an adaptive adjacency matrix optimization strategy based on a triple constraint mechanism is proposed to solve the modal alignment and deviation problems represented by a multi-modal diagram in a data-driven branch, redundant noise is eliminated by adopting global regularization constraint, and unique feature representation in a modal is enhanced through modal specificity; a deep association rule is mined in combination with a cross-modal interaction module, the modeling ability of a heart and brain emotional state is improved, a multi-view image convolutional network is further designed, global features and local features are extracted, features of a cognitive heuristic branch and a data driven branch are combined by adopting an attention mechanism-based image fusion network, a domain confrontation strategy is introduced, and a cognitive network is constructed. And the generalization of the method is enhanced.

The invention provides a multi-modal emotion recognition software development method for voice interaction of an intelligent robot, and the method achieves the synchronous collection and fusion of multi-modal data through the weighted fusion preprocessing of voice signals, facial expression video streams and text contents. A DCNN and LSTM combined model is adopted to deeply extract emotional features, and an attention mechanism is introduced to strengthen key information capture. And a comprehensive emotion representation vector is calculated by using a cross-modal emotion association network CMAN, so that the emotion distinguishing capability is improved. An emotion classifier combining an SVM and a neural network is constructed, and refined output of emotion categories and intensity is achieved. Through real-time feedback and adaptive optimization steps, the model can continuously learn and adapt to user emotion expression habits, and accuracy and user satisfaction are improved. According to the invention, more humanized and accurate multi-modal emotion recognition technical support is provided for voice interaction of the intelligent robot.

The invention discloses a multi-modal emotion recognition method based on cross-modal interaction of a state space model, and belongs to the technical field of multi-modal emotion recognition, and the method comprises the steps: firstly, extracting low-level features of voice and text, and extracting S4D features through employing a structured state space sequence model of a diagonal version, the unique sequence modeling capability and the robust feature extraction capability are fully utilized, and the complex time correlation of sound and text data is effectively captured; secondly, in order to fuse the modal features of the sound and the text, a plurality of cross-modal fusion strategies are provided to dynamically learn the weight between the modals; besides, in order to better distinguish different types of sentiments, a triple loss is designed and is combined with a cross entropy loss, so that the accuracy of the model in sentiment classification is improved; finally, an ablation experiment is carried out, and the effectiveness of the method is verified. According to the invention, the performance and generalization ability of the multi-modal emotion recognition system are effectively improved.

The invention discloses a multi-modal dialogue emotion recognition method, and relates to the technical field of multi-modal emotion recognition and human-computer interaction, and the method comprises the steps: obtaining an expression time sequence feature and a posture time sequence feature through a face recognition model and a posture recognition tool; the attention module performs adaptive weighted fusion on the expression and posture features to obtain fused visual modal features; constructing a new expression of the context information, obtaining an emotional expression based on a prompted emotional modeling technology, and extracting a text modal feature through a text encoder; using the data vectorization model to extract modal features of the voice corresponding to the speaker; a jump connection multi-head attention cross-modal fusion method is provided, cross-modal alignment and fusion are carried out on multi-modal modal features, and then emotion recognition is carried out through an emotion classifier module. According to the method, the problems of insufficient key emotion clue recognition and insufficient fusion in traditional multi-modal emotion recognition are effectively solved, and the accuracy and robustness of emotion recognition are improved.

Disclosed are a multi-mode emotion recognition method, a system, an electronic device, and a storage medium. The method includes obtaining a spectrogram of a voice to be recognized and a corresponding text and inputting the spectrogram and the text into a multi-mode emotion recognition model to obtain an emotion recognition result output by the multi-mode emotion recognition model. The multi-mode emotion recognition model is trained based on a sample spectrogram, and a corresponding sample text, and a sample emotion recognition result, and is configured to extract a feature from the spectrogram and the text by a self-attention mechanism to obtain the voice features and the text feature, fuse the text feature and voice feature to obtain a multi-mode fusion feature, and make an emotion classification decision to obtain an emotion recognition result based on the text feature, the voice feature, and the multi-mode fusion feature.

The invention discloses an emotion recognition method, system and device based on multi-modal adaptive fusion and a storage medium, and relates to the technical field of artificial intelligence, and the method comprises the steps: selecting a pre-training model, respectively extracting the original features of an audio and a video, carrying out the preliminary extraction of the audio through a convolution layer, carrying out the multi-module processing of the video, and keeping the time sequence information. Constructing an attention module to generate an attention matrix and interaction features, and adjusting the original features by using the matrix; and inputting the weighted and fused features into a convolutional network to extract advanced time sequence features, performing pooling compression on the advanced time sequence features in a time dimension, splicing audio and video features, and finally sending the spliced audio and video features into a full-connection layer classifier to obtain an emotion classification result. According to the method, the weights of different features can be dynamically adjusted, so that the audio and visual features are effectively fused, the accuracy and robustness of emotion recognition are improved, the weighted recall rate and the unweighted recall rate are remarkably improved, and the method has high calculation efficiency and expandability.

The invention discloses a multi-modal emotion recognition method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring text, video and audio data of a user and respectively performing feature extraction to obtain text features, audio features and facial features; the three features are input into a pre-trained multi-modal emotion recognition model, the multi-modal emotion recognition model comprises a first fusion module, a second fusion module, a third fusion module, a fourth fusion module and a classification module, the audio features and the text features are fused through the first fusion module, and audio text features are obtained; fusing the facial features and the text features by using a second fusion module to obtain facial text features; performing feature enhancement on the text features by using a third fusion module to obtain enhanced text features; fusing the audio text features, the face text features and the enhanced text features by using a fourth fusion module to obtain multi-modal features; and classifying the multi-modal features by using a classification module to obtain a sentiment classification result of the user.

The invention discloses a multi-mode emotion continuous recognition method for medical treatment, belongs to the technical field of artificial intelligence and medical treatment information, and mainly aims to simulate the dynamic change process of emotion by establishing a Neural ODEs framework and overcome the static property and discreteness of emotion modeling in a traditional method. Through a causal inference technology, emotional features are separated from individual-independent physiological differences, and the generalization ability across individuals is improved. A self-supervised learning method is utilized, the synergistic effect between the EEG and the eye movement signal is improved through cross-modal contrast learning, and the emotion recognition precision is enhanced. The calculation complexity is reduced through a dynamic sparse attention mechanism, and meanwhile, focusing is performed on a key time slice in emotion recognition. Through multi-task joint learning, the model learns multiple tasks such as emotion intensity regression and tested identity recognition during emotion classification, and the personalized emotion recognition capability is improved.

The embodiment of the invention provides an incomplete multi-mode emotion recognition method based on a diffusion model and a federal model, and belongs to the technical field of data recognition. The method specifically comprises the steps that a client downloads a global model to the local; the client extracts incomplete multi-modal features through a shallow feature extractor, the missing modal features are input as random noise, the extracted original features are input into a global model for reverse diffusion denoising, and finally the missing modal features are complemented; sending the complemented full-modal features into an emotion classifier to perform an emotion classification task; the central node receives the model parameters uploaded by the clients, evaluates the contribution degree of each client, calculates the reputation of the clients, preferentially determines the final weight of each client according to the reputation of the clients, and carries out aggregation according to the final weight; and the central node updates the global model according to the local model set and the aggregated weight. Through the scheme disclosed by the invention, the recognition accuracy and safety are improved.

The invention discloses a music stave sentiment classification method and system based on multi-level distillation, and belongs to the technical field of music information processing. The method comprises the steps of firstly collecting music stave data and converting the data into stave data vectors, then performing feature extraction by using a long short-term memory network, then constructing a teacher network and a student network for knowledge distillation, and realizing multi-level knowledge transmission through temperature scaling, KL divergence loss and mask feature distillation. And finally, training a lightweight classification model to complete sentiment classification. The knowledge distillation technology is creatively applied to staff sentiment classification, the classification accuracy is effectively improved through an online multi-level distillation mode, and the technical problems that a traditional method lacks semantic information and a self-supervised model is not suitable for sentiment tasks are solved. The method has the main advantages of high classification precision, light model weight, capability of effectively capturing music emotion features and the like.

The invention provides a multi-modal sentiment analysis method based on a diffusion model and self-paced learning. The method comprises the following steps: firstly, dividing a data set into a missing image modal data set and a complete modal data set according to image modal integrity; thirdly, constructing a feature alignment diffusion model, and performing image generation; training the diffusion model by adopting a self-paced learning strategy and a missing image data set; and based on the trained diffusion model, guiding a reverse process through text features to generate feature representation of the missing image. And carrying out weighted fusion on the generated image features and text features by using an attention mechanism, and dynamically adjusting contribution weights of all modalities to generate a complete multi-modal feature representation. And finally, integrating a missing modal completion result and the complete modal features to form a unified multi-modal representation, inputting the unified multi-modal representation into a multi-modal sentiment classification module, and outputting a sentiment classification result. According to the method, the problem of multi-modal sentiment analysis under random missing of image modals is effectively solved, and the generation quality and semantic consistency are improved.

The invention relates to the technical field of artificial intelligence and the field of financial science and technology, and discloses a service strategy adjustment method based on emotion recognition, which comprises the following steps: acquiring voice data in a service session, performing segmentation processing, extracting feature data, inputting the feature data into an emotion classification model to recognize emotion tags, and generating an emotion tag sequence; and further generating an emotional fluctuation graph, analyzing a service anomaly probability, generating a risk early warning signal when the service anomaly probability reaches a preset threshold, and adjusting a dialogue strategy of the service session based on the risk early warning signal. The emotion label is identified through the emotion classification model, and the service abnormity probability is analyzed in combination with the emotion fluctuation graph, so that the risk early warning signal is generated when the service abnormity probability reaches the preset threshold value, the service strategy is adjusted in time, real-time monitoring and service dynamic adjustment of the customer emotion state are realized, the customer satisfaction can be effectively improved, and the customer experience is improved. And economic loss and reputation loss caused by complaints are reduced.

The invention discloses a multi-modal image-text emotion recognition method and system based on a dynamic routing hybrid expert model, and aims to solve the technical problem that the model recognition effect is poor due to the fact that an existing multi-modal image-text emotion recognition method generally adopts a static fusion mechanism. The method comprises the steps that image data and text data are acquired, the image data and the text data are input into a preset multi-mode image-text emotion recognition network, and the preset multi-mode image-text emotion recognition network comprises a target encoder, a dynamic routing hybrid expert model and an emotion classifier; encoding the image data and the text data through a target encoder, and outputting image global features and text global features; performing multi-modal feature fusion on the image global features and the text global features to generate image text fusion features; performing dynamic expert calculation on the image text fusion features by adopting a dynamic routing hybrid expert model, and outputting weighted features; and inputting the weighted features into an emotion classifier to generate a target multi-modal image-text emotion recognition result.

The invention provides an event description text generation method, device and equipment based on video data, which can be applied to the field of computer vision. The method comprises the steps that video scene graph data is constructed based on to-be-processed video data, the video scene graph data comprises object nodes and edge relations, and the object nodes represent character objects in the to-be-processed video data and object objects related to the character objects; processing the to-be-processed video data based on the trained emotion classifier to obtain a target emotion attribute node, the target emotion attribute node representing the emotion attribute of the character object; updating the video scene graph data based on the target emotional attribute node to obtain a video knowledge graph, the video knowledge graph comprising an edge relationship between the object node and the target emotional attribute node; and processing the video knowledge graph by using the trained decoder to obtain a target event description text for describing the behavior of the character object.

The invention particularly relates to a multi-modal natural dialogue context sentiment analysis method based on personalized sentiment feature generation. The method comprises the steps of obtaining a training set; constructing a multi-modal sentiment analysis model for performing feature fusion through a cross-modal attention mechanism based on input visual, sound and text three-modal features, outputting a sentiment classification result, and calculating clustering center features of each sentiment type; generating personalized multi-modal features of the specific speaker by using the initialization information; training the multi-modal emotion analysis model by using the training set, performing parameter fine tuning on the model in the training stage by using the personalized multi-modal features of the specific speaker, and optimizing the emotion recognition ability for the specific speaker; and applying the fine-tuned multi-modal sentiment analysis model to the new dialogue data, and outputting a final sentiment classification result. According to the method, the personalized adaptive process related to the speaker is combined with the universal multi-modal sentiment analysis model, so that the overall recognition efficiency and precision are improved.