1431 results about "Emotionality" patented technology

This invention relates to a gait emotion recognition method, system, storage medium, and computer device based on spatiotemporal graph convolution. The method includes the following steps: S1, data augmentation by reversing the temporal direction of gait; S2, obtaining deep emotion features and prior emotion features respectively through a spatiotemporal graph convolutional network and prior feature statistical methods; S3, performing nonlinear mapping on the prior emotion features using a feature mapping layer; S4, inputting the fused features of the deep emotion features and prior emotion features into an emotion classifier to obtain the emotion category. The feature mapping layer of this invention achieves more effective feature fusion by performing nonlinear mapping on prior features; it also introduces causal temporal convolution to replace general temporal convolution, effectively extracting fine-grained temporal features by enhancing temporal correlation and cross-period feature fusion. Furthermore, a walking direction recognition auxiliary task is designed to accelerate the training and convergence speed of the model, enhancing the ability to extract temporal-dependent features and the performance of emotion recognition.

The invention belongs to the field of artificial intelligence, and provides a dialogue interaction system based on multi-mode emotion perception and knowledge graph dynamic enhancement. A user edge terminal obtains multi-modal data, a lightweight Transform fusion network is adopted, the multi-modal data is converted into a fusion feature vector through cross-modal attention fusion and dynamic weight adjustment, and the fusion feature vector and historical conversations in a preset round are compressed in real time; the cloud service platform inputs the compressed data into DKGE, mining and fusing feature vectors and entity knowledge and emotional relations implied in historical dialogues in real time in the dialogue interaction process to update a dynamic knowledge graph, performing knowledge enhancement processing based on the dynamic knowledge graph, and constructing an initial reply prototype of the current round of interaction of the user; inputting the fusion feature vector and the updated dynamic knowledge graph into a dialogue strategy model, and determining a response strategy and a knowledge calling direction of the current round of dialogue; and the edge terminal generates real-time interaction reply information according to the initial reply prototype, the response strategy and the knowledge calling direction.

Aiming at the problems of insufficient language understanding depth, weak personalized dialogue generation ability, lack of continuous learning and long-term user state modeling and the like in the current emotion recognition and psychological intervention technology, the invention provides a user emotion recognition and psychological intervention method combined with a large language model (LLM). According to the method, the potential emotional state is identified by analyzing free text information input by a user by utilizing the powerful capabilities of a large language model in the aspects of natural language understanding, emotional modeling and text generation; constructing a multi-round dialogue context, and reasoning a psychological change trend of the user; in combination with a psychological knowledge base, personalized and mild psychological intervention dialogue content with a dredging effect is generated. The system supports recognition and classification of various emotional states such as depression, anxiety and alonity, is suitable for various interaction scenes (such as APPs, webpages and social robots), and can greatly improve the precision of emotion recognition and the timeliness and effectiveness of psychological intervention. The emotion recognition and psychological intervention method based on the large language model provides solid technical support for constructing an intelligent, continuous and personalized psychological health management system, and has wide application prospects and profound social significance.

The embodiment of the invention provides a vehicle-mounted emotion interaction method and device based on multi-dimensional recognition, and the method and device achieve the precise judgment of the emotion of a driver through innovatively constructing an emotion fusion recognition model and integrating the facial expression, voice emotion, driving behavior and physiological state features. And designing a scene-based self-adaptive interaction strategy, and establishing an interaction triggering threshold value for intelligent matching in combination with external environment data and a danger level. An interaction effect evaluation mechanism is introduced, an interaction strategy model is continuously optimized through an online learning module, and dynamic adjustment of personalized interaction content is achieved. According to the method, the defects of the traditional technology in the aspects of emotion recognition, interaction strategies, effect evaluation and the like are effectively overcome, and the intelligent level and the user experience of vehicle-mounted emotion interaction are remarkably improved.

A system and method for real-time, identity-free personalization using deformable emotional trait vectors to dynamically adapt digital and voice-based experiences. Each user session is modeled as a behavioral object known as a Vectra, composed of fluidic traits—such as mass, viscosity, temperature, volatility, and texture—that evolve continuously in response to live behavioral, contextual, environmental, and voice-derived signals. These Vectras traverse a dynamically warped emotional space, the Vectraverse, influenced by ambient conditions including time of day, noise level, inventory urgency, and engagement rhythm. Gravitational pull toward predefined emotional goal attractors modulates system behavior, while a goal mutation engine reclassifies session intent when confidence decays or friction spikes. Outputs include tone modulation, content pacing, offer framing, and gamified reward logic—all executed without storing identity, login credentials, or historical profiles. The system supports modular deployment across voice, screen, signage, mobile, and in-room environments, and integrates with large language models, AI agents, and third-party personalization stacks via privacy-safe APIs and federated learning. Designed for zero-ID personalization, the platform enables emotionally intelligent, context-aware engagement across any surface or session.

The embodiment of the invention provides a digital human interaction system and method based on multi-modal emotion recognition, and belongs to the technical field of digital human interaction. The system comprises a multi-modal sensing module used for collecting multi-modal data and preprocessing the multi-modal data to generate a standardized data stream; the cross-modal fusion and emotion recognition module is used for carrying out interactive modeling on the multi-modal features and outputting a current emotion label and emotion intensity; the reaction planning module is used for generating a composite reaction strategy; and the digital human rendering module is used for mapping the composite reaction strategy into control signals corresponding to the voice, the facial expression and the action respectively, and driving a digital human to execute corresponding voice output, facial expression change and limb action through the control signals so as to realize interaction. According to the method, multi-modal data are deeply fused through the cross-modal graph neural network and comparative learning, the weight is dynamically adjusted in combination with the modal confidence, and the emotion recognition accuracy and robustness are improved.

The invention discloses a multi-modal emotion fusion analysis method and system, and the method comprises the steps: carrying out the feature extraction of multi-modal emotion data modal by modal through a feature extraction module, and generating a text original feature, an audio original feature and a visual original feature; performing cross-modal alignment interactive fusion on the original text features, the original audio features and the original visual features based on a unified semantic alignment module, and constructing collaborative fusion features; performing mode and channel double-layer dynamic fusion optimization by adopting a dynamic fusion regulation and control module according to the text original feature, the audio original feature, the visual original feature and the collaborative fusion feature, and determining a unified fusion feature; performing hierarchical residual semantic gating enhancement based on the unified fusion features according to a high-order semantic abstraction module to generate semantic enhancement features; and inputting the semantic enhancement features into an emotion prediction module, and outputting an emotion analysis result. Based on the above scheme, a more stable, accurate and reliable emotion recognition result can be provided.

The invention relates to the technical field of artificial intelligence, in particular to a multi-modal emotion recognition and interaction adjusting system and method based on uncertainty evaluation, and solves the defects of uncertainty processing, robustness of interaction strategies, complementary information mining degree among modals and the like in the man-machine interaction process in the prior art. Depth application finiteness is caused by lack of modeling for feature uncertainty after fusion. The uncertainty of an emotion recognition result is quantified through technologies such as multi-modal feature fusion and Bayesian neural network / model integration, an interaction strategy is dynamically adjusted according to an uncertainty score, emotion clarification or conservative response is triggered in a high-uncertainty scene, the robustness of human-computer interaction and the user experience are improved, and the user experience is improved. The method is suitable for intelligent customer service, government affair consultation, medical inquiry and other scenes with high requirements for emotion interaction accuracy.

The invention discloses a fine emotion control TTS method and device based on a large model, equipment and a medium, relates to the technical field of artificial intelligence, and can generate a voice service with fine emotion expression and improve the user experience in the high-sensitivity emotion interaction fields of banks, financial customer service, insurance consultation, medical hospital guide and the like. Acquiring an emotional feature vector of the input text based on a preset large language model; encoding the emotion feature vector to obtain a voice encoding vector; determining an emotion curve vector of the input text based on a pre-trained neural network model; and generating target voice corresponding to the input text based on the voice coding vector and the emotion curve vector. According to the method, emotion feature vectors are extracted through a large language model, a dynamic emotion curve is generated in combination with a neural network, speech synthesis is cooperatively driven, emotion fineness and continuity are remarkably improved, and the problems of traditional TTS emotion expression roughening and fragmentation are solved.

The invention discloses a multi-mode emotion continuous recognition method for medical treatment, belongs to the technical field of artificial intelligence and medical treatment information, and mainly aims to simulate the dynamic change process of emotion by establishing a Neural ODEs framework and overcome the static property and discreteness of emotion modeling in a traditional method. Through a causal inference technology, emotional features are separated from individual-independent physiological differences, and the generalization ability across individuals is improved. A self-supervised learning method is utilized, the synergistic effect between the EEG and the eye movement signal is improved through cross-modal contrast learning, and the emotion recognition precision is enhanced. The calculation complexity is reduced through a dynamic sparse attention mechanism, and meanwhile, focusing is performed on a key time slice in emotion recognition. Through multi-task joint learning, the model learns multiple tasks such as emotion intensity regression and tested identity recognition during emotion classification, and the personalized emotion recognition capability is improved.

The invention provides an intelligent customer service dynamic intention recognition system based on a semantic analysis model, and the system collects the multi-dimensional data of a user through a data collection module, constructs a user portrait through a feature extraction module, extracts the portrait features, carries out the semantic analysis of multiple rounds of historical dialogue data, and extracts preference features. And constructing a dynamic user-entity association graph to extract GNN node features. Multi-modal data of a user is analyzed through an emotion recognition module, emotion features are recognized, portrait features, preference features and emotion features are fused and analyzed based on an MLP model to obtain a final emotion state, and the portrait features, the preference features, GNN node features and the emotion features are fused through an intention recognition module to obtain a final emotion state. According to the method, dynamic intention analysis is carried out on the basis of the Transform sequence-to-sequence model, the current intention of the user is recognized, the recognition accuracy is high, the dynamically changing intention is adjusted in real time, and more accurate and targeted answers or services are provided for the user.

The invention discloses a multi-scene self-adaptive man-machine interaction system and method based on emotion recognition, relates to the technical field of man-machine interaction, and solves the technical problems of realizing fusion perception of multi-modal emotion features and improving the accuracy of emotion judgment in complex scenes. According to the method, facial, voice and text emotion features are extracted by adopting a multi-modal fusion technology, the limitation of single-modal recognition is solved, an emotion-scene association rule base and a user portrait are constructed, real-time scene classification is combined, accurate mapping of emotions, scenes and demands is realized, one-step interaction of strategies is avoided, and the user experience is improved. Language interaction adaptation is designed from the form, content and style three-dimensional degree, it is ensured that languages are natural and fit scenes, functional response adaptation improves efficiency through priority ranking and execution mode optimization, environment linkage adaptation is combined with user emotion dynamic adjustment directions, collaborative linkage of languages, functions and environments is achieved, and strategy splitting is avoided.

The invention provides a smart home control method, a readable storage medium and a smart home system. The method comprises the steps that character characteristics of target home equipment are set at least according to equipment functions of the target home equipment, the character characteristics comprise character dimensions, and the character dimensions comprise at least one of gentle, cold, humor and fine; according to at least part of the voice data, the behavior data and the indoor environment data of the user, the emotional state of the user is determined, and the behavior data comprises use condition data of the user for the multiple intelligent devices; analyzing the emotional state and the character traits by using a large model, and at least determining an interaction strategy of the target home device, the interaction strategy comprising a response style and response content; and under the condition that the emotional interaction function of the target home equipment is started, at least controlling the target home equipment to interact with the user according to the interaction strategy. According to the invention, the personification and intelligence level of the smart home equipment is improved.

The invention relates to the technical field of intelligent assessment of psychological states, and discloses an intelligent dynamic assessment method for psychological states of teenagers based on multi-modal data. The method comprises the steps that multi-modal psychological data of a target assessment object is acquired, the multi-modal psychological data comprises voice emotion information and expression behavior relations, and dynamic features are extracted from the multi-modal psychological data; performing feature fusion processing on the psychological data by adopting a trained machine learning model to generate a multi-modal feature set containing a mapping relation between modal type identifiers and time sequence codes; based on the multi-modal feature set, performing time sequence correlation analysis on the dynamic features through a state analysis network to obtain a fusion result containing emotion dimension features and cognitive mode features; and inputting a fusion result into an evaluation model for dynamic state deduction processing, and outputting psychological state evaluation elements. According to the invention, comprehensive and dynamic assessment of the psychological state can be realized.

The invention discloses a digital human construction method and device based on a heterogeneous emotion semantic graph and long-sequence emotion modeling, and the method comprises the steps: obtaining multi-modal emotion input data of a text, voice and a visual image, extracting features, and constructing a multi-modal emotion feature set with a timestamp; constructing a heterogeneous emotion semantic graph which comprises user entity nodes, modal feature nodes and emotion concept nodes, modeling a semantic association, state transition and conflict suppression relationship through a multi-type edge structure, and introducing a dynamic evolution and conflict discrimination mechanism; performing time sequence modeling on the emotional state sequence by utilizing a local-global double-layer emotional modeling mechanism, and respectively capturing short-time fluctuation and long-time trend; performing cross-modal fusion on the emotional state and the modal features, and decoding the emotional state and the modal features into behavior parameters for controlling expressions, voices and actions of the digital human; and multi-modal emotion expression of the digital human is driven. Compared with the prior art, the emotion recognition accuracy and expression continuity and naturalness can be effectively improved.

This disclosure provides a method, apparatus, electronic device, and storage medium for animal language conversion, relating to the field of artificial intelligence technology, specifically machine learning, deep learning, and natural language processing. The specific implementation involves: acquiring multimodal data related to the animal, including animal vocal data, animal behavioral data, and animal physical characteristics data; preprocessing the multimodal data to obtain fused multimodal data; identifying the animal's current emotion based on the fused multimodal data to obtain an emotion recognition result; and performing semantic mapping and language translation on the emotion recognition result to convert the animal language into human language, obtaining a language conversion result. This disclosure can accurately identify the animal's current emotional state and convert it into human language, thereby achieving deeper emotional communication and understanding between animals and humans, and improving the accuracy and efficiency of cross-species communication.

The invention discloses an original script-oriented AI autonomous plot structure adaptive generation system, and relates to the technical field of creation assistance, and the system specifically comprises the following modules: a structure extraction analysis module, an emotional role analysis module, a plot inference module, a scene generation optimization module, a conservation target generation module, an adaptive control module, and a constraint punishment module. According to the method, a multi-level narrative structure and a causal relationship graph are constructed to form an emotion vector and a trajectory curve, an optimal causal path is generated based on a graph neural network, a graph convolution / attention mechanism and a graph generation algorithm, and emotion toning and conservation target driven text generation are performed on scene and dialogue levels. Structural consistency and emotional arcs are optimized in real time in combination with self-adaptive control, plot path weighting and rewriting triggering are performed by utilizing a multi-dimensional emotional space and a neural oscillator network, intelligent structured management of a script is realized, and script creation efficiency and quality are improved.

The invention belongs to the technical field of data analysis, and provides a questionnaire-based sentiment analysis method, system and equipment in order to solve the problem of inaccurate user sentiment analysis in the existing questionnaire. Using a pre-trained language model to extract a semantic vector of the topic text, and combining with the co-occurrence frequency of the multiple topic options to generate node-level local features; carrying out dynamic reasoning by adopting an improved graph neural network model, calculating a dynamic attention coefficient based on semantic similarity calculated by a topic text semantic vector and a jump probability between topics, and weighting and aggregating neighbor features, so as to obtain context-associated topic node features; the global emotion is calculated by using the PageRank thought, a final emotion analysis result is obtained, dynamic context association generated due to questionnaire jump logic is effectively captured, and the reliability of grasping the overall emotion venation of the user is improved.

The invention discloses an accompanying robot emotion generation method based on hierarchical Transform sparse quantization. The method comprises the following steps: obtaining a multi-modal context feature vector sequence; constructing a hierarchical Transform neural network, and outputting a first emotion potential vector representation; generating a sparse graph based on the first emotion potential vector representation in combination with a hierarchical Transform neural network, and obtaining a sparse Transform neural network; a sparse Transform neural network with mixed precision is generated; deploying a mixed precision sparse Transform neural network on an edge processor of the accompanying robot to obtain a second emotion potential vector representation; performing gating fusion on the second emotion potential vector representation and the first emotion potential vector representation to generate an emotion response category and emotion expression intensity; and forming a multi-channel emotion expression result. According to the method, higher robustness and expression continuity are shown in a multi-round dialogue and multi-modal fusion scene.

The invention provides a TTS speech synthesis method for emotion perception enhancement under multi-modal fusion driving, and the method comprises the following steps: S1, carrying out the collection and preprocessing of multi-modal data, the multi-modal data comprising text data, speech data and facial expression data; s2, extracting and analyzing emotion features; s3, emotion perception speech synthesis model training; s4, speech synthesis and post-processing; s5, performing model evaluation and optimization; by collecting and analyzing multi-mode data such as texts, voices and facial expressions, emotion features can be captured more comprehensively and accurately, complementary information among different modes is fully mined through application of a multi-mode fusion network and a collaborative attention mechanism, the emotion expression of the synthesized voices is closer to real emotions, and the user experience is improved. And the precision of emotion perception is greatly improved.

The invention relates to the technical field of robot control, and discloses a spiking neural network-based robot control method, which comprises the following steps of: preprocessing collected multi-modal data to obtain an emotion pulse signal; inputting the emotion pulse signal into a pre-constructed emotion pulse neural network, and outputting a comprehensive emotion pulse; inputting the comprehensive emotion pulse into a central pattern generator, and outputting a behavior rhythm; acquiring an environment feedback signal generated by executing the behavior rhythm, and adjusting a connection weight according to the environment feedback signal; optimizing the comprehensive emotion pulse according to the connection weight, and converting the optimized comprehensive emotion pulse into emotion interaction voice information; and adjusting the emotion intensity according to the optimized comprehensive emotion pulse and the multi-modal data. According to the method, data are collected through the multi-mode sensor to generate emotion pulses, the emotion pulses are input into the central mode generator to generate behavior rhythms after SNN processing, weight optimization, emotional speech generation and emotional steady-state control setting are combined with the STDP algorithm, and the efficiency of a robot service scene is improved.

The invention belongs to the technical field of artificial intelligence, and particularly relates to a service-oriented robot-oriented multi-modal emotion recognition method and system, and the method comprises the steps: collecting audio and video stream data of emotion changes of a user, and separating visual and voice data; extracting visual and voice emotion features through a pre-training model, and calculating prediction probability distribution of each mode; constructing a bimodal confidence quantitative model based on the distribution to obtain each modal confidence; and fusing the features by adopting a sectional type dynamic weight distribution strategy so as to identify the emotional state of the user. Visual and voice modes are fused, feature alignment is realized in combination with dynamic time warping, spatial optimization performance is shared and expressed through a confidence model, a dynamic weight strategy and a cross-modal time sequence cooperation module, and the method has high recognition accuracy, high robustness and real-time processing capacity in a complex environment and is suitable for various service scenes.

The invention discloses a digital human interaction method and system based on a large model. The method comprises the following steps: acquiring a multi-mode instruction; generating contextual information based on the basic settings and historical information of the digital human; inputting the multi-mode instruction and the context information into a core model to obtain an initial text response; adjusting the initial text response based on the emotional state and character traits of the digital person to obtain a text reply; generating an expression reply according to the text reply, and controlling digital human output; receiving feedback information input by a user, and generating interaction data; and optimizing model parameters of the core model based on the interaction data. According to the method and the device, the reply which better fits the user demand and the emotional state is generated by acquiring the multi-mode instruction and combining the basic setting and the historical information of the digital human. The digital human continuously accumulates knowledge from the interaction of the user, and the behavior mode and evolution character are optimized, so that the continuous learning and growth of the digital human are realized.

The invention discloses a psychological health analysis method and apparatus, an electronic device and a storage medium, through the application, dynamic psychological assessment is generated through multi-modal data fusion analysis, and personalized scenarized dynamic interaction content is generated based on an assessment result and an interaction history, so that the psychological health analysis efficiency is improved. Meanwhile, continuous learning and correlation analysis are carried out on the psychological state of the user by utilizing a multi-dimensional memory architecture, so that the system can understand the change of the user state and adjust an interaction strategy like a human consultant, and therefore, the technical problems of insufficient emotional distraction and credibility of the user due to the adoption of a standardized reply mode in the prior art can be solved, and the user experience is improved. The technical effects of enhancing the emotion affinity of the system, establishing a continuous and credible interaction relationship and improving the durability of the psychological intervention effect are achieved.

The invention relates to the technical field of psychological health, and discloses a psychological health multi-modal evaluation method, system and device and computer equipment, and the method comprises the steps: obtaining multi-modal data, and carrying out the feature extraction of the multi-modal data, and obtaining multi-modal features; performing emotion-oriented cross-modal attention mechanism analysis on the multi-modal features to respectively obtain a preset-dimension emotion scale vector, an emotion semantic feature, a multi-dimensional emotion feature and an uncertainty quantitative feature; fusing the preset dimension emotion scale vector, the emotion semantic feature, the multi-dimensional emotion feature and the uncertainty quantitative feature to obtain a multi-modal fusion feature; and obtaining a mental health assessment result based on the multi-modal fusion features and a preset multi-task learning framework. Through a cross-modal attention mechanism, deep semantic fusion of five modals of vision, audio, physiology, text and behavior is realized, and intelligent mapping from original multi-modal data to accurate psychological state judgment is realized through a deep learning technology.

The invention, which relates to the field of the intelligent accompanying system, discloses an interactive digital person accompanying system for the old with cognitive impairment, comprising a data acquisition module, an emotion recognition module, a first matching module, a second matching module, a personalized interaction generation module and a reinforcement learning adaptive module. The data acquisition module acquires multi-modal sensing data of a target old person; the emotion recognition module outputs an emotion label and an intensity value through feature extraction and Transform fusion; a first matching module screens corpus fragments according to emotion intensity triggering conditions; the second matching module performs secondary screening based on the retrieval weight; the personalized interaction generation module generates digital twin interaction data through probability distribution sampling; the reinforcement learning adaptive module dynamically adjusts the corpus weight according to the interaction reward value; according to the method, the dual guarantee of emotion guarding and physiological safety is realized, the anxiety emotion of the old with cognitive impairment is remarkably relieved, and the novelty of interaction and high emotion relieving efficiency can be kept.

The invention discloses a causal perception sentiment analysis method and system based on thinking chain reasoning, and belongs to the technical field of computer vision. The method comprises the following steps: acquiring and reading a multi-modal sentiment analysis data set; extracting video features from the video data in the multi-modal sentiment analysis data set, including voice, text and visual modal features; using the training set and the test set to train and verify the causal perception emotion polarity alignment model; inputting the test set into the trained causal perception emotion polarity alignment model to obtain an emotion state prediction result; video features are input into a causal perception emotion polarity alignment model, and emotion clues are extracted through thinking chain prompt and a self-supervision verification mechanism; then performing causal intervention and anti-factual reasoning on each modal feature by using an emotion clue to obtain a causal-related single-modal feature; and finally, obtaining joint feature representation from the causal-related single-mode features through cross-mode interaction by using a multi-mode representation learning method, and predicting an emotional state.

The invention discloses an elderly accompanying robot system based on multi-mode emotion cognition reinforcement learning. The system comprises a robot body, a multi-source data acquisition and feature extraction unit, a multi-data-source dynamic feature fusion unit, an emotional state estimation and modeling unit and a personalized interaction strategy generation unit. The multi-source data acquisition and feature extraction unit is used for extracting voice signals, physiological signals and behavior signals; the multi-data-source dynamic feature fusion unit is used for realizing cross-modal coupling through a learnable attention mechanism; the emotional state estimation and modeling unit is used for constructing a three-dimensional emotional vector space, predicting an emotional state through a mode of fusing time sequence characteristics and reinforcement learning, and converting a multi-modal signal into a quantifiable emotional index; and the personalized interaction strategy generation unit is used for generating a personalized interaction strategy for controlling the robot body.

The invention discloses a cognitive disorder nostalgic audio-video intervention therapy method, which comprises the following steps of: building a scene containing nostalgic music art, classical movies and retro original sound, playing matched audio and visual materials to a patient, and guiding the patient to perform structured intervention activities such as puzzle, theme recall or emotion sharing. Therefore, memory is stimulated, cognition is exercised, and emotion expression is promoted; according to the invention, by integrating three layouts of audios of nostalgic music art, classical movies and retro original sounds with matched visual elements, a multi-scene nostalgic sensory environment is constructed, and auditory and visual channels of a patient can be synchronously stimulated; compared with single auditory or visual stimulation, the compound sensory stimulation can more effectively activate a plurality of brain areas related to memory and emotion in the brain, so that long-term memory of deep sleep is more intensively aroused, and the strength and awakening efficiency of reminiscence stimulation are greatly improved.

The invention relates to the field of multi-modal sentiment analysis, in particular to a multi-modal sentiment analysis method based on task association perception learning. According to the scheme, the method comprises the steps that multi-modal data including text, audio and visual modal data are obtained, feature extraction is conducted on the multi-modal data, a double-branch comparison module is constructed for each kind of modal data, each double-branch comparison module comprises a fusion branch and a comparison branch, and the fusion branch is connected with the comparison branch; input of a fusion branch and a comparison branch of each double-branch comparison module is an extracted text modal feature, an extracted audio modal feature and an extracted visual modal feature; and finally, feature fusion and prediction output are carried out, so that the accuracy and robustness of multi-modal sentiment analysis are improved. The method is suitable for multi-modal sentiment analysis.