Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

1812 results about "Emotion identification" patented technology

Emotion recognition method based on large model and related device

The invention relates to the field of data processing, in particular to an emotion recognition method based on a large model and a related device. The method comprises the following steps: firstly, acquiring to-be-processed multi-modal data; the multi-modal data at least comprises text information, audio information and image information input by a user; the image information at least comprises a facial expression image and / or a limb action image. And inputting the multi-modal data into an emotion recognition model for emotion state analysis to obtain a predicted emotion state corresponding to the user. And finally, performing confidence evaluation based on the predicted emotional state, and generating a user emotion recognition result based on the predicted emotional state and a corresponding evaluation result. According to the method, richer emotion details are captured from the multi-modal data through the emotion recognition model, cross-modal emotion intelligent recognition is realized, the accuracy of emotion recognition is further improved by combining confidence evaluation, the emotion recognition efficiency is improved, and the application range of emotion recognition is expanded.
Owner:BEIJING LIXIN INTELLIGENT TECH CO LTD

Psychological accompanying dialogue method based on multi-agent collaboration and storage medium

The invention relates to the field of natural language understanding in an artificial intelligence technology man-machine dialogue system, in particular to a psychological accompanying dialogue method based on multi-agent collaboration and a storage medium. The psychological accompanying dialogue with individuation, high naturalness and accurate emotion recognition is provided, the user experience is effectively improved, and the psychological support requirement of the user is met. According to the main scheme, the method comprises the steps of performing multi-mode emotion evaluation on user input to obtain user emotion state description; retrieving related dialogue states and external knowledge from a shared memory module according to user questions and user emotional state description; according to the retrieved dialogue state and external knowledge, the strategy planning module generates a reply strategy to obtain a strategy group; the dialogue content generation module calls a corresponding vertical domain agent according to the generated reply strategy to generate specific reply content; and storing the user questions and model replies in the current round into the short-term memory of the shared memory module, and updating the long-term memory. The method is used for AI dialogues.
Owner:GUANGDONG DIGITAL IND INTELLIGENT TECH CO LTD

Online teaching optimization method and system based on emotion recognition

The invention relates to the technical field of online teaching management, and discloses an online teaching optimization method and system based on emotion recognition, and the method comprises the steps: collecting the real-time emotion data of a learner through a multi-modal sensor, carrying out the fusion of a graph convolutional neural network to generate a feature vector, and optimizing a teaching strategy parameter through a meta-reinforcement learning model, the dynamic course generation model combines an improved genetic algorithm and a knowledge graph to optimize a teaching content sequence, and the hierarchical teaching control model realizes knowledge path planning, interactive adjustment and learning state evaluation, and generates a teaching control instruction. According to the method, personalized online teaching optimization is realized, the state of a learner can be accurately grasped, the teaching interaction mode is optimized, the course content is dynamically adjusted, the learning effect is comprehensively and accurately evaluated, the problems that traditional online teaching lacks personalization, the state of the learner is difficult to grasp and the like are effectively solved, the online teaching quality and the learning experience are improved, and the learning experience is improved. And the online education development is promoted.
Owner:XUECHENG CENTURY BEIJING INFORMATION TECHCO

Comprehensive regulation and control method and system based on multi-mode electromagnetic and ultrasonic stimulation

The invention discloses a comprehensive regulation and control method and system based on multi-mode electromagnetic and ultrasonic stimulation, and the method comprises the steps: collecting brain wave signals in real time through a multi-electrode EEG system, synchronously collecting skin electroreaction and heart rate variability signals, preprocessing the collected signals, and carrying out the feature extraction; the extracted signal features are subjected to multi-modal feature fusion through a CNN, and the multi-modal features are subjected to time sequence dependency analysis through a long-short-term memory network to generate a real-time emotional state quantization vector and identify a current emotional state category. Through synchronous acquisition and feature fusion of multi-electrode EEG, EDA and HRV signals, a real-time brain function connection network is constructed, accurate recognition and stimulation strategy making are performed on emotion regulation and control nodes based on graph theory analysis, the accuracy and reliability of emotion recognition are improved, personalized regulation and control of individual emotion states are realized, and the emotion regulation and control efficiency is improved. The cognitive function and language expression ability of an individual can be improved, and movement coordination disorder caused by emotional fluctuation is reduced.
Owner:BEIJING TIANTAN HOSPITAL AFFILIATED TO CAPITAL MEDICAL UNIV

Electroencephalogram and synchronous physiological signal emotion recognition method based on cross-modal comparative learning and multi-scale representation

The invention relates to the technical field of emotion recognition, and discloses an electroencephalogram and synchronous physiological signal emotion recognition method based on cross-modal comparative learning and multi-scale characterization, in a pre-training stage, features are extracted from physiological signals of different modalities through encoders corresponding to the modalities, and modal characterization related to emotion stimulation is learned; the modal representations are mapped to a potential space through a mapping layer to calculate the similarity, then the contrast loss of each modal is calculated, and contrast learning in the modal is carried out; performing cross-modal comparative learning on the representation based on each modal extracted by an encoder, and aligning the representation of different modals to a shared potential space; according to the invention, the encoder is pre-trained at different time scales by using short-term and long-term emotion changes in each mode; in the fine tuning stage, emotion recognition training is carried out by using a pre-trained encoder, an initialized modal fusion module and a classifier, and the model weight is optimized based on the emotion label of the physiological signal.
Owner:HEFEI UNIV OF TECH

System platform for psychological assessment and emotion feedback

The invention provides a system platform for psychological assessment and emotion feedback. The system platform comprises a multi-source data acquisition module, an emotion data analysis module, a psychological state assessment module, an intervention scheme generation module, an emotion change feedback module, a scheme optimization module and a data security protection module. The emotion recognition accuracy is improved by synchronously acquiring physiological-behavior-environment data and combining with a dynamic feature fusion method, a personalized intervention strategy is generated based on a reinforcement learning algorithm, real-time feedback is realized, and a data security protection system is constructed by adopting federal learning and homomorphic encryption technologies, so that the data security is ensured.
Owner:AIR FORCE MEDICAL CENT PLA

User emotion recognition and psychological intervention system and method based on large language model

Aiming at the problems of insufficient language understanding depth, weak personalized dialogue generation ability, lack of continuous learning and long-term user state modeling and the like in the current emotion recognition and psychological intervention technology, the invention provides a user emotion recognition and psychological intervention method combined with a large language model (LLM). According to the method, the potential emotional state is identified by analyzing free text information input by a user by utilizing the powerful capabilities of a large language model in the aspects of natural language understanding, emotional modeling and text generation; constructing a multi-round dialogue context, and reasoning a psychological change trend of the user; in combination with a psychological knowledge base, personalized and mild psychological intervention dialogue content with a dredging effect is generated. The system supports recognition and classification of various emotional states such as depression, anxiety and alonity, is suitable for various interaction scenes (such as APPs, webpages and social robots), and can greatly improve the precision of emotion recognition and the timeliness and effectiveness of psychological intervention. The emotion recognition and psychological intervention method based on the large language model provides solid technical support for constructing an intelligent, continuous and personalized psychological health management system, and has wide application prospects and profound social significance.
Owner:CHANGCHUN UNIV OF TECH

EEG emotion recognition system and method based on multi-scale space-time diagram convolution and comparative learning

The invention relates to the field of deep learning and emotion recognition, in particular to an EEG emotion recognition system and method based on multi-scale space-time diagram convolution and comparative learning, and the method comprises the steps: carrying out the preprocessing of an EEG signal; constructing positive and negative sample pairs based on the emotion categories and the identities of the subjects; extracting hidden space representation of the electroencephalogram signal by adopting a stack auto-encoder, and dividing the electroencephalogram signal into a plurality of functional brain regions to obtain a brain source signal after dimension reduction; constructing an undirected graph structure of the brain source signal; constructing a multi-scale space-time diagram convolutional network as a feature extraction network, inputting an undirected graph structure, dynamically learning and updating an adjacent matrix, expressing features of brain source signals in the undirected graph structure, and extracting emotion related features; based on a multi-scale space-time diagram convolution and comparative learning framework, an EEG emotion recognition model is built and trained to eliminate individual differences. According to the method, the multi-dimensional information of the EEG signals is fully utilized, and the accuracy, robustness and generalization performance of emotion recognition are remarkably improved.
Owner:SOUTH CHINA UNIV OF TECH

Real-time emotion recognition and regulation method and device combined with brain-computer interface

The invention discloses a real-time emotion recognition, regulation and control method and device combined with a brain-computer interface. The method comprises the following steps that S1, electroencephalogram signals of a user are collected in real time through an electroencephalogram sensor; s2, preprocessing the original electroencephalogram data; s3, processing the electroencephalogram signals in the S2 by adopting a time-frequency analysis technology, and extracting time-frequency characteristics related to the emotional state; s4, inputting the extracted time-frequency features into a trained long-short-term memory network model, performing time sequence analysis, and identifying the emotional state of the user; s5, according to the emotion state recognized in the step S4, combining historical emotion data and individual features of the user, and dynamically adjusting an emotion regulation strategy; and S6, comprehensively evaluating the emotional state of the user through a multi-modal data fusion technology. By combining the advanced time-frequency analysis technology, the long-short term memory network model and the multi-modal data fusion technology, accurate recognition and dynamic regulation and control of the emotional state of the user are achieved.
Owner:HUNAN VOCATIONAL COLLEGE OF LABOR & PERSONNEL

Dynamic calibration method and system of vehicle-mounted emotion recognition system

The invention provides a dynamic calibration method and system for a vehicle-mounted emotion recognition system, and the method comprises the steps: S1, obtaining multi-source data which comprises a facial image, a voice signal and a physiological signal; the obtained multi-source data are preprocessed, and preprocessed multi-source data are obtained; s2, performing feature extraction based on the preprocessed facial image, the voice signal and the physiological signal to obtain a facial expression feature vector, an audio feature vector and a physiological state feature vector; s3, evaluating the current environment credibility based on an environment credibility evaluation function; s4, dynamically distributing the weight of the multi-source data according to the credibility of the current environment and the real-time scene; and S5, constructing a multi-modal fusion vector based on the dynamically distributed weight of the multi-source data, the facial expression feature vector, the audio feature vector and the physiological state feature vector, and performing emotion recognition by using the constructed emotion recognition model based on the multi-modal fusion vector.
Owner:SHANGHAI PUFAFEN ELECTRONIC TECH CO LTD

Multi-modal signal fusion emotion recognition method based on attention mechanism

The invention discloses a multi-modal signal fusion emotion recognition method based on an attention mechanism. The method comprises the following steps: firstly, preprocessing physiological signal data; constructing a channel attention module for the multi-channel electroencephalogram data; respectively extracting EEG and other physiological signal features by using the EEGNet; a shared-private encoder is introduced to decouple shared features among the multiple modes and private features of each mode; and finally, providing a cross-modal cross attention fusion mechanism to realize multi-modal feature interaction and effective fusion. According to the method, multi-modal signals are combined, the limitation of a single mode is overcome, and a sharing-private feature separation mechanism and a cross-modal cross fusion mechanism are introduced, so that modal collaborative modeling and effective integration are realized, and the accuracy and generalization ability of emotion recognition are improved.
Owner:HOHAI UNIV

Multi-mode-based dog training method and system for intelligently correcting pet behaviors

A dog training method for intelligently correcting pet behaviors based on multi-modality comprises the steps that multi-modality perception data of a pet is obtained, the multi-modality perception data comprises acoustic signals, motion signals and track data, an acoustic feature tensor, a motion feature tensor and a position feature tensor are constructed, and a unified coupling tensor is formed; inputting the coupling tensor into a pre-trained behavior recognition model, and outputting a behavior tag; the emotion recognition module outputs an emotion label, and updates a character label in a sliding window mode; inputting the behavior label, the emotion label and the character label into an intervention strategy generation model and outputting an adaptive intervention strategy, wherein the intervention strategy comprises a pacifying type strategy and a stimulation type strategy; according to a historical feedback effect record, adjusting a feedback intensity parameter of the intervention strategy, calculating a feedback convergence index and an emotion recovery index, and dynamically correcting a feedback level and a delay strategy of the intervention strategy.
Owner:SHENZHEN TIZE TECH CO LTD

AI digital human interaction system based on emotion recognition

The invention discloses an AI digital human interaction system based on emotion recognition, relates to the technical field of emotion calculation, and improves the accuracy of emotion recognition through multi-modal emotion data acquisition and processing of an emotion primary recognition module and a secondary recognition module. The system effectively distinguishes complex emotions, such as fatigue and depression, of the trainee in the virtual education task by calculating the first emotion coefficient and the second emotion coefficient; secondary emotion recognition is carried out, so that misjudgment is reduced, and the emotion state recognition accuracy is ensured; through an emotion feedback module, the system dynamically adjusts the expression, intonation and interaction frequency of the AI digital human according to a second emotion coefficient, so that the emotion feedback is more natural, and the emotion recognition sense of the student is enhanced; besides, the culture adaptation analysis module collects cross-culture emotion data based on the culture background of the trainee, calculates and evaluates a culture adaptation coefficient, and automatically adjusts an emotion recognition and feedback strategy, thereby improving the adaptability of the system in a multi-culture environment and the accuracy of emotion expression.
Owner:ICE CHAN (NANTONG) INTELLIGENT TECHNOLOGY CO LTD

Self-learning multi-modal emotion recognition method based on multi-scale cavity attention

The invention provides a self-learning multi-modal emotion recognition method based on multi-scale cavity attention, and solves the problem of low recognition precision caused by different importance of basic action units of a face and different distances between key action units, and the problem of different modality confidence during decision-level fusion. The method comprises the following steps: preprocessing a facial expression image, inputting the facial expression image into a multi-scale cavity attention convolution module, extracting features through a parallel three-branch convolution structure, splicing the features, calibrating through an attention mechanism to obtain an enhanced feature map, and sending the enhanced feature map to a full connection layer to recognize emotion; an original electroencephalogram signal is input into a time-frequency-space three-dimensional feature extraction network, the signal is decomposed, differential entropy features are calculated and processed by a global attention module comprising a frequency spectrum attention module, a space attention module and a time attention module, time-frequency-space multi-dimensional feature representation is output, and emotions are recognized by a full connection layer; and finally, inputting the emotion recognition result of the facial expression and the electroencephalogram signal into a self-learning weight module, and obtaining a final emotion recognition result through dynamic weighted fusion.
Owner:DALIAN UNIV

Deep learning-based pet dog emotion recognition method and system

The invention relates to the technical field of pet emotion recognition, in particular to a pet dog emotion recognition method and system based on deep learning. Comprising the steps of collecting pet dynamic data, pet physiological data and scene data to obtain a structured data set; labeling the structured data set through a cross validation labeling mechanism to obtain a labeled data set; multi-modal features are extracted based on the labeled data set, and multi-modal feature integration is carried out through a cascade SEblock array to obtain multi-modal fusion features; adversarial sample data generated by a stress scene simulator is injected in a training stage, and deep learning model training is performed based on the adversarial sample data and the multi-modal fusion features to obtain a pet emotion recognition model; and performing emotion recognition on a to-be-recognized pet through the pet emotion recognition model to obtain a pet emotion recognition result. According to the method, the data quality and the model robustness of pet emotion recognition are improved, and then the human-pet interaction quality is improved.
Owner:HANGZHOU AXO BIOTECHNOLOGY CO LTD

Emotion recognition method based on space-time multi-scale attention convolutional neural network

The invention discloses an emotion recognition method based on a space-time multi-scale attention convolutional neural network, and the method comprises the steps: collecting electroencephalogram data of a subject, and carrying out the preprocessing of the electroencephalogram data, so as to obtain electroencephalogram (EEG) data containing a space dimension and a time dimension; constructing a lightweight convolutional neural network comprising a double-flow spatial-temporal feature construction layer, a mixed attention mechanism layer, a high-order fusion layer and a classification layer; wherein the double-flow spatial-temporal feature construction layer comprises a temporal feature extraction module and a parallel spatial feature extraction module; the mixed attention mechanism layer combines a channel attention mechanism, a space attention mechanism and a self-attention mechanism; the high-order fusion layer is used for carrying out re-learning from the learned global convolution kernel to the representation of the local hemisphere convolution kernel; and recognizing the EEG data by using the trained lightweight convolutional neural network to obtain an emotion recognition result of the subject. The precision and efficiency of emotion recognition driven by electroencephalogram signals are improved by constructing a lightweight model with a small number of parameters.
Owner:SHIHEZI UNIVERSITY

Multi-mode XR emotion interaction method, system and device and storage medium

InactiveCN119251438ARealize precise perceptionRealize active regulationBiological modelsImage data processingPersonalizationMood
The invention discloses a multi-modal XR emotion interaction method, system and device and a storage medium. The method comprises the steps of collecting multi-modal data of a user through a multi-modal perception interface of an XR device; preprocessing the collected multi-modal data, and performing emotion recognition on the preprocessed multi-modal data by adopting a deep neural network to obtain an emotion recognition result; constructing an emotion knowledge base, mapping the emotion recognition result into the emotion knowledge base, and reasoning and analyzing the current emotion; and automatically classifying the emotion according to the personalized emotion model of the user, and intervening and regulating the emotion of the user according to the current emotion of the user, the personalized emotion model and an emotion regulation reinforcement learning algorithm. Omnibearing, accurate, dynamic and personalized perception and quantitative analysis of the emotional state of the user are realized, and the emotional interaction experience in an XR scene can be remarkably improved.
Owner:SHENZHEN YUANXIANG VISION TECH GRP CO LTD

Vehicle-mounted emotion interaction method and device based on multi-dimensional recognition

The embodiment of the invention provides a vehicle-mounted emotion interaction method and device based on multi-dimensional recognition, and the method and device achieve the precise judgment of the emotion of a driver through innovatively constructing an emotion fusion recognition model and integrating the facial expression, voice emotion, driving behavior and physiological state features. And designing a scene-based self-adaptive interaction strategy, and establishing an interaction triggering threshold value for intelligent matching in combination with external environment data and a danger level. An interaction effect evaluation mechanism is introduced, an interaction strategy model is continuously optimized through an online learning module, and dynamic adjustment of personalized interaction content is achieved. According to the method, the defects of the traditional technology in the aspects of emotion recognition, interaction strategies, effect evaluation and the like are effectively overcome, and the intelligent level and the user experience of vehicle-mounted emotion interaction are remarkably improved.
Owner:SHENZHEN ZHI HUI LIN NETWORK TECH CO LTD

Electroencephalogram emotion recognition method and system based on deep neural network

The invention relates to the technical field of electroencephalogram signal processing, and discloses an electroencephalogram emotion recognition method and system based on a deep neural network. The method comprises the following steps: collecting and preprocessing a multi-channel EEG signal; constructing a graph data structure, extracting multi-domain features by taking electroencephalogram channels as nodes, and constructing a self-adaptive dynamic adjacency matrix; constructing a graph convolution long and short-term memory network, learning spatial features by GNN, and extracting time dependence by LSTM; enhancing emotion capture by using a multi-scale time-frequency feature fusion method in combination with STF and CWT; constructing global topological information of an FCN brain extraction region in combination with brain network features; and outputting alertness and other emotion indexes by means of the classification model. According to the method, graph structure learning and time sequence modeling are combined, EEG signal emotion recognition is optimized, and personalized adaptation and emotion recognition accuracy is improved.
Owner:NANCHANG UNIV +1

Facial expression and context-based academic emotion recognition method and device

The invention discloses an academic emotion recognition method and device based on facial expressions and contexts. According to the method, accurate emotion modeling is realized by utilizing a visual language pre-training model CLIP and combining student facial expressions in a video sequence and contextual information of a learning situation. Specifically, according to the method, firstly, collaborative modeling is carried out on facial expressions and context information in a learning scene, so that the ability of recognizing the learning state (such as focusing on learning and distracting) of students is remarkably improved; and secondly, the strong language and visual feature alignment capability of the CLIP pre-training model is utilized, so that the dependence on large-scale video training data is effectively reduced, and the generalization capability and practicability of the model are improved. An accurate and feasible solution is provided for academic emotion recognition, and the method is suitable for monitoring and analysis of academic emotions of students in an education scene.
Owner:ZHEJIANG GONGSHANG UNIVERSITY

Emotion recognition method based on millimeter wave radar and robot

The invention relates to an emotion recognition method based on a millimeter-wave radar and a robot. The method comprises the following steps: receiving a transmitting signal of the millimeter-wave radar and a corresponding echo signal, converting the transmitting signal and the corresponding echo signal into a point cloud format, and removing static reflection to obtain dynamic point cloud data and a timestamp of a surrounding environment; grouping the dynamic point cloud data into clusters; obtaining a centroid estimation value of the target, and tracking the centroid estimation value; sending the millimeter wave signal which horizontally moves at a constant speed to a target person, and receiving echo signals which are reflected by the target person and are uniformly spaced; point spread function optimization is carried out, and SAR image reconstruction is carried out by using a fast frequency domain near-field imaging method; calculating to obtain phase change of the human body signal, and separating to obtain respiration and heartbeat signals of the target person; and extracting emotion related features to obtain the current emotion of the target person. Compared with the prior art, the specific position of the human body can be accurately recognized, and non-contact privacy protection emotion recognition can be remotely achieved.
Owner:SOUTHEAST UNIV

Cognitive disorder man-machine interaction method and system based on emotion analysis

The invention discloses a cognitive impairment man-machine interaction method and system based on emotion analysis, and relates to the field of cognitive impairment man-machine interaction, and the method comprises the steps: collecting the physiological information, language information and expression information of a user in real time, carrying out the feature fusion, forming a time series data set, and carrying out the emotion recognition and classification; based on a user emotion recognition classification result, establishing a generative adversarial network to generate man-machine interaction dialogue content matched with the current context and emotion in real time; when it is detected that the emotion recognition result of the user is negative emotion, a cognitive disorder emotion pacifying strategy is retrieved based on big data, emotion pacifying is conducted on the user, and related data are recorded; by analyzing historical related data of negative emotions of a user, an emotion prediction model is constructed, possible emotion changes are recognized in advance, and a man-machine interaction pacifying strategy is adjusted.
Owner:THE SECOND HOSPITAL OF HEBEI MEDICAL UNIV

Pet pacifying method

The invention discloses a pet pacifying method, and relates to the technical field of emotion recognition, and the pet pacifying method comprises the following steps: obtaining pet behavior data, and inputting the pet behavior data into a pet emotion recognition model to recognize the current emotion type of a pet; according to the current emotion type of the pet, determining a pacifying strategy corresponding to the current emotion type; and pacifying the pet based on the pacifying strategy. According to the method, the emotion of the pet is recognized from multiple dimensions, the accuracy of pet emotion recognition is improved, whether the pet is in an anger, happy or calm state is judged, corresponding strategies are selected for pacifying according to different emotional states of the pet, and the anxiety, tension and other emotions of the pet can be relieved more effectively.
Owner:SHENZHEN SHUJU TECHNOLOGY CO LTD

Method and apparatus for text sentiment recognition, computer storage medium, and electronic device

The present application relates to the field of natural language processing or other related fields, and discloses a method and apparatus for text sentiment recognition, a computer storage medium, and an electronic device. The method comprises: obtaining text to be recognized, wherein said text at least comprises an attribute word and an emotion word, and the attribute word refers to a subject that the emotion word describes; and inputting said text to a sentiment task model for processing to obtain a sentiment recognition result, wherein the sentiment task model is configured to determine an attribute category to which the attribute word belongs and an emotion category to which the emotion word belongs, determine all permutation and combination results of the attribute word, the emotion word, the emotion category, and the attribute category, and determine a target permutation and combination result among all the permutation and combination results as the sentiment recognition result. The present application solves problems in the related art that accuracy of recognition results of text sentiment recognition is low and content of the recognition results is incomplete.
Owner:CHINA TELECOM CORP LTD

Pet emotion recognition system and method

The invention discloses a pet emotion recognition system and method, and relates to the technical field of pet emotion recognition, and the system comprises a data acquisition module, a data analysis module, an environment analysis module, an emotion analysis module, an emotion comparison module, and an abnormality processing module. The data acquisition module is used for acquiring classified pet historical emotion data, pet emotion data acquired in real time and environment data acquired in real time, and then sorting the classified pet historical emotion data; the technical key points are as follows: analyzing and processing historical emotion data of the pet, identifying the emotion of the pet from two perspectives of sound and physiology, knowing the relationship between the emotion of the pet and the sound and physiology, and then identifying the emotion of the pet in real time based on the relationship between the emotion of the pet and the sound and physiology. The external environment and the state of the pet are analyzed in the pet emotion recognition process, the pet emotion recognition accuracy can be improved, and the good use prospect is achieved.
Owner:HANGZHOU AXO BIOTECHNOLOGY CO LTD

Affective haptic regulation method based on multimodal fusion

Disclosed are an affective haptic regulation system and method based on multimodal fusion, including a haptic optimal parameter adjustment module, a haptic generation module, a visual-auditory generation module, a multi-physiological signal acquisition module, a multi-sensory signal acquisition module, and a multimodal fusion emotion recognition module. The system can fuse multi-physiological signal features with audio and haptic modal features by acquiring a plurality of physiological signals of a user, accurately identify a current affective state of the user in real time through advanced data processing and analysis technology, seek for a haptic parameter with the help of an optimization theory, and achieve proactive regulation of affective state of the user; and the system can overcome the limitations of traditional subjective scale methods, effectively reduce the influence of unstable physiological signals on emotion recognition results, and significantly improve the accuracy of affective detection in the affective haptic regulation system.
Owner:SOUTHEAST UNIV

Multi-modal emotion recognition method based on zero-knowledge machine learning

The invention relates to a multi-modal emotion recognition method based on zero-knowledge machine learning, and belongs to the crossing field of artificial intelligence and information security. Key features of local face images, voice signals and posture data multi-modal data are extracted through an attention module, then the key features are transmitted to a server after homomorphic encryption, training of a convolutional neural network (CNN) model is carried out in an encryption domain, and finally a result is sent back to the local for decryption and emotion recognition. The method has the advantages that by using the homomorphic encryption technology, in the whole data processing and transmission process, privacy data of a user is always kept in an encrypted state, and due to fusion of multi-modal features, the emotion recognition accuracy is improved, the robustness of the system in a complex environment is enhanced, the emotion recognition result is more stable and reliable, and the user experience is improved. The method can be widely applied to multiple fields of intelligent monitoring systems, user sentiment analysis, human-computer interaction, mental health monitoring and the like.
Owner:JILIN UNIVERSITY

Service content generation method and device based on emotion recognition, equipment and medium

The invention relates to the technical field of artificial intelligence and the field of financial science and technology, and discloses a service content generation method based on emotion recognition, and the method comprises the steps: collecting voice data, extracting voiceprint features, and building a user voiceprint file; extracting Mel frequency cepstrum coefficient features and Mel energy difference coefficient features, and inputting the extracted features into an emotion classification model for emotion classification recognition to obtain an emotion state of the user; the current emotional state is compared with a historical normal emotional state, emotional fluctuation is analyzed, and whether the emotional state of the user is abnormal or not is judged; and when it is detected that the emotional state is abnormal, target service content is determined from the personalized service library in combination with the emotional state and the user voiceprint file. According to the method, the voice features are extracted and the user voiceprint file is established, so that the emotion fluctuation of the user can be accurately analyzed, whether abnormity exists or not is judged, accurate recommendation service is provided in combination with the personalized features and historical behavior data of the user, and the intelligent and personalized degree of vehicle-mounted voice interaction is improved.
Owner:PING AN TECH (SHENZHEN) CO LTD

Emotion detection system based on facial recognition

The invention discloses an emotion detection system based on facial recognition, and relates to the technical field of computer vision and emotion calculation. A video stream time sequence analysis module is used for extracting a facial micro-expression image sequence of continuous frames, a time sequence feature vector containing a micro-expression intensity gradient, an illumination robustness coefficient and a facial action unit cooperation feature is generated, and a multi-mode dynamic sensing module is combined to carry out real-time analysis on an emotion classification probability, voice emotion parameters and physiological signals. And the fusion decision module performs dynamic weighted fusion on the multi-modal data based on the scene adaptive weight, and finally generates a comprehensive emotion score. Through multi-modal time sequence modeling and a dynamic weight optimization mechanism, the accuracy and environmental adaptability of emotion recognition are remarkably improved, and real-time perception and accurate decision making of customer emotion are realized in a target scene.
Owner:NORTHEAST FORESTRY UNIV

Emotion recognition method and device based on artificial intelligence

The embodiment of the invention provides an emotion recognition method and device based on artificial intelligence, and identity verification is realized through voiceprint features by creatively integrating multi-modal data of facial images, voices and body actions. And a multi-level attention fusion network is designed, and intelligent fusion of modal interior and cross-modal features is realized by using a feature attention layer and a modal attention layer. An emotion change time sequence model is constructed in combination with medical record data, the physiological indexes are fused for evaluation and correction, and dynamic evaluation and accurate monitoring of the emotional state are achieved. According to the method, the defects of the traditional technology in the aspects of multi-modal fusion, time sequence modeling, medical monitoring and the like are effectively overcome, and the accuracy and the practical value of emotion recognition in the medical scene are remarkably improved.
Owner:SHENZHEN ZHI HUI LIN NETWORK TECH CO LTD