Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

31 results about "Microexpression" patented technology

A microexpressionis the innate result of a voluntary and an involuntary emotional response occurring simultaneously and conflicting with one another. This occurs when the amygdala (the emotion center of the brain) responds appropriately to the stimuli that the individual experiences and the individual wishes to conceal this specific emotion. This results in the individual very briefly displaying their true emotions followed by a false emotional reaction. Human emotions are an unconscious bio-psycho-social reaction that derives from the amygdala and they typically last 0.5–4.0 seconds, although a microexpression will typically last less than 1/2 of a second. Unlike regular facial expressions it is either very difficult or virtually impossible to hide microexpression reactions. Microexpressions cannot be controlled as they happen in a fraction of a second, but it is possible to capture someone's expressions with a high speed camera and replay them at much slower speeds. Microexpressions express the seven universal emotions: disgust, anger, fear, sadness, happiness, contempt, and surprise. Nevertheless, in the 1990s, Paul Ekman expanded his list of emotions, including a range of positive and negative emotions not all of which are encoded in facial muscles. These emotions are amusement, embarrassment, anxiety, guilt, pride, relief, contentment, pleasure, and shame.

Cerebral stroke upper limb rehabilitation training system and method based on dynamic reward feedback

PendingCN121601151APhysical therapies and activitiesChiropractic devicesMicroexpressionEmotional arousal
The embodiment of the invention discloses a cerebral apoplexy upper limb rehabilitation training system and method based on dynamic reward feedback. A data acquisition module is used for acquiring physiological signals, motion signals, emotional state data and training performance data; the motion intention decoding module is used for performing motion intention feature extraction and reliability evaluation on the physiological signals according to an improved MREE-Net + + fusion algorithm, performing weighted fusion after feature weights are adjusted according to a reliability result, obtaining a unified motion intention feature vector, performing motion intention decoding, and obtaining a motion intention intensity index and a motion intention vector; the dynamic reward decision module is used for processing the facial micro-expression data according to an improved VGG-Face model to obtain an emotion titer, and obtaining an emotion awakening degree according to the voice signal; a reward action is generated according to an improved deep reinforcement learning algorithm; and the adaptive training regulation and control module is used for generating a personalized virtual training scene according to the generative adversarial network and regulating and controlling the training intensity according to the fatigue index. The rehabilitation training effect can be improved.
Owner:SHANGHAI SECOND REHABILITATION HOSPITAL (SHANGHAI BAOSHAN NO 1 STEEL HOSPITAL)

Method and device for realizing fast micro-expression recognition processing based on bidirectional optical flow, processor and computer readable storage medium thereof

ActiveCN117456578BImage enhancementImage analysisMicroexpressionOptical flow
The present application relates to a kind of based on two-way optical flow to realize the method for fast micro-expression recognition processing, comprising the following steps: according to visual system acquisition tester face micro-expression video segment information;Extract the emotional video segment in emotional memory library, and the facial muscle movement situation of micro-expression in emotional video segment is captured by positive and negative two-way optical flow;Extract method extracts key frame in emotional video segment, and the redundant frame in continuous sequence image is eliminated;Call the optical flow information between key frame in optical flow information memory library.The present application also relates to a kind of two-way optical flow to realize fast micro-expression recognition device, processor and storage medium.The method for fast micro-expression recognition processing based on two-way optical flow of the present application, device, processor and its computer readable storage medium, the facial muscle movement situation of micro-expression is captured by positive and negative two-way optical flow, the micro-expression of tester is identified using muscle movement trend, and the micro-expression recognition accuracy is improved.
Owner:SHANGHAI UNIV

Emotion analysis method and system for children based on eye movement and micro-expression association

The application belongs to the technical field of biomedical signal processing and computer vision, and specifically discloses a child emotion analysis method and system based on eye movement and micro-expression association, which comprises the following steps: synchronously acquiring an eye movement data stream and a facial video stream, triggering a dynamic analysis process by identifying a mode conflict between high physiological arousal and low emotional expression intensity after generating a physiological arousal and emotion hypothesis through preliminary analysis, generating a guide strategy according to a conflict degree, and enhancing original data in a time domain or a space domain to facilitate deep analysis, capturing suppressed micro-expression signals in the enhanced data to correct the preliminary emotion hypothesis, and finally fusing a physiological arousal level, a corrected emotion category and a quantified expression inhibition degree to output a multi-dimensional psychological state analysis result containing an emotion category, a physiological arousal level and an expression inhibition degree, so as to realize quantitative and multi-dimensional evaluation of real emotions of children.
Owner:NANTONG UNIV

Deep learning model training method and system for in-vehicle user micro-expression recognition

The invention provides a deep learning model training method and system for in-vehicle user micro-expression recognition, and the method and system achieve the precise recognition of the emotion of a vehicle owner through the fusion of facial expressions and voice features. In the model construction process, a self-attention mechanism is introduced, and the mechanism can enhance the capture ability of the model to micro expressions and subtle voice changes, and effectively improve the accuracy of emotion classification. And meanwhile, a localized data processing and edge computing technology is adopted, so that all emotion recognition and in-vehicle environment regulation operations are locally completed on the vehicle-mounted terminal. According to the method, the real-time requirement can be met, the privacy risk caused by uploading the user data to the cloud end is avoided, the personalized requirements of the vehicle owner can be learned based on the emotion change of the vehicle owner in combination with the historical behavior data of the vehicle owner, and then the settings such as customized in-vehicle environment adjustment, automatic adjustment of the air conditioner temperature, the air speed and seat heating of thousands of people are achieved, and the user experience is improved. And the driving comfort and safety are obviously improved.
Owner:TONGLING PFAFEN ELECTRONIC TECHNOLOGY CO LTD

Deep learning-based eating micro-expression and food satisfaction correlation analysis method and application

PendingCN122244925ADigital data information retrievalCharacter and pattern recognitionFood preferenceMicroexpression
This invention relates to the field of health data analysis technology, and particularly to a method and application for analyzing the correlation between eating micro-expressions and food satisfaction based on deep learning. The method includes the following steps: S1, capturing facial video images of users during the eating process using a camera device, and extracting eating facial feature sequences from them; S2, separating chewing actions and facial muscle movements from the eating facial feature sequences based on the spatial displacement differences of facial key points, and obtaining target micro-expression feature vectors. In this invention, by cross-evaluating the captured eating micro-expression emotional state with the user's physiological health indicators, nutritional threshold constraints and weight corrections are applied to the initial food preferences generated based on emotions. This allows for strict control of the intake of core risk nutrients while catering to the user's personal taste satisfaction, ultimately generating personalized recommended recipes that balance emotional experience and medical health standards.
Owner:YUNNAN AGRICULTURAL UNIVERSITY

Depression early screening and risk assessment platform based on artificial intelligence

The invention relates to the technical field of personal health risk assessment, in particular to a depression early screening and risk assessment platform based on artificial intelligence, and the platform can achieve the following steps through the mutual cooperation of a plurality of modules: obtaining a facial image and an audio of a patient to be detected in each historical observation period, obtaining a face image in each historical question and answer period and a reply delay corresponding to each question and answer question; performing mouth corner raising change analysis according to the face image in the historical observation period; performing non-positive emotion analysis processing according to the facial image in the historical observation period; performing response function analysis processing according to the audio in the historical observation period; determining a micro-expression sudden increase coefficient corresponding to the historical question and answer period; and determining a depression risk assessment auxiliary value corresponding to the to-be-detected patient. According to the method, the depression risk assessment auxiliary value is quantified relatively objectively by analyzing the facial image, the audio and the like of the patient, and the accuracy of the quantification of the depression risk assessment auxiliary value is improved.
Owner:XIAN CENT HOSPITAL

A method for simulating facial expressions in an interactive humanoid robot based on artificial intelligence

ActiveCN121093995BHumanoid robot naoMicroexpression
This invention discloses an artificial intelligence-based method for simulating facial expressions in an interactive humanoid robot, belonging to the field of facial expression simulation technology. The method includes: acquiring and preprocessing user brainwave signals to generate power spectral density features; inputting the power spectral density features into a pre-trained emotion classification model to output the user's emotion category and intensity value; mapping the emotion category and intensity value to an emotional expression and outputting biomimetic muscle contraction commands; adjusting the facial muscle fibers of the humanoid robot according to the biomimetic muscle contraction commands to dynamically adjust the surface texture details of the humanoid robot and generate simulated micro-expressions; capturing user facial feedback data through camera eyes, evaluating the user's acceptance of the simulated micro-expressions based on the user's facial feedback data, and outputting an expression optimization strategy. This invention achieves bio-realistic micro-expressions in a humanoid robot through a dual closed-loop modeling approach of deformation matching and user feedback-driven expression optimization.
Owner:BEIJING YUNSHANGHUI INFORMATION TECH CO LTD

Question and answer emotion evaluation method and device based on action unit au and micro-expression

The application discloses a kind of based on action unit AU and micro-expression question and answer mood evaluation method and device, obtain the question and answer video of user to be detected, the first image about non-basic question and answer in question and answer video is input into face detection model and obtains first face image, first face image is input into the multi-task identification model based on AU and micro-expression, output multiple first AU scores and multiple first micro-expression scores of first face image. By the corresponding relationship between multiple AU, multiple micro-expression and multiple emotional parameters, multiple first AU scores and multiple first micro-expression scores are weighted to obtain multiple target emotional values corresponding to multiple emotional parameters. Compare multiple target emotional values with multiple basic emotional values corresponding to multiple emotional parameters to obtain comparison result. The question and answer mood of user to be detected is evaluated by comparison result. In this way, the analysis standard of question and answer mood is set, and the fairness and credibility of evaluating question and answer mood are improved.
Owner:太保科技有限公司

Air rifle athlete training emotion monitoring method based on phase entanglement cooperation

The invention provides an air rifle athlete training emotion monitoring method based on phase entanglement collaboration, which is used for realizing deep fusion of context perception on the basis of deep spatio-temporal features extracted from electroencephalogram signals and facial micro-expressions by a phase entanglement collaboration network and a micro-expression recognition model. Therefore, the complex emotional state of the air rifle athlete can be synchronously and accurately recognized in a high-pressure and fast-changing training environment, and then high-precision data support is provided for monitoring and intervention of the emotional state of the air rifle athlete.
Owner:WENHUA UNIV

Emotion recognition method and system based on multi-modal biological characteristics

PendingCN121808672AExpress emotions naturallyEye movement characteristics leadMicroexpressionSpeech sound
The invention discloses an emotion recognition method and system based on multi-modal biological characteristics, and relates to the technical field of emotion recognition, and the system comprises a multi-modal module, a calculation module, a dynamic module and a summary module. A coupling unit on the summarizing module also calculates a first coupling degree between the eye movement features and the micro-expression features and a second coupling degree between the eye movement features and the voice features through the eye movement feature fluctuation degree, the micro-expression feature fluctuation degree, the voice feature fluctuation degree and a coupling formula respectively, and when the value of the first coupling degree is closer to 1, the second coupling degree is closer to 1; the micro-expression features and the eye movement features are more synchronous, the micro-expression features and the eye movement features of the tested person are closer to natural physiological reactions, and the micro-expression features are not deliberately controlled; when the numerical value of the second coupling degree is closer to 1, the voice features and the eye movement features are more synchronous, further, the voice features and the eye movement features of the testee are closer to natural physiological reactions, and the voice features are not deliberately controlled.
Owner:FOURTH MILITARY MEDICAL UNIVERSITY

Determining pain level and pain source from microexpressions using machine learning

PendingCN122341311APhysical medicine and rehabilitationMicroexpression
An apparatus includes a facial expression detection device and a computing device, the computing device being configured to receive facial pattern input from the facial expression detector, identify one or more micro-expressions in the facial pattern input, and predict one or more of pain intensity and pain source based on the identified one or more micro-expressions. Various aspects of the disclosed embodiments aim to identify pain sources non-contactly based on pain-induced micro-expressions using machine learning.
Owner:HUAWEI TECH CO LTD

Psychological risk early warning method and system for micro-expression image sequence

PendingCN121237418AMedical data miningHealth-index calculationMicroexpressionMedicine
The invention discloses a psychological risk early warning method and system for a micro-expression image sequence, and relates to the technical field of computer vision and psychological health monitoring, and the method comprises the steps: extracting a continuous face image sequence from a video stream, and calculating an eyebrow displacement feature and a mouth corner deformation feature through face key point tracking; comparing with a preset threshold value to mark an eyebrow event and a mouth corner event; counting the occurrence frequency of the event in a preset monitoring window, and calculating an event coupling coefficient according to the time relevance of the event; performing weighted fusion on the interbrow event frequency, the mouth corner event frequency and the coupling coefficient to obtain a basic risk value; if the basic risk value is smaller than a preset risk threshold value, outputting a low-risk early warning signal; if yes, entropy value characteristics are calculated according to the probability distribution of the event types, and a high-risk or medium-risk early warning signal is judged to be output according to the entropy value characteristics; according to the method, the emotion change of an individual under the synergistic effect of multiple micro expressions can be judged, and personalized psychological risk early warning is provided.
Owner:SHIHEZI UNIVERSITY

A cross-domain facial emotion recognition method based on cue learning

PendingCN122135420ASemantic analysisBiological modelsPattern recognitionContextual cueing
This invention discloses a cross-domain emotion recognition method based on cue learning, belonging to the field of multimodal emotion recognition technology. In the feature alignment stage, this invention introduces a spatial-channel collaborative attention module within the CLIP multimodal framework to enhance the capture of micro-expression features in low-light blurred regions, achieving effective alignment of emotional features in the visible and low-light domains. In the recognition and inference stage, this method employs a semantically guided contextual cue learning approach, fully utilizing the knowledge of the pre-trained CLIP model to construct a text representation more adapted to the target domain features for each emotion category. A visual-text dual-path collaborative optimization framework is designed, achieving effective alignment and robust recognition of emotional features in the visible and low-light domains through cross-modal contrastive learning and domain adversarial loss. This invention significantly improves the model's generalization ability and recognition performance in cross-illumination domain scenarios.
Owner:ANHUI UNIV OF SCI & TECH

Three-dimensional digital human expression generation system based on voice emotion decoupling

PendingCN121837455ABiological modelsAnimationMicroexpressionAnimation
The invention discloses a three-dimensional digital human expression generation system based on voice emotion decoupling, and belongs to the technical field of computer graphics and artificial intelligence. The method comprises the following steps of: firstly, decoupling semantic contents and emotion styles from voice through a double-flow encoder, and ensuring characteristic orthogonality by utilizing cyclic consistency constraint; secondly, constructing a multi-modal physical feature enhancement branch, extracting Mel-frequency spectrum, fundamental frequency (F0) and energy (Entry) features of the speech in parallel, and after encoding, injecting the features into a content space to supplement high-frequency acoustic details; then, generating a basic action by utilizing a Transform decoder which introduces dual position coding; and finally, explicitly injecting emotional features in a residual form through a rear emotional attention fusion module. According to the method, the problems of high-definition digital population type smoothness, detail loss and emotion expression dilution in a deep network in the prior art are effectively solved, the three-dimensional facial animation with accurate mouth shape, rich micro expressions and high emotion explosive power can be generated, and the method is suitable for scenes such as meta universe, virtual live broadcast and movie and television production.
Owner:DALIAN NATIONALITIES UNIVERSITY +1

Facial micro-expression pain recognition and analysis method based on deep learning

The invention discloses a deep learning-based facial micro-expression pain recognition and analysis method. The method comprises the following steps of S1, constructing an original data set; s2, preprocessing the obtained original data set; s3, based on the first-layer generative adversarial network and the preprocessed original data set, generating facial micro-expression images under different pain levels through a first-layer generator; s4, based on the second-layer generative adversarial network, extracting pain level features from the micro-expression image generated by the first-layer generative adversarial network and the real micro-expression data; s5, processing the extracted micro-expression features and pain grade scores through a fuzzy logic module; s6, based on a fuzzy logic reasoning result, performing defuzzification processing on the pain level; and S7, according to the real-time analysis result, continuously monitoring the pain state of the patient, and adjusting the sensitivity of the system to the micro-expression under different environmental conditions. According to the method, the facial micro-expression capture precision and the pain level identification accuracy in a complex environment are improved.
Owner:THE SECOND AFFILIATED HOSPITAL TO NANCHANG UNIV

Micro-expression monitoring method and system based on action feature recognition

PendingCN121686413ACharacter and pattern recognitionDriver/operatorMicroexpression
The invention discloses a micro-expression monitoring method and system based on action feature recognition, and relates to the technical field of micro-expression monitoring methods, and the method comprises the steps: determining a plurality of sub-expression features based on the recognition of a facial expression image of a driver; the micro-expression dynamic graph of the driver is determined according to the multiple sub-expression features, the trunk action combination and the primary driving event of the driver, the micro-expression monitoring area of the driver is determined based on the micro-expression dynamic graph of the driver and the head action combination, and the micro-expression combination accuracy of the driver is improved. Therefore, a micro-expression combination is determined based on a plurality of micro-expression key features and a primary driving event of the driver; the type of the current micro-expression is determined according to the emotion interaction information of the driver, the micro-expression combination and the current micro-expression image, and the fatigue level of the driver is determined according to the type of the current micro-expression, the corresponding expression duration and the eye image of the driver, so that the accuracy of the fatigue level of the driver is improved.
Owner:BEIJING JIUZHOU ANHUA INFORMATION SECURITY TECH CO LTD

Multimodal digital human interaction system and method supporting instant hot update

The invention discloses a multi-modal digital human interaction system and method supporting instant hot update, and relates to the technical field of multi-modal digital human interaction.The method comprises the steps that a multi-modal interaction instruction input by a user is received, the multi-modal interaction instruction is analyzed, and semantic description of a target digital human is obtained; synchronously acquiring the voice, the facial micro-expression and the physiological sensing signal of the user to generate an emotional state index; while keeping real-time interaction of a main rendering channel of the current digital person uninterrupted, calling a lightweight generation model in a background channel according to semantic description to construct three-dimensional appearance assets of a target digital person, and establishing a mapping relationship between phonemes corresponding to the target digital person and expression bases; and after monitoring that the assets of the target digital person in the background independent thread are ready, extracting historical voices and the expression weight sequence corresponding to the historical voices from the cross-modal memory cache, and performing expression-driven pre-calculation on the target digital person in combination with the future phoneme sequence of the current voice stream and the mapping relationship.
Owner:NINGXIA UNIVERSITY

Facial expression capture model training method, facial micro-expression driving method and facial micro-expression driving device

The invention discloses a facial expression capture model training method, a facial micro-expression driving method and a facial micro-expression driving device, and belongs to the field of facial expression recognition. The invention discloses a training method of a facial expression capture model. The training method comprises the following steps of 1, obtaining a sample face image set; 2, extracting texture features in the sample face image set; 3, obtaining face images of different expressions; 4, obtaining expression parameters corresponding to the face images with different expressions; and 5, constructing a facial expression capture model, and carrying out training and verification. The method solves the problem that the presentation of the facial expression in the two-dimensional space is not accurate enough due to information loss or distortion in the prior art, and can more accurately capture the change of the facial expression, thereby improving the accuracy of facial expression recognition, improving the accuracy and fineness of facial micro-expression capture, and improving the user experience. Therefore, the facial expression capture model is driven to reflect the emotion change of the character more truly.
Owner:上海视觉艺术学院

Suicide risk prediction method and system based on facial micro-expression recognition

The invention relates to the technical field of crisis prediction, and discloses a suicide risk prediction method and system based on facial micro-expression recognition, and the method comprises the steps: collecting micro-expression sequences of MDD and BD patients and suicide risk crowds, carrying out the normalization and cutting of the micro-expression sequences, and carrying out the data dimension reduction processing through employing KPCA, and obtaining a processed micro-expression sequence; constructing a multi-label graph model based on a fuzzy measurement function, and performing fuzzy processing on the processed micro-expression sequence to obtain a facial feature set; a sparrow search algorithm is adopted to screen out features related to suicide risk prediction from the facial feature set, and a target feature set is obtained; constructing a suicide risk identification model based on fusion of a graph convolutional neural network and a long short-term memory network, inputting the target feature set into the suicide risk identification model, and outputting an identification result by the suicide risk identification model through space-time sequence feature fusion; according to the invention, effective prediction of the suicide risk is realized, early diagnosis of mental diseases is assisted, and the recognition rate and prediction precision are improved.
Owner:NANJING GENERAL HOSPITAL NANJING MILLITARY COMMAND P L A

Micro-expression emotion recognition method and system based on multi-modal time sequence decoupling

The invention provides a micro-expression emotion recognition method and system based on multi-modal time sequence decoupling, and the method comprises the steps: collecting and preprocessing the face video data, and obtaining a standardized video clip; extracting global time sequence features and geometric features by adopting a hierarchical emotion dynamic feature extraction framework; establishing a generative adversarial decoupling network, and separating the global time sequence features into emotion features and noise features; splicing the emotional features with the geometric features to obtain enhanced features; a dual-path network architecture is combined with the enhanced features, emotion classification and emotion feature regression are carried out respectively, and an emotion prediction probability and an emotion feature regression vector are obtained; and based on the enhanced features, the emotion prediction probability and the emotion feature regression vector, obtaining an emotion state score, and based on the emotion state score, obtaining a micro-expression emotion recognition result. According to the method, more comprehensive and more objective quantitative analysis on the emotional state is provided, and a richer decision basis is provided for emotional understanding and human-computer interaction.
Owner:SHENZHEN INSTITUTE OF INFORMATION TECHNOLOGY +2

A Micro-expression Recognition Method Based on AU Structural Dependency Enhancement and Multi-level Prototype Aggregation

This invention discloses a micro-expression recognition method based on AU structural dependency enhancement and multi-level prototype aggregation, belonging to the field of computer vision and emotion computing. The method includes the following steps: Step 1: Obtain micro-expression video sequences from a publicly available dataset, preprocess them to obtain start frame and keyframe image pairs; Step 2: Construct a micro-expression recognition model based on AU structural dependency enhancement and multi-level prototype aggregation; Step 3: Pre-train and fine-tune the micro-expression recognition model, simultaneously constructing an overall loss function to constrain the model, and then training and optimizing the model to obtain a trained micro-expression recognition model; Step 4: Obtain facial expression samples of the current person, and perform micro-expression recognition based on the trained micro-expression recognition model. This invention can simultaneously focus on local subtle movements, model AU structural dependencies, and integrate multi-level emotional semantics, improving the model's recognition performance and robustness in complex and weak signal scenarios.
Owner:SHANDONG UNIV OF SCI & TECH

A Driving Behavior Analysis Method and System Based on Emotion Perception

The application discloses a driving behavior analysis method and system based on emotion perception, and relates to the technical field of driver state perception.The driving behavior analysis system based on emotion perception comprises an emotion perception correction module and a driving behavior analysis module.The application introduces a micro-expression feature branch, a space-time attention branch and a perception weight adjustment branch, accurately captures small changes and time sequence features in the facial expression of a driver, and efficiently identifies emotional fluctuations of the driver even in the case that the facial expression is not rich or the emotional expression is weak, so that the emotion perception model can adapt to individualized emotional expression modes of different drivers, and the accuracy and reliability of emotion identification are improved.
Owner:JIANGXI UNIVERSITY OF FINANCE AND ECONOMICS

System and method for classifying activity of users based on micro-expression and emotion using AI

ActiveUS12511937B2Buying/selling/leasing transactionsMachine learningActivity classificationMicroexpression
A system and method for automatically classifying an activity of a user 102 during a proposal by an agent 104 to a user based on micro-expression and emotion of the user that provides a succeeding response to the agent 104 such that the proposal becomes successful using an artificial intelligence model is provided. The system includes a facial micro-expression unit 106, an expression analyser 110, the artificial intelligence model 112. The facial micro-expression unit 106 captures an interactive sequence of audio-visual information. The expression analyser 110 processes the interactive sequence of audio-visual information using the artificial intelligence model to determine an emotion and intensity of emotion of the user. The expression analyser 110 creates a record of a set of questions and responses. The expression analyzer 110 provides the succeeding response to the agent based on the created record using a wearable device 114.
Owner:RN CHIDAKASHI TECH PTE LTD

A multi-modal multi-task emotion recognition method for faces obscured by head-mounted displays

ActiveCN118779820BSensorsPsychotechnic devicesPattern recognitionMicroexpression
This invention discloses a multimodal, multi-task emotion recognition method for faces occluded by head-mounted displays. First, keyframes are identified, and regions of interest (ROIs) are located within these keyframes. Based on this, facial occlusion is applied to obtain training video sequences. Simultaneously, EEG signals from the ROIs and surrounding physiological signals are used as training signals to obtain training data. Then, a multimodal, multi-task emotion recognition model is constructed. A half-face encoding module with five 3D convolutional layers extracts spatial-temporal features from the video sequences. A physiological signal perception module is constructed using a dual-stream Transformer structure and an interactive modal fusion module to obtain physiological features, enhancing the emotion recognition operation applied to physiological signals. The half-face features are then concatenated with the physiological features to obtain the emotion feature F used for recognition. j Send to two classification layers CL V CL A This invention enhances emotion recognition during micro-expression events, achieving emotion prediction accuracy comparable to full-face images through the fusion of half-face and physiological signals, and possessing the ability to accurately identify complex emotional states.
Owner:UNIV OF ELECTRONICS SCI & TECH OF CHINA

Food sensory experience evaluation and personalized recommendation system based on electroencephalogram multi-mode fusion

The invention relates to the technical field of food sensory evaluation, in particular to a food sensory experience evaluation and personalized recommendation system based on electroencephalogram multi-modal fusion, which comprises a food stimulation module, a physiological reaction acquisition module, a computer system, a machine learning module and a personalized recommendation feedback module, according to the system, multi-modal physiological data such as electroencephalogram signals, electrocardiosignals and facial micro-expressions of a subject in the food stimulation process are collected, topological features are extracted by adopting a topological data analysis method, emotion state manifold characterization is constructed, topology preserving mapping from the emotion state to food attributes is achieved, and the food quality is improved. The core innovation of the invention lies in that a continuous coherence theory is adopted to process multi-modal physiological signals, an 8-dimensional emotional state manifold is constructed, emotional-food attribute mapping is realized through a multilayer radial basis function network, topology is designed to maintain constraints to ensure the stability of a structural relationship, and the system can objectively evaluate the sensory experience of food and improve the quality of food. The defect that a traditional evaluation method is high in subjectivity is overcome.
Owner:BOHAI UNIV

Micro-expression detection method and system based on multi-modal large model, medium and equipment

The invention relates to the field of artificial intelligence and emotion recognition, and discloses a micro-expression detection method and system based on a multi-modal large model, a medium and equipment, and the method comprises the steps: carrying out the face detection of each frame of video, extracting a face region in the video through a deep learning face detection model, and carrying out the unified adjustment of the image size after cutting; a face AU regression network is adopted to extract a face AU feature value of each frame of image so as to represent the motion intensity of face muscles, and face dynamic change information is constructed in combination with a time sequence; and combining the AU features of each frame of image with the image data to generate a language prompt suitable for a multi-modal large language model, training and optimizing the multi-modal large language model to perform emotion reasoning, and performing quantitative evaluation on an emotion reasoning result. According to the method, the accuracy problem caused by tiny change of facial muscles in micro-expression detection is solved; and the reasoning capability and accuracy of the model are improved.
Owner:UNIV OF CHINESE ACAD OF SCI

AI robot emotional state image recognition system based on micro expression analysis

PendingCN121962780ACharacter and pattern recognitionMicroexpressionData acquisition
The invention relates to the technical field of computer vision and robots, in particular to a robot emotional state image recognition system based on micro-expression analysis, which comprises a data acquisition and frequency domain locking step: acquiring a facial time sequence image, measuring a mechanical background vibration frequency and delimiting a target enhancement frequency band; a phase amplification and enhancement step: extracting a local phase in a complex pyramid domain, and performing Euler video amplification on a component in a frequency band; a residual error decoupling and modeling step: constructing an actual quantization and ideal smooth motion model, and calculating a difference value to generate a quantization residual error map; a state judgment and feedback step: extracting space-time pulse characteristics, judging an emotion calculation conflict state and generating a tuning instruction; according to the method, micron-sized tremor is effectively captured, and a bottom layer driving state is converted into visual emotion features.
Owner:深圳市永迦电子科技有限公司

Micro-expression recognition method and system based on context awareness and multi-modal routing

PendingCN122435653AMicroexpressionContext data
The application relates to the field of artificial intelligence and emotion computing technology, and particularly discloses a micro-expression recognition method and system based on context perception and multi-modal routing, which comprises the following steps: respectively performing heterogeneous feature extraction on a visual micro-motion video segment, an audio segment and transcription text of a current sentence, and an audio segment and transcription text of context data; performing cross-modal fusion and sequence compression; generating context type embedding according to a context type label; extracting context perception sentence features through a context cross-attention mechanism; obtaining visual classification confidence based on a visual micro-motion feature tensor, and comparing the visual classification confidence with a preset threshold to generate an emotion classification result. Through a layer focusing fusion network, the application realizes high-dimensional heterogeneous feature extraction and emotion inducement analysis on facial micro-deformation caused by a multi-modal stimulus context, so that accurate prediction of concealed micro-expression is realized.
Owner:JILIN UNIVERSITY

Multimodal emotion recognition method and apparatus

The application relates to the field of artificial intelligence, and provides a multi-modal emotion recognition method and device. The method comprises the following steps: determining a micro-expression knowledge graph, a micro-motion knowledge graph and a text emotion analysis vector of a target user; fusing the micro-expression knowledge graph and the micro-motion knowledge graph, and determining a preset emotion vector according to a fusion result; determining an emotion analysis result according to the preset emotion vector and the text emotion analysis vector; and determining the emotion of the target user according to the emotion analysis result. The multi-modal emotion recognition method provided in the embodiment of the application constructs a knowledge graph based on visual, voice and text multi-modal entities, comprehensively obtains emotion recognition results from multi-modal data, provides more possibilities of emotion judgment, and thus improves the accuracy of emotion recognition.
Owner:CHINA MOBILE (XIONGAN) ICT CO LTD +2