Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

66 results about "Microexpression" patented technology

A microexpressionis the innate result of a voluntary and an involuntary emotional response occurring simultaneously and conflicting with one another. This occurs when the amygdala (the emotion center of the brain) responds appropriately to the stimuli that the individual experiences and the individual wishes to conceal this specific emotion. This results in the individual very briefly displaying their true emotions followed by a false emotional reaction. Human emotions are an unconscious bio-psycho-social reaction that derives from the amygdala and they typically last 0.5–4.0 seconds, although a microexpression will typically last less than 1/2 of a second. Unlike regular facial expressions it is either very difficult or virtually impossible to hide microexpression reactions. Microexpressions cannot be controlled as they happen in a fraction of a second, but it is possible to capture someone's expressions with a high speed camera and replay them at much slower speeds. Microexpressions express the seven universal emotions: disgust, anger, fear, sadness, happiness, contempt, and surprise. Nevertheless, in the 1990s, Paul Ekman expanded his list of emotions, including a range of positive and negative emotions not all of which are encoded in facial muscles. These emotions are amusement, embarrassment, anxiety, guilt, pride, relief, contentment, pleasure, and shame.

Multi-modal interaction method and system of digital human intelligent agent

The invention relates to the field of multi-modal interaction analysis, in particular to a multi-modal interaction method and system of a digital human agent. The method comprises the following steps: acquiring a real-time face image and a voice signal input stream of an interactive user based on an intelligent agent; performing real-time micro-expression recognition and deep emotion analysis based on the real-time facial image to obtain real-time emotion features of the user; performing time sequence evolution analysis on the real-time emotion characteristics of the user, performing holographic user emotion deep mining, and constructing a user emotion holographic characteristic spectrum; carrying out adaptive acoustic gain processing on the voice signal input stream, and carrying out voice-emotion association analysis based on the user emotion holographic characteristic spectrum to generate a voice-emotion linkage mapping spectrum; and carrying out eyeball fixation point migration tracking based on the user emotion holographic feature map and the real-time face image, and generating a user interaction depth intention signal. Through the real-time deep semantic understanding and emotion perception ability, the intelligent agent interaction intelligence and response accuracy are improved.
Owner:GUANGDONG HUITONG INFORMATION TECH CO LTD

Emotion detection system based on facial recognition

The invention discloses an emotion detection system based on facial recognition, and relates to the technical field of computer vision and emotion calculation. A video stream time sequence analysis module is used for extracting a facial micro-expression image sequence of continuous frames, a time sequence feature vector containing a micro-expression intensity gradient, an illumination robustness coefficient and a facial action unit cooperation feature is generated, and a multi-mode dynamic sensing module is combined to carry out real-time analysis on an emotion classification probability, voice emotion parameters and physiological signals. And the fusion decision module performs dynamic weighted fusion on the multi-modal data based on the scene adaptive weight, and finally generates a comprehensive emotion score. Through multi-modal time sequence modeling and a dynamic weight optimization mechanism, the accuracy and environmental adaptability of emotion recognition are remarkably improved, and real-time perception and accurate decision making of customer emotion are realized in a target scene.
Owner:NORTHEAST FORESTRY UNIV

Emotion accompanying robot system based on multi-dimensional perception and interaction method

The invention relates to an emotion accompanying robot system based on multi-dimensional perception and an interaction method, and belongs to the technical field of intelligent robots. The system comprises a sensing module which captures micro-expression, voice, body temperature, touch and other multi-source data through a biological radar array, multispectral imaging and a touch sensing fabric; the decision-making module quantifies the emotion intensity by using an emotion state calculation engine, constructs a user personalized emotion file in combination with the dynamic knowledge graph, and generates a dynamic emotion graph; and the execution module realizes anthropomorphic emotion expression through bionic face driving, joint compliance control and thermal feedback. The interaction method comprises the steps of data capture, emotion file construction, dynamic graph generation, interactive execution and feedback adjustment, definition of an emotion intensity quantification formula, a graph edge weight model and the like. According to the method, physiological signals, environment situations and bionic expression are fused, multi-modal precise emotion perception and safe interaction are achieved, man-machine naturalness is improved, the method is suitable for elderly accompanying, psychological counseling and other scenes, and the emotion accompanying effect is enhanced.
Owner:CHONGQING UNIV OF POSTS & TELECOMM

Anger emotion recognition method based on video stream

The invention discloses an angry emotion recognition method based on a video stream, and belongs to the technical field of emotion recognition. The method comprises the following steps: preprocessing a video stream of a visitor to obtain a time-space aligned standardized facial image sequence, a 3D skeleton sequence and a voice segment; performing feature extraction on the standardized facial image sequence by using a 3D-ResNet34 network and a micro-expression optical flow enhancement algorithm to obtain optical flow facial features; performing feature extraction on the 3D skeleton sequence by using a space-time diagram convolutional network to obtain limb action features; performing feature extraction on the voice segments to obtain voice features; and performing fusion processing on the facial features, the body movement features and the voice features by using a multi-head space-time cross attention mechanism and an expansion time sequence convolution algorithm, and outputting an angry emotion recognition result through dynamic weight distribution classification. According to the method, the angry emotion recognition accuracy of the system is greatly improved on the whole.
Owner:SICHUAN CANCER HOSPITAL

Mental and psychological auxiliary inquiry system based on multi-mode AI

PendingCN120600279AMedical communicationTherapiesMicroexpressionPsychiatric consultation
The invention provides a mental and psychological auxiliary inquiry system based on multi-modal AI, and belongs to the technical field of artificial intelligence and medical crossing, and the mental and psychological auxiliary inquiry system comprises a multi-modal data acquisition module which is used for collecting and analyzing multi-modal data of a patient in the inquiry process in real time based on the artificial intelligence technology, the multi-modal data is related to voice emotion, micro-expression, semantics and face emotion; the abnormal node labeling module is used for establishing a double-standard knowledge graph to analyze the multi-modal data, labeling abnormal data nodes of the patient, and generating a visual inquiry path in combination with contribution weights of the multi-modal data automatically optimized based on a dynamic weight adjustment mechanism; and the scheme generation module is used for performing comparative analysis on the visual inquiry path and a historical case database to generate a personalized treatment scheme. And real-time and objective auxiliary decision support is provided for doctors.
Owner:BEIJING HAOXINQING MOBILE MEDICAL TECH CO LTD

Psychological tutoring system based on digital human technology

The invention provides a psychological tutoring system based on a digital human technology, and relates to the technical field of psychological health. The system captures the emotional state of a user in real time in a multi-modal interaction mode, generates a real-time emotional state group of the user by combining multi-dimensional data fusion of facial micro-expressions, voice spectrums and dialogue texts, and maps the real-time emotional state group to a three-dimensional emotional space for accurate emotion recognition. Based on an emotion-behavior mapping rule, the system dynamically adjusts the digital person and provides an intervention scheme. And when the emotion fluctuation of the user is detected to exceed a preset threshold value, the system automatically triggers a digital human hiding mechanism, gradually weakens visual, auditory and action stimulation, and provides psychological pacification and guidance. According to the method, the accuracy of psychological state recognition and the pertinence of intervention are remarkably improved, real-time and continuous psychological health support is provided for the user, and the method is suitable for psychological health assistance and emotional state analysis.
Owner:BEIJING XUENENGTONG TECHNOLOGY GROUP CO LTD

Three-dimensional emotion resonance interaction system fused with multi-modal perception

The invention discloses a three-dimensional emotion resonance interaction system fused with multi-modal perception, and belongs to the field of artificial intelligence. The emotion micro-expression fine-grained recognition module is used for extracting facial local muscle micro-motion features and amplifying and enhancing micro-expression detail changes through sub-pixel-level motion; the voice emotion spectrum dynamic modeling module is used for analyzing an audio emotion change trend; the individual emotional process depth tracking module is used for modeling an individual emotional state evolution track according to historical interaction records; the three-dimensional emotion situation synchronous rendering module is used for inputting and synchronously displaying emotion changes in a virtual environment; the self-adaptive personalized emotion modeling module dynamically learns an individual emotion response mode and constructs an emotion preference map; the interactive memory evolution module evolves and updates the individual emotion model; and the emotion resonance enhancement regulation and control module is used for dynamically regulating interaction contents and expression modes based on resonance intensity. The method has the beneficial effects that fine-grained perception, accurate understanding and natural expression of three-dimensional emotion resonance interaction are realized.
Owner:SHANGHAI JUNJI INTELLIGENT TECH CO LTD

Device and method for assisting in detecting depression degree by utilizing emotional picture cognitive deviation

The invention relates to a device and a method for assisting in detecting depression degree by utilizing emotional picture cognitive deviation, and relates to the technical field of biomedical engineering and medical instruments, and the device comprises an emotional picture library module which is used for storing a pre-selected emotional picture set; the display interaction module is used for displaying pictures to the user and recording classification selection and reaction time of the user; the data acquisition module is used for acquiring user facial micro-expressions and synchronously storing the facial micro-expressions and classification results; the data processing module is used for calculating positive picture error rate related features and normalizing behavior features and micro expression features into feature vectors; the depression index calculation module is used for outputting depression indexes; and the result output module is used for generating a visual report. The depression index is calculated based on behavior data, deviation is avoided, multi-modal features are fused to improve quantization precision, low-cost non-intrusive screening is achieved through a touch screen and a camera, and an embedded processor is integrated to improve the integration level and generalizability.
Owner:AFFILIATED HUSN HOSPITAL OF FUDAN UNIV

Interactive communication humanoid robot expression simulation method based on artificial intelligence

The invention discloses an interactive humanoid robot expression simulation method based on artificial intelligence, and relates to the technical field of expression simulation, and the method comprises the steps: collecting a brain wave signal of a user, carrying out the preprocessing, and generating a power spectrum density feature; inputting the power spectral density features into a pre-trained emotion classification model, and outputting an emotion category and an emotion intensity value of the user; performing emotional expression mapping on the emotional category and the emotional intensity value, and outputting a bionic muscle contraction instruction; performing facial adjustment on facial muscle fibers of the humanoid robot according to the bionic muscle contraction instruction, dynamically adjusting surface texture details of the humanoid robot, and generating a simulated micro expression; and capturing user face feedback data through camera shooting eyeballs, evaluating the acceptance degree of the user on the simulated micro-expression based on the user face feedback data, and outputting an expression optimization strategy. According to the method, through deformation matching and expression optimization double-closed-loop modeling driven by user feedback, biological simulation of micro-expressions of the humanoid robot is realized.
Owner:BEIJING YUNSHANGHUI INFORMATION TECH CO LTD

Cerebral stroke upper limb rehabilitation training system and method based on dynamic reward feedback

The embodiment of the invention discloses a cerebral apoplexy upper limb rehabilitation training system and method based on dynamic reward feedback. A data acquisition module is used for acquiring physiological signals, motion signals, emotional state data and training performance data; the motion intention decoding module is used for performing motion intention feature extraction and reliability evaluation on the physiological signals according to an improved MREE-Net + + fusion algorithm, performing weighted fusion after feature weights are adjusted according to a reliability result, obtaining a unified motion intention feature vector, performing motion intention decoding, and obtaining a motion intention intensity index and a motion intention vector; the dynamic reward decision module is used for processing the facial micro-expression data according to an improved VGG-Face model to obtain an emotion titer, and obtaining an emotion awakening degree according to the voice signal; a reward action is generated according to an improved deep reinforcement learning algorithm; and the adaptive training regulation and control module is used for generating a personalized virtual training scene according to the generative adversarial network and regulating and controlling the training intensity according to the fatigue index. The rehabilitation training effect can be improved.
Owner:SHANGHAI SECOND REHABILITATION HOSPITAL (SHANGHAI BAOSHAN NO 1 STEEL HOSPITAL)

Healthcare accompanying robot based on multi-modal facial feature fusion

The invention relates to the field of health care robots, and particularly discloses a health care accompanying robot based on multi-modal facial feature fusion, which is characterized in that a facial expression video stream of an elderly object is acquired and refined preprocessing is carried out, and then fine features of each area of the face are captured; then, the fine features of all the areas of the face are intelligently recognized, and higher weights are given to the face micro expressions; and then capturing the time dynamic state of the expression, and further focusing on the key moment and mode of expression change in combination with a time attention module so as to generate a more accurate final emotion vector. And finally, an appropriate action instruction is generated for the robot based on the emotion vector, so that the defect that a traditional method depends on a single macro expression and ignores a contradictory signal is overcome, and accurate and unambiguous understanding and response to the emotion state of the elderly user are achieved.
Owner:ZHEJIANG FUBAO INTELLIGENT TECH CO LTD

Artificial intelligence psychological assessment method and device based on multiple modes

The invention discloses an artificial intelligence psychological assessment method and equipment based on multiple modes, particularly relates to the technical field of psychological assessment analysis, and aims to solve the problem of high detection error rate caused by biological characteristic distortion due to privacy desensitization in existing psychological assessment. The method comprises the following steps of: dynamically generating an evaluation instruction matched with a user behavior, collecting a facial micro-expression video, segmenting and quantifying a privacy interference level based on region pixels, and fuzzifying a non-sensitive region while keeping key micro-expression region characteristics; synchronously acquiring a voice spectrum waveform and reconstructing voiceprint tone modification and fundamental frequency waveform; extracting text sentiment keywords to construct a cross-stage sentiment association chain to repair semantic fracture caused by local desensitization; and according to the privacy interference level, dynamically adjusting the fusion weight of the video and the voice to generate a stage evaluation parameter and a time sequence label, and finally constructing a psychological state evolution graph and outputting an evaluation report, so that the emotional feature fidelity and the psychological state analysis precision are remarkably improved.
Owner:HANGZHOU XINWA TECHNOLOGY CO LTD

VEM-Token world model robot expression function construction method

According to the VEM-Token world model robot expression function construction method, a traditional NLP-Token method is not adopted, a VEM-fe expression function model is established, a robot can directly understand emotions and learn expressions, expression vectors of multiple beats are synthesized and aligned into synthesized expressions, expression conflicts among the multiple expression vectors are eliminated, and the expression function construction efficiency is improved. The synthetic expression from the current beat to the next beat is smoothly transited, and a physical space robot or a virtual digital robot is adopted to learn, understand and display facial expressions and dialogue expressions in social contact. According to the consistency, simplification, fuzziness and compensatory of expressions found by researches, through micro-expression simulation of eyes, a mouth, a head and dialogues, real-time expressions and dialogue expression spectrums, supervised learning and reinforcement learning training are adopted to obtain direct social and industrial expression libraries; the robot is designed to face government public service, hotel reception, catering reception, bank hall consultation, shopping mall and exclusive shop shopping guide and the like, and one-to-one and one-to-many conversation with customers is achieved.
Owner:GREATER BAY AREA STAR BIOTECH (SHENZHEN) CO LTD

Micro-expression real-time emotion studying and judging method and system

The invention relates to the crossing field of computer vision and emotion calculation, and particularly provides a micro-expression real-time emotion research and judgment method and device, and the method comprises the following steps: S1, multi-mode data collection; s2, micro-expression feature extraction; s3, multi-modal fusion optimization is carried out; s4, real-time emotion tracking; and S5, edge-cloud collaborative study and judgment. Compared with the prior art, the method has the advantages that the dynamic compensation algorithm can effectively improve the micro-expression detection rate, the edge calculation can greatly reduce the response time, the feature compression algorithm can save the bandwidth, and meanwhile, the method supports recognition of various basic emotions and composite emotions, and a self-adaptive adjustment mechanism can switch analysis modes in different scenes such as medical treatment and security.
Owner:INSPUR SOFTWARE TECH CO LTD

Method and device for realizing fast micro-expression recognition processing based on bidirectional optical flow, processor and computer readable storage medium thereof

The present application relates to a kind of based on two-way optical flow to realize the method for fast micro-expression recognition processing, comprising the following steps: according to visual system acquisition tester face micro-expression video segment information;Extract the emotional video segment in emotional memory library, and the facial muscle movement situation of micro-expression in emotional video segment is captured by positive and negative two-way optical flow;Extract method extracts key frame in emotional video segment, and the redundant frame in continuous sequence image is eliminated;Call the optical flow information between key frame in optical flow information memory library.The present application also relates to a kind of two-way optical flow to realize fast micro-expression recognition device, processor and storage medium.The method for fast micro-expression recognition processing based on two-way optical flow of the present application, device, processor and its computer readable storage medium, the facial muscle movement situation of micro-expression is captured by positive and negative two-way optical flow, the micro-expression of tester is identified using muscle movement trend, and the micro-expression recognition accuracy is improved.
Owner:SHANGHAI UNIV

Micro-expression-based emotion recognition method, device, equipment and medium

This application relates to the field of digital medicine, specifically to a method, device, equipment and medium for emotion recognition based on micro-expressions. This method comprehensively considers the two factors that the climax frame in a micro-expression video frame sequence contains more facial information and that the optical flow information can reflect the subtle changes in the face. RGB The two types of climax frames, climax frames and optical flow climax frames, obtain more and more subtle facial features from the micro-expression video to be identified, thereby improving the accuracy of emotion recognition of micro-expressions. At the same time, this method RGB The climax frames and optical flow climax frames are enhanced to increase the number of micro-expression data samples. RGB The number of climax frames and optical flow climax frames increases the number of micro-expression videos in the micro-expression data sample set in disguise. The greatly increased number of samples can improve the accuracy of the training model and further improve the accuracy of emotion recognition of micro-expressions.
Owner:PING AN TECH (SHENZHEN) CO LTD

Emotion analysis method and system for children based on eye movement and micro-expression association

The application belongs to the technical field of biomedical signal processing and computer vision, and specifically discloses a child emotion analysis method and system based on eye movement and micro-expression association, which comprises the following steps: synchronously acquiring an eye movement data stream and a facial video stream, triggering a dynamic analysis process by identifying a mode conflict between high physiological arousal and low emotional expression intensity after generating a physiological arousal and emotion hypothesis through preliminary analysis, generating a guide strategy according to a conflict degree, and enhancing original data in a time domain or a space domain to facilitate deep analysis, capturing suppressed micro-expression signals in the enhanced data to correct the preliminary emotion hypothesis, and finally fusing a physiological arousal level, a corrected emotion category and a quantified expression inhibition degree to output a multi-dimensional psychological state analysis result containing an emotion category, a physiological arousal level and an expression inhibition degree, so as to realize quantitative and multi-dimensional evaluation of real emotions of children.
Owner:NANTONG UNIV

Driver state determination method and device, electronic equipment and storage medium

The invention discloses a driver state determination method and device, electronic equipment and a storage medium, and relates to the technical field of vehicle detection. The method comprises the steps of determining a target micro-expression type of a driver in a vehicle at a current moment based on collected target facial image data of the driver in the vehicle and a trained micro-expression recognition model; based on a preset natural language processing algorithm, performing voice processing on the collected target voice stream data of the driver in the vehicle, and determining target semantics and target emotion types of the driver in the vehicle at the current moment; determining a target emotion type of the driver in the vehicle at the current moment based on the target micro-expression type, the target semantics and the target emotion type; and determining a target psychological state of the driver in the vehicle at the current moment based on the target emotion type and a preset emotion psychological state mapping table. According to the method, the psychological state of the driver in the vehicle can be analyzed and dredged, so that the accuracy of the monitoring result of state monitoring of the driver is improved.
Owner:VOYAH AUTOMOBILE TECH CO LTD

User experience improving system based on emotion recognition

The invention discloses a user experience improving system based on emotion recognition, and the system comprises an emotion capturing module which is used for capturing the micro-expression of a user and converting the micro-expression into digital image information; the emotion analysis module is used for analyzing the image information and determining the emotion category of the user; the response strategy module is used for triggering voice pacifying or switching to manual service intervention according to the emotion category of the user; the central control module is used for coordinating the work of each module; according to the method, the micro-expression of the user can be captured through the camera, accurate recognition of the emotional state of the user is achieved, the system can more sensitively perceive the requirements and confusion of the user, and a foundation is laid for providing personalized services subsequently; the method supports a response mechanism of triggering voice pacifying or switching to manual service intervention according to the emotion category of the user, can timely and effectively relieve the negative emotion of the user, provides timely and intimate help and support for the user, and enhances the credibility of the user to the system.
Owner:INSPUR FINANCIAL INFORMATION TECHNOLOGY CO LTD

Deep learning model training method and system for in-vehicle user micro-expression recognition

The invention provides a deep learning model training method and system for in-vehicle user micro-expression recognition, and the method and system achieve the precise recognition of the emotion of a vehicle owner through the fusion of facial expressions and voice features. In the model construction process, a self-attention mechanism is introduced, and the mechanism can enhance the capture ability of the model to micro expressions and subtle voice changes, and effectively improve the accuracy of emotion classification. And meanwhile, a localized data processing and edge computing technology is adopted, so that all emotion recognition and in-vehicle environment regulation operations are locally completed on the vehicle-mounted terminal. According to the method, the real-time requirement can be met, the privacy risk caused by uploading the user data to the cloud end is avoided, the personalized requirements of the vehicle owner can be learned based on the emotion change of the vehicle owner in combination with the historical behavior data of the vehicle owner, and then the settings such as customized in-vehicle environment adjustment, automatic adjustment of the air conditioner temperature, the air speed and seat heating of thousands of people are achieved, and the user experience is improved. And the driving comfort and safety are obviously improved.
Owner:TONGLING PFAFEN ELECTRONIC TECHNOLOGY CO LTD

Deep learning-based eating micro-expression and food satisfaction correlation analysis method and application

This invention relates to the field of health data analysis technology, and particularly to a method and application for analyzing the correlation between eating micro-expressions and food satisfaction based on deep learning. The method includes the following steps: S1, capturing facial video images of users during the eating process using a camera device, and extracting eating facial feature sequences from them; S2, separating chewing actions and facial muscle movements from the eating facial feature sequences based on the spatial displacement differences of facial key points, and obtaining target micro-expression feature vectors. In this invention, by cross-evaluating the captured eating micro-expression emotional state with the user's physiological health indicators, nutritional threshold constraints and weight corrections are applied to the initial food preferences generated based on emotions. This allows for strict control of the intake of core risk nutrients while catering to the user's personal taste satisfaction, ultimately generating personalized recommended recipes that balance emotional experience and medical health standards.
Owner:YUNNAN AGRICULTURAL UNIVERSITY

Depression early screening and risk assessment platform based on artificial intelligence

The invention relates to the technical field of personal health risk assessment, in particular to a depression early screening and risk assessment platform based on artificial intelligence, and the platform can achieve the following steps through the mutual cooperation of a plurality of modules: obtaining a facial image and an audio of a patient to be detected in each historical observation period, obtaining a face image in each historical question and answer period and a reply delay corresponding to each question and answer question; performing mouth corner raising change analysis according to the face image in the historical observation period; performing non-positive emotion analysis processing according to the facial image in the historical observation period; performing response function analysis processing according to the audio in the historical observation period; determining a micro-expression sudden increase coefficient corresponding to the historical question and answer period; and determining a depression risk assessment auxiliary value corresponding to the to-be-detected patient. According to the method, the depression risk assessment auxiliary value is quantified relatively objectively by analyzing the facial image, the audio and the like of the patient, and the accuracy of the quantification of the depression risk assessment auxiliary value is improved.
Owner:XIAN CENT HOSPITAL

A method for simulating facial expressions in an interactive humanoid robot based on artificial intelligence

This invention discloses an artificial intelligence-based method for simulating facial expressions in an interactive humanoid robot, belonging to the field of facial expression simulation technology. The method includes: acquiring and preprocessing user brainwave signals to generate power spectral density features; inputting the power spectral density features into a pre-trained emotion classification model to output the user's emotion category and intensity value; mapping the emotion category and intensity value to an emotional expression and outputting biomimetic muscle contraction commands; adjusting the facial muscle fibers of the humanoid robot according to the biomimetic muscle contraction commands to dynamically adjust the surface texture details of the humanoid robot and generate simulated micro-expressions; capturing user facial feedback data through camera eyes, evaluating the user's acceptance of the simulated micro-expressions based on the user's facial feedback data, and outputting an expression optimization strategy. This invention achieves bio-realistic micro-expressions in a humanoid robot through a dual closed-loop modeling approach of deformation matching and user feedback-driven expression optimization.
Owner:BEIJING YUNSHANGHUI INFORMATION TECH CO LTD

Learning emotion adaptive adjustment method and system based on micro-expression analysis

The invention discloses a learning emotion self-adaptive adjustment method and system based on micro-expression analysis, and belongs to the technical field of learning machines and micro-expression analys.The method comprises the steps that firstly, a face video stream of a learner is obtained in real time and divided into a plurality of time sliding windows; secondly, performing face region detection to obtain a face region image, and selecting a plurality of ROI regions from the face region image; then, calculating an optical flow vector matrix of the ROI in each frame, and calculating a facial local motion matrix of the ROI; calculating a single-frame micro-expression space feature vector of each frame, and forming a micro-expression space-time fusion matrix: finally, performing structured phase extraction on the micro-expression, and mapping real-time learning emotion of a learner, so as to perform real-time adjustment on learning parameters, and completing adaptive adjustment of the learning emotion. Through recognition of micro expressions and structured phase extraction, the learning machine can sense the emotional state of the learner in real time and make adjustment in time, and the learning efficiency of the learner is remarkably improved.
Owner:SHENZHEN ACCELERATION TECHNOLOGY CO LTD

Question and answer emotion evaluation method and device based on action unit au and micro-expression

The application discloses a kind of based on action unit AU and micro-expression question and answer mood evaluation method and device, obtain the question and answer video of user to be detected, the first image about non-basic question and answer in question and answer video is input into face detection model and obtains first face image, first face image is input into the multi-task identification model based on AU and micro-expression, output multiple first AU scores and multiple first micro-expression scores of first face image. By the corresponding relationship between multiple AU, multiple micro-expression and multiple emotional parameters, multiple first AU scores and multiple first micro-expression scores are weighted to obtain multiple target emotional values corresponding to multiple emotional parameters. Compare multiple target emotional values with multiple basic emotional values corresponding to multiple emotional parameters to obtain comparison result. The question and answer mood of user to be detected is evaluated by comparison result. In this way, the analysis standard of question and answer mood is set, and the fairness and credibility of evaluating question and answer mood are improved.
Owner:太保科技有限公司

Facial expression analysis method based on deep learning

The invention relates to the technical field of facial expression analysis, in particular to a facial expression analysis method based on deep learning, which comprises the following steps: acquiring facial feature point coordinates, analyzing displacement and density distribution, obtaining a facial activity intensity index, calling muscle action data, analyzing synchronization rate and direction consistency, and constructing muscle group interaction data. The method comprises the following steps: tracking facial actions, identifying action time, analyzing an action change rate, calculating an action change contribution ratio, analyzing a ratio variable amplitude, adjusting a weight, obtaining an action contribution adjustment result, and comparing a facial region action collaboration degree to obtain a facial coordination output index. According to the method, through fine comparison of facial feature point displacement and density distribution, the measurement precision of facial activity intensity is improved, emotion information extraction is more accurate, muscle group interaction analysis particularly strengthens measurement of synchronous activity rate and direction consistency, and the recognition capability of micro expressions and fast change expressions is improved.
Owner:BOZHOU SMART HEALTH TECHNOLOGY CO LTD +1

Air rifle athlete training emotion monitoring method based on phase entanglement cooperation

The invention provides an air rifle athlete training emotion monitoring method based on phase entanglement collaboration, which is used for realizing deep fusion of context perception on the basis of deep spatio-temporal features extracted from electroencephalogram signals and facial micro-expressions by a phase entanglement collaboration network and a micro-expression recognition model. Therefore, the complex emotional state of the air rifle athlete can be synchronously and accurately recognized in a high-pressure and fast-changing training environment, and then high-precision data support is provided for monitoring and intervention of the emotional state of the air rifle athlete.
Owner:WENHUA UNIV

Conversation psychological state assessment method based on multi-modal data

The invention provides a conversation psychological state assessment method based on multi-modal data. The method is applied to scenes such as century review conversation and the like. Multi-modal data such as micro expressions, voices and intonations, physiological parameters and semantic contents of a talked object are obtained through videos, microphones and non-contact physiological monitoring; fusing the multi-modal features with the individual feature vectors, inputting a psychological state recognition model, and recognizing the emotion category, the psychological elasticity level and the risk evolution trend of the talked object; further, based on cross analysis of the psychological elasticity index and semantic emotion, dynamically calculating a psychological load value, judging a psychological tolerance boundary and outputting a safety level; and finally, according to the risk type, the change rate and the context sensitivity, forming a three-level intervention suggestion of risk type, evolution rate and context association. Real-time evaluation and trend prediction of the psychological state of the conversation object are achieved, the method has the advantages of being non-contact, intelligent and high in interpretability, and the safety and scientificity of the conversation process are effectively improved.
Owner:GUODIAN DADU RIVER DAGANGSHAN HYDROPOWER DEV

Emotion recognition method and system based on multi-modal biological characteristics

PendingCN121808672AExpress emotions naturallyEye movement characteristics leadMicroexpressionSpeech sound
The invention discloses an emotion recognition method and system based on multi-modal biological characteristics, and relates to the technical field of emotion recognition, and the system comprises a multi-modal module, a calculation module, a dynamic module and a summary module. A coupling unit on the summarizing module also calculates a first coupling degree between the eye movement features and the micro-expression features and a second coupling degree between the eye movement features and the voice features through the eye movement feature fluctuation degree, the micro-expression feature fluctuation degree, the voice feature fluctuation degree and a coupling formula respectively, and when the value of the first coupling degree is closer to 1, the second coupling degree is closer to 1; the micro-expression features and the eye movement features are more synchronous, the micro-expression features and the eye movement features of the tested person are closer to natural physiological reactions, and the micro-expression features are not deliberately controlled; when the numerical value of the second coupling degree is closer to 1, the voice features and the eye movement features are more synchronous, further, the voice features and the eye movement features of the testee are closer to natural physiological reactions, and the voice features are not deliberately controlled.
Owner:FOURTH MILITARY MEDICAL UNIVERSITY

Service response method based on micro-expression recognition and speech emotion analysis

The invention discloses a service response method based on micro-expression recognition and voice sentiment analysis. The method comprises the following steps: acquiring facial image data and voice information in a user interaction process by using a camera and a microphone through a data acquisition module; performing real-time processing on the facial image data by adopting a micro-expression recognition module, and recognizing micro-expression features of the user in the process; processing the voice information by using a voice emotion analysis module, and analyzing voice features of the voice information to determine emotion types contained in the voice features; the micro-expression feature data and the voice emotion data are fused through a data fusion module, and feature vectors comprehensively reflecting the emotion state of the user are generated; according to the invention, through fusion of micro-expression recognition and voice emotion analysis, more comprehensive and accurate perception of the emotion of the user is realized, so that highly personalized service response is provided, the user experience is effectively improved, and the integrating degree between the service and the user demand is enhanced.
Owner:INSPUR FINANCIAL INFORMATION TECHNOLOGY CO LTD