Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

88 results about "Microexpression" patented technology

A microexpressionis the innate result of a voluntary and an involuntary emotional response occurring simultaneously and conflicting with one another. This occurs when the amygdala (the emotion center of the brain) responds appropriately to the stimuli that the individual experiences and the individual wishes to conceal this specific emotion. This results in the individual very briefly displaying their true emotions followed by a false emotional reaction. Human emotions are an unconscious bio-psycho-social reaction that derives from the amygdala and they typically last 0.5–4.0 seconds, although a microexpression will typically last less than 1/2 of a second. Unlike regular facial expressions it is either very difficult or virtually impossible to hide microexpression reactions. Microexpressions cannot be controlled as they happen in a fraction of a second, but it is possible to capture someone's expressions with a high speed camera and replay them at much slower speeds. Microexpressions express the seven universal emotions: disgust, anger, fear, sadness, happiness, contempt, and surprise. Nevertheless, in the 1990s, Paul Ekman expanded his list of emotions, including a range of positive and negative emotions not all of which are encoded in facial muscles. These emotions are amusement, embarrassment, anxiety, guilt, pride, relief, contentment, pleasure, and shame.

Multi-modal interaction method and system of digital human intelligent agent

The invention relates to the field of multi-modal interaction analysis, in particular to a multi-modal interaction method and system of a digital human agent. The method comprises the following steps: acquiring a real-time face image and a voice signal input stream of an interactive user based on an intelligent agent; performing real-time micro-expression recognition and deep emotion analysis based on the real-time facial image to obtain real-time emotion features of the user; performing time sequence evolution analysis on the real-time emotion characteristics of the user, performing holographic user emotion deep mining, and constructing a user emotion holographic characteristic spectrum; carrying out adaptive acoustic gain processing on the voice signal input stream, and carrying out voice-emotion association analysis based on the user emotion holographic characteristic spectrum to generate a voice-emotion linkage mapping spectrum; and carrying out eyeball fixation point migration tracking based on the user emotion holographic feature map and the real-time face image, and generating a user interaction depth intention signal. Through the real-time deep semantic understanding and emotion perception ability, the intelligent agent interaction intelligence and response accuracy are improved.
Owner:GUANGDONG HUITONG INFORMATION TECH CO LTD

Emotional state evaluation system and method based on multiple modes

The invention discloses a multi-modal-based emotional state evaluation method, which comprises the following steps: S1, collecting multi-modal data of a user to be tested, the multi-modal data comprising voice data, facial micro-expression data and facial heart rate data; s2, constructing a multi-modal deep neural network, inputting the multi-modal data into the multi-modal deep neural network, and dynamically adjusting the weight of the modal data by the multi-modal deep neural network by using an attention mechanism so as to output and obtain fusion features; and S3, judging the emotional state of the to-be-detected user based on the fused features. Accordingly, the invention further discloses a multi-mode-based emotional state evaluation system, and the multi-mode-based emotional state evaluation system can be used for implementing the emotional state evaluation method, so that the physiological status of the patient is more accurately and comprehensively considered, the emotional state of the patient is evaluated, and the accuracy of the emotional state evaluation is improved. Therefore, professional psychologists are assisted in diagnosing depression.
Owner:BEIJING ZHENXIN PRIME MINISTER INTELLIGENT TECHNOLOGY CO LTD

Emotion detection system based on facial recognition

The invention discloses an emotion detection system based on facial recognition, and relates to the technical field of computer vision and emotion calculation. A video stream time sequence analysis module is used for extracting a facial micro-expression image sequence of continuous frames, a time sequence feature vector containing a micro-expression intensity gradient, an illumination robustness coefficient and a facial action unit cooperation feature is generated, and a multi-mode dynamic sensing module is combined to carry out real-time analysis on an emotion classification probability, voice emotion parameters and physiological signals. And the fusion decision module performs dynamic weighted fusion on the multi-modal data based on the scene adaptive weight, and finally generates a comprehensive emotion score. Through multi-modal time sequence modeling and a dynamic weight optimization mechanism, the accuracy and environmental adaptability of emotion recognition are remarkably improved, and real-time perception and accurate decision making of customer emotion are realized in a target scene.
Owner:NORTHEAST FORESTRY UNIV

Emotion accompanying robot system based on multi-dimensional perception and interaction method

The invention relates to an emotion accompanying robot system based on multi-dimensional perception and an interaction method, and belongs to the technical field of intelligent robots. The system comprises a sensing module which captures micro-expression, voice, body temperature, touch and other multi-source data through a biological radar array, multispectral imaging and a touch sensing fabric; the decision-making module quantifies the emotion intensity by using an emotion state calculation engine, constructs a user personalized emotion file in combination with the dynamic knowledge graph, and generates a dynamic emotion graph; and the execution module realizes anthropomorphic emotion expression through bionic face driving, joint compliance control and thermal feedback. The interaction method comprises the steps of data capture, emotion file construction, dynamic graph generation, interactive execution and feedback adjustment, definition of an emotion intensity quantification formula, a graph edge weight model and the like. According to the method, physiological signals, environment situations and bionic expression are fused, multi-modal precise emotion perception and safe interaction are achieved, man-machine naturalness is improved, the method is suitable for elderly accompanying, psychological counseling and other scenes, and the emotion accompanying effect is enhanced.
Owner:CHONGQING UNIV OF POSTS & TELECOMM

Related song pushing system and method based on face micro-expression recognition

The invention discloses a related song pushing system based on face micro-expression recognition, and relates to the field of song pushing. The problem that limitation exists when the emotional state of the user is deduced through existing voice recognition or text analysis and music is recommended according to the emotional state is solved. The system comprises a data acquisition module, a face detection module, a micro-expression recognition and classification module, an emotional state inference module and a music recommendation module. The data acquisition module acquires a face video stream and transmits the face video stream to the face detection module, the face detection module detects face information in each frame in the face video stream and transmits the face information to the micro-expression recognition and classification module, and the micro-expression recognition and classification module performs micro-expression classification and performs micro-expression classification. The classification result is input to the emotional state inference module; the emotional state deduction module deduces the emotional state of the user and inputs the emotional state of the user to the music recommendation module; and the music recommendation module selects matched music for recommendation according to the emotional state of the user. The method is applied to the music personalized classification field.
Owner:赵政凯

Anger emotion recognition method based on video stream

The invention discloses an angry emotion recognition method based on a video stream, and belongs to the technical field of emotion recognition. The method comprises the following steps: preprocessing a video stream of a visitor to obtain a time-space aligned standardized facial image sequence, a 3D skeleton sequence and a voice segment; performing feature extraction on the standardized facial image sequence by using a 3D-ResNet34 network and a micro-expression optical flow enhancement algorithm to obtain optical flow facial features; performing feature extraction on the 3D skeleton sequence by using a space-time diagram convolutional network to obtain limb action features; performing feature extraction on the voice segments to obtain voice features; and performing fusion processing on the facial features, the body movement features and the voice features by using a multi-head space-time cross attention mechanism and an expansion time sequence convolution algorithm, and outputting an angry emotion recognition result through dynamic weight distribution classification. According to the method, the angry emotion recognition accuracy of the system is greatly improved on the whole.
Owner:SICHUAN CANCER HOSPITAL

Student classroom attention analysis system based on facial multi-modal emotion fusion

The invention provides a student classroom attention analysis system based on facial multi-modal emotion fusion, aiming at solving the problem that an effective micro-expression recognition method for judging the attention analysis of students on teaching content is lacked in the current intelligent education classroom environment. The invention mainly solves the technical problem of providing a facial micro-expression recognition method based on multi-modal fusion aiming at the deficiency of facial micro-expression recognition analysis in an intelligent teaching classroom at present, and the facial micro-expression recognition method comprises the following steps of: dynamically enhancing local detail features of a facial image by utilizing coding features of a facial action unit; according to the method, the image difference subtle influence caused by low intensity of micro-expression motion is reduced, whether the students hide the real emotional states by using micro-expressions or not is analyzed, the attention condition of the students in class is analyzed according to the recognition condition of the micro-expressions and information such as the usual performance of the students and the teacher-student communication frequency, and then the teaching content of teachers is dynamically adjusted.
Owner:CHANGZHI UNIV

Mental and psychological auxiliary inquiry system based on multi-mode AI

PendingCN120600279AMedical communicationTherapiesMicroexpressionPsychiatric consultation
The invention provides a mental and psychological auxiliary inquiry system based on multi-modal AI, and belongs to the technical field of artificial intelligence and medical crossing, and the mental and psychological auxiliary inquiry system comprises a multi-modal data acquisition module which is used for collecting and analyzing multi-modal data of a patient in the inquiry process in real time based on the artificial intelligence technology, the multi-modal data is related to voice emotion, micro-expression, semantics and face emotion; the abnormal node labeling module is used for establishing a double-standard knowledge graph to analyze the multi-modal data, labeling abnormal data nodes of the patient, and generating a visual inquiry path in combination with contribution weights of the multi-modal data automatically optimized based on a dynamic weight adjustment mechanism; and the scheme generation module is used for performing comparative analysis on the visual inquiry path and a historical case database to generate a personalized treatment scheme. And real-time and objective auxiliary decision support is provided for doctors.
Owner:BEIJING HAOXINQING MOBILE MEDICAL TECH CO LTD

Psychological tutoring system based on digital human technology

The invention provides a psychological tutoring system based on a digital human technology, and relates to the technical field of psychological health. The system captures the emotional state of a user in real time in a multi-modal interaction mode, generates a real-time emotional state group of the user by combining multi-dimensional data fusion of facial micro-expressions, voice spectrums and dialogue texts, and maps the real-time emotional state group to a three-dimensional emotional space for accurate emotion recognition. Based on an emotion-behavior mapping rule, the system dynamically adjusts the digital person and provides an intervention scheme. And when the emotion fluctuation of the user is detected to exceed a preset threshold value, the system automatically triggers a digital human hiding mechanism, gradually weakens visual, auditory and action stimulation, and provides psychological pacification and guidance. According to the method, the accuracy of psychological state recognition and the pertinence of intervention are remarkably improved, real-time and continuous psychological health support is provided for the user, and the method is suitable for psychological health assistance and emotional state analysis.
Owner:BEIJING XUENENGTONG TECHNOLOGY GROUP CO LTD

Three-dimensional emotion resonance interaction system fused with multi-modal perception

The invention discloses a three-dimensional emotion resonance interaction system fused with multi-modal perception, and belongs to the field of artificial intelligence. The emotion micro-expression fine-grained recognition module is used for extracting facial local muscle micro-motion features and amplifying and enhancing micro-expression detail changes through sub-pixel-level motion; the voice emotion spectrum dynamic modeling module is used for analyzing an audio emotion change trend; the individual emotional process depth tracking module is used for modeling an individual emotional state evolution track according to historical interaction records; the three-dimensional emotion situation synchronous rendering module is used for inputting and synchronously displaying emotion changes in a virtual environment; the self-adaptive personalized emotion modeling module dynamically learns an individual emotion response mode and constructs an emotion preference map; the interactive memory evolution module evolves and updates the individual emotion model; and the emotion resonance enhancement regulation and control module is used for dynamically regulating interaction contents and expression modes based on resonance intensity. The method has the beneficial effects that fine-grained perception, accurate understanding and natural expression of three-dimensional emotion resonance interaction are realized.
Owner:SHANGHAI JUNJI INTELLIGENT TECH CO LTD

Device and method for assisting in detecting depression degree by utilizing emotional picture cognitive deviation

The invention relates to a device and a method for assisting in detecting depression degree by utilizing emotional picture cognitive deviation, and relates to the technical field of biomedical engineering and medical instruments, and the device comprises an emotional picture library module which is used for storing a pre-selected emotional picture set; the display interaction module is used for displaying pictures to the user and recording classification selection and reaction time of the user; the data acquisition module is used for acquiring user facial micro-expressions and synchronously storing the facial micro-expressions and classification results; the data processing module is used for calculating positive picture error rate related features and normalizing behavior features and micro expression features into feature vectors; the depression index calculation module is used for outputting depression indexes; and the result output module is used for generating a visual report. The depression index is calculated based on behavior data, deviation is avoided, multi-modal features are fused to improve quantization precision, low-cost non-intrusive screening is achieved through a touch screen and a camera, and an embedded processor is integrated to improve the integration level and generalizability.
Owner:AFFILIATED HUSN HOSPITAL OF FUDAN UNIV

Emotion recognition method and device

The embodiment of the invention provides an emotion recognition method and device, and the method comprises the steps: determining an initial sample group, and obtaining initial face data based on the initial sample group; performing data annotation on the face data, and determining annotated data; performing data preprocessing based on the annotation data, and determining target face data; performing feature extraction based on the target facial data, and determining expression features; training a deep learning model based on the expression features, and determining an emotion recognition model; and acquiring to-be-recognized face data through a camera, and determining emotion data from the to-be-recognized face data through the emotion recognition model. The method realizes automatic detection and recognition of micro-expressions by constructing a deep learning model, has high precision, high efficiency and wide application prospect, and can be applied to the fields of man-machine interaction, intelligent monitoring, mental health and the like.
Owner:BEIJING HAOXINQING MOBILE MEDICAL TECH CO LTD

Interactive communication humanoid robot expression simulation method based on artificial intelligence

The invention discloses an interactive humanoid robot expression simulation method based on artificial intelligence, and relates to the technical field of expression simulation, and the method comprises the steps: collecting a brain wave signal of a user, carrying out the preprocessing, and generating a power spectrum density feature; inputting the power spectral density features into a pre-trained emotion classification model, and outputting an emotion category and an emotion intensity value of the user; performing emotional expression mapping on the emotional category and the emotional intensity value, and outputting a bionic muscle contraction instruction; performing facial adjustment on facial muscle fibers of the humanoid robot according to the bionic muscle contraction instruction, dynamically adjusting surface texture details of the humanoid robot, and generating a simulated micro expression; and capturing user face feedback data through camera shooting eyeballs, evaluating the acceptance degree of the user on the simulated micro-expression based on the user face feedback data, and outputting an expression optimization strategy. According to the method, through deformation matching and expression optimization double-closed-loop modeling driven by user feedback, biological simulation of micro-expressions of the humanoid robot is realized.
Owner:BEIJING YUNSHANGHUI INFORMATION TECH CO LTD

Emotion evaluation method and system

The invention belongs to the technical field of emotion evaluation. The emotion evaluation method comprises the steps that a video frame of a played video in a preset time period and an expression frame of micro-expression change of a user when the user watches the video are obtained, and the expression frame represents the facial expression of the user recorded by a frame of image; performing corresponding matching on the video frame and the expression frame to obtain to-be-analyzed data; inputting the to-be-analyzed data into a target convolutional neural network to obtain a micro-expression data analysis result; and obtaining an emotion evaluation result of the user according to the micro-expression data analysis result. Through the method, the related characteristics of the expression change are analyzed to summarize the emotional state of the testee when the testee receives the test, and a reasonable reference analysis material is provided for related services of psychological assessment.
Owner:NO 15 INST OF CHINA ELECTRONICS TECH GRP

Cerebral stroke upper limb rehabilitation training system and method based on dynamic reward feedback

The embodiment of the invention discloses a cerebral apoplexy upper limb rehabilitation training system and method based on dynamic reward feedback. A data acquisition module is used for acquiring physiological signals, motion signals, emotional state data and training performance data; the motion intention decoding module is used for performing motion intention feature extraction and reliability evaluation on the physiological signals according to an improved MREE-Net + + fusion algorithm, performing weighted fusion after feature weights are adjusted according to a reliability result, obtaining a unified motion intention feature vector, performing motion intention decoding, and obtaining a motion intention intensity index and a motion intention vector; the dynamic reward decision module is used for processing the facial micro-expression data according to an improved VGG-Face model to obtain an emotion titer, and obtaining an emotion awakening degree according to the voice signal; a reward action is generated according to an improved deep reinforcement learning algorithm; and the adaptive training regulation and control module is used for generating a personalized virtual training scene according to the generative adversarial network and regulating and controlling the training intensity according to the fatigue index. The rehabilitation training effect can be improved.
Owner:SHANGHAI SECOND REHABILITATION HOSPITAL (SHANGHAI BAOSHAN NO 1 STEEL HOSPITAL)

Dynamic micro-expression recognition and psychological abnormality mining system

The invention discloses a dynamic micro-expression recognition and psychological anomaly mining system, which comprises a data acquisition module, a multi-dimensional feature extraction module, a feature fusion module, an emotion classification module, a psychological anomaly evaluation module and a feedback and interaction module. According to the system, the accuracy of emotion recognition is greatly improved through multi-modal data collection and feature extraction including micro expressions, voices, limb actions and physiological signals, the psychological abnormal state of a user can be accurately predicted through historical data and behavior pattern analysis, accurate health scoring and early warning are provided, and the health of the user can be accurately evaluated according to an evaluation result. The system can provide personalized emotion regulation suggestions in real time or give out early warning when psychological abnormality is found, and the psychological health management level of the user is improved.
Owner:CHANGZHOU FEISI INTELLIGENT TECHNOLOGY CO LTD

Digital human auxiliary diagnosis method and device, electronic equipment and readable storage medium

The invention provides a digital human auxiliary diagnosis method and device, electronic equipment and a readable storage medium, and is applied to the technical field of medical data processing. The method comprises the following steps: processing question and answer audio information of a patient to generate question semantic information and sound feature information of the patient; processing the real-time facial video information and the physiological status information of the patient to generate facial micro-expression feature information and physiological index feature information; processing the sound feature information, the facial micro-expression feature information and the physiological index feature information based on a multi-modal fusion network to generate real-time emotional state information of the patient; processing the real-time emotional state information of the patient, the question semantic information of the patient and the preference information of the patient to generate target reply style information of the digital person; and processing the real-time operation information of the hospital department of the target area and the historical case record information of the patient based on the question and answer planning network, generating resource allocation result information, and generating digital human reply information.
Owner:BEIJING TSINGHUA CHANGGUNG HOSPITAL

Old people micro-expression recognition method and device based on elimination of initial basic expression state influence, processor and storage medium thereof

The invention relates to an old people micro-expression recognition method based on elimination of an initial basic expression state influence, and the method comprises the following steps: (1) collecting the face micro-expression video clip information of the old people of a testee according to a visual system, and storing the face micro-expression video clip information in a face data storage memory bank; (2) extracting emotional video clips in the face data storage memory bank, and extracting facial features of the current elderly by removing initial basic expression state features; (3) extracting facial change features of the current micro-expression of the old people through a multi-scale convolutional neural network, and acquiring an area of interest in the micro-expression of the old people by using an incremental area of interest extraction module; and (4) taking the acquired features of the senior micro-surface emotional interest area as the input of a full-connection neural network to classify the real facial expressions of the senior. The invention also relates to a corresponding device, a processor and a computer readable storage medium.
Owner:SHANGHAI UNIV

Emotion analysis method and system based on artificial intelligence

The invention provides an emotion analysis method based on artificial intelligence, and the method comprises the steps: collecting a target video material of a person, and extracting face information and voice information in the target video material; converting the voice information into text information, and obtaining text emotion intensity according to the text information; performing feature extraction on the voice information to obtain voice information, and obtaining voice emotion intensity according to the voice information; identifying facial feature points of the face information, obtaining micro-expressions according to the facial feature points, and calculating facial emotion intensity according to the micro-expressions; obtaining a comprehensive emotion score according to the text emotion intensity, the voice emotion intensity and the face emotion intensity; and obtaining an emotion analysis result according to the comprehensive emotion score. According to the method, text, voice and facial information are highly fused, and the emotion intensity of three dimensions is comprehensively considered by using an artificial intelligence algorithm, so that the accuracy of emotion judgment is greatly improved.
Owner:AIR FORCE MEDICAL CENT PLA

Drawing projection marriage matching test system and method based on visual converter

The invention provides a drawing projection marriage matching test system based on a visual converter and a method thereof. According to the system, drawing behaviors, emotion feedback and interaction data of a user are collected, multi-modal feature extraction is carried out through a visual converter, and drawing styles, psychological features and emotion preferences of the user are deeply analyzed through a self-supervised clustering algorithm. A unique emotion feedback loop is embedded into a system, and micro expressions, gestures and emotion changes shown by a user in the drawing process are captured in real time, so that more accurate personalized marriage matching recommendation is realized. The system generates a comprehensive love and marriage matching report, provides customized love and marriage suggestions in combination with potential characters, behavior modes and emotional requirements of users, and is matched with emotional management and mental health support tools. The innovative system improves the scientificity, interactivity and deep emotion association of love and marriage matching, has high application value, and can be widely applied to the fields of love and marriage platforms, emotion consultation and intelligent mental health.
Owner:CHONGQING HOLOGRAPHIC PAINTING INTELLIGENT TECHNOLOGY CO LTD

Healthcare accompanying robot based on multi-modal facial feature fusion

The invention relates to the field of health care robots, and particularly discloses a health care accompanying robot based on multi-modal facial feature fusion, which is characterized in that a facial expression video stream of an elderly object is acquired and refined preprocessing is carried out, and then fine features of each area of the face are captured; then, the fine features of all the areas of the face are intelligently recognized, and higher weights are given to the face micro expressions; and then capturing the time dynamic state of the expression, and further focusing on the key moment and mode of expression change in combination with a time attention module so as to generate a more accurate final emotion vector. And finally, an appropriate action instruction is generated for the robot based on the emotion vector, so that the defect that a traditional method depends on a single macro expression and ignores a contradictory signal is overcome, and accurate and unambiguous understanding and response to the emotion state of the elderly user are achieved.
Owner:ZHEJIANG FUBAO INTELLIGENT TECH CO LTD

Micro-expression emotion recognition method based on over-parameterization

The invention relates to a micro-expression emotion recognition method based on over-parameterization, and the method specifically comprises the steps: S1, preprocessing: obtaining a micro-expression video, carrying out the face detection of each frame in the video, and carrying out the preprocessing operation of face alignment and cutting; s2, feature extraction: calculating optical flow images of the face images of the peak frame and the initial frame; s3, inputting the optical flow graph obtained by feature extraction into a convolutional neural network model for training; s4, performing structural re-parameterization processing on the trained multi-branch convolutional neural network model; and S5, inputting a to-be-detected micro-expression video into the single-branch convolutional neural network model after structure re-parameterization for reasoning, and identifying the micro-expression. In the training stage, model parameters are improved, and the expression ability of the model is improved, so that the micro-expression discrimination precision is improved, and the reasoning speed is optimized; in the training stage, parameters are reduced in a structure re-parameterization mode, and model reasoning is accelerated while precision is reserved.
Owner:NINGBO KANGNING HOSPITAL (NINGBO MENTAL DISEASE PREVENTION & CONTROL CENT NINGBO INST OF MICROCIRCULATION & HYOSCYAMS) +1

Artificial intelligence psychological assessment method and device based on multiple modes

The invention discloses an artificial intelligence psychological assessment method and equipment based on multiple modes, particularly relates to the technical field of psychological assessment analysis, and aims to solve the problem of high detection error rate caused by biological characteristic distortion due to privacy desensitization in existing psychological assessment. The method comprises the following steps of: dynamically generating an evaluation instruction matched with a user behavior, collecting a facial micro-expression video, segmenting and quantifying a privacy interference level based on region pixels, and fuzzifying a non-sensitive region while keeping key micro-expression region characteristics; synchronously acquiring a voice spectrum waveform and reconstructing voiceprint tone modification and fundamental frequency waveform; extracting text sentiment keywords to construct a cross-stage sentiment association chain to repair semantic fracture caused by local desensitization; and according to the privacy interference level, dynamically adjusting the fusion weight of the video and the voice to generate a stage evaluation parameter and a time sequence label, and finally constructing a psychological state evolution graph and outputting an evaluation report, so that the emotional feature fidelity and the psychological state analysis precision are remarkably improved.
Owner:HANGZHOU XINWA TECHNOLOGY CO LTD

VEM-Token world model robot expression function construction method

According to the VEM-Token world model robot expression function construction method, a traditional NLP-Token method is not adopted, a VEM-fe expression function model is established, a robot can directly understand emotions and learn expressions, expression vectors of multiple beats are synthesized and aligned into synthesized expressions, expression conflicts among the multiple expression vectors are eliminated, and the expression function construction efficiency is improved. The synthetic expression from the current beat to the next beat is smoothly transited, and a physical space robot or a virtual digital robot is adopted to learn, understand and display facial expressions and dialogue expressions in social contact. According to the consistency, simplification, fuzziness and compensatory of expressions found by researches, through micro-expression simulation of eyes, a mouth, a head and dialogues, real-time expressions and dialogue expression spectrums, supervised learning and reinforcement learning training are adopted to obtain direct social and industrial expression libraries; the robot is designed to face government public service, hotel reception, catering reception, bank hall consultation, shopping mall and exclusive shop shopping guide and the like, and one-to-one and one-to-many conversation with customers is achieved.
Owner:GREATER BAY AREA STAR BIOTECH (SHENZHEN) CO LTD

Micro-expression recognition method and device based on AU guidance

The invention discloses a micro-expression recognition method and device based on AU guidance, and the method comprises the steps: carrying out the preprocessing of a micro-expression sample video, and obtaining a micro-expression sequence; generating a facial muscle movement text description for each micro-expression sample through an AU coding label of the public data set and a facial movement coding system; sending the processed micro-expression sequence into a visual encoder to obtain encoded visual motion features; sending the motion text description into a text encoder to obtain text motion features; by generating AU-guided fine-grained micro-expression facial muscle motion text description and utilizing CLIP comparison loss to perform cross-modal semantic alignment, perception and capture of a model on weak local micro-expression motion can be improved, remarkable space-time micro-expression motion features are learned, the motion features are effectively converted into emotional features, and the motion features are effectively converted into emotional features. And the Focal Loss classification loss is used to solve the problem of micro-expression data class imbalance, so that the recognition performance of the model is greatly improved.
Owner:UNIV OF SCI & TECH OF CHINA

Micro-expression real-time emotion studying and judging method and system

The invention relates to the crossing field of computer vision and emotion calculation, and particularly provides a micro-expression real-time emotion research and judgment method and device, and the method comprises the following steps: S1, multi-mode data collection; s2, micro-expression feature extraction; s3, multi-modal fusion optimization is carried out; s4, real-time emotion tracking; and S5, edge-cloud collaborative study and judgment. Compared with the prior art, the method has the advantages that the dynamic compensation algorithm can effectively improve the micro-expression detection rate, the edge calculation can greatly reduce the response time, the feature compression algorithm can save the bandwidth, and meanwhile, the method supports recognition of various basic emotions and composite emotions, and a self-adaptive adjustment mechanism can switch analysis modes in different scenes such as medical treatment and security.
Owner:INSPUR SOFTWARE TECH CO LTD

Method and device for realizing fast micro-expression recognition processing based on bidirectional optical flow, processor and computer readable storage medium thereof

The present application relates to a kind of based on two-way optical flow to realize the method for fast micro-expression recognition processing, comprising the following steps: according to visual system acquisition tester face micro-expression video segment information;Extract the emotional video segment in emotional memory library, and the facial muscle movement situation of micro-expression in emotional video segment is captured by positive and negative two-way optical flow;Extract method extracts key frame in emotional video segment, and the redundant frame in continuous sequence image is eliminated;Call the optical flow information between key frame in optical flow information memory library.The present application also relates to a kind of two-way optical flow to realize fast micro-expression recognition device, processor and storage medium.The method for fast micro-expression recognition processing based on two-way optical flow of the present application, device, processor and its computer readable storage medium, the facial muscle movement situation of micro-expression is captured by positive and negative two-way optical flow, the micro-expression of tester is identified using muscle movement trend, and the micro-expression recognition accuracy is improved.
Owner:SHANGHAI UNIV

Micro-expression-based emotion recognition method, device, equipment and medium

This application relates to the field of digital medicine, specifically to a method, device, equipment and medium for emotion recognition based on micro-expressions. This method comprehensively considers the two factors that the climax frame in a micro-expression video frame sequence contains more facial information and that the optical flow information can reflect the subtle changes in the face. RGB The two types of climax frames, climax frames and optical flow climax frames, obtain more and more subtle facial features from the micro-expression video to be identified, thereby improving the accuracy of emotion recognition of micro-expressions. At the same time, this method RGB The climax frames and optical flow climax frames are enhanced to increase the number of micro-expression data samples. RGB The number of climax frames and optical flow climax frames increases the number of micro-expression videos in the micro-expression data sample set in disguise. The greatly increased number of samples can improve the accuracy of the training model and further improve the accuracy of emotion recognition of micro-expressions.
Owner:PING AN TECH (SHENZHEN) CO LTD

Emotion analysis method and system for children based on eye movement and micro-expression association

The application belongs to the technical field of biomedical signal processing and computer vision, and specifically discloses a child emotion analysis method and system based on eye movement and micro-expression association, which comprises the following steps: synchronously acquiring an eye movement data stream and a facial video stream, triggering a dynamic analysis process by identifying a mode conflict between high physiological arousal and low emotional expression intensity after generating a physiological arousal and emotion hypothesis through preliminary analysis, generating a guide strategy according to a conflict degree, and enhancing original data in a time domain or a space domain to facilitate deep analysis, capturing suppressed micro-expression signals in the enhanced data to correct the preliminary emotion hypothesis, and finally fusing a physiological arousal level, a corrected emotion category and a quantified expression inhibition degree to output a multi-dimensional psychological state analysis result containing an emotion category, a physiological arousal level and an expression inhibition degree, so as to realize quantitative and multi-dimensional evaluation of real emotions of children.
Owner:NANTONG UNIV

Driver state determination method and device, electronic equipment and storage medium

The invention discloses a driver state determination method and device, electronic equipment and a storage medium, and relates to the technical field of vehicle detection. The method comprises the steps of determining a target micro-expression type of a driver in a vehicle at a current moment based on collected target facial image data of the driver in the vehicle and a trained micro-expression recognition model; based on a preset natural language processing algorithm, performing voice processing on the collected target voice stream data of the driver in the vehicle, and determining target semantics and target emotion types of the driver in the vehicle at the current moment; determining a target emotion type of the driver in the vehicle at the current moment based on the target micro-expression type, the target semantics and the target emotion type; and determining a target psychological state of the driver in the vehicle at the current moment based on the target emotion type and a preset emotion psychological state mapping table. According to the method, the psychological state of the driver in the vehicle can be analyzed and dredged, so that the accuracy of the monitoring result of state monitoring of the driver is improved.
Owner:VOYAH AUTOMOBILE TECH CO LTD