Dynamic micro-expression recognition and psychological abnormality mining system

Abstract

The invention discloses a dynamic micro-expression recognition and psychological anomaly mining system, which comprises a data acquisition module, a multi-dimensional feature extraction module, a feature fusion module, an emotion classification module, a psychological anomaly evaluation module and a feedback and interaction module. According to the system, the accuracy of emotion recognition is greatly improved through multi-modal data collection and feature extraction including micro expressions, voices, limb actions and physiological signals, the psychological abnormal state of a user can be accurately predicted through historical data and behavior pattern analysis, accurate health scoring and early warning are provided, and the health of the user can be accurately evaluated according to an evaluation result. The system can provide personalized emotion regulation suggestions in real time or give out early warning when psychological abnormality is found, and the psychological health management level of the user is improved.

Description

Technical Field

[0001] The present invention relates to emotion recognition and psychological anomaly assessment, and in particular to a dynamic micro-expression recognition and psychological anomaly mining system, and belongs to the technical field of mental health assessment. Background Art

[0002] With the rapid development of artificial intelligence and affective computing technologies, the demand for identifying users' emotional states and mental health conditions is growing. Traditional emotion recognition technologies usually rely on single-modal data, such as facial expressions or voice. However, such single-modal methods cannot effectively reflect the user's true emotional state, especially when the user shows micro-expressions, the emotional state may be difficult to detect. In addition, the assessment of psychological abnormalities often requires long-term observation and the accumulation of a large amount of data, but current systems are usually lacking in this regard.

[0003] Therefore, the present invention proposes a multimodal emotion recognition and psychological abnormality assessment system based on micro-expressions, voice, body movements and physiological signals, aiming to improve the accuracy and timeliness of emotion recognition and to effectively evaluate and provide feedback on psychological abnormalities. Summary of the invention

[0004] The purpose of the present invention is to provide a dynamic micro-expression recognition and psychological anomaly mining system to solve the problems raised in the above background technology.

[0005] To achieve the above-mentioned purpose, the present invention provides the following technical scheme: a dynamic micro-expression recognition and psychological abnormality mining system, comprising a data acquisition module, a multidimensional feature extraction module, a feature fusion module, a sentiment classification module, a psychological abnormality assessment module and a feedback and interaction module, wherein the output of the data acquisition module is connected to the input of the multidimensional feature extraction module, the output of the multidimensional feature extraction module is connected to the input of the feature fusion module, the output of the feature fusion module is connected to the input of the sentiment classification module, the output of the sentiment classification module is connected to the input of the psychological abnormality assessment module, and the output of the psychological abnormality assessment module is connected to the input of the feedback and interaction module.

[0006] As a preferred technical solution of the present invention, the data acquisition module includes a micro-expression acquisition unit, a voice acquisition unit, a body movement acquisition unit and a physiological signal acquisition unit, which are respectively used to collect corresponding emotional data.

[0007] As a preferred technical solution of the present invention, the multidimensional feature extraction module includes a micro-expression feature extraction unit, a speech feature extraction unit, a body movement feature extraction unit and a physiological feature extraction unit, which are respectively used to extract emotional features from the collected data.

[0008] As a preferred technical solution of the present invention, the feature fusion module includes a feature standardization unit, a weighted fusion unit and a modality consistency check unit, which are used to perform standardization, weighted fusion and consistency detection on each modality feature.

[0009] As a preferred technical solution of the present invention, the emotion classification module includes a feature preprocessing unit, a classification model training unit, an emotion prediction unit and a classification result optimization unit, which are used to preprocess, classify and optimize the emotion features.

[0010] As a preferred technical solution of the present invention, the psychological abnormality assessment module includes a historical data storage unit, a psychological abnormality prediction unit, a behavior pattern analysis unit and a psychological health scoring unit, which are used to assess the user's psychological health status.

[0011] As a preferred technical solution of the present invention, the feedback and interaction module includes a feedback generation unit, an early warning system unit and a user interaction unit, which are used to provide emotion regulation suggestions and trigger early warnings.

[0012] Compared with the prior art, the beneficial effects of the present invention are: the dynamic micro-expression recognition and psychological abnormality mining system of the present invention greatly improves the accuracy of emotion recognition through multimodal data collection and feature extraction, including micro-expressions, voice, body movements and physiological signals. Through historical data and behavioral pattern analysis, it can accurately predict the user's abnormal psychological state and provide accurate health scores and warnings. According to the evaluation results, the system can provide personalized emotion regulation suggestions in real time, or issue warnings when psychological abnormalities are found, thereby improving the user's mental health management level. DETAILED DESCRIPTION

[0013] The technical solutions in the embodiments of the present invention are described clearly and completely. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0014] The present invention provides a dynamic micro-expression recognition and psychological anomaly mining system, comprising a data acquisition module, a multidimensional feature extraction module, a feature fusion module, a sentiment classification module, a psychological anomaly assessment module and a feedback and interaction module, wherein the output of the data acquisition module is connected to the input of the multidimensional feature extraction module, the output of the multidimensional feature extraction module is connected to the input of the feature fusion module, the output of the feature fusion module is connected to the input of the sentiment classification module, the output of the sentiment classification module is connected to the input of the psychological anomaly assessment module, and the output of the psychological anomaly assessment module is connected to the input of the feedback and interaction module.

[0015] Data acquisition module: This module is used to collect the user's micro-expression, voice, body movement and physiological signal data. The acquisition units include a micro-expression acquisition unit, a voice acquisition unit, a body movement acquisition unit and a physiological signal acquisition unit.

[0016] Multi-dimensional feature extraction module: extracts emotional features from the collected data. The extraction units include micro-expression feature extraction unit, speech feature extraction unit, body movement feature extraction unit and physiological feature extraction unit.

[0017] Feature fusion module: The emotional features of different modalities are fused. The fusion unit includes a feature normalization unit, a weighted fusion unit, and a modality consistency check unit.

[0018] Sentiment classification module: Classification is performed based on the fused sentiment features. The classification unit includes a feature preprocessing unit, a classification model training unit, a sentiment prediction unit, and a classification result optimization unit.

[0019] Psychological abnormality assessment module: Evaluates the user's psychological abnormality through historical data and behavior pattern analysis. The assessment unit includes a historical data storage unit, a psychological abnormality prediction unit, a behavior pattern analysis unit and a mental health scoring unit.

[0020] Feedback and interaction module: Generates evaluation results and provides real-time interactive feedback. The feedback unit includes feedback generation unit, early warning system unit and user interaction unit.

[0021] Each module is connected through internal data transmission. The output of the data acquisition module is used as the input of the multidimensional feature extraction module. After being processed by the feature fusion module, the emotion classification module classifies the emotion state according to the fusion features. Finally, the psychological abnormality assessment module predicts and evaluates psychological abnormalities based on the classification results. The feedback and interaction module provides real-time feedback and early warning.

[0022] The data acquisition module includes a micro-expression acquisition unit, a voice acquisition unit, a body movement acquisition unit and a physiological signal acquisition unit, which are respectively used to collect corresponding emotion data:

[0023] Micro-expression acquisition unit: uses cameras and facial recognition technology to capture subtle facial expression changes, inputs user facial expressions, and outputs micro-expression data;

[0024] Voice collection unit: collects user voice data through a microphone, analyzes characteristics such as intonation, speaking speed, volume, etc., inputs the user's voice signal, and outputs voice data;

[0025] Body movement acquisition unit: captures the user's body movements and posture changes through a camera or sensor, inputs the user's body movements, and outputs body movement data;

[0026] Physiological signal acquisition unit: uses sensors to collect the user's heart rate, skin galvanic response, blood pressure and other physiological data, inputs the user's physiological signals, and outputs physiological signal data;

[0027] Furthermore, the multi-dimensional feature extraction module includes a micro-expression feature extraction unit, a voice feature extraction unit, a body movement feature extraction unit and a physiological feature extraction unit, which are respectively used to extract emotional features from the collected data:

[0028] Micro-expression feature extraction unit: analyzes facial muscle movements, extracts key features of micro-expressions, such as facial action coding system features, inputs micro-expression data, and outputs micro-expression features;

[0029] Speech feature extraction unit: analyzes speech signals, extracts emotional features in speech, such as intonation, frequency, rhythm, etc., and inputs speech data and speech features;

[0030] Body movement feature extraction unit: extracts emotional features from body movements, such as gestures, posture changes, and movement amplitudes, inputs body movement data, and outputs body movement features;

[0031] Physiological feature extraction unit: Analyze physiological signals, extract physiological features that reflect emotional states, such as heart rate changes, the amplitude of skin electrical response, etc., input physiological signal data, and output physiological features.

[0032] The feature fusion module includes a feature standardization unit, a weighted fusion unit and a modality consistency check unit, which are used to perform standardization, weighted fusion and consistency check on each modality feature:

[0033] Feature normalization unit: normalizes the emotional features of each modality to ensure that the feature values ​​are within a unified range for easy subsequent fusion. It inputs the feature data of each modality and outputs the standardized feature data.

[0034] Weighted fusion unit: According to the weights of different emotional features, it performs weighted processing to generate comprehensive emotional features, inputs standardized micro-expressions, voice, body movements and physiological features, and outputs weighted emotional features;

[0035] Modality consistency check unit: Checks the consistency of emotional features of different modalities, identifies and handles conflicting features, such as emotional classification conflicts between micro-expressions and voice signals, inputs weighted fused emotional features, consistency detection results and adjusted comprehensive emotional features.

[0036] The sentiment classification module includes a feature preprocessing unit, a classification model training unit, a sentiment prediction unit and a classification result optimization unit, which are used to preprocess, classify and optimize sentiment features:

[0037] Feature preprocessing unit: processes the fused emotional features, removes redundant information or noise to improve classification accuracy, inputs comprehensive emotional features, and outputs preprocessed feature data;

[0038] Classification model training unit: trains sentiment classification models through machine learning algorithms, such as support vector machines and convolutional neural networks, inputs pre-processed historical sentiment data, and outputs the trained sentiment classification model;

[0039] Emotion prediction unit: input the preprocessed emotion features into the trained emotion classification model to predict the user's emotional state (such as happiness, anxiety, sadness, etc.), input the preprocessed emotion features, and output the emotion classification results;

[0040] Classification result optimization unit: adjusts and optimizes the classification results according to specific scenarios or individual historical data to ensure the accuracy of classification, inputs preliminary classification results and individual historical data, and outputs optimized sentiment classification results.

[0041] The psychological anomaly assessment module includes a historical data storage unit, a psychological anomaly prediction unit, a behavior pattern analysis unit and a psychological health scoring unit, which are used to assess the user's psychological health status:

[0042] Historical data storage unit: stores the user's historical emotional data and behavior patterns for subsequent psychological abnormality assessment, inputs the user's emotional classification data, and outputs the stored historical emotional data;

[0043] Psychological abnormality prediction unit: Based on the emotion classification results and historical data, predict the user's possible psychological abnormalities, such as depression, anxiety, etc., input the emotion classification results and historical emotion data, and output the psychological abnormality prediction results;

[0044] Behavior pattern analysis unit: analyzes changes in user behavior patterns, detects whether there are abnormal behavior characteristics, assists in the assessment of psychological abnormalities, inputs user behavior data, and outputs behavior pattern analysis results;

[0045] Mental health scoring unit: Generates the user's mental health score based on emotion classification and behavior patterns for quantitative evaluation of mental state, inputs psychological abnormality prediction results and behavior pattern analysis results, and outputs a mental health score.

[0046] The feedback and interaction module includes a feedback generation unit, an early warning system unit and a user interaction unit, which are used to provide emotion regulation suggestions and trigger early warnings:

[0047] Feedback generation unit: Generates a user report based on the psychological abnormality assessment results, including the current emotional state, mental health score, etc., inputs the psychological abnormality assessment results, and outputs a user feedback report;

[0048] Early warning system unit: When a serious psychological abnormality is detected, it automatically triggers an early warning, notifies the user or his guardian, inputs the psychological health score and abnormality assessment results, and outputs early warning information;

[0049] User interaction unit: Provides real-time interactive feedback, such as emotion regulation suggestions or guides users to perform emotion regulation activities, based on the emotional state detected by the system, inputs emotion classification and abnormality assessment results, and outputs interactive prompts or emotion regulation suggestions.

[0050] Each unit performs its specific function independently, ensuring that the system can more accurately and flexibly perform micro-expression recognition, emotion classification, and psychological anomaly mining. This modular structure also facilitates the expansion and optimization of the system.

[0051] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. Dynamic micro-expression recognition and psychological anomaly mining system, including data acquisition module, multi-dimensional feature extraction module, feature fusion module, emotion classification module, psychological anomaly assessment module and feedback and interaction module, characterized in that: The output of the data acquisition module is connected to the input of the multidimensional feature extraction module, the output of the multidimensional feature extraction module is connected to the input of the feature fusion module, the output of the feature fusion module is connected to the input of the sentiment classification module, the output of the sentiment classification module is connected to the input of the psychological abnormality assessment module, and the output of the psychological abnormality assessment module is connected to the input of the feedback and interaction module.

2. The dynamic micro-expression recognition and psychological anomaly mining system according to claim 1, characterized in that: The data acquisition module includes a micro-expression acquisition unit, a voice acquisition unit, a body movement acquisition unit and a physiological signal acquisition unit.

3. The dynamic micro-expression recognition and psychological anomaly mining system according to claim 1, characterized in that: The multi-dimensional feature extraction module includes a micro-expression feature extraction unit, a speech feature extraction unit, a body movement feature extraction unit and a physiological feature extraction unit.

4. The dynamic micro-expression recognition and psychological anomaly mining system according to claim 1, characterized in that: The feature fusion module includes a feature normalization unit, a weighted fusion unit and a modality consistency checking unit.

5. The dynamic micro-expression recognition and psychological anomaly mining system according to claim 1, characterized in that: The emotion classification module includes a feature preprocessing unit, a classification model training unit, an emotion prediction unit and a classification result optimization unit.

6. The dynamic micro-expression recognition and psychological anomaly mining system according to claim 1, characterized in that: The psychological abnormality assessment module includes a historical data storage unit, a psychological abnormality prediction unit, a behavior pattern analysis unit and a psychological health scoring unit.

7. The dynamic micro-expression recognition and psychological anomaly mining system according to claim 1, characterized in that: The feedback and interaction module includes a feedback generation unit, an early warning system unit and a user interaction unit.