Psychological stress detection method, device, equipment and storage medium

Abstract

The application discloses a psychological stress detection method, device, equipment and storage medium, and the method comprises the steps of obtaining face image information of a user to be detected; inputting the face image information into a preset face video coding network to output a coding feature vector; detecting whether the face image information contains a preset face action, and outputting a motion feature vector according to a detection result corresponding to the preset face action; detecting whether the face image information contains a preset face emotion, and outputting an emotion feature vector according to a detection result corresponding to the preset face emotion; and generating a psychological stress detection result corresponding to the user to be detected according to the coding feature vector, the motion feature vector and the emotion feature vector, thereby completing psychological stress detection on the user to be detected. The method is not only suitable for medical scenes, but also can be widely applied to other high-concurrency scenes, such as education scenes. The method can identify the psychological stress of the user in a timely manner, thereby providing more targeted help.

Description

Technical Field

[0001] This application relates to the field of deep learning technology, and in particular to a method, apparatus, device and storage medium for detecting psychological stress. Background Technology

[0002] In the medical field, especially in the area of ​​psychological counseling and treatment, psychological stress is a significant factor affecting individual mental health in modern society. Prolonged stress can lead to psychological problems such as anxiety and depression, and even harm physical health. Timely and accurate detection of psychological stress levels is crucial for mental health intervention and management.

[0003] Traditional methods for detecting psychological stress mainly rely on physiological signal acquisition devices, such as heart rate monitors and skin conductance sensors. While these methods can obtain physiological indicators, they have significant drawbacks: First, the use of specialized equipment increases the cost of detection and requires users to wear or touch the devices, making them inconvenient and difficult to popularize in daily life. Second, physiological signal detection is easily affected by environmental factors (such as skin condition and exercise) and cannot directly reflect key psychological stress characteristics such as an individual's emotional expression and facial micro-expressions.

[0004] Therefore, a method is urgently needed to solve at least one of the above problems. Summary of the Invention

[0005] This application provides a method, device, equipment, and storage medium for detecting psychological stress, aiming to solve the problems of traditional psychological stress detection methods that mainly rely on physiological signal acquisition equipment, which increases detection costs, lacks convenience, is difficult to popularize, and is susceptible to interference from environmental factors, failing to directly reflect key psychological stress characteristics such as an individual's emotional expression and facial micro-expressions.

[0006] Firstly, this application provides a method for detecting psychological stress, including...

[0007] Obtain facial image information of the user to be detected;

[0008] The facial image information is input into a preset facial video coding network, which encodes and represents the facial image information and outputs an encoded feature vector.

[0009] Detect whether the facial image information contains a preset facial action, and output the action feature vector corresponding to the facial image information based on the detection result corresponding to the preset facial action;

[0010] Detect whether the facial image information contains a preset facial emotion, and output the emotion feature vector corresponding to the facial image information based on the detection result corresponding to the preset facial emotion;

[0011] The psychological stress detection result corresponding to the user to be detected is generated based on the encoded feature vector, action feature vector and emotion feature vector, thus completing the psychological stress detection of the user to be detected.

[0012] In some embodiments, obtaining the facial image information of the user to be tested includes: asking the user to be tested questions according to preset psychological stress counseling questions; and obtaining the facial image information corresponding to the user's answers to the psychological stress counseling questions.

[0013] In some embodiments, the number of preset facial actions is multiple; the step of detecting whether the facial image information contains preset facial actions and outputting the action feature vector corresponding to the facial image information based on the detection result corresponding to the preset facial actions includes: inputting the facial image information to multiple facial action units respectively, each facial unit parsing the facial image information and outputting the action detection result corresponding to the facial image information; each facial action unit corresponds to one preset facial action; generating an action feature matrix based on the action detection result output by each facial action unit; and encoding the action feature matrix to generate the action feature vector.

[0014] In some embodiments, the number of preset facial emotions is multiple; the facial image information includes multiple frames of facial images; the step of detecting whether the facial image information contains preset facial emotions and outputting the emotion feature vector corresponding to the facial image information based on the detection results corresponding to the preset facial emotions includes: detecting the emotion detection result corresponding to each frame of the facial image based on each preset facial emotion; generating an emotion feature matrix based on the emotion detection results corresponding to each preset facial emotion; and encoding the emotion feature matrix to generate the emotion feature vector.

[0015] In some embodiments, the encoded feature vector, action feature vector, and emotion feature vector have the same dimension; generating the psychological stress detection result corresponding to the user to be detected based on the encoded feature vector, action feature vector, and emotion feature vector includes: performing feature fusion on the encoded feature vector, action feature vector, and emotion feature vector based on a self-attention mechanism to generate a fused feature vector; inputting the fused feature vector into a preset psychological stress detection model, the psychological stress detection model parsing the fused feature vector and outputting the psychological stress detection result.

[0016] For example, the feature fusion of the encoded feature vector, action feature vector, and emotion feature vector based on the self-attention mechanism includes: fusing the encoded feature vector and action feature vector based on cross-attention to obtain a first sub-fused feature vector; wherein, in the fusion of the first sub-fused feature vector, the action feature vector serves as query information, and the encoded feature vector serves as key information and value information; fusing the emotion feature vector and action feature vector based on cross-attention to obtain a second sub-fused feature vector; wherein, in the fusion of the second sub-fused feature vector, the action feature vector serves as query information, and the emotion feature vector serves as key information and value information; fusing the first sub-fused feature vector and the second sub-fused feature vector based on cross-attention to obtain the fused feature vector; wherein, in the fusion of the fused feature vector, the first sub-fused feature vector serves as query information, and the second sub-fused feature vector serves as key information and value information.

[0017] In some embodiments, inputting the facial image information into a preset facial video coding network includes: training a video visual coding network to be trained based on a preset facial image dataset to generate the facial video coding network, so as to input the facial image information into the trained facial video coding network and output the coding feature vector.

[0018] Secondly, this application also provides a psychological stress detection device, the psychological stress detection device comprising:

[0019] The image acquisition unit is used to acquire facial image information of the user to be detected.

[0020] An information input unit is used to input the facial image information into a preset facial video coding network, wherein the facial video coding network encodes and represents the facial image information and outputs an encoded feature vector;

[0021] An action detection unit is used to detect whether the facial image information contains a preset facial action, and output the action feature vector corresponding to the facial image information based on the detection result corresponding to the preset facial action.

[0022] An emotion detection unit is used to detect whether the facial image information contains a preset facial emotion, and outputs an emotion feature vector corresponding to the facial image information based on the detection result corresponding to the preset facial emotion.

[0023] The stress detection unit is used to generate the psychological stress detection result corresponding to the user to be detected based on the encoded feature vector, action feature vector and emotion feature vector, and to complete the psychological stress detection of the user to be detected.

[0024] Thirdly, this application also provides a computer device, the computer device including a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and, when executing the computer program, implement the psychological stress detection method as described above.

[0025] Fourthly, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the psychological stress detection method described above.

[0026] This application discloses a method, apparatus, device, and storage medium for detecting psychological stress. The method uses devices such as cameras to capture facial images of users (which can be derived from multiple frames of a video) for subsequent feature analysis. This process can be combined with a pre-set question-and-answer scenario for psychological stress counseling, simultaneously capturing facial images while the user answers questions to ensure that the acquired images are correlated with real stress-related emotions / movements. The facial image information is then input into a pre-trained facial video coding network (such as ViViT (Video Vision)). The Transformer (video visual coding network) temporally encodes dynamic facial images (multi-frame sequences), outputting encoded feature vectors that characterize overall facial dynamic changes and capture the temporal correlation of facial expressions (such as the persistence and changing patterns of micro-expressions). Based on preset facial action units (such as downturned corners of the mouth, raised eyebrows, etc.) defined by a facial action coding system (such as FACS), it detects whether each frame of the facial image contains these actions. A binary feature matrix is ​​generated based on the detection results (action presence / absence), and after encoding, an action feature vector is obtained, representing anatomical action features related to psychological stress. For preset facial emotion categories (such as sadness, anxiety, tension, etc.), the emotional expression of each frame is detected, generating an emotion feature matrix (binary representation of emotion presence / absence), which is then encoded to obtain an emotion feature vector, representing stress-related emotional states. The encoded feature vectors (overall dynamic features), action feature vectors (anatomical action features), and emotion feature vectors (emotional expression features) are fused, and a comprehensive detection result is generated through feature interaction analysis (such as cross-attention mechanisms), achieving a quantitative assessment of psychological stress.

[0027] The proposed method eliminates the need for wearing physiological sensors (such as heart rate monitors), completing detection solely through facial image capture via camera. This reduces costs and avoids the discomfort of contact-based detection, making it suitable for various scenarios including psychological counseling, employee monitoring, and education. It can simultaneously capture images during the question-and-answer process related to stress, ensuring detection is completed within natural conversation, enhancing user experience and data accuracy. By capturing temporal changes (such as expression duration and movement continuity) across multiple frames using a facial video coding network, it overcomes the lack of dynamic information in existing static image analysis, more accurately reflecting the dynamic evolution of psychological stress. Simultaneously, it analyzes facial motion units (based on objective motion classification using FACS) and subjective emotional expressions, extracting stress-related features from two levels: "motion details" (such as the muscle movement of drooping corners of the mouth) and "emotional categories" (such as sadness), avoiding the limitations of single-dimensional analysis.

[0028] By leveraging the complementary strengths of three feature types—encoding feature vectors (overall dynamics), action feature vectors (local actions), and emotion feature vectors (emotional labels)—a more comprehensive representation of stress characteristics is constructed, covering multiple layers of information from macroscopic temporal sequence to microscopic actions and from objective physiological manifestations to subjective emotional expression. Through fusion strategies such as self-attention mechanisms, complex relationships between features of different dimensions are modeled (e.g., the co-occurrence of specific actions and emotions on stress), significantly improving feature utilization efficiency and detection accuracy compared to traditional simple concatenation or weighted fusion.

[0029] By combining computer vision (video coding), psychology (FACS action units, emotion classification), and deep learning (feature fusion), an interdisciplinary solution is formed, providing a new technical path for psychological stress detection. Furthermore, pre-training and fine-tuning mechanisms can optimize the facial video coding network, adapting it to different user groups and scenarios, improving model robustness, and avoiding overfitting issues on a single dataset.

[0030] In summary, the proposed method, through a three-layer architecture of "dynamic video coding + multi-dimensional feature detection + intelligent fusion," overcomes the limitations of existing technologies that rely on special equipment and single-dimensional analysis. It achieves non-contact, high-precision psychological stress detection, demonstrating significant improvements in convenience, comprehensiveness, and accuracy, and possesses broad practical application value.

[0031] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0032] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 This is a schematic flowchart illustrating the steps of a psychological stress detection method provided in an embodiment of this application;

[0034] Figure 2 This is a schematic flowchart illustrating the steps of a unique identifier information generation method provided in an embodiment of this application;

[0035] Figure 3 This is a schematic block diagram of a psychological stress detection device provided in an embodiment of this application;

[0036] Figure 4 This is a schematic block diagram of the structure of a computer device provided in an embodiment of this application.

[0037] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Detailed Implementation

[0038] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0039] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0040] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0041] It should also be understood that the term "and / or" as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0042] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0043] Psychological stress is a significant factor affecting individual mental health in modern society. Prolonged stress can lead to psychological problems such as anxiety and depression, and even harm physical health. Timely and accurate detection of psychological stress is crucial for mental health intervention and management. Traditional methods for detecting psychological stress primarily rely on physiological signal acquisition devices, such as heart rate monitors and skin conductance sensors. While these methods can obtain physiological indicators, they have significant drawbacks: Firstly, the use of specialized equipment increases detection costs and requires users to wear or touch the devices, making them inconvenient and difficult to popularize in daily life. Secondly, physiological signal detection is easily affected by environmental factors (such as skin condition and exercise) and cannot directly reflect key psychological stress characteristics such as an individual's emotional expression and facial micro-expressions.

[0044] With the development of computer vision technology, psychological stress detection based on facial expressions has gradually become a research hotspot. However, most existing technologies remain at the level of static facial image analysis, assessing stress only through a single dimension (such as facial emotion classification) or simple features (such as local expression intensity), which has the following shortcomings:

[0045] 1. Lack of dynamic features: Static images cannot capture the dynamic process of facial expressions changing over time (such as the duration of micro-expressions and the temporal changes of action units), and this dynamic information is an important basis for the detection of psychological stress.

[0046] 2. Insufficient multi-dimensional analysis: Existing methods do not combine the collaborative analysis of facial motion units (such as anatomical motion units defined by FACS), emotional expression and overall facial video coding features. The feature extraction dimension is single and it is difficult to fully reflect the complex correlation of psychological stress.

[0047] 3. Simple feature fusion method: Traditional methods often use basic fusion strategies such as splicing and weighted averaging, which cannot effectively model the interaction relationship between features of different dimensions, resulting in limited detection accuracy.

[0048] To resolve the above issues, please refer to [link / reference]. Figure 1 , Figure 1 This is a schematic flowchart illustrating the steps of a psychological stress detection method according to an embodiment of this application. The method can be implemented using a computer device, which can be deployed on a single server or a server cluster. It can also be deployed on a handheld terminal, laptop, wearable device, or robot, etc.

[0049] It should be noted that the method provided in this application embodiment can be widely applied to various high-concurrency scenarios, such as medical and educational scenarios. This application embodiment does not limit the application scenarios of the provided method.

[0050] Furthermore, the collection and use of user information involved in the methods provided in this application are all carried out with the authorization and permission of the relevant users, and will not infringe on the users' privacy.

[0051] like Figure 1 As shown, this psychological stress detection method specifically includes steps S101 to S105:

[0052] Step S101: Obtain facial image information of the user to be detected.

[0053] Specifically, this step involves collecting the user's facial video data in real time using a camera, with the core being the dynamic capture of facial expression changes in natural conversation scenarios.

[0054] For example, during video capture, the system or a human guides the user to answer stress-related questions (such as a doctor asking, "Have you been feeling anxious lately?" in a medical setting, or a teacher asking, "Do you feel nervous before an exam?" in an educational setting). While the user answers, the camera continuously captures facial video at a preset frame rate (e.g., 25 frames per second), forming a video sequence containing K-frame images. The camera can be integrated into computers, mobile phones, smart terminals, and other devices, supports indoor / outdoor environments, and has adaptive light adjustment capabilities to ensure clear facial images.

[0055] Simultaneously, face detection (such as the MTCNN algorithm) and face alignment can be performed on the original video, standardized facial regions (such as 112×112 pixels) can be cropped, background interference can be removed, and a preprocessed facial video sequence can be generated.

[0056] For example, when applied in a medical setting, in a psychological counseling room or online medical platform, when a patient is conversing with a psychologist, the system automatically captures facial video of the patient (with the patient's authorization) and simultaneously records the conversation (optional). The doctor guides the patient to express their emotions through preset questions (such as "What physical reactions do you experience when you are under a lot of work stress lately?"), and the video capture covers the entire process of the patient's answer, capturing micro-expression changes (such as drooping corners of the mouth, furrowed brows, and other stress-related expressions).

[0057] For example, in educational settings, during classroom interactions or one-on-one communication after class, teachers can guide students to express themselves by asking questions (such as, "Even after repeated explanations of this problem, do you still find it difficult?"), and the system can capture facial video of the student's response. For online teaching, where students use devices with cameras to participate in the course, the system automatically captures video during interactive sessions, avoiding disruption to the normal teaching process.

[0058] By capturing video during conversational scenarios, continuous facial movements of users in a natural state of expression can be obtained, which better reflects the time-series characteristics of emotional changes (such as the duration and intensity fluctuations of stress) compared to static images. No sensors or special equipment are required from the user; data collection can be completed using only a regular camera, increasing user acceptance and making it suitable for natural interaction needs in medical privacy protection and educational scenarios.

[0059] Step S102: Input the facial image information into a preset facial video coding network, whereby the facial video coding network encodes and represents the facial image information and outputs a coded feature vector.

[0060] Specifically, a Video Vision Transformer (ViViT) network is used to perform global feature encoding on the preprocessed facial video sequence. This can be achieved using a pre-trained ViViT model (e.g., pre-trained on the large-scale video dataset Kinetics-400), with input being a sequence of K facial images and output being an H-dimensional encoded feature vector A (embedding vector). A self-built stress-related facial video dataset (containing samples of different stress levels) is used to fine-tune the pre-trained ViViT model, optimizing its encoding ability for stress-related facial dynamic features (such as muscle movement trajectories and expression duration). End-to-end supervised learning is used during training, with stress level labels as the input loss function (e.g., cross-entropy loss). The encoded feature vector A integrates spatiotemporal information from the video sequence, including spatial features of a single frame (such as facial contours and texture) and temporal features across frames (such as the speed and amplitude of expression changes).

[0061] For example, in medical applications, during medical data processing, a patient's facial video is input into a finely tuned ViViT network to generate encoded features reflecting their overall facial dynamics. For instance, anxious patients may exhibit dynamic features such as frequent blinking and facial muscle tension when answering stressful questions. ViViT can capture these cross-frame changes and output corresponding feature vectors, providing global temporal features for subsequent stress level analysis.

[0062] For example, when applied in educational scenarios, ViViT can identify facial expression patterns of inattention or high stress in student videos (such as prolonged eye wandering and persistent drooping of the mouth). The generated encoded feature vectors can reflect the emotional stability of students when answering questions, helping teachers to judge the students' learning stress levels.

[0063] ViViT models the dependencies between video frames using the Transformer mechanism, making it better at capturing temporal features of facial expression changes compared to traditional 2D convolutional networks, thus improving the temporal analysis capabilities of stress detection. It initializes the network using a pre-trained model, reducing reliance on large-scale labeled data, making it suitable for scenarios in the medical and educational fields where data annotation costs are high.

[0064] Step S103: Detect whether the facial image information contains a preset facial action, and output the action feature vector corresponding to the facial image information based on the detection result corresponding to the preset facial action.

[0065] Specifically, stress-related action unit (AU) features can be extracted based on the Facial Action Coding System (FACS). For example, AU detection can be performed by using a dedicated FACS algorithm (such as the OpenFace library) to detect M stress-related AUs in each frame of the video (e.g., AU12 upward-turned corners of the mouth, AU15 downward-turned corners of the mouth, AU4 downward-turned eyebrows, etc.). Each AU is marked as 1 when it appears in the frame and 0 otherwise, generating a K×M binary matrix (K is the number of frames, M is the number of AUs). Temporal coding involves inputting the K×M matrix into a Transformer Encoder network, training the model through unsupervised masked prediction, learning the temporal sequence patterns of AU occurrences (e.g., frames where AU15 continuously appears under stress), and outputting an H-dimensional action feature vector B. By analyzing psychological research literature and labeled data, combinations of AUs significantly associated with stress (e.g., AU4 + AU15 often correspond to negative emotions) are selected to reduce interference from irrelevant features.

[0066] For example, in a medical setting, during psychotherapy, the system detects AU combinations in a patient's video. For instance, identifying persistent AU4 (brows pulled down) and AU15 (corners of the mouth pulled down) indicates potential anxiety or stress. The action feature vector B quantifies the frequency and duration of these action units, helping doctors determine the intensity and persistence of stress.

[0067] For example, when applied in an educational setting, if frequent AU20 (horizontal stretching of the corners of the mouth, corresponding to a tense expression) and AU25 (separation of the lips, corresponding to anxiety) are detected when students answer questions, the action feature vector B will reflect the temporal distribution of these stress-related actions. Teachers can use this to determine whether students are under significant stress during learning tasks (such as tension when solving problems).

[0068] AU detection based on FACS is supported by psychological theory, ensuring the scientific nature of feature extraction and avoiding subjective judgment bias. By capturing the dynamic correlation of AUs in the time dimension through Transformer Encoder (such as the coordinated occurrence of multiple AUs during a stressful outburst), it can better reflect the logic of emotional changes than single-frame detection.

[0069] Step S104: Detect whether the facial image information contains a preset facial emotion, and output the emotion feature vector corresponding to the facial image information based on the detection result corresponding to the preset facial emotion.

[0070] Specifically, facial emotion features related to stress are extracted. N basic emotions strongly associated with stress (such as anger, sadness, fear, boredom, and tension) are selected, and an emotion recognition model (such as a CNN network trained on the FER+ dataset) is used to detect the emotion category for each frame of the video. When the corresponding emotion appears, it is marked as 1, generating a K×N binary matrix. Similar to action features, the K×N matrix is ​​input into a Transformer Encoder network, which captures the temporal transition patterns of emotions (such as the transition from calm to tension) through unsupervised learning, outputting an H-dimensional emotion feature vector C. Multiple emotions are allowed in a single frame (such as "tension + worry"), and the binary matrix preserves the information of complex emotions, avoiding information loss from single emotion labels.

[0071] For example, when applied in a medical setting, if consecutive frames of "sadness + tension" are detected in a patient's video, the emotion feature vector C will highlight the duration of such complex emotions, helping doctors identify the emotional triggers behind stress (such as tension reactions accompanying depressive tendencies).

[0072] For example, when applied in educational scenarios, if students experience conflicting emotions of "fear + boredom" during exams or classroom presentations (such as fear of failure but lack of interest in the content), the emotional feature vector C can quantify this complex emotional state, allowing teachers to adjust teaching content or communication methods accordingly (such as simplifying task difficulty or increasing interactivity).

[0073] It supports simultaneous detection of multiple emotions, more closely resembling real-life emotional expressions (stress is often accompanied by a mixture of emotions), thus enhancing the semantic richness of the detection model. It captures emotional fluctuation trajectories through temporal encoding (such as the abrupt emotional shifts when stress suddenly increases), providing a time anchor for real-time intervention (e.g., timely pausing of teaching in educational settings to soothe students' emotions).

[0074] Step S105: Generate the psychological stress detection result corresponding to the user to be detected based on the encoded feature vector, action feature vector and emotion feature vector, and complete the psychological stress detection of the user to be detected.

[0075] Specifically, a cross-attention mechanism is used to fuse three types of features (encoded feature A, action feature B, and emotion feature C). For example, using action feature B as the query and encoded feature A as the key and value, a fused feature W is generated through cross-attention, focusing on the spatiotemporal localization of action units in global video coding (e.g., which frame segments' AU changes are strongly correlated with overall facial expression dynamics). Using emotion feature C as the query and encoded feature A as the key and value, a fused feature V is generated, focusing on the correlation between emotional expression and global facial expression dynamics (e.g., facial movement patterns corresponding to tension). Using fused feature W as the query and fused feature V as the key and value, a final fused feature U is generated, capturing the interaction between action and emotion (e.g., the co-occurrence pattern of specific AU combinations and emotion categories under stress). Feature U is input into a deep neural network (DNN), which outputs four-class classification results (no stress, mild, moderate, and high stress) through fully connected layers and a softmax layer. End-to-end supervised learning is used during training to optimize the cross-entropy loss function.

[0076] For example, when applied in medical scenarios, in medical diagnosis, the fusion feature U can integrate the patient's dynamic facial expression coding (such as the frequency of frowning), stress-related AU (such as the duration of AU4), and complex emotions (such as anxiety + sadness). After the DNN model outputs the stress level, doctors can combine it with clinical experience to formulate intervention plans (such as prioritizing cognitive behavioral therapy for patients with high stress).

[0077] For example, when applied in educational scenarios, in classroom stress monitoring, the system generates students' stress levels in real time. If "moderate stress" is detected, teachers can analyze the specific causes through feature U: for example, the action feature shows frequent AU15 (corner of the mouth drooping), the emotional feature shows "tense", and combined with the dynamic of eye avoidance in the coding feature, it can be judged that the student may be under pressure due to the difficulty of the question, so as to adjust the difficulty of the question or provide prompts.

[0078] The cross-attention mechanism explicitly models the dependencies between features of different dimensions (such as how action units affect emotional expression and how emotional changes are mapped to global facial expression dynamics), avoiding the information isolation problem of simply splicing features and improving detection accuracy. Through multi-dimensional feature fusion, the model can adapt to different needs in medical and educational scenarios (medical focus on emotional trigger analysis, while education focuses on real-time intervention guidance), without the need to train models separately for different scenarios, thus reducing deployment costs.

[0079] When applied in medical settings, the method provides mental health professionals with an objective and quantifiable stress assessment tool, aiding in the diagnosis of stress-related mental illnesses such as anxiety and depression. Dynamic feature analysis identifies stress triggers (such as sudden facial expressions caused by specific questions in conversations), optimizing treatment plans. Non-contact detection protects patient privacy, and multi-dimensional features avoid misjudgments based on a single indicator (e.g., distinguishing between physiological facial expression changes and psychological stress responses), improving diagnostic accuracy (compared to single heart rate monitoring, combining facial dynamics with the detection results in a 30%+ improvement in F1 score).

[0080] When applied in educational settings, the system can help teachers monitor students' classroom stress levels in real time and identify learning difficulties (such as inattention caused by stress). For online teaching, the system can automatically flag high-stress students and prompt teachers to provide personalized guidance. It offers low-cost deployment (utilizing existing cameras) and supports monitoring of large student groups. Dynamic feature analysis can distinguish between short-term stress (such as nervousness during exams) and long-term stress (such as persistent low mood), providing a basis for tiered intervention (e.g., providing immediate encouragement to students with short-term stress and referring students with long-term stress to psychological counseling).

[0081] No specialized sensors are required, lowering the barrier to entry and making it suitable for areas with insufficient medical resources and educational settings. Through feature fusion, key stress-related facial expression patterns can be traced (e.g., "AU4+AU15 persistent appearance + tension" corresponds to moderate stress), providing transparent evidence for human decision-making and enhancing the credibility of applications.

[0082] In some embodiments, obtaining the facial image information of the user to be tested includes: asking the user to be tested questions according to preset psychological stress counseling questions; and obtaining the facial image information corresponding to the user's answers to the psychological stress counseling questions.

[0083] This embodiment aims to guide users to generate natural responses through pre-set psychological stress counseling questions, while simultaneously collecting facial image information of their answers to provide raw data for subsequent stress detection. The core logic is to stimulate the user's psychological state through specific questions, making stress-related characteristics such as facial expressions and micro-movements more prominent.

[0084] By combining psychological theories (such as stress assessment scales) and historical data, we design closed-ended / open-ended questions covering multiple dimensions such as work stress, emotional distress, and social stress. Examples include, "Have you felt anxious about deadlines in the past week?" and "What is your first reaction when faced with criticism from others?". These questions are posed to users via text or voice through an interactive interface (such as an app, webpage, or smart device), supporting single-turn or multi-turn dialogues (adjusted according to the depth of stress assessment).

[0085] Facial image information acquisition uses a camera (such as a mobile phone front camera or a professional camcorder) to record the user's facial video in real time while answering questions. A frame rate of ≥15fps is recommended to capture micro-expression details. Questioning and image acquisition are synchronized through timestamps to ensure that each frame corresponds to a specific question context. For example, the frowning action when the user answers "yes" is strongly correlated with the question. The original video is then subjected to face detection (such as the MTCNN algorithm), face alignment (standardizing facial poses), and video frame extraction (extracting at fixed intervals or dynamic keyframes) to generate a sequence of facial images to be processed.

[0086] By guiding users through questions, the system places them in realistic stress assessment scenarios, avoiding feature sparsity caused by aimless data collection and improving the quality of input data for subsequent detection models. Questions directly correspond to facial reactions, facilitating subsequent analysis of stress-related facial features triggered by specific questions (e.g., furrowed brows corresponding to anxiety), providing causal evidence for multimodal stress detection. Interactive questioning enhances the targeting of the detection process, reduces the mechanical feeling of passive user cooperation, makes facial expressions more natural and realistic, and reduces data noise.

[0087] In some embodiments, the number of preset facial movements is multiple; such as Figure 2 As shown, the step of detecting whether the facial image information contains a preset facial action and outputting the action feature vector corresponding to the facial image information according to the detection result corresponding to the preset facial action includes steps S103a to S103c.

[0088] Step S103a. The facial image information is input to multiple facial motion units respectively. Each facial motion unit parses the facial image information and outputs the motion detection result corresponding to the facial image information. Each facial motion unit corresponds to a preset facial motion.

[0089] Step S103b. Generate a motion feature matrix based on the motion detection results output by the facial motion unit;

[0090] Step S103c. Encode the action feature matrix to generate the action feature vector.

[0091] Steps S103a to S103c are based on the Facial Action Coding System (FACS). The facial image is input into multiple independent facial action units (each unit corresponds to a preset facial action, such as blinking or drooping of the corner of the mouth). By detecting the existence or intensity of each action, an action feature matrix is ​​generated and encoded into a vector to represent the quantitative features of facial muscle movement.

[0092] By selecting basic action units (AUs) from FACS, such as AU1 (inner eyebrow raise), AU4 (eyebrow lower), and AU12 (corner of mouth raise), a preset action set is formed (usually containing 10-20 core AUs, covering key areas such as the forehead, eyes, and mouth). Each unit is an independent detection model (such as a lightweight CNN, like MobileNet), with a single frame of facial image as input (preprocessed to a fixed size, such as 128×128), and output as a binary classification result (whether the action occurred, or a probability value), or a regression value (action intensity, a continuous value from 0 to 1). Using datasets labeled with FACS-AU (such as CK+, BP4D), the cross-entropy loss function or the mean squared error loss function is used to ensure that each unit focuses on the detection of specific actions. For multi-frame facial image sequences, each unit outputs the detection result for each frame, forming an original matrix with dimensions of "number of frames × number of preset actions" (e.g., 100 frames × 15 actions, each element being a probability value of 0-1). The original matrix is ​​encoded using a fully connected layer or a temporal model (such as LSTM) to compress dimensionality while preserving temporal action features, ultimately generating a fixed-length action feature vector (e.g., 256-dimensional). Based on FACS, action units enable refined detection of facial muscle movements, avoiding the coarse-grained limitations of traditional expression recognition which only focuses on overall emotion, and capturing subtle, stress-related hidden features such as micro-movements (e.g., slight twitching of one corner of the mouth). Each action unit is trained independently and supports flexible expansion or replacement; for example, an AU17 (chin-raising) detection unit can be added to adapt to different stress scenarios, improving model robustness. Multi-frame detection generates and encodes matrices, effectively capturing dynamic changes in facial movements (e.g., the process of brows changing from relaxed to furrowed), providing a data foundation for temporal analysis of stress levels.

[0093] In some embodiments, the number of preset facial emotions is multiple; the facial image information includes multiple frames of facial images; the step of detecting whether the facial image information contains preset facial emotions and outputting the emotion feature vector corresponding to the facial image information based on the detection results corresponding to the preset facial emotions includes: detecting the emotion detection result corresponding to each frame of the facial image based on each preset facial emotion; generating an emotion feature matrix based on the emotion detection results corresponding to each preset facial emotion; and encoding the emotion feature matrix to generate the emotion feature vector.

[0094] For facial videos containing multiple frames, this embodiment detects the preset facial emotions (such as anxiety, tension, calmness) in each frame, generates the emotion probability distribution for each frame, and further integrates them into an emotion feature matrix and encodes it into a vector to represent the emotional change patterns of the user during the process of answering questions.

[0095] Combining psychological emotion classification (such as the Plutchik emotion wheel) and stress detection needs, define target emotion categories, such as "anxiety," "tension," "confusion," "calm," and "relaxation," forming a set of 5-8 preset emotion categories.

[0096] Single-frame emotion detection employs an end-to-end emotion recognition model (e.g., ResNet+LSTM, or fine-tuned pre-trained model). It takes a single-frame facial image as input and outputs the probability distribution of each preset emotion (e.g., the vector [0.7, 0.2, 0.1] represents 70% anxiety, 20% tension, and 10% other). Using emotion-labeled datasets (e.g., FER+, AffectNet), transfer learning is combined to optimize the detection accuracy for stress-related emotions (e.g., anxiety). The emotion probability vectors for each frame are arranged chronologically to form an emotion feature matrix with dimensions of "frame number × preset emotion number" (e.g., 100 frames × 5 emotions, with each row representing the emotion distribution for one frame). Through a temporal convolutional network (TCN) or global average pooling, the matrix is ​​compressed into a fixed-length emotion feature vector (e.g., 256 dimensions), preserving the evolution trend of emotions over time (e.g., the probability of anxiety gradually increases).

[0097] Unlike single-frame emotion recognition, this method captures continuous changes in emotion (such as abrupt shifts from calm to tension) through multi-frame detection, more closely reflecting the dynamic process of psychological stress. It complements action features (actions represent muscle movement, while emotions represent overall state); for example, the combined feature of "frowning (action) + anxiety (emotion)" is more discriminative for stress detection. By averaging multi-frame probability distributions to process single-frame noise (such as detection errors caused by blinking), the stability of the emotion feature vector is ensured, making it particularly suitable for uncontrolled environments (such as changes in lighting or slight user movement).

[0098] In some embodiments, the encoded feature vector, action feature vector, and emotion feature vector have the same dimension; generating the psychological stress detection result corresponding to the user to be detected based on the encoded feature vector, action feature vector, and emotion feature vector includes: performing feature fusion on the encoded feature vector, action feature vector, and emotion feature vector based on a self-attention mechanism to generate a fused feature vector; inputting the fused feature vector into a preset psychological stress detection model, the psychological stress detection model parsing the fused feature vector and outputting the psychological stress detection result.

[0099] Under the premise that the dimensions of the encoded feature vector (the overall video feature generated in Example 6), the action feature vector (Example 2), and the emotion feature vector (Example 3) are the same, the interaction relationship of the three types of features is dynamically captured through the self-attention mechanism to generate a fused feature vector, which is input into the preset model to output the psychological stress detection result (such as stress level: low, medium, high).

[0100] To ensure the three types of feature vectors have consistent dimensions (e.g., all 256-dimensional), a fully connected layer is used to adjust for initial dimensionality mismatches (e.g., the original 512-dimensional encoding vector is reduced to 256 dimensions via linear transformation). The three types of vectors are used as combinations of query, key, and value, respectively, and attention weights are calculated to capture dependencies between features. Q, K, and V matrices are generated: the encoding vector (E), action vector (A), and emotion vector (M) are concatenated into a feature matrix F = [E; A; M], and linear projection yields Q = FW_Q, K = FW_K, and V = FW_V (W is the learnable parameter matrix). Attention scores are calculated: the weight matrix α = softmax(QK^T / √d), where d is the dimension (256), avoiding excessively large values. A fusion vector F' = αV is generated, and gradient stability is maintained through residual connections and layer normalization.

[0101] The stress detection model uses a pre-defined multilayer perceptron (MLP) or Transformer decoder for inference, taking a fused feature vector as input and outputting stress detection results (numerical scores for regression tasks and probability distributions for classification tasks). Model training uses a dataset labeled with stress levels (e.g., psychological scale ratings as supervision signals) and employs mean squared error (mean squared error) or cross-entropy (cross-entropy) loss functions to optimize end-to-end performance.

[0102] Self-attention mechanisms allow actions, emotions, and encoded features to interact with each other. For example, head pose information in encoded features can enhance the accuracy of action features in locating micro-expressions, creating synergistic gains among features. For different users or problem scenarios, the attention mechanism automatically adjusts the importance of the three types of features (e.g., some users' stress is mainly reflected in changes in movement, while others are reflected in emotional fluctuations), improving the model's generalization ability. Unifying feature dimensions avoids information loss, ensures the mathematical feasibility of fusion operations, and simplifies model design (eliminating the need for complex dimension transformation layers).

[0103] For example, the feature fusion of the encoded feature vector, action feature vector, and emotion feature vector based on the self-attention mechanism includes: fusing the encoded feature vector and action feature vector based on cross-attention to obtain a first sub-fused feature vector; wherein, in the fusion of the first sub-fused feature vector, the action feature vector serves as query information, and the encoded feature vector serves as key information and value information; fusing the emotion feature vector and action feature vector based on cross-attention to obtain a second sub-fused feature vector; wherein, in the fusion of the second sub-fused feature vector, the action feature vector serves as query information, and the emotion feature vector serves as key information and value information; fusing the first sub-fused feature vector and the second sub-fused feature vector based on cross-attention to obtain the fused feature vector; wherein, in the fusion of the fused feature vector, the first sub-fused feature vector serves as query information, and the second sub-fused feature vector serves as key information and value information.

[0104] This example details a specific implementation of the self-attention mechanism, which fuses features in stages through three cross-attention operations: first, the action vector is used as the query to fuse the encoding vector; then, the action vector is used as the query to fuse the emotion vector; and finally, the sub-fusion vectors of the first two are used as the query for final fusion, forming a hierarchical feature integration.

[0105] Phase 1: Encoding-Action Feature Fusion (Obtaining the first sub-fusion vector F1); Query: Action feature vector A (256-dimensional); Key, Value: Encoding feature vector E (256-dimensional, same dimension as A); Operation steps: Q_A = A*W_Q1, K_E = E*W_K1, V_E = E*W_V1 (W_Q1, W_K1, W_V1 are learnable matrices, 256×256 in dimension, keeping the dimension unchanged); Attention score α1 = softmax(Q_A*K_E^T / √256); F1 = α1*V_E+A (residual connection, preserving the original action features); Layer normalization: F1 = LN(F1).

[0106] Second stage: Emotion-action feature fusion (obtaining the second sub-fusion vector F2): Query: Action feature vector A (same as the first stage, maintaining consistency); Key, Value: Emotion feature vector M (256 dimensions); The operation steps are similar to the first stage, replacing K / V with the projection matrix of M to obtain F2 = α2*V_M+A, which is then output after layer normalization.

[0107] Phase 3: Sub-fusion vector deep fusion (obtaining the final fusion vector Fusion): Query: First sub-fusion vector F1; Key, Value: Second sub-fusion vector F2. Corresponding operation steps: Q_F1=F1*W_Q3, K_F2=F2*W_K3, V_F2=F2*W_V3; Attention score α3=softmax(Q_F1*K_F2^T / √256); Fusion=α3*V_F2+F1 (residual connection);

[0108] The final normalized layer is then input into the pressure detection model.

[0109] By using a fixed action vector as the initial query, the model is forced to prioritize the correlation between action features and other modalities (since action is one of the most direct physiological responses to stress), aligning with the causal chain logic of "stress → muscle tension → facial movements" in psychology. The first and second stages strengthen the local correlations between action and encoding, and action and emotion, respectively. The third stage integrates global correlations, avoiding feature contamination caused by direct fusion and improving the hierarchical nature of feature expression (similar to the "part → whole" process in human cognition). Cross-attention weights allow analysis of the contribution of different features (e.g., a high F1 weight for a user indicates that their stress is more dependent on the action-encoding feature correlation), providing direction for subsequent model optimization.

[0110] In some embodiments, inputting the facial image information into a preset facial video coding network includes: training a video visual coding network to be trained based on a preset facial image dataset to generate the facial video coding network, so as to input the facial image information into the trained facial video coding network and output the coding feature vector.

[0111] This embodiment trains a video visual coding network using a pre-set facial image dataset, enabling it to extract high-level semantic features (encoded feature vectors) from facial videos to represent the overall structural information of the video (such as head movement trajectory and spatiotemporal changes in facial regions).

[0112] By collecting facial videos from multiple scenarios (including samples with different stress levels), stress labels were added (e.g., scores from 1 to 10 using professional psychological assessment scales). Strategies such as random cropping, brightness adjustment, and time jitter (randomizing frame order) were employed to improve model generalization and avoid overfitting.

[0113] Employing 3D CNNs (such as C3D and I3D) or 2D CNN+LSTM (for processing temporal information), the input consists of N consecutive frames of facial images (e.g., N=16), and the output is a fixed-dimensional encoding vector (e.g., 256-dimensional). Stress labels are used as supervisory signals (for regression tasks), or contrastive learning objectives are designed (e.g., narrowing the encoding distance between samples of the same stress level and widening the distance between samples of different stress levels) to improve the discriminative power of the encoding vectors.

[0114] During the training phase, input video clips are used to obtain encoded vectors through forward propagation of the network. The loss (e.g., MSE) with respect to the labels is calculated, and the network parameters are updated through backpropagation until convergence. During the inference phase, the facial video of the user to be detected is input into the trained network, and the encoded feature vector is directly output as a high-level semantic representation of the overall spatiotemporal features of the video.

[0115] To avoid the tediousness and subjectivity of manually designing features (such as manually extracting facial keypoint coordinates), a data-driven approach is used to learn stress-related implicit features (such as complex patterns like frequent head tremors). 3D CNN or LSTM structures effectively capture the temporal dynamics of facial expressions (such as the duration of a smile and changes in blink frequency), and these temporal features are crucial for stress detection (static images struggle to reflect the duration of stress). Encoded vectors, as video-level global features, can complement local features such as actions and emotions, covering multi-scale information from details (single-frame actions) to the global picture (the entire video), thus enhancing the information richness of the detection model.

[0116] In some embodiments, to address the problem of "scarcity of high-stress samples" in psychological stress detection, a dedicated GAN model is designed to generate realistic stress-related facial expression images (such as micro-expression sequences of anxiety and tension), thereby expanding the diversity of the training dataset and solving the model bias problem caused by data imbalance.

[0117] The stress expression GAN architecture design includes: Generator (G): Inputting a low-dimensional random vector (containing stress intensity encoding, such as continuous values ​​from 0 to 1), it generates a facial image (128×128×3) through multiple transposed convolutions, and introduces conditional input (such as the semantic vector of the corresponding question text) to ensure that the generated expression is context-dependent. Discriminator (D): Dual-task output—① Determining whether the image is a real sample or a generated sample; ② Regressing and predicting the stress intensity label corresponding to the image, guiding the generator to learn stress-related features (such as the mapping relationship between the tightness of the corners of the mouth and the stress value).

[0118] Adversarial training and data fusion include: Training data: Based on real stress video datasets (such as BP4D-Spontaneous), the stress intensity of each frame is labeled through feature engineering (weighted calculation combining psychological scale scores and facial movement intensity). Conditional constraints: BERT encoded vectors of the question text are added to the generator input, ensuring that generated expressions only appear in specific question contexts (e.g., a combination of frowning and wandering eyes for a "work deadline" question), avoiding meaningless generation. Data augmentation: Valid generated samples (after manual screening) are mixed into the original training set at a 1:1 ratio to balance the distribution of high and low stress samples.

[0119] Temporal consistency enhancement is achieved by introducing video-level GANs (such as VGAN) to ensure that the generated multi-frame image sequence conforms to the temporal logic of facial movements (such as the eyebrows not instantly changing from relaxed to furrowed), and by optimizing the inter-frame motion coherence through optical flow constraint loss function.

[0120] To address the challenge of obtaining realistic high-stress samples in clinical-grade stress detection, this approach enhances the model's ability to recognize extreme stress states by controllably generating supplementary data. By combining question text with conditional facial expression generation, the augmented data more closely resembles real-world detection scenarios (unlike the random generation of unconditional GANs), avoiding the introduction of irrelevant noise features. After balancing the dataset, the model's overfitting to "calm expressions" is reduced, particularly improving sensitivity to micro-expressions (such as slight eyelid tremors), making it suitable for covert stress detection.

[0121] In some embodiments, a speech modality (speech signal when answering questions) is introduced on the basis of the original facial features. A cross-modal interactive attention mechanism is used to achieve deep fusion of speech features (such as tone, speech rate, and fundamental frequency jitter) and facial features (action, emotion, and coding features) to construct a multimodal stress detection model.

[0122] Acoustic features are extracted using OpenSMILE to extract 600+ low-level features such as MFCC and Prosody, and then reduced to 128 dimensions using T-SNE. Semantic features are obtained by performing sentiment analysis on the speech text (e.g., the anxiety probability output by the BERT classifier) ​​and concatenating it with the acoustic features to form a speech feature vector (256 dimensions, aligned with the dimensions of facial features).

[0123] Voice-guided facial attention: Using voice features as the query and the concatenated vector of three types of facial features (encoding E, action A, emotion M) as the key / value pair, the attention weight of voice to facial expression is calculated (e.g., when the speech rate increases, the model should pay more attention to AU25 (lip separation) in mouth action feature A); Facial-guided voice attention: Using facial fusion features as the query and voice features as the key / value pair, voice anomalies corresponding to changes in facial expression are captured (e.g., whether the fundamental frequency of the voice increases when frowning). Collaborative fusion layer: The bidirectional attention output and the original features are weighted and fused through a gating mechanism to generate the final multimodal feature vector (256 dimensions).

[0124] By simultaneously optimizing the main stress detection task (MSE / CE loss) and the cross-modal alignment auxiliary task (such as contrastive loss, which forces speech-facial features of the same stress level to be close in the embedding space).

[0125] The tense vibrato in speech (acoustic feature) and pupil constriction in the face (motor feature) form synergistic evidence, resolving the ambiguity of a single modality (e.g., a smile may correspond to relaxation or feigned stress). An interactive attention mechanism explicitly models the correspondence between "speech content" and "facial expression / action" (e.g., the micro-expression contradiction when saying "I'm fine"), capturing the "inconsistency between words and actions" under stress and improving detection accuracy (especially suitable for scenarios where stress is feigned). When noisy environments cause speech signal distortion, the model can automatically increase the attention weight of facial features; conversely, it relies on speech features when the face is occluded, enhancing robustness.

[0126] In some embodiments, to address the "black box" problem of psychological testing models, an interpretability module is designed. Through feature attribution analysis and attention visualization, the contribution of facial movements / emotions to stress detection results is clarified, assisting psychological counselors in verifying the model's decision-making logic and enhancing user trust.

[0127] Gradient-weighted class activation mapping (Grad-CAM) is used to calculate the gradient of the CNN layer output during the action feature vector generation process, locating the region in the facial image that has the greatest impact on the stress detection result (e.g., the activation intensity of AU4 corresponding to the eyebrow region), and generating a heatmap that is superimposed on the original image. Based on the correlation coefficient (e.g., Pearson coefficient) between the output value of each facial action unit (AU detector in Example 2) and the final stress score, a "key action list" is generated (e.g., "AU4 (eyebrow down) contributes 32%, AU10 (upper lip lift) contributes 15%"). During the self-attention fusion stage, the attention weights of each frame in generating the encoded feature vector are recorded, and a "stress-sensitive frame curve" is plotted (e.g., the weight suddenly increases in frame 45, corresponding to the facial expression change when answering the core stress question). The emotion distribution of each frame in the emotion feature matrix is ​​visualized, forming a time-emotion two-dimensional heatmap, intuitively presenting the cumulative process of anxiety as the question progresses.

[0128] By developing visualization tools, users can click to view the detection results: ① heat map of key facial areas; ② Top 3 contributing action units and their occurrence frame numbers; ③ attribution curve of stress score changes over time (e.g., stress increase at a certain time period is jointly triggered by "continuous blinking + drooping corner of mouth").

[0129] This helps mental health professionals quickly pinpoint the specific manifestations of user stress (e.g., "a furrowed brow for 2 seconds" corresponds to a high-stress prediction), aiding in the design of subsequent intervention plans, rather than simply relying on model output. Users can intuitively see which of their expressions were captured by the model, reducing resistance to AI detection, especially in the healthcare field to meet compliance requirements (the algorithm's decision-making basis needs to be explained). Attribution analysis identifies model errors (e.g., incorrect focus on irrelevant actions), allowing for targeted adjustments to training data labeling or model structure, forming a closed loop of "detection-explanation-optimization."

[0130] In some embodiments, for differences in facial features among different users (such as some people habitually frowning when stressed, while others habitually blinking), meta-learning technology is used to train a "learning-to-learn" model, enabling it to quickly adapt to personalized detection patterns with a small amount of new user data, thus solving the problem of insufficient generalization ability of traditional models to individual differences.

[0131] Historical data is divided into "user tasks," with each task containing a small number of labeled samples (e.g., 5 stress video clips, each containing 10 frames) as the support set and the remaining samples as the query set. The MAML (Model-Agnostic Meta-Learning) framework is adopted, with a shared feature extractor (face encoding network + action / emotion detector) as the base network, supplemented by rapidly updatable personalized parameter layers (e.g., fusion layer weights specific to each user).

[0132] The meta-training process includes: Inner loop update (user adaptation): For new users, personalized parameters are quickly adjusted using gradient descent with their support set data (only the last 1-2 layers are updated, and the basic feature extractor is frozen), generating a temporary personalized model. Outer loop optimization (meta-learning): The base network is trained on multiple user tasks to ensure good generalization of the initial parameters, guaranteeing rapid convergence after the inner loop update (e.g., achieving 80% performance with only 5 steps of gradient descent). During the first detection of a new user, videos of 3-5 question answers are collected as the support set, and a personalized model is generated through inner loop update. Subsequent detections directly use this model to process real-time data without retraining.

[0133] To address the issue of "missed detections due to significant individual user differences," for example, for users "without a noticeable frowning habit," the model learns to focus on their unique stress characteristics (such as slight nasal flaring) using a small amount of data. Traditional models rely on statistical patterns from large amounts of cross-user data, while meta-learning models can capture the individual-specific "stress-expression" mapping relationship, making them particularly suitable for clinical scenarios where customized testing plans need to be developed for each patient.

[0134] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of the psychological stress detection device 200 provided in the embodiments of this application. The psychological stress detection device 200 is used to perform the steps of the psychological stress detection methods shown in the above embodiments. The psychological stress detection device 200 can be a single server or a server cluster, or it can be a terminal, such as a handheld terminal, a laptop computer, a wearable device, or a robot.

[0135] like Figure 3 As shown, the psychological stress detection device 200 includes:

[0136] Image acquisition unit 201 is used to acquire facial image information of the user to be detected;

[0137] The information input unit 202 is used to input the facial image information into a preset facial video coding network, wherein the facial video coding network encodes the facial image information and outputs an encoded feature vector;

[0138] The motion detection unit 203 is used to detect whether the facial image information contains a preset facial action, and output the motion feature vector corresponding to the facial image information according to the detection result corresponding to the preset facial action.

[0139] The emotion detection unit 204 is used to detect whether the facial image information contains a preset facial emotion, and output the emotion feature vector corresponding to the facial image information according to the detection result corresponding to the preset facial emotion.

[0140] The stress detection unit 205 is used to generate the psychological stress detection result corresponding to the user to be detected based on the encoded feature vector, action feature vector and emotion feature vector, and to complete the psychological stress detection of the user to be detected.

[0141] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the above-described apparatus and modules can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0142] The aforementioned device can be implemented as a computer program, which can be used in, for example... Figure 4 It runs on the computer device shown.

[0143] Please see Figure 4 , Figure 4 This is a schematic block diagram of a computer device in one embodiment. The computer device may be a server.

[0144] See Figure 4 The computer device includes a processor, memory, and network interface connected via a system bus, wherein the memory may include non-volatile storage media and internal memory.

[0145] Non-volatile storage media can store operating systems and computer programs. These computer programs include program instructions that, when executed, cause the processor to perform any psychological stress detection method.

[0146] The processor provides computing and control capabilities, supporting the operation of the entire computer device.

[0147] Internal memory provides an environment for the execution of computer programs in non-volatile storage media. When the computer program is executed by the processor, it enables the processor to perform any psychological stress detection method.

[0148] This network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0149] It should be understood that the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.

[0150] In one embodiment, the processor is configured to run a computer program stored in memory to perform the following steps:

[0151] Obtain facial image information of the user to be detected;

[0152] The facial image information is input into a preset facial video coding network, which encodes and represents the facial image information and outputs an encoded feature vector.

[0153] Detect whether the facial image information contains a preset facial action, and output the action feature vector corresponding to the facial image information based on the detection result corresponding to the preset facial action;

[0154] Detect whether the facial image information contains a preset facial emotion, and output the emotion feature vector corresponding to the facial image information based on the detection result corresponding to the preset facial emotion;

[0155] The psychological stress detection result corresponding to the user to be detected is generated based on the encoded feature vector, action feature vector and emotion feature vector, thus completing the psychological stress detection of the user to be detected.

[0156] In some embodiments, obtaining the facial image information of the user to be tested includes: asking the user to be tested questions according to preset psychological stress counseling questions; and obtaining the facial image information corresponding to the user's answers to the psychological stress counseling questions.

[0157] In some embodiments, the number of preset facial actions is multiple; the step of detecting whether the facial image information contains preset facial actions and outputting the action feature vector corresponding to the facial image information based on the detection result corresponding to the preset facial actions includes: inputting the facial image information to multiple facial action units respectively, each facial unit parsing the facial image information and outputting the action detection result corresponding to the facial image information; each facial action unit corresponds to one preset facial action; generating an action feature matrix based on the action detection result output by each facial action unit; and encoding the action feature matrix to generate the action feature vector.

[0158] In some embodiments, the number of preset facial emotions is multiple; the facial image information includes multiple frames of facial images; the step of detecting whether the facial image information contains preset facial emotions and outputting the emotion feature vector corresponding to the facial image information based on the detection results corresponding to the preset facial emotions includes: detecting the emotion detection result corresponding to each frame of the facial image based on each preset facial emotion; generating an emotion feature matrix based on the emotion detection results corresponding to each preset facial emotion; and encoding the emotion feature matrix to generate the emotion feature vector.

[0159] In some embodiments, the encoded feature vector, action feature vector, and emotion feature vector have the same dimension; generating the psychological stress detection result corresponding to the user to be detected based on the encoded feature vector, action feature vector, and emotion feature vector includes: performing feature fusion on the encoded feature vector, action feature vector, and emotion feature vector based on a self-attention mechanism to generate a fused feature vector; inputting the fused feature vector into a preset psychological stress detection model, the psychological stress detection model parsing the fused feature vector and outputting the psychological stress detection result.

[0160] For example, the feature fusion of the encoded feature vector, action feature vector, and emotion feature vector based on the self-attention mechanism includes: fusing the encoded feature vector and action feature vector based on cross-attention to obtain a first sub-fused feature vector; wherein, in the fusion of the first sub-fused feature vector, the action feature vector serves as query information, and the encoded feature vector serves as key information and value information; fusing the emotion feature vector and action feature vector based on cross-attention to obtain a second sub-fused feature vector; wherein, in the fusion of the second sub-fused feature vector, the action feature vector serves as query information, and the emotion feature vector serves as key information and value information; fusing the first sub-fused feature vector and the second sub-fused feature vector based on cross-attention to obtain the fused feature vector; wherein, in the fusion of the fused feature vector, the first sub-fused feature vector serves as query information, and the second sub-fused feature vector serves as key information and value information.

[0161] In some embodiments, inputting the facial image information into a preset facial video coding network includes: training a video visual coding network to be trained based on a preset facial image dataset to generate the facial video coding network, so as to input the facial image information into the trained facial video coding network and output the coding feature vector.

[0162] The embodiments of this application also provide a computer-readable storage medium storing a computer program, the computer program including program instructions, and the processor executing the program instructions to implement any of the psychological stress detection methods provided in the embodiments of this application.

[0163] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiments, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the computer device.

[0164] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for detecting psychological stress, characterized in that, include: Obtain facial image information of the user to be detected; The facial image information is input into a preset facial video coding network, which encodes and represents the facial image information and outputs an encoded feature vector. The system detects whether the facial image information contains a preset facial action, and outputs the action feature vector corresponding to the facial image information based on the detection result corresponding to the preset facial action; the number of preset facial actions is multiple. The system detects whether the facial image information contains a preset facial emotion, and outputs an emotion feature vector corresponding to the facial image information based on the detection result corresponding to the preset facial emotion; the number of preset facial emotions is multiple; the facial image information includes multiple frames of facial images. Based on the encoded feature vector, action feature vector, and emotion feature vector, a psychological stress detection result corresponding to the user to be detected is generated, thus completing the psychological stress detection of the user to be detected. Wherein, the encoded feature vector, action feature vector, and emotion feature vector have the same dimension; the step of generating the psychological stress detection result corresponding to the user to be detected based on the encoded feature vector, action feature vector, and emotion feature vector includes: performing feature fusion on the encoded feature vector, action feature vector, and emotion feature vector based on a self-attention mechanism to generate a fused feature vector; inputting the fused feature vector into a preset psychological stress detection model, the psychological stress detection model parsing the fused feature vector and outputting the psychological stress detection result; The feature fusion based on the self-attention mechanism of the encoded feature vector, action feature vector, and emotion feature vector includes: fusing the encoded feature vector and action feature vector based on cross-attention to obtain a first sub-fused feature vector; wherein, in the fusion of the first sub-fused feature vector, the action feature vector serves as query information, and the encoded feature vector serves as key and value information; fusing the emotion feature vector and action feature vector based on cross-attention to obtain a second sub-fused feature vector; wherein, in the fusion of the second sub-fused feature vector, the action feature vector serves as query information, and the emotion feature vector serves as key and value information; fusing the first sub-fused feature vector and the second sub-fused feature vector based on cross-attention to obtain the fused feature vector; wherein, in the fusion of the fused feature vector, the first sub-fused feature vector serves as query information, and the second sub-fused feature vector serves as key and value information.

2. The method according to claim 1, characterized in that, The acquisition of the facial image information of the user to be detected includes: The user to be tested is asked questions based on preset psychological stress counseling questions; Obtain the facial image information corresponding to the user's answer to the psychological stress counseling question.

3. The method according to claim 1, characterized in that, The step of detecting whether the facial image information contains a preset facial action, and outputting the action feature vector corresponding to the facial image information based on the detection result corresponding to the preset facial action, includes: The facial image information is input to multiple facial motion units, each facial unit parses the facial image information and outputs the motion detection result corresponding to the facial image information; each facial motion unit corresponds to a preset facial motion. A motion feature matrix is ​​generated based on the motion detection results output by each facial motion unit; The action feature matrix is ​​encoded to generate the action feature vector.

4. The method according to claim 1, characterized in that, The step of detecting whether the facial image information contains a preset facial emotion, and outputting an emotion feature vector corresponding to the facial image information based on the detection result corresponding to the preset facial emotion, includes: Based on each preset facial emotion, the emotion detection result corresponding to each frame of the facial image is detected; An emotion feature matrix is ​​generated based on the emotion detection results corresponding to each preset facial emotion; The emotion feature vector is generated by encoding the emotion feature matrix.

5. The method according to claim 1, characterized in that, The step of inputting the facial image information into a preset facial video coding network includes: The video visual coding network to be trained is trained based on a preset facial image dataset to generate the facial video coding network. The facial image information is then input into the trained facial video coding network, and the encoded feature vector is output.

6. A psychological stress detection device for implementing the method as described in any one of claims 1-5, characterized in that, The psychological stress detection device includes: The image acquisition unit is used to acquire facial image information of the user to be detected. An information input unit is used to input the facial image information into a preset facial video coding network, wherein the facial video coding network encodes and represents the facial image information and outputs an encoded feature vector; An action detection unit is used to detect whether the facial image information contains a preset facial action, and output the action feature vector corresponding to the facial image information based on the detection result corresponding to the preset facial action. An emotion detection unit is used to detect whether the facial image information contains a preset facial emotion, and outputs an emotion feature vector corresponding to the facial image information based on the detection result corresponding to the preset facial emotion. The stress detection unit is used to generate the psychological stress detection result corresponding to the user to be detected based on the encoded feature vector, action feature vector and emotion feature vector, and to complete the psychological stress detection of the user to be detected.

7. A computer device, characterized in that, The device includes a memory and a processor, wherein the memory stores computer-readable instructions that, when executed by the processor, cause the processor to perform the steps of the psychological stress detection method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that causes one or more processors to perform the steps of the psychological stress detection method as described in any one of claims 1 to 5.

Images