Mental health and safety management system based on expression recognition

Abstract

The invention discloses a psychological health and safety management system based on expression recognition. The psychological health and safety management system comprises a face data acquisition module, an edge calculation module, a cloud psychological assessment module, a dynamic risk prediction module and a grading intervention execution module. The face data acquisition module is used for acquiring a user face video stream and environmental parameters in real time, and the edge calculation module is internally provided with a lightweight expression recognition model and is used for outputting acquired face data as micro-expression feature vectors and physiological parameters. The cloud psychological assessment module is used for converting the micro-expression feature vectors and the physiological parameters into psychological state quantized values, and the dynamic risk prediction module generates a future N-hour risk curve based on a hidden Markov model. The psychological health state of the user can be comprehensively and objectively evaluated in real time based on the expression of the user, so that a personalized psychological health management scheme is formulated for the user, the psychological health management effect of the user is greatly improved, and the personal safety and work safety problems caused by psychological health are solved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of mental health management, and particularly relates to a mental health and safety management system based on expression recognition. BACKGROUND

[0002] Mental health management is to adjust and manage the mental health status of individuals through cognitive, emotional, behavioral and other intervention means, so as to improve the quality of life, stress resistance and happiness. It is essentially different from traditional "mental disease treatment", and is more focused on "prevention" and "growth", involving emotional regulation, stress coping, cognitive optimization, interpersonal relationship maintenance and other key dimensions.

[0003] The existing mental health management methods include but are not limited to emotional management, stress management, positive attitude cultivation, healthy lifestyle, social support, self-cognition and growth, and seeking professional help. Among them, when seeking professional help, the user's mental health status needs to be evaluated first, and the main basis for evaluation is the user's self-statement or the realization through filling out the scale, for example, the user answers the general mental health questionnaire, and then the user's mental condition is evaluated according to the evaluation rules and the user's answer results. However, these evaluation methods cannot be targeted to the actual psychology of the user, and there are problems of not real-time, not comprehensive and not objective, which will also affect the subsequent mental health management of the user.

[0004] Therefore, in view of the above technical problems, it is necessary to provide a mental health and safety management system based on expression recognition.

[0005] The information disclosed in this BACKGROUND section is only intended to increase an understanding of the general context in which the present application can be practiced. It should not be taken as an acknowledgement that this information forms a prior art that was known to those skilled in the art, before the filing date of the present application. SUMMARY

[0006] The purpose of the present application is to provide a mental health and safety management system based on expression recognition, which can solve the problems in the background art.

[0007] In order to achieve the above-mentioned purpose, the technical scheme provided by an embodiment of the present application is as follows: A mental health and safety management system based on expression recognition, comprising a facial data acquisition module, an edge computing module, a cloud psychological evaluation module, a dynamic risk prediction module and a hierarchical intervention execution module.

[0008] The face data acquisition module is used for acquiring a user face video stream and environment parameters in real time, the edge computing module is internally provided with a lightweight expression recognition model, which is used for outputting the collected face data as a micro-expression feature vector and physiological parameters, the cloud psychological evaluation module is used for converting the micro-expression feature vector and the physiological parameters into a psychological state quantitative value, the dynamic risk prediction module uses a user history and a current psychological state quantitative value sequence as an observation sequence based on a hidden Markov model, predicts a multi-dimensional psychological risk index change trend in the future N hours, and draws a dynamic risk curve according to the prediction result, and the graded intervention execution module triggers a differentiated safety management response strategy according to a risk level, which is used for personal state early warning, facilitates self-emotion adjustment, or is subjected to regional cloud management and control and human management and protection, so as to cope with work and study.

[0009] In one or more embodiments of the present application, the face data acquisition module comprises an image preprocessing unit, which is used for preprocessing the acquired user face video stream and environment parameters, and the preprocessing method comprises extracting target data in the face image with the same weight parameter and performing de-redundancy processing on the target data through a convolutional neural network.

[0010] The face data acquisition module can simultaneously collect expression information of one or more persons and independently analyze respectively. Meanwhile, the face data acquisition module can also identify a risk individual or a crowd, discover a non-safe place, and perform risk early warning to safety personnel or others, perform safety control on the others, and prevent non-safe events from occurring.

[0011] In one or more embodiments of the present application, the micro-expression feature vector and the physiological parameters comprise an eye periphery tremor frequency, a single-side mouth corner droop, a brow center contraction duration, a pupil diameter change rate, and a face blood flow fluctuation.

[0012] In one or more embodiments of the present application, the cloud psychological evaluation module comprises a DSM-5 mapping model, which is trained through the following steps: S1, collecting an annotated video data set of clinically diagnosed patients; S2, extracting an AU unit combination feature of a facial physiological action coding system; S3, establishing a non-linear mapping relationship between the AU combination and a depression / anxiety scale score by using an XGBoost algorithm.

[0013] In one or more embodiments of the present application, the dynamic risk prediction module updates a user state transition matrix and an observation probability matrix at a preset period, and triggers an upgrade warning when a predicted multi-dimensional psychological risk index or a specific dimension thereof continuously exceeds a corresponding threshold value for M consecutive periods, wherein the multi-dimensional psychological risk index includes but is not limited to anxiety, depression tendency, stress level, and emotion stability.

[0014] In one or more embodiments of the present application, the mental health management system further comprises an information storage module for storing data information and a privacy protection module for encrypting the information storage module, and the video data can be anonymized in the edge computing module.

[0015] In one or more embodiments of the present application, the information storage module comprises a user module and a management module, the user module stores user personal information and emergency contact information, and the management module stores mental health training courses, psychologist information and medical institution information.

[0016] In one or more embodiments of the present application, the privacy protection module uses AES symmetric encryption algorithm to protect user personal and emergency contact privacy.

[0017] In one or more embodiments of the present application, the privacy protection module uses federated machine learning to update cloud model parameters in real time.

[0018] In one or more embodiments of the present application, the hierarchical intervention execution module comprises a three-level response mechanism, wherein the first level is used to push mental health training courses, the second level is used to make an appointment with an online psychologist, and the third level is used to send positioning information and medical institution navigation to the preset emergency contact, and trigger background area cloud management and control and human management.

[0019] Compared with the prior art, the mental health and safety management system based on expression recognition can evaluate the mental health status of the user in real time, comprehensively and objectively based on the expression of the user, so as to formulate a personalized mental health management scheme for the user, and greatly improve the mental health management effect of the user and improve the personal safety and work safety problems caused by mental health. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiment or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments described in the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0021] Figure 1 The system block diagram of the mental health and safety management system based on expression recognition in an embodiment of the present application; Figure 2 The principle diagram of the mental health and safety management system based on expression recognition in an embodiment of the present application; Figure 3 DSM-5 mapping model training flowchart in an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order for those skilled in the art to better understand the technical solutions in the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, not all. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present disclosure.

[0023] As shown in the figure, an emotional recognition-based mental health and safety management system in an embodiment of the present application includes a facial data acquisition module, an edge computing module, a cloud psychological assessment module, a dynamic risk prediction module, and a hierarchical intervention execution module. Figure 1

[0024] The facial data acquisition module is used to acquire user facial video stream and environmental parameters in real time. The edge computing module is internally provided with a lightweight expression recognition model, which is used to output the collected facial data as micro-expression feature vectors and physiological parameters. The cloud psychological assessment module is used to convert the micro-expression feature vectors and physiological parameters into psychological state quantitative values. The dynamic risk prediction module uses the historical and current psychological state quantitative value sequences of the user as observation sequences to predict the change trend of multi-dimensional psychological risk indicators within N hours in the future based on a hidden Markov model, and draws a dynamic risk curve for the prediction results, which is used to identify risk inflection points and high-risk periods. The hierarchical intervention execution module triggers differentiated safety management response strategies according to the risk level, which is used for personal state early warning, convenient self-emotion adjustment, or regional cloud control and human management, so as to cope with work and study.

[0025] The lightweight expression recognition model is compressed from a complex large-scale facial expression recognition model to a lightweight version with fewer parameters and lower computational complexity through model distillation technology. Among them, the model distillation technology generates feature information through a pre-trained “teacher model” (such as an improved ResNet18) to guide the learning of a “student model” (such as ShuffleNetV2), and finally generates a lightweight model with fewer parameters and high computational efficiency.

[0026] Among them, the facial data acquisition module can adopt a high-definition camera to capture user facial video, and synchronously collect environmental light intensity and sound tone data.

[0027] ​In addition, the facial data acquisition module includes an image preprocessing unit, which is used to preprocess the acquired user facial video stream and environmental parameters. The preprocessing method includes extracting target data from the facial image with the same weight parameters and performing de-redundancy processing on the target data through a convolutional neural network.

[0028] In this embodiment, the facial data collection module can perform multi-face recognition within its field of view, simultaneously collecting facial expressions from one or more people and analyzing them independently. Furthermore, the facial data collection module can identify risky individuals or groups, alerting security personnel to any unsafe locations, allowing them to conduct safety checks and prevent unsafe incidents.

[0029] Preferably, the facial data acquisition module and the edge computing module can be integrated into devices including but not limited to portable monitoring bracelets or other types of surveillance cameras.

[0030] Specifically, the lightweight expression recognition model is a proprietary micro-expression detection module based on MobileNetV3. It can extract basic expressions, micro-expression indicators, and physiological parameters in real time. Basic expressions include happiness, sadness, anger, fear, disgust, and surprise. Micro-expression feature vectors include: frequency of periocular tremors, unilateral droop of the mouth corner, and duration of brow contraction. Physiological parameters include pupil diameter change rate and facial blood flow fluctuations.

[0031] In addition, facial blood flow fluctuations were measured using the PPG signal inversion method, which specifically included: extracting the PPG signal from the facial ROI area; and calculating the LF / HF ratio of heart rate variability (HRV) through wavelet transform.

[0032] like Figures 1 to 3 As shown, the cloud psychological assessment module includes a DSM-5 mapping model, which is trained through the following steps: S1. Collect annotated video datasets of clinically diagnosed patients; S2, extracting the AU unit combination features of the facial physiological action coding system; S3. The XGBoost algorithm is used to establish a nonlinear mapping relationship between AU combinations and depression / anxiety scale scores.

[0033] Among them, the facial action coding system decomposes facial muscle movements into 44 action units (AUs), and quantifies facial expression changes through combined coding. For example, AU6 corresponds to cheek lifting (happiness), AU12 is associated with pulling the corners of the mouth (contempt), the combination of AU6+AU12 may represent a happy smile, AU4+AU6 corresponds to (anger), AU1+AU2 corresponds to (sadness), etc.

[0034] In addition, the mental state quantification algorithm is as follows: def mental_state_calculation(features): depression_score = 0.3 * eyebrow_asymmetry + 0.5 * blink_rate - 0.2 * smile_intensity anxiety_level = 0.4 * pupil_dilation_variance + 0.6 * voice_tremor return DSM-5_mapping(depression_score, anxiety_level).

[0035] As shown in Figures 1 to 2 , the dynamic risk prediction module updates the user state transition matrix and observation probability matrix at a preset period (e.g., 8 hours), and triggers an escalation alert when the predicted multi-dimensional psychological risk indicators or specific dimensions thereof (e.g., anxiety risk) exceed the corresponding threshold for M consecutive periods (e.g., 3 periods). The multi-dimensional psychological risk indicators include but are not limited to depression tendency, stress level, emotional stability, etc.

[0036] The multi-dimensional psychological risk indicators include but are not limited to anxiety, depression tendency, stress level, emotional stability, etc.

[0037] In addition, the Hidden Markov Model (HMM) is a probabilistic model used to process time series data, which explains the observed data through a sequence of hidden states. The Hidden Markov Model is defined by the following three elements: the initial state probability distribution: the probability of the initial hidden state of the system. The state transition probability matrix (A): describes the transition probabilities between hidden states. The observation probability matrix (B): the probability of generating different observations under each hidden state.

[0038] Specifically, the core parameters of the Hidden Markov Model, such as the initial state distribution, state transition probability matrix, and observation probability matrix, are trained and updated regularly using user historical data to adapt to individual differences and changes in behavior patterns. As shown in Figure 1 , the mental health management system also includes an information storage module and a privacy protection module. The information storage module is used to store data information, and the privacy protection module is used to encrypt and protect the information storage module. At the same time, the video data can be anonymized in the edge computing module.

[0039] The information storage module includes a user module and a management module. The user module stores personal information and emergency contact information. The management module stores mental health training courses, mental health knowledge in different forms, information of psychological consultants, and information of medical institutions.

[0040] In addition, the privacy protection module uses the AES symmetric encryption algorithm to protect the privacy of the user and the emergency contact.

[0041] Specifically, the privacy protection module uses federated machine learning to update the cloud model parameters in real time.

[0042] As shown in Figures 1 to 3 The hierarchical intervention execution module includes a three-level response mechanism. The first level response corresponds to mild, the second level response corresponds to moderate, and the third level response corresponds to severe. The first level response is used to push mental health training courses, such as mindfulness breathing guidance videos. The second level response is used to make an appointment with an online psychological consultant, and an online psychological counseling session can also be started. The third level response is used to send location information and medical institution navigation to the preset emergency contact, and trigger the background regional cloud management and control. This part of the crowd is given special attention to avoid entering non-safe places such as rooftops and riverbanks, thereby avoiding the occurrence of safety accidents.

[0043] For example, when the mental health and safety management system is applied to a campus environment, the mental health and safety management system can be connected with cameras in the campus, including but not limited to indoor cameras, outdoor cameras, and gate cameras. These cameras can form a regional sky net in the campus. Each camera can simultaneously collect the expression information of one or more people and independently analyze it. When a camera identifies a student with a high-risk signal of mental problems, the hierarchical intervention execution module triggers a three-level response mechanism, and the cameras in the regional sky net of the campus will dynamically track the student. When the student is in an unsafe place such as a rooftop or a riverbank, the mental health and safety management system will quickly send an early warning message to the background manager to enable the manager to intervene in time, thereby avoiding the occurrence of safety accidents.

[0044] For another example, the mental health and safety management system can also be applied to a work system and connected with attendance machines, computers, mobile phones, or other cameras in the work system. The staff can use the mental health and safety management system to test their mental health status every day. If the test shows mild abnormalities, the hierarchical intervention execution module triggers a first level response to push mental health training courses to the staff and remind them to pay attention to their work methods and adjust their emotions. If the test shows severe abnormalities, the camera of the attendance machine can accurately identify the staff when they perform attendance check-in, trigger a third level response, and timely feedback to the manager, who can intervene by adjusting the work tasks of the staff or other ways to relieve their emotions.

[0045] In summary, the mental health management method of the present application comprises the following steps: S01, capturing user facial data in real time and extracting micro-expression features; S02, inputting the features into a DSM-5 mapping model to output a psychological state score; S03, predicting risk trends in combination with historical data; S04, executing intervention strategy closed loop according to risk level.

[0046] It is apparent for those skilled in the art that the present disclosure is not limited to the details of the above exemplary embodiments, and the present disclosure can be implemented in other specific forms without departing from the spirit or essential characteristics of the present disclosure. Therefore, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present disclosure is defined by the appended claims rather than the above description, and all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present disclosure. Any reference signs in the claims should not be regarded as limiting the claims involved.

[0047] In addition, it should be understood that although the present specification is described in terms of embodiments, not every embodiment contains only one independent technical solution, and the description of the specification is only for the sake of clarity, and those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can be appropriately combined to form other embodiments that those skilled in the art can understand.

Claims

1. A mental health and safety management system based on facial expression recognition, characterized in that: It includes facial data collection module, edge computing module, cloud psychological assessment module, dynamic risk prediction module and graded intervention execution module; The facial data acquisition module is used to obtain the user's facial video stream and environmental parameters in real time. The edge computing module is equipped with a lightweight expression recognition model for outputting the collected facial data as micro-expression feature vectors and physiological parameters. The cloud psychological assessment module is used to convert the micro-expression feature vectors and physiological parameters into psychological state quantitative values. The dynamic risk prediction module is based on the hidden Markov model and uses the user's historical and current psychological state quantitative value sequences as the observation sequence to predict the change trend of multi-dimensional psychological risk indicators in the next N hours, and draws the prediction results into a dynamic risk curve. The graded intervention execution module triggers differentiated security management response strategies according to the risk level, which is used for personal status warning to facilitate self-adjustment of emotions, or is controlled and maintained by the regional cloud to cope with work and study.

2. A mental health and safety management system based on facial expression recognition according to claim 1, characterized in that: The facial data acquisition module includes an image preprocessing unit, which is used to preprocess the acquired user facial video stream and environmental parameters. The preprocessing method includes extracting target data from the facial image with the same weight parameters and performing redundancy removal on the target data through a convolutional neural network.

3. The mental health and safety management system based on facial expression recognition according to claim 1, characterized in that: The micro-expression feature vectors and physiological parameters include: frequency of eyelid tremors, degree of unilateral mouth corner droop, duration of eyebrow contraction, rate of pupil diameter change, and facial blood flow fluctuation.

4. The mental health and safety management system based on facial expression recognition according to claim 1, characterized in that: The cloud psychological assessment module includes a DSM-5 mapping model, which is trained by the following steps: S1. Collect annotated video datasets of clinically diagnosed patients; S2, extracting the AU unit combination features of the facial physiological action coding system; S3. The XGBoost algorithm is used to establish a nonlinear mapping relationship between AU combinations and depression / anxiety scale scores.

5. The mental health and safety management system based on facial expression recognition according to claim 1, characterized in that: The dynamic risk prediction module updates the user state transition matrix and the observation probability matrix at a preset period. When the predicted multidimensional psychological risk index or its specific dimension exceeds the corresponding threshold for M consecutive periods, an upgrade warning is triggered.

6. The mental health and safety management system based on facial expression recognition according to claim 1, characterized in that: The mental health management system also includes an information storage module and a privacy protection module. The information storage module is used to store data information, and the privacy protection module is used to encrypt and protect the information storage module. At the same time, the video data can be anonymized in the edge computing module.

7. The mental health and safety management system based on facial expression recognition according to claim 6, characterized in that: The information storage module includes a user module and a management module. The user module stores user personal information and emergency contact information, and the management module stores mental health training courses, psychological counselor information and medical institution information.

8. The mental health and safety management system based on facial expression recognition according to claim 7, characterized in that: The privacy protection module uses the AES symmetric encryption algorithm to protect the privacy of users and emergency contacts.

9. The mental health and safety management system based on facial expression recognition according to claim 8, characterized in that: The privacy protection module uses federated machine learning to update cloud model parameters in real time.

10. The mental health and safety management system based on facial expression recognition according to claim 7, characterized in that: The hierarchical intervention execution module includes a three-level response mechanism, wherein the first-level response is used to push mental health training courses, the second-level response is used to make appointments with online psychological counselors, and the third-level response is used to send location information and medical institution navigation to preset emergency contacts, and trigger background regional cloud management or manual maintenance.

Images