Emotional risk grading early warning and intervention system and method based on wrist physiological signals

Abstract

The invention discloses an emotional risk grading early warning and intervention system and method based on wrist physiological signals, and relates to the technical field of medical monitoring. The system comprises a wrist physiological signal acquisition module, a signal preprocessing module, an emotion recognition module, an emotion risk grading early warning module, an intervention module and an interaction management module which are in communication connection in sequence, the wrist physiological signal acquisition module is used for acquiring a wrist physiological signal of a user; the signal preprocessing module is used for preprocessing the acquired wrist physiological signal; the emotion recognition module recognizes a user emotion state; the emotional risk grading and early warning module performs risk grading and early warning on the emotional state of the user; the intervention module performs intervention according to the user emotion risk grade; and the interaction management module obtains user emotion risk grading real-time data, receives early warning and adjusts an intervention scheme. The wrist physiological signal can be continuously monitored in real time, the individual emotional state is recognized, risk grading evaluation and early warning are carried out, and personalized intervention is provided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical monitoring, in particular to an emotion risk grading early warning and intervention system and method based on wrist physiological signals. BACKGROUND

[0002] Traditional psychological assessment relies on subjective scales and discrete clinical observation, which has poor timeliness and insufficient objectivity. In order to truly reflect the daily emotions of the target personnel, the best method is to monitor the emotions normally. Wrist signal acquisition technology has great potential in the field of health monitoring due to its convenience and non-invasiveness. However, in the process of wrist signal acquisition, it is easy to be disturbed by various interferences such as motion artifacts, environmental noise and signal interference, which leads to the decline of signal quality and seriously affects the accuracy of subsequent analysis.

[0003] Therefore, the present application provides an emotion risk grading early warning and intervention system and method based on wrist physiological signals to solve the problems existing in the prior art, which is a problem that needs to be solved by those skilled in the art. SUMMARY

[0004] Therefore, the present application provides an emotion risk grading early warning and intervention system and method based on wrist physiological signals to solve the problems existing in the prior art, which is a problem that needs to be solved by those skilled in the art.

[0005] In order to achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0006] An emotion risk grading early warning and intervention system based on wrist physiological signals, comprising:

[0007] a wrist physiological signal acquisition module, a signal preprocessing module, an emotion recognition module, an emotion risk grading early warning module, an intervention module and an interaction management module connected in sequence;

[0008] The wrist physiological signal acquisition module is used to acquire the wrist physiological signals of the user.

[0009] The signal preprocessing module is used to preprocess the acquired wrist physiological signals of the user.

[0010] The emotion recognition module is used to recognize the emotional state of the user according to the preprocessed wrist physiological signals.

[0011] The emotion risk grading early warning module is used to grade and warn the user's emotions according to the user's emotional state.

[0012] The intervention module is used to intervene according to the user's emotional risk grading.

[0013] An interaction management module is configured to acquire user emotional risk grading real-time data, receive early warnings, and adjust intervention plans.

[0014] The system can further include a wrist physiological signal acquisition module, which integrates a photoplethysmogram sensor, an electrocardiogram sensor, a galvanic skin response sensor, a skin temperature sensor, and a three-axis accelerometer in a wristband device.

[0015] The photoplethysmogram sensor is configured to monitor a PPG signal.

[0016] The electrocardiogram sensor is configured to monitor an ECG signal.

[0017] The galvanic skin response sensor is configured to monitor a GSR signal on the surface of the wrist skin.

[0018] The skin temperature sensor is configured to monitor changes in the temperature of the wrist skin.

[0019] The three-axis accelerometer is configured to monitor a body activity posture.

[0020] The system can further include a signal preprocessing module, which is configured to preprocess the acquired wrist physiological signals, including:

[0021] Filtering to remove power frequency and motion noise of the wrist signals.

[0022] Signal segmentation of the wrist signals according to a time window.

[0023] Extraction of time domain features, frequency domain features, and nonlinear features of the wrist signals.

[0024] Feature fusion of PPG signal features, ECG signal features, GSR signal features, wrist skin temperature change signal features, and body activity posture signal features to obtain a wrist signal feature set.

[0025] The system can further include an emotion recognition module, which is configured to construct a deep learning-based multi-scale convolutional emotion recognition model, with the wrist signal feature set as the input and basic emotional dimensions or specific emotional states as the output, including but not limited to calm, mild stress, high stress, anxiety, and sadness.

[0026] The deep learning-based multi-scale convolutional emotion recognition model is trained on a labeled data set to recognize basic emotional dimensions or specific emotional states based on the wrist signals.

[0027] The system can further include an emotional risk grading and early warning module, which is configured to grade and warn the user's emotions, specifically:

[0028] The user's emotions are graded according to the recognized emotional states, the intensity, duration, and change trend of the emotional states, and historical warning records.

[0029] The risk threshold is dynamically set according to different individuals, different situations and different times, and is divided into four risk levels, including: low risk, medium risk, high risk and emergency risk.

[0030] Low risk, no need for early warning, only record logs;

[0031] Medium risk, reminded by vibration or soft light;

[0032] High risk: reminded by strong vibration and prompt sound, and preset emergency contact person is informed;

[0033] Emergency risk: strong alarm, automatic call to preset emergency contact person, and help signal containing location information is sent.

[0034] The system described above, optionally, an intervention module, according to the user emotional risk grading, intervention, including: self-help intervention and external linkage intervention;

[0035] Self-help intervention, specifically: guided exercise, distraction and seeking support;

[0036] External linkage intervention, specifically: informing caregivers / family members, providing user status and location information; informing professional agencies / medical personnel, accessing remote medical platform or community mental health service system, and starting professional intervention process.

[0037] The system described above, optionally, a user interaction and management module includes: Web and mobile APP;

[0038] The user obtains real-time physiological signals, emotional state curve and risk level history through the user interaction and management module, receives early warning information and intervention suggestions, and manages emergency contact person.

[0039] An emotional risk grading early warning and intervention method based on wrist physiological signals, applied to any one of the emotional risk grading early warning and intervention systems based on wrist physiological signals described above, comprising:

[0040] S1, obtaining user wrist physiological signals;

[0041] S2, preprocessing the obtained user wrist physiological signals;

[0042] S3, constructing a multi-scale convolution emotion recognition model based on deep learning, training the multi-scale convolution emotion recognition model based on deep learning on a labeled data set, and identifying user emotional state;

[0043] S4, according to the user emotional state, risk grading and early warning of user emotion;

[0044] S5, intervention according to user emotional risk grading real-time data, real-time adjustment of intervention scheme;

[0045] S6, the user, the guardian and the medical staff access the system through the Web end and the mobile APP, obtain real-time physiological signals, emotional state curves and risk level history, receive early warning information and intervention suggestions, and manage emergency contacts.

[0046] According to the technical scheme, compared with the prior art, the present application provides an emotional risk grading early warning and intervention system and method based on wrist physiological signals, which has the following beneficial effects:

[0047] The present application constructs a dynamic risk assessment model by fusing PPG signal, ECG signal, GSR signal, wrist skin temperature change and body activity posture and other multi-dimensional data, realizes the paradigm shift from passive treatment to active intervention, uses the convenience of wearable technology and the objectivity of physiological signals, provides an innovative active monitoring and intervention means for the field of mental health, through accurate signal processing, robust emotion and risk identification model, graded early warning mechanism and personalized and timely intervention strategy, realizes early discovery and effective response to individual emotional crisis under the premise of respecting privacy and ethics, and finally improves the level of mental health and crisis response ability. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of the provided drawings.

[0049] Figure 1 The structure diagram of the emotional risk grading early warning and intervention system based on wrist physiological signals provided by the present application is shown in the figure.

[0050] Figure 2 The flowchart of the emotional risk grading early warning and intervention method based on wrist physiological signals provided by the present application is shown in the figure. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present application will be described in detail below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0052] In this application, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment. Without more limitation, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, article or equipment including the element.

[0053] Referring to Figure 1 The present application discloses a wrist physiological signal-based emotional risk grading early warning and intervention system, comprising:

[0054] The wrist physiological signal acquisition module, the signal preprocessing module, the emotion recognition module, the emotional risk grading early warning module, the intervention module and the interaction management module are sequentially connected in communication;

[0055] The wrist physiological signal acquisition module is used for acquiring the wrist physiological signal of a user;

[0056] The signal preprocessing module is used for preprocessing the acquired wrist physiological signal of the user;

[0057] The emotion recognition module is used for recognizing the emotional state of the user according to the preprocessed wrist physiological signal;

[0058] The emotional risk grading early warning module is used for grading and early warning the emotional risk of the user according to the emotional state of the user;

[0059] The intervention module is used for intervening according to the emotional risk grading of the user;

[0060] The interaction management module is used for acquiring real-time data of the emotional risk grading of the user, receiving early warning and adjusting the intervention scheme.

[0061] Further, the wrist physiological signal acquisition module integrates the photoplethysmogram sensor, the electrocardio sensor, the galvanic skin response sensor, the skin temperature sensor and the three-axis accelerometer in the wristband device;

[0062] The photoplethysmogram sensor is accurately aligned with the radial artery distribution position of the wrist through the photoplethysmogram sensor, and the sensor is ensured to be in close contact with the skin of the wrist through the flexible fitting technology, the PPG signal is monitored, and the autonomic nervous system activity (sympathetic / parasympathetic nerve balance) is reflected, which is the core indicator of the emotional state;

[0063] Through the electrocardio sensor, the ECG electrode layout of the electrocardio sensor in the wrist wearable device is optimized based on the wrist muscle distribution position, and the ECG signal is monitored;

[0064] Through the galvanic skin response sensor, the galvanic skin response sensor in the wrist wearable device is optimized and designed based on the wrist nerve distribution position, the galvanic skin response signal on the wrist skin surface is monitored, the skin conductance level / response is measured, and the sympathetic nervous excitability is directly reflected, which is highly sensitive to stress, anxiety and emotional arousal;

[0065] Through the skin temperature sensor, the change of the wrist skin temperature is monitored, and the change of the emotion (such as tension, anger) may cause peripheral vasoconstriction to cause the fluctuation of the skin temperature;

[0066] Through the three-axis accelerometer, the body activity posture is monitored, which is used to identify the motion artifact (interference signal), the activity state (rest / motion), and certain specific postures (such as curling) may also be related to the emotional state.

[0067] Further, the signal preprocessing module pre-processes the acquired wrist physiological signals of the user, including:

[0068] Filtering to remove the power frequency and motion noise of the wrist signal;

[0069] Signal segmentation of the wrist signal according to a time window;

[0070] Extracting the time domain features, frequency domain features and nonlinear features of the wrist signal;

[0071] Feature fusion of the PPG signal features, ECG signal features, GSR signal features, wrist skin temperature change signal features and body activity posture signal features to obtain a wrist signal feature set;

[0072] Multi-modal signal fusion is usually better than single signal effect;

[0073] Fusion strategy: Transformer fusion, regarding the features of different modalities as sequence input of Transformer for cross-modal interaction modeling.

[0074] Further, the emotion recognition module constructs a multi-scale convolution emotion recognition model based on deep learning, the input is the wrist signal feature set, and the output is the basic emotional dimension or specific emotional state, including but not limited to: calm, mild stress, high stress, anxiety, sadness;

[0075] The multi-scale convolution emotion recognition model based on deep learning is trained on the labeled data set, and the basic emotional dimension or specific emotional state is recognized according to the wrist signal.

[0076] Further, the emotional risk grading and early warning module grades and warns the user's emotions, specifically:

[0077] According to the identified emotional state, the intensity, duration, trend of change and historical warning records of the emotional state, the user's emotions are graded for risk:

[0078] Considering the individual baseline, the current situation (such as normal physiological index change after strenuous exercise), time (such as different baseline at night), dynamically setting the risk threshold, divided into four risk levels, including: low risk, medium risk, high risk, emergency risk;

[0079] Low risk, no warning, only record log;

[0080] Medium risk, reminded by vibration or soft light;

[0081] High risk: reminded by strong vibration and prompt sound, notify the preset emergency contact;

[0082] Emergency risk: strong alarm, automatic call to preset emergency contact, send help signal containing location information.

[0083] Further, the intervention module, according to the user's emotional risk grading, carries out intervention, including: self-help intervention and external linkage intervention;

[0084] Self-help intervention, specifically: guided exercise, distraction and seeking support;

[0085] External linkage intervention, specifically: notify the caregiver / family member, provide user status and location information; notify professional agencies / medical personnel, access remote medical platform or community mental health service system, start professional intervention process.

[0086] Further, the user interaction and management module includes: Web and mobile APP;

[0087] Users obtain real-time physiological signals, emotional state curve and risk level history through the user interaction and management module, receive warning information and intervention suggestions, and manage emergency contacts;

[0088] Users need to give explicit informed consent (especially for sensitive data and high-risk warning); users have full control over data sharing objects (especially emergency contacts).

[0089] Referring to Figure 2 A kind of emotional risk grading and early warning and intervention method based on wrist physiological signals, applied to any one of the above-mentioned emotional risk grading and early warning and intervention system based on wrist physiological signals, comprising:

[0090] S1, obtain the wrist physiological signal of user;

[0091] S2, pre-processing the acquired user wrist physiological signals;

[0092] S3, constructing a multi-scale convolution emotion recognition model based on deep learning, training the multi-scale convolution emotion recognition model based on deep learning on a labeled data set, and identifying the user emotional state;

[0093] S4, classifying and warning the user emotion according to the user emotional state;

[0094] S5, intervening and adjusting the intervention scheme in real time according to the user emotional risk classification real-time data;

[0095] S6, the user, the caregiver and the medical staff access the system through the Web end and the mobile APP, obtain the real-time physiological signal, the emotional state curve and the risk level history, receive the warning information and the intervention suggestion, and manage the emergency contact person.

[0096] Each of the embodiments in the specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment mainly describes the difference from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, it is described more simply, and the related parts can be referred to the part of the method embodiment. The above-described system and system embodiment are only illustrative, and the units described as separate components can be or can not be physically separated, and the components displayed as units can be or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment according to the actual needs. Those skilled in the art can understand and implement it without creative labor.

[0097] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A system for classifying, warning, and intervening in emotional risk based on wrist physiological signals, characterized in that, include: The system is sequentially connected to a wrist physiological signal acquisition module, a signal preprocessing module, an emotion recognition module, an emotion risk classification and early warning module, an intervention module, and an interaction management module. The wrist physiological signal acquisition module is used to acquire the user's wrist physiological signals; The signal preprocessing module is used to preprocess the acquired physiological signals from the user's wrist. The emotion recognition module is used to identify the user's emotional state based on preprocessed wrist physiological signals; The emotional risk classification and early warning module is used to classify and warn users about the risks of their emotions based on their emotional state. The intervention module is used to intervene based on the user's emotional risk level. The interaction management module is used to acquire real-time data on user emotional risk levels, receive early warnings, and adjust intervention plans.

2. The emotional risk grading, early warning, and intervention system based on wrist physiological signals according to claim 1, characterized in that, The wrist physiological signal acquisition module integrates a photoplethysmography (PPG) sensor, an electrocardiogram (ECG) sensor, a skin conductance sensor, a skin temperature sensor, and a triaxial accelerometer into the wristband device. PPG signals are monitored using a photoplethysmography (PPG) sensor. ECG signals are monitored using a cardiac sensor; The electrical conductance GSR signal on the skin surface of the wrist is monitored using a skin conductance response sensor. The wrist skin temperature is monitored using a skin temperature sensor. Body posture during activity is monitored using a three-axis accelerometer.

3. The emotional risk grading, early warning, and intervention system based on wrist physiological signals according to claim 2, characterized in that, The signal preprocessing module preprocesses the acquired physiological signals from the user's wrist, including: Filtering removes power frequency and motion noise from wrist signals; Segment the wrist signal according to the time window; Extract the time-domain, frequency-domain, and nonlinear features of the wrist signal; The wrist signal feature set is obtained by fusing the features of PPG signal, ECG signal, GSR signal, wrist skin temperature change signal, and body movement posture signal.

4. The emotional risk grading, early warning, and intervention system based on wrist physiological signals according to claim 3, characterized in that, The emotion recognition module constructs a multi-scale convolutional emotion recognition model based on deep learning. The input is a set of wrist signal features, and the output is a basic emotion dimension or a specific emotion state, including but not limited to: calm, mild stress, high stress, anxiety, and sadness. A deep learning-based multi-scale convolutional emotion recognition model is trained on a labeled dataset to identify basic emotion dimensions or specific emotion states based on wrist signals.

5. The emotional risk grading, early warning, and intervention system based on wrist physiological signals according to claim 4, characterized in that, The emotion risk classification and early warning module classifies and warns users' emotions based on risk levels, specifically: Based on the identified emotional state, its intensity, duration, trend of change, and historical warning records, the user's emotions are classified into risk levels. Risk thresholds are dynamically set based on individual differences, contexts, and time periods, and are divided into four risk levels: low risk, medium risk, high risk, and emergency risk. Low risk, no warning required, only log entries; Medium risk is indicated by vibration or soft light. High risk: Notifies preset emergency contacts via strong vibration and alert tone; Emergency Risk: Loud alarm, automatic call to preset emergency contacts, and sending a distress signal containing location information.

6. The emotional risk grading, early warning, and intervention system based on wrist physiological signals according to claim 5, characterized in that, The intervention module provides interventions based on the user's emotional risk level, including self-help intervention and external collaborative intervention. Self-help interventions include: guided practice, distraction, and seeking support. External collaborative intervention specifically involves: notifying caregivers / family members and providing user status and location information; notifying professional institutions / medical personnel, connecting to remote medical platforms or community mental health service systems, and initiating professional intervention procedures.

7. The emotional risk grading, early warning, and intervention system based on wrist physiological signals according to claim 6, characterized in that, The user interaction and management module includes: a web interface and a mobile app; Users can obtain real-time physiological signals, emotional state curves, and historical risk levels through the user interaction and management module, receive early warning information and intervention suggestions, and manage emergency contacts.

8. A method for grading, warning, and intervention of emotional risk based on wrist physiological signals, applied to the emotional risk grading, warning, and intervention system based on wrist physiological signals as described in any one of claims 1-7, comprising: S1. Acquire physiological signals from the user's wrist; S2. Preprocess the acquired physiological signals from the user's wrist; S3. Construct a deep learning-based multi-scale convolutional emotion recognition model, train the deep learning-based multi-scale convolutional emotion recognition model on a labeled dataset, and identify the user's emotional state. S4. Based on the user's emotional state, classify and warn about the risks associated with the user's emotions; S5. Intervene based on real-time data of user emotional risk levels and adjust the intervention plan in real time; S6. Users, caregivers, and medical staff can access the system via the web or mobile app to obtain real-time physiological signals, emotional state curves, and historical risk levels, receive early warning information and intervention suggestions, and manage emergency contacts.

Images