Sleep state recognition system, method, and sleep intervention system
By collecting and analyzing brainwave and electrocardiogram data, the system identifies users' sleep states and provides precise intervention, solving the problem of existing technologies being unable to quickly and accurately identify short-term light sleep states, thus improving the effectiveness of sleep aid products.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANGHAI SHULI INTELLIGENT TECH CO LTD
- Filing Date
- 2023-08-22
- Publication Date
- 2026-07-07
AI Technical Summary
Existing sleep aids cannot quickly and accurately identify a user's current sleep state, especially short periods of light sleep, which makes it impossible to effectively help users enter and maintain optimal sleep depth.
By collecting users' brainwave and electrocardiogram data, a feature extraction network is used to extract features, and a splicing and fusion network is used to obtain the probability value of sleep state. The current sleep state is identified by combining the quantitative value range of sleep state, and the data interference is smoothed by the quantitative value optimization module of sleep state to achieve accurate identification.
It enables rapid and accurate identification of users' sleep states, and provides precise intervention through a sleep intervention system to help users enter and maintain optimal sleep depth and improve sleep quality.
Smart Images
Figure CN117064400B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of electroencephalogram (EEG) signal processing, and specifically relates to sleep state recognition systems, methods, and sleep intervention systems. Background Technology
[0002] Humans need sleep, and sleep occupies only one-third of a person's life. Numerous studies have shown that sufficient sleep at night is beneficial to health. Besides the long nighttime sleep, short afternoon naps also have many benefits. An afternoon nap is a short, light sleep that can eliminate fatigue, leaving you refreshed and alert for the afternoon, lowering blood pressure, reducing cardiovascular disease, improving memory, and enhancing performance. However, if a nap is too long, the sleep depth increases, and the negative effects of being awakened are amplified. For example, being awakened after a 30-minute nap can cause grogginess, and being awakened after an hour can have adverse health effects. Generally, a nap of around 15 minutes is optimal.
[0003] Existing sleep aids are generally designed for users with sleep difficulties, helping them enter deep sleep. Currently, there are no sleep aids specifically for short periods of light sleep. The main challenge for such products is how to quickly and accurately identify the user's current sleep state. Since all sleep aids rely on the user's current state, only by accurately knowing the user's current state can they quickly assist the user in achieving short periods of light sleep. Summary of the Invention
[0004] Objective of the Invention: This invention addresses the problems existing in the prior art by proposing a sleep state recognition system. It primarily obtains probability values for different sleep states by extracting and fusing features from the user's electroencephalogram (EEG) and electrocardiogram (ECG) data. Then, it quantifies the user's current sleep state based on the probability values of all sleep states, and determines the user's current sleep state according to the range of this quantified value. This allows for the rapid and accurate determination of the user's current sleep state.
[0005] Technical solution: To achieve the above objectives, the present invention provides a sleep state recognition system, including a data acquisition module, a feature extraction module, and a fusion feature recognition module;
[0006] The data acquisition module is used to collect EEG and ECG data in real time after the user is about to fall asleep.
[0007] The feature extraction module uses a feature extraction network to extract features from the EEG and ECG data collected by the data acquisition module.
[0008] The fusion feature recognition module is used to splice and fuse the EEG and ECG features extracted by the feature extraction module to obtain the probability of different sleep states; based on the probability of different sleep states, the current user's sleep state quantification value V is obtained; and based on the range of the current user's sleep state quantification value V, the current user's sleep state is obtained.
[0009] Furthermore, the feature extraction module employs an EEG data feature extraction network to sequentially extract features from M seconds of EEG data. The network architecture of this EEG data feature extraction network includes, in sequence, an input layer, a 2D convolutional layer, a channel convolutional layer, an average pooling layer, a 2D separable convolutional layer, a depthwise separable convolutional layer, an average pooling layer, a Flatten layer, and a fully connected layer. This allows for accurate and rapid extraction of the desired features from the EEG data.
[0010] Furthermore, the feature extraction module employs an ECG data feature extraction network to sequentially extract features from every 10M seconds of ECG data. The network architecture of this ECG data feature extraction network sequentially includes an input layer, a first convolutional layer, a first average pooling layer, a second convolutional layer, a second average pooling layer, a third convolutional layer, a third average pooling layer, a Flatten layer, and a fully connected layer. This not only ensures sufficient ECG data collection but also enables accurate and rapid extraction of the required features from the ECG data.
[0011] Furthermore, it also includes a sleep state quantification value optimization module. This module statistically analyzes the sleep state quantification values obtained from the current iteration and the previous L consecutive iterations, identifies the most frequent state among the L+1 iterations as the current state, and averages the corresponding sleep state quantification values to obtain the current user's sleep state quantification value V. This effectively reduces data interference and yields more accurate results.
[0012] The present invention also provides a sleep intervention system, which uses the above-mentioned sleep state recognition system to identify the user's state in real time, obtain the user's sleep state quantitative value, and control the sleep-aid device to perform sleep intervention on the user based on the changes in the user's sleep quantitative value within a specified time.
[0013] Furthermore, sleep interventions for users include: sleep onset intervention, light sleep intervention, and wakefulness intervention, among which sleep aids include music, electrical stimulation, and heat therapy.
[0014] The present invention also provides a sleep state recognition method, comprising the following steps:
[0015] Step 1: Collect EEG and ECG data in real time from when the user is preparing to sleep until they fall asleep;
[0016] Step 2: Use a feature extraction model to extract features from the EEG and ECG data collected in Step 1, respectively;
[0017] Step 3: Use a splicing and fusion network to splice and fuse the EEG features and ECG features obtained in Step 2 to obtain the probability values of the awake state, the sleep state, the light sleep state, and the deep sleep state.
[0018] Step 4: According to the formula V=∑s i p i Obtain the current user's sleep state quantification value V, where i represents the number of the different sleep states; p i The probability value of its i-th sleep state, s i The value representing the i-th sleep state;
[0019] Step 5: Based on the obtained quantitative value V of the current user's sleep state, and combined with the range of the quantitative value of the sleep state under different states, obtain the current user's sleep state; where the range of the quantitative value V is 0-10, it indicates that the user is awake; the range of the quantitative value V is 11-20, it indicates that the user is falling asleep; the range of the quantitative value V is 21-30, it indicates that the user is in light sleep; and the range of the quantitative value V is 31-40, it indicates that the user is in deep sleep.
[0020] If the user state represented by the obtained sleep state quantification value V is inconsistent with the state represented by the highest probability value among the four states, the state represented by the highest probability value shall be taken as the current user state, and the current user sleep state quantification value V shall be modified to the maximum value of the corresponding state range.
[0021] Furthermore, it also includes step 6: statistically analyzing the sleep state quantification values obtained from the current time and the previous L consecutive times, taking the most frequent state in L+1 times as the current state, and averaging the corresponding sleep state quantification values of the current state to obtain the current user's sleep state quantification value V.
[0022] The present invention also discloses a sleep intervention method, which uses the above-mentioned sleep state recognition method to identify the user's state in real time, obtain the user's sleep state quantitative value, and control the sleep-assistance device to perform sleep intervention on the user according to the change rate of the user's sleep state quantitative value within one minute within a specified time, so that the user's sleep state quantitative value V is between 21 and 30.
[0023] The present invention also provides a computer-readable medium for storing software, the software including instructions executable by one or more computers, the instructions causing the one or more computers to perform operations including the flow of the sleep state recognition method described above.
[0024] The present invention also provides a computer system, comprising:
[0025] One or more processors;
[0026] The memory stores operable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations, including the flow of the sleep state recognition method described above.
[0027] Beneficial effects: Compared with the prior art, the present invention can accurately and quickly identify the current sleep state of the user and provide precise sleep intervention for the user. Attached Figure Description
[0028] Figure 1 This is a schematic diagram of the sleep state recognition system in Example 1;
[0029] Figure 2 This is a schematic diagram of the sleep intervention system in Example 2. Detailed Implementation
[0030] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0031] Example 1:
[0032] like Figure 1 As shown in the figure, this embodiment provides a sleep state recognition system, including a data acquisition module, a feature extraction module, and a fusion feature recognition module.
[0033] The data acquisition module includes a basic information acquisition submodule and a physiological information acquisition submodule. The basic information acquisition submodule collects the user's gender, age, and sleep status; the physiological information acquisition submodule is used to collect the user's brainwave and electrocardiogram data in real time.
[0034] The feature extraction module is used to extract features from the user's electroencephalogram (EEG) and electrocardiogram (ECG) data collected by the data acquisition module. The feature extraction module includes sub-modules for EEG data feature extraction and ECG data feature extraction.
[0035] The EEG data feature extraction submodule mainly includes the following steps when extracting features from EEG data:
[0036] Step A1: Take 1 second of EEG data and compare the amplitude of the 1 second EEG data with a first threshold. If it exceeds the first threshold, delete the entire data segment and take another 1 second of EEG data to perform step A1. If it does not exceed the first threshold, proceed to step A2. In this embodiment, the first threshold is preferably 1 mV.
[0037] Step A2: Filter and remove artifacts from the EEG data after the judgment in Step A1;
[0038] Step A3: Input the processed EEG data from step A2 into the EEG data feature extraction network to obtain a 64-dimensional vector, which represents the extracted features of the EEG data. The network architecture of the EEG data feature extraction network used in this embodiment is shown in Table 1.
[0039] Table 1
[0040]
[0041]
[0042] The ECG data feature extraction submodule mainly includes the following steps when extracting features from ECG data:
[0043] Step B1: Take 10 seconds of ECG data and perform low-pass filtering on the 10 seconds of ECG data;
[0044] Step B2: Downsample the low-pass filtered data from step B1 to 200Hz.
[0045] Step B3: Input the ECG data processed in Step B2 into the ECG data feature extraction network to obtain a 32-dimensional vector, which is the feature of the extracted ECG data. The network architecture of the ECG data feature extraction network used in this embodiment is shown in Table 2.
[0046] Table 2:
[0047]
[0048]
[0049] The fusion feature recognition module is used to concatenate and fuse the EEG and ECG features extracted by the feature extraction module to obtain the probability of each state; according to the formula V=∑s i p iThe current user's sleep state quantification value V is obtained, and the current user's sleep state is determined based on the range of the current user's sleep state quantification value V; where i represents the number of different sleep states, and in this embodiment, four sleep states are set, i = 1, 2, 3, 4; p i The probability value of its i-th sleep state, p i The range is 0-100%; s i The value represents the i-th sleep state. When i is 1, it represents the awake state, s1 = 0; when i is 2, it represents the sleep state, s2 = 15; when i is 3, it represents the light sleep state, s3 = 25; when i is 4, it represents the deep sleep state, s4 = 40.
[0050] The feature recognition module includes a feature splicing and fusion submodule, a sleep state quantification submodule, and a recognition submodule.
[0051] The feature splicing and fusion submodule mainly fuses EEG features and ECG features through a splicing and fusion network to obtain probability values for four states. The splicing and fusion network architecture used in this embodiment is shown in Table 3. That is, the output layer can obtain the probabilities of the awake state, the sleep state, the light sleep state, and the deep sleep state.
[0052] Table 3:
[0053] Layer type Output dimension Activation function Explanation Fully connected layer (32) No Learn relationships between features Fully connected layer (4) No Learn relationships between features Output layer (4) Softmax Output probabilities
[0054] The sleep state quantification submodule uses the formula V=∑s i p i The current user's sleep state quantification value V is obtained, where p i The probability values of different sleep states output by the feature splicing and fusion submodule.
[0055] The identification submodule uses the current user's sleep state quantification value V obtained from the sleep state quantification submodule, combined with the range of sleep state quantification values for different states, to determine the current user's sleep state. The sleep state quantification value V ranges from 0 to 40. A sleep state quantification value V of 0-10 indicates that the target user is awake; a sleep state quantification value V of 11-20 indicates that the target user is falling asleep and is slightly drowsy; a sleep state quantification value V of 21-30 indicates that the target user is in a light sleep state, which is the state the target user needs to maintain during short-term sleep; a sleep state quantification value V of 31-40 indicates that the target user is in a deep sleep state, and waking up in this state would have an impact on the target user's health.
[0056] In this embodiment, when the probability of being awake is 70%, the probability of being asleep is 5%, the probability of being in light sleep is 25%, and the probability of being in deep sleep is 0%, the most probable state is being awake. The sleep state quantification submodule calculates the probability based on the formula V = ∑s i p i The current user's sleep state quantification value is V = 7. The range of the sleep state quantification value V is 0-10, which also indicates that the current user is awake. The recognition submodule outputs that the current user is awake.
[0057] If the user state represented by the sleep state quantization value V obtained by the feature recognition module is inconsistent with the state represented by the highest probability value among the four states obtained by the feature splicing and fusion submodule, the state represented by the highest probability value among the four states obtained by the feature splicing and fusion submodule shall be taken as the current user state.
[0058] When the probability of being awake is 60%, the probability of being asleep is 0%, the probability of being in light sleep is 0%, and the probability of being in deep sleep is 40%, the most probable state is being awake. The sleep state quantification submodule calculates the probability based on the formula V = ∑s i p i The current user's sleep state quantification value V = 16 is obtained, which exceeds the range of 0-10 for the awake state. The sleep quantification value is corrected to 10. The recognition submodule outputs that the current user is in an awake state.
[0059] The EEG data feature extraction network, ECG data feature extraction network, and splicing and fusion network used in this embodiment require training before use. Training data comes from publicly available datasets and data collected during development. The data includes both EEG and ECG data, labeled with four different sleep states: awake, falling asleep, light sleep, and deep sleep. Publicly available data uses pre-defined labels, while internally collected data uses manually labeled labels. Labels are edited using one-hot encoding, and the loss function is cross-entropy. Data preprocessing is the same as in practice. After training, the classifier can predict the sleep state of the selected samples.
[0060] Example 2:
[0061] This embodiment provides a sleep intervention system, including a data acquisition module, a feature extraction module, a fusion feature recognition module, a sleep state quantification and optimization module, and a sleep assistance module.
[0062] The data acquisition module includes a basic information acquisition submodule and a physiological information acquisition submodule. The basic information acquisition submodule collects the user's gender, age, and sleep status; the physiological information acquisition submodule is used to collect the user's electroencephalogram (EEG) and electrocardiogram (ECG) data.
[0063] The feature extraction module is used to extract features from the user's electroencephalogram (EEG) and electrocardiogram (ECG) data collected by the data acquisition module. The feature extraction module includes sub-modules for EEG data feature extraction and ECG data feature extraction.
[0064] The EEG data feature extraction submodule mainly includes the following steps when extracting features from EEG data:
[0065] Step A1: Take 1 second of EEG data and compare the amplitude of the 1 second EEG data with a first threshold. If it exceeds the first threshold, delete the entire data segment and take another 1 second of EEG data to perform step A1. If it does not exceed the first threshold, proceed to step A2. In this embodiment, the first threshold is preferably 1 mV.
[0066] Step A2: Filter and remove artifacts from the EEG data after the judgment in Step A1;
[0067] Step A3: Input the processed EEG data from step A2 into the EEG data feature extraction network to obtain a 64-dimensional vector, which represents the extracted features of the EEG data. The network architecture of the EEG data feature extraction network used in this embodiment is shown in Table 4.
[0068] Table 4
[0069]
[0070]
[0071] The ECG data feature extraction submodule mainly includes the following steps when extracting features from ECG data:
[0072] Step B1: Take 10 seconds of ECG data and perform low-pass filtering on the 10 seconds of ECG data;
[0073] Step B2: Downsample the low-pass filtered data from step B1 to 200Hz.
[0074] Step B3: Input the ECG data processed in Step B2 into the ECG data feature extraction network to obtain a 32-dimensional vector, which is the feature of the extracted ECG data. The network architecture of the ECG data feature extraction network used in this embodiment is shown in Table 5.
[0075] Table 5:
[0076]
[0077]
[0078] The fusion feature recognition module is used to concatenate and fuse the EEG and ECG features extracted by the feature extraction module to obtain the probability of each state; according to the formula V=∑s i p i The current user's sleep state quantification value V is obtained, and the current user's sleep state is determined based on the range of the current user's sleep state quantification value V; where i represents the number of different sleep states, and in this embodiment, four sleep states are set, i = 1, 2, 3, 4; p i The probability value of its i-th sleep state, p i The range is 0-100%; s i The value represents the i-th sleep state. When i is 1, it represents the awake state, s1 = 0; when i is 2, it represents the sleep state, s2 = 15; when i is 3, it represents the light sleep state, s3 = 25; when i is 4, it represents the deep sleep state, s4 = 40.
[0079] The feature recognition module includes a feature splicing and fusion submodule, a sleep state quantification submodule, and a recognition submodule.
[0080] The feature splicing and fusion submodule mainly fuses EEG features and ECG features through a splicing and fusion network to obtain probability values for four states. The splicing and fusion network architecture used in this embodiment is shown in Table 6. That is, the output layer can obtain the probability of awake state, sleep state, light sleep state and deep sleep state.
[0081] Table 6:
[0082] Layer type Output dimension Activation function Explanation Fully connected layer (32) No Learn relationships between features Fully connected layer (4) No Learn relationships between features Output layer (4) Softmax Output probabilities
[0083] The sleep state quantification submodule uses the formula V=∑s i p i The current user's sleep state quantification value V is obtained, where p i The probability values of different sleep states output by the feature splicing and fusion submodule.
[0084] The identification submodule uses the current user's sleep state quantification value V obtained from the sleep state quantification submodule, combined with the range of sleep state quantification values for different states, to determine the current user's sleep state. The sleep state quantification value V ranges from 0 to 40. A sleep state quantification value V of 0-10 indicates that the target user is awake; a sleep state quantification value V of 11-20 indicates that the target user is falling asleep and is slightly drowsy; a sleep state quantification value V of 21-30 indicates that the target user is in a light sleep state, which is the state the target user needs to maintain during short-term sleep; a sleep state quantification value V of 31-40 indicates that the target user is in a deep sleep state, and waking up in this state would have an impact on the target user's health.
[0085] In this embodiment, when the probability of the awake state output by the feature splicing and fusion submodule is 70%, the probability of the sleep state is 5%, the probability of the light sleep state is 25%, and the probability of the deep sleep state is 0%, the most probable state is the awake state. The sleep state quantification submodule uses the formula V=∑s i p i The current user's sleep state quantification value is V = 7. The range of the sleep state quantification value V is 0-10, which also indicates that the current user is awake. The recognition submodule outputs that the current user is awake.
[0086] If the user state represented by the sleep state quantization value V obtained by the feature recognition module is inconsistent with the state represented by the highest probability value among the four states obtained by the feature splicing and fusion submodule, the state represented by the highest probability value among the four states obtained by the feature splicing and fusion submodule shall be taken as the current user state.
[0087] When the probability of being awake is 60%, the probability of being asleep is 0%, the probability of being in light sleep is 0%, and the probability of being in deep sleep is 40%, the most probable state is being awake. The sleep state quantification submodule calculates the probability based on the formula V = ∑s i p i The current user's sleep state quantification value V = 16 is obtained, which exceeds the range of 0-10 for the awake state. The sleep quantification value is corrected to 10. The recognition submodule outputs that the current user is in an awake state.
[0088] The sleep state quantification value optimization module is used to optimize the sleep state quantification value obtained by the fusion feature recognition module. It mainly calculates the sleep state quantification value obtained from the current time and the previous 9 consecutive times, and the most frequent state among the 10 times is the current state. The sleep state quantification value corresponding to the current state is averaged to obtain the current user's sleep state quantification value V.
[0089] In this embodiment, the sleep quantification values obtained within 10 seconds are 11, 12, 13, 14, 15, 16, 17, 33, 5, and 29. Seven of these values correspond to the state of falling asleep, therefore the user is determined to be currently asleep. The average of these seven sleep quantification values yields the current user's sleep state quantification value V, which is 14. This method can mitigate data interference and obtain more accurate prediction results.
[0090] The sleep assistance module obtains the current user's state based on the quantitative value V of the current user's sleep state, and controls the sleep assistance device to intervene in the user's sleep based on the current user's sleep state.
[0091] Types of sleep interventions for users include: sleep onset intervention, helping users fall asleep quickly; light sleep intervention, preventing users from transitioning from light sleep to deep sleep; and wake-up intervention, waking the user. Sleep aids include devices with intervention methods including, but not limited to, music, electrical stimulation, and heat therapy. Different sleep quantification values affect the intensity of the intervention. For example, when a sleep aid is implementing a rapid sleep onset intervention, the device continuously plays specific frequency sleep-inducing music while the target user is awake. As the target user's sleep quantification value rapidly increases, indicating they are falling asleep, the volume of the sleep-inducing music decreases. When the target user's sleep quantification value further increases, indicating they are entering a light sleep state, the sleep-inducing music is turned off. The intensity of the intervention varies for different users. For example, if a target user's historical data indicates they fall asleep quickly, the sleep-inducing music can be turned off when the user falls asleep.
[0092] The EEG data feature extraction network, ECG data feature extraction network, and splicing and fusion network used in this embodiment require training before use. Training data comes from publicly available datasets and data collected during development. The data includes both EEG and ECG data, labeled with four different sleep states: awake, falling asleep, light sleep, and deep sleep. Publicly available data uses pre-defined labels, while internally collected data uses manually labeled labels. Labels are edited using one-hot encoding, and the loss function is cross-entropy. Data preprocessing is the same as in practice. After training, the classifier can predict the sleep state of the selected samples.
[0093] Example 3:
[0094] This embodiment provides a sleep state recognition method, which specifically includes the following steps:
[0095] Step 1: Collect basic user information, including the user's gender, age, sleep status, and EEG and ECG data from preparing to sleep to falling asleep.
[0096] Step 2: Use a feature extraction model to extract features from the EEG and ECG data collected in Step 1, respectively;
[0097] The main steps involved in feature extraction from brainwaves are as follows:
[0098] Step A1: Take 1 second of EEG data and compare the amplitude of the 1 second EEG data with a first threshold. If it exceeds the first threshold, delete the entire data segment and take another 1 second of EEG data to perform step A1. If it does not exceed the first threshold, proceed to step A2. In this embodiment, the first threshold is preferably 1 mV.
[0099] Step A2: Filter and remove artifacts from the EEG data after the judgment in Step A1;
[0100] Step A3: Input the processed EEG data from Step A2 into the EEG data feature extraction network to obtain a 64-dimensional vector, which represents the extracted features of the EEG data. The EEG feature extraction network adopts the network architecture disclosed in Table 1 above.
[0101] Feature extraction from electrocardiograms mainly includes the following steps:
[0102] Step B1: Take 10 seconds of ECG data and perform low-pass filtering on the 10 seconds of ECG data;
[0103] Step B2: Downsample the low-pass filtered data from step B1 to 200Hz.
[0104] Step B3: Input the ECG data processed in Step B2 into the ECG data feature extraction network to obtain a 32-dimensional vector, which is the feature of the extracted ECG data. The ECG data feature extraction network adopts the network architecture disclosed in Table 2 above.
[0105] Step 3: Use a splicing and fusion network to splice and fuse the EEG features and ECG features obtained in Step 2 to obtain the probability values of the awake state, the sleep state, the light sleep state, and the deep sleep state; the splicing and fusion network adopts the network architecture disclosed in Table 3 above.
[0106] Step 4: According to the formula V=∑s i p i Obtain the quantified value V of the current user's sleep state, where i represents the number of different sleep states, i = 1, 2, 3, 4; p i The probability value of its i-th sleep state, p i The range is 0-100%; s iThe value represents the i-th sleep state. When i is 1, it represents the awake state, s1 = 0; when i is 2, it represents the sleep state, s2 = 15; when i is 3, it represents the light sleep state, s3 = 25; when i is 4, it represents the deep sleep state, s4 = 40.
[0107] Step 5: Based on the obtained quantitative value V of the current user's sleep state, and combined with the range of the quantitative value of the sleep state under different states, obtain the current user's sleep state; where the quantitative value V ranges from 0 to 10, indicating that the user is awake; the quantitative value V ranges from 11 to 20, indicating that the user is falling asleep; the quantitative value V ranges from 21 to 30, indicating that the user is in light sleep; and the quantitative value V ranges from 31 to 40, indicating that the user is in deep sleep.
[0108] If the user state represented by the obtained sleep state quantification value V is inconsistent with the state represented by the highest probability value among the four states, the state represented by the highest probability value shall be taken as the current user state, and the current user sleep state quantification value V shall be modified to the maximum value of the corresponding state range.
[0109] Step 6: Statistically analyze the sleep state quantification values obtained from the current time and the previous 9 consecutive times. The state with the most occurrences among the 10 times is the current state. Then, average the sleep state quantification values corresponding to the current state to obtain the current user's sleep state quantification value V.
[0110] Example 4:
[0111] This embodiment provides a sleep intervention method, which mainly includes the following methods: using the sleep state recognition method disclosed in Embodiment 3 to identify the user's sleep state in real time; based on the target user's sleep quantification value and the rate of change of the sleep quantification value, combined with the rapid sleep onset intervention intensity table and the light sleep intervention intensity table, to obtain whether the user needs to be intervened and the intensity of the intervention; and then controlling the sleep-aiding device to perform sleep intervention on the user.
[0112] The sleep intervention intensity scale is only for users with a sleep quantification score between 0 and 20, and its goal is to help users fall asleep quickly. The sleep intervention ends when the user's sleep quantification score exceeds 20.
[0113] The light sleep intervention intensity scale is only for users whose sleep quantification value is between 20 and 40. The light sleep intervention scale ensures that users are in a light sleep state before being woken up.
[0114] The methods for using the sleep onset intervention intensity and light sleep intervention intensity scales are similar to the adjustment methods. The following describes the rapid sleep onset intervention intensity scale.
[0115] The rapid sleep onset intervention intensity table is a two-dimensional table. The first column represents the user's current sleep quantification value, and the first row represents the rate of change of the user's sleep quantification value, which is the difference between the current sleep quantification value and the sleep quantification value one minute ago. The values in the table represent the intervention intensity, and the intervention method is music intervention. The intervention intensity section is shown in Table 7:
[0116] Table 7:
[0117] -3 -2 -1 0 1 2 1 1 1 1 2 1 1 2 1 1 1 2 1 1 3 2 1 1 2 1 0 4 2 2 1 1 0 0 5 2 2 2 1 0 0 6 3 2 2 0 0 0
[0118] For example, if a user's sleep quantification value was 4 one minute ago and is currently 6, corresponding to row 6 and column 7 of the table, the intervention intensity is 0, meaning no intervention is performed. Specifically, if the sleep duration is less than one minute, the change rate of the sleep quantification value is 0.
[0119] Generally, in sleep intervention scenarios, if the target user's quantitative sleep value changes positively and their sleep state is deep, no intervention will be implemented. The detection and implementation cycle for sleep intervention is 1 minute, with the intervention intensity adjusted every minute according to the above rules.
[0120] For first-time users, the intensity scales for light sleep intervention and rapid sleep onset intervention are determined based on the user's age, gender, sleep status questionnaire, and type of assistive device. Further adjustments are made based on user data.
[0121] The adjustment methods include the following:
[0122] Sleep onset time refers to the time it takes for a user to fall asleep and enter the light sleep stage. If the sleep onset time is within the standard range, no changes will be made; if it is not within the standard range, the sleep onset intervention intensity scale will be adjusted.
[0123] In cases where insufficient historical data is available for a user, the ideal sleep time for that user is calculated as the average sleep time ± 2 standard deviation of users of the same age, gender, and mental state. When sufficient sleep history is available, the ideal sleep time is calculated based on the average sleep time and standard deviation of the target user's most recent 10 sleep episodes. The calculation method is: average sleep time ± 2 standard deviation.
[0124] If, including the current sleep attempt period, the target user experiences two consecutive sleep attempts with a sleep duration shorter than the normal range, then the intensity of the rapid sleep onset intervention for the corresponding sleep aid device category should be adjusted to reduce the intervention intensity. Conversely, if, including the current sleep attempt period, the target user experiences two consecutive sleep attempts with a sleep duration exceeding the normal range, then the intensity of the rapid sleep onset intervention for the corresponding sleep aid device category should be adjusted to increase the intervention intensity.
[0125] The method of using the light sleep intervention chart is similar to that of the rapid sleep onset intervention chart, but the adjustment conditions for the light sleep intervention chart are slightly different. The adjustment method is as follows:
[0126] If the target user's sleep quantification score remains between 21 and 40, no adjustment is needed. If the target user's sleep quantification score exceeds 30 during two consecutive light sleep phases, adjust the light sleep intervention intensity scale to increase the intervention intensity. If the target user's sleep quantification score is less than 21 during two consecutive light sleep phases, adjust the light sleep intervention intensity scale to decrease the intervention intensity.
[0127] In this embodiment, the method of light sleep intervention is to play music. Increasing the intervention intensity can be achieved by increasing the volume, extending the playback time, etc. Increasing the volume means raising the volume of the music, making it slightly louder than before. Increasing the playback time means that, at the user's preferred volume, the music plays for a longer period and stops playing for a shorter period. For example, with an intervention intensity of 1, music is played for 5 seconds and then stopped for 10 seconds; with an intervention intensity of 2, music is played for 6 seconds and then stopped for 9 seconds.
[0128] An intervention intensity scale derived from sleep state and sleep state change rate is more in line with users' sleep habits, enabling users to better experience short periods of light sleep.
[0129] The present invention also provides a computer system comprising: one or more processors; and a memory storing operable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations, the operations including the flow of the aforementioned sleep state recognition method.
[0130] It should be understood that the aforementioned sleep state recognition method of the present invention can be applied to any computer system containing data storage and data processing capabilities. The aforementioned computer system can be at least one electronic processing system or electronic device including a processor and memory, such as a PC, whether it be a personal PC, a commercial PC, a graphics processing PC, or a server-class PC. These PCs enable wired and / or wireless data transmission, especially image data, through data interfaces and / or network interfaces.
[0131] In other embodiments, the computer system may also be a server, especially a cloud server, with data storage, processing, and network communication functions.
[0132] A typical computer system includes at least one processor, memory, and a network interface connected by a system bus. The network interface is used to communicate with other devices / systems.
[0133] The processor is used to provide computation and control for the system.
[0134] Memory includes non-volatile memory and cache.
[0135] Non-volatile memory typically has massive storage capacity and can store operating systems and computer programs. These computer programs may include operable instructions that, when executed by one or more processors, enable one or more processors to perform the sleep state recognition method of the foregoing embodiments of the present invention.
[0136] In a necessary or reasonable implementation, the aforementioned computer system, whether a PC device or a server, may include more or fewer components or combinations thereof than those shown in the diagram, or may employ different hardware, software, or other components or different deployment methods.
Claims
1. A sleep state recognition system, characterized in that: It includes a data acquisition module, a feature extraction module, and a fusion feature recognition module; The data acquisition module is used to collect EEG and ECG data in real time after the user is about to fall asleep. The feature extraction module uses a feature extraction network to extract features from the EEG and ECG data collected by the data acquisition module. The feature fusion recognition module is used to concatenate and fuse the EEG and ECG features extracted by the feature extraction module to obtain the probabilities of different sleep states; based on the probabilities of different sleep states, a quantitative value of the current user's sleep state is obtained. Quantified based on the current user's sleep status The current user's sleep state is obtained from the range in which the user is located; where, according to the formula Get the current user's sleep state quantitative value , where i represents the number of different sleep states; The probability value of its i-th sleep state, This represents the numerical value of the i-th sleep state; when i is 1, it represents the awake state. When i is 2, it indicates a sleep state. When i is 3, it indicates a light sleep state. When i is 4, it indicates a deep sleep state. Among them, the quantitative values of sleep state The range is 0-10, indicating that the user is awake; the quantification value for sleep status. A range of 11-20 indicates that the user is asleep; sleep state quantification value. A range of 21-30 indicates that the user is in a light sleep state; quantified sleep state values. A range of 31-40 indicates that the user is in a deep sleep state; Among them, if a quantitative value of sleep state is obtained If the user's current state does not match the state with the highest probability value among the four possible states, the state with the highest probability value will be taken as the current user's current state, and the current user's sleep state will be quantified. Modify it to the maximum value within the range of the corresponding state.
2. The sleep state recognition system according to claim 1, characterized in that: The feature extraction module uses an EEG data feature extraction network to extract features from M seconds of EEG data. The network architecture of the EEG data feature extraction network includes an input layer, a 2D convolutional layer, a channel convolutional layer, an average pooling layer, a 2D separable convolutional layer, a depthwise separable convolutional layer, an average pooling layer, a Flatten layer, and a fully connected layer.
3. The sleep state recognition system according to claim 1, characterized in that: The feature extraction module uses an ECG data feature extraction network to extract features from ECG data every 10M seconds. The network architecture of the ECG data feature extraction network includes an input layer, a first convolutional layer, a first average pooling layer, a second convolutional layer, a second average pooling layer, a third convolutional layer, a third average pooling layer, a Flatten layer, and a fully connected layer.
4. The sleep state recognition system according to claim 1, characterized in that: It also includes a sleep state quantification value optimization module, which statistically analyzes the sleep state quantification values obtained in the current time and the previous L consecutive times, and the most frequent state in the L+1 times is the current state. The sleep state quantification value corresponding to the current state is averaged to obtain the current user's sleep state quantification value V.
5. A sleep intervention system, characterized in that: The sleep state recognition system described in claim 1 identifies the user's state in real time, obtains the user's sleep state quantification value, and controls the sleep aid device to intervene in the user's sleep based on the changes in the user's sleep quantification value within a specified time, thereby improving the user's sleep state quantification value. Between 21 and 30.
6. The sleep intervention system according to claim 5, characterized in that: The sleep aid device includes music, electrical stimulation, and heat therapy.
7. A method for sleep state recognition, characterized in that: Includes the following steps: Step 1: Collect EEG and ECG data in real time from when the user is preparing to sleep until they fall asleep; Step 2: Use a feature extraction model to extract features from the EEG and ECG data collected in Step 1, respectively; Step 3: Use a splicing and fusion network to splice and fuse the EEG features and ECG features obtained in Step 2 to obtain the probability values of the awake state, the sleep state, the light sleep state, and the deep sleep state. Step 4: According to the formula Get the current user's sleep state quantitative value , where i represents the number of different sleep states; The probability value of its i-th sleep state, This represents the numerical value of the i-th sleep state; when i is 1, it represents the awake state. When i is 2, it indicates a sleep state. When i is 3, it indicates a light sleep state. When i is 4, it indicates a deep sleep state. ; Step 5: Quantify the current user's sleep state based on the obtained numerical values. By combining the range of sleep state quantification values under different states, the current user's sleep state is obtained; where the sleep state quantification values are... The range is 0-10, indicating that the user is awake; the quantification value for sleep status. A range of 11-20 indicates that the user is asleep; sleep state quantification value. A range of 21-30 indicates that the user is in a light sleep state; quantified sleep state values. A range of 31-40 indicates that the user is in a deep sleep state; Among them, if a quantitative value of sleep state is obtained If the user's current state does not match the state with the highest probability value among the four possible states, the state with the highest probability value will be taken as the current user's current state, and the current user's sleep state will be quantified. Modify it to the maximum value within the range of the corresponding state.
8. The sleep state recognition method according to claim 7, characterized in that: It also includes step 6: statistically analyzing the sleep state quantification values obtained from the current time and the previous L consecutive times, taking the most frequent state in the L+1 times as the current state, and averaging the corresponding sleep state quantification values of the current state to obtain the current user's sleep state quantification value V.
9. A sleep intervention method, characterized in that: The sleep state recognition method described in claim 7 is used to identify the user's state in real time, obtain the user's sleep state quantification value, and control the sleep aid device to intervene in the user's sleep based on the rate of change of the user's sleep quantification value within one minute within a specified time, so as to improve the user's sleep state quantification value. Between 21 and 30.
10. A computer system, characterized in that, include: One or more processors; The memory stores operable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations, including the flow of the sleep state recognition method as described in any one of claims 7-8.