Sleep state detection method and wearable device

Abstract

The invention provides a sleep state detection method and wearable equipment, and relates to the technical field of wearable equipment. The wearable device obtains PPG data and ACC data. Afterwards, the wearable device can process the PPG data to obtain an R-R interval RRI sequence, and process the ACC data to obtain a motion sequence. Afterwards, the wearable device can predict a sleep state of a wearer of the wearable device based on the RRI sequence and the motion sequence. Wherein the sleep state of the wearer comprises a rapid eye movement state, a deep sleep state, a light sleep state or a waking state. In the application, the accuracy of detecting the sleep state of the user can be improved, so that the wearing experience of the user is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of wearable devices, and in particular to a sleep state detection method and a wearable device. Background Art

[0002] With the rapid development of terminal technology, various types of electronic devices, such as mobile phones, tablet computers, and wearable devices represented by smart watches, have become indispensable products in people's lives. In daily use, users can often use wearable devices to detect their own physiological conditions to understand whether their bodies are healthy.

[0003] In some cases, smart watches can analyze the user's sleep state based on the detected physiological data (for example, heart rate, acceleration value during physical activity, etc.), and then determine the user's sleep quality. Therefore, how to accurately analyze the user's sleep state to improve the user's wearing experience has become an urgent problem to be solved. Summary of the invention

[0004] The embodiments of the present application provide a sleep state detection method and a wearable device, which are used to improve the accuracy of user sleep state detection, thereby enhancing the user's wearing experience.

[0005] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0006] In a first aspect, a sleep state detection method is provided, which is applied to a wearable device, in which the wearable device obtains PPG data and ACC data. Afterwards, the wearable device can process the PPG data to obtain an RR interval RRI sequence, and process the ACC data to obtain a motion sequence. Afterwards, the wearable device can predict the sleep state of the wearer of the wearable device based on the RRI sequence and the motion sequence; wherein the sleep state of the wearer includes rapid eye movement state, deep sleep state, light sleep state or awake state.

[0007] In this application, the wearable device can obtain four classification results including rapid eye movement state, deep sleep state, light sleep state and awake state through the RRI sequence and motion sequence. In this way, the wearer's sleep state can be fully detected, the probability of false detection of sleep state can be reduced, and the detection accuracy of sleep state can be improved, which provides a basis for the subsequent timely sleep intervention of the wearer according to the sleep state, and improves the wearer's user experience.

[0008] In a possible implementation manner of the first aspect, the process of the wearable device acquiring PPG data and ACC data may specifically include: when a wearer's wearing operation on the wearable device is detected, the wearable device collects PPG data in real time through a PPG sensor, and collects ACC data through an ACC sensor.

[0009] In this application, if the wearable device is worn on a designated part of the wearer's body, the wearable device can collect PPG data in real time through the PPG sensor, and collect ACC data through the ACC sensor. In this way, the wearer's sleep condition can be detected in real time, so that the wearable device can intervene in the wearer's sleep in time according to the sleep condition, so as to improve the wearer's sleep quality and facilitate the wearer to fall asleep better. In addition, since PPG data can also be used to detect the wearer's physiological condition (such as blood oxygen, blood pressure, etc.), if the PPG sensor can collect PPG data in real time, the wearable device can detect the wearer's physiological condition in real time according to the PPG data, so that the user can view his own physiological condition in real time, and can remind the user to pay attention to rest in time when the wearer's physiological condition is in an abnormal state, thereby improving the user's wearing experience. In addition, since ACC data can also be used to detect the wearer's exercise condition (such as the number of steps, etc.), if the ACC sensor can collect ACC data in real time, the wearable device can detect the wearer's exercise condition in real time according to the ACC data, so that the user can understand his own exercise condition, and then reasonably adjust the exercise plan to ensure that the wearer can exercise healthily.

[0010] In a possible implementation manner of the first aspect, the process of the wearable device acquiring PPG data and ACC data may specifically include: when a wearer's wearing operation on the wearable device is detected, the wearable device may acquire PPG data within a first preset time period, and acquire ACC data within a second preset time period.

[0011] In the present application, since the PPG data and ACC data are used to detect the sleep state of the wearer, and the wearer generally sleeps at a fixed time period, the wearable device can collect PPG data and ACC data only during the fixed time period, thereby improving the utilization rate of collection resources.

[0012] In a possible implementation of the first aspect, the process of the wearable device acquiring PPG data and ACC data may specifically include: the wearable device acquiring the ACC data. Then, the wearable device may perform feature extraction on the ACC data to obtain a motion sequence, wherein the motion sequence is used to indicate whether the wearer of the wearable device is in a motion state. Then, when the motion sequence indicates that the wearer is not in a motion state, the wearable device acquires PPG data.

[0013] In the present application, the wearable device needs to obtain PPG data only when the motion sequence indicates that the wearer is not in motion, that is, the wearer may be in a sleeping state. In this way, the accuracy of sleep state detection can be ensured while improving the utilization rate of acquisition resources, thereby reducing the waste of storage resources.

[0014] In a possible implementation of the first aspect, the process of the wearable device acquiring PPG data and ACC data may specifically include: when the first condition is met, the wearable device receives a data collection instruction from the electronic device. Thereafter, in response to the data collection instruction, the wearable device acquires the PPG data and ACC data.

[0015] Among them, the first condition includes at least one of the following: the current time is within the preset sleep period; and the electronic device is in a screen-off state, and the screen-off duration reaches a preset duration.

[0016] In this application, considering that the electronic device will automatically turn off the screen if it does not receive the wearer's touch operation for a long time, that is, the electronic device will switch from the on screen state to the off screen state, or, when the wearer presses the power off button, the electronic device can switch from the on screen state to the off screen state. Therefore, the wearable device can obtain PPG data and ACC data when receiving the data collection instruction sent by the electronic device. In this way, the waste of subsequent computing resources caused by invalid sensor collection can be reduced, and the utilization rate of computing resources can be improved.

[0017] In addition, considering that the wearer may not use the mobile phone for a long time due to work during the day, if the electronic device is in the off-screen state and the off-screen time reaches the preset time, it means that the wearer may fall asleep. Therefore, the wearable device can obtain PPG data and ACC data after receiving the data collection instruction sent by the electronic device. In this way, the smart watch can achieve accurate detection, reduce the waste of subsequent computing resources caused by invalid sensor collection, and improve the utilization rate of computing resources.

[0018] In a possible implementation of the first aspect, the process of the wearable device acquiring PPG data and ACC data may specifically include: the wearable device acquiring a pressure value between the wearable device and the wearer's wrist, wherein the pressure value is used to characterize the fit between the dial of the wearable device and the wearer's wrist. Thereafter, when the pressure value is within a preset pressure range, the wearable device may acquire the PPG data and the ACC data.

[0019] In this application, when the pressure value is within the preset pressure range, it means that the wearable device is worn appropriately, so the wearable device can directly obtain PPG data and ACC data. In this way, accurate data collection can be achieved, reducing the occurrence of inaccurate data collection caused by wearing the smart watch too loose or too tight, thereby improving the detection accuracy of the sleep state.

[0020] In a possible implementation of the first aspect, the method further includes: when the current time is within the preset sleep period and the pressure value is not within the preset pressure range, the wearable device may send a prompt message, wherein the prompt message is used to prompt the wearer to adjust the tightness of the wearable device.

[0021] In the present application, when the pressure value is not within the preset pressure range, it indicates that the wearable device is worn too loosely or too tightly. Therefore, the wearable device can output a prompt message when the current time is within the preset sleep period and the pressure value is not within the preset pressure range. In this way, the wearer can be notified in time to adjust the tightness of the wearable device, thereby providing a basis for subsequent accurate data collection, thereby improving the detection accuracy of the sleep state.

[0022] In a possible implementation of the first aspect, the method further includes: the wearable device interpolates the RRI sequence to obtain a target RRI sequence with the same sampling rate as the motion sequence. Afterwards, the wearable device can predict the sleep state of the wearer based on the target RRI sequence and the motion sequence.

[0023] In the present application, by interpolating the RRI sequence, the processed RRI sequence (that is, the target RRI sequence) can be made to have the same sampling rate as the motion sequence. In this way, time alignment between different sequences can be achieved, that is, different sequences at the same acquisition time can be aligned, reducing the occurrence of situations where the detection results of the sleep state are affected by differences in time points between sequences (such as the target RRI sequence of the first second and the motion sequence of the second second as input at the same time), which provides a basis for subsequent accurate detection of the sleep state.

[0024] In a possible implementation of the first aspect, the process of the wearable device predicting the sleep state of the wearer may specifically include: the wearable device may input the target RRI sequence and the motion sequence into a sleep state detection model to obtain the sleep state of the wearer. The sleep state detection model has the ability to predict the sleep state based on the target RRI sequence and the motion sequence.

[0025] In the present application, since the wearable device can output four-classification results through only one state detection model, it can reduce the waste of computing resources caused by multi-model detection, improve the utilization rate of computing resources, and thus improve the detection efficiency of sleep state.

[0026] In a possible implementation of the first aspect, the sleep state detection model includes a CNN layer, which is used to extract features from the target RRI sequence and the running sequence, and the CNN layer includes a convolution kernel; the process of the wearable device predicting the sleep state of the wearer may specifically include: the wearable device may continuously use the RRI sequence and the motion sequence within the first time period as input, run the sleep state detection model, and the sleep state detection model slides the convolution kernel with a preset step size to output the wearer's continuous sleep state within the first time period.

[0027] In the present application, the sleep state of the wearer during the time period in which the convolution kernel is located is detected by sliding the convolution kernel on the input data (that is, the RRI sequence and the motion sequence). In this way, real-time detection of the sleep state can be achieved, so that the smart watch can intervene in the wearer in time according to the real-time detection results, thereby improving the wearer's sleep condition and further improving the wearer's sleep quality.

[0028] In a possible implementation of the first aspect, the width of the convolution kernel is a preset convolution time, which is the time length of the input data included in a convolution kernel, and the height of the convolution kernel is the number of parameters of the input data, which includes an RRI sequence and a motion sequence.

[0029] In this application, since the height of the convolution kernel is the number of parameters of the input data, that is, the convolution kernel includes all input parameters, the sleep state can be detected more comprehensively, so that the smart watch can detect the wearer's sleep state more accurately, thereby providing a basis for subsequent timely sleep intervention of the wearer according to the sleep state, thereby improving the wearer's user experience.

[0030] In a possible implementation of the first aspect, the method further includes: the wearable device acquires skin resistance data. Afterwards, the wearable device may perform feature extraction on the skin resistance data to obtain a skin resistance sequence. The sampling rate of the skin resistance sequence is the same as the sampling rate of the motion sequence. Afterwards, the wearable device may predict the sleep state of the wearer of the wearable device based on the skin resistance sequence, the RRI sequence, and the motion sequence.

[0031] In this application, the wearable device can also predict the sleep state of the wearer based on the skin resistance sequence. In this way, the sleep state of the wearer can be detected from multiple angles, so that the smart watch can detect the sleep state of the wearer more accurately, thereby providing a basis for the subsequent sleep intervention of the wearer in a timely manner according to the sleep state, thereby improving the wearer's user experience.

[0032] In a possible implementation manner of the first aspect, the method further includes: the wearable device performing sleep intervention on the wearer according to the sleep state.

[0033] In this application, after obtaining the sleep state of the wearer, the wearable device can perform sleep intervention on the wearer according to the sleep state. In this way, the wearer's sleep condition can be improved, and a sleep improvement service matching the current sleep state can be provided, thereby improving the wearer's sleep quality.

[0034] In a second aspect, the present application provides a wearable device, comprising a PPG sensor, an ACC sensor, a display screen, a memory and one or more processors; the PPG sensor, the ACC sensor, the display screen, the memory and the processor are coupled; the PPG sensor is used to collect PPG data, and the ACC sensor is used to collect ACC data; the display screen is used to display an image generated by the processor, and the memory is used to store computer program code, and the computer program code includes computer instructions; when the processor executes the computer instructions, the wearable device executes the method described above.

[0035] In a possible implementation manner of the second aspect, the wearable device further includes a GSR sensor; the GSR sensor is coupled to the PPG sensor, the ACC sensor, the display screen, the memory, and the processor; and the GSR sensor is used to collect skin resistance data.

[0036] In a third aspect, the present application provides a computer-readable storage medium, comprising computer instructions, which, when executed on an electronic device, enables the electronic device to execute the method described above.

[0037] In a fourth aspect, the present application provides a computer program product, which, when executed on an electronic device, enables the electronic device to execute the method described above.

[0038] In a fifth aspect, a chip is provided, comprising: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected via an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the method as described above.

[0039] Among them, the beneficial effects that can be achieved by the electronic device described in the second aspect, the computer-readable storage medium described in the third aspect, the computer program product described in the fourth aspect, and the chip described in the fifth aspect provided above can refer to the beneficial effects in the first aspect and any possible design method thereof, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic diagram of wearing a wearable device provided in an embodiment of the present application;

[0041] Figure 2 A schematic diagram of detecting the sleep state of a wearer according to a PPG signal provided in an embodiment of the present application;

[0042] Figure 3 A schematic diagram of detecting the sleep state of a wearer according to an ACC signal provided in an embodiment of the present application;

[0043] Figure 4 A schematic diagram of the hardware structure of a smart watch provided in an embodiment of the present application;

[0044] Figure 5 A flowchart of a sleep state detection method provided in an embodiment of the present application;

[0045] Figure 6 A schematic diagram of a waveform of a photoplethysmography signal provided in an embodiment of the present application;

[0046] Figure 7 A schematic diagram of a target RRI sequence and a motion sequence when the sampling rates are the same provided in an embodiment of the present application;

[0047] Figure 8 A schematic diagram of a convolution kernel sliding on input data provided in an embodiment of the present application;

[0048] Fig. 9 A schematic diagram of a process for detecting the sleep state of a wearer provided in an embodiment of the present application;

[0049] Fig.10A schematic diagram of a sleep state detection process provided in an embodiment of the present application;

[0050] Fig.11 A schematic diagram of a target RRI sequence, a motion sequence, and a skin resistance sequence when the sampling rates are the same provided in an embodiment of the present application;

[0051] Fig.12 A schematic diagram of the structure of a communication system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The technical scheme in the embodiment of the present application will be described below in conjunction with the accompanying drawings in the embodiment of the present application. Wherein, in the description of the present application, unless otherwise specified, the "and / or" in the present application is only a kind of association relationship describing the associated object, indicating that there can be three kinds of relationships, for example, A and / or B, which can be represented by: A exists alone, A and B exist at the same time, and B exists alone, wherein A, B can be singular or plural. And, in the description of the present application, unless otherwise specified, "multiple" refers to two or more than two. "At least one of the following (individuals)" or its similar expressions refers to any combination of these items, including any combination of single items (individuals) or plural items (individuals). For example, at least one of a, b, or c (individuals) can be represented by: a, b, c, ab, ac, bc, or abc, wherein a, b, c can be single or multiple. In addition, in order to facilitate the clear description of the technical scheme of the embodiment of the present application, in the embodiment of the present application, the words "first", "second" and the like are used to distinguish the same items or similar items with substantially the same functions and effects. Those skilled in the art will appreciate that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit the difference. At the same time, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way for easy understanding.

[0053] In some application scenarios, wearable devices (such as Figure 1 The smart watch 200 shown can be fixed on the wrist of the wearer (or user) through a strap. The wearable device can detect the wearer's physiological parameters and display the physiological parameters (such as heart rate) to let the wearer know the current physiological condition.

[0054] In some embodiments, Figure 2As shown, the wearable device can obtain the PPG signal (or PPG data) obtained by the photoplethysmographic (PPG) sensor to obtain the RR interval (RR interval, RRI) sequence. Among them, the RR interval refers to the time limit between two adjacent R waves on the electrocardiogram. In the QRS complex, the inflection point where the Q wave is located is the starting point of the R wave. The time from this starting point to the next starting point is called the RR interval, that is, the interval time of each heart beat. The QRS complex is used to reflect the changes in the depolarization potential and time of the left and right ventricles. Under normal circumstances, the time limit of the RR interval should be between 0.6-1.0 seconds. If the RR interval is less than 0.6 seconds, it means that the wearer has tachycardia; if the RR interval is less than or greater than 1.0 seconds, it means that the wearer has bradycardia. In addition, when the RR intervals are not equal, it means that the wearer has arrhythmia, and obvious RR intervals will appear on the electrocardiogram of atrial fibrillation.

[0055] Specifically, the wearable device can obtain the PPG signal collected by the PPG sensor. After that, the wearable device can perform peak detection on the PPG signal to obtain the RRI sequence.

[0056] When the above-mentioned RRI sequence is obtained, the wearable device can extract heart rate variability (HRV) information from the RRI sequence. The HRV information may include at least one of the mean, standard deviation and root mean square of the difference between adjacent RR intervals of the RRI sequence per minute. Afterwards, the wearable device can detect the wearer's first sleep state based on the HRV information to obtain the wearer's sleep condition. The sleep condition is used to characterize the wearer's sleep quality. It can be understood that the longer the wearer is in a deep sleep state, the higher the wearer's sleep quality; the longer the wearer is in a rapid eye movement state, the lower the wearer's sleep quality.

[0057] In some embodiments, the wearable device may input the above HRV information into a first state detection model to obtain the first sleep state of the wearer. Among them, the first sleep state may include at least one of a rapid eye movement (REM) state, a deep sleep state, and a shallow sleep state. The rapid eye movement state refers to a sleep state in which the brain wave frequency becomes faster and the amplitude becomes lower during sleep, and also shows signs such as increased heart rate, increased blood pressure, muscle relaxation, and constant left and right swinging of the eyeballs. The deep sleep state is also called a deep sleep state, which refers to a sleep state in which the cerebral cortex cells are in a fully resting state. The shallow sleep state is also called a light sleep state, which refers to a sleep state in which the body's activity frequency is high and the sleep sensitivity is high. That is to say, users in a shallow sleep state are prone to turning over and dreaming, and are more likely to be awakened than users in a deep sleep state.

[0058] The first state detection model is a model that can detect the user's first sleep state. Exemplarily, the first state detection model can be a decision tree model, a support vector machine (SVM) model, a temporal convolutional network (TCN), etc. As long as it is a classification model, it can be used as the first state detection model, and there is no specific limitation.

[0059] It can be seen that the first sleep state of the wearer detected by the above wearable device through the PPG signal only includes the rapid eye movement state, deep sleep state and light sleep state, and does not include the awake state. In other words, the sleep state detection method cannot fully detect the sleep state of the wearer, which makes the sleep state detection method limited, and then the sleep state may be misdetected due to incomplete sleep state detection (such as misdetecting a sleep state that is not in the first sleep state as the first sleep state, that is, misdetecting the awake state as the rapid eye movement state), which makes the sleep state detection accuracy low, and ultimately affects the user's wearing experience.

[0060] In other embodiments, Figure 3 As shown, the wearable device can obtain a motion sequence through an accelerometer (ACC) sensor to obtain an ACC signal (or ACC data). The motion sequence is used to characterize whether the wearer is in motion. Specifically, the wearable device can obtain an ACC signal collected by the ACC sensor. Afterwards, the wearable device can perform feature extraction on the ACC signal to obtain the above motion sequence.

[0061] Afterwards, the wearable device can detect the second sleep state of the wearer according to the above motion sequence to obtain the sleep condition of the wearer. The second sleep state may include a sleep state and / or a wake state. The sleep state refers to the state of a person when sleeping. The wake state refers to the state of consciousness in which the cerebral cortex and the entire body are awake. The sleep condition is used to characterize whether the wearer is in a sleep state.

[0062] In one implementation, the wearable device can input the above motion sequence into a state machine to obtain the second sleep state of the wearer. The state machine can perform state transition according to a preset sleep state (or second sleep state) according to the motion sequence, that is, the wearable device can determine whether the wearer is in a sleep state according to the motion sequence through the state machine.

[0063] In another implementation, the wearable device may input the above motion sequence into a second state detection model to obtain the second sleep state of the wearer. The second state detection model is a model that can detect the second sleep state of the user. Exemplarily, the second state detection model may be the same as the above first state detection model, or may be different from the above first state detection model, without specific limitation.

[0064] Exemplarily, the training process of the second state detection model may specifically include: the wearable device obtains a motion sequence set, wherein the motion sequence set includes multiple motion sequences, and each motion sequence carries a label of the second sleep state. Afterwards, the wearable device may train the pre-built second state detection model according to the motion sequence set to obtain a trained second state detection model, thereby realizing classification of the wearer's second sleep state.

[0065] It can be seen that the second sleep state of the wearer detected by the above wearable device through the ACC signal only includes the sleep state and the awake state, and cannot distinguish the sleep state. In other words, the sleep state detection method cannot detect the wearer's sleep condition, and cannot further analyze the wearer's sleep quality, and then intervene in the wearer's sleep, resulting in a low wearing experience for the user.

[0066] Therefore, in order to accurately detect the sleep state of the wearer, an embodiment of the present application provides a sleep state detection method, in which a wearable device obtains an RRI sequence and a motion sequence. The RRI sequence is obtained based on the PPG signal, and the motion sequence is obtained based on the ACC signal. Afterwards, the wearable device can input the RRI sequence and the motion sequence into the state detection model to obtain the user's sleep state. The sleep state includes rapid eye movement state, deep sleep state, light sleep state or awake state.

[0067] In an embodiment of the present application, the wearable device simultaneously inputs the RRI sequence and the motion sequence into the state detection model, and can obtain four classification results including rapid eye movement state, deep sleep state, light sleep state and awake state. In this way, not only can the wearer's sleep state be fully detected, the probability of false detection of the sleep state can be reduced, the detection accuracy of the sleep state can be improved, and the user's wearing experience can be improved. In addition, the wearable device can output four classification results through only one state detection model, which can reduce the waste of computing resources caused by multi-model detection, improve the utilization of computing resources, and thus improve the detection efficiency of the sleep state.

[0068] Exemplarily, the wearable device is a device such as a smart watch or a smart bracelet that is in contact with the wearer and can collect the wearer's physiological parameters. Figure 4 , taking the wearable device as a smart watch as an example, the hardware structure of the wearable device is described.

[0069] like Figure 4 As shown, the smart watch 200 includes: a watch body and a wristband (or watch strap) connected to each other, wherein the watch body may include a front shell ( Figure 4 (not shown), touch screen 210 (also called touch panel), display screen 220, bottom case ( Figure 4 2), a processor 230, a memory 250, a microphone (MIC) 260, a communication module 270, a PPG sensor 281, an ACC sensor 282, and an ambient light sensor 283. Although not shown, the smart watch may further include a power supply, a power management system, an antenna, a speaker, a gyroscope sensor, etc. Those skilled in the art will appreciate that Figure 4 The smart watch structure shown in the figure does not constitute a limitation on the smart watch, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0070] The functional components of the smart watch 200 are introduced below:

[0071] The touch panel 210, also known as the touchpad, can collect touch operations performed by the watch user thereon (such as operations performed by the user using fingers, stylus, or any other suitable object or accessory on or near the touch panel) and drive a corresponding connected device according to a pre-set program.

[0072] The display screen 220 can be used to display information input by the user or information provided to the user and various menus of the watch. Optionally, the display screen 220 can be configured in the form of LCD, OLED, etc. Further, the touch panel 210 can cover the display screen 220. When the touch panel 210 detects a touch operation on or near it, it is transmitted to the processor 230 to determine the type of touch event, and then the processor 230 provides corresponding visual output on the display screen 220 according to the type of touch event. Although in Figure 4 In the embodiment, the touch panel 210 and the display screen 220 are used as two independent components to realize the input and output functions of the watch, but in some embodiments, the touch panel 210 and the display screen 220 can be integrated to realize the input and output functions of the watch.

[0073] The processor 230 is used to perform system scheduling, control the display screen, touch screen, process data sent by sensors (such as the above-mentioned PPG sensor 281, ACC sensor 282, ambient light sensor 283), etc. The processor 230 can also be called a main control unit, which can include a computing unit that can process data, etc.

[0074] The memory 250 is used to store software programs and data. The processor 230 executes various functional applications and data processing of the watch by running the software programs and data stored in the memory. The memory 250 mainly includes a program storage area and a data storage area, wherein the program storage area can store an operating system and an application required for at least one function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created by using the watch (such as audio data, a phone book, etc.). In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as a disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0075] Communication module 270, the smart watch can exchange information with other electronic devices (such as mobile phones, tablet computers, etc.) through the communication module 270. Exemplarily, the communication module 270 may include a wireless communication module and a mobile communication module. Optionally, the wireless communication module may include a Bluetooth (BT) module, a global navigation satellite system (GNSS), and a wireless local area network (WLAN) (such as a wireless fidelity (Wi-Fi) network). The mobile communication module can provide solutions for wireless communications such as 2G / 3G / 4G / 5G applied to smart watches.

[0076] The PPG sensor 281 can be used to measure the user's heart rate, and / or physiological data of the human body such as blood oxygen saturation. Taking the detection of the user's heart rate as an example, in a specific implementation, the PPG sensor 281 can emit a light signal of a specified wavelength, which can illuminate the arterial blood vessels under the skin tissue and be reflected back to the PPG sensor 281. When the heart beats, the contraction and expansion of the blood vessels will change the blood volume in the arterial blood vessels, thereby affecting the absorption or attenuation of the light signal by the arterial blood vessels, thereby affecting the reflection of the light signal. The PPG sensor 281 can detect the user's heart rate based on the changes in the reflected light signal. It should be noted that the PPG sensor 281 can also detect the user's heart rate in other ways, and the present application does not constitute a limitation to this.

[0077] The ACC sensor 282 can be used to detect the magnitude of the acceleration of the smart watch 200 in all directions (generally three axes, i.e., the x-axis, y-axis, and z-axis). In some embodiments, the ACC sensor 282 can detect the number of steps, step frequency, and stride length of the user in different sports modes such as walking, running, or cycling, and provide real-time sports data for the user. In other embodiments, the ACC sensor 282 can also detect the user's sleeping state, that is, whether the user is in a sleeping state or in a wakeful state.

[0078] The ambient light sensor 283 is used to detect the lighting conditions of the environment in which the smart watch is located.

[0079] It should be understood that the illustrated smart watch 200 is merely one example of a wearable device, and that the smart watch 200 may have more or fewer components than shown, may combine two or more components, or may have a different configuration of components. Figure 4 The various components shown in the EMBODIMENTS 2000 may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0080] Based on the electronic device described above, an embodiment of the present application provides a method for detecting a sleep state. The method can be applied to a wearable device health detection (such as sleep detection) scenario, that is, the wearable device in the method is an electronic device with a health detection function. The following will take a smart watch as an example to illustrate the method of the embodiment of the present application. Specifically, Figure 5 As shown, the sleep state detection method may include S601 to S604.

[0081] S601, the smart watch collects PPG data through the PPG sensor and collects ACC data through the ACC sensor.

[0082] Specifically, when the wearer's wearing operation on the smart watch is detected, the smart watch can collect PPG data through the PPG sensor and collect ACC data through the ACC sensor. It can be understood that only when the smart watch is worn on a designated part of the wearer's body (such as the wrist) can the smart watch detect the wearer's sleep state and then determine whether the wearer's current sleep quality is good, that is, whether sleep intervention is needed for the wearer. If the smart watch is not worn on the designated part of the wearer's body, the smart watch does not need to collect data to avoid unnecessary power consumption loss.

[0083] Generally, the above-mentioned smart watch can transmit a light signal into the wearer's skin tissue (or can be understood as blood, blood vessels, etc. in the skin tissue) through a PPG sensor (such as a light-emitting diode (LED) in a PPG sensor). A PPG sensor (such as a photodiode (PD) in a PPG sensor) can receive the light signal reflected back through the skin tissue. The light signal includes the pulsation information of the wearer's blood vessels, and the pulsation information is used as PPG data. It should be understood that after the PD receives the reflected light signal, it can convert the light signal into an electrical signal, and after analog-to-digital conversion, convert the electrical signal into a digital signal that can be used by the smart watch. The above-mentioned PPG data may actually refer to the digital signal.

[0084] In some embodiments, if the smart watch is worn on a designated part of the wearer's body, the smart watch can collect PPG data in real time through the PPG sensor, and collect ACC data through the ACC sensor. In this way, the wearer's sleep condition can be detected in real time, so that the smart watch can intervene in the wearer's sleep in time according to the sleep condition, so as to improve the wearer's sleep quality and facilitate the wearer to fall asleep better. In addition, since PPG data can also be used to detect the wearer's physiological condition (such as blood oxygen, blood pressure, etc.), if the PPG sensor can collect PPG data in real time, the smart watch can detect the wearer's physiological condition in real time according to the PPG data, so that the user can check his own physiological condition in real time, and can remind the user to take a rest in time when the wearer's physiological condition is in an abnormal state, thereby improving the user's wearing experience. In addition, since ACC data can also be used to detect the wearer's exercise condition (such as the number of steps, etc.), if the ACC sensor can collect ACC data in real time, the smart watch can detect the wearer's exercise condition in real time according to the ACC data, so that the user can understand his own exercise condition, and then reasonably adjust the exercise plan to ensure that the wearer can exercise healthily.

[0085] In other embodiments, in order to reduce unnecessary waste of resources, the smart watch can collect PPG data through the PPG sensor within a first preset time period, and collect ACC data through the ACC sensor within a second preset time period. The first preset time period and the second preset time period may be the same or different, and are not specifically limited. It can be understood that since the PPG data and ACC data in the embodiments of the present application are used to detect the sleep state of the wearer, and the wearer generally sleeps at a fixed time period, the smart watch can only collect PPG data and ACC data within the fixed time period, so that the utilization rate of the collection resources can be improved.

[0086] Generally, the ACC data is used to detect whether the wearer is in a sleeping state, that is, the smart watch only needs to judge whether the wearer has fallen asleep based on the ACC data, and the PPG data is used to detect which stage of the sleep stage (that is, the first sleep state) the wearer is in, that is, after the wearer has fallen asleep, the smart watch can detect the sleep stage of the wearer through the PPG data. Among them, the first sleep state includes at least one of the rapid eye movement state, the deep sleep state and the light sleep state. Therefore, the smart watch can collect PPG data through the PPG sensor when the ACC data is collected through the ACC sensor and the motion sequence obtained through the ACC data indicates that the wearer is not in a motion state, that is, the wearer may be in a sleeping state. In other words, the smart watch can not collect PPG data first, that is, only collect ACC data. And, when the wearer is in a sleeping state, the smart watch can not collect ACC data, but only collect PPG data. In this way, the utilization rate of collection resources can be improved while ensuring the accuracy of sleep state detection, thereby reducing the waste of storage resources.

[0087] In one implementation, the first preset time period and the second preset time period may be pre-set according to actual conditions. In one example, taking the first preset time period and the second preset time period as the same, if most wearers sleep between 8 pm and 8 am the next day, the smart watch may set the time period between 8 pm and 8 am the next day as the first preset time period and the second preset time period. In another example, taking the first preset time period and the second preset time period as different, if most wearers fall asleep between 8 pm and 11 pm, and the ACC data corresponding to the second preset time period is only used to detect whether the wearer is in a sleeping state, the smart watch may set the time period between 8 pm and 11 pm as the second preset time period. If most wearers are sleeping between 10 pm and 8 am the next day, and the PPG data corresponding to the first preset time period is only used to detect which sleep stage the wearer is in, the smart watch may set the time period between 10 pm and 8 am the next day as the first preset time period.

[0088] In another implementation, the first preset time period and the second preset time period can be set according to the historical sleep status of the wearer of the smart watch. In one example, taking the first preset time period and the second preset time period as the same, if the wearer of the smart watch usually sleeps between 11 pm and 7 am the next day, the smart watch can set the time period between 11 pm and 7 am the next day as the first preset time and the second preset time period. In another example, taking the first preset time and the second preset time period as different, if the wearer of the smart watch usually falls asleep between 11 pm and 1 am the next day, and the ACC data corresponding to the second preset time period is only used to detect whether the wearer is in a sleeping state, the smart watch can set the time period between 11 pm and 1 am the next day as the second preset time period. If the wearer of the smart watch usually sleeps between 12 pm and 8 am the next day, and the PPG data corresponding to the first preset time period is only used to detect which sleep stage the wearer is in, the smart watch can set the time period between 12 pm and 8 am the next day as the first preset time period.

[0089] It can be understood that when the wearer is in a sleeping state or about to fall asleep, the electronic device (such as a mobile phone) will not be triggered to perform the corresponding operation, that is, the electronic device will not receive any touch operation (such as a click operation, etc.) of the wearer, and the electronic device belonging to the wearer will establish a communication connection with the above-mentioned smart watch. If the electronic device does not receive the wearer's touch operation for a long time, the screen will automatically turn off, that is, the electronic device will be converted from the screen-on state to the screen-off state, or, when the wearer presses the power-off button, the electronic device can be converted from the screen-on state to the screen-off state. Therefore, the electronic device can be in the screen-off state, and the screen-off time reaches the preset time. Send a data collection instruction to the smart watch. Among them, the data collection instruction is used to indicate that the wearer is resting, so as to instruct the smart watch to trigger the PPG sensor to collect PPG data, and to trigger the ACC sensor to collect ACC data. Afterwards, when the smart watch receives the data collection instruction, in response to the data collection instruction, the smart watch can collect PPG data through the PPG sensor, and collect ACC data through the ACC sensor. In this way, the waste of subsequent computing resources caused by invalid sensor collection can be reduced, and the utilization rate of computing resources can be improved.

[0090] Furthermore, considering that the wearer may not use the mobile phone for a long time due to daytime work, the electronic device can first determine whether the current time (or the first time) is within the preset sleep period (such as 22:00-9:00 am the next day). Only when the current time is within the preset sleep period, if the electronic device is in the screen-off state, and the screen-off time reaches the preset time, the electronic device can send the above data collection instruction to the smart watch. In this way, accurate detection of the smart watch can be achieved, reducing the waste of subsequent computing resources caused by invalid sensor collection, and improving the utilization rate of computing resources.

[0091] Among them, the above-mentioned preset sleep period can be pre-set according to the actual situation (such as the general sleep situation of most wearers), or it can be set according to the historical sleep situation of the wearer of the smart watch, and there is no specific limitation.

[0092] In some embodiments, it is considered that the tightness with which the wearer wears the smart watch will affect the accuracy of the data collected by the sensor. That is to say, if the smart watch is worn too loosely, or the smart watch is worn too tightly, the PPG data and ACC data will be collected inaccurately, thereby affecting the subsequent detection accuracy of the sleep state. Therefore, the smart watch can collect the pressure value between the smart watch and the wearer's wrist through the pressure sensor, wherein the pressure value is used to characterize the fit between the dial of the smart watch and the wearer's wrist. Afterwards, the ACC data and PPG data are collected only when the pressure value is within the preset pressure range. In this way, accurate data collection can be achieved, reducing the occurrence of inaccurate data collection due to the smart watch being worn too loosely or too tightly, thereby improving the detection accuracy of the sleep state.

[0093] It can be understood that when the above pressure value is within the preset pressure range, it means that the wearing degree of the smart watch is relatively appropriate. Therefore, the smart watch can directly collect PPG data through the PPG sensor, and collect ACC data through the ACC sensor, that is, there is no need to output prompt information about the wearing tightness. When the pressure value is not within the preset pressure range, it means that the smart watch is worn too loose or too tight. Therefore, in order to improve the accuracy of data collection and further improve the accuracy of sleep state detection, the smart watch can output prompt information when the current time is within the preset sleep period and the pressure value is not within the preset pressure range. Among them, the prompt information is used to prompt the wearer to adjust the wearing tightness of the smart watch.

[0094] The above-mentioned preset pressure interval is preset according to actual conditions. Specifically, when the pressure value is greater than the first preset pressure and less than the second preset pressure, the smart watch can determine that the pressure value is within the preset pressure interval. The first preset pressure is less than the second preset pressure. In other words, the preset pressure interval can be determined based on the first preset pressure and the second preset pressure.

[0095] In one example, when the pressure value is less than or equal to the first preset pressure, it means that the smart watch is worn too loosely. Therefore, in order to improve the accuracy of data collection, the smart watch can output a first prompt message to prompt the user that the current wearing degree is relatively loose. Exemplarily, the first prompt message output by the smart watch can be a voice output of the first prompt message (such as the voice output of the smart watch that the wearing degree is relatively loose, please wear it tighter) and / or the smart watch displays the first prompt message (such as the smart watch displays the prompt message "The wearing degree is relatively loose, please wear it tighter"). In this way, the occurrence of data collection being affected by wearing the smart watch too loose can be reduced, the accuracy of data collection can be improved, and the detection accuracy of the sleep state can be improved.

[0096] In one example, when the pressure value is greater than or equal to the second preset pressure, it means that the smart watch is worn too tightly. Therefore, in order to improve the accuracy of data collection, the smart watch can output a second prompt message to prompt the user that the current wearing degree is too tight. Exemplarily, the smart watch outputs the second prompt message by voice output (such as the smart watch outputs a prompt message that the wearing degree is too tight, please wear it looser), and / or the smart watch displays the second prompt message (such as the smart watch displays a prompt message that "the wearing degree is too tight, please wear it looser"). In this way, the occurrence of data collection being affected by wearing the smart watch too tightly can be reduced, the accuracy of data collection can be improved, and the detection accuracy of the sleep state can be improved.

[0097] It should be noted that the personal information used in the technical solution of this application is limited to information for which the individual’s separate consent has been obtained, including but not limited to notifying and reminding the user to read the relevant user agreement (notification) and sign the agreement (authorization) including authorization of relevant user information before the user uses the function.

[0098] Specifically, since the above-mentioned PPG data and ACC sensor can be used to detect the current sleep state of the wearer, the smart watch can timely intervene in the wearer's sleep according to the sleep state to improve the wearer's sleep quality. Therefore, in order to protect the user's privacy, it is necessary to determine whether the wearer has turned on the sleep detection function before the PPG sensor collects PPG data and the ACC sensor collects ACC data. When the sleep detection function is turned on, that is, when the smart watch receives the wearer's operation to turn on the sleep detection function, the smart watch can collect PPG data through the PPG sensor and ACC data through the ACC sensor to determine the wearer's sleep state.

[0099] It should be noted that the order in which the smart watch collects PPG data and ACC data is not limited. For example, the smart watch can collect PPG data through the PPG sensor and ACC data through the ACC sensor at the same time. For another example, the smart watch can first collect PPG data through the PPG sensor, and then collect ACC data through the ACC sensor. For another example, the smart watch can also first collect ACC data through the ACC sensor, and then collect PPG data through the PPG sensor.

[0100] S602, the smart watch processes the above PPG data and ACC data respectively to obtain a target detection sequence, wherein the target detection sequence includes an RRI sequence and a motion sequence.

[0101] Specifically, after collecting the above-mentioned PPG data and ACC data, the smart watch can process the PPG data and ACC data respectively to obtain a target detection sequence. Among them, the target detection sequence is used to detect the current sleep state of the wearer. The target detection sequence may include an RRI sequence and a motion sequence. The RRI sequence is used to characterize the interval time of each heart beat, that is, the interval time of each heart beat. The motion sequence is used to characterize whether the wearer is in motion.

[0102] In one implementation, for the above PPG data, the smart watch can perform peak detection on the PPG data to obtain an RRI sequence. In one example, the smart watch can perform peak detection on the PPG data by finding the local maximum value (or peak) and minimum value (or trough) in the PPG data. In another example, the smart watch can also perform peak detection on the PPG data by using a first-order derivative (such as a Li derivative) with an adaptive threshold.

[0103] In some embodiments, the smart watch can perform peak detection on the above PPG data to obtain the peak point of at least one waveform in the PPG data. Afterwards, the smart watch can calculate the distance between the peak points of adjacent waveforms and determine the distance as an RR interval. Afterwards, the smart watch can determine the above RRI sequence based on the RR intervals corresponding to the peak points of all waveforms in the PPG data. In other words, the RRI sequence is obtained based on the RR intervals corresponding to the peak points of all waveforms in the PPG data.

[0104] It should be noted that the waveform of the PPG signal can reflect the physiological characteristics of the wearer. For example, Figure 6 As shown, taking the PPG signal including 4 waveforms as an example, each waveform represents a complete pulse waveform cycle. Among them, point A1 in waveform 1, point A2 in waveform 2, point A3 in waveform 3, and point A4 in waveform 4 are all peak points, that is, R waves. In some embodiments, the wearable device can determine the RR interval, that is, RR1, by calculating the distance between point A1 and point A2, and can determine RR2 by calculating the distance between point A2 and point A3, and can determine RR3 by calculating the distance between point A3 and point A4.

[0105] In another implementation, for the above ACC data, the smart watch can perform feature extraction on the ACC data to obtain a motion sequence.

[0106] In some embodiments, the smart watch can calculate the above ACC data according to a preset calculation interval to obtain a motion value. It can be understood that the above motion sequence can include at least one motion value, that is, the motion sequence is composed based on at least one motion value. Afterwards, the smart watch can determine whether the wearer is in motion based on the motion value. In one example, the motion value can be obtained by calculating energy. In another example, the motion value can also be obtained by calculating variance.

[0107] It should be noted that the above preset calculation interval can be set according to actual needs. In this embodiment, in order to facilitate subsequent sleep state detection, the preset calculation interval can be set to 1s, that is, the smart watch calculates the exercise value once every 1s. In other embodiments, the preset calculation interval can also be set to 5s, 10s, etc., without specific limitation.

[0108] In one implementation, the smart watch can determine whether the exercise value is greater than a preset value. If the exercise value is greater than the preset value, the smart watch can determine that the wearer is in motion; or, if the exercise value is less than or equal to the preset value, the smart watch can determine that the wearer is stationary, that is, the wearer is not in motion.

[0109] For example, taking the above motion value obtained by energy-based calculation as an example, the motion value can be calculated by the following formula 1:

[0110]

[0111] Among them, E() is energy, that is, the motion value; i is the above-mentioned preset interval (such as 1s); t is the sampling rate of ACC data (such as 100Hz); x is the acceleration value of the smart watch on the x-axis; y is the acceleration value of the smart watch on the y-axis; z is the acceleration value of the smart watch on the z-axis.

[0112] Specifically, after the smart watch obtains the exercise value through the above formula 1, the smart watch can determine whether the exercise value is greater than a preset value (that is, a preset energy value). If the exercise value is greater than the preset energy value, the smart watch can determine that the wearer is in an exercise state. If the exercise value is less than or equal to the preset energy value, the smart watch can determine that the wearer is not in an exercise state.

[0113] For example, when the exercise value is calculated based on the variance, the smart watch can determine whether the exercise value is greater than a preset value (that is, a preset variance value). If the exercise value is greater than the preset variance value, the smart watch can determine that the wearer is in exercise. If the exercise value is less than or equal to the preset variance value, the smart watch can determine that the wearer is not in exercise.

[0114] S603, the smart watch interpolates and normalizes the above RRI sequence to obtain a target RRI sequence.

[0115] Specifically, after obtaining the above RRI sequence, the smart watch can perform interpolation processing on the RRI sequence to obtain an interpolated RRI sequence.

[0116] It should be noted that the above-mentioned RRI sequence is used to characterize the interval time of each heartbeat, and the interval time of the heartbeat of the same wearer at different sampling times is also different. The above-mentioned motion sequence is calculated according to the preset calculation interval and the above-mentioned ACC data, that is to say, the interval time corresponding to the motion sequence is the same, that is, the sampling rate of the motion sequence is fixed. Therefore, in order to ensure the accuracy of subsequent sleep state detection, the smart watch can interpolate the RRI sequence to obtain the interpolated RRI sequence. In this way, it can ensure that the sampling rate of the target RRI sequence is consistent with the sampling rate of the motion sequence, thereby providing a basis for the subsequent accurate detection of the sleep state.

[0117] Exemplarily, the above-mentioned smartwatch may interpolate the RRI sequence by polynomial interpolation, spline function interpolation, etc., without specific limitation. Among them, the polynomial interpolation is to use the function values ​​of the function f(x) at several known points in a certain interval to make an appropriate specific function (or polynomial), and use the value of this specific function as the approximate value of the function f(x) at other points in the interval. The spline function interpolation is a method of simulating the original data in the form of a spline function. The spline function is a type of piecewise (piece) smooth function that also has a certain degree of smoothness at the intersection of each segment.

[0118] In one implementation, after obtaining the above-mentioned interpolated RRI sequence, the smart watch can normalize the interpolated RRI sequence to obtain a target RRI sequence. Among them, the sampling rate of the target RRI sequence is the same as the sampling rate of the above-mentioned motion sequence. It can be understood that by normalizing the interpolated RRI sequence, the interpolation result can be improved, that is, the interpolated RRI sequence can be improved, so that the target RRI sequence can be spliced ​​with the above-mentioned motion sequence into a time series of the same length (such as 1s), that is, the sampling time corresponding to the target RRI sequence and the motion sequence is the same, which is the above-mentioned preset calculation interval. In this way, it can provide a basis for subsequently improving the accuracy of sleep state detection.

[0119] For example, Figure 7 As shown, after the above RRI sequence is interpolated and normalized, the sampling time corresponding to the target RRI sequence and the above motion sequence is the same. Taking the sampling time of 1 second as an example, the first column of cells in the schematic diagram includes the data corresponding to the target RRI sequence and the motion sequence in the first second. The second column of cells includes the data corresponding to the target RRI sequence and the motion sequence in the second second. The third column of cells includes the data corresponding to the target RRI sequence and the motion sequence in the third second. The fourth column of cells includes the data corresponding to the target RRI sequence and the motion sequence in the fourth second. The fifth column of cells includes the data corresponding to the target RRI sequence and the motion sequence in the fifth second. The sixth column of cells includes the data corresponding to the target RRI sequence and the motion sequence in the sixth second. The seventh column of cells includes the data corresponding to the target RRI sequence and the motion sequence in the seventh second.

[0120] In some embodiments, the smart watch may not perform step S603, that is, the smart watch may directly detect the sleep state of the wearer based on the above RRI sequence and motion sequence, without interpolating or normalizing the RRI sequence. In this way, the detection efficiency of the sleep state may be improved.

[0121] In other embodiments, the smart watch may only interpolate the RRI sequence without normalizing the interpolated RRI sequence. Afterwards, the smart watch can directly detect the sleep state of the wearer based on the interpolated RRI sequence and the motion sequence. In this way, the detection efficiency of the sleep state can be improved.

[0122] S604, the smart watch inputs the above target RRI sequence and motion sequence into the sleep state detection model to obtain the sleep state of the wearer.

[0123] Specifically, after obtaining the above target RRI sequence, the smart watch can predict the wearer's sleep state based on the target RRI sequence and the motion sequence. The sleep state is used to characterize the wearer's current sleep state. The sleep state may include rapid eye movement state, deep sleep state, light sleep state or awake state.

[0124] In one implementation, the smartwatch may input the target RRI sequence and motion sequence into a sleep state detection model to obtain the sleep state of the wearer, wherein the sleep state detection model has the ability to predict the sleep state based on the target RRI sequence and motion sequence.

[0125] In some embodiments, after obtaining the above-mentioned target RRI sequence, the smart watch can first vector-combine the target RRI sequence and the motion sequence to obtain a combined sequence table. Afterwards, the smart watch can input the combined sequence table into the sleep state detection model to obtain the sleep state of the wearer. It can be understood that by interpolating and normalizing the above-mentioned RRI sequence, the sampling rate of the target RRI sequence and the motion sequence can be made the same. Therefore, by vector-combining the target RRI sequence and the motion sequence, time alignment between different sequences can be achieved, that is, different sequences under the same acquisition time are aligned, and the occurrence of the detection result of the sleep state affected by the difference in time points between sequences (such as the target RRI sequence of the first second and the motion sequence of the second second as input at the same time) is reduced, which provides a basis for the subsequent accurate detection of the sleep state.

[0126] Among them, the above-mentioned sleep state detection model is a classification model. In this embodiment, the classification model can be a convolutional neural network (CNN). Specifically, the classification process of the convolutional neural network is to obtain the score of the current sample belonging to each class through the fully connected layer after the convolution layer and the pooling layer, and then classify according to the scores of different classes through the normalized exponential function (softmax) to obtain the class to which the current sample belongs. Among them, softmax is an activation function for multi-classification problems. In other embodiments, the classification model can also be a decision tree, a support vector machine (SVM), etc., as long as it is a classification model, it can be used as a sleep state detection model, without specific limitation.

[0127] In one implementation, the sleep state detection model may include at least one of N layers of convolutional neural networks, one layer of long short-term memory (LSTM), one layer of full connection, and softmax. In this implementation, N is 3. In other implementations, N may also be other values, for example, N may be 2, 5, etc., without specific limitation.

[0128] Among them, the CNN layer (or convolutional layer) is used to perform feature extraction operations on the above-mentioned target RRI sequence and motion sequence. The LSTM layer is used to process data with time series properties, that is, it is used to process the target RRI sequence and motion sequence. The fully connected layer is used to connect the features extracted by the convolutional layer and output the final classification result. In other words, the fully connected layer is used to combine and integrate the features. Softmax is used to normalize the output sum to become the probability distribution of the predicted category.

[0129] Specifically, the smart watch can input the above target RRI sequence and motion sequence into the three-layer convolution layer to obtain the first output value (or the first output feature). Afterwards, the smart watch can input the first output value into the LSTM layer to obtain the second output value (or the second output feature). Afterwards, the smart watch can input the second output value into the fully connected layer and softmax to obtain the four-classification result, that is, to obtain the sleep state of the wearer.

[0130] It can be understood that each convolution layer includes multiple convolution kernels, which slide on the input data (that is, the target RRI sequence and the motion sequence) according to a preset sliding step size (or called preset step size). Among them, the preset sliding step size refers to the length (or called time) corresponding to each sliding of the convolution kernel in the convolution operation, that is, the length corresponding to one sliding of the convolution kernel. Exemplarily, the preset sliding step size is pre-set according to actual needs. For example, the preset sliding step size can be 30s, 15s, etc., without specific limitation.

[0131] In some embodiments, the smart watch can continuously use the target RRI sequence and motion sequence within the first time period as input, run the above-mentioned sleep state detection model, and the sleep state detection model slides the above-mentioned convolution kernel with a preset step size to output the wearer's continuous sleep state within the first time period. Among them, the first time period is the time period corresponding to the smart watch needs to perform sleep state detection, that is, the time period when the wearer is already asleep or about to fall asleep. In one example, the first time period can be a time period pre-set according to the sleeping habits of most wearers. For example, the first time period can be from 20:00 to 10:00 am the next day. In another example, the first time period can also be a time period pre-set according to the sleeping habits of the wearer to whom the smart watch belongs. For example, the first time period can be from 23:00 to 8:00 am the next day.

[0132] In this embodiment, the sleep state of the wearer during the time period (that is, the preset convolution time described below) in which the convolution kernel is located is detected by sliding the convolution kernel on the input data. In this way, real-time detection of the sleep state can be achieved, so that the smart watch can intervene in the wearer in time according to the real-time detection results, thereby improving the wearer's sleep condition and further improving the wearer's sleep quality.

[0133] Among them, the size of the above-mentioned convolution kernel is determined based on the preset convolution time and the number of parameters corresponding to the input data (or called input value), that is, the width of the convolution kernel is the preset convolution time, and the height of the convolution kernel is the number of parameters of the input data. The preset convolution time is the time length of the input data that can be contained in a convolution kernel. It can be understood that since the parameters corresponding to the input data in this embodiment are the target RRI sequence and the motion sequence, the number of parameters is 2. If the input data also includes data of other parameters, then the number of parameters can also be the sum of the numbers corresponding to other parameters, and there is no specific limitation.

[0134] For example, Figure 8As shown, the time corresponding to each cell is 15s, that is, one cell corresponds to 15s of input data. Specifically, the convolution kernel A slides on the input data according to the preset sliding step size. It can be seen that the parameter corresponding to the cells in the first row is the target RRI sequence, and the parameter corresponding to the cells in the second row is the motion sequence. Therefore, it can be explained that the sliding direction of the convolution kernel is horizontal sliding, and the convolution kernel slides from left to right. Among them, the width of the convolution kernel A is the preset convolution time (60s), that is, each convolution kernel contains the target RRI sequence and motion sequence obtained by the smart watch for 60s. The height of the convolution kernel is the number of parameters of the input data (2). It can be understood that the input data includes the data corresponding to the target RRI sequence and the data corresponding to the motion sequence. Therefore, the number of parameters can be 2. In addition, the preset sliding step size of the convolution kernel is 30s, that is, the smart watch can output the sleep status of the wearer once every 30s.

[0135] In one implementation, the three layers of CNN included in the above-mentioned sleep state detection model can be named CNN layer one, CNN layer two, and CNN layer three, respectively. Among them, CNN layer one can include at least one of a first preset number of convolution kernels, maximum pooling, and activation functions. The first preset number is a number pre-set according to actual conditions. In this embodiment, the first preset number is 64. In other embodiments, the first preset number can also be other values, for example, it can be 32, etc., which are not specifically limited. Maximum pooling is one of the commonly used pooling operations in convolutional neural networks. Through the maximum pooling operation, the spatial dimension of the feature vector can be reduced, thereby reducing the number of parameters and computational complexity of the model and enhancing the robustness of the model. Exemplarily, the activation function can be a rectified linear unit (ReLU), a sigmoid function, a hyperbolic tangent function (tanh), etc., which are not specifically limited.

[0136] Among them, the above-mentioned CNN layer two includes at least one of a first preset number of convolution kernels, maximum pooling and activation functions. The above-mentioned CNN layer three includes at least one of a second preset number of convolution kernels, maximum pooling and activation functions. The second preset number is twice the above-mentioned first preset number, that is, the number of convolution kernels in the CNN layer three can be 128, 64, etc., without specific limitation. It can be understood that since the CNN layer three includes a second preset number of convolution kernels, the feature vector corresponding to the first output value input to the LSTM layer is the second preset number*1. In this embodiment, the feature vector is 128*1.

[0137] For example, Fig. 9 As shown, after obtaining the above-mentioned target RRI sequence and motion sequence, the smart watch can input the target RRI sequence and motion sequence as input data into CNN layer 1, CNN layer 2 and CNN layer 3 in sequence, and input the output result (or the above-mentioned first output feature) into the LSTM layer. Among them, the CNN layer 1 includes 64 convolution kernels, maximum pooling and activation function. CNN layer 2 includes 64 convolution kernels. CNN layer 3 includes 128 convolution kernels.

[0138] Among them, the feature vector size of the output feature of the above-mentioned LSTM layer is set to a third preset number, and the time length corresponding to the first output feature passing through the LSTM layer should not be less than the time length of the entire night's input data, that is, the sequence length passing through the LSTM layer is the maximum length after the CNN layer performs a convolution operation.

[0139] In one implementation, the length of time for inputting data overnight can be pre-set according to actual conditions. For example, under normal circumstances, the sleep duration of the wearer will not exceed 15 hours, so the smart watch can set the length of time for inputting data overnight to 15 hours. And, taking the above-mentioned input data corresponding to 15s for each cell as an example, it can be understood that 15 hours is equivalent to 54000s, so the maximum length is the length of 3600 cells.

[0140] In another implementation, the length of the night input data can also be pre-set according to the historical sleep status of the wearer of the smart watch. For example, under normal circumstances, the sleep duration of the wearer of the smart watch will not exceed 10 hours, so the smart watch can set the length of the night input data to 10 hours. And, taking the above-mentioned input data corresponding to 15s for each cell as an example, it can be understood that 10 hours is equivalent to 36000s, so the maximum length is the length of 2400 cells.

[0141] In some embodiments, the training process of the above-mentioned sleep state detection model may specifically include: the smart watch obtains an input data set. Among them, the input data set includes multiple input data and each input data carries a real state label, and the real state label is used to characterize the sleep state corresponding to the input data. For example, if the real state label is awake, therefore, the input data corresponding to the real state label is the data obtained by the smart watch when the wearer is in an awake state. The input data includes an RRI sequence and a motion sequence, and the sampling rate of the RRI sequence is the same as that of the motion sequence. Afterwards, for each input data, the smart watch can input the input data into the sleep state detection model to be trained to obtain a predicted state label. Afterwards, the smart watch can adjust the parameters of the sleep state detection model to be trained according to the predicted state label and the real state label carried by the input data to obtain a trained sleep state detection model.

[0142] For example, Fig.10 As shown, the sleep state detection process may specifically include: the smart watch can obtain PPG data through the PPG sensor, and perform peak detection on the PPG data to obtain an RRI sequence. Also, the smart watch can obtain ACC data through the ACC sensor, and perform feature extraction on the ACC data to obtain a motion sequence. Afterwards, the smart watch can interpolate and normalize the RRI sequence to obtain a target RRI sequence. Afterwards, the smart watch can vectorize the target RRI sequence and the motion sequence, and input the combined sequence table into three convolutional layers, LSTM layers, and softmax in turn to obtain the wearer's sleep state. Among them, the sleep state can be a rapid eye movement state, a deep sleep state, a light sleep state, and an awake state.

[0143] In one implementation, the input data may also include skin resistance data (or skin resistance signal). The skin resistance data is used to reflect the sweat gland secretion of the wearer. The skin resistance signal may be collected by a galvanic skin response (GSR) sensor. The GSR sensor is used to measure changes in skin conductivity to provide information about the wearer's physiological and emotional state. It is understood that if the wearer's mood fluctuates greatly (such as sudden tension), the more the wearer's sweat glands secrete, the wearer's body conductivity will be enhanced, thereby reducing the skin resistance value.

[0144] Specifically, the smart watch can collect skin resistance data through the above-mentioned GSR sensor. Afterwards, the smart watch can perform feature extraction on the skin resistance data to obtain a skin resistance sequence. Among them, the sampling rate of the skin resistance sequence is the same as the sampling rate of the above-mentioned motion sequence. Afterwards, the smart watch can input the skin resistance sequence, the above-mentioned target RRI sequence and the motion sequence into the above-mentioned sleep state detection model to obtain the sleep state of the wearer. In this way, the sleep state of the wearer can be detected from multiple angles, so that the smart watch can detect the sleep state of the wearer more accurately, thereby providing a basis for the subsequent timely sleep intervention of the wearer according to the sleep state, thereby improving the wearer's user experience.

[0145] For example, Fig.11 As shown, the sampling times corresponding to the above-mentioned skin resistance sequence, target RRI sequence and motion sequence are the same. Taking the sampling time of 1 second as an example, the first column of cells in the schematic diagram includes the data corresponding to the target RRI sequence, motion sequence and skin resistance sequence in the first second. The second column of cells includes the data corresponding to the target RRI sequence, motion sequence and skin resistance sequence in the second second. The third column of cells includes the data corresponding to the target RRI sequence, motion sequence and skin resistance sequence in the third second. The fourth column of cells includes the data corresponding to the target RRI sequence, motion sequence and skin resistance sequence in the fourth second. The fifth column of cells includes the data corresponding to the target RRI sequence, motion sequence and skin resistance sequence in the fifth second. The sixth column of cells includes the data corresponding to the target RRI sequence, motion sequence and skin resistance sequence in the sixth second. The seventh column of cells includes the data corresponding to the target RRI sequence, motion sequence and skin resistance sequence in the seventh second.

[0146] In some embodiments, after obtaining the sleep state of the wearer, the smart watch can perform sleep intervention on the wearer according to the sleep state. In this way, the wearer's sleep condition can be improved, and sleep improvement services matching the current sleep state can be provided, thereby improving the wearer's sleep quality.

[0147] In one example, when the wearer is in a deep sleep state, the smartwatch can intercept all notification messages. In this way, the wearer's sleep may be affected less by the reminder of notification messages, providing a good sleeping environment for the wearer and improving the wearer's experience.

[0148] In another example, when the wearer is in a rapid eye movement state, the smart watch can play sleep-aiding music and other sleep-aiding content to help the wearer quickly enter a deep sleep state and enhance the wearer's user experience.

[0149] In other embodiments, after obtaining the sleep state of the wearer, the smart watch can send the sleep state to a target electronic device (such as a mobile phone) in real time. The target electronic device is a device that establishes a communication connection with the smart watch. The communication connection includes a wired communication connection or a wireless communication connection. Fig.12 The communication system shown introduces the process of communication between the target electronic device 100 and the smart watch 200.

[0150] like Fig.12 As shown, the communication system may include a target electronic device 100 and a smart watch 200. Exemplarily, the target electronic device may be a mobile phone, a tablet computer, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc. The present application does not impose any restrictions on the specific type of the electronic device 100.

[0151] In a possible implementation, the target electronic device 100 can establish a wireless communication connection with the smart watch 200, and the target electronic device 100 and the smart watch 200 can transmit data information to each other through the wireless communication connection. For example, the smart watch 200 can send data information such as the wearer's sleep data to the target electronic device 100 through the wireless communication connection, and the target electronic device 100 can send data information such as the display screen state and / or the device state of the target electronic device 100 to the smart watch 200 through the wireless communication connection. Among them, the sleep data can refer to: the time occupied by each sleep state such as the light sleep state, the deep sleep state and the rapid eye movement state during the entire sleep period of the user, and the proportion of each in the sleep period. The data information sent between the target electronic device 100 and the smart watch 200 will be described in detail in subsequent embodiments, and will not be repeated here. Specifically, the wireless communication connection can be one or more wireless communication connections such as Bluetooth, wireless fidelity direct (wireless fidelity direct, WIFI direct) or wireless fidelity software access point (wireless fidelity software access point, WIFI softAP).

[0152] In another possible implementation, the target electronic device 100 can also establish a wired communication connection with the smart watch 200 to exchange data. For example, the target electronic device 100 can establish a wired connection with the smart watch 200 via a universal serial bus (USB), and transmit data information to each other based on the wired communication connection.

[0153] Afterwards, when the target electronic device receives the sleep state of the wearer, the target electronic device can perform sleep intervention on the wearer according to the sleep state. In other words, the process of performing sleep intervention on the wearer can be performed by the smart watch or by the target electronic device that establishes a communication connection with the smart watch, without specific limitation.

[0154] In some other embodiments, the smart watch can send the time occupied by each sleep state in the entire sleep period to the target electronic device when detecting that the wearer's sleep state switches from the first sleep state to the awake state. In this way, the target device can analyze the wearer's sleep quality according to the time occupied by each sleep state and provide a sleep analysis report for the wearer, so that the wearer can clearly understand his or her own sleep situation and improve the wearer's use experience.

[0155] In some embodiments, the present application provides a computer-readable storage medium, including computer instructions, which, when executed on an electronic device, enable the electronic device to execute the method described above.

[0156] In some embodiments, the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the method as described above.

[0157] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0158] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0159] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0160] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0161] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0162] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method for detecting a sleeping state, characterized in that: Applied to a wearable device, the method comprises: The wearable device acquires photoplethysmography (PPG) data and accelerometer (ACC) data; The wearable device processes the PPG data to obtain an RR interval RRI sequence, and processes the ACC data to obtain a motion sequence; The wearable device predicts the sleep state of the wearer of the wearable device based on the RRI sequence and the motion sequence; wherein the sleep state includes rapid eye movement state, deep sleep state, light sleep state or awake state.

2. The method according to claim 1, characterized in that The method further comprises: The wearable device acquires skin resistance data; The wearable device performs feature extraction on the skin resistance data to obtain a skin resistance sequence; The wearable device predicts a sleep state of a wearer of the wearable device based on the RRI sequence and the motion sequence, including: The wearable device predicts a sleep state of a wearer of the wearable device based on the skin resistance sequence, the RRI sequence, and the motion sequence.

3. The method according to claim 1 or 2, characterized in that: The wearable device predicts a sleep state of a wearer of the wearable device based on the RRI sequence and the motion sequence, including: The wearable device takes the RRI sequence and the motion sequence as input, runs a sleep state detection model, and outputs the sleep state of the wearer; Among them, the sleep state detection model has the ability to predict the sleep state based on the RRI sequence and the motion sequence.

4. The method according to claim 3, characterized in that The sleep state detection model includes a convolutional neural network (CNN) layer, the CNN layer is used to extract features from the RRI sequence and the running sequence, and the CNN layer includes a convolution kernel; The wearable device takes the RRI sequence and the motion sequence as input, runs a sleep state detection model, and outputs the sleep state of the wearer, including: The wearable device continuously takes the RRI sequence and the motion sequence within a first time period as input, runs the sleep state detection model, and the sleep state detection model slides the convolution kernel with a preset step size to output the wearer's continuous sleep state within the first time period.

5. The method according to claim 4, characterized in that The width of the convolution kernel is the preset convolution time, the preset convolution time is the time length of the input data included in a convolution kernel, the height of the convolution kernel is the number of parameters of the input data, and the input data includes the RRI sequence and the motion sequence.

6. The method according to any one of claims 1 to 5, characterized in that After the wearable device processes the PPG data to obtain an RR interval RRI sequence, the method further includes: The wearable device performs interpolation processing on the RRI sequence to obtain a target RRI sequence having the same sampling rate as the motion sequence; The wearable device predicts a sleep state of a wearer of the wearable device based on the RRI sequence and the motion sequence, including: The wearable device predicts the sleep state of the wearer of the wearable device based on the target RRI sequence and the motion sequence.

7. The method according to any one of claims 1 to 6, characterized in that Before the wearable device acquires the photoplethysmography (PPG) data and the accelerometer (ACC) data, the method further includes: When the first condition is met, the wearable device receives a data collection instruction from the electronic device; The wearable device acquires photoplethysmography (PPG) data and accelerometer (ACC) data, including: In response to the data collection instruction, the wearable device acquires the PPG data and the ACC data; The first condition includes at least one of the following: The current time is within the preset sleep period; and, The electronic device is in a screen-off state, and the screen-off duration reaches a preset duration.

8. The method according to any one of claims 1 to 7, characterized in that The wearable device acquires photoplethysmography (PPG) data and accelerometer (ACC) data, including: When the pressure value between the wearable device and the wearer's wrist is within a preset pressure range, the wearable device acquires the PPG data and the ACC data.

9. The method according to claim 8, characterized in that The method further comprises: When the current time is within the preset sleep period and the pressure value is not within the preset pressure range, the wearable device sends a prompt message, and the prompt message is used to prompt the wearer to adjust the wearing tightness of the wearable device.

10. The method according to any one of claims 1 to 9, characterized in that The wearable device acquires photoplethysmography (PPG) data and accelerometer (ACC) data, including: The wearable device acquires the ACC data; When the motion sequence obtained based on the ACC data indicates that the wearer is not in motion, the wearable device acquires the PPG data.

11. A wearable device, characterized in that: The wearable device includes a PPG sensor, an ACC sensor, a display screen, a memory, and one or more processors; the PPG sensor, the ACC sensor, the display screen, the memory, and the processor are coupled; The PPG sensor is used to collect PPG data, and the ACC sensor is used to collect ACC data; The display screen is used to display the image generated by the processor, and the memory is used to store computer program code, wherein the computer program code includes computer instructions; when the processor executes the computer instructions, the wearable device executes the method as described in any one of claims 1 to 10.

12. The wearable device according to claim 11, characterized in that: The wearable device further includes a skin electrical GSR sensor; the GSR sensor is coupled with the PPG sensor, the ACC sensor, the display screen, the memory, and the processor; The GSR sensor is used to collect skin resistance data.

13. A computer-readable storage medium, characterized in that: The invention comprises computer instructions, and when the computer instructions are executed on a wearable device, the wearable device is caused to execute the method according to any one of claims 1 to 10.

14. A computer program product, characterized in that The invention comprises computer instructions, and when the computer instructions are executed on a wearable device, the wearable device is caused to execute the method according to any one of claims 1 to 10.

Images