Sleep state detection method and wearable device

Abstract

The present application provides a sleep state detection method and a wearable device, relating to the technical field of wearable devices. The wearable device obtains PPG data and ACC data. The wearable device can then process the PPG data to obtain an R-R interval (RRI) sequence, and process the ACC data to obtain a motion sequence. The wearable device can then predict the sleep state of the wearer of the wearable device based on the RRI sequence and the motion sequence; wherein the sleep state of the wearer includes rapid eye movement (REM) state, deep sleep state, light sleep state, or awake state. In the present application, the accuracy of user sleep state detection can be improved, thereby enhancing the user's wearing experience.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of wearable devices, and in particular to a method for detecting a sleep state and a wearable device. Background Art

[0002] With the rapid development of terminal technology, various types of electronic devices, such as mobile phones, tablets, and wearable devices such as smart watches, have become indispensable products in people's lives. In daily use, users often use wearable devices to monitor their physiological conditions and understand their physical health.

[0003] In some cases, smartwatches can analyze a user's sleep state and thus determine their sleep quality based on detected physiological data (e.g., heart rate, acceleration during physical activity, etc.). Therefore, how to accurately analyze a user's sleep state to improve the user's wearing experience has become an urgent problem to be solved. Summary of the Invention

[0004] The embodiments of the present application provide a sleep state detection method and a wearable device for improving the accuracy of user sleep state detection, thereby enhancing the user's wearing experience.

[0005] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0006] In a first aspect, a sleep state detection method is provided for use with a wearable device. In this method, the wearable device acquires PPG data and ACC data. The wearable device then processes the PPG data to obtain a sequence of RR intervals (RRIs), and processes the ACC data to obtain a motion sequence. Based on the RRI and motion sequences, the wearable device can predict the sleep state of the wearer of the wearable device. The sleep state of the wearer can be rapid eye movement (REM), deep sleep, light sleep, or awake.

[0007] In this application, the wearable device can obtain four classification results including rapid eye movement state, deep sleep state, light sleep state and awake state through the RRI sequence and motion sequence. In this way, the wearer's sleep state can be comprehensively detected, the probability of false detection of sleep state can be reduced, and the detection accuracy of sleep state can be improved. This provides a basis for subsequent sleep intervention based on the wearer's sleep state, and improves the wearer's user experience.

[0008] In a possible implementation of the first aspect, the process of the wearable device acquiring PPG data and ACC data may specifically include: upon detecting a wearer performing a wearing operation on the wearable device, the wearable device collecting PPG data in real time through a PPG sensor, and collecting ACC data through an ACC sensor.

[0009] In this application, if a wearable device is worn on a designated part of the wearer's body, the wearable device can collect PPG data in real time via a PPG sensor and ACC data via an ACC sensor. This allows real-time monitoring of the wearer's sleep status, allowing the wearable device to provide timely sleep intervention based on the wearer's sleep status, thereby improving the wearer's sleep quality and helping the wearer fall asleep more easily. Furthermore, since PPG data can also be used to monitor the wearer's physiological conditions (such as blood oxygen levels and blood pressure), if the PPG sensor can collect PPG data in real time, the wearable device can use this data to monitor the wearer's physiological conditions in real time. This allows the user to monitor their physiological conditions in real time and, if the wearer's physiological conditions are abnormal, can promptly remind the user to rest, thereby enhancing the user's wearing experience. Furthermore, since ACC data can also be used to monitor the wearer's exercise status (such as step count), if the ACC sensor can collect ACC data in real time, the wearable device can use this data to monitor the wearer's exercise status in real time. This allows the user to understand their exercise status and adjust their exercise plan accordingly, ensuring healthy exercise.

[0010] In a possible implementation of the first aspect, the process of the wearable device acquiring PPG data and ACC data may specifically include: upon detecting a wearer performing a wearing operation on the wearable device, the wearable device may acquire PPG data within a first preset time period, and acquire ACC data within a second preset time period.

[0011] In this application, since PPG data and ACC data are used to detect the wearer's sleep state, and the wearer generally sleeps at a fixed time period, the wearable device can only collect PPG data and ACC data during the fixed time period, thereby improving the utilization rate of collection resources.

[0012] In one possible implementation of the first aspect, the process of the wearable device acquiring PPG data and ACC data may specifically include: the wearable device acquiring the ACC data. Thereafter, the wearable device may perform feature extraction on the ACC data to obtain a motion sequence, where the motion sequence is used to indicate whether the wearer of the wearable device is in motion. Thereafter, if the motion sequence indicates that the wearer is not in motion, the wearable device acquires PPG data.

[0013] In this application, the wearable device needs to obtain PPG data only when the motion sequence indicates that the wearer is not in motion, that is, the wearer may be in a sleeping state. In this way, while ensuring the accuracy of sleep state detection, the utilization rate of acquisition resources can be improved, thereby reducing the waste of storage resources.

[0014] In one possible implementation of the first aspect, the process of the wearable device acquiring PPG data and ACC data may specifically include: if a first condition is met, the wearable device receiving a data collection instruction from an electronic device. Thereafter, in response to the data collection instruction, the wearable device acquires the PPG data and ACC data.

[0015] Among them, the first condition includes at least one of the following: the current time is within the preset sleep period; and the electronic device is in the screen-off state, and the screen-off time reaches the preset time.

[0016] In this application, if the electronic device does not receive any touch operation from the wearer for a long time, it will automatically turn off the screen. In other words, the electronic device will switch from the screen-on state to the screen-off state. Alternatively, if the wearer presses the power button, the electronic device can switch from the screen-on state to the screen-off state. Therefore, the wearable device can obtain PPG data and ACC data when receiving the data collection instruction sent by the electronic device. In this way, the waste of subsequent computing resources caused by ineffective sensor collection can be reduced, and the utilization rate of computing resources can be improved.

[0017] Furthermore, considering that the wearer may not use their phone for extended periods during the day due to work, if the electronic device's screen is off for a preset period, it indicates that the wearer may have fallen asleep. Therefore, the wearable device can acquire PPG and ACC data upon receiving data collection instructions from the electronic device. This enables accurate detection of the smartwatch, reduces subsequent computing resource waste caused by ineffective sensor data collection, and improves computing resource utilization.

[0018] In one possible implementation of the first aspect, the process of the wearable device acquiring PPG data and ACC data may specifically include: the wearable device acquiring a pressure value between the wearable device and a wearer's wrist, where the pressure value is used to indicate the fit between the dial of the wearable device and the wearer's wrist. Thereafter, if the pressure value is within a preset pressure range, the wearable device may acquire the PPG data and ACC data.

[0019] In this application, when the pressure value is within the preset pressure range, it indicates that the wearable device is being worn appropriately. Therefore, the wearable device can directly obtain PPG data and ACC data. This allows for accurate data collection, reducing the likelihood of inaccurate data collection due to wearing the smartwatch too loosely or too tightly, thereby improving the accuracy of sleep state detection.

[0020] In a possible implementation of the first aspect, the method further includes: if the current time is within a preset sleep period and the pressure value is not within a preset pressure range, the wearable device may issue a prompt message, wherein the prompt message is used to prompt the wearer to adjust the tightness of the wearable device.

[0021] In the present application, when the pressure value is not within the preset pressure range, it indicates that the wearable device is worn too loosely or too tightly. Therefore, the wearable device can output a prompt message when the current time is within the preset sleep period and the pressure value is not within the preset pressure range. In this way, the wearer can be notified in time to adjust the tightness of the wearable device, thereby providing a basis for subsequent accurate data collection, thereby improving the detection accuracy of the sleep state.

[0022] In one possible implementation of the first aspect, the method further includes: the wearable device interpolating the RRI sequence to obtain a target RRI sequence having the same sampling rate as the motion sequence. The wearable device may then predict the wearer's sleep state based on the target RRI sequence and the motion sequence.

[0023] In this application, by interpolating the RRI sequence, the sampling rate of the processed RRI sequence (that is, the target RRI sequence) can be made the same as that of the motion sequence. In this way, time alignment between different sequences can be achieved, that is, different sequences at the same acquisition time can be aligned, reducing the occurrence of situations where the detection results of the sleep state are affected by differences in the time points between sequences (such as the target RRI sequence of the first second and the motion sequence of the second second as input at the same moment), thereby providing a basis for subsequent accurate detection of the sleep state.

[0024] In one possible implementation of the first aspect, the wearable device may predict the wearer's sleep state by inputting a target RRI sequence and a motion sequence into a sleep state detection model to obtain the wearer's sleep state. The sleep state detection model may be capable of predicting the sleep state based on the target RRI sequence and the motion sequence.

[0025] In this application, since the wearable device can output four-category results through only one state detection model, it can reduce the waste of computing resources caused by multi-model detection, improve the utilization rate of computing resources, and thus improve the detection efficiency of sleep state.

[0026] In a possible implementation of the first aspect, the sleep state detection model includes a CNN layer, which is used to extract features from the target RRI sequence and the motion sequence, and the CNN layer includes a convolution kernel. The process of the wearable device predicting the wearer's sleep state may specifically include: the wearable device may continuously use the RRI sequence and the motion sequence within the first time period as input, run the sleep state detection model, and the sleep state detection model slides the convolution kernel with a preset step size to output the wearer's continuous sleep state within the first time period.

[0027] In this application, the sleep state of the wearer during the time period in which the convolution kernel is located is detected by sliding the convolution kernel on the input data (that is, the RRI sequence and the motion sequence). In this way, real-time detection of the sleep state can be achieved, so that the smart watch can intervene in the wearer in time according to the real-time detection results, thereby improving the wearer's sleep condition and thus improving the wearer's sleep quality.

[0028] In a possible implementation of the first aspect, the width of the convolution kernel is a preset convolution time, which is the time length of the input data included in a convolution kernel, and the height of the convolution kernel is the number of parameters of the input data, which includes an RRI sequence and a motion sequence.

[0029] In this application, since the height of the convolution kernel is the number of parameters of the input data, that is, the convolution kernel includes all input parameters, the sleep state can be detected more comprehensively, so that the smart watch can detect the wearer's sleep state more accurately, thereby providing a basis for subsequent timely sleep intervention of the wearer according to the sleep state, thereby improving the wearer's user experience.

[0030] In one possible implementation of the first aspect, the method further includes: the wearable device acquiring skin resistance data. The wearable device may then perform feature extraction on the skin resistance data to obtain a skin resistance sequence. The sampling rate of the skin resistance sequence is the same as the sampling rate of the motion sequence. The wearable device may then predict the sleep state of the wearer of the wearable device based on the skin resistance sequence, the RRI sequence, and the motion sequence.

[0031] In this application, the wearable device can also predict the wearer's sleep state based on the skin resistance sequence. In this way, the wearer's sleep state can be detected from multiple angles, allowing the smartwatch to more accurately detect the wearer's sleep state, thereby providing a basis for subsequent sleep intervention based on the wearer's sleep state, thereby improving the wearer's user experience.

[0032] In a possible implementation manner of the first aspect, the method further includes: the wearable device performing sleep intervention on the wearer based on the sleep state.

[0033] In this application, after the wearer's sleep state is determined, the wearable device can perform sleep intervention on the wearer based on the sleep state. In this way, the wearer's sleep condition can be improved, and sleep improvement services that match the current sleep state can be provided, thereby improving the wearer's sleep quality.

[0034] In a second aspect, the present application provides a wearable device, comprising a PPG sensor, an ACC sensor, a display screen, a memory, and one or more processors; the PPG sensor, the ACC sensor, the display screen, the memory, and the processor are coupled; the PPG sensor is used to collect PPG data, and the ACC sensor is used to collect ACC data; the display screen is used to display an image generated by the processor, and the memory is used to store computer program code, wherein the computer program code comprises computer instructions; when the processor executes the computer instructions, the wearable device performs the method described above.

[0035] In a possible implementation of the second aspect, the wearable device further includes a GSR sensor; the GSR sensor is coupled to the PPG sensor, the ACC sensor, the display screen, the memory, and the processor; and the GSR sensor is used to collect skin resistance data.

[0036] In a third aspect, the present application provides a computer-readable storage medium comprising computer instructions, which, when executed on an electronic device, enables the electronic device to execute the method described above.

[0037] In a fourth aspect, the present application provides a computer program product, which, when executed on an electronic device, enables the electronic device to execute the method described above.

[0038] In a fifth aspect, a chip is provided, comprising: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected via an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the method as described above.

[0039] Among them, the beneficial effects that can be achieved by the electronic device described in the second aspect, the computer-readable storage medium described in the third aspect, the computer program product described in the fourth aspect, and the chip described in the fifth aspect provided above can refer to the beneficial effects in the first aspect and any possible design method thereof, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic diagram of wearing a wearable device provided in an embodiment of the present application;

[0041] Figure 2 A schematic diagram of detecting a wearer's sleep state based on a PPG signal provided in an embodiment of the present application;

[0042] Figure 3 A schematic diagram of detecting the sleep state of a wearer based on an ACC signal provided in an embodiment of the present application;

[0043] Figure 4 A schematic diagram of the hardware structure of a smartwatch provided in an embodiment of the present application;

[0044] Figure 5 A flowchart of a sleep state detection method provided in an embodiment of the present application;

[0045] Figure 6 A schematic diagram of a waveform of a photoplethysmography signal provided in an embodiment of the present application;

[0046] Figure 7 A schematic diagram of a target RRI sequence and a motion sequence when the sampling rates are the same is provided in an embodiment of the present application;

[0047] Figure 8 A schematic diagram of a convolution kernel sliding on input data provided in an embodiment of the present application;

[0048] Figure 9 A schematic diagram of a process for detecting the sleep state of a wearer provided in an embodiment of the present application;

[0049] Figure 10A schematic diagram of a sleep state detection process provided in an embodiment of the present application;

[0050] Figure 11 A schematic diagram of a target RRI sequence, a motion sequence, and a skin resistance sequence when the sampling rates are the same, provided in an embodiment of the present application;

[0051] Figure 12 A schematic diagram of the structure of a communication system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application. Wherein, in the description of the present application, unless otherwise specified, the "and / or" in the present application is merely a kind of association relationship describing the associated objects, indicating that there can be three kinds of relationships, for example, A and / or B, which can represent: A exists alone, A and B exist at the same time, and B exists alone, wherein A and B can be singular or plural. Moreover, in the description of the present application, unless otherwise specified, "multiple" refers to two or more than two. "At least one of the following" or its similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, wherein a, b, c can be single or multiple. In addition, in order to facilitate the clear description of the technical solutions in the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art will understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit differences. At the same time, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or design. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way for easy understanding.

[0053] In some application scenarios, wearable devices (such as Figure 1 The smartwatch 200 shown can be fixed to the wrist of the wearer (or user) via a watch strap. The wearable device can detect physiological parameters of the wearer and display the physiological parameters (such as heart rate) to inform the wearer of the current physiological condition.

[0054] In some embodiments, as Figure 2As shown, the wearable device can obtain a sequence of RR intervals (RRIs) using PPG signals (or PPG data) acquired by a photoplethysmographic (PPG) sensor. The RR interval refers to the time interval between two adjacent R waves on an electrocardiogram (ECG). In a QRS complex, the inflection point of the Q wave is the starting point of the R wave. The time from one starting point to the next is called the RR interval, which is the interval between each heartbeat. The QRS complex reflects changes in the potential and timing of left and right ventricular depolarization. Under normal circumstances, the RR interval should be between 0.6 and 1.0 seconds. If the RR interval is less than 0.6 seconds, the wearer is experiencing tachycardia; if the RR interval is less than or equal to 1.0 seconds, the wearer is experiencing bradycardia. Furthermore, unequal RR intervals indicate arrhythmia, and atrial fibrillation will show noticeable unequal RR intervals on an ECG.

[0055] Specifically, the wearable device can obtain the PPG signal collected by the PPG sensor. Then, the wearable device can perform peak detection on the PPG signal to obtain the RRI sequence.

[0056] When the above-mentioned RRI sequence is obtained, the wearable device can extract heart rate variability (HRV) information from the RRI sequence. The HRV information may include at least one of the mean, standard deviation, and root mean square of the difference between adjacent RR intervals of the RRI sequence per minute. Afterwards, the wearable device can detect the wearer's first sleep state based on the HRV information to obtain the wearer's sleep condition. The sleep condition is used to characterize the wearer's sleep quality. It can be understood that the longer the wearer is in a deep sleep state, the higher the wearer's sleep quality; the longer the wearer is in a rapid eye movement state, the lower the wearer's sleep quality.

[0057] In some embodiments, the wearable device can input the above-mentioned HRV information into a first state detection model to obtain the wearer's first sleep state. The first sleep state may include at least one of a rapid eye movement (REM) state, a deep sleep state, and a light sleep state. The rapid eye movement state refers to a sleep state in which the brain wave frequency becomes faster and the amplitude becomes lower during sleep, and also exhibits physical signs such as increased heart rate, increased blood pressure, muscle relaxation, and constant left and right swaying of the eyeballs. The deep sleep state is also called a deep sleep state, which refers to a sleep state in which the cerebral cortex cells are in a fully rested state. The light sleep state is also called a light sleep state, which refers to a sleep state in which the body's activity frequency is higher and the sleep sensitivity is higher. That is to say, users in a light sleep state are prone to turning over and dreaming, and are more likely to be awakened than users in a deep sleep state.

[0058] The first state detection model is a model capable of detecting the user's first sleep state. Exemplarily, the first state detection model can be a decision tree model, a support vector machine (SVM) model, or a temporal convolutional network (TCN). Any classification model can serve as the first state detection model, without specific limitation.

[0059] It can be seen that the wearer's first sleep state detected by the above-mentioned wearable device through the PPG signal only includes the rapid eye movement state, deep sleep state and light sleep state, and does not include the awake state. In other words, this sleep state detection method cannot fully detect the wearer's sleep state, which makes the sleep state detection method limited. As a result, the incomplete sleep state detection may lead to false sleep state detection (such as misdetecting a sleep state that is not in the first sleep state as the first sleep state, that is, misdetecting the awake state as the rapid eye movement state), resulting in low accuracy of sleep state detection, which ultimately affects the user's wearing experience.

[0060] In other embodiments, Figure 3 As shown, a wearable device can obtain a motion sequence using an accelerometer (ACC) sensor's ACC signal (or ACC data). This motion sequence is used to characterize whether the wearer is in motion. Specifically, the wearable device can obtain the ACC signal collected by the ACC sensor. The wearable device can then perform feature extraction on the ACC signal to obtain the aforementioned motion sequence.

[0061] The wearable device can then detect the wearer's second sleep state based on the aforementioned motion sequence and determine the wearer's sleep status. The second sleep state can include a sleeping state and / or a wakeful state. The sleeping state refers to the state a person exhibits while sleeping. The wakeful state refers to a state in which the cerebral cortex and the entire body are awake and conscious. The sleep status indicates whether the wearer is asleep.

[0062] In one implementation, the wearable device can input the aforementioned motion sequence into a state machine to obtain the wearer's second sleep state. The state machine can transition to a pre-set sleep state (or second sleep state) based on the motion sequence. In other words, the wearable device can use the state machine to determine whether the wearer is in a sleep state based on the motion sequence.

[0063] In another implementation, the wearable device can input the aforementioned motion sequence into a second state detection model to obtain the wearer's second sleep state. The second state detection model is a model capable of detecting the user's second sleep state. For example, the second state detection model can be the same as or different from the first state detection model, without limitation.

[0064] Exemplarily, the training process of the second state detection model may include: the wearable device acquiring a motion sequence set, where the motion sequence set includes multiple motion sequences, each motion sequence carrying a label for the second sleep state. The wearable device may then train a pre-built second state detection model based on the motion sequence set to obtain a trained second state detection model, thereby classifying the wearer's second sleep state.

[0065] It can be seen that the wearer's second sleep state detected by the above wearable device through the ACC signal only includes the sleep state and the awake state, and cannot distinguish between sleep states. In other words, this sleep state detection method cannot detect the wearer's sleep situation, further analyze the wearer's sleep quality, and then intervene in the wearer's sleep, resulting in a poor user experience.

[0066] Therefore, in order to accurately detect the wearer's sleep state, an embodiment of the present application provides a sleep state detection method, in which a wearable device obtains an RRI sequence and a motion sequence. The RRI sequence is obtained based on the PPG signal, and the motion sequence is obtained based on the ACC signal. Afterwards, the wearable device can input the RRI sequence and motion sequence into a state detection model to obtain the user's sleep state. The sleep state includes rapid eye movement state, deep sleep state, light sleep state or awake state.

[0067] In an embodiment of the present application, the wearable device simultaneously inputs the RRI sequence and the motion sequence into the state detection model, and can obtain four classification results including rapid eye movement state, deep sleep state, light sleep state and awake state. In this way, not only can the wearer's sleep state be comprehensively detected, the probability of false detection of the sleep state can be reduced, the detection accuracy of the sleep state can be improved, and the user's wearing experience can be enhanced. In addition, the wearable device can output four classification results through only one state detection model, which can reduce the waste of computing resources caused by multi-model detection, improve the utilization rate of computing resources, and thus improve the detection efficiency of the sleep state.

[0068] For example, the wearable device is a smartwatch, smart bracelet, or other device that is in contact with the wearer and can collect the wearer's physiological parameters. Figure 4 , taking the wearable device as a smart watch as an example, the hardware structure of the wearable device is described.

[0069] like Figure 4 As shown, the smart watch 200 includes: a watch body and a wristband (or watch strap) connected to each other, wherein the watch body may include a front shell ( Figure 4 Not shown), touch screen 210 (also called touch panel), display screen 220, bottom case ( Figure 4 (not shown), as well as a processor 230, a memory 250, a microphone (MIC) 260, a communication module 270, a PPG sensor 281, an ACC sensor 282, and an ambient light sensor 283. Although not shown, the smart watch may further include a power supply, a power management system, an antenna, a speaker, a gyroscope sensor, etc. It will be understood by those skilled in the art that Figure 4 The smart watch structure shown in the figure does not constitute a limitation to the smart watch, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0070] The following describes the functional components of the smartwatch 200:

[0071] The touch panel 210, also known as the touchpad, can collect touch operations performed by the watch user (such as operations performed by the user using a finger, stylus, or any other suitable object or accessory on or near the touch panel) and drive the corresponding connected device according to a pre-set program.

[0072] The display screen 220 can be used to display information input by the user or information provided to the user, as well as various menus of the watch. Optionally, the display screen 220 can be configured in the form of an LCD, an OLED, etc. Further, the touch panel 210 can cover the display screen 220. When the touch panel 210 detects a touch operation on or near it, it transmits the information to the processor 230 to determine the type of touch event. Subsequently, the processor 230 provides corresponding visual output on the display screen 220 according to the type of touch event. Although in Figure 4 In the embodiment, the touch panel 210 and the display screen 220 are used as two independent components to realize the input and output functions of the watch, but in some embodiments, the touch panel 210 and the display screen 220 can be integrated to realize the input and output functions of the watch.

[0073] The processor 230 is used to perform system scheduling, control the display screen and touch screen, and process data sent by sensors (such as the PPG sensor 281, ACC sensor 282, and ambient light sensor 283). The processor 230 can also be called a main control unit, which can include a computing unit that can process data.

[0074] The memory 250 is used to store software programs and data. The processor 230 executes the various functional applications and data processing of the watch by running the software programs and data stored in the memory. The memory 250 mainly includes a program storage area and a data storage area. The program storage area can store the operating system and at least one application required for a function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created by using the watch (such as audio data, a phone book, etc.). In addition, the memory can include high-speed random access memory and non-volatile memory, such as a disk storage device, a flash memory device, or other volatile solid-state storage device.

[0075] Communication module 270, the smart watch can exchange information with other electronic devices (such as mobile phones, tablets, etc.) through the communication module 270. Exemplarily, the communication module 270 may include a wireless communication module and a mobile communication module. Optionally, the wireless communication module may include a Bluetooth (BT) module, a global navigation satellite system (GNSS), and a wireless local area network (WLAN) (such as a wireless fidelity (Wi-Fi) network). The mobile communication module can provide solutions for wireless communications such as 2G / 3G / 4G / 5G applied to the smart watch.

[0076] The PPG sensor 281 can be used to measure the user's heart rate, and / or physiological data of the human body such as blood oxygen saturation. Taking the detection of the user's heart rate as an example, in a specific implementation method, the PPG sensor 281 can emit a light signal of a specified wavelength, which can illuminate the artery under the skin tissue and be reflected back to the PPG sensor 281. When the heart beats, the contraction and expansion of the blood vessels will change the blood volume in the artery, thereby affecting the absorption or attenuation of the light signal by the artery, thereby affecting the reflection of the light signal. The PPG sensor 281 can detect the user's heart rate based on the changes in the reflected light signal. It should be noted that the PPG sensor 281 can also detect the user's heart rate in other ways, and this application does not constitute a limitation to this.

[0077] The ACC sensor 282 can be used to detect the magnitude of the acceleration of the smart watch 200 in various directions (generally three axes, namely the x-axis, y-axis, and z-axis). In some embodiments, the ACC sensor 282 can detect the number of steps, cadence, and stride length of the user in different exercise modes such as walking, running, or cycling, providing the user with real-time exercise data. In other embodiments, the ACC sensor 282 can also detect the user's sleep status, that is, whether the user is asleep or awake.

[0078] The ambient light sensor 283 is used to detect the lighting conditions of the environment in which the smart watch is located.

[0079] It should be understood that the illustrated smart watch 200 is merely one example of a wearable device, and that the smart watch 200 may have more or fewer components than shown, may combine two or more components, or may have a different configuration of components. Figure 4 The various components shown in the drawings may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0080] Based on the electronic device described above, an embodiment of the present application provides a method for detecting a sleep state. The method can be applied to health detection (such as sleep detection) scenarios of wearable devices. That is, the wearable device in the method is an electronic device with a health detection function. The following will take the wearable device as a smart watch as an example to illustrate the method of the embodiment of the present application. Specifically, Figure 5 As shown, the sleep state detection method may include S601 to S604.

[0081] S601: The smart watch collects PPG data through the PPG sensor and collects ACC data through the ACC sensor.

[0082] Specifically, when the wearer is detected to be wearing the smartwatch, the smartwatch can collect PPG data through the PPG sensor and collect ACC data through the ACC sensor. It can be understood that only when the smartwatch is worn on a designated part of the wearer's body (such as the wrist) can the smartwatch detect the wearer's sleep state and then determine whether the wearer's current sleep quality is good, that is, whether sleep intervention is needed for the wearer. If the smartwatch is not worn on the designated part of the wearer's body, the smartwatch does not need to collect data to avoid unnecessary power consumption.

[0083] Generally, the above-mentioned smart watch can transmit a light signal into the wearer's skin tissue (or can be understood as blood, blood vessels, etc. in the skin tissue) through a PPG sensor (such as a light-emitting diode (LED) in a PPG sensor). The PPG sensor (such as a photodiode (PD) in a PPG sensor) can receive the light signal reflected back through the skin tissue. The light signal includes the pulsation information of the wearer's blood vessels, and the pulsation information is used as PPG data. It should be understood that after the PD receives the reflected light signal, it can convert the light signal into an electrical signal, and through analog-to-digital conversion, convert the electrical signal into a digital signal that can be used by the smart watch. The above-mentioned PPG data may actually refer to the digital signal.

[0084] In some embodiments, if a smartwatch is worn on a designated area of ​​the wearer's body, the smartwatch can collect PPG data in real time via a PPG sensor and ACC data via an ACC sensor. This allows for real-time monitoring of the wearer's sleep status, enabling the smartwatch to implement timely sleep interventions based on this information to improve sleep quality and help the wearer fall asleep more easily. Furthermore, since PPG data can also be used to monitor the wearer's physiological status (such as blood oxygen levels and blood pressure), if the PPG sensor can collect PPG data in real time, the smartwatch can use this data to monitor the wearer's physiological status in real time. This allows the user to monitor their physiological status in real time and, if the wearer's physiological status is abnormal, can promptly remind the user to rest, thereby enhancing the user's wearing experience. Furthermore, since ACC data can also be used to monitor the wearer's exercise status (such as step count), if the ACC sensor can collect ACC data in real time, the smartwatch can use this data to monitor the wearer's exercise status in real time. This allows the user to understand their exercise status and adjust their exercise plan accordingly, ensuring a healthy and active lifestyle.

[0085] In other embodiments, to reduce unnecessary resource waste, the smartwatch may collect PPG data via the PPG sensor during a first preset time period, and collect ACC data via the ACC sensor during a second preset time period. The first preset time period and the second preset time period may be the same or different, and are not specifically limited. It is understood that since the PPG data and ACC data in the embodiments of the present application are used to detect the wearer's sleep state, and the wearer generally sleeps during fixed time periods, the smartwatch may only collect PPG data and ACC data during these fixed time periods, thereby improving the utilization of collection resources.

[0086] Generally, the ACC data is used to detect whether the wearer is asleep. In other words, the smartwatch only needs to determine whether the wearer is asleep based on the ACC data. The PPG data is used to detect which sleep stage the wearer is in (i.e., the first sleep state). After the wearer has fallen asleep, the smartwatch can use the PPG data to detect the wearer's sleep stage. The first sleep state includes at least one of rapid eye movement (REM), deep sleep, and light sleep. Therefore, the smartwatch can collect PPG data using the PPG sensor when the ACC sensor collects ACC data and the motion sequence derived from the ACC data indicates that the wearer is not moving, indicating that the wearer may be asleep. In other words, the smartwatch can skip collecting PPG data and only collect ACC data. Furthermore, if the wearer is asleep, the smartwatch can skip collecting ACC data and only collect PPG data. This ensures the accuracy of sleep state detection while improving the utilization of acquisition resources and reducing storage waste.

[0087] In one implementation, the first and second preset time periods can be pre-set based on actual circumstances. In one example, assuming the first and second preset time periods are the same, if most wearers sleep between 8 PM and 8 AM the following day, the smartwatch can set the time period between 8 PM and 8 AM the following day as the first and second preset time periods. In another example, assuming the first and second preset time periods are different, if most wearers sleep between 8 PM and 11 PM, and the ACC data corresponding to the second preset time period is only used to detect whether the wearer is asleep, the smartwatch can set the time period between 8 PM and 11 PM as the second preset time period. If most wearers sleep between 10 PM and 8 AM the following day, and the PPG data corresponding to the first preset time period is only used to detect which sleep stage the wearer is in, the smartwatch can set the time period between 10 PM and 8 AM the following day as the first preset time period.

[0088] In another implementation, the first and second preset time periods can be set based on the smartwatch wearer's sleep history. In one example, if the first and second preset time periods are the same, and the smartwatch wearer typically sleeps between 11 PM and 7 AM the following morning, the smartwatch may set the time period between 11 PM and 7 AM as the first and second preset time periods. In another example, if the first and second preset time periods are different, and the smartwatch wearer typically sleeps between 11 PM and 1 AM the following morning, and the ACC data corresponding to the second preset time period is only used to detect whether the wearer is asleep, the smartwatch may set the time period between 11 PM and 1 AM as the second preset time period. If the smartwatch wearer typically sleeps between 12 PM and 8 AM the following morning, and the PPG data corresponding to the first preset time period is only used to detect which sleep stage the wearer is in, the smartwatch may set the time period between 12 PM and 8 AM as the first preset time period.

[0089] It is understood that when the wearer is asleep or about to fall asleep, the electronic device (such as a mobile phone) will not trigger corresponding operations. In other words, the electronic device will not receive any touch operations (such as clicks) from the wearer, and the wearer's electronic device will establish a communication connection with the smartwatch. However, if the electronic device does not receive any touch operations from the wearer for a long time, it will automatically turn off the screen. In other words, the electronic device will switch from the on screen state to the off screen state. Alternatively, if the wearer presses the power button, the electronic device can switch from the on screen state to the off screen state. Therefore, when the screen is off and the screen has been off for a predetermined period of time, the electronic device can send a data collection instruction to the smartwatch. The data collection instruction indicates that the wearer is resting and instructs the smartwatch to trigger the PPG sensor to collect PPG data and the ACC sensor to collect ACC data. Subsequently, when the smartwatch receives the data collection instruction, it can collect PPG data through the PPG sensor and ACC data through the ACC sensor in response to the data collection instruction. This can reduce the waste of subsequent computing resources caused by ineffective sensor data collection and improve computing resource utilization.

[0090] Furthermore, considering that the wearer may not use the phone for a long time due to daytime work, the electronic device can first determine whether the current time (or the first time) is within the preset sleep period (e.g., 22:00 to 9:00 am the next day). Only when the current time is within the preset sleep period, if the electronic device is in the off state and the screen-off duration reaches the preset duration, the electronic device can send the above data collection instruction to the smartwatch. In this way, accurate detection of the smartwatch can be achieved, reducing the waste of subsequent computing resources caused by invalid sensor data collection, and improving the utilization of computing resources.

[0091] Among them, the above-mentioned preset sleep period can be pre-set according to the actual situation (such as the general sleep conditions of most wearers), or can be set according to the historical sleep conditions of the wearer of the smart watch, without specific limitation.

[0092] In some embodiments, it is considered that the tightness with which the wearer wears the smartwatch will affect the accuracy of the data collected by the sensor. That is to say, if the smartwatch is worn too loosely or too tightly, the PPG data and ACC data will be inaccurately collected, thereby affecting the accuracy of subsequent sleep state detection. Therefore, the smartwatch can collect the pressure value between the smartwatch and the wearer's wrist through a pressure sensor, wherein the pressure value is used to characterize the fit between the smartwatch dial and the wearer's wrist. Afterwards, the ACC data and PPG data will be collected only when the pressure value is within a preset pressure range. In this way, accurate data collection can be achieved, reducing the occurrence of inaccurate data collection due to the smartwatch being worn too loosely or too tightly, thereby improving the accuracy of sleep state detection.

[0093] It can be understood that when the above pressure value is within the preset pressure range, it indicates that the smart watch is worn appropriately. Therefore, the smart watch can directly collect PPG data through the PPG sensor and collect ACC data through the ACC sensor, that is, there is no need to output prompt information about the wearing tightness. When the pressure value is not within the preset pressure range, it indicates that the smart watch is worn too loose or too tight. Therefore, in order to improve the accuracy of data collection and further improve the accuracy of sleep state detection, the smart watch can output a prompt message when the current time is within the preset sleep period and the pressure value is not within the preset pressure range. Among them, the prompt message is used to prompt the wearer to adjust the wearing tightness of the smart watch.

[0094] The preset pressure range is pre-set based on actual conditions. Specifically, when the pressure value is greater than a first preset pressure and less than a second preset pressure, the smartwatch can determine that the pressure value is within the preset pressure range. The first preset pressure is less than the second preset pressure. In other words, the preset pressure range can be determined based on the first and second preset pressures.

[0095] In one example, when the pressure value is less than or equal to the first preset pressure, it indicates that the smartwatch is worn too loosely. Therefore, to improve the accuracy of data collection, the smartwatch can output a first prompt message to remind the user that the current wearing degree is too loose. Exemplarily, the first prompt message output by the smartwatch can be a voice output of the first prompt message (such as the smartwatch voice output of the prompt message "The wearing degree is too loose, please wear it tighter"), and / or the smartwatch displays the first prompt message (such as the smartwatch displays the prompt message "The wearing degree is too loose, please wear it tighter"). In this way, the occurrence of data collection being affected by wearing the smartwatch too loose can be reduced, the accuracy of data collection can be improved, and the detection accuracy of sleep state can be improved.

[0096] In one example, when the pressure value is greater than or equal to the second preset pressure, it indicates that the smartwatch is worn too tightly. Therefore, to improve the accuracy of data collection, the smartwatch can output a second prompt message to remind the user that the current wearing degree is too tight. Exemplarily, the second prompt message output by the smartwatch can be a voice output of the second prompt message (such as the smartwatch voice output of the second prompt message "wearing degree is too tight, please wear it looser"), and / or the smartwatch displays the second prompt message (such as the smartwatch displays the prompt message "wearing degree is too tight, please wear it looser"). In this way, the occurrence of data collection being affected by wearing the smartwatch too tightly can be reduced, the accuracy of data collection can be improved, and the accuracy of sleep state detection can be improved.

[0097] It should be noted that the personal information used in the technical solution of this application is limited to information for which the individual’s separate consent has been obtained, including but not limited to notifying and reminding the user to read the relevant user agreement (notification) and sign the agreement (authorization) including authorization of relevant user information before the user uses the function.

[0098] Specifically, since the aforementioned PPG data and ACC sensor can be used to detect the wearer's current sleep state, the smartwatch can timely intervene in the wearer's sleep based on the sleep state to improve the wearer's sleep quality. Therefore, in order to protect the user's privacy, it is necessary to determine whether the wearer has turned on the sleep detection function before the PPG sensor collects PPG data and the ACC sensor collects ACC data. Only when the sleep detection function is turned on, that is, when the smartwatch receives the wearer's operation to turn on the sleep detection function, can the smartwatch collect PPG data through the PPG sensor and ACC data through the ACC sensor to determine the wearer's sleep state.

[0099] It should be noted that there is no restriction on the order in which a smartwatch collects PPG data and ACC data. For example, a smartwatch can simultaneously collect PPG data through the PPG sensor and ACC data through the ACC sensor. For another example, a smartwatch can first collect PPG data through the PPG sensor and then collect ACC data through the ACC sensor. For another example, a smartwatch can first collect ACC data through the ACC sensor and then collect PPG data through the PPG sensor.

[0100] S602: The smartwatch processes the PPG data and the ACC data respectively to obtain a target detection sequence, wherein the target detection sequence includes an RRI sequence and a motion sequence.

[0101] Specifically, after collecting the PPG data and ACC data, the smartwatch can process the PPG data and ACC data to obtain a target detection sequence. The target detection sequence is used to detect the wearer's current sleep state. The target detection sequence can include an RRI sequence and a motion sequence. The RRI sequence is used to represent the interval between each heartbeat, that is, the interval between each heartbeat. The motion sequence is used to indicate whether the wearer is in a state of exercise.

[0102] In one implementation, the smartwatch can perform peak detection on the PPG data to obtain an RRI sequence. In one example, the smartwatch can perform peak detection on the PPG data by finding local maxima (also known as peaks) and minima (also known as troughs) in the PPG data. In another example, the smartwatch can also perform peak detection on the PPG data using a first-order derivative (such as the Li derivative) with an adaptive threshold.

[0103] In some embodiments, the smartwatch can perform peak detection on the PPG data to obtain the peak point of at least one waveform in the PPG data. The smartwatch can then calculate the distance between the peak points of adjacent waveforms and determine this distance as an RR interval. The smartwatch can then determine the RRI sequence based on the RR intervals corresponding to the peak points of all waveforms in the PPG data. In other words, the RRI sequence is derived based on the RR intervals corresponding to the peak points of all waveforms in the PPG data.

[0104] It should be noted that the waveform of the PPG signal can reflect the physiological characteristics of the wearer. For example, Figure 6 As shown in the figure, a PPG signal includes four waveforms, each representing a complete pulse waveform cycle. Point A1 in waveform 1, point A2 in waveform 2, point A3 in waveform 3, and point A4 in waveform 4 are all peak points, i.e., R waves. In some embodiments, the wearable device can determine the RR interval, i.e., RR1, by calculating the distance between points A1 and A2, RR2 by calculating the distance between points A2 and A3, and RR3 by calculating the distance between points A3 and A4.

[0105] In another implementation, the smartwatch may perform feature extraction on the ACC data to obtain a motion sequence.

[0106] In some embodiments, the smartwatch can calculate the ACC data at preset calculation intervals to obtain a motion value. It is understood that the motion sequence can include at least one motion value, that is, the motion sequence is composed of at least one motion value. The smartwatch can then determine whether the wearer is in a motion state based on the motion value. In one example, the motion value can be obtained by calculating energy. In another example, the motion value can also be obtained by calculating variance.

[0107] It should be noted that the preset calculation interval can be set according to actual needs. In this embodiment, to facilitate subsequent sleep state detection, the preset calculation interval can be set to 1 second, that is, the smartwatch calculates the exercise value every 1 second. In other embodiments, the preset calculation interval can also be set to 5 seconds, 10 seconds, etc., without specific limitation.

[0108] In one implementation, the smartwatch can determine whether the exercise value is greater than a preset value. If the exercise value is greater than the preset value, the smartwatch can determine that the wearer is in exercise; or if the exercise value is less than or equal to the preset value, the smartwatch can determine that the wearer is stationary, that is, the wearer is not in exercise.

[0109] For example, taking the above-mentioned motion value obtained by energy-based calculation as an example, the motion value can be calculated using the following formula 1:

[0110]

[0111] Wherein, E() is energy, that is, motion value; i is the preset interval (e.g., 1s); t is the sampling rate of ACC data (e.g., 100Hz); x is the acceleration value of the smartwatch on the x-axis; y is the acceleration value of the smartwatch on the y-axis; and z is the acceleration value of the smartwatch on the z-axis.

[0112] Specifically, after the smartwatch obtains the exercise value using the above formula 1, the smartwatch can determine whether the exercise value is greater than a preset value (i.e., a preset energy value). If the exercise value is greater than the preset energy value, the smartwatch can determine that the wearer is in an exercise state. If the exercise value is less than or equal to the preset energy value, the smartwatch can determine that the wearer is not in an exercise state.

[0113] For example, if the exercise value is calculated based on variance, the smartwatch can determine whether the exercise value is greater than a preset value (i.e., a preset variance value). If the exercise value is greater than the preset variance value, the smartwatch can determine that the wearer is exercising. If the exercise value is less than or equal to the preset variance value, the smartwatch can determine that the wearer is not exercising.

[0114] S603: The smartwatch performs interpolation and normalization on the RRI sequence to obtain a target RRI sequence.

[0115] Specifically, after obtaining the above RRI sequence, the smart watch can perform interpolation processing on the RRI sequence to obtain an interpolated RRI sequence.

[0116] It should be noted that, since the above-mentioned RRI sequence is used to characterize the interval time of each heartbeat, and the interval time of the heartbeat of the same wearer at different sampling times is also different. The above-mentioned motion sequence is calculated according to the above-mentioned ACC data according to the preset calculation interval, that is, the interval time corresponding to the motion sequence is the same, that is, the sampling rate of the motion sequence is fixed. Therefore, in order to ensure the accuracy of subsequent sleep state detection, the smart watch can interpolate the RRI sequence to obtain the interpolated RRI sequence. In this way, it can ensure that the sampling rate of the target RRI sequence is consistent with the sampling rate of the motion sequence, thereby providing a basis for the subsequent accurate detection of the sleep state.

[0117] Exemplarily, the smartwatch may interpolate the RRI sequence using polynomial interpolation, spline function interpolation, or other methods, without limitation. Polynomial interpolation utilizes the known values ​​of the function f(x) at several points within a certain interval to create an appropriate specific function (or polynomial), and uses the values ​​of this specific function as an approximation of the function f(x) at other points within the interval. Spline function interpolation simulates the original data using a spline function. A spline function is a type of function that is piecewise smooth and exhibits a certain degree of smoothness at the intersections of its segments.

[0118] In one implementation, after obtaining the above-mentioned interpolated RRI sequence, the smart watch can normalize the interpolated RRI sequence to obtain a target RRI sequence. The sampling rate of the target RRI sequence is the same as the sampling rate of the above-mentioned motion sequence. It can be understood that by normalizing the interpolated RRI sequence, the interpolation result can be improved, that is, the interpolated RRI sequence can be improved, so that the target RRI sequence can be spliced ​​with the above-mentioned motion sequence into a time series of the same length (such as 1s), that is, the sampling time corresponding to the target RRI sequence and the motion sequence is the same, which is the above-mentioned preset calculation interval. In this way, it can provide a basis for subsequently improving the accuracy of sleep state detection.

[0119] For example, Figure 7 As shown, after the above RRI sequence is interpolated and normalized, the target RRI sequence obtained has the same sampling time as the motion sequence. Taking the sampling time as 1 second as an example, the first column of cells in the schematic diagram includes the data corresponding to the target RRI sequence and the motion sequence in the first second. The second column of cells includes the data corresponding to the target RRI sequence and the motion sequence in the second second. The third column of cells includes the data corresponding to the target RRI sequence and the motion sequence in the third second. The fourth column of cells includes the data corresponding to the target RRI sequence and the motion sequence in the fourth second. The fifth column of cells includes the data corresponding to the target RRI sequence and the motion sequence in the fifth second. The sixth column of cells includes the data corresponding to the target RRI sequence and the motion sequence in the sixth second. The seventh column of cells includes the data corresponding to the target RRI sequence and the motion sequence in the seventh second.

[0120] In some embodiments, the smartwatch may not perform step S603. That is, the smartwatch may directly detect the wearer's sleep state based on the RRI sequence and motion sequence, without interpolating or normalizing the RRI sequence. This can improve the efficiency of sleep state detection.

[0121] In other embodiments, the smartwatch may only interpolate the RRI sequence without normalizing it. The smartwatch can then directly detect the wearer's sleep state based on the interpolated RRI sequence and the motion sequence. This can improve sleep state detection efficiency.

[0122] S604: The smartwatch inputs the target RRI sequence and motion sequence into a sleep state detection model to obtain the sleep state of the wearer.

[0123] Specifically, after obtaining the target RRI sequence, the smartwatch can predict the wearer's sleep state based on the target RRI sequence and the motion sequence. The sleep state is used to represent the wearer's current sleep state. The sleep state can include rapid eye movement state, deep sleep state, light sleep state, or awake state.

[0124] In one implementation, the smartwatch may input the target RRI sequence and motion sequence into a sleep state detection model to determine the wearer's sleep state. The sleep state detection model has the ability to predict the sleep state based on the target RRI sequence and motion sequence.

[0125] In some embodiments, after obtaining the above-mentioned target RRI sequence, the smart watch can first perform vector combination of the target RRI sequence and the motion sequence to obtain a combined sequence table. Afterwards, the smart watch can input the combined sequence table into the sleep state detection model to obtain the sleep state of the wearer. It can be understood that by interpolating and normalizing the above-mentioned RRI sequence, the sampling rate of the target RRI sequence and the motion sequence can be made the same. Therefore, by performing vector combination of the target RRI sequence and the motion sequence, time alignment between different sequences can be achieved, that is, different sequences at the same acquisition time are aligned, reducing the occurrence of situations where the detection results of the sleep state are affected by differences in the time points between sequences (such as the target RRI sequence of the first second and the motion sequence of the second second as input at the same time), providing a basis for subsequent accurate detection of the sleep state.

[0126] Among them, the above-mentioned sleep state detection model is a classification model. In this embodiment, the classification model can be a convolutional neural network (CNN). Specifically, the classification process of the convolutional neural network is to obtain the score of the current sample belonging to each class through the fully connected layer after the convolution layer and the pooling layer, and then classify it according to the scores of different classes through the normalized exponential function (softmax) to obtain the class to which the current sample belongs. Among them, softmax is an activation function for multi-classification problems. In other embodiments, the classification model can also be a decision tree, support vector machine (SVM), etc., as long as it is a classification model, it can be used as a sleep state detection model, without specific limitation.

[0127] In one implementation, the sleep state detection model may include at least one of N convolutional neural network layers, one long short-term memory (LSTM) layer, one fully connected layer, and a softmax layer. In this embodiment, N is 3. In other embodiments, N can also be other values, for example, 2, 5, etc., without limitation.

[0128] The CNN layer (also known as the convolutional layer) is used to extract features from the target RRI and motion sequences. The LSTM layer is used to process time series data, specifically the target RRI and motion sequences. The fully connected layer connects the features extracted by the convolutional layer and outputs the final classification result. In other words, this fully connected layer is used to combine and integrate features. Softmax is used to normalize the output sum to form a probability distribution for the predicted category.

[0129] Specifically, the smartwatch can input the target RRI sequence and motion sequence into three convolutional layers to obtain a first output value (or first output feature). The smartwatch can then input the first output value into the LSTM layer to obtain a second output value (or second output feature). The smartwatch can then input the second output value into the fully connected layer and softmax to obtain a four-class classification result, that is, the sleep state of the wearer.

[0130] It can be understood that each convolution layer includes multiple convolution kernels, which slide on the input data (that is, the target RRI sequence and the motion sequence) according to a preset sliding step (or called a preset step). The preset sliding step refers to the length (or time) corresponding to each sliding of the convolution kernel in the convolution operation, that is, the length corresponding to one sliding of the convolution kernel. Exemplarily, the preset sliding step is pre-set according to actual needs. For example, the preset sliding step can be 30s, 15s, etc., without specific limitation.

[0131] In some embodiments, the smart watch can continuously use the target RRI sequence and motion sequence within a first time period as input, run the above-mentioned sleep state detection model, and the sleep state detection model slides the above-mentioned convolution kernel with a preset step size to output the wearer's continuous sleep state within the first time period. The first time period is the time period corresponding to the smart watch needing to perform sleep state detection, that is, the time period when the wearer has fallen asleep or is about to fall asleep. In one example, the first time period can be a time period pre-set according to the sleeping habits of most wearers. For example, the first time period can be from 20:00 to 10:00 am the next day. In another example, the first time period can also be a time period pre-set according to the sleeping habits of the wearer of the smart watch. For example, the first time period can be from 23:00 to 8:00 am the next day.

[0132] In this embodiment, the convolution kernel is slid on the input data to detect the sleep state of the wearer during the time period (that is, the preset convolution time described below) in which the convolution kernel is located. In this way, real-time detection of the sleep state can be achieved, so that the smart watch can intervene in the wearer in a timely manner according to the real-time detection results, thereby improving the wearer's sleep condition and thus improving the wearer's sleep quality.

[0133] Among them, the size of the above-mentioned convolution kernel is determined based on the preset convolution time and the number of parameters corresponding to the input data (or input value), that is, the width of the convolution kernel is the preset convolution time, and the height of the convolution kernel is the number of parameters of the input data. The preset convolution time is the time length of the input data that can be contained in a convolution kernel. It can be understood that since the parameters corresponding to the input data in this embodiment are the target RRI sequence and the motion sequence, the number of parameters is 2. If the input data also includes data of other parameters, then the number of parameters can also be the sum of the numbers corresponding to other parameters, and there is no specific limitation.

[0134] For example, Figure 8As shown, the time corresponding to each cell is 15s, that is, one cell corresponds to 15s of input data. Specifically, the convolution kernel A slides on the input data according to the preset sliding step. It can be seen that the parameters corresponding to the cells in the first row are the target RRI sequence, and the parameters corresponding to the cells in the second row are the motion sequence. Therefore, it can be explained that the sliding direction of the convolution kernel is horizontal sliding, and the convolution kernel slides from left to right. Among them, the width of the convolution kernel A is the preset convolution time (60s), that is, each convolution kernel contains the target RRI sequence and motion sequence obtained by the smart watch for 60s. The height of the convolution kernel is the number of parameters of the input data (2). It can be understood that the input data includes data corresponding to the target RRI sequence and data corresponding to the motion sequence. Therefore, the number of parameters can be 2. In addition, the preset sliding step of the convolution kernel is 30s, that is, the smart watch can output the wearer's sleep status once every 30s.

[0135] In one implementation, the three layers of CNN included in the above-mentioned sleep state detection model can be named CNN layer one, CNN layer two, and CNN layer three, respectively. Among them, CNN layer one can include at least one of a first preset number of convolution kernels, maximum pooling, and activation functions. The first preset number is a number pre-set according to actual conditions. In this embodiment, the first preset number is 64. In other embodiments, the first preset number can also be other values, for example, it can be 32, etc., which are not specifically limited. Max pooling is one of the commonly used pooling operations in convolutional neural networks. Through this max pooling operation, the spatial dimension of the feature vector can be reduced, thereby reducing the number of parameters and computational complexity of the model and enhancing the robustness of the model. Exemplarily, the activation function can be a rectified linear unit (ReLU), a sigmoid function, a hyperbolic tangent function (tanh), etc., which are not specifically limited.

[0136] Among them, the above-mentioned CNN layer 2 includes at least one of a first preset number of convolution kernels, maximum pooling and activation functions. The above-mentioned CNN layer 3 includes at least one of a second preset number of convolution kernels, maximum pooling and activation functions. The second preset number is twice the above-mentioned first preset number, that is, the number of convolution kernels in the CNN layer 3 can be 128, 64, etc., and is not specifically limited. It can be understood that since the CNN layer 3 includes the second preset number of convolution kernels, the feature vector corresponding to the first output value input to the LSTM layer is the second preset number * 1. In this embodiment, the feature vector is 128*1.

[0137] For example, Figure 9 As shown in the figure, after obtaining the above-mentioned target RRI sequence and motion sequence, the smartwatch can input the target RRI sequence and motion sequence as input data into CNN layer 1, CNN layer 2, and CNN layer 3 in sequence, and input the output result (or the above-mentioned first output feature) into the LSTM layer. Among them, the CNN layer 1 includes 64 convolution kernels, maximum pooling, and activation functions. CNN layer 2 includes 64 convolution kernels. CNN layer 3 includes 128 convolution kernels.

[0138] Among them, the feature vector size of the output feature of the above-mentioned LSTM layer is set to a third preset number, and the time length corresponding to the first output feature passing through the LSTM layer should not be less than the time length of the entire night input data, that is, the sequence length passing through the LSTM layer is the maximum length after the convolution operation of the CNN layer.

[0139] In one implementation, the length of time for overnight data input can be pre-set based on actual conditions. For example, if a wearer normally sleeps no more than 15 hours, the smartwatch can set the length of overnight data input to 15 hours. Furthermore, using the example of each cell corresponding to 15 seconds of input data, it can be understood that 15 hours is equivalent to 54,000 seconds, so the maximum length is 3,600 cells.

[0140] In another implementation, the length of the overnight data input period can be pre-set based on the smartwatch wearer's sleep history. For example, if the smartwatch wearer typically sleeps no more than 10 hours, the smartwatch can set the length of the overnight data input period to 10 hours. Furthermore, using the example of each cell corresponding to 15 seconds of input data, it can be understood that 10 hours is equivalent to 36,000 seconds, so the maximum length is 2,400 cells.

[0141] In some embodiments, the training process of the above-mentioned sleep state detection model may specifically include: the smart watch obtains an input data set. The input data set includes multiple input data and each input data carries a real state label, and the real state label is used to characterize the sleep state corresponding to the input data. For example, if the real state label is awake, the input data corresponding to the real state label is the data obtained by the smart watch when the wearer is awake. The input data includes an RRI sequence and a motion sequence, and the sampling rate of the RRI sequence is the same as that of the motion sequence. Afterwards, for each input data, the smart watch can input the input data into the sleep state detection model to be trained to obtain a predicted state label. Afterwards, the smart watch can adjust the parameters of the sleep state detection model to be trained according to the predicted state label and the real state label carried by the input data to obtain a trained sleep state detection model.

[0142] For example, Figure 10 As shown, the sleep state detection process may specifically include: the smart watch can obtain PPG data through the PPG sensor, and perform peak detection on the PPG data to obtain an RRI sequence. Also, the smart watch can obtain ACC data through the ACC sensor, and perform feature extraction on the ACC data to obtain a motion sequence. Afterwards, the smart watch can interpolate and normalize the RRI sequence to obtain a target RRI sequence. Afterwards, the smart watch can vectorize the target RRI sequence and the motion sequence, and input the combined sequence table into three convolutional layers, LSTM layers, and softmax in sequence to obtain the wearer's sleep state. Among them, the sleep state can be a rapid eye movement state, a deep sleep state, a light sleep state, and an awake state.

[0143] In one implementation, the input data may also include skin resistance data (or skin resistance signal). The skin resistance data is used to reflect the wearer's sweat gland secretion. The skin resistance signal may be collected via a galvanic skin response (GSR) sensor. The GSR sensor is used to measure changes in skin conductivity to provide information about the wearer's physiological and emotional state. It is understood that if the wearer's emotions fluctuate significantly (such as sudden nervousness), the wearer's sweat glands secrete more, and the wearer's body conductivity increases, thereby reducing the skin resistance value.

[0144] Specifically, the smartwatch can collect skin resistance data through the above-mentioned GSR sensor. Afterwards, the smartwatch can perform feature extraction on the skin resistance data to obtain a skin resistance sequence. The sampling rate of the skin resistance sequence is the same as the sampling rate of the above-mentioned motion sequence. Afterwards, the smartwatch can input the skin resistance sequence, the above-mentioned target RRI sequence, and the motion sequence into the above-mentioned sleep state detection model to obtain the wearer's sleep state. In this way, the wearer's sleep state can be detected from multiple angles, allowing the smartwatch to detect the wearer's sleep state more accurately, thereby providing a basis for subsequent timely sleep intervention of the wearer based on the sleep state, thereby improving the wearer's user experience.

[0145] For example, Figure 11 As shown, the sampling times corresponding to the above-mentioned skin resistance sequence, target RRI sequence and motion sequence are the same. Taking the sampling time of 1 second as an example, the first column of cells in the schematic diagram includes the data corresponding to the target RRI sequence, motion sequence and skin resistance sequence in the first second. The second column of cells includes the data corresponding to the target RRI sequence, motion sequence and skin resistance sequence in the second second. The third column of cells includes the data corresponding to the target RRI sequence, motion sequence and skin resistance sequence in the third second. The fourth column of cells includes the data corresponding to the target RRI sequence, motion sequence and skin resistance sequence in the fourth second. The fifth column of cells includes the data corresponding to the target RRI sequence, motion sequence and skin resistance sequence in the fifth second. The sixth column of cells includes the data corresponding to the target RRI sequence, motion sequence and skin resistance sequence in the sixth second. The seventh column of cells includes the data corresponding to the target RRI sequence, motion sequence and skin resistance sequence in the seventh second.

[0146] In some embodiments, after obtaining the sleep status of the wearer, the smartwatch can perform sleep intervention based on the sleep status of the wearer. In this way, the wearer's sleep condition can be improved, and sleep improvement services that match the current sleep status can be provided, thereby improving the wearer's sleep quality.

[0147] In one example, if the wearer is in a deep sleep state, the smartwatch can intercept all notification messages. This can reduce the impact of notification messages on the wearer's sleep, provide a good sleeping environment for the wearer, and enhance the wearer's user experience.

[0148] In another example, when the wearer is in a rapid eye movement state, the smart watch can play sleep-aiding music and other sleep-aiding content to help the wearer quickly enter a deep sleep state and improve the wearer's user experience.

[0149] In other embodiments, after obtaining the sleep state of the wearer, the smart watch can send the sleep state to the target electronic device (such as a mobile phone) in real time. The target electronic device is a device that establishes a communication connection with the smart watch. The communication connection includes a wired communication connection or a wireless communication connection. Figure 12 The communication system shown introduces the process of communication between the target electronic device 100 and the smart watch 200.

[0150] like Figure 12 As shown, the communication system may include a target electronic device 100 and a smartwatch 200. For example, the target electronic device may be a mobile phone, a tablet computer, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), or other electronic device. This application does not impose any restrictions on the specific type of the electronic device 100.

[0151] In one possible implementation, the target electronic device 100 can establish a wireless communication connection with the smartwatch 200, and the target electronic device 100 and the smartwatch 200 can transmit data information to each other via the wireless communication connection. For example, the smartwatch 200 can transmit data information such as the wearer's sleep data to the target electronic device 100 via the wireless communication connection, and the target electronic device 100 can transmit data information such as the display status and / or the device status of the target electronic device 100 to the smartwatch 200 via the wireless communication connection. The sleep data can refer to the time occupied by each sleep state, such as light sleep, deep sleep, and rapid eye movement, during the user's entire sleep period, as well as the proportion of each sleep state in the sleep period. The data information transmitted between the target electronic device 100 and the smartwatch 200 will be described in detail in subsequent embodiments and will not be repeated here. Specifically, the wireless communication connection can be one or more wireless communication connections such as Bluetooth, wireless fidelity direct (WIFI direct), or wireless fidelity software access point (WIFI softAP).

[0152] In another possible implementation, the target electronic device 100 may also establish a wired communication connection with the smartwatch 200 to exchange data. For example, the target electronic device 100 and the smartwatch 200 may establish a wired connection via a universal serial bus (USB) and transmit data information to each other based on the wired communication connection.

[0153] Afterwards, if the target electronic device receives the wearer's sleep status, it can perform sleep intervention on the wearer based on the sleep status. In other words, the sleep intervention process can be performed by the smartwatch or by the target electronic device that has established a communication connection with the smartwatch, without limitation.

[0154] In yet other embodiments, upon detecting that the wearer's sleep state has switched from a first sleep state to an awake state, the smartwatch can transmit the time occupied by each sleep state during the entire sleep period to a target electronic device. This allows the target device to analyze the wearer's sleep quality based on the time occupied by each sleep state and provide the wearer with a sleep analysis report, helping the wearer to clearly understand their sleep status and improving the wearer's user experience.

[0155] In some embodiments, the present application provides a computer-readable storage medium including computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the method described above.

[0156] In some embodiments, the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the method described above.

[0157] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0158] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0159] The units described as separate components may or may not be physically separate, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple places. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0160] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0161] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0162] The above content is only a specific embodiment of this application, but the scope of protection of this application is not limited to this. Any changes or replacements within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for detecting a sleep state, characterized in that: Applied to a wearable device, the method includes: When the pressure value between the wearable device and the wearer's wrist is within a preset pressure range, the wearable device obtains photoplethysmography (PPG) data and accelerometer (ACC) data; the pressure value is used to represent the degree of fit between the wearable device and the wearer's wrist; The wearable device processes the PPG data to obtain a target RR interval RRI sequence, and processes the ACC data to obtain a motion sequence; the target RRI sequence is used to represent the interval time between each heartbeat of the wearer; the motion sequence is used to represent whether the wearer is in a motion state; the sampling frequency of the target RRI sequence is consistent with that of the motion sequence; The wearable device continuously uses the target RRI sequence and the motion sequence within a first time period as input and runs a sleep state detection model, wherein the sleep state detection model periodically outputs the sleep state of the wearer with a preset step size; wherein the sleep state detection model has the ability to predict the sleep state based on the RRI sequence and the motion sequence; The sleep state includes rapid eye movement state, deep sleep state, light sleep state or awake state.

2. The method according to claim 1, characterized in that The method further comprises: The wearable device acquires skin resistance data; The wearable device performs feature extraction on the skin resistance data to obtain a skin resistance sequence; The wearable device continuously uses the target RRI sequence and the motion sequence in the first time period as input and runs a sleep state detection model, including: The wearable device continuously takes the target RRI sequence, the motion sequence and the skin resistance sequence in the first time period as input to run a sleep state detection model.

3. The method according to claim 1, characterized in that The sleep state detection model includes a convolutional neural network (CNN) layer, which is used to extract features from the target RRI sequence and the motion sequence, and the CNN layer includes a convolution kernel; The sleep state detection model periodically outputs the sleep state of the wearer at a preset step size, including: The sleep state detection model slides the convolution kernel with a preset step size and periodically outputs the wearer's continuous sleep state within the first time period with the preset step size.

4. The method according to claim 3, characterized in that The width of the convolution kernel is the preset convolution time, the preset convolution time is the time length of the input data included in a convolution kernel, the height of the convolution kernel is the number of parameters of the input data, and the input data includes the target RRI sequence and the motion sequence.

5. The method according to any one of claims 1 to 4, characterized in that The wearable device processes the PPG data to obtain a target RR interval RRI sequence, including: The wearable device processes the PPG data to obtain an RR interval RRI sequence; The wearable device performs interpolation processing on the RRI sequence to obtain a target RRI sequence having the same sampling rate as the motion sequence.

6. The method according to any one of claims 1 to 4, characterized in that Before the wearable device acquires photoplethysmography (PPG) data and accelerometer (ACC) data, the method further includes: When the first condition is met, the wearable device receives a data collection instruction from the electronic device; The wearable device acquires photoplethysmography (PPG) data and accelerometer (ACC) data, including: In response to the data collection instruction, the wearable device obtains the PPG data and the ACC data; The first condition includes at least one of the following: The current time is within the preset sleep period; and, The electronic device is in a screen-off state, and the screen-off duration reaches a preset duration.

7. The method according to claim 1, characterized in that The method further comprises: When the current time is within the preset sleep period and the pressure value is not within the preset pressure range, the wearable device sends a prompt message, and the prompt message is used to prompt the wearer to adjust the tightness of wearing the wearable device.

8. The method according to any one of claims 1 to 4, characterized in that The wearable device acquires photoplethysmography (PPG) data and accelerometer (ACC) data, including: The wearable device obtains the ACC data; When the motion sequence obtained based on the ACC data indicates that the wearer is not in a motion state, the wearable device obtains the PPG data.

9. A wearable device, characterized in that: The wearable device includes a PPG sensor, an ACC sensor, a display screen, a memory, and one or more processors; the PPG sensor, the ACC sensor, the display screen, the memory, and the processor are coupled; The PPG sensor is used to collect PPG data, and the ACC sensor is used to collect ACC data; The display screen is used to display the image generated by the processor, and the memory is used to store computer program code, wherein the computer program code includes computer instructions; when the processor executes the computer instructions, the wearable device executes the method as described in any one of claims 1 to 8.

10. The wearable device according to claim 9, wherein: The wearable device further includes a skin electrical GSR sensor; the GSR sensor is coupled to the PPG sensor, the ACC sensor, the display screen, the memory, and the processor; The GSR sensor is used to collect skin resistance data.

11. A computer-readable storage medium, characterized in that The invention comprises computer instructions, which, when executed on a wearable device, enable the wearable device to perform the method according to any one of claims 1 to 8.

12. A computer program product, characterized in that The invention comprises computer instructions, which, when executed on a wearable device, enable the wearable device to perform the method according to any one of claims 1 to 8.

Images