Sleep detection method, related apparatus, and communication system

Abstract

The present application provides a sleep detection method, a related apparatus, and a communication system. The method utilizes the combination of data obtained by various detections such as heart rate detection, activity level statistics, going-to-bed action detection, walking feature detection, ambient light detection, ambient sound detection, and use detection of an electronic device such as a mobile phone and / or a tablet personal computer to determine the sleep onset time of a user, thereby improving the accuracy of sleep onset time detection. Moreover, the method further combines heart rate data, activity level data, as well as walking features and action features of going to bed and getting out of bed to determine the going-to-bed time and getting-out-of-bed time of the user.

Description

Sleep detection method, related device and communication system

[0001] This application claims priority to Chinese patent application No. 202410063004.8 filed with the State Intellectual Property Office of China on January 15, 2024, and priority to Chinese patent application entitled “A sleep detection method, related device and communication system”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] The present application relates to the field of terminal technology, and in particular to a sleep detection method, related devices, and a communication system. Background Art

[0003] With the development of electronic devices and research on users' sleep status, more and more electronic devices (such as smart watches and smart bracelets) can detect users' sleep status. For example, electronic devices can detect the time a user falls asleep, the time a user wakes up, and the stages of their sleep state. However, electronic devices usually perform sleep detection based on the user's heart rate data and motion data, and the detection results often have errors. For example, in scenarios where a user is lying quietly in bed playing with a mobile phone, listening to music, or watching TV, the user's heart rate data is usually low and the user's activity level is low. Electronic devices may determine that a user has fallen asleep when the user is not actually asleep. Summary of the Invention

[0004] This application provides a sleep detection method, related device, and communication system. This method determines the user's sleep onset time by combining data obtained from multiple detections, including heart rate detection, activity statistics, bed-going movement detection, walking characteristics detection, ambient light detection, ambient sound detection, and detection of the use of electronic devices such as mobile phones and / or tablets, thereby improving the accuracy of sleep onset detection.

[0005] In a first aspect, the present application provides a sleep detection method. Specifically, the method comprises obtaining a user's heart rate and motion data, the motion data including acceleration and / or angular velocity; obtaining ambient light brightness, ambient sound volume, and usage data of one or more electronic devices, the usage data of one or more electronic devices including the screen off time of one or more electronic devices; determining activity level, walking characteristics, and bed-going movement characteristics based on the motion data; and determining the user's sleep time based on the heart rate, activity level, walking characteristics, bed-going movement characteristics, ambient light brightness, ambient sound volume, and usage data of one or more electronic devices.

[0006] The one or more electronic devices may include one or more of the following: a mobile phone, a tablet computer, a television, and a laptop computer.

[0007] It can be seen that the above method combines multiple factors: heart rate, motion data, ambient light brightness, ambient sound volume, and usage data of one or more electronic devices to determine the user's bedtime, which can improve the accuracy of bedtime detection.

[0008] In combination with the first aspect, in some embodiments, a first sleeping point is determined based on bed-going action characteristics, a second sleeping point is determined based on activity level, a third sleeping point is determined based on walking characteristics, a fourth sleeping point is determined based on heart rate, a fifth sleeping point is determined based on ambient light brightness, a sixth sleeping point is determined based on ambient sound volume, and a seventh sleeping point is determined based on usage data of one or more electronic devices; the user's bedtime is determined based on the first, second, third, fourth, fifth, sixth, and seventh sleeping points.

[0009] Among them, the first sleep point is the time when the action of going to bed occurs, the second sleep point is the time when the activity level is less than the activity level threshold, the third sleep point is the time when the user changes from a walking state to a non-walking state, the fifth sleep point is the time when the ambient light brightness is less than the brightness threshold, the sixth sleep point is the time when the ambient sound volume is less than the volume threshold, and the seventh sleep point is the time when one or more electronic devices turn off the screen.

[0010] In combination with the first aspect, in some embodiments, the first time is determined based on the second sleep point and the fourth sleep point; when the first sleep point, the third sleep point, the fifth sleep point, the sixth sleep point, and the seventh sleep point are all earlier than the first time, the first time is determined as the user's bedtime.

[0011] The first time is any time between the second sleep onset point and the fourth sleep onset point, or is the average of the second sleep onset point and the fourth sleep onset point.

[0012] As can be seen, the phone can use the second sleep time determined by activity level and the fourth sleep time determined by heart rate as a benchmark, with the first, third, and fifth to seventh sleep time points as auxiliary indicators to determine the user's bedtime. As you can understand, activity level and heart rate can directly reflect whether the user has fallen asleep. The aforementioned bed-going movements, walking characteristics, ambient light brightness, ambient sound volume, and the phone screen being off can reflect the user's preparation for bed. It may take some time for the user to actually fall asleep after preparing for bed. If the first and third sleep time points are earlier than the first time, it may indicate that the user took some time to fall asleep after getting into bed. If the fifth sleep time point is earlier than the first time, it may indicate that the ambient light dimmed before the user fell asleep. If the sixth sleep time point is earlier than the first time, it may indicate that the ambient sound volume decreased before the user fell asleep. If the seventh sleep time point is earlier than the first time, it may indicate that the user turned off the screen of their mobile phone or other electronic device before falling asleep.

[0013] In combination with the first aspect, in some embodiments, when the fifth and sixth sleep points are both earlier than the seventh sleep point, and the seventh sleep point is later than the first time, a second time is determined based on the seventh sleep point, and the second time is determined as the user's bedtime, and the second time is later than the seventh sleep point.

[0014] It can be seen that when the user falls asleep based on activity level and heart rate, the user may be lying quietly in bed using their phone, meaning they are not actually asleep. Furthermore, it often takes some time for the user to fall asleep after turning off the phone screen. Therefore, the second time after the phone or other electronic device turns off can be used as the user's bedtime. This combination of sleep detection based on the usage of mobile phones and other electronic devices can improve the accuracy of sleep onset detection.

[0015] In combination with the first aspect, in some embodiments, when the sixth sleep point and the seventh sleep point are both earlier than the fifth sleep point, the fifth sleep point is later than the first time, and the time difference between the fifth sleep point and the first time is less than or equal to the first difference, a third time is determined based on the fifth sleep point, and the third time is determined as the user's sleep time, and the third time is later than the fifth sleep point.

[0016] It can be seen that when the sixth and seventh sleep onset points are both earlier than the fifth sleep onset point, the user may have turned off the audio playing in the sleeping environment and turned off the screens of electronic devices such as mobile phones before turning off the lights to go to bed. When the fifth sleep onset point is later than the above-mentioned first time and the time difference between it and the first time is less than or equal to the first difference, the user may have been lying quietly in bed with the lights on for a while before turning off the lights to prepare for sleep. Therefore, the user's actual sleep onset time may be slightly later than the time when the ambient light brightness dims. The above-mentioned sleep detection combined with ambient light brightness can improve the accuracy of sleep onset detection.

[0017] In combination with the first aspect, in some embodiments, when the fifth sleep point and the seventh sleep point are both earlier than the sixth sleep point, the sixth sleep point is later than the first time, and the time difference between the sixth sleep point and the first time is less than or equal to the second difference, the fourth time is determined based on the sixth sleep point, and the fourth time is determined as the user's sleep time, and the fourth time is later than the sixth sleep point.

[0018] It can be seen that if the fifth and seventh sleep time points are both earlier than the sixth sleep time point, the user may have turned off the phone screen and lights before the audio playback was turned off. If the sixth sleep time point is later than the first time mentioned above and the time difference between the sixth sleep time point and the first time is less than or equal to the second difference, the user may have quietly listened to music in bed for a while and then actively turned off the audio playback. Therefore, the user's actual sleep time may be slightly later than the sixth sleep time point (i.e., the time when the ambient sound volume decreases).

[0019] In combination with the first aspect, in some embodiments, when the sixth sleep point is later than the first time, the time difference between the sixth sleep point and the first time is greater than the second difference, and the fifth sleep point and the seventh sleep point are both earlier than the first time, the fifth time is determined based on the first time, and the fifth time is determined as the user's sleep time, and the fifth time is later than the first time.

[0020] In combination with the first aspect, in some embodiments, when the sixth sleep point and the seventh sleep point are later than the first time, the time difference between the sixth sleep point and the first time is greater than the second difference, and the fifth sleep point is earlier than the first time, the sixth time is determined based on the seventh sleep point, and the sixth time is determined as the user's sleep time, and the sixth time is later than the seventh sleep point.

[0021] In combination with the first aspect, in some embodiments, when the fifth and sixth sleep points are later than the first time, the time difference between the fifth sleep point and the first time is less than or equal to the first difference, the time difference between the sixth sleep point and the first time is greater than the second difference, and the seventh sleep point is earlier than the first time, the seventh time is determined based on the fifth sleep point, and the seventh time is determined as the user's sleep time, and the seventh time is later than the fifth sleep point.

[0022] In combination with the first aspect, in some embodiments, when the fifth and sixth sleep points are later than the first time, the time difference between the fifth sleep point and the first time is greater than the first difference, the time difference between the sixth sleep point and the first time is greater than the second difference, and the seventh sleep point is earlier than the first time, the eighth time is determined based on the first time, and the eighth time is determined as the user's sleep time, and the eighth time is later than the first time.

[0023] It can be seen that when the sixth sleeping point is later than the above-mentioned first time and the time difference between the sixth sleeping point and the first time is greater than the second difference, the user may fall asleep during the audio playback. The audio continues to play and is not paused after the user falls asleep. The moment when the ambient sound volume decreases (i.e., the sixth sleeping point) cannot reflect the user's actual sleeping time. Among them, if the above-mentioned fifth sleeping point and / or seventh sleeping point are within the time period between the first time and the sixth sleeping point, the user may have turned off the lights and / or turned off the mobile phone screen after lying quietly in bed for a while and continued to listen to the audio. Therefore, when the moment when the ambient sound volume decreases (i.e., the sixth sleeping point) cannot reflect the user's actual sleeping time, the mobile phone can determine the sleeping point based on the fifth sleeping point and / or the seventh sleeping point. The above embodiment combines the brightness of the ambient light, the volume of the ambient sound and the usage of the mobile phone to perform sleep detection, which can improve the accuracy of sleeping time detection.

[0024] In combination with the first aspect, in some embodiments, a first time period is determined according to the user's bedtime, and the user's first heart rate and first motion data in the first time period are obtained; a second time period is determined according to the user's bedtime, and the user's second heart rate and second motion data in the second time period are obtained; a first activity amount, a first walking characteristic, and a first going-to-bed action characteristic are determined according to the first motion data, and a second activity amount, a second walking characteristic, and a first getting-out-of-bed action characteristic are determined according to the second motion data; the user's going-to-bed time and getting-out-of-bed time are determined according to the first heart rate, the second heart rate, the first activity amount, the first walking characteristic, the first going-to-bed action characteristic, the second activity amount, the second walking characteristic, and the first getting-out-of-bed action characteristic.

[0025] As can be seen, the above method combines walking characteristics, heart rate, activity level, and in-and-out movement detection around sleep onset and wake-up times to determine the user's bedtime and exit times. Because a user's walking behavior changes significantly when entering and exiting bed, and the walking characteristics during the in-and-out phases are symmetrical, this method of detecting sleep based on walking characteristics can improve the accuracy of in-and-out time detection.

[0026] In combination with the first aspect, in some embodiments, a first bed-going point is determined based on a first heart rate and a first activity level; a first bed-getting point is determined based on a second heart rate and a second activity level; a second bed-going point and a second bed-getting point are determined based on a first walking characteristic and a second walking characteristic; a third bed-going point is determined based on a first bed-going movement characteristic; a third bed-getting point is determined based on a first bed-getting movement characteristic; a user's bedtime is determined based on the first bed-going point, the second bed-going point, and the third bed-getting point; and a user's bedtime is determined based on the first bed-getting point, the second bed-getting point, and the third bed-getting point.

[0027] Among them, the first going-in-bed point is the time when the first heart rate drops to the first heart rate threshold and the first activity level drops to the first activity level threshold; the first getting-out-of-bed point is the time when the second heart rate rises to the second heart rate threshold and / or the second activity level rises to the second activity level threshold; the second going-in-bed point is the time when the last walking behavior occurs in the first time period, and the second getting-out-of-bed point is the time when the first walking behavior occurs in the second time period; the third going-in-bed point is the time when the going-in-bed action occurs, and the third getting-out-of-bed point is the time when the getting-in-bed action occurs.

[0028] In combination with the first aspect, in some embodiments, when the time difference between any two of the first, second, and third going-to-bed points is less than a third difference, the average of the first, second, and third going-to-bed points is determined as the user's going-to-bed time, or any time between the earliest and latest of the first, second, and third going-to-bed points is determined as the user's going-to-bed time; when the time difference between any two of the first, second, and third getting-out-of-bed points is less than a fourth difference, the average of the first, second, and third getting-out-of-bed points is determined as the user's getting-out-of-bed time, or any time between the earliest and latest of the first, second, and third getting-out-of-bed points is determined as the user's getting-out-of-bed time.

[0029] In combination with the first aspect, in some embodiments, when the time difference between the second and third bedgoing points is less than the third difference, and the time difference between the first and second bedgoing points is greater than or equal to the third difference, the average of the second and third bedgoing points is determined as the user's bedtime, or any time between the second and third bedgoing points is determined as the user's bedtime; when the time difference between the second and third bedgoing points is less than the fourth difference, and the time difference between the first and second bedgoing points is greater than or equal to the fourth difference, the average of the second and third bedgoing points is determined as the user's bedtime, or any time between the second and third bedgoing points is determined as the user's bedtime.

[0030] The first time period includes a time period before the user's bedtime, and the second time period includes a time period after the user's bedtime.

[0031] In combination with the first aspect, in some embodiments, the user's sleep falling time, sleep waking time, bedtime and bedtime are displayed.

[0032] In combination with the first aspect, in some embodiments, the user's heart rate and motion data are obtained by a wearable device, and the wearable device includes one or more of the following: a smart watch, a smart bracelet.

[0033] In a second aspect, the present application provides a sleep detection method that can be applied to a communication system including a smartwatch and a mobile phone. The smartwatch can obtain the user's heart rate and motion data, including acceleration and / or angular velocity; the smartwatch can also obtain ambient light brightness and ambient sound volume; the mobile phone can obtain usage data of one or more electronic devices, including the screen-off time of one or more electronic devices; the mobile phone can determine activity level, walking characteristics, and bed-going movement characteristics based on the motion data; and the mobile phone can determine the user's sleep time based on the heart rate, activity level, walking characteristics, bed-going movement characteristics, ambient light brightness, ambient sound volume, and usage data of one or more electronic devices.

[0034] The one or more electronic devices may include one or more of the following: a mobile phone, a tablet computer, a television, and a laptop computer.

[0035] In some embodiments, the smartwatch can send the user's heart rate and exercise data to the phone. The smartwatch can also send the ambient light brightness and ambient sound volume to the phone.

[0036] It can be seen that the above method combines multiple factors: heart rate, motion data, ambient light brightness, ambient sound volume, and usage data of one or more electronic devices to determine the user's bedtime, which can improve the accuracy of bedtime detection.

[0037] In combination with the second aspect, in some embodiments, the mobile phone can determine the first sleeping point based on the characteristics of going to bed, the mobile phone can determine the second sleeping point based on the amount of activity, the mobile phone can determine the third sleeping point based on the walking characteristics, the mobile phone can determine the fourth sleeping point based on the heart rate, the mobile phone can determine the fifth sleeping point based on the ambient light brightness, the mobile phone can determine the sixth sleeping point based on the ambient sound volume, and the mobile phone can determine the seventh sleeping point based on the usage data of one or more electronic devices; the mobile phone can determine the user's bedtime based on the first sleeping point, the second sleeping point, the third sleeping point, the fourth sleeping point, the fifth sleeping point, the sixth sleeping point, and the seventh sleeping point.

[0038] Among them, the first sleep point is the time when the action of going to bed occurs, the second sleep point is the time when the activity level is less than the activity level threshold, the third sleep point is the time when the user changes from a walking state to a non-walking state, the fifth sleep point is the time when the ambient light brightness is less than the brightness threshold, the sixth sleep point is the time when the ambient sound volume is less than the volume threshold, and the seventh sleep point is the time when one or more electronic devices turn off the screen.

[0039] Optionally, one or more of the first to sixth sleeping points may be determined by a smartwatch, wherein the smartwatch may send the sleeping points determined by itself to the mobile phone.

[0040] In combination with the second aspect, in some embodiments, the mobile phone can determine the first time based on the second sleeping point and the fourth sleeping point; when the first sleeping point, the third sleeping point, the fifth sleeping point, the sixth sleeping point, and the seventh sleeping point are all earlier than the first time, the mobile phone can determine the first time as the user's bedtime.

[0041] The first time is any time between the second sleep onset point and the fourth sleep onset point, or is the average of the second sleep onset point and the fourth sleep onset point.

[0042] As can be seen, the phone can use the second sleep time determined by activity level and the fourth sleep time determined by heart rate as a benchmark, with the first, third, and fifth to seventh sleep time points as auxiliary indicators to determine the user's bedtime. As you can understand, activity level and heart rate can directly reflect whether the user has fallen asleep. The aforementioned bed-going movements, walking characteristics, ambient light brightness, ambient sound volume, and the phone screen being off can reflect the user's preparation for bed. It may take some time for the user to actually fall asleep after preparing for bed. If the first and third sleep time points are earlier than the first time, it may indicate that the user took some time to fall asleep after getting into bed. If the fifth sleep time point is earlier than the first time, it may indicate that the ambient light dimmed before the user fell asleep. If the sixth sleep time point is earlier than the first time, it may indicate that the ambient sound volume decreased before the user fell asleep. If the seventh sleep time point is earlier than the first time, it may indicate that the user turned off the screen of their mobile phone or other electronic device before falling asleep.

[0043] In combination with the second aspect, in some embodiments, when the fifth and sixth sleep points are both earlier than the seventh sleep point, and the seventh sleep point is later than the first time, the mobile phone can determine a second time based on the seventh sleep point, and determine the second time as the user's bedtime, and the second time is later than the seventh sleep point.

[0044] It can be seen that when the user falls asleep based on activity level and heart rate, the user may be lying quietly in bed using their phone, meaning they are not actually asleep. Furthermore, it often takes some time for the user to fall asleep after turning off the phone screen. Therefore, the second time after the phone or other electronic device turns off can be used as the user's bedtime. This combination of sleep detection based on the usage of mobile phones and other electronic devices can improve the accuracy of sleep onset detection.

[0045] In combination with the second aspect, in some embodiments, when the sixth sleep point and the seventh sleep point are both earlier than the fifth sleep point, the fifth sleep point is later than the first time, and the time difference between the fifth sleep point and the first time is less than or equal to the first difference, the mobile phone can determine a third time based on the fifth sleep point, and determine the third time as the user's bedtime, and the third time is later than the fifth sleep point.

[0046] It can be seen that when the sixth and seventh sleep onset points are both earlier than the fifth sleep onset point, the user may have turned off the audio playing in the sleeping environment and turned off the screens of electronic devices such as mobile phones before turning off the lights to go to bed. When the fifth sleep onset point is later than the above-mentioned first time and the time difference between it and the first time is less than or equal to the first difference, the user may have been lying quietly in bed with the lights on for a while before turning off the lights to prepare for sleep. Therefore, the user's actual sleep onset time may be slightly later than the time when the ambient light brightness dims. The above-mentioned sleep detection combined with ambient light brightness can improve the accuracy of sleep onset detection.

[0047] In combination with the second aspect, in some embodiments, when the fifth sleep point and the seventh sleep point are both earlier than the sixth sleep point, the sixth sleep point is later than the first time, and the time difference between the sixth sleep point and the first time is less than or equal to the second difference, the mobile phone can determine the fourth time based on the sixth sleep point, and determine the fourth time as the user's bedtime, and the fourth time is later than the sixth sleep point.

[0048] It can be seen that if the fifth and seventh sleep time points are both earlier than the sixth sleep time point, the user may have turned off the phone screen and lights before the audio playback was turned off. If the sixth sleep time point is later than the first time mentioned above and the time difference between the sixth sleep time point and the first time is less than or equal to the second difference, the user may have quietly listened to music in bed for a while and then actively turned off the audio playback. Therefore, the user's actual sleep time may be slightly later than the sixth sleep time point (i.e., the time when the ambient sound volume decreases).

[0049] In combination with the second aspect, in some embodiments, when the sixth sleep point is later than the first time, the time difference between the sixth sleep point and the first time is greater than the second difference, and the fifth sleep point and the seventh sleep point are both earlier than the first time, the mobile phone can determine the fifth time based on the first time, and determine the fifth time as the user's bedtime, and the fifth time is later than the first time.

[0050] In combination with the second aspect, in some embodiments, when the sixth and seventh sleep points are later than the first time, the time difference between the sixth sleep point and the first time is greater than the second difference, and the fifth sleep point is earlier than the first time, the mobile phone can determine the sixth time based on the seventh sleep point, and determine the sixth time as the user's bedtime, and the sixth time is later than the seventh sleep point.

[0051] In combination with the second aspect, in some embodiments, when the fifth and sixth sleep points are later than the first time, the time difference between the fifth sleep point and the first time is less than or equal to the first difference, the time difference between the sixth sleep point and the first time is greater than the second difference, and the seventh sleep point is earlier than the first time, the mobile phone can determine the seventh time based on the fifth sleep point, and determine the seventh time as the user's bedtime, and the seventh time is later than the fifth sleep point.

[0052] In combination with the second aspect, in some embodiments, when the fifth and sixth sleep points are later than the first time, the time difference between the fifth sleep point and the first time is greater than the first difference, the time difference between the sixth sleep point and the first time is greater than the second difference, and the seventh sleep point is earlier than the first time, the mobile phone can determine the eighth time based on the first time, and determine the eighth time as the user's bedtime, and the eighth time is later than the first time.

[0053] It can be seen that when the sixth sleeping point is later than the above-mentioned first time and the time difference between the sixth sleeping point and the first time is greater than the second difference, the user may fall asleep during the audio playback. The audio continues to play and is not paused after the user falls asleep. The moment when the ambient sound volume decreases (i.e., the sixth sleeping point) cannot reflect the user's actual sleeping time. Among them, if the above-mentioned fifth sleeping point and / or seventh sleeping point are within the time period between the first time and the sixth sleeping point, the user may have turned off the lights and / or turned off the mobile phone screen after lying quietly in bed for a while and continued to listen to the audio. Therefore, when the moment when the ambient sound volume decreases (i.e., the sixth sleeping point) cannot reflect the user's actual sleeping time, the mobile phone can determine the sleeping point based on the fifth sleeping point and / or the seventh sleeping point. The above embodiment combines the brightness of the ambient light, the volume of the ambient sound and the usage of the mobile phone to perform sleep detection, which can improve the accuracy of sleeping time detection.

[0054] In combination with the second aspect, in some embodiments, the mobile phone can determine a first time period based on the user's bedtime, and obtain the user's first heart rate and first motion data in the first time period; the mobile phone can determine a second time period based on the user's wake-up time, and obtain the user's second heart rate and second motion data in the second time period; the mobile phone can determine the first activity amount, the first walking characteristic, and the first going-to-bed action characteristic based on the first motion data, and determine the second activity amount, the second walking characteristic, and the first getting-out-of-bed action characteristic based on the second motion data; the mobile phone can determine the user's going-to-bed time and getting-out-of-bed time based on the first heart rate, the second heart rate, the first activity amount, the first walking characteristic, the first going-to-bed action characteristic, the second activity amount, the second walking characteristic, and the first getting-out-of-bed action characteristic.

[0055] Among them, the above-mentioned first heart rate, second motion data, second heart rate, and second motion data can be obtained by the mobile phone from the smart watch.

[0056] As can be seen, the above method combines walking characteristics, heart rate, activity level, and in-and-out movement detection around sleep onset and wake-up times to determine the user's bedtime and exit times. Because a user's walking behavior changes significantly when entering and exiting bed, and the walking characteristics during the in-and-out phases are symmetrical, this method of detecting sleep based on walking characteristics can improve the accuracy of in-and-out time detection.

[0057] In combination with the second aspect, in some embodiments, the mobile phone can determine a first bed-going point based on a first heart rate and a first activity level; the mobile phone can determine a first bed-getting point based on a second heart rate and a second activity level; the mobile phone can determine a second bed-going point and a second bed-getting point based on a first walking characteristic and a second walking characteristic; the mobile phone can determine a third bed-going point based on a first bed-going action characteristic; the mobile phone can determine a third bed-getting point based on a first bed-getting action characteristic; the mobile phone can determine a user's bedtime based on the first bed-going point, the second bed-going point, and the third bed-getting point; the mobile phone can determine a user's bedtime based on the first bed-getting point, the second bed-getting point, and the third bed-getting point.

[0058] Among them, the first going-in-bed point is the time when the first heart rate drops to the first heart rate threshold and the first activity level drops to the first activity level threshold; the first getting-out-of-bed point is the time when the second heart rate rises to the second heart rate threshold and / or the second activity level rises to the second activity level threshold; the second going-in-bed point is the time when the last walking behavior occurs in the first time period, and the second getting-out-of-bed point is the time when the first walking behavior occurs in the second time period; the third going-in-bed point is the time when the going-in-bed action occurs, and the third getting-out-of-bed point is the time when the getting-in-bed action occurs.

[0059] In combination with the second aspect, in some embodiments, when the time difference between any two of the first, second, and third going-to-bed times is less than a third difference, the mobile phone may determine the average of the first, second, and third going-to-bed times as the user's going-to-bed time, or determine any time between the earliest and latest of the first, second, and third going-to-bed times as the user's going-to-bed time; when the time difference between any two of the first, second, and third getting-out-of-bed times is less than a fourth difference, the mobile phone may determine the average of the first, second, and third getting-out-of-bed times as the user's getting-out-of-bed time, or determine any time between the earliest and latest of the first, second, and third getting-out-of-bed times as the user's getting-out-of-bed time.

[0060] In combination with the second aspect, in some embodiments, when the time difference between the second and third going-to-bed points is less than the third difference, and the time difference between the first and second going-to-bed points is greater than or equal to the third difference, the mobile phone may determine the average of the second and third going-to-bed points as the user's going-to-bed time, or determine any time between the second and third going-to-bed points as the user's going-to-bed time; when the time difference between the second and third getting-out-of-bed points is less than the fourth difference, and the time difference between the first and second getting-out-of-bed points is greater than or equal to the fourth difference, the mobile phone may determine the average of the second and third getting-out-of-bed points as the user's getting-out-of-bed time, or determine any time between the second and third getting-out-of-bed points as the user's getting-out-of-bed time.

[0061] The first time period includes a time period before the user's bedtime, and the second time period includes a time period after the user's bedtime.

[0062] In conjunction with the second aspect, in some embodiments, the smart watch and / or mobile phone can display the user's sleep time, sleep time, bed time, and bedtime.

[0063] In a third aspect, the present application provides a sleep detection method, which can be applied to a communication system including a processing device and a data acquisition device. The data acquisition device can obtain the user's heart rate and motion data, the motion data including acceleration and / or angular velocity; the data acquisition device can obtain ambient light brightness, ambient sound volume, and usage data of one or more electronic devices, the usage data of one or more electronic devices including the screen off time of one or more electronic devices; the processing device can determine the activity level, walking characteristics, and bed-going action characteristics based on the motion data; and determine the user's sleep time based on the heart rate, activity level, walking characteristics, bed-going action characteristics, ambient light brightness, ambient sound volume, and usage data of one or more electronic devices.

[0064] The one or more electronic devices may include one or more of the following: a mobile phone, a tablet computer, a television, and a laptop computer.

[0065] It can be seen that the above method combines multiple factors: heart rate, motion data, ambient light brightness, ambient sound volume, and usage data of one or more electronic devices to determine the user's bedtime, which can improve the accuracy of bedtime detection.

[0066] In combination with the third aspect, in some embodiments, the processing device can determine the first sleeping point based on the characteristics of the going to bed action, the processing device can determine the second sleeping point based on the activity amount, the processing device can determine the third sleeping point based on the walking characteristics, and determine the fourth sleeping point based on the heart rate. The processing device can determine the fifth sleeping point based on the ambient light brightness, the processing device can determine the sixth sleeping point based on the ambient sound volume, and the processing device can determine the seventh sleeping point based on the usage data of one or more electronic devices; the processing device can determine the user's bedtime based on the first sleeping point, the second sleeping point, the third sleeping point, the fourth sleeping point, the fifth sleeping point, the sixth sleeping point, and the seventh sleeping point.

[0067] Among them, the first sleep point is the time when the action of going to bed occurs, the second sleep point is the time when the activity level is less than the activity level threshold, the third sleep point is the time when the user changes from a walking state to a non-walking state, the fifth sleep point is the time when the ambient light brightness is less than the brightness threshold, the sixth sleep point is the time when the ambient sound volume is less than the volume threshold, and the seventh sleep point is the time when one or more electronic devices turn off the screen.

[0068] In combination with the third aspect, in some embodiments, the processing device may determine the first time based on the second sleep point and the fourth sleep point; when the first sleep point, the third sleep point, the fifth sleep point, the sixth sleep point, and the seventh sleep point are all earlier than the first time, the processing device may determine the first time as the user's bedtime.

[0069] The first time is any time between the second sleep onset point and the fourth sleep onset point, or is the average of the second sleep onset point and the fourth sleep onset point.

[0070] As can be seen, the above embodiment can use the second sleep onset point determined based on activity level and the fourth sleep onset point determined based on heart rate as a benchmark, with the first, third, and fifth to seventh sleep onset points as auxiliary indicators to determine the user's bedtime. It is understandable that activity level and heart rate can relatively directly reflect whether the user has fallen asleep. The aforementioned bed-going movements, walking characteristics, ambient light brightness, ambient sound volume, and the phone screen being off can reflect the user's preparation for sleep. It may take some time for the user to actually fall asleep after preparing to sleep. If the first and third sleep onset points are earlier than the first time, it may indicate that the user fell asleep some time after going to bed. If the fifth sleep onset point is earlier than the first time, it may indicate that the ambient light dimmed before the user fell asleep. If the sixth sleep onset point is earlier than the first time, it may indicate that the ambient sound volume decreased before the user fell asleep. If the seventh sleep onset point is earlier than the first time, it may indicate that the user turned off the screen of an electronic device such as a mobile phone before falling asleep.

[0071] In combination with the third aspect, in some embodiments, when the fifth sleep point and the sixth sleep point are both earlier than the seventh sleep point, and the seventh sleep point is later than the first time, the processing device can determine a second time based on the seventh sleep point, and determine the second time as the user's sleep time, and the second time is later than the seventh sleep point.

[0072] It can be seen that when the user falls asleep based on activity level and heart rate, the user may be lying quietly in bed using their phone, meaning they are not actually asleep. Furthermore, it often takes some time for the user to fall asleep after turning off the phone screen. Therefore, the second time after the phone or other electronic device turns off can be used as the user's bedtime. This combination of sleep detection based on the usage of mobile phones and other electronic devices can improve the accuracy of sleep onset detection.

[0073] In combination with the third aspect, in some embodiments, when the sixth sleep point and the seventh sleep point are both earlier than the fifth sleep point, the fifth sleep point is later than the first time, and the time difference between the fifth sleep point and the first time is less than or equal to the first difference, the processing device can determine the third time based on the fifth sleep point, and determine the third time as the user's sleep time, and the third time is later than the fifth sleep point.

[0074] It can be seen that when the sixth and seventh sleep onset points are both earlier than the fifth sleep onset point, the user may have turned off the audio playing in the sleeping environment and turned off the screens of electronic devices such as mobile phones before turning off the lights to go to bed. When the fifth sleep onset point is later than the above-mentioned first time and the time difference between it and the first time is less than or equal to the first difference, the user may have been lying quietly in bed with the lights on for a while before turning off the lights to prepare for sleep. Therefore, the user's actual sleep onset time may be slightly later than the time when the ambient light brightness dims. The above-mentioned sleep detection combined with ambient light brightness can improve the accuracy of sleep onset detection.

[0075] In combination with the third aspect, in some embodiments, when the fifth sleep point and the seventh sleep point are both earlier than the sixth sleep point, the sixth sleep point is later than the first time, and the time difference between the sixth sleep point and the first time is less than or equal to the second difference, the processing device can determine the fourth time based on the sixth sleep point, and determine the fourth time as the user's sleep time, and the fourth time is later than the sixth sleep point.

[0076] It can be seen that if the fifth and seventh sleep time points are both earlier than the sixth sleep time point, the user may have turned off the phone screen and lights before the audio playback was turned off. If the sixth sleep time point is later than the first time mentioned above and the time difference between the sixth sleep time point and the first time is less than or equal to the second difference, the user may have quietly listened to music in bed for a while and then actively turned off the audio playback. Therefore, the user's actual sleep time may be slightly later than the sixth sleep time point (i.e., the time when the ambient sound volume decreases).

[0077] In combination with the third aspect, in some embodiments, when the sixth sleep point is later than the first time, the time difference between the sixth sleep point and the first time is greater than the second difference, and the fifth sleep point and the seventh sleep point are both earlier than the first time, the processing device can determine the fifth time based on the first time, and determine the fifth time as the user's sleep time, and the fifth time is later than the first time.

[0078] In combination with the third aspect, in some embodiments, when the sixth sleep point and the seventh sleep point are later than the first time, the time difference between the sixth sleep point and the first time is greater than the second difference, and the fifth sleep point is earlier than the first time, the processing device can determine the sixth time based on the seventh sleep point, and determine the sixth time as the user's sleep time, and the sixth time is later than the seventh sleep point.

[0079] In combination with the third aspect, in some embodiments, when the fifth and sixth sleep points are later than the first time, the time difference between the fifth sleep point and the first time is less than or equal to the first difference, the time difference between the sixth sleep point and the first time is greater than the second difference, and the seventh sleep point is earlier than the first time, the processing device can determine the seventh time based on the fifth sleep point, and determine the seventh time as the user's sleep time, and the seventh time is later than the fifth sleep point.

[0080] In combination with the third aspect, in some embodiments, when the fifth and sixth sleep points are later than the first time, the time difference between the fifth sleep point and the first time is greater than the first difference, the time difference between the sixth sleep point and the first time is greater than the second difference, and the seventh sleep point is earlier than the first time, the processing device can determine the eighth time based on the first time, and determine the eighth time as the user's sleep time, and the eighth time is later than the first time.

[0081] It can be seen that when the sixth sleeping point is later than the above-mentioned first time and the time difference between the sixth sleeping point and the first time is greater than the second difference, the user may fall asleep during the audio playback. The audio continues to play and is not paused after the user falls asleep. The moment when the ambient sound volume decreases (i.e., the sixth sleeping point) cannot reflect the user's actual sleeping time. Among them, if the above-mentioned fifth sleeping point and / or seventh sleeping point are within the time period between the first time and the sixth sleeping point, the user may have turned off the lights and / or turned off the mobile phone screen after lying quietly in bed for a while and continued to listen to the audio. Therefore, when the moment when the ambient sound volume decreases (i.e., the sixth sleeping point) cannot reflect the user's actual sleeping time, the mobile phone can determine the sleeping point based on the fifth sleeping point and / or the seventh sleeping point. The above embodiment combines the brightness of the ambient light, the volume of the ambient sound and the usage of the mobile phone to perform sleep detection, which can improve the accuracy of sleeping time detection.

[0082] In combination with the third aspect, in some embodiments, the processing device can determine a first time period based on the user's bedtime, and obtain the user's first heart rate and first motion data in the first time period; the processing device can determine a second time period based on the user's wake-up time, and obtain the user's second heart rate and second motion data in the second time period; the processing device can determine the first activity amount, the first walking characteristic, and the first going-to-bed action characteristic based on the first motion data, and the processing device can determine the second activity amount, the second walking characteristic, and the first getting-out-of-bed action characteristic based on the second motion data; the processing device can determine the user's going-to-bed time and getting-out-of-bed time based on the first heart rate, the second heart rate, the first activity amount, the first walking characteristic, the first going-to-bed action characteristic, the second activity amount, the second walking characteristic, and the first getting-out-of-bed action characteristic.

[0083] The first heart rate, the first motion data, the second heart rate, and the second motion data may be acquired by the processing device from the data acquisition device.

[0084] As can be seen, the above method combines walking characteristics, heart rate, activity level, and in-and-out movement detection around sleep onset and wake-up times to determine the user's bedtime and exit times. Because a user's walking behavior changes significantly when entering and exiting bed, and the walking characteristics during the in-and-out phases are symmetrical, this method of detecting sleep based on walking characteristics can improve the accuracy of in-and-out time detection.

[0085] In combination with the third aspect, in some embodiments, the processing device can determine the first bed-going point based on the first heart rate and the first activity level; the processing device can determine the first bed-getting point based on the second heart rate and the second activity level; the processing device can determine the second bed-going point and the second bed-getting point based on the first walking characteristics and the second walking characteristics; the processing device can determine the third bed-going point based on the first bed-getting action characteristics; the processing device can determine the third bed-getting point based on the first bed-getting action characteristics; the processing device can determine the user's bed-going time based on the first bed-going point, the second bed-getting point, and the third bed-getting point; the processing device can determine the user's bed-getting time based on the first bed-getting point, the second bed-getting point, and the third bed-getting point.

[0086] Among them, the first going-in-bed point is the time when the first heart rate drops to the first heart rate threshold and the first activity level drops to the first activity level threshold; the first getting-out-of-bed point is the time when the second heart rate rises to the second heart rate threshold and / or the second activity level rises to the second activity level threshold; the second going-in-bed point is the time when the last walking behavior occurs in the first time period, and the second getting-out-of-bed point is the time when the first walking behavior occurs in the second time period; the third going-in-bed point is the time when the going-in-bed action occurs, and the third getting-out-of-bed point is the time when the getting-in-bed action occurs.

[0087] In combination with the third aspect, in some embodiments, when the time difference between any two of the first, second, and third going-to-bed points is less than the third difference, the processing device may determine the average of the first, second, and third going-to-bed points as the user's going-to-bed time, or determine any time between the earliest and latest of the first, second, and third going-to-bed points as the user's going-to-bed time; when the time difference between any two of the first, second, and third getting-out-of-bed points is less than the fourth difference, the processing device may determine the average of the first, second, and third getting-out-of-bed points as the user's getting-out-of-bed time, or determine any time between the earliest and latest of the first, second, and third getting-out-of-bed points as the user's getting-out-of-bed time.

[0088] In combination with the third aspect, in some embodiments, when the time difference between the second and third bedgoing points is less than the third difference, and the time difference between the first and second bedgoing points is greater than or equal to the third difference, the processing device may determine the average of the second and third bedgoing points as the user's bedtime, or determine any time between the second and third bedgoing points as the user's bedtime; when the time difference between the second and third bedgoing points is less than the fourth difference, and the time difference between the first and second bedgoing points is greater than or equal to the fourth difference, the processing device may determine the average of the second and third bedgoing points as the user's bedtime, or determine any time between the second and third bedgoing points as the user's bedtime.

[0089] The first time period includes a time period before the user's bedtime, and the second time period includes a time period after the user's bedtime.

[0090] In a fourth aspect, the present application provides an electronic device, which includes a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to call the computer program, so that the electronic device can implement any possible method as described in the first aspect.

[0091] In a fifth aspect, the present application provides a computer-readable storage medium comprising instructions, which, when executed on an electronic device, enables the electronic device to execute any possible implementation method as in the first aspect.

[0092] In a sixth aspect, the present application provides a computer program product, which may include computer instructions. When the computer instructions are run on an electronic device, the electronic device executes any possible implementation method as in the first aspect.

[0093] In a seventh aspect, the present application provides a chip, which is applied to an electronic device. The chip includes one or more processors, and the processor is used to call computer instructions to enable the electronic device to execute any possible implementation method as in the first aspect.

[0094] It is understandable that the electronic device provided in the fourth aspect, the computer-readable storage medium provided in the fifth aspect, the computer program product provided in the sixth aspect, and the chip provided in the seventh aspect are all used to execute the methods provided in the embodiments of the present application. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding methods and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0095] FIG1 is a schematic diagram of a communication system provided in an embodiment of the present application;

[0096] FIG2 is a schematic structural diagram of an electronic device 100 provided in an embodiment of the present application;

[0097] FIG3 is a schematic structural diagram of another electronic device 100 provided in an embodiment of the present application;

[0098] FIG4 is a schematic diagram of a method for determining the time to fall asleep provided by an embodiment of the present application;

[0099] FIG5 is a schematic diagram of a method for determining the time to go to and from bed provided by an embodiment of the present application;

[0100] FIG6 is a schematic diagram of a sleep detection result provided in an embodiment of the present application;

[0101] FIG7 is a schematic diagram of another communication system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0102] The technical solutions in the embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application. Among them, in the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to be used as limitations on the present application. As used in the specification and claims of the present application, the singular expressions "a", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one or more (including two). The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist; for example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0103] References to "one embodiment" or "some embodiments" etc. described in this specification mean that the specific features, structures or characteristics described in conjunction with the embodiment are included in one or more embodiments of the present application. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in another way. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in another way. The term "connected" includes direct and indirect connections, unless otherwise stated. "First" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated.

[0104] In the embodiments of this application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplarily" or "for example" is intended to present the relevant concepts in a concrete manner.

[0105] The present application provides a sleep detection method, which determines the user's bedtime by combining data obtained from multiple detections such as heart rate detection, activity statistics, bed-going movement detection, walking characteristic detection, ambient light detection, ambient sound detection, and detection of the use of electronic devices such as mobile phones and / or tablets, thereby improving the accuracy of sleep time detection.

[0106] In addition, the present application can also combine the heart rate data, activity data, walking characteristics and movement characteristics detected by the wearable device to determine the user's bedtime and bedtime. The above bedtime and bedtime can help users better understand their sleep status.

[0107] The sleep detection method provided in this application can be applied to the communication system 10 .

[0108] FIG1 exemplarily shows a schematic diagram of a communication system 10 .

[0109] As shown in FIG1 , a communication system 10 may include a mobile phone and a smartwatch. A communication connection may be established between the mobile phone and the smartwatch. For example, the communication connection may be a Bluetooth connection, a wireless local area network (WLAN) connection, or the like. This application does not limit the manner of the above-mentioned communication connection.

[0110] In some embodiments, the smart watch may include a heart rate detection device, an ambient light sensor, an audio input device, a motion sensor, and the like. The heart rate detection device may be used to detect the user's heart rate data. For example, the heart rate detection device may generate a photoplethysmography (PPG) signal and use the PPG signal to determine the user's heart rate data. The ambient light sensor may be used to detect the brightness of the ambient light in the environment in which the smart watch is located. The audio input device may be used to collect sound signals. For example, the audio input device may include a microphone. The motion sensor may be used to collect motion data, such as acceleration data, angular velocity data, and the like. The motion sensor may include, but is not limited to, an accelerometer, a gyroscope, and the like.

[0111] The mobile phone may also include one or more devices such as an ambient light sensor, an audio input device, and a motion sensor. The mobile phone can detect whether the screen is off. When the screen is off, the phone can also detect whether an audio playback application is running on the phone, that is, whether the phone is playing audio when the screen is off.

[0112] In some embodiments, the smartwatch can send one or more data items, such as detected heart rate data, ambient light brightness data, ambient sound brightness data, and motion data, to the mobile phone. The mobile phone can then determine the user's bedtime based on the data from the smartwatch and the phone's usage. The bedtime can indicate the time it takes for the user to transition from wakefulness to sleep.

[0113] The smartwatch or mobile phone can also determine the user's sleep time based on the heart rate data, exercise data, etc. detected by the smartwatch. The sleep time can indicate the time when the user wakes up.

[0114] A smartwatch or phone can also detect the user's bedtime and wake-up time. It's understandable that a user may not fall asleep immediately after going to bed, and may not wake up immediately after waking up. Therefore, the bedtime is earlier than or the same as the bedtime, and the wake-up time is later than or the same as the wake-up time. The bedtime can also be called the wake-up time. The smartwatch or phone can extract walking characteristics and in-and-out motion characteristics from the motion data detected by the smartwatch. Based on the bedtime and wake-up times, the smartwatch or phone can combine the walking characteristics and in-and-out motion characteristics to determine the bedtime and wake-up times.

[0115] In some embodiments, the smartwatch and / or mobile phone can display the user's sleep data. The sleep data may include but is not limited to bedtime, sleep time, wake-up time, and time to get out of bed.

[0116] The communication system 10 is not limited to smart watches and mobile phones, and can also include more devices. For example, the communication system 10 can also include wearable devices such as smart bracelets and smart glasses, tablet computers, televisions, speakers, etc. In some embodiments, the mobile phone can establish a communication connection with the wearable devices, tablet computers, televisions, speakers, and other devices in the communication system 10. In addition to the heart rate data, motion data, ambient light brightness data, and ambient sound volume data detected by the smart watch, the mobile phone can also determine the user's sleep data based on the usage of one or more devices such as smart glasses, tablet computers, televisions, and speakers. For example, the mobile phone can determine whether the user has fallen asleep based on one or more data such as the wearing status of the smart glasses, the on / off status of the tablet computer, the on / off status of the TV, and the audio playback status of the speaker. It is understandable that the change of smart glasses from the wearing state to the non-wearing state (i.e., the smart glasses are removed) can indicate that the user is about to fall asleep. The tablet computer being in the on-screen state can indicate that the user is still using the tablet computer, i.e., the user has not fallen asleep. The tablet computer going from the on-screen state to the off-screen state can indicate that the user is about to fall asleep. Similarly, a bright screen on a TV can indicate that the user is still watching TV, meaning they are not asleep yet. A speaker playing audio can indicate that the user is still listening to audio, meaning they are not asleep yet.

[0117] The structure of the electronic device involved in this application is introduced below.

[0118] FIG. 2 exemplarily shows a schematic structural diagram of the electronic device 100 .

[0119] As shown in Figure 2, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0120] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0121] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0122] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0123] Processor 110 may also include a memory for storing instructions and data. In some examples, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the instruction or data again, it can directly retrieve it from the memory. This avoids repeated accesses, reduces processor 110 latency, and thus improves system efficiency.

[0124] In the present application, a computer program may be stored in the memory, configured to cause a controller or processor to implement the sleep detection method of the present application via an interface or protocol. For example, the computer program stored in the memory may be used to: detect heart rate; determine the brightness of ambient light; determine the volume of ambient sound; determine activity level based on motion data from a motion sensor; extract walking characteristics and movement characteristics of getting in and out of bed from the motion data; determine the time to fall asleep; determine the time to wake up from sleep; determine the time to go to bed; determine the time to get out of bed; and so on.

[0125] The USB interface 130 is an interface that complies with the USB standard. The USB interface 130 can be used to connect a charger to charge the electronic device 100, or to transfer data between the electronic device 100 and peripheral devices. The USB interface 130 can also be used to connect headphones to play audio through the headphones.

[0126] The charging management module 140 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. While charging the battery 142, the charging management module 140 can also power the electronic device through the power management module 141.

[0127] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to provide power to the processor 110, the internal memory 121, the external memory, the display 194, the camera 193, and the wireless communication module 160.

[0128] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0129] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0130] The mobile communication module 150 can provide wireless communication solutions, including 2G / 3G / 4G / 5G, for the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low-noise amplifier (LNA), and the like. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, filter and amplify the received electromagnetic waves, and transmit them to the modem processor for demodulation. The mobile communication module 150 can also amplify the signals modulated by the modem processor and convert them into electromagnetic waves for radiation via the antenna 1.

[0131] The wireless communication module 160 can provide wireless communication solutions including WLAN (such as wireless fidelity (Wi-Fi) network), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc. applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signal, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0132] The electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing that connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering.

[0133] The display screen 194 is used to display images, videos, etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194 , where N is a positive integer greater than 1.

[0134] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0135] The ISP is used to process data fed back by the camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then transmitted to the ISP for processing and converted into an image visible to the naked eye.

[0136] The camera 193 is used to capture still images or videos. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0137] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0138] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU can enable intelligent cognitive applications in electronic device 100, such as image recognition, face recognition, speech recognition, and text comprehension.

[0139] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0140] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0141] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0142] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some examples, the audio module 170 can be set in the processor 110, or some functional modules of the audio module 170 can be set in the processor 110. The speaker 170A, also known as the "speaker", is used to convert audio electrical signals into sound signals. The receiver 170B, also known as the "earpiece", is used to convert audio electrical signals into sound signals. The microphone 170C, also known as the "microphone" or "microphone", is used to convert sound signals into electrical signals. The headphone jack 170D is used to connect wired headphones.

[0143] The speaker 170A and receiver 170B described above may constitute an audio output device of the electronic device 100. The audio output device of the electronic device 100 is not limited to the speaker 170A and receiver 170B, and may also include other devices for playing audio. The microphone 170C described above may constitute an audio input device of the electronic device 100. The audio input device of the electronic device 100 is not limited to the microphone 170C, and may also include other devices for collecting sound signals.

[0144] The sensor module 180 may include a pressure sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a gravity sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.

[0145] The gyroscope sensor may be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (ie, x, y, and z axes) may be determined by the gyroscope sensor.

[0146] The accelerometer can detect the magnitude of acceleration of the electronic device 100 in all directions (generally three axes). When the electronic device 100 is stationary, it can detect the magnitude and direction of gravity. The accelerometer can also be used to identify the electronic device's posture, enabling applications such as switching between landscape and portrait modes and pedometers.

[0147] In some embodiments, when the electronic device 100 is a device worn on the user's wrist (such as a smart watch or smart bracelet), the electronic device 100 may further detect the user's arm posture based on a gyroscope sensor and / or an accelerometer. The electronic device 100 may further use the user's arm posture data when detecting the user's sleeping state.

[0148] Buttons 190 include a power button, a volume button, etc. Motor 191 can generate vibration prompts. Indicator 192 can be an indicator light that can be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0149] The SIM card interface 195 is used to connect a SIM card. A SIM card can be connected to and disconnected from the electronic device 100 by inserting or removing it from the SIM card interface 195. The electronic device 100 may support one or N SIM card interfaces, where N is a positive integer greater than 1. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some examples, the electronic device 100 uses an eSIM, or embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0150] FIG3 exemplarily shows a structural diagram of another electronic device 100 .

[0151] As shown in Figure 3, the electronic device 100 may include a heart rate detection module, a motion data acquisition module, an activity statistics module, a feature extraction module, a screen on / off detection module, an ambient sound detection module, an ambient light detection module, a falling asleep and exiting sleep detection module, and a getting in and out of bed detection module.

[0152] The heart rate detection module can be used to detect the user's heart rate. In some embodiments, the heart rate detection module can collect PPG signals and determine the heart rate based on the PPG signals. The present application embodiment does not limit the above-mentioned heart rate detection method.

[0153] In some embodiments, the heart rate detection module can also determine heart rate variability (HRV) based on the heart rate data. HRV can be used to indicate the irregularity of the heartbeat. The irregularity of the heartbeat is dominated by the autonomic nervous system. Therefore, HRV can reflect the health of the nervous system. The higher the HRV, the better the cardiovascular function and stress resistance. HRV can also reflect the working condition of the autonomic nervous system. The autonomic nervous system can include the sympathetic nervous system for fighting or fleeing and the parasympathetic nervous system for relaxing or digesting. When the user is in a sympathetic nervous system-dominated mode for fighting or fleeing, the user's HRV is lower. When the user is in a parasympathetic nervous system-dominated mode for relaxing or digesting, the user's HRV is higher.

[0154] The electronic device 100 can determine whether the user has fallen asleep based on HRV. In one possible implementation, the electronic device 100 can determine the user's sleep time based on the frequency index of HRV. The frequency domain index of HRV can include high frequency power (HFP) and normalized high frequency power (nHFP). HFP can be the variance during a normal heartbeat in the high frequency range and can represent the activity of the parasympathetic nerves. nHFP can be a quantitative indicator of parasympathetic nerve activity. The time corresponding to the rising edge of nHFP can be the time when the user falls asleep.

[0155] The motion data acquisition module can be used to acquire motion data collected by the motion sensor, such as acceleration data, angular velocity data, etc.

[0156] The activity volume statistics module can be used to determine the activity volume based on the motion data. The activity volume can indicate whether the electronic device 100 is moving and whether the posture has changed. Among them, the faster the electronic device 100 moves and the greater the posture change, the greater the activity volume. When the electronic device 100 is worn on the user or held by the user, the activity volume of the electronic device 100 can reflect the activity volume of the user. For example, the electronic device 100 is a smart watch worn on the user's wrist. The activity volume determined by the activity volume statistics module of the electronic device 100 can reflect the activity of the user's arm. A small activity volume determined by the activity volume statistics module can indicate that the user's arm remains basically still and the user is likely to fall asleep. A large activity volume determined by the activity volume statistics module can indicate that the user's arm is moving frequently and the user is unlikely to fall asleep.

[0157] The feature extraction module can be used to extract walking features and action features based on motion data.

[0158] The aforementioned walking characteristics may include characteristics of a user transitioning from a walking state to a non-walking state, as well as characteristics of a user transitioning from a non-walking state to a walking state. It is understood that getting in and out of bed are two opposite processes. The walking characteristics of the user during the getting in and out phase are symmetrical. During the walking process, the user's legs alternately retract and extend their legs. For example, with the left leg in the back and the right leg in the front, the left leg first retracts and then extends, causing the left leg to be in the front and the right leg to be in the back. Then, the right leg first retracts and then extends, causing the left leg to be in the back and the right leg to be in the front again. During the getting in bed phase, the user transitions from a walking state to a non-walking state. The user walks to the edge of the bed, retracts their legs, stops walking, and lies down on the bed. Therefore, the walking characteristics of getting in bed may include: an increase in vertical acceleration when retracting their legs, and a decrease in horizontal acceleration when retracting their legs. During the getting out of bed phase, the user transitions from a non-walking state to a walking state. The user moves to the edge of the bed and extends their legs to begin walking. Therefore, the walking characteristics of getting out of bed may include: the vertical acceleration decreases when the legs are extended, and the horizontal acceleration increases when the legs are extended.

[0159] The above-mentioned action characteristics may include the action characteristics of getting into bed and the action characteristics of getting out of bed. The action characteristics may reflect the type of action performed by the user.

[0160] In some embodiments, the feature extraction module may use a feature extraction algorithm to extract the walking features and motion features. The feature extraction algorithm may include a principal component analysis (PCA) algorithm, a support vector machine (SVM) algorithm, and the like. The present embodiment does not limit the feature extraction algorithm.

[0161] The screen on / off detection module may be used to detect whether the screen of the electronic device 100 is in a screen on state or a screen off state.

[0162] The ambient sound detection module can be used to detect the volume of the sound in the environment where the electronic device 100 is located. The ambient sound detection module can obtain the sound collected by the electronic device 100 and analyze the volume of the sound.

[0163] The ambient light detection module may be used to detect the brightness of ambient light in the environment where the electronic device 100 is located. For example, the ambient light detection module may include an ambient light sensor.

[0164] The sleeping and waking up detection module can be used to determine the user's sleeping and waking up time based on one or more data such as the heart rate data detected by the above-mentioned heart rate detection module, the motion data detected by the motion data acquisition module, the activity volume determined by the activity volume statistics module, the walking characteristics and movement characteristics determined by the feature extraction module, the screen on and off status determined by the screen on and off detection module, the ambient sound volume determined by the ambient sound detection module, and the ambient light brightness determined by the ambient light detection module.

[0165] In some embodiments, the electronic device 100 may receive data indicating the usage of other electronic devices. The sleep detection module in the electronic device 100 may also determine the user's sleep onset and wake-up times in combination with the data indicating the usage of other electronic devices.

[0166] In some embodiments, the electronic device 100 may not include a heart rate detection module. The electronic device 100 may obtain heart rate data from other electronic devices (such as wearable devices such as smart watches). Optionally, the electronic device 100 uses one or more data such as motion data, ambient sound volume, and ambient light brightness detected by other devices when determining the user's sleep time.

[0167] The bed entry and exit detection module may be used to determine the user's bed entry and exit times based on one or more data such as bedtime, wake-up time, the aforementioned heart rate data, activity level, walking characteristics, and motion characteristics.

[0168] The specific method for determining the user's sleeping time and the time for getting in and out of bed will be described in subsequent embodiments.

[0169] The electronic device 100 is not limited to the modules shown in Figure 3. The electronic device 100 may also include more or fewer modules than those shown in Figure 3, or combine or separate some modules. The modules shown in Figure 3 are merely exemplary illustrations of the present application.

[0170] The structures of electronic devices (such as mobile phones, smart watches, etc.) in the communication system 10 shown in Figure 1 can refer to the device structures shown in Figures 2 and 3.

[0171] The present application provides a sleep detection method, which can obtain the user's heart rate, motion data, ambient light brightness, ambient sound volume, and usage data of one or more electronic devices. The above motion data may include acceleration and / or angular velocity, and the usage data of the above one or more electronic devices may include the screen off time of the one or more electronic devices. The method can determine the amount of activity, walking characteristics, and bed-going action characteristics based on the above motion data, and determine the user's sleep time based on the heart rate, activity, walking characteristics, bed-going action characteristics, ambient light brightness, ambient sound volume, and usage data of the above one or more electronic devices. The sleep time can also be called the sleep onset point.

[0172] The one or more electronic devices may include, but are not limited to, mobile phones, tablets, televisions, speakers, and laptops. In addition to screen-off time, usage data for the one or more electronic devices may also include one or more of the following: the duration the tablet screen is off, the duration the tablet receives user actions while the screen is on, the duration the tablet plays audio while the screen is off, the duration the television plays multimedia content, the duration the television is turned off, the duration the speaker plays audio, and the duration the speaker stops playing audio.

[0173] FIG4 exemplarily shows a schematic diagram of a method for determining the time to fall asleep provided by the present application.

[0174] In some embodiments, the present application can determine multiple suspected sleep onset points based on motion data, heart rate data, ambient light brightness, ambient sound volume, and the screen on / off status of the mobile phone and / or tablet detected by the wearable device, and then determine a single sleep onset point based on the multiple suspected sleep onset points. This single sleep onset point can represent the user's sleep onset time.

[0175] Here we take the example of using smart watches and mobile phones to determine the time to go to bed.

[0176] As shown in FIG4 , the smart watch may include a motion sensor, a heart rate detection module, an ambient light sensor, and an audio input device.

[0177] The smartwatch can collect motion data such as acceleration data and angular velocity data through the motion sensor. In some embodiments, the smartwatch can detect the bed-going action, activity statistics, and walking characteristics based on the motion data.

[0178] Among them, the smart watch can use feature extraction algorithms such as PCA and SVM to extract motion features from motion data. Then, the smart watch can use the motion features as input to the bed-going action detection model and use the bed-going action detection module to determine the motion features of the bed-going action. The smart watch can determine the time when the bed-going action occurs as suspected sleep point 1. Suspected sleep point 1 can also be called the first sleep point. The bed-going action detection model can be a trained neural network model that can be used to identify the features of the bed-going action. The embodiment of the present application does not limit the type of bed-going action detection model.

[0179] The smartwatch can determine the amount of activity based on the motion data, and determine the suspected sleeping point 2 based on the amount of activity. The suspected sleeping point 2 can also be called the second sleeping point. It is understandable that after the user falls asleep, the body usually remains still or occasionally moves slightly. Therefore, the amount of activity detected by the smartwatch after the user falls asleep is relatively small. The smartwatch can determine whether the amount of activity within a period of time is less than the activity threshold. If the amount of activity within a period of time is less than the activity threshold, the smartwatch can determine any time point within this period of time as the suspected sleeping point 2. The amount of activity within the above period of time being less than the activity threshold may include the average amount of activity in this period of time being less than the activity threshold. The duration of the above period of time and the activity threshold can both be preset. The embodiment of the present application does not limit the value of the duration of the above period of time and the activity threshold.

[0180] The smartwatch can also use a feature extraction algorithm to extract walking features from the motion data and, based on the walking features, determine the last walking time as suspected sleep onset point 3. Suspected sleep onset point 2 can also be called the third sleep onset point. The last walking time can include the time when the walking state changes to the non-walking state.

[0181] The smartwatch can detect the user's heart rate through a heart rate detection module and determine suspected sleep onset point 4 based on the heart rate. Suspected sleep onset point 4 can also be referred to as the fourth sleep onset point. In some embodiments, the smartwatch can determine HRV based on the heart rate and determine the time when the rising edge of the HRV frequency feature nHFP occurs as suspected sleep onset point 4.

[0182] The smartwatch can detect the brightness of the ambient light through the ambient light sensor, and determine the time when the brightness of the ambient light is less than the brightness threshold as the suspected sleeping point 5. The suspected sleeping point 5 can also be called the fifth sleeping point. It is understandable that users usually turn off the lights before going to bed and fall asleep in a dimly lit environment. The smartwatch can determine whether the brightness of the ambient light is less than the brightness threshold within a period of time. If the brightness of the ambient light is less than the brightness threshold during this period of time, the smartwatch can determine any time point within this period of time as the suspected sleeping point 5. The brightness of the ambient light within the above period of time is less than the brightness threshold, which may include the average brightness of the ambient light within this period of time being less than the brightness threshold. The above brightness threshold can be preset. The embodiment of the present application does not limit the value of the brightness threshold.

[0183] The smartwatch can collect ambient sound through an audio input device and detect the volume of the ambient sound. The smartwatch can determine the time when the volume of the ambient sound is less than the volume threshold as the suspected sleeping point 6. The suspected sleeping point 6 can also be called the sixth sleeping point. It is understandable that the user's sleeping environment is usually quiet. The smartwatch can determine whether the volume of the ambient sound is less than the volume threshold within a period of time. If the volume of the ambient sound is less than the volume threshold during this period of time, the smartwatch can determine any time point within this period of time as the suspected sleeping point 6. The volume of the ambient sound is less than the volume threshold during the above period of time, which may include the average volume of the ambient sound during this period of time being less than the volume threshold. The above volume threshold can be preset. The embodiment of the present application does not limit the value of the volume threshold.

[0184] The mobile phone can perform screen on / off detection to determine whether the phone is in the screen on state or the screen off state. It is understandable that the user will no longer watch the phone screen after falling asleep, and the phone screen will be in the screen off state for a long time. After detecting that the phone has entered the screen off state, the mobile phone can determine whether the phone has been in the screen off state for a preset time. If the phone has been in the screen off state for a preset time, the phone can determine the time when it entered the screen off state (i.e. the time when the screen was turned off) as the suspected sleeping point 7. Suspected sleeping point 7 can also be called the seventh sleeping point.

[0185] Optionally, the mobile phone can also determine the above-mentioned suspected sleeping point 7 in combination with the user operation on the mobile phone. For example, if the user is ready to fall asleep without turning off the mobile phone screen, the mobile phone screen may not turn off automatically, but remain in the bright screen state. In the bright screen state, the mobile phone can detect whether there is any user operation acting on the mobile phone within a preset time period. The above-mentioned preset time period can be 1 minute, 2 minutes, and so on. The embodiment of the present application does not limit the duration of the above-mentioned preset time period. If the mobile phone does not detect any user operation within the preset time period in the bright screen state, the mobile phone can determine any time within the preset time period as the suspected sleeping point 7.

[0186] In some embodiments, one or more of the suspected sleep onset points 1 to 7 can be determined by a mobile phone. For example, a smartwatch can send motion data detected by a motion sensor to the mobile phone. The mobile phone can detect bedtime movements, activity statistics, and walking characteristics based on the motion data to determine suspected sleep onset points 1 to 3. The smartwatch can send the heart rate detected by the heart rate detection module to the mobile phone. The mobile phone can determine suspected sleep onset point 4 based on the heart rate. The smartwatch can send ambient light data detected by the ambient light sensor to the mobile phone. The mobile phone can determine suspected sleep onset point 5 based on the ambient light data from the smartwatch. Optionally, the mobile phone has an ambient light sensor. The mobile phone can determine suspected sleep onset point 5 based on the ambient light data detected by its ambient light sensor. The smartwatch can send the volume of ambient sound collected by an audio input device to the mobile phone. The mobile phone can determine suspected sleep onset point 6 based on the volume of the ambient sound from the smartwatch. Optionally, the mobile phone has an audio input device. The mobile phone can collect ambient sound using its audio input device and further determine suspected sleep onset point 6 based on the volume of the ambient sound.

[0187] The mobile phone can combine the suspected sleeping points 1 to 7 to obtain the sleeping point.

[0188] In some embodiments, the mobile phone can use the suspected sleeping point 2 determined based on the amount of activity and the suspected sleeping point 4 determined based on the heart rate as a benchmark, and use the suspected sleeping point 1, suspected sleeping point 3, suspected sleeping point 5 to suspected sleeping point 7 as auxiliary to determine the sleeping point. It can be understood that the amount of activity and heart rate can more directly reflect whether the user has fallen asleep. The above-mentioned bed-going actions, walking characteristics, the brightness of the ambient light, the volume of the ambient sound, and the phone screen being turned off can reflect that the user is preparing to fall asleep. It may take some time for the user to prepare to fall asleep and actually fall asleep. For example, after going to bed, the user may play with the phone for a while before falling asleep. After turning off the lights, the user may meditate for a while before falling asleep.

[0189] After the user falls asleep, the amount of activity will decrease and the heart rate will drop to close to the resting heart rate. If the user's activity is large (for example, the user frequently turns over, waves his arms, etc.), the user's heart rate will also be relatively high. Therefore, the suspected sleeping point 2 and the suspected sleeping point 4 are usually the same or the difference is small. The mobile phone can detect whether the difference between the suspected sleeping point 2 and the suspected sleeping point 4 is less than the first preset time difference. For example, the above-mentioned first preset time difference can be 2 minutes, or 5 minutes, etc. This application does not limit the value of the above-mentioned first preset time difference. If the difference between the suspected sleeping point 2 and the suspected sleeping point 4 is less than the first preset time difference, the mobile phone can determine the time point t1 based on the suspected sleeping point 2 and the suspected sleeping point 4. Time point t1 can also be called the first time. Among them, time point t1 can be the suspected sleeping point 2, or the suspected sleeping point 4, or the average of the suspected sleeping point 2 and the suspected sleeping point 4, or any time point between the suspected sleeping point 2 and the suspected sleeping point 4.

[0190] Furthermore, the mobile phone can determine whether suspected sleeping point 1, suspected sleeping point 3, suspected sleeping point 5 to suspected sleeping point 7 are earlier or later than time point t1. If suspected sleeping point 1, suspected sleeping point 3, suspected sleeping point 5 to suspected sleeping point 7 are all earlier than time point t1, the mobile phone can determine time point t1 as the sleeping point. It can be understood that suspected sleeping point 1 and suspected sleeping point 3 are before time point t1, which can indicate that the user fell asleep after a period of time (i.e., the time difference between suspected sleeping point 1 and time point t1, or the time difference between suspected sleeping point 3 and time point t1) after going to bed. Suspected sleeping point 5 is before time point t1, which can indicate that the ambient light dimmed before the user fell asleep. Suspected sleeping point 6 is before time point t1, which can indicate that the volume of the ambient sound decreased before the user fell asleep. Suspected sleeping point 7 is before time point t1, which can indicate that the user turned off the mobile phone screen before falling asleep.

[0191] If the suspected sleeping point 5 is after time point t1, the mobile phone can determine whether the time difference between the suspected sleeping point 5 and time point t1 is greater than the second preset time difference. The second preset time difference can be 10 minutes, or 20 minutes, etc. The embodiment of the present application does not limit the value of the second preset time difference. The time difference between the suspected sleeping point 5 and time point t1 is greater than the second preset time difference, which may indicate that the user fell asleep in an environment with strong ambient light brightness. For example, the user may fall asleep without turning off the lights. Therefore, when the time difference between the suspected sleeping point 5 and time point t1 is greater than the second preset time difference, the suspected sleeping point 5 may not be used as an evaluation factor for determining the sleeping point.

[0192] The time difference between the suspected sleep onset point 5 and time point t1 is less than or equal to the second preset time difference, which may indicate that the ambient light dimmed within a short period of time after time point t1. For example, the user lies quietly in bed with the lights on for a while before turning off the lights and preparing to fall asleep. When the user lies quietly in bed with the lights on, the user's activity level is low and the heart rate is relatively slow. The above-mentioned suspected sleep onset point 2 and suspected sleep onset point 4 may be time points during the period when the user lies quietly in bed with the lights on. Therefore, when the time difference between the suspected sleep onset point 5 and time point t1 is less than or equal to the second preset time difference, the user's actual sleep onset time is later than time point t1 and close to the suspected sleep onset point 5.

[0193] If suspected sleep onset point 6 is after time t1, the mobile phone can determine whether the time difference between suspected sleep onset point 6 and time t1 is greater than a third preset time difference. The third preset time difference can be 10 minutes, 20 minutes, or so on. The embodiment of the present application does not limit the value of the third preset time difference. If the time difference between suspected sleep onset point 6 and time t1 is greater than the third preset time difference, it may indicate that the user fell asleep in an environment with a high ambient sound volume. For example, a user may start playing music before falling asleep and fall asleep while the music is playing. The music may continue playing without being paused after the user falls asleep. In this case, the user may be lying quietly in bed, listening to music while preparing to fall asleep. When the user is not actually asleep, the user's activity level and heart rate may already indicate that they are asleep. In other words, suspected sleep onset point 2 and suspected sleep onset point 4 may be time points before the user actually falls asleep. Therefore, if the time difference between suspected sleep onset point 6 and time t1 is greater than the third preset time difference, the user's actual sleep onset time is slightly later than time t1.

[0194] The time difference between the suspected sleep point 6 and time point t1 is less than or equal to the third preset time difference, which can indicate that the volume of the ambient sound decreases within a short period of time after time point t1. For example, the user lies quietly in bed and listens to music for a while, then turns off the music and prepares to fall asleep. When the user lies quietly in bed listening to music, the amount of activity is small and the heart rate is also relatively slow. The above-mentioned suspected sleep point 2 and suspected sleep point 4 may be time points when the user lies quietly in bed listening to music and has not yet fallen asleep. Therefore, when the time difference between the suspected sleep point 6 and time point t1 is less than or equal to the third preset time difference, the user's actual sleep time is later than time point t1 and close to the suspected sleep point 6.

[0195] Suspected sleep onset point 7 is after time t1, which may indicate that the user was still using their phone after t1. For example, the user was quietly lying in bed using their phone. While quietly lying in bed using their phone, their activity level is low and their heart rate is relatively slow. Therefore, suspected sleep onset points 2 and 4 may be times when the user was quietly lying in bed using their phone. The user's actual sleep onset time is later than time t1 and closer to suspected sleep onset point 7.

[0196] If suspected sleep onset points 1 and 3 occur after time t1, this may indicate that the user got up and went back to bed after falling asleep. For example, after falling asleep, the user might get up to use the restroom or drink water, then go back to bed. The presence of suspected sleep onset points 1 and 3 after time t1 does not affect the phone's determination of the user's actual sleep onset time.

[0197] In some embodiments, when both suspected sleep onset point 5 and suspected sleep onset point 6 are earlier than suspected sleep onset point 7, and suspected sleep onset point 7 is later than time point t1, the mobile phone may determine time point t2 based on suspected sleep onset point 7. Time point t2 may also be referred to as a second time. Time point t2 is later than suspected sleep onset point 7. For example, time point t2 may be a time point that is separated from suspected sleep onset point 7 by a preset time period after suspected sleep onset point 7. For example, the preset time period may be 3 minutes, 5 minutes, 10 minutes, or the like. The mobile phone may determine time point t2 as the sleep onset point.

[0198] Understandably, when a phone detects a user falling asleep based on activity and heart rate, the user may still be quietly lying in bed using their phone, meaning they haven't actually fallen asleep. Furthermore, it often takes some time for a user to fall asleep after turning off the phone screen. Therefore, the phone determines the time t2 after the screen turns off as the sleep onset point. This combination of sleep detection and phone usage can improve the accuracy of sleep onset detection.

[0199] In some embodiments, when both suspected sleep onset point 6 and suspected sleep onset point 7 are earlier than suspected sleep onset point 5, suspected sleep onset point 5 is later than the aforementioned time point t1, and the time difference between suspected sleep onset point 5 and time point t1 is less than or equal to a second preset time difference, the mobile phone may determine time point t3 based on suspected sleep onset point 5. The second preset time difference may also be referred to as a first difference. Time point t3 may also be referred to as a third time. Time point t3 is later than suspected sleep onset point 5. For example, time point t3 may be a time point after suspected sleep onset point 5 and separated from suspected sleep onset point 5 by a preset time period. The mobile phone may determine time point t3 as the sleep onset point.

[0200] It is understandable that when both suspected sleeping point 6 and suspected sleeping point 7 are earlier than suspected sleeping point 5, the user may have turned off the audio being played in the sleeping environment and turned off the screen of the mobile phone before turning off the lights to go to sleep. When the suspected sleeping point 5 is later than the above-mentioned time point t1 and the time difference between it and time point t1 is less than or equal to the second preset time difference, the user may have been lying quietly in bed with the lights on for a while before turning off the lights to prepare for sleep. Therefore, the user's actual sleeping time may be slightly later than the time when the ambient light brightness dims. The above-mentioned combination of sleep detection with ambient light brightness can improve the accuracy of sleep time detection.

[0201] In some embodiments, when both suspected sleep onset point 5 and suspected sleep onset point 7 are earlier than suspected sleep onset point 6, suspected sleep onset point 6 is later than the aforementioned time point t1, and the time difference between suspected sleep onset point 6 and time point t1 is less than or equal to a third preset time difference, the mobile phone may determine time point t4 based on suspected sleep onset point 6. The third preset time difference may also be referred to as a second difference. Time point t4 may also be referred to as a fourth time. Time point t4 is later than suspected sleep onset point 6. For example, time point t4 may be a time point after suspected sleep onset point 6 and separated from suspected sleep onset point 6 by a preset time period. The mobile phone may determine time point t4 as the sleep onset point.

[0202] Understandably, if both suspected sleep onset points 5 and 7 are earlier than suspected sleep onset point 6, the user may have turned off the phone screen and lights before the audio playback was turned off. If suspected sleep onset point 6 is later than time point t1 and the time difference between it and time point t1 is less than or equal to the third preset time difference, the user may have quietly listened to music in bed for a while before actively turning off the audio playback. Therefore, the user's actual sleep onset time may be slightly later than suspected sleep onset point 6 (i.e., the time when the ambient sound volume decreases).

[0203] When the suspected sleeping point 6 is later than the time point t1 and the time difference between the suspected sleeping point 6 and the time point t1 is greater than the third preset time difference, the mobile phone can determine whether the suspected sleeping point 5 and the suspected sleeping point 7 are earlier than the time point t1.

[0204] If both the suspected sleeping point 5 and the suspected sleeping point 7 are earlier than the above-mentioned time point t1, the mobile phone can determine the time point t5 based on the time point t1. Time point t5 can also be called the fifth time. Time point t5 is later than time point t1. For example, time point t5 can be a time point that is separated from time point t1 by a preset time period after time point t1. The mobile phone can determine time point t5 as the sleeping point. It can be understood that the suspected sleeping point 5 and the suspected sleeping point 7 are both earlier than the above-mentioned time point t1, which may indicate that the user turns off the lights and turns off the mobile phone screen before lying quietly in bed to prepare for sleep. However, the user may be listening to audio while lying quietly in bed. Therefore, the mobile phone can determine the user's sleeping time in combination with the volume of the ambient sound to improve the accuracy of the sleeping time detection.

[0205] If suspected sleep onset point 5 is earlier than time t1, and suspected sleep onset point 7 is later than time t1, the mobile phone can determine the sleep onset time based on suspected sleep onset point 7. For example, the sleep onset time determined by the mobile phone based on suspected sleep onset point 7 can be the sixth time. The sixth time is later than suspected sleep onset point 7. The sleep onset time determined based on suspected sleep onset point 7 (e.g., the sixth time) can be a time point later than suspected sleep onset point 7 by a preset period of time.

[0206] If the suspected sleeping point 5 is later than time point t1, and the suspected sleeping point 7 is earlier than time point t1, the mobile phone can determine the sleeping point based on the suspected sleeping point 5 when the time difference between the suspected sleeping point 5 and time point t1 is less than or equal to the second preset time difference. Exemplarily, the sleeping point determined by the mobile phone based on the suspected sleeping point 5 can be the seventh time. The seventh time is later than the suspected sleeping point 5. Among them, the sleeping point determined based on the suspected sleeping point 5 (such as the seventh time) can be a time point that is a preset time period later than the suspected sleeping point 5. If the suspected sleeping point 5 is later than time point t1, and the suspected sleeping point 7 is earlier than time point t1, the mobile phone can use the time point t5 determined by the above-mentioned time point t1 as the sleeping point when the time difference between the suspected sleeping point 5 and time point t1 is greater than the second preset time difference. Alternatively, the sleeping point determined by the mobile phone based on the time point t1 is the eighth time. The eighth time is later than time point t1. The eighth time and the above-mentioned time point t5 can be the same as or different.

[0207] If both the suspected sleeping point 5 and the suspected sleeping point 7 are later than the time point t1, the mobile phone can determine the sleeping point according to the order of the suspected sleeping point 5 and the suspected sleeping point 7. For details, please refer to the description of the above embodiment. No further details will be given here.

[0208] It is understandable that when the suspected sleeping point 6 is later than the above-mentioned time point t1 and the time difference between it and the time point t1 is greater than the third preset time difference, the user may fall asleep during the audio playback. The audio continues to play and is not paused after the user falls asleep. The moment when the ambient sound volume decreases (i.e., the suspected sleeping point 6) cannot reflect the user's actual sleep time. Among them, if the above-mentioned suspected sleeping point 5 and / or suspected sleeping point 7 are within the time period between time point t1 and suspected sleeping point 6, the user may have turned off the lights and / or turned off the mobile phone screen after lying quietly in bed for a while and continued to listen to the audio. Therefore, when the moment when the ambient sound volume decreases (i.e., the suspected sleeping point 6) cannot reflect the user's actual sleep time, the mobile phone can determine the sleep point based on the suspected sleeping point 5 and / or the suspected sleeping point 7. The above embodiment combines the brightness of the ambient light, the volume of the ambient sound and the usage of the mobile phone to perform sleep detection, which can improve the accuracy of sleep time detection.

[0209] The above method of determining the sleep onset point based on suspected sleep onset points 1 to 7 is merely an example of the present invention and should not be construed as limiting the present invention. The smartwatch and / or mobile phone may also use other methods to determine the sleep onset point based on suspected sleep onset points 1 to 7.

[0210] Not limited to the above-mentioned suspected sleeping points 1 to 7, the smart watch and / or mobile phone can also generate more or fewer suspected sleeping points, and then determine the sleeping point based on all the suspected sleeping points to improve the accuracy of sleeping time detection.

[0211] Exemplarily, the mobile phone can also obtain the usage of the tablet computer and determine a suspected sleeping point based on the usage of the tablet computer. The usage of the tablet computer may include but is not limited to: the time when the tablet computer screen is turned off, the length of time the screen is off, the time when the tablet computer receives user operations when the screen is on, the time when the tablet computer plays audio when the screen is off, etc. The suspected sleeping point determined based on the usage of the tablet computer may be a time point that reflects the time when the user stops using the tablet computer. It is understandable that the user using the tablet computer may indicate that the user has not fallen asleep yet. If the user stops using the tablet computer, the user may be about to fall asleep. That is, the actual time the user falls asleep is usually later than the suspected sleeping point determined based on the usage of the tablet computer.

[0212] The mobile phone can also obtain the usage of the TV and determine a suspected sleeping point based on the usage of the TV. The usage of the TV may include but is not limited to: the length of time the TV plays multimedia content, the time the TV is turned off, etc. The suspected sleeping point determined based on the usage of the TV may be the time point that reflects the user stopping watching TV. It is understandable that when the TV is turned off slightly later than the suspected sleeping point determined based on the activity level and heart rate, the user may lie quietly in bed and watch TV for a while before actively turning off the TV. When the TV is turned off a long time later than the suspected sleeping point determined based on the activity level and heart rate, the user may have fallen asleep while watching TV. After the user falls asleep, the TV continues to play and is not turned off.

[0213] The phone can also monitor speaker usage and determine a suspected falling asleep point based on this information. This information may include, but is not limited to, the duration of audio playback and the time the speaker stops playing. The suspected falling asleep point determined based on speaker usage may reflect the time when the user stopped listening to audio.

[0214] In some embodiments, the aforementioned electronic devices, such as smart watches, mobile phones, tablet computers, televisions, and speakers, can all be electronic devices associated with the same user. The present application embodiments do not limit the method for associating electronic devices with users. For example, if two electronic devices are associated with the same user, it can mean that the device accounts logged into the two electronic devices are the same user's accounts.

[0215] In some embodiments, the smartwatch and / or mobile phone can also detect the user's bedtime. Specifically, the smartwatch can determine the user's bedtime based on activity level and heart rate. Optionally, in addition to activity level and heart rate, the mobile phone can also determine the user's bedtime based on data such as the alarm clock set on the phone, the time when the phone screen turns on and the user begins using the phone, etc. The present embodiments do not limit the method for detecting the bedtime.

[0216] FIG5 exemplarily shows a schematic diagram of a method for determining the time to go to and from bed provided by the present application.

[0217] In some embodiments, the present application can determine the user's bedtime and wake-up time based on the time of falling asleep, the time of waking up from sleep, the heart rate data and motion data detected by the wearable device.

[0218] Here we take the example of using a smartwatch and a mobile phone to determine the time to go to bed.

[0219] As shown in FIG5 , the method for determining the time to go to and from bed may include steps S511 to S520 .

[0220] S511. Obtain PPG data.

[0221] S512: Acquire motion data detected by the motion sensor.

[0222] S513: Determine the heart rate based on the PPG data.

[0223] In some embodiments, the smartwatch can obtain PPG data through a heart rate detection module and detect motion data through a motion sensor. The smartwatch can determine the user's heart rate based on the PPG data. The smartwatch can send the heart rate and motion data to a mobile phone, which can then detect the user's bedtimes and when they get in and out of bed.

[0224] S514: Determine the activity amount based on the exercise data.

[0225] S515: Extract walking features from the motion data.

[0226] S516: Extract motion features from the motion data.

[0227] The above methods for determining the amount of activity and extracting walking characteristics and motion characteristics can refer to the introduction of the above embodiments.

[0228] The mobile phone can determine the user's sleep onset and wake-up time by combining data such as heart rate, activity level, walking characteristics, and motion characteristics. For details, please refer to the description of the embodiment shown in FIG4 .

[0229] S517: Determine the suspected time of going to bed and suspected time of getting out of bed using an in-and-out bed recognition algorithm based on the heart rate and activity level.

[0230] In some embodiments, the mobile phone can obtain heart rate and activity level for a period of time before bedtime, as well as heart rate and activity level for a period of time after bedtime. The duration of the period before bedtime and the period after bedtime can be 30 minutes, 40 minutes, 1 hour, etc. The embodiment of the present application does not limit the value of this duration.

[0231] Based on the heart rate and activity level before falling asleep, the phone can use an in-and-out recognition algorithm to determine a suspected bedtime. Based on the heart rate and activity level after waking up, the phone can use an in-and-out recognition algorithm to determine a suspected bedtime. The in-and-out recognition algorithm can determine suspected bedtimes and suspected bedtimes based on the principle that the user's heart rate and activity level decrease significantly after going to bed, and increase significantly after getting out of bed.

[0232] For example, the suspected bedtime determined by the above-mentioned in-and-out recognition algorithm can be the time before sleep onset when the user's heart rate begins to drop to heart rate threshold 1 and the activity level begins to drop to activity level threshold 1. Prior to the suspected bedtime, the user's heart rate is above heart rate threshold 1 for most of the time, and / or the user's activity level is above activity level threshold 1 for most of the time. After the suspected bedtime, the user's heart rate is below heart rate threshold 1 for most of the time, and the activity level is below activity level threshold 1 for most of the time. The suspected getting-out-of-bed time determined by the above-mentioned algorithm can be the time after wake-up time when the heart rate begins to rise to heart rate threshold 2 and / or the activity level begins to rise to activity level threshold 2. Prior to the suspected getting-out-of-bed time, the user's heart rate is below heart rate threshold 2 for most of the time, and the activity level is below activity level threshold 2 for most of the time. After the suspected getting-out-of-bed time, the user's heart rate is above heart rate threshold 2 for most of the time, and / or the activity level is above activity level threshold 2 for most of the time. The embodiments of the present application do not limit the specific implementation of the above-mentioned in-and-out recognition algorithm.

[0233] S518: Identify the time of getting in and out of bed based on the walking characteristics.

[0234] Users usually walk before going to bed, but rarely walk after going to bed. Users usually do not walk before getting out of bed, but walk after getting out of bed.

[0235] In some embodiments, the mobile phone can obtain walking characteristics for a period of time before falling asleep, and walking characteristics for a period of time after waking up. Based on the changes in the user's walking behavior when getting in and out of bed, and the principle that the walking characteristics during the getting in and out of bed phase are symmetrical, the mobile phone can determine the time when the user walks in and out of bed. The time of walking in and out of bed can include the time of walking in bed and the time of walking out of bed. The time of walking in bed can indicate the time when the last walking behavior occurred before falling asleep. The time of walking out of bed can indicate the time when the first walking behavior occurred after waking up.

[0236] S519: Identify the time when the action of getting into bed and the action of getting out of bed occur based on the action features.

[0237] The actions that most users take when going to bed every day are similar. For example, the action of going into bed may include but is not limited to: sitting up on the bed after reaching the bedside, then lifting the legs and placing them on the bed, and finally lying down and covering the quilt. The actions that most users take when getting out of bed every day are also similar. For example, the action of getting out of bed may include but is not limited to: sitting up on the bed, then moving to the bedside, moving the legs under the bed, putting on shoes and standing up. The mobile phone can detect the user's getting into bed and getting out of bed actions based on pre-trained bed-going action detection models and pre-trained bed-getting action detection models. The above-mentioned bed-going action detection model and getting out of bed action detection model can be neural network models. The embodiments of the present application do not limit the types of bed-going action detection models and getting out of bed action detection models.

[0238] In some embodiments, the mobile phone can obtain motion features from a period before falling asleep and walking features from a period after waking up. The mobile phone can use the motion features before falling asleep as input to a bed-going motion detection model, and use the bed-going motion detection model to detect the bed-going motion, thereby determining the time when the bed-going motion occurred. The mobile phone can use the motion features after waking up as input to a bed-getting motion detection model, and use the bed-getting motion detection model to detect the bed-getting motion, thereby determining the time when the bed-getting motion occurred.

[0239] In some embodiments, the above-mentioned bed-entering action detection model and bed-getting action detection model can also be self-learning, thereby improving the accuracy of bed-entering action and bed-getting action detection.

[0240] S520: Obtain the time of going to bed and the time of getting out of bed by combining the suspected time of going to bed, the suspected time of getting out of bed, the time of walking in and out of bed, and the time of the action of going to bed and the action of getting out of bed.

[0241] In some embodiments, when the suspected bedtime, the time of walking to bed, and the time when the going-to-bed action occurs are close, the mobile phone can use any one of the suspected bedtime, the time of walking to bed, and the time when the going-to-bed action occurs as the bedtime, or use the average of the suspected bedtime, the time of walking to bed, and the time when the going-to-bed action occurs as the bedtime, or select any one of the suspected bedtime, the time of walking to bed, and the time when the going-to-bed action occurs to be the bedtime.

[0242] When the suspected time of getting out of bed, the time of getting out of bed and walking, and the time of getting out of bed are close, the mobile phone can use any one of the suspected time of getting out of bed, the time of getting out of bed and walking, and the time of getting out of bed as the time of getting out of bed, or use the average of the suspected time of getting out of bed, the time of getting out of bed and walking, and the time of getting out of bed as the time of getting out of bed, or select any one of the suspected time of getting out of bed, the time of getting out of bed and walking, and the time of getting out of bed as the time of getting out of bed.

[0243] The proximity of the suspected bed-going time, the time of walking in bed, and the time of the bed-going action may indicate that the time difference between the two times is less than a preset time difference (such as the third difference value). The proximity of the suspected bed-getting time, the time of walking in bed, and the time of the bed-getting action may indicate that the time difference between the two times is less than a preset time difference (such as the fourth difference value).

[0244] Among them, if the suspected bedtime is later than the time of walking in bed and the time difference between the suspected bedtime and the time of walking in bed is greater than the preset time difference, it means that the user may have a lot of activity after going to bed. For example, the user frequently tossing and turning after going to bed, or doing exercises such as stretching in bed before going to bed. This causes the smartwatch to still detect a large amount of activity and a high heart rate after the user goes to bed. Therefore, if the suspected bedtime is later than the time of walking in bed and the time difference between the suspected bedtime and the time of walking in bed is greater than the preset time difference, the suspected bedtime can be ignored as an evaluation factor for determining the bedtime. In this case, the mobile phone can determine the bedtime based on the time of walking in bed and the time when the bed-going action occurred.

[0245] If the suspected time of getting out of bed is earlier than the time of getting out of bed and walking, and the time difference between the suspected time of getting out of bed and walking is greater than the preset time difference, it means that the user may not have gotten out of bed after waking up and has been active in bed. For example, after waking up, the user may use the mobile phone in bed, or do morning stretching and other exercises in bed. This causes the smartwatch to detect a large amount of activity and a high heart rate before the user gets out of bed. Therefore, when the suspected time of getting out of bed is earlier than the time of getting out of bed and walking, and the time difference between the suspected time of getting out of bed and walking is greater than the preset time difference, the suspected time of getting out of bed may not be used as an evaluation factor for the time of getting out of bed. At this time, the mobile phone can determine the time of getting out of bed based on the time of getting out of bed and walking and the time when the action of getting out of bed occurs.

[0246] The mobile phone is not limited to the amount of activity, heart rate, walking characteristics and motion characteristics in the period before falling asleep. The mobile phone can also determine the time to go to bed by combining the amount of activity, heart rate, walking characteristics and motion characteristics in the period after falling asleep. This can avoid the situation where the user falls asleep quickly after going to bed, and the amount of data from going to bed to falling asleep is too small, resulting in inaccurate detection. Similarly, the mobile phone is not limited to the amount of activity, heart rate, walking characteristics and motion characteristics in the period after waking up. The mobile phone can also determine the time to get out of bed by combining the amount of activity, heart rate, walking characteristics and motion characteristics in the period before waking up. This can avoid the situation where the user gets up immediately after waking up, and the amount of data from waking up to getting out of bed is too small, resulting in inaccurate detection.

[0247] In some embodiments, if no getting-out-of-bed time is detected based on activity level, heart rate, walking characteristics, or motion characteristics within a preset time period after the wake-up time, the mobile phone may determine the preset time after the wake-up time as the getting-out-of-bed time. The length of the preset time period after the wake-up time may be one hour, two hours, or the like. The preset time after the wake-up time may be one hour, two hours, or the like after the wake-up time. This is not limited in this embodiment of the present application. The failure to detect a getting-out-of-bed time based on activity level and heart rate may indicate that the user's activity level and heart rate remain low after the wake-up time. In other words, the getting-out-of-bed recognition algorithm cannot determine a suspected getting-out-of-bed time based on activity level and heart rate. The failure to detect a getting-out-of-bed time based on walking characteristics may indicate that the user has not walked since the wake-up time. In other words, the mobile phone cannot detect the time of getting-out-of-bed walking based on walking characteristics. The failure to detect a getting-out-of-bed time based on motion characteristics may indicate that the user has not made any movements to get out of bed since the wake-up time. In other words, the mobile phone cannot detect the getting-out-of-bed movement based on motion characteristics, and thus cannot determine the time when the getting-out-of-bed movement occurred.

[0248] In some embodiments, one or more of the above steps S514 to S520 may also be executed by a smart watch.

[0249] As shown in the method shown in FIG5 , this application combines walking characteristics, heart rate, activity level, and in-and-out motion detection around sleep onset and wakeup times to determine a user's bedtime and exit times. Because a user's walking behavior changes significantly when entering and exiting bed, and the walking characteristics during the in-and-out phases are symmetrical, this method of detecting sleep based on walking characteristics can improve the accuracy of in-and-out time detection.

[0250] The present application provides a sleep detection method. The method can determine a first time period based on the user's sleep onset time and obtain the user's first heart rate and first motion data in the first time period. The method can determine a second time period based on the user's sleep onset time and obtain the user's second heart rate and second motion data in the second time period. The method can determine a first activity level, a first walking characteristic, and a first bed-going motion characteristic based on the first motion data, and determine a second activity level, a second walking characteristic, and a first bed-getting motion characteristic based on the second motion data. Then, the method can determine the user's bed-going time and bed-getting time based on the first heart rate, the second heart rate, the first activity level, the first walking characteristic, the first bed-going motion characteristic, the second activity level, the second walking characteristic, and the first bed-getting motion characteristic.

[0251] The first time period may include a period of time before the sleep onset time. The second time period may include a period of time after the sleep onset time. Optionally, the first time period may further include a period of time after the sleep onset time. The second time period may further include a period of time before the sleep onset time.

[0252] In some embodiments, the first time period and the second time period may be the same length.

[0253] In some embodiments, the mobile phone can determine a first time to go to bed based on a first heart rate and a first amount of activity, and determine a first time to get out of bed based on a second heart rate and a second amount of activity. The first time to go to bed and the first time to get out of bed can be determined based on an algorithm for identifying when going to and from bed. The first time to go to bed can also be referred to as a suspected time to go to bed. The first time to get out of bed can also be referred to as a suspected time to get out of bed. The first time to go to bed can be the time when the first heart rate drops to a first heart rate threshold and the first amount of activity drops to a first activity threshold. The first time to get out of bed can be the time when the second heart rate rises to a second heart rate threshold and / or the second amount of activity rises to a second activity threshold. The first heart rate threshold can also be referred to as heart rate threshold 1. The first activity threshold can also be referred to as activity threshold 1. The second heart rate threshold can also be referred to as heart rate threshold 2. The second activity threshold can also be referred to as activity threshold 2.

[0254] The mobile phone can determine a second bed entry point and a second bed exit point based on the first walking characteristic and the second walking characteristic. The second bed entry point and the second bed exit point can be determined based on the principle that the user's walking characteristics during the bed entry phase and the bed exit phase are symmetrical. The second bed entry point and the second bed exit point are determined based on the time at which the symmetrical features of the first and second walking characteristics occur. The second bed entry point can also be referred to as the time of walking during bed entry. The second bed exit point can also be referred to as the time of walking after getting out of bed. The second bed entry point can be the time of the last walking activity during the first time period. The second bed exit point can be the time of the first walking activity during the second time period.

[0255] The mobile phone can determine a third bed-entry point based on the first bed-entry action characteristic, and a third bed-getting-out point based on the first bed-getting-out action characteristic. The third bed-entry point can be determined based on a bed-entry action detection model. The third bed-getting-out point can be determined based on a bed-getting-out action detection model. The third bed-entry point can be the time when the bed-entry action occurs. The third bed-getting-out point can be the time when the bed-getting-out action occurs.

[0256] In some embodiments, when the time difference between any two of the first, second, and third going-to-bed times is less than a third difference, the mobile phone may determine the average of the first, second, and third going-to-bed times as the user's bedtime, or determine any time between the earliest and latest of the first, second, and third going-to-bed times as the user's bedtime.

[0257] In some embodiments, when the time difference between any two of the first, second, and third getting-out-of-bed times is less than a fourth difference, the mobile phone may determine the average of the first, second, and third getting-out-of-bed times as the user's getting-out-of-bed time, or determine any time between the earliest and latest of the first, second, and third getting-out-of-bed times as the user's getting-out-of-bed time.

[0258] In some embodiments, if the time difference between the second and third bedtimes is less than a third difference, and the time difference between the first and second bedtimes is greater than the third difference, the mobile phone may determine the user's bedtime based on the second and third bedtimes. For example, the mobile phone may determine the average of the second and third bedtimes as the user's bedtime, or any time between the second and third bedtimes as the user's bedtime. In other words, if the time difference between the first and second bedtimes is too large, the first bedtime may not be used as a factor in bedtime assessment.

[0259] In some embodiments, if the time difference between the second and third getting-out-of-bed points is less than a fourth difference, and the time difference between the first and second getting-out-of-bed points is greater than the fourth difference, the mobile phone may determine the user's getting-out-of-bed time based on the second and third getting-out-of-bed points. For example, the mobile phone may determine the average of the second and third getting-out-of-bed points as the user's getting-out-of-bed time, or any time between the second and third getting-out-of-bed points as the user's getting-out-of-bed time. In other words, if the time difference between the first and second getting-out-of-bed points is too large, the first getting-out-of-bed point may not be used as a factor in evaluating the user's getting-out-of-bed time.

[0260] In some embodiments, when sleep data such as the time of falling asleep, getting in and out of bed, etc. is detected, the smartwatch and / or mobile phone can display the sleep data. Here, the mobile phone is used as an example to illustrate the display of sleep data.

[0261] FIG6 exemplarily shows a schematic diagram of a sleep detection result.

[0262] 6 , the mobile phone may display a user interface 610. The user interface 610 may include a time option 611, a sleep duration 612, a bedtime 613, a bedtime 614, a sleep onset time 615, and a sleep wake-up time 616.

[0263] The time option 611 can be used to select a time period for which to view sleep data. For example, the time option 611 may include a "Day Option," a "Week Option," a "Month Option," a "Year Option," and the like. The "Day Option" can be used to select a sleep data period for one day (e.g., yesterday). The "Week Option" can be used to select a sleep data period for one week (e.g., the most recent week). The "Month Option" can be used to select a sleep data period for one month (e.g., the most recent month). The "Year Option" can be used to select a sleep data period for one year (e.g., the most recent year).

[0264] The sleep duration 612 may be used to indicate the sleep duration of the user, that is, the duration from the time of falling asleep to the time of waking up.

[0265] Bedtime 613 may be used to indicate the user's bedtime.

[0266] The time to get out of bed 614 may be used to indicate the time when the user gets out of bed.

[0267] The sleep time 615 may be used to indicate the sleep time of the user.

[0268] The wake-up time 616 may be used to indicate the user's wake-up time.

[0269] In this way, users can understand how much time it takes from going to bed to falling asleep, and how much time it takes from waking up to getting up based on the above sleep data, so as to adjust their sleeping habits and improve their sleep quality. In addition, the mobile phone can provide sleep suggestions to users based on the time it takes from the time the user goes to bed to the time the user falls asleep, and the time it takes from the time the user wakes up to the time the user gets out of bed. For example, when it is detected that the use of electronic devices such as mobile phones and / or tablets affects the user's bedtime, the sleep suggestions provided by the mobile phone may include but are not limited to: reducing the time of using electronic devices such as mobile phones and tablets after going to bed can help you fall asleep faster. For another example, when it is detected that ambient light affects the user's bedtime, the sleep suggestions provided by the mobile phone may include but are not limited to: turning off the lights as soon as possible after going to bed can help you fall asleep faster. The above sleep suggestions are only illustrative of this application and should not constitute a limitation on this application.

[0270] Please refer to FIG. 7 , which exemplarily shows a schematic diagram of a communication system 20 provided in the present application.

[0271] The sleep detection method provided in this application can be applied to the communication system 20 .

[0272] As shown in FIG7 , the communication system 20 may include a processing device, a storage device, and a data acquisition device.

[0273] The data collection device can be used to collect heart rate data, motion data (such as acceleration, angular velocity, etc.), ambient light brightness data, ambient sound volume data, and usage data of one or more electronic devices. The usage data of the one or more electronic devices may include the screen off time of the one or more electronic devices.

[0274] In some embodiments, the data acquisition device may include one or more sensors. For example, motion sensors such as ambient light sensors, acceleration sensors, and angular velocity sensors. The data acquisition device may also include a heart rate detection device, an audio input device, and the like. The devices included in the data acquisition device may all be configured on one device, or may also be configured on multiple devices. The embodiments of the present application do not limit the form of existence of the data acquisition device. For example, the sensor, heart rate detection device, and audio input device included in the data acquisition device may all be devices on a wearable device (such as a smart watch). Alternatively, the sensor and heart rate detection device included in the data acquisition device may be devices on a smart watch, and the audio input device may be a device on a mobile phone. Alternatively, the heart rate detection device included in the data acquisition device may be an independent device, and the sensor and audio input device may be devices on a smart watch.

[0275] In some embodiments, the data collection device can transmit the collected data to a processing device. The processing device can determine the user's sleep data, such as the time of falling asleep, the time of waking up, the time of going to bed, and the time of getting out of bed, based on the data collected by the data collection device. The processing device can detect bed-going movements, count activity levels, and detect walking characteristics based on motion data. The processing device can also determine suspected sleep onset points 1 through 7 shown in FIG4 based on bed-going movements, activity levels, walking characteristics, heart rate, ambient light brightness, ambient sound volume, and electronic device usage data. The processing device can then combine suspected sleep onset points 1 through 7 to determine the sleep onset point. The processing device can also extract walking characteristics and motion characteristics from the motion data and determine the time of getting in and out of bed, and the time of getting in and out of bed movements. The processing device can determine the suspected bed-going time and suspected bed-getting time based on heart rate and activity levels. Furthermore, the processing device can determine the user's bed-going time and bed-getting time. The specific process for the processing device to determine sleep data can be referred to the methods shown in FIG4 and FIG5 . This will not be further described here.

[0276] In some embodiments, the processing device may include a processor configured on one or more devices. The embodiments of the present application do not limit the existence form of the processing device. For example, the processing device may include a processor of a smart watch and a mobile phone. The detection of bed-going movements, activity statistics and walking characteristics based on motion data can be performed by the processor of the mobile phone. Determining the suspected sleeping points 1 to 3 and suspected sleeping point 7 shown in Figure 4 above, and obtaining the sleeping point by combining the suspected sleeping points 1 to the suspected sleeping points can also be performed by the processor of the mobile phone. Determining the suspected sleeping points 4 to suspected sleeping points 6 shown in Figure 4 above can be performed by the processor of the smart watch. Alternatively, the processing device can also be an independent device. For example, the processor can be the processor of a mobile phone. The above steps of determining the user's sleep data based on the data collected by the data acquisition device can all be performed by the processor of the mobile phone.

[0277] In some embodiments, the data acquisition device may send the collected data to a storage device. The processor device may then retrieve the data collected by the data acquisition device from the storage device. The processor device may also send the detected sleep data of the user to the storage device. The storage device may store the data collected by the data acquisition device, the sleep data detected by the processor device, and the like.

[0278] The storage device may further store a computer program for causing the processing device to detect sleep data. The processing device may retrieve the computer program from the storage device and execute the computer program to detect sleep data.

[0279] In some embodiments, the storage device may include memory configured on one or more devices. The embodiments of the present application do not limit the form of the storage device. For example, the storage device may include the memory of a smartwatch and a mobile phone. Alternatively, the storage device may include only the memory of a smartwatch or only the memory of a mobile phone.

[0280] In some embodiments, the communication system 20 may further include additional devices. For example, the communication system 20 may further include a display device. The display device may include a display screen on one or more devices. The display device may be used to display sleep data detected by the processing device, such as sleep onset time, sleep exit time, bedtime, and bedtime.

[0281] It is understood that the user interfaces described in the embodiments of this application are merely exemplary interfaces and do not limit the scope of this application. In other embodiments, the user interfaces may employ different layouts, include more or fewer controls, and add or remove other functional options. As long as they are based on the same inventive concept provided by this application, they are all within the scope of protection of this application.

[0282] It should be noted that, without causing any contradiction or conflict, any feature in any embodiment of the present application, or any part of any feature, can be combined, and the combined technical solution is also within the scope of the embodiments of the present application.

[0283] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A sleep detection method, characterized in that, The method includes: Obtaining the user's heart rate and exercise data, where the exercise data includes acceleration and / or angular velocity; Obtaining the ambient light brightness, ambient sound volume, and usage data of one or more electronic devices, where the usage data of the one or more electronic devices includes the screen off time of the screens of the one or more electronic devices; Determining the activity level, walking characteristics, and bed - getting action characteristics based on the exercise data; Determining the user's bedtime based on the heart rate, the activity level, the walking characteristics, the bed - getting action characteristics, the ambient light brightness, the ambient sound volume, and the usage data of the one or more electronic devices.

2. The method according to claim 1, characterized in that, The determining the user's bedtime based on the heart rate, the activity level, the walking characteristics, the bed - getting action characteristics, the ambient light brightness, the ambient sound volume, and the usage data of the one or more electronic devices specifically includes: Determining a first bedtime point according to the bed - getting action characteristics, a second bedtime point according to the activity level, a third bedtime point according to the walking characteristics, a fourth bedtime point according to the heart rate, a fifth bedtime point according to the ambient light brightness, a sixth bedtime point according to the ambient sound volume, and a seventh bedtime point according to the usage data of the one or more electronic devices; Determining the user's bedtime based on the first bedtime point, the second bedtime point, the third bedtime point, the fourth bedtime point, the fifth bedtime point, the sixth bedtime point, and the seventh bedtime point.

3. The method according to claim 2, wherein The first bedtime point is the time when the bed - getting action occurs, the second bedtime point is the time when the activity level is less than the activity threshold, the third bedtime point is the time when the user changes from a walking state to a non - walking state, the fifth bedtime point is the time when the ambient light brightness is less than the brightness threshold, the sixth bedtime point is the time when the ambient sound volume is less than the volume threshold, and the seventh bedtime point is the time when the one or more electronic devices are turned off.

4. The method according to claim 2 or 3, characterized in that, The determining the user's bedtime based on the first bedtime point, the second bedtime point, the third bedtime point, the fourth bedtime point, the fifth bedtime point, the sixth bedtime point, and the seventh bedtime point specifically includes: Determining a first time according to the second bedtime point and the fourth bedtime point; When the first bedtime point, the third bedtime point, the fifth bedtime point, the sixth bedtime point, and the seventh bedtime point are all earlier than the first time, determining the first time as the user's bedtime.

5. The method according to claim 4, characterized in that The first time is any time between the second bedtime point and the fourth bedtime point, or the average value of the second bedtime point and the fourth bedtime point.

6. The method according to claim 4 or 5, characterized in that, The method further includes: When the fifth bedtime point and the sixth bedtime point are both earlier than the seventh bedtime point, and the seventh bedtime point is later than the first time, determining a second time according to the seventh bedtime point and determining the second time as the user's bedtime, where the second time is later than the seventh bedtime point.

7. The method according to any one of claims 4 to 6, characterized in that, The method further includes: When both the sixth sleep onset point and the seventh sleep onset point are earlier than the fifth sleep onset point, the fifth sleep onset point is later than the first time, and the time difference between the fifth sleep onset point and the first time is less than or equal to a first difference value, a third time is determined according to the fifth sleep onset point, and the third time is determined as the user's sleep onset time, where the third time is later than the fifth sleep onset point.

8. The method according to any one of claims 4 to 7, characterized in that, The method further includes: When both the fifth sleep onset point and the seventh sleep onset point are earlier than the sixth sleep onset point, the sixth sleep onset point is later than the first time, and the time difference between the sixth sleep onset point and the first time is less than or equal to a second difference value, a fourth time is determined according to the sixth sleep onset point, and the fourth time is determined as the user's sleep onset time, where the fourth time is later than the sixth sleep onset point.

9. The method according to any one of claims 4 - 8, characterized in that The method further includes: When the sixth sleep onset point is later than the first time, the time difference between the sixth sleep onset point and the first time is greater than the second difference value, and both the fifth sleep onset point and the seventh sleep onset point are earlier than the first time, a fifth time is determined according to the first time, and the fifth time is determined as the user's sleep onset time, where the fifth time is later than the first time.

10. The method according to any one of claims 4-9, characterized in that, The method further includes: When the sixth sleep onset point and the seventh sleep onset point are later than the first time, the time difference between the sixth sleep onset point and the first time is greater than the second difference value, and the fifth sleep onset point is earlier than the first time, a sixth time is determined according to the seventh sleep onset point, and the sixth time is determined as the user's sleep onset time, where the sixth time is later than the seventh sleep onset point.

11. The method according to any one of claims 4 to 10, characterized in that The method further includes: When the fifth sleep onset point and the sixth sleep onset point are later than the first time, the time difference between the fifth sleep onset point and the first time is less than or equal to the first difference value, the time difference between the sixth sleep onset point and the first time is greater than the second difference value, and the seventh sleep onset point is earlier than the first time, a seventh time is determined according to the fifth sleep onset point, and the seventh time is determined as the user's sleep onset time, where the seventh time is later than the fifth sleep onset point.

12. The method according to any one of claims 4 to 11, characterized in that, The method further includes: When the fifth sleep onset point and the sixth sleep onset point are later than the first time, the time difference between the fifth sleep onset point and the first time is greater than the first difference value, the time difference between the sixth sleep onset point and the first time is greater than the second difference value, and the seventh sleep onset point is earlier than the first time, an eighth time is determined according to the first time, and the eighth time is determined as the user's sleep onset time, where the eighth time is later than the first time.

13. The method according to any one of claims 1-12, characterized in that, The method further includes: Determine a first time period according to the user's sleep onset time, and obtain a first heart rate and first exercise data of the user during the first time period; Determine a second time period according to the user's wake-up time, and obtain a second heart rate and second exercise data of the user during the second time period; Determine the first activity amount, the first walking feature, and the first bed - getting action feature according to the first motion data, and determine the second activity amount, the second walking feature, and the first bed - leaving action feature according to the second motion data; Determine the user's bed - getting time and bed - leaving time according to the first heart rate, the second heart rate, the first activity amount, the first walking feature, the first bed - getting action feature, the second activity amount, the second walking feature, and the first bed - leaving action feature.

14. The method according to claim 13, wherein The determining of the user's bed - getting time and bed - leaving time according to the first heart rate, the second heart rate, the first activity amount, the first walking feature, the first bed - getting action feature, the second activity amount, the second walking feature, and the first bed - leaving action feature specifically includes: Determine the first bed - getting point according to the first heart rate and the first activity amount; Determine the first bed - leaving point according to the second heart rate and the second activity amount; Determine the second bed - getting point and the second bed - leaving point according to the first walking feature and the second walking feature; Determine the third bed - getting point according to the first bed - getting action feature; Determine the third bed - leaving point according to the first bed - leaving action feature; Determine the user's bed - getting time according to the first bed - getting point, the second bed - getting point, and the third bed - getting point; Determine the user's bed - leaving time according to the first bed - leaving point, the second bed - leaving point, and the third bed - leaving point.

15. The method according to claim 14, wherein The first bed - getting point is the time when the first heart rate drops to the first heart rate threshold and the first activity amount drops to the first activity amount threshold. The first bed - leaving point is the time when the second heart rate rises to the second heart rate threshold and / or the second activity amount rises to the second activity amount threshold; The second bed - getting point is the time when the last walking behavior occurs within the first time period, and the second bed - leaving point is the time when the first walking behavior occurs within the second time period. The third bed - getting point is the time when the bed - getting action occurs, and the third bed - leaving point is the time when the bed - leaving action occurs.

16. The method according to claim 14 or 15, characterized in that, The determining of the user's bed - getting time according to the first bed - getting point, the second bed - getting point, and the third bed - getting point; The determining of the user's bed - leaving time according to the first bed - leaving point, the second bed - leaving point, and the third bed - leaving point specifically includes: In the case where the time difference between any two of the first bed - getting point, the second bed - getting point, and the third bed - getting point is less than the third difference value, determine the average value of the first bed - getting point, the second bed - getting point, and the third bed - getting point as the user's bed - getting time, or determine any time between the earliest time and the latest time among the first bed - getting point, the second bed - getting point, and the third bed - getting point as the user's bed - getting time; When the time difference between any two of the first getting-out-of-bed point, the second getting-out-of-bed point, and the third getting-out-of-bed point is less than a fourth difference value, determine the average value of the first getting-out-of-bed point, the second getting-out-of-bed point, and the third getting-out-of-bed point as the getting-out-of-bed time of the user, or determine any time between the earliest time and the latest time among the first getting-out-of-bed point, the second getting-out-of-bed point, and the third getting-out-of-bed point as the getting-out-of-bed time of the user.

17. The method according to any one of claims 14 - 16, characterized in that, Determine the going-to-bed time of the user according to the first going-to-bed point, the second going-to-bed point, and the third going-to-bed point; Determine the getting-out-of-bed time of the user according to the first getting-out-of-bed point, the second getting-out-of-bed point, and the third getting-out-of-bed point, which specifically includes: When the time difference between the second going-to-bed point and the third going-to-bed point is less than a third difference value, and the time difference between the first going-to-bed point and the second going-to-bed point is greater than or equal to the third difference value, determine the average value of the second going-to-bed point and the third going-to-bed point as the going-to-bed time of the user, or determine any time between the second going-to-bed point and the third going-to-bed point as the going-to-bed time of the user; When the time difference between the second getting-out-of-bed point and the third getting-out-of-bed point is less than a fourth difference value, and the time difference between the first getting-out-of-bed point and the second getting-out-of-bed point is greater than or equal to the fourth difference value, determine the average value of the second getting-out-of-bed point and the third getting-out-of-bed point as the getting-out-of-bed time of the user, or determine any time between the second getting-out-of-bed point and the third getting-out-of-bed point as the getting-out-of-bed time of the user.

18. The method according to any one of claims 13 - 17, characterized in that, The first time period includes the time period before the user's falling asleep time, and the second time period includes the time period after the user's waking-up time.

19. The method according to any one of claims 13 - 18, characterized in that, The method further includes: Display the user's falling asleep time, waking-up time, going-to-bed time, and getting-out-of-bed time.

20. The method according to any one of claims 1-19, characterized in that, The heart rate and motion data of the user are obtained by a wearable device, and the wearable device includes one or more of the following: smart watch, smart bracelet.

21. The method according to any one of claims 1-20, characterized in that, The one or more electronic devices include one or more of the following: mobile phone, tablet computer, television, laptop computer.

22. An electronic device, characterized in that, The electronic device includes a memory and a processor. Among them, the memory is used to store a computer program; the processor is used to call the computer program so that the electronic device executes the method described in any one of claims 1-21.

23. A computer-readable storage medium stores instructions, characterized in that, When the instruction runs on the electronic device, the electronic device executes the method described in any one of claims 1-21.

24. A computer program product, characterized in that, The computer program product contains computer instructions. When the computer instructions run on the electronic device, the electronic device executes the method described in any one of claims 1-21.

Images