Electronic device, information processing system, sleep state determination method and program

Abstract

Improve the accuracy of determining the sleep state of sleeping individuals. [Solution] The electronic device 100 includes a control unit 110 that acquires multiple first analysis data in a first period, sets a first threshold based on the acquired multiple first analysis data, determines the user's sleep state based on the determination data acquired in a second period after the first period and the first threshold, and sets the first threshold to be changed at least once in the second period.

Description

Technical Field

[0001] This invention relates to an electronic device, an information processing system, a sleep state determination method, and a program.

Background Art

[0002] As a device for detecting a person's sleep state and waking up the person during a period of light sleep (REM period), for example, the device disclosed in Patent Document 1 is known. This device includes a body motion sensor that measures the body motion of a person going to bed, measures the heart rate of the person going to bed from the signal of the body motion sensor, calculates the standard deviation of the measured heart rate every predetermined time, and determines the period during which the calculated standard deviation is equal to or greater than a reference value as the period of light sleep state.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] As described above, in the conventional device, since the sleep state is determined by comparison with a certain reference value (threshold value), there is a possibility of erroneously determining the sleep state due to accidental changes in the body motion of the person going to bed or the influence of noise.

[0005] This invention has been made in view of the above circumstances, and an object thereof is to provide an electronic device, an information processing system, a sleep state determination method, and a program capable of improving the determination accuracy of the sleep state of a person going to bed.

Means for Solving the Problems

[0006] To achieve the above objective, the electronic device according to the present invention includes a control unit that acquires a plurality of first data in a first period, sets a first threshold based on the plurality of acquired first data, determines the user's sleep state based on determination data acquired in a second period after the first period and the first threshold, and sets the first threshold to be changed at least once in the second period. [Effects of the Invention]

[0007] According to the present invention, the accuracy of determining the sleep state of a person sleeping can be improved. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows an example of the configuration of an electronic device according to the first embodiment. [Figure 2] This figure shows an example of a pulse wave. [Figure 3] This diagram illustrates the analysis period and the judgment period. [Figure 4] This figure shows an example of a heart rate graph. [Figure 5] This figure shows an example of a heart rate graph after applying the Min filter. [Figure 6] This diagram illustrates the operation of the Min filter. [Figure 7] This is a diagram illustrating the histogram of the standard deviation of heart rate. [Figure 8] This diagram illustrates the heart rate data used to derive the heart rate threshold. [Figure 9] This diagram illustrates when the alarm is triggered. [Figure 10] This diagram illustrates the phenomenon where heart rate suddenly drops to a low value. [Figure 11] This diagram illustrates a method for obtaining heart rate data used to derive the heart rate threshold without using a histogram. [Figure 12] This is a flowchart of the analysis process according to the first embodiment. [Figure 13]It is a flowchart of the determination process according to the first embodiment. [Figure 14] It is a diagram for explaining the threshold value in Modified Example 4. [Figure 15] It is a diagram for explaining the threshold value in Modified Example 5. [Figure 16] It is a diagram for explaining the threshold value in Modified Example 6. [Figure 17] It is a diagram for explaining the threshold value in Modified Example 7. [Figure 18] It is a flowchart of the second determination process according to Modified Examples 4 to 7. [Figure 19] It is a diagram showing a configuration example of the information processing system according to the second embodiment.

Mode for Carrying Out the Invention

[0009] Hereinafter, embodiments will be described with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals.

[0010] The electronic device 100 according to the first embodiment is a wearable device worn by a user on the body, and specifically, for example, a smartwatch. The electronic device 100 acquires a biological signal of the user by a built-in sensor, estimates the sleep state of the user, determines the timing of light sleep as the wake-up recommended time, and wakes up the user with an electronic sound or a vibrator.

[0011] As shown in FIG. 1, the electronic device 100 includes, as functional components, a control unit 110, a storage unit 120, a sensor unit 130, an input unit 140, an output unit 150, and a communication unit 160. The control unit 110 is composed of a processor such as a CPU (Central Processing Unit), and executes various processes described later by executing a program stored in the storage unit 120. Since the control unit 110 supports a multi-thread function for executing a plurality of processes in parallel, the various processes described later can be executed in parallel. In addition, the control unit 110 also has a clock function and a timer function, and can measure the date and time. The memory unit 120 is composed of a ROM (Read Only Memory), a flash memory, a RAM (Random Access Memory), etc. In the ROM, programs executed by the CPU of the control unit 110 and data necessary in advance for executing the programs are stored. The flash memory is a writable non-volatile memory, in which data to be saved even after the power is turned off is stored. In the RAM, data created or changed during program execution is stored.

[0012] The sensor unit 130 includes a pulse wave sensor that detects the user's volume pulse wave and an acceleration sensor that detects the user's body movement. The pulse wave sensor irradiates light from the surface of the user's skin and observes the reflected light or transmitted light, thereby detecting changes in the amount of light absorption by oxyhemoglobin in the blood at a certain sampling frequency (for example, 250 Hz). Since this change in the amount of light absorption corresponds to the volume change of blood vessels, as shown in FIG. 2, a pulse wave 201 that captures the volume change of blood vessels as a waveform can be obtained by the pulse wave sensor. And since the volume change (pulse wave) of blood vessels is considered to be synchronized with the heartbeat, the timing when the amplitude of the pulse wave peaks can be estimated as the timing when the heartbeat occurs (hereinafter referred to as "pulse timing"). Also, the time interval between two adjacent peaks on the time axis in the pulse wave can be estimated as the R-R interval (RRI). In FIG. 2, the timings when the amplitude of the pulse wave 201 peaks correspond to the pulse timings 201t, 202t, 203t, and the intervals between the peaks of the pulse wave 201 correspond to the R-R intervals 202i, 203i. And when the unit of the R-R interval is seconds, the control unit 110 can calculate the heart rate (HR: Heart Rate) per minute by dividing 60 seconds by the R-R interval. In this embodiment, the control unit 110 calculates the heart rate per minute once per second based on the pulse wave detected by the sensor unit 130. Furthermore, the sensor unit 130 also includes an acceleration sensor, which detects the user's body movement by detecting acceleration in each of the three axes (X, Y, and Z axes, which are perpendicular to each other). In this embodiment, the control unit 110 samples the detected values ​​from the acceleration sensor at a certain sampling frequency (e.g., 32 Hz) to obtain a data stream of acceleration.

[0013] The input unit 140 consists of, for example, a push button switch, a touch panel, etc. The input unit 140 is a user interface for receiving user operations such as setting the alarm time (scheduled wake-up time), setting the alarm operation content (electronic sound only, vibrator only, electronic sound + vibrator, etc.), and turning the alarm function on / off. The output unit 150 includes, for example, a display unit such as a liquid crystal display or an organic EL (electroluminescence) display, a sound output unit such as a speaker or buzzer, a vibration unit such as a vibrator, etc. However, the output unit 150 may include only some of these. The display unit of the output unit 150 displays, for example, the current time. Also, if the electronic device 100 is used as an alarm clock, the output unit 150 will, for example, sound an alarm at the recommended wake-up time. The communication unit 160 is a communication interface for exchanging data with other external devices. This communication interface can be wireless or wired. For example, the electronic device 100 can transmit the determined wake-up time to an external server or the like via the communication unit 160.

[0014] As shown in Figure 3, the electronic device 100 acquires multiple biosignals (data within the first period) of the user during the analysis period (first period) while the user is sleeping, performs statistical processing, and derives a threshold for determining whether or not the user is in light sleep. Then, during the judgment period (second period) immediately preceding the set alarm time, it acquires biosignals (data within the second period), estimates the user's sleep state based on the threshold, determines the timing when sleep becomes lighter (when the value of the data within the second period exceeds the threshold) as the recommended wake-up time, and activates the alarm (e.g., by sounding an electronic beep or activating a vibrator). In this embodiment, the analysis period is 3 hours, and the judgment period can be selected by the user from 15 minutes, 30 minutes, 45 minutes, or 60 minutes. In the example shown in Figure 3, the alarm time is set to 6:00 AM and the evaluation period is set to 60 minutes. In this case, the analysis period is from 2:00 AM to 5:00 AM, and the evaluation period is from 5:00 AM to 6:00 AM. During the evaluation period, the sleep state becomes lighter than the threshold at 5:40 AM, so the alarm is activated at this time.

[0015] The inventor conducted tests on multiple subjects, simultaneously measuring sleep stages using a simple electroencephalograph, heart rate, and acceleration, and obtained the following findings from the analysis of the test results. (a) Heart rate is higher during light sleep and lower during normal sleep (deep or moderate sleep). (b) The standard deviation of heart rate is larger during light sleep and smaller during normal sleep. (c) Acceleration is high when sleep is light and low when sleep is not light. (d) Heart rate, standard deviation of heart rate, and acceleration not only differ from subject to subject, but also show subtle differences from day to day even in the same subject. In this embodiment, the threshold for determination is derived using biosignal data acquired during the analysis period immediately preceding the determination period for determining the recommended wake-up time, thereby minimizing the influence of individual differences and daily variations.

[0016] Furthermore, due to errors in the values ​​detected by the sensor unit 130 and errors in calculating heart rate from pulse waves, the heart rate acquired by the control unit 110 often contains outliers. For example, Figure 4 shows an example of a user's heart rate from around 3:45 to 4:45 on a given day. In Figure 4, the portion where the heart rate exceeds 70 is a sudden increase in value and can be considered an outlier. Such outliers occur, for example, when an algorithm that calculates heart rate from pulse waves mistakenly identifies a time when it is not the peak of the pulse wave as a peak. Conversely, if the algorithm misses the peak of the pulse wave, it will acquire outliers where the heart rate suddenly drops. However, by employing an algorithm that does not miss the peak of the pulse wave as much as possible, it is possible to minimize outliers where the heart rate suddenly drops. In reality, outliers can occur due to various other causes, but typical outliers include those that occur abruptly, as seen in the area enclosed by the dotted ellipse 301 in Figure 4, and those that occur repeatedly, as seen in the area enclosed by the dotted rectangle 302 in Figure 4. In this embodiment, outlier removal is performed using a filter (for example, a Min filter described later) to remove abrupt outliers, and outlier removal is performed using standard deviation to remove repeatedly occurring outliers.

[0017] In this embodiment, an algorithm is employed to minimize the loss of peaks in the pulse wave when calculating heart rate from the pulse wave. In this case, the possibility of outliers occurring, where the heart rate is higher than the actual value, increases, but the possibility of outliers occurring, where the heart rate is lower than the actual value, becomes very low. Therefore, by correcting the heart rate data using a Min filter that replaces the heart rate with the minimum value within a predetermined period (window), it becomes possible to almost completely eliminate outliers. For example, applying a Min filter to the heart rate graph shown in Figure 4 yields the heart rate graph shown in Figure 5. Comparing Figure 4 and Figure 5, it can be seen that even at the timing of outliers with heart rates above 70, which occur in the area enclosed by the dotted ellipse 301 in Figure 4, no outliers occur in Figure 5.

[0018] The Min filter will be explained in more detail with reference to Figure 6. Figure 6 is an enlarged view of the area from 3:50 to 3:53 in Figures 4 and 5. The Min filter is a filter that replaces each heart rate with the minimum heart rate within a certain period immediately preceding it (the first window period). In this embodiment, the first window period is set to 60 seconds. For example, in Figure 6, the original heart rate at time t is 73, but the minimum heart rate within the first window period from 60 seconds before time t to time t is 53, so the heart rate after applying the Min filter at time t is 53. Similarly, in Figure 6, the original heart rate at time s is 63, but the minimum heart rate within the first window period from 60 seconds before time s to time s is 52, so the heart rate after applying the Min filter at time s is 52. Thus, the Min filter is a filter that replaces the heart rate at each time point with the minimum value within the window (first window) of the length of the first window period. The control unit 110 then applies the Min filter to each of the multiple heart rate data points (data within the first period) acquired by shifting this first window over time (for example, by sliding it by 1 second) during the analysis period (first period), thereby generating a set of heart rate data points (first corrected data set) that have been corrected to remove sudden outliers. As will be described later, the control unit 110 also applies the Min filter to each of the multiple heart rate data points (data within the second period) acquired by shifting this first window over time (for example, by sliding it by 1 second) during the judgment period (second period), thereby generating a set of heart rate data points (second corrected data set) that have been corrected to remove sudden outliers.

[0019] Furthermore, the inventor discovered that if the heart rate data is correct, its standard deviation will fall within a certain (predetermined) range. First, if the standard deviation is too small, it can be said that it is mechanical data without biological fluctuations, so it can be seen that the heart rate data contains some kind of error. Also, since there is a limit to the magnitude of fluctuations in the heart rate of a normal person, if the standard deviation is too large, it can be seen that normal data has not been obtained. From the above, if the heart rate is normal, its standard deviation will fall within a predetermined range. Therefore, the control unit 110 can determine that the heart rate has not been obtained correctly if the standard deviation of the heart rate does not fall within the predetermined range. However, as mentioned above, the heart rate obtained in this embodiment may contain outliers, and if outliers are present, the standard deviation will be larger than the original value. Therefore, in this embodiment, the control unit 110 creates a histogram of the standard deviation of the heart rate and uses only the data that is below the 25th percentile (first quartile) of the standard deviation to determine the recommended wake-up time. A histogram, as shown in Figure 7, is a representation of the number of data points (standard deviation in Figure 7) that fall within the range of values ​​on the horizontal axis, using bins 303 on the vertical axis. Specifically, the control unit 110 slides the second window (30 seconds in this embodiment) by 1 second for each data point in the heart rate data, and calculates the standard deviation for each data point in the second window. This yields 10,800 standard deviations within the analysis period (3 hours = 180 minutes = 10,800 seconds), and creating a histogram of these results in a histogram like the one shown in Figure 7. Here, the 25th percentile value of the standard deviation is approximately 0.833 in the example in Figure 7. The control unit 110 extracts data in range A (heart rate data after applying the Min filter) that corresponds to standard deviations below the 25th percentile value. This extracted data (extracted data) is then used for analysis, and data in range B, which corresponds to standard deviations above the 25th percentile value, is treated as an error value and not used for analysis.

[0020] When the standard deviation of the data in the second window is calculated in the area where consecutive outliers occur (for example, the area enclosed by the dotted rectangle 302 in Figure 4), it becomes larger than the standard deviation of the other areas (exceeding the 25th percentile). Therefore, by using the data in the range where it is below the 25th percentile (range A in Figure 7), consecutive outliers can be removed. In this way, the control unit 110 uses the standard deviation to generate a set of standard deviation data (data Dn in range A) to be used in the analysis process described later. Then, as shown in Figure 8, the control unit 110 uses the heart rate data Dn for the time period in which the standard deviation is less than or equal to the 25th percentile (heart rate after applying the Min filter) to derive the heart rate threshold HR_TH (a heart rate threshold for determining whether or not the sleep state is light) using the following equation (1). HR_TH = Dn_ave + h × Dn_std …(1) However, the values ​​in equation (1) are as follows: Dn_ave = DnAverage value of all data Dn_std=DnStandard deviation of all data h = adjustment coefficient (in this embodiment, "2") Let me provide some supplementary explanation regarding equation (1). When the heart rate becomes somewhat higher than the average value (Dn_ave), it can be assumed that the user's sleep state has become lighter. In other words, it is clear that the heart rate threshold should be set to the average value + α. However, the magnitude of heart rate variability varies from person to person, and for people with large variability, it is necessary to set the value of α to a larger value so that they do not easily exceed the heart rate threshold. Furthermore, when the heart rate variability is large, the standard deviation (Dn_std) becomes large. Therefore, in equation (1) above, by adding h × Dn_std to Dn_ave, a more reliable heart rate threshold (that can absorb individual differences in heart rate) is set.

[0021] The control unit 110 then determines that it is the recommended time to wake up if the heart rate after applying the Min filter during the judgment period becomes greater than the heart rate threshold HR_TH derived by equation (1), and activates an alarm to wake the user. For example, Figure 9 shows a graph of a user's heart rate after applying the Min filter on a certain evening. If the heart rate threshold HR_TH is 72 and the judgment period starts at 5:00, the control unit 110 determines that 05:36, when the heart rate after applying the Min filter becomes greater than 72 after 5:00, is the recommended time to wake up and activates an alarm. However, although very rare, there are cases where the heart rate suddenly becomes an abnormally small outlier, such as the heart rate value at time 05:22 shown in Figure 10. In this case, the heart rate after applying the Min filter between 05:21 and 05:23 will also be this outlier, making it impossible to correctly determine the sleep state using the heart rate threshold. Therefore, if the heart rate after applying the Min filter is below the abnormality detection threshold (for example, 10), the control unit 110 will determine the sleep state using an acceleration threshold instead of the heart rate.

[0022] The control unit 110 also derives the acceleration threshold based on the acceleration during the analysis period. In this embodiment, the control unit 110 derives the acceleration threshold ACC_TH from the average value (ACC_ave) and standard deviation (ACC_std) of the acceleration during the analysis period using the following equation (2). ACC_TH=ACC_ave+k×ACC_std …(2) However, the values ​​in equation (2) are as follows: ACC_ave = Average value of all acceleration data during the analysis period ACC_std = Standard deviation of all acceleration data during the analysis period k = adjustment coefficient (8 in this embodiment) Let me provide some supplementary explanation regarding equation (2). When the acceleration becomes somewhat greater than the average value (ACC_ave), it can be assumed that the user's sleep state has become lighter. In other words, it is clear that the acceleration threshold should be set to the average value + α. However, the magnitude of acceleration fluctuations varies from person to person, and for people with large fluctuations, it is necessary to set a larger value for α so that they do not easily exceed the acceleration threshold. Furthermore, if the acceleration fluctuation is large, the standard deviation (ACC_std) will be large. Therefore, in equation (2) above, by adding k × ACC_std to ACC_ave, a more reliable acceleration threshold (that can absorb individual differences in acceleration) is set.

[0023] Furthermore, when deriving the heart rate threshold, the control unit 110 can determine the 25th percentile value without creating a histogram of the standard deviation of heart rate. For example, as shown in Figure 11, if the control unit 110 sorts the standard deviation of heart rate (STD[ti], STD[tj], STD[tk], ...) at each time point (ti, tj, tk, ...) in ascending order, the value corresponding to 1 / 4 of the total from the smallest value is the 25th percentile value. Therefore, the heart rate threshold can also be derived by calculating the mean (Dn_ave) and standard deviation (Dn_std) for the heart rate data after applying the Min filter (HRmin[ti], ..., HRmin[tj]) corresponding to the time points (ti, ..., tj in this example) of the standard deviation from the smallest value up to that value. Furthermore, the derivation of the heart rate threshold (first threshold) can be done by performing some statistical processing on the heart rate data, and it is not necessarily required to use the 25th percentile of the standard deviation of heart rate. For example, the median of the standard deviation of heart rate can be used, or the x-th percentile of the standard deviation of heart rate can be used for any x (where x is between 0 and 100). In addition, in equation (1), Dn_med (median of all Dn data) or Dn_mode (mode obtained from all Dn data) can be used instead of Dn_ave (mean of all Dn data). Also, in equation (1), the variance of all Dn data or a predetermined value can be used instead of Dn_std (standard deviation of all Dn data).

[0024] Similarly, the derivation of the acceleration threshold (second threshold) can be done by performing some statistical processing on the acceleration data. For example, in equation (2), instead of ACC_ave (mean value of all acceleration data during the analysis period), ACC_med (median value of all acceleration data during the analysis period) or ACC_mode (mode value obtained from all acceleration data during the analysis period) may be used. Also, in equation (2), instead of ACC_std (standard deviation of all acceleration data during the analysis period), the variance of all acceleration data during the analysis period or a predetermined value may be used.

[0025] Next, the analysis process performed during the analysis period will be explained with reference to Figure 12. The analysis process starts when the time reaches the start of the analysis period and ends when the analysis period ends, setting the first and second thresholds. For example, if the alarm time is set to 6:00 and the judgment period is set to 60 minutes, the analysis period will be from 2:00 to 5:00 as shown in Figure 3, so the analysis process will start at 2:00.

[0026] First, the control unit 110 acquires biological signals from the sensor unit 130 (step S101). More specifically, the control unit 110 acquires pulse waves with the pulse wave sensor and acceleration with the acceleration sensor. Then, the control unit 110 calculates the sum and sum of squares of accelerations acquired in step S101 (step S102). Specifically, if the sum of accelerations is stored in variable SA, the sum of squares in variable TA, and the acceleration acquired in step S101 in variable A, then in step S102, A is added to variable SA and the square of A is added to TA. By calculating the sum and sum of squares of accelerations sequentially in this way, the derivation of the acceleration threshold in step S108, described later, can be performed in a short time. Then, the control unit 110 calculates the heart rate from the pulse wave acquired in step S101 (step S103). Simply put, the heart rate can be calculated by determining the heart rate interval (sec) from the pulse wave and dividing 60 by the heart rate interval. However, in this embodiment, in order to minimize outliers, it is desirable to calculate the heart rate using an algorithm that does not miss the peaks of the pulse wave as much as possible. The control unit 110 then determines the minimum heart rate within the most recent first window period (step S104). In this embodiment, the first window period is 1 minute, but it may be a shorter period (e.g., 30 seconds) or a longer period (e.g., 2 minutes). A shorter first window period makes it easier to track changes in heart rate, allowing the alarm to be activated immediately when the user's sleep becomes lighter. On the other hand, a longer first window period can better suppress the influence of outliers in heart rate.

[0027] Next, the control unit 110 calculates the standard deviation of the heart rate within the most recent second window period (step S105). In this embodiment, the second window period is 30 seconds, but it can be a shorter period (e.g., 15 seconds) or a longer period (e.g., 1 minute). The standard deviation calculated here determines whether the heart rate calculated in step S103 is a normal value or not. Therefore, a shorter second window period allows for quicker detection of abnormal heart rates. On the other hand, a longer second window period allows for a more accurate determination of whether the heart rate is a normal value or not. Then, the control unit 110 creates a histogram of the standard deviations obtained in step S105 (step S106). At this time, the control unit 110 calculates the sum and sum of squares of the minimum heart rates obtained in step S104 for each bin of the histogram. By sequentially calculating the sum and sum of squares of the minimum heart rates for each bin of the histogram in this way, the derivation of the heart rate threshold in step S109, described later, can be performed in a short time. The control unit 110 then determines whether the analysis period has ended (step S107). If the analysis period has not ended (step S107; No), the process returns to step S101.

[0028] If the analysis period has ended (step S107; Yes), the control unit 110 derives the acceleration threshold using equation (2) described above (step S108). Then, the control unit 110 calculates the 25th percentile value for the histogram created in step S106 and derives the heart rate threshold using the above-mentioned equation (1) (step S109). Then, the control unit 110 determines whether the standard deviation (Dn_std: Dn = standard deviation of all data) calculated in step S105 is within a predetermined range (step S110). If the standard deviation is within the predetermined range (step S110; Yes), the control unit 110 turns on the flag variable HR_flg and sets it to use both the heart rate threshold and the acceleration threshold in the determination process described later (step S111), and terminates the analysis process. On the other hand, if the standard deviation is not within a predetermined range (step S110; No), the control unit 110 turns OFF the flag variable HR_flg and sets it to use only the acceleration threshold in the judgment process described later (step S112), and terminates the analysis process. The judgment condition in step S110 is one of the conditions for determining whether or not the heart rate was correctly acquired during the analysis period, and is also called the first condition.

[0029] Next, the determination process performed during the determination period will be explained with reference to Figure 13. The determination process starts when the time reaches the determination period, and determines the user's sleep state based on the acquired determination data (heart rate, acceleration) at predetermined time intervals (for example, every second for heart rate, and every 1 / 32 second for acceleration), and the user's sleep state based on the acquired determination data and the thresholds (first threshold, second threshold) set in the analysis process. For example, if the alarm time is set to 6:00 and the determination period is set to 60 minutes, the determination period will be from 5:00 to 6:00 as shown in Figure 3, so the determination process will start at 5:00.

[0030] First, the control unit 110 determines whether the flag variable HR_flg is ON or OFF (step S201). However, if neither ON nor OFF has been set for the flag variable HR_flg yet (if the analysis process is not yet complete), the control unit 110 waits until either ON or OFF is set (until the analysis process is complete), and then performs the determination in step S201.

[0031] If the flag variable HR_flg is not ON (step S201; No), the control unit 110 obtains acceleration from the acceleration sensor of the sensor unit 130 (step S202). Then, the control unit 110 determines whether the acceleration obtained in step S202 is greater than the acceleration threshold set in the analysis process (step S203). If the acceleration is greater than the acceleration threshold (step S203; Yes), the control unit 110 determines that the current time is the recommended wake-up time, which is when the sleep state is light, and activates the alarm (step S205), ending the determination process. If the acceleration is below the acceleration threshold (step S203; No), the control unit 110 determines whether the judgment period has ended (step S204). If the judgment period has not ended (step S204; No), the process returns to step S202. If the judgment period has ended (step S204; Yes), the set alarm time has arrived, so the control unit 110 proceeds to step S205 (alarm activation).

[0032] On the other hand, if the flag variable HR_flg is ON in step S201 (step S201; Yes), the control unit 110 acquires a pulse wave using the pulse wave sensor and an acceleration sensor using the acceleration sensor (step S206). Then, the control unit 110 calculates the heart rate from the pulse wave acquired in step S206 (step S207), and uses a Min filter to find the minimum value of the heart rate within the most recent first window period (step S208). Then, the control unit 110 determines whether the minimum heart rate obtained in step S208 is within a predetermined reference range (whether it is greater than the abnormality detection threshold (e.g., 10)) (step S209). In this embodiment, similar to the analysis process in the analysis period, the Min filter also slides the first window by 1 second in the determination process in the determination period to obtain the minimum heart rate within the most recent first window period. Therefore, the determination in step S209 is performed every second. If the minimum heart rate is within a predetermined reference range (less than or equal to the abnormality detection threshold) (step S209; No), the control unit 110 determines whether the acceleration obtained in step S206 is greater than the acceleration threshold set in the analysis process until the next heart rate is calculated (step S210).

[0033] If the acceleration is greater than the acceleration threshold (step S210; Yes), the control unit 110 determines that the current time, which is a time when the sleep state is light, is the recommended time to wake up, and proceeds to step S205 (alarm activated). If the acceleration is less than or equal to the acceleration threshold (step S210; No), the control unit 110 determines whether the judgment period has ended or not (step S212). If the judgment period has not ended (step S212; No), it returns to step S206. If the judgment period has ended (step S212; Yes), the set alarm time has arrived, so the control unit 110 proceeds to step S205 (alarm activated).

[0034] On the other hand, if in step S209 the minimum value of the heart rate is outside the range of a predetermined reference value (greater than the abnormality determination threshold) (step S209; Yes), the control unit 110 determines whether the minimum value of the heart rate obtained in step S208 is greater than the heart rate threshold set in the analysis process (step S211). If the minimum heart rate is below the heart rate threshold (step S211; No), the system proceeds to step S212. On the other hand, if the minimum heart rate is greater than the heart rate threshold (step S211; Yes), the control unit 110 determines that the current time is the recommended wake-up time, which is a period of light sleep, and proceeds to step S205 (alarm activated). The determination condition in step S209 is one of the conditions for determining whether the heart rate was correctly acquired during the determination period, and is also called the second condition.

[0035] Through the analysis and determination processes described above, the control unit 110 of the electronic device 100 acquires multiple data points within the first period, which are the user's biosignal data, during the first period. It generates a first correction data set by performing a predetermined filter process on the acquired data points within the first period. It generates a standard deviation data set by performing a predetermined statistical process on the data points within the first period. Based on the generated first correction data set and the standard deviation data set, it derives a threshold value. Based on the data points within the second period acquired in the second period after the first period, and the threshold value, it determines the user's sleep state. Therefore, the control unit 110 can determine that the timing when the user's sleep state is light is the recommended time to wake up. By activating the alarm at that timing, the user can wake up smoothly.

[0036] As explained above, the control unit 110 can remove two types of outliers that occur in the heart rate data (sudden outliers and consecutive outliers). Specifically, sudden outliers can be removed by a Min filter, and consecutive outliers can be removed using the standard deviation (for example, by using the average heart rate and standard deviation corresponding to the time when the standard deviation is below the 25th percentile). Therefore, the control unit 110 can determine the user's sleep state more appropriately.

[0037] Furthermore, if the control unit 110 determines that a normal heart rate cannot be obtained, it will use acceleration instead of heart rate to determine the user's sleep state, allowing for a more accurate determination of the user's sleep state compared to using only heart rate. For example, if the strap of the electronic device 100 (e.g., a smartwatch) loosens while the user is sleeping, the heart rate may not be detected properly. However, acceleration can still be detected without problems in such cases, so by using acceleration data instead of heart rate data, the sleep state can be determined more accurately.

[0038] Furthermore, the control unit 110 calculates multiple standard deviations of heart rate data within the second window period by sliding the second window over time, extracts a predetermined percentage of these multiple standard deviations from the smallest value (for example, those below the 25th percentile), and derives the heart rate threshold using the data corresponding to the extracted standard deviations from the data included in the first corrected data group (multiple heart rate data after applying the Min filter). Thus, the heart rate threshold can be derived using data from which outliers of consecutively occurring heart rates have been removed.

[0039] In the determination process according to the first embodiment described above (Figure 13), the possibility that the heart rate falls outside the range of a predetermined reference value is considered in step S209. However, in reality, the possibility of the heart rate falling outside the range of a predetermined reference value is very small, so the analysis process (Figure 12) does not take this possibility into consideration. However, as Modification 1, processing that takes this possibility into consideration may be performed. In Modification 1, if the heart rate calculated in step S103 is outside the range of a predetermined reference value (below the abnormality detection threshold), the control unit 110 ignores that heart rate as an outlier. By doing so, in Modification 1, even if an outlier (abnormally small) heart rate outside the range of a predetermined reference value occurs suddenly during the analysis period, it is possible to set a heart rate threshold that minimizes the influence of that outlier.

[0040] In the analysis process (Figure 12) and determination process (Figure 13) according to the first embodiment described above, a Min filter was used to remove outliers where the heart rate suddenly becomes a large value. However, depending on the algorithm used to calculate the heart rate from the pulse wave, outliers where the heart rate suddenly becomes a large value may rarely occur, while conversely, outliers where the heart rate becomes a small value may occur relatively often. When such an algorithm is adopted, as a modification 2, it is desirable to use a Max filter instead of a Min filter, which replaces the heart rate with the maximum value within the most recent predetermined period (window). In this modified example 2, step S209 of the judgment process (Figure 13) determines whether the maximum heart rate is within a predetermined reference range (whether it is less than the abnormality judgment threshold (e.g., 200)). By using the Max filter, in modified example 2, even if a sudden outlier of a small heart rate occurs, the heart rate threshold can be set and the recommended wake-up time determined in a way that minimizes the influence of that outlier.

[0041] Depending on the algorithm used to calculate heart rate from pulse waves, the heart rate may sometimes be unexpectedly large or small. When using such an algorithm, it is preferable to use a Median filter, which replaces the heart rate with the median value within a given period (window), instead of a Min filter or Max filter, as shown in Modification 3. In this modified example 3, in step S209 of the judgment process (Figure 13), two types of abnormality judgment thresholds (a first abnormality judgment threshold and a second abnormality judgment threshold) are prepared, and it is determined whether the heart rate falls between the first abnormality judgment threshold (e.g., 10) and the second abnormality judgment threshold (e.g., 200). By using a median filter, in modified example 3, even if there are sudden outliers of small or large values ​​in the heart rate, it is possible to set the heart rate threshold and determine the recommended wake-up time while minimizing the influence of these outliers.

[0042] In the first embodiment and its modifications described above, the sleep state was determined using constant threshold values ​​(heart rate threshold HR_TH and acceleration threshold ACC_TH) derived from equations (1) and (2). If the thresholds are kept constant in this way, for example, if the heart rate threshold derived from equation (1) is 50, and the heart rate during the determination period (heart rate after applying the Min filter) changes as shown in Figure 14, the alarm will be activated simultaneously with the start of the determination period (at 4:45). While it can be considered unavoidable since the heart rate value indicates that the sleep state is light at 4:45, the user will be woken up considerably earlier than the originally set alarm time (at 5:45), which may cause the user to feel unwell or uncomfortable. This was also confirmed in subjective tests conducted by the inventor on multiple subjects using the first embodiment described above. In the modified example 4, as shown by the dashed line 401 in Figure 14, the initial threshold for the judgment period is set to a value greater than the value derived by equation (1) (50.0) (60.0), and the threshold is linearly decreased so that it becomes the value derived by equation (1) (50.0) at the end of the judgment period. Specifically, for example, the initial heart rate threshold HR_TH_INIT is derived by equation (3) below, and the heart rate threshold HR_TH(t) is derived by equation (4). HR_TH_INIT=(Dn_ave+h×Dn_std)×ch …(3) HR_TH(t)=HR_TH_INIT-(HR_TH_INIT -(Dn_ave+h×Dn_std))×t / x …(4) however, Dn_ave = DnAverage value of all data Dn_std=DnStandard deviation of all data h = adjustment coefficient (in this modified example 4, it is "2") ch = Initial value adjustment coefficient (in this modified example 4, it is "1.2") x = length of the evaluation period (minutes) t = elapsed time (minutes) from the start of the judgment period. By setting a larger initial threshold value and decreasing it over time, the alarm is triggered around 5:36 in the example shown in Figure 14. In subjective tests conducted by the inventor with multiple subjects, it was confirmed that the 4th modified example (waking up around 5:36) resulted in a more pleasant awakening than the first embodiment described above (waking up at 4:45). Note that the heart rate data after applying the Min filter in Figure 14 is for the purpose of aiding the understanding of the present invention and does not represent data from an actual alarm being triggered around 5:36, and the same applies to Figures 15, 16, and 17 described later.

[0043] This concept can also be applied to acceleration thresholds. That is, the initial threshold at the start of the judgment period is set to a value greater than the value derived by equation (2), and the threshold is linearly decreased until it reaches the value derived by equation (2) at the end of the judgment period. Specifically, for example, the initial acceleration threshold AC_TH_INIT is derived using equation (5) below, and the acceleration threshold AC_TH(t) is derived using equation (6). AC_TH_INIT=(AC_ave+k×AC_std)×ca …(5) AC_TH(t)=AC_TH_INIT-(AC_TH_INIT -(AC_ave+k×AC_std))×t / x …(6) however, AC_ave = Average value of all acceleration data during the analysis period. AC_std = Standard deviation of all acceleration data during the analysis period k = adjustment coefficient (8 in this example 4) ca = Initial value adjustment coefficient (in this modified example 4, it is "1.8") x = length of the evaluation period (minutes) t = elapsed time (minutes) from the start of the judgment period.

[0044] Furthermore, in the above-mentioned modification 4, the threshold was linearly decreased over time within the judgment period. In this case, the threshold can be expressed as a linear function of time, which has the advantage of simplifying the formula for deriving the threshold. However, the function that determines the threshold is not limited to a linearly decreasing function. The threshold can be set using any function that is monotonically decreasing in the broad sense, including nonlinear functions. Examples of nonlinear functions include monotonically decreasing step functions, monotonically decreasing functions that are convex upwards, and monotonically decreasing functions that are convex downwards (e.g., the beta function), but the function that determines the threshold is not limited to these. For example, as modification 5, the threshold can be decreased stepwise using a step function, as shown in Figure 15. This function divides the judgment period into multiple intervals, with the threshold remaining constant within each interval and gradually decreasing when crossing intervals. The length of the intervals and the amount of decrease when crossing intervals do not have to be constant. In the example shown in Figure 15, the alarm is triggered around 5:36. In this case, for example, there is an advantage in that the threshold can be easily derived by storing the intervals where the threshold is constant and the threshold value within those intervals in the storage unit 120. Furthermore, as variation 6, as shown in Figure 16, the threshold can be decreased by a broadly monotonically decreasing, downward-convex function. In this example as well, the alarm will be activated around 5:36. For example, if the judgment period is relatively long, using variation 6 has the advantage that the alarm will be less likely to be activated in the first half of the judgment period, making it easier for the user to get enough sleep. Furthermore, as variation 7, as shown in Figure 17, the threshold can be decreased by a broadly monotonically decreasing, downward-convex function. In this example, the alarm will be triggered around 5:29. For example, if the judgment period is relatively short, using variation 7 makes it easier for the alarm to be triggered within the judgment period, thus reducing the likelihood of the alarm being forcibly triggered at the end of the judgment period, which has the advantage of waking the user more easily. Furthermore, it is possible to combine these variations. For example, by linearly decreasing the threshold until partway through the judgment period and then setting it to a constant value (a combination of Variation 4 and Variation 5), an effect similar to Variation 7 can be achieved with a simpler calculation formula.

[0045] In modified examples 4 to 7, it is necessary to set the threshold value to change over time (at least once) during the determination period. Therefore, the control unit 110 executes the second determination process shown in Figure 18, rather than the determination process shown in Figure 13. The second determination process (Figure 18) is simply the determination process (Figure 13) with steps S200, S213, and S214 added. The processes in these steps will be explained below. First, when the control unit 110 starts the second determination process, it derives an initial threshold and sets the threshold to that initial threshold (step S200). For example, the initial threshold for heart rate HR_TH_INIT can be derived using equation (3) above, and the initial threshold for acceleration AC_TH_INIT can be derived using equation (5) above. After step S200, the process proceeds to step S201. Furthermore, if the determination in step S204 is No, the control unit 110 updates the value of the acceleration threshold according to the elapsed time t in the determination period (step S213) before returning to step S202, and then returns to step S202. The updated threshold value can be derived, for example, by equation (6) described above. Furthermore, if the determination in step S212 is No, the control unit 110 updates the values ​​of the heart rate threshold and the acceleration threshold according to the elapsed time t in the determination period (step S214) before returning to step S206, and then returns to step S206. The updated heart rate threshold value can be derived, for example, by equation (4) above, and the updated acceleration threshold value can be derived, for example, by equation (6) above. Other processes are the same as those in the judgment process described above (Figure 13), so their explanation will be omitted. As described above, the second judgment process sets the threshold to a value greater than the threshold in the first embodiment at the start of the judgment period, and sets the threshold to change in a broadly monotonically decreasing manner so that it becomes the threshold in the first embodiment at the end of the judgment period. This makes it less likely for the alarm to sound near the start of the judgment period, thus reducing the probability of being woken up considerably earlier than the set alarm time (end of the judgment period), and providing the user with a more pleasant awakening. In other words, it becomes possible to balance the user's desire to "wake up well" and their desire to "sleep for a long time" as much as possible.

[0046] Furthermore, in the first embodiment and its modifications described above, the electronic device 100 alone acquired sensing data such as heart rate and acceleration as time-series vital signals, and derived thresholds based on the mean value and standard deviation of the sensing data. However, these processes may be distributed and executed by multiple devices. For example, in a second embodiment, as shown in Figure 19, an information processing system 2000 comprising a wrist device 210 (electronic device), a smartphone 220, and a server 230 (information processing device) may perform the same processing as the electronic device 100 in the first embodiment. In the second embodiment, for example, the wrist device 210 and the smartphone 220 communicate using a low-power short-range wireless communication standard such as Bluetooth®, while the smartphone 220 and the server 230 communicate using a wireless communication standard capable of relatively long-range communication, such as Wi-Fi (Local Area Network) or LTE (Long Term Evolution). For example, the wrist device 210 acquires time-series sensing data (vital signals during the analysis period and vital signals during the judgment period) and transmits the acquired vital signals during the analysis period (including acquisition time) and time information for the judgment period (start time and end time (or length of the judgment period)) to the smartphone 220. The smartphone 220 then analyzes the vital signals transmitted by the wrist device 210 and transmits the analysis results (for example, the mean and standard deviation of vital signals during the analysis period) and time information for the judgment period to the server 230. The server 230 then derives a threshold for the vital signals based on the analysis results transmitted by the smartphone 220 and transmits the derived threshold to the smartphone 220 according to the time during the judgment period (however, it is desirable to consider the time required for communication). The smartphone 220 then receives the threshold transmitted by the server 230 and transmits it to the wrist device 210. In this way, the wrist device 210 can receive the threshold transmitted by the smartphone 220 and determine the user's sleep state (i.e., whether or not to activate the alarm) based on the received threshold and vital signals during the determination period.

[0047] By performing distributed processing in this manner, the processing load on the list device 210 can be reduced, and the server 230 can perform more complex processing to derive thresholds. Furthermore, various modifications are conceivable in the second embodiment as well. For example, the smartphone 220 may transmit the vital signals received from the wrist device 210 directly to the server 230 without analyzing the vital signals, and the server 230 may perform vital signal analysis, threshold derivation, etc. Alternatively, the smartphone 220 may perform the function of the server 230 without a server 230, or conversely, the server 230 and the wrist device 210 may communicate directly without a smartphone 220. For example, one possible modification is that the wrist device 210, as an electronic device, transmits the vital signals (first data) acquired during the first period (analysis period) to the server 230, as an information processing device, and during the second period (determination period), the server 230 transmits the first threshold derived from the first data to the wrist device 210 while changing it in a broadly monotonically decreasing manner. In variation 8, the server 230 sets a first threshold based on the first data, and sets the first threshold to be changed at least once in a broadly monotonically decreasing manner as time progresses during the second period. The server then transmits the first threshold set in this manner to the list device 210, and the list device 210 determines the user's sleep state based on the vital signals (second data) acquired during the second period and the first threshold received from the server 230 during the second period. In the first embodiment described above, the electronic device 100 derived the first threshold based only on vital signals (first data) during the analysis period (first period) (i.e., data only for that day). However, in the modified example 8, for example, the server 230 can also accumulate vital signals during the analysis period and judgment period over a long period in the past (for example, each day of the most recent month). Therefore, it becomes possible to derive a more accurate threshold based on this accumulated large amount of data (for example, using deep learning). Furthermore, users could provide feedback on how easily they woke up each day using their alarms, and this feedback data could be used to derive a more accurate threshold.

[0048] Furthermore, as yet another variation of the second embodiment (variation 9), for example, during the analysis period (first period), the vital signals acquired by the wrist device 210 are transmitted to the smartphone 220 or server 230, the smartphone 220 or server 230 analyzes the vital signals and derives a threshold value for the vital signals based on the analysis results. Then, during the determination period (second period), the vital signals acquired by the wrist device 210 are transmitted to the smartphone 220 or server 230, the smartphone 220 or server 230 determines the sleep state based on the received vital signals and the threshold value derived from the vital signals during the analysis period, and transmits the determination result (i.e., whether or not to activate the alarm) to the wrist device 210. In modification 9, the wrist device 210 only needs to acquire vital signals and activate alarms, while other processing can be left to the smartphone 220 or server 230, thus further reducing the processing load on the wrist device 210.

[0049] It should be noted that the present invention is not limited to the embodiments described above, and various modifications and applications are possible. For example, in the embodiments and modifications described above, the sensor unit 130 included a pulse wave sensor for detecting pulse waves and an acceleration sensor for detecting body movement. However, the types of sensors included in the sensor unit 130 are not limited to these. The sensor unit 130 can include any sensor that detects (acquires) biosignals used to determine the recommended wake-up time. Furthermore, if the electronic device 100 can receive biosignals or feature quantities for estimating sleep states from an external device via the communication unit 160, for example, the electronic device 100 does not need to include the sensor unit 130, and may acquire biosignals, etc., via the communication unit 160.

[0050] Furthermore, the electronic device 100 is not limited to wearable devices such as smartwatches. The electronic device 100 may also be an information device such as a smartphone that acquires the user's heart rate and acceleration from other wearable devices (such as smartwatches) via wireless communication. In this case, the information device as the electronic device 100 may perform the analysis and judgment processing, the information device itself may perform the alarm function, or the alarm information may be transmitted to a wearable device and the wearable device may output an alarm sound.

[0051] Furthermore, in the embodiments and modifications described above, the electronic device 100 uses heart rate (first data) and acceleration (second data) as biosignal data used to determine the user's sleep state. However, the biosignals used for determination are not limited to these. Biosignals that the electronic device 100 can use include body movement (detected by acceleration sensors installed on the head, arms, chest, legs, torso, etc.), electromyography (detected by electromyography sensors installed on the head (temples and around the nose), arms, chest, legs, torso, etc.), sweat (detected by skin potential meters and humidity sensors), heart rate (detected by electrocardiographs, pressure sensors installed under the bed (detection of electrocardiogram waveforms), pulse wave sensors installed on the head, arms, chest, legs, torso, etc.), etc.

[0052] In the embodiments and modifications described above, the electronic device 100 was equipped with an input unit 140, an output unit 150, and a communication unit 160. However, these are not essential components, and the electronic device 100 does not need to be equipped with them.

[0053] Furthermore, each function of the electronic device 100 can also be performed by a computer such as a regular PC (Personal Computer). Specifically, in the above embodiment, it was described that the program for analysis processing and other operations performed by the electronic device 100 is pre-stored in the ROM of the storage unit 120. However, a computer capable of realizing the above functions may be configured by distributing the program on a computer-readable recording medium such as a flexible disk, CD-ROM (Compact Disc Read Only Memory), DVD (Digital Versatile Disc), MO (Magneto-Optical Disc), memory card, or USB (Universal Serial Bus) memory, and then loading and installing the program into the computer.

[0054] Furthermore, when the above-mentioned processing is performed through a division of labor between the OS (Operating System) and the application program, or through collaboration between the OS and the application program, only the application program may be stored on a recording medium or storage device. It is also possible to superimpose the program onto a carrier wave and distribute it over a network. For example, the above program may be posted on a bulletin board system (BBS) on a network and distributed over the network. This program can then be launched and executed under the control of the OS, just like other application programs, to perform the above-mentioned processing.

[0055] Furthermore, the control unit 110 may consist of any single processor, such as a single processor, multi-processor, or multi-core processor, or it may be configured by combining any of these processors with processing circuits such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field-Programmable Gate Array).

[0056] Although preferred embodiments of the present invention have been described above, the present invention is not limited to these specific embodiments, and the present invention includes the invention described in the claims and its equivalents. [Explanation of Symbols]

[0057] 100...Electronic equipment, 110...Control unit, 120...Storage unit, 130...Sensor unit, 140...Input unit, 150...Output unit, 160...Communication unit

Claims

1. Multiple data points are acquired during the first period. Based on the acquired first data, a first threshold is set. Based on the judgment data acquired in the second period following the first period and the first threshold, the user's sleep state is determined. The first threshold is set to be changed at least once during the second period. Equipped with a control unit, electronic equipment.

2. The control unit, The first threshold is set to change in a broadly monotonically decreasing manner during the second period. The electronic device according to claim 1.

3. The control unit, The first threshold is set to decrease linearly during the second period. The electronic device according to claim 2.

4. The control unit, The first threshold is set to decrease stepwise during the second period. The electronic device according to claim 2.

5. The control unit, The first threshold is set to decrease as an upwardly convex function during the second period. The electronic device according to claim 2.

6. The control unit, The first threshold is set to decrease as a downward-convex function during the second period. The electronic device according to claim 2.

7. The control unit, A first set of corrected data is generated by performing a predetermined filtering process on the acquired first data, A set of standard deviation data is generated by performing a predetermined statistical process on multiple of the aforementioned first data. The first threshold is set based on the generated first correction data set and the standard deviation data set. The electronic device according to claim 1.

8. The control unit, Multiple sets of second data, different from the first data, were also acquired during the first period. Based on the acquired second data, a second threshold is set. The user's sleep state is determined using the aforementioned determination data and at least one of the first threshold or the second threshold. The threshold of at least one of the first threshold or the second threshold used in the determination is set to be changed at least once during the second period. The electronic device according to claim 1.

9. The first data mentioned above is heart rate data, The second data mentioned above is acceleration data. The electronic device according to claim 8.

10. An information processing system comprising an electronic device worn on the human body and an information processing device capable of communicating with the electronic device, The aforementioned electronic device is Multiple data points are acquired during the first period. The acquired multiple first data are transmitted to the information processing device. The first threshold is received from the information processing device. The user's sleep state is determined based on the determination data acquired in the second period following the first period and the received first threshold. The aforementioned information processing device is The electronic device receives a plurality of first data transmitted during the first period, Based on the multiple first data received, the first threshold is set to be changed at least once during the second period. The set first threshold is transmitted to the electronic device. Information processing system.

11. The control unit, Multiple data points are acquired during the first period. Based on the acquired first data, a first threshold is set. Based on the judgment data acquired in the second period following the first period and the first threshold, the user's sleep state is determined. The first threshold is set to be changed at least once during the second period. Methods for determining sleep state.

12. On the computer, Multiple data points are acquired during the first period. Based on the acquired first data, a first threshold is set. Based on the judgment data acquired in the second period following the first period and the first threshold, the user's sleep state is determined. The first threshold is set to be changed at least once during the second period. A program to execute a process.

Images