Method and system for marking sleep states

Abstract

The present disclosure relates to methods and systems for labeling sleep states. A system, computer-readable storage medium, and method capable of estimating a user's sleep state directly or indirectly based on sensor data from mobile sensors and / or optical sensors.

Description

[0001] Divisional Statement

[0002] This application is a divisional application of Chinese Patent Application No. 201780063092.X, filed September 6, 2017, which is based on and claims priority to U.S. Provisional Patent Application No. 62 / 384,188, filed September 6, 2016, entitled "METHODS AND SYSTEMS FOR LABELING SLEEP STATES," and U.S. Patent Application No. 15 / 438,643, filed February 21, 2017, entitled "METHODS AND SYSTEMS FOR LABELING SLEEP STATES," which are hereby incorporated by reference in their entirety.

[0003] Cross Reference to Related Applications

[0004] This application claims the benefit of priority under 35 U.S.C. § 119(e) to U.S. Provisional Patent Application No. 62 / 384,188, filed September 6, 2016, entitled "METHODS AND SYSTEMS FOR LABELING SLEEP STATES," and U.S. Patent Application No. 15 / 438,643, filed February 21, 2017, entitled "METHODS AND SYSTEMS FOR LABELING SLEEP STATES," which are hereby incorporated by reference in their entirety. TECHNICAL FIELD

[0005] The present disclosure relates to wearable devices. In particular, the example devices described herein are capable of estimating a sleep state of a user based on sensor data from a movement sensor and / or an optical sensor, either directly or indirectly. BACKGROUND

[0006] In traditional sleep stage scoring, an expert analyzes readings of a sleeper's brain activity. This process is a highly manual process that involves the expertise of the scorer. As a result, sleep stage scores can differ from scorer to scorer and from sleeper to sleeper due to errors or differences in the method. Furthermore, such sleep stage scoring requires specialized equipment, such as an electroencephalogram (EEG) system, which makes it difficult to perform outside of a laboratory setting. SUMMARY

[0007] The details of one or more implementations of the subject matter described in this specification are set forth in the accompanying drawings and the following description. Other features, aspects, and advantages will become apparent from the description, the drawings, and the claims.

[0008] In some implementations, a sleep monitoring system is provided that includes a wearable electronic device to be worn by a user, the wearable electronic device including: a set of one or more motion sensors to generate motion data representative of motion of the wearable electronic device over a first time window; and a set of one or more optical sensors to generate cardiopulmonary pulse-related data detected by the wearable electronic device over the first time window. Such implementations can also include: a set of one or more processors configured to receive data from the set of motion sensors and the set of one or more optical sensors; and a non-transitory machine-readable storage medium operably coupled to the set of one or more processors and having stored therein instructions that, when executed, cause the set of one or more processors to: extract movement features from the motion data covering the first time window; extract pulse data features from the cardiopulmonary pulse-related data covering the first time window; and use the movement features and the pulse data features to cause a classifier to label a time period associated with the first time window with an identifier indicative of a first sleep stage selected from a plurality of sleep stages.

[0009] In some additional implementations, the plurality of sleep stages can include two or more sleep stages drawn from a sleep stage such as a wake sleep stage, an artifact / off-wrist sleep stage, a light sleep stage, a deep sleep stage, and a random eye movement (REM) sleep stage.

[0010] In some additional or alternative implementations, the set of one or more optical sensors can include a photoplethysmogram sensor, the set of one or more motion sensors can include an accelerometer; or the set of one or more optical sensors can include a photoplethysmogram sensor and the set of one or more motion sensors can include an accelerometer.

[0011] In some additional or alternative implementations, the first time window and the time period can be the same duration and have the same start and end points.

[0012] In some additional or alternative implementations, the movement features and the pulse data features together can include one or more features such as: a cumulative movement index extracted from the motion data; a first elapsed time between a first time that is within a first time window and a second time that is before the first time window at which the cumulative movement index last exceeded a first threshold amount; a second elapsed time between a third time that is within the first time window and a fourth time that is after the first time window at which the cumulative movement index first exceeded a second threshold amount; a third elapsed time between a fifth time that is within the first time window and a most recent time that is outside the first time window at which the cumulative movement index first exceeded a third threshold amount; a first number of time windows since the cumulative movement index last exceeded a fourth threshold amount before the first time window; a second number of time windows after the first time window until the cumulative movement index first exceeded a fifth threshold amount; a variability of inter-beat intervals in the cardiopulmonary pulse-related data assessed by sample entropy; a variability of the cardiopulmonary pulse-related data assessed by sample entropy; a root mean square of successive differences of inter-beat intervals; a root mean square of successive differences of the cardiopulmonary pulse-related data; a low frequency spectral power of inter-beat intervals in a frequency range of 0.04 Hz to 0.15 Hz; a low frequency spectral power of the cardiopulmonary pulse-related data in a frequency range of 0.04 Hz to 0.15 Hz; a high frequency spectral power of inter-beat intervals in a frequency range of 0.15 Hz to 0.4 Hz; a high frequency spectral power of the cardiopulmonary pulse-related data in a frequency range of 0.15 Hz to 0.4 Hz; a variability of an envelope of the cardiopulmonary pulse-related data; a variability of an envelope of inter-beat intervals; a variability of a detrended respiratory rate extracted from the cardiopulmonary pulse-related data; an inter-percentile span of a heart rate extracted from the cardiopulmonary pulse-related data or from inter-beat intervals; a normalized detrended heart rate extracted from the cardiopulmonary pulse-related data or from inter-beat intervals; and a cross-correlation of each of one or more pulse shapes in the cardiopulmonary pulse-related data with a preceding pulse shape in the cardiopulmonary pulse-related data (wherein the pulse shapes are normalized to a common duration prior to the cross-correlation).

[0013] In some additional or alternative implementations, the movement features and the pulse data features together can include at least one of: a time since last movement, a time until next movement, a time to most recent movement, a variability of inter-beat intervals extracted from the cardiopulmonary pulse-related data assessed using sample entropy, a variability of a detrended respiratory rate extracted from the cardiopulmonary pulse-related data, and a cross-correlation of each of one or more pulse shapes in the cardiopulmonary pulse-related data with a preceding pulse shape in the cardiopulmonary pulse-related data (wherein the pulse shapes are normalized to a common duration prior to the cross-correlation).

[0014] In some additional or alternative implementations, the instructions that cause the classifier to label the time period can include instructions that, when executed, cause the set of one or more processors to transmit the movement features and the pulse data features to a server system that executes the classifier, and at least some of the one or more processors of the set of one or more processors can be part of the server system.

[0015] In some additional or alternative implementations, the instructions that cause the classifier to label the time period can include instructions that, when executed, cause the set of one or more processors to execute a classifier such as, for example, a nearest neighbor classifier, a random forest classifier, and a linear discriminant classifier.

[0016] In some additional or alternative implementations, the classifier can be trained using movement features and pulse data features extracted from benchmark movement data and benchmark cardiopulmonary pulse-related data collected for a population of sleep study subjects.

[0017] In some additional or alternative implementations, the instructions can further cause the set of one or more processors to impute missing data points in the cardiopulmonary pulse-related data before the one or more processors are caused to generate the pulse data features.

[0018] In some additional or alternative implementations, the instructions can further cause the set of one or more processors to change a label of the time period associated with the first time window based on labels of a plurality of consecutive time periods, the plurality of consecutive time periods including the time period associated with the first time window, to satisfy the pattern constraint.

[0019] In some additional implementations, the pattern constraint can be satisfied when a time period is labeled with an indicator that indicates a wakeful sleep stage and an adjacent time period to the time period is labeled with an indicator that indicates a deep sleep stage. In such implementations, the label for the time period can be changed from the indicator that indicates the wakeful sleep stage to the indicator that indicates the deep sleep stage.

[0020] In some additional or alternative implementations, the instructions can further cause the set of one or more processors to: obtain a confidence number associated with the time period, each confidence number for a different sleep stage of the plurality of sleep stages; and label the time period with an identifier that indicates a sleep stage of the plurality of sleep stages that has a highest confidence number for the time period.

[0021] In some implementations, a method can be provided that includes receiving cardiopulmonary pulse-related data obtained from a set of one or more optical sensors in a wearable electronic device over a first time window; receiving motion data obtained from a set of one or more motion sensors in the wearable electronic device over the first time window, the motion data including at least one of a quantification of movement experienced by the wearable electronic device or data derived therefrom; and using the motion data and the cardiopulmonary pulse-related data to label a time period associated with the first time window with an indicator indicating a first sleep stage selected from a plurality of sleep stages.

[0022] In some additional implementations, the method can further include extracting movement features from the motion data covering the first time window; extracting pulse data features from the cardiopulmonary pulse-related data covering the first time window; and using the movement features and the pulse data features to cause a classifier to select the first sleep stage from the plurality of sleep stages.

[0023] In some additional or alternative implementations, the classifier can be a nearest neighbor classifier, a random forest classifier, or a linear discriminant classifier.

[0024] In some additional or alternative implementations, the method can further include padding missing data points in the cardiopulmonary pulse-related data prior to extracting the pulse data features.

[0025] In some additional or alternative implementations, the method can further include training the classifier using movement features and pulse data features extracted from benchmark motion data and benchmark cardiopulmonary pulse-related data collected for a population of sleep study subjects.

[0026] In some additional or alternative implementations, the movement features and the pulse data features, collectively, can include one or more features such as: a cumulative movement index extracted from the motion data; a first elapsed time between a first time that is within the first time window and a second time that is before the first time window at which the cumulative movement index last exceeded a first threshold amount; a second elapsed time between a third time that is within the first time window and a fourth time that is after the first time window at which the cumulative movement index first exceeded a second threshold amount; a third elapsed time between a fifth time that is within the first time window and a most recent time that is outside of the first time window at which the cumulative movement index first exceeded a third threshold amount; a first number of time windows since the cumulative movement index last exceeded a fourth threshold amount before the first time window; a second number of time windows after the first time window until the cumulative movement index first exceeded a fifth threshold amount; a variability of interbeat intervals in the cardiopulmonary pulse-related data assessed by sample entropy; a variability of the cardiopulmonary pulse-related data assessed by sample entropy; a root mean square deviation of interbeat intervals; a root mean square deviation of the cardiopulmonary pulse-related data; a low frequency spectral power of interbeat intervals; a high frequency spectral power of interbeat intervals; a low frequency spectral power of the cardiopulmonary pulse-related data; a high frequency spectral power of the cardiopulmonary pulse-related data; a variability of an envelope of the cardiopulmonary pulse-related data; a variability of an envelope of interbeat intervals; a variability of a detrended respiratory rate extracted from the cardiopulmonary pulse-related data; a percentiles span of a heart rate extracted from the cardiopulmonary pulse-related data or from interbeat intervals; a normalized detrended heart rate extracted from the cardiopulmonary pulse-related data or from interbeat intervals; and a cross-correlation of each of one or more pulse shapes in the cardiopulmonary pulse-related data with a preceding pulse shape in the cardiopulmonary pulse-related data (wherein the pulse shapes can be normalized to a common duration prior to the cross-correlation).

[0027] In some additional or alternative implementations, the first time window and the time period can be the same duration and have the same start and end points.

[0028] In some additional or alternative implementations, the plurality of sleep stages can include two or more sleep stages drawn from sleep stages such as a wake sleep stage, an artifact / loss of armband sleep stage, a light sleep stage, a deep sleep stage, and a rapid eye movement (REM) sleep stage.

[0029] In some additional or alternative implementations, the time period can be one of a plurality of time periods, each time period marked with an indicator indicating a corresponding sleep stage selected from a plurality of sleep stages, and the method can further include: detecting that the first sleep stage causes the sleep stage corresponding to each time period of the plurality of time periods to match a sleep stage rule, the sleep stage rule indicating a series of sleep stages that are invalid; and in response to detecting that the first sleep stage causes the sleep stage corresponding to each time period of the plurality of time periods to match the sleep stage rule, replacing the first sleep stage with a second sleep stage in accordance with an update rule.

[0030] In some additional or alternative implementations, the update rule can be executed in accordance with a majority rule analysis of the plurality of sleep stages to select the second sleep stage.

[0031] In some additional or alternative implementations, the marked time period can be part of a plurality of marked time periods, and the method can further include generating an estimate of a duration of sleep time for each type of label used in the plurality of marked time periods.

[0032] In some implementations, an apparatus can be provided that includes a communication interface, one or more processors, and a memory. The one or more processors can be communicatively connected with the memory and the communication interface, and the memory can have stored therein computer-executable instructions that, when executed, cause the one or more processors to: receive sleep stage data for a first user, the sleep stage data including data indicating time intervals associated with a first sleep session of the first user and data indicating, for each time interval, a sleep stage associated with the time interval, wherein the sleep stage associated with each time interval is selected from a predetermined set of different sleep stages; and generate a first graphical user interface component indicating a relative percentile breakdown of total time spent in each sleep stage of the first sleep session.

[0033] In some additional implementations, the predetermined set of different sleep stages can include one or more sleep stages drawn from a sleep stage such as pseudowake or dozing sleep stage, wake sleep stage, random eye movement (REM) sleep stage, light sleep stage, or deep sleep stage.

[0034] In some additional or alternative implementations, the memory can also have computer-executable instructions stored therein that, when executed, also cause the one or more processors to: receive representative personal sleep stage data indicating a representative relative percentile breakdown of total time spent in each sleep stage for a plurality of sleep sessions of a first user; and modify the first graphical user interface component to indicate the representative relative percentile breakdown of total time spent in each sleep stage for the plurality of sleep sessions of the first user in addition to the relative percentile breakdown of total time spent in each sleep stage for the first sleep session.

[0035] In some additional or alternative implementations, the representative personal sleep stage data indicating a representative relative percentile breakdown of total time spent in each sleep stage for a plurality of sleep sessions of a first user can be a relative percentile breakdown of an average of total time spent in each sleep stage for the plurality of sleep sessions of the first user.

[0036] In some additional or alternative implementations, the plurality of sleep sessions of the first user can include a plurality of sleep sessions across at least one of the following time intervals, such as, for example, a week prior to a time of generation of the first graphical user interface component, four weeks prior to the time of generation of the first graphical user interface component, 15 days prior to the time of generation of the first graphical user interface component, or 30 days prior to generation of the first graphical user interface component.

[0037] In some additional or alternative implementations, the memory can also have computer-executable instructions stored therein that, when executed, also cause the one or more processors to: receive representative peer group sleep stage data indicating a representative peer group relative percentile of total time spent in each sleep stage for a plurality of sleep sessions of a plurality of users; and modify the first graphical user interface component to indicate the representative peer group relative percentile breakdown of total time spent in each sleep stage for the plurality of sleep sessions of the plurality of users in addition to the relative percentile breakdown of total time spent in each sleep stage for the first sleep session.

[0038] In some additional or alternative implementations, the representative peer group relative percentile breakdown of total time spent in each sleep stage for a plurality of sleep sessions of a plurality of users can include a range of percentiles associated with each sleep stage.

[0039] In some additional or alternative implementations, the plurality of users can be users within a first age range, and can be of a same gender as the first user, and the first user can be within the first age range.

[0040] In some implementations, an apparatus can be provided that includes a communication interface, one or more processors, and a memory. The one or more processors can be communicatively connected with the memory and the communication interface, and the memory can have stored therein computer-executable instructions that, when executed, cause the one or more processors to: receive sleep stage data for a first user, the sleep stage data including data indicating time intervals associated with a first sleep session of the first user and data indicating, for each time interval, a sleep stage associated with the time interval, wherein the sleep stage associated with each time interval can be selected from a predetermined set of different sleep stages; generate a first graphical user interface component that includes a sleep graph displaying the time intervals; receive a first user input; and in response to receiving the first user input, modify the first graphical user interface to highlight time intervals associated with a first sleep stage of the predetermined set of different sleep stages and provide textual content that is customized based on the sleep stage data for the first user related to the first sleep stage.

[0041] In some additional implementations, the predetermined set of different sleep stages can include at least one of a pseudo or off-wrist sleep stage, a wake sleep stage, a random eye movement (REM) sleep stage, a light sleep stage, or a deep sleep stage.

[0042] In some additional or alternative implementations, the textual content that is customized based on the sleep stage data for the first user related to the first sleep stage can include text such as text indicating a number of intervals during the sleep session that the first user was in the first sleep stage and / or text indicating a percentile number based on an amount of total time spent in the first sleep stage during the sleep session relative to an amount of total time spent in all sleep stages of the predetermined set of different sleep stages during the sleep session.

[0043] In some additional or alternative implementations, the memory can also have stored therein computer-executable instructions that, when executed, cause the one or more processors to: receive a plurality of user inputs, the plurality of user inputs including the first user input; in response to receiving each user input, modify the first graphical user interface to highlight time intervals associated with a different sleep stage of the predetermined set of different sleep stages and provide textual content that is customized based on the sleep stage data for the first user related to the sleep stage.

[0044] In some additional or alternative implementations, the sleep map can include: a vertical axis having different elevations indicated for each sleep stage of a predetermined set of different sleep stages and a horizontal axis representing time; horizontal segments, each horizontal segment corresponding to a different time interval and being located at an elevation corresponding to a sleep stage of the corresponding time interval. In such implementations, the memory can also have computer-executable instructions stored therein that, when executed, cause the one or more processors to highlight the time intervals associated with the first sleep stage by marking a region between the horizontal axis and the horizontal segments located at elevations associated with the first sleep stage with a different graphical appearance than a region between the horizontal axis and the horizontal segments located at elevations associated with sleep stages of the predetermined set of different sleep stages that are different from the first sleep stage.

[0045] In some additional or alternative implementations, the memory can also have computer-executable instructions stored therein that, when executed, cause the one or more processors to generate a second graphical user interface component indicating a relative percentile breakdown of total time spent in each sleep stage of the first sleep session.

[0046] In some additional or alternative implementations, the predetermined set of different sleep stages can include one or more sleep stages, such as pseudo- or dozing sleep stages, wakeful sleep stages, random eye movement (REM) sleep stages, light sleep stages, or deep sleep stages.

[0047] In some additional or alternative implementations, the memory can also have computer-executable instructions stored therein that, when executed, further cause the one or more processors to: receive representative personal sleep stage data indicating a representative relative percentile breakdown of total time spent in each sleep stage for a plurality of sleep sessions of the first user; and modify the second graphical user interface component to indicate the representative relative percentile breakdown of total time spent in each sleep stage for the plurality of sleep sessions of the first user in addition to the relative percentile breakdown of total time spent in each sleep stage of the first sleep session.

[0048] In some additional or alternative implementations, the representative personal sleep stage data indicating a representative relative percentile breakdown of total time spent in each sleep stage for a plurality of sleep sessions of the first user can be a relative percentile breakdown of an average of total time spent in each sleep stage for the plurality of sleep sessions of the first user.

[0049] In some additional or alternative implementations, the plurality of sleep sessions of the first user can include a plurality of sleep sessions across at least one of the following time intervals, such as: a week prior to the time of generating the second graphical user interface component, four weeks prior to the time of generating the second graphical user interface component, 15 days prior to the time of generating the second graphical user interface component, or 30 days prior to the time of generating the second graphical user interface component.

[0050] In some additional or alternative implementations, the memory can also have stored therein computer-executable instructions that, when executed, cause the one or more processors to: receive representative peer group sleep stage data indicating representative peer group relative percentile breakdowns of total time spent in each sleep stage for a plurality of sleep sessions of a plurality of users; and modify the second graphical user interface component to indicate the representative peer group relative percentile breakdowns of total time spent in each sleep stage for the plurality of sleep sessions of the plurality of users in addition to the relative percentile breakdowns of total time spent in each sleep stage for the first sleep session.

[0051] In some additional or alternative implementations, the representative peer group relative percentile breakdowns of total time spent in each sleep stage for the plurality of sleep sessions of the plurality of users can include percentile ranges associated with each sleep stage.

[0052] In some additional or alternative implementations, the plurality of users can be users within a first age range, and can be of a same gender as the first user, and the first user can be within the first age range.

[0053] In some implementations, an apparatus can be provided that includes a communication interface, one or more processors, and a memory. In such implementations, the one or more processors can be communicatively connected with the memory and the communication interface, and the memory can have stored therein computer-executable instructions that, when executed, cause the one or more processors to: receive sleep stage data for a first user, the sleep stage data including data indicating time intervals associated with a first sleep session of the first user and data indicating, for each time interval, a sleep stage associated with the time interval (where the sleep stage associated with each time interval can be selected from a predetermined set of different sleep stages); identify a plurality of display time intervals, each display time interval being associated with one of a set of different sleep stages (where each display time interval of the plurality of display time intervals that is associated with a wakeful sleep stage of the set of different sleep stages can be coextensive with one different time interval that is associated with the wakeful sleep stage and has a duration that is greater than a first threshold amount, and each display time interval of the plurality of display time intervals that is associated with a sleep stage of the set of different sleep stages that is different from the wakeful sleep stage can also be coextensive with one different time interval that is associated with the sleep stage, or across one or more contiguous time intervals of the time intervals, each contiguous time interval of the one or more contiguous time intervals a) being associated with the sleep stage or b) being associated with the wakeful sleep stage and having a duration that is less than or equal to a threshold amount); and display a hypnogram based on the display time intervals and the different sleep stages.

[0054] In some additional implementations, the memory can also have stored therein computer-executable instructions that, when executed, further cause the one or more processors to: display the hypnogram by displaying a first graphical element representing each display time interval. In such implementations, each first graphical element can have: a dimension extending along a horizontal direction that is proportional to a duration of the display time interval represented by the first graphical element; a horizontal position based on a start time of the display time interval represented by the first graphical element; and a vertical position based on the sleep stage associated with the display time interval.

[0055] In some additional or alternative implementations, the memory can also have stored therein computer-executable instructions which, when executed, further cause the one or more processors to cause display of one or more second graphical elements on the sleep graph, each of the one or more second graphical elements representing a different one of the one or more time intervals associated with the wakeful sleep stage and having a duration less than or equal to the threshold amount.

[0056] In some additional or alternative implementations, the memory can also have stored therein computer-executable instructions which, when executed, further cause the one or more processors to cause display of the second graphical element on the sleep graph at the same elevation as the first graphical element representing the particular time interval associated with the wakeful sleep stage.

[0057] In some additional or alternative implementations, the memory can also have stored therein computer-executable instructions which, when executed, further cause the one or more processors to cause display of vertically extending lines on the sleep graph, each of the vertically extending lines spanning between one end of each of the second graphical elements and a corresponding one of the first graphical elements extending at the same horizontal position as the one end of the second graphical elements. In such implementations, the vertically extending lines can have a different format than the vertical lines used to connect the first graphical elements together.

[0058] In some implementations, an apparatus can be provided that includes a housing configured to be worn on a human body, a heart rate sensor located within the housing and configured to generate time-varying data of cardiovascular behavior, one or more processors, and a memory. In such implementations, the one or more processors can be communicatively connected with the heart rate sensor and the memory, and the memory can have stored therein computer-executable instructions which, when executed, cause the one or more processors to: a) obtain data from the heart rate sensor at a first sampling rate, b) extract a series of interbeat intervals from the data, c) determine a reference value associated with values of the interbeat intervals in the series of interbeat intervals, d) determine a difference between the reference value and values of each of the interbeat intervals in the series of interbeat intervals to produce corresponding adjusted interbeat interval values, and e) store interbeat interval information based on the adjusted interbeat interval values in the memory in association with the reference value.

[0059] In some additional implementations, the apparatus can further include a wireless communication interface, and the memory can have further computer-executable instructions stored therein that, when executed, further cause the one or more processors to: perform a) through e) for a plurality of time periods and cause the wireless communication interface to: transmit, for each of the plurality of time periods, interbeat interval information for the sequence of interbeat intervals for the time period, and transmit, for each of the plurality of time periods, a reference value for the sequence of interbeat intervals for the time period that is associated with the interbeat interval information for the time period.

[0060] In some additional or alternative implementations, the reference value can be within a range of minimum / maximum values of the interbeat intervals in the sequence of interbeat intervals. In some other additional or alternative implementations, the reference value can be an arithmetic mean of the interbeat intervals in the sequence of interbeat intervals, a median of the interbeat intervals in the sequence of interbeat intervals, a mode of the interbeat intervals in the sequence of interbeat intervals, or a measure of central tendency of the interbeat intervals in the sequence of interbeat intervals.

[0061] In some additional or alternative implementations, the memory can have further computer-executable instructions stored therein that, when executed, further cause the one or more processors to quantize the interbeat interval values prior to storing the interbeat interval information based on the adjusted interbeat interval values in the memory. In such implementations, the quantization can reduce a number of unique values in the interbeat interval information based on the adjusted interbeat interval values compared to a number of unique values of the interbeat intervals in the sequence of interbeat intervals extracted from the data.

[0062] In some additional or alternative implementations, the reference value can be an arithmetic mean, a median, or a mode, and the memory can have further computer-executable instructions stored therein that, when executed, further cause the one or more processors to quantize the adjusted interbeat interval values that are within a first threshold of the reference value according to a first quantization step size and quantize the adjusted interbeat interval values that are outside the first threshold of the reference value according to a second quantization step size that is greater than the first quantization step size.

[0063] In some additional or alternative implementations, the first threshold can be based on a predetermined fixed value. In some other additional or alternative implementations, the memory can have further computer-executable instructions stored therein that, when executed, further cause the one or more processors to determine the first threshold based on the interbeat intervals in the sequence of interbeat intervals and store the first threshold in the memory in association with the interbeat interval information.

[0064] In some additional or alternative implementations, the memory can have further computer-executable instructions stored therein that, when executed, further cause the one or more processors to quantize at least one of the interbeat interval values and the adjusted interbeat interval values prior to storing the interbeat interval information based on the adjusted interbeat interval values in the memory. In such implementations, the quantization can reduce a number of unique values in the interbeat interval information based on the adjusted interbeat interval values as compared to a number of unique values of the interbeat intervals in the sequence of interbeat intervals extracted from the data.

[0065] In some additional or alternative implementations, the reference value can be an arithmetic mean, a median, or a mode, and the memory can have further computer-executable instructions stored therein that, when executed, further cause the one or more processors to: quantize at least one of the interbeat interval values or the adjusted interbeat interval values according to a first quantization step size for the interbeat interval values or the adjusted interbeat interval values that are within a first threshold of the reference value, and quantize at least one of the interbeat interval values or the adjusted interbeat interval values according to a second quantization step size that is greater than the first quantization step size for the interbeat interval values or the adjusted interbeat interval values that are outside the first threshold of the reference value.

[0066] In some additional or alternative implementations, the first threshold can be based on a predetermined fixed value. In some other additional or alternative such implementations, the memory can have further computer-executable instructions stored therein that, when executed, further cause the one or more processors to: determine the first threshold based on the interbeat values in the sequence of interbeat intervals, and store the first threshold in the memory in association with the interbeat interval information.

[0067] In some additional or alternative implementations, the first sampling rate can be 25 Hz or higher, and the memory can have further computer-executable instructions stored therein that, when executed, further cause the one or more processors to: temporarily store the interbeat interval values as 11 bits or higher, temporarily store the adjusted interbeat values prior to quantization as 9 bits or lower, and store the adjusted interbeat values as 6 bits or lower.

[0068] In some implementations, a method can be provided that includes: a) obtaining data from a heart rate sensor at a first sampling rate, b) extracting an interbeat interval sequence from the data, c) determining a reference value associated with a value of an interbeat interval in the interbeat interval sequence, d) determining a difference between the reference value and the value of each interbeat interval in the interbeat interval sequence to produce a corresponding adjusted interbeat interval value, and e) storing interbeat interval information based on the adjusted interbeat interval values in a computer-readable memory in association with the reference value.

[0069] In some additional implementations, the method can further include performing a) through e) for a plurality of time periods and causing a wireless communication interface to: transmit interbeat interval information for the interbeat interval sequence for each of the plurality of time periods, and transmit the reference value for the interbeat interval sequence for each of the plurality of time periods in association with the interbeat interval information for that time period.

[0070] In some additional or alternative implementations, the reference value can be within a range of minimum / maximum values of the interbeat intervals in the interbeat interval sequence. In some additional or alternative implementations, the reference value can be an arithmetic mean, median, or mode of the interbeat intervals in the interbeat interval sequence. In some additional or alternative implementations, the reference value can be a mode of the interbeat intervals in the interbeat interval sequence. In some additional or alternative implementations, the reference value can be a measure of central tendency of the interbeat intervals in the interbeat interval sequence.

[0071] In some additional or alternative implementations, the method can further include quantizing at least one of the interbeat interval values and the adjusted interbeat interval values prior to storing the interbeat interval information based on the adjusted interbeat interval values in the memory. In such implementations, the quantizing can reduce a number of unique values in the interbeat interval information based on the adjusted interbeat interval values as compared to a number of unique values of the interbeat intervals in the interbeat interval sequence extracted from the data. In some additional such implementations, the reference value can be an arithmetic mean, median, or mode of the interbeat intervals in the interbeat interval sequence, and the quantizing can include quantizing at least one of the interbeat interval values or the adjusted interbeat interval values according to a first quantization step size for interbeat interval values or adjusted interbeat interval values that are within a first threshold of the reference value, and quantizing at least one of the interbeat interval values or the adjusted interbeat interval values according to a second quantization step size that is greater than the first quantization step size for interbeat interval values or adjusted interbeat interval values that are outside the first threshold of the reference value. In some additional such implementations, the first threshold can be based on a predetermined fixed value, while in some other such implementations, the method can further include determining the first threshold based on the interbeat values in the interbeat interval sequence and storing the first threshold in the memory in association with the interbeat interval information.

[0072] In some additional or alternative implementations, the first sampling rate can be 25 Hz or higher, and the method can further include temporarily storing the interbeat interval values as 11-bit or higher values, temporarily storing the adjusted interbeat values prior to quantization as 9-bit or lower values, and storing the adjusted interbeat values as 6-bit or lower values.

[0073] In some implementations, a sleep monitoring system can be provided that includes a wearable electronic device to be worn by a user. The wearable electronic device can include one or more motion sensors to generate motion data representative of motion of the wearable electronic device and one or more optical sensors to generate cardiopulmonary pulse-related data representative of circulatory system characteristics of the user. The wearable electronic device can further include one or more processors configured to receive data from the one or more motion sensors and the one or more optical sensors and a non-transitory machine-readable storage medium operably coupled to the one or more processors and having stored therein computer-executable instructions that, when executed, cause the one or more processors to, for each of a plurality of time segments: extract one or more movement features from the motion data for one or more first time windows associated with the time segment (where each first time window can be associated with a different one of the one or more movement features); extract one or more pulse data features from the cardiopulmonary pulse-related data for one or more second time windows associated with the time segment, each second time window being associated with a different one of the one or more pulse data features; and classify the time segment with a sleep stage selected from a plurality of potential sleep stages based on the one or more movement features extracted from the motion data for the one or more first time windows associated with the time segment and the one or more pulse data features extracted from the cardiopulmonary pulse-related data for the one or more second time windows associated with the time segment.

[0074] In some additional implementations, the one or more processors can be located in the wearable electronic device.

[0075] In some additional or alternative implementations, the wearable electronic device can further include a communication interface configured to communicate the motion data and the cardiopulmonary pulse-related data to the one or more processors, and the one or more processors can be located remotely from the wearable electronic device.

[0076] In some additional or alternative implementations, the one or more processors can be located in a portable electronic device selected from the group consisting of: a smartphone, a tablet computer, and a laptop computer.

[0077] In some additional or alternative implementations, the one or more processors can be located in one or more servers.

[0078] In some additional or alternative implementations, the first time window, the second time window, or at least two of the first time window and the second time window can have the same start point and the same end point, and thus can have the same duration.

[0079] In some additional or alternative implementations, the first time window, the second time window, or at least two of the first time window and the second time window can have the same start point and the same end point as the time period associated therewith, and thus can have the same duration as the time period.

[0080] In some additional or alternative implementations, the non-transitory machine- readable storage medium can further have stored therein computer-executable instructions that, when executed, cause the one or more processors to apply one or more post-classification rules to the plurality of time periods, wherein each post-classification rule compares sleep stage classifications of two or more time periods and changes one or more of the sleep stage classifications of one or more of the plurality of time periods in response to the one or more post-classification rules.

[0081] In some additional or alternative implementations, the plurality of potential sleep stage classifications can include at least a wake classification and a deep sleep classification, and the one or more post-classification rules can include a post-classification rule that causes a time period having a wake classification that is adjacent to a time period having a deep sleep classification to change the stage classification of the time period having the wake classification from the wake classification to the deep sleep classification.

[0082] In some additional or alternative implementations, the non-transitory machine- readable storage medium can further have stored therein computer-executable instructions that, when executed, cause the one or more processors to, in classifying each time period, determine, for each time period, a confidence number for each potential sleep stage classification based on one or more movement features extracted from motion data for one or more first time windows associated with the time period and one or more pulse data features extracted from cardiopulmonary pulse-related data for one or more second time windows associated with the time period, and classify the time period with a sleep stage classification corresponding to the potential sleep stage classification having the highest confidence number for the time period.

[0083] In some additional or alternative implementations, the one or more optical sensors can include a photoplethysmogram sensor, the one or more motion sensors can include an accelerometer, or the one or more optical sensors can include a photoplethysmogram sensor and the one or more motion sensors include an accelerometer.

[0084] In some additional or alternative implementations, the one or more movement features and the one or more pulse data features for each time period together can include one or more features such as: a cumulative movement index extracted from the motion data; a first elapsed time between a first time within the time period and a second time prior to the time period at which the cumulative movement index last exceeded a first threshold amount; a second elapsed time between a third time within the time period and a fourth time after the time period at which the cumulative movement index first exceeded a second threshold amount; a third elapsed time between a fifth time within the time period and a most recent time outside of the time period at which the cumulative movement index first exceeded a third threshold amount; a first number of time windows prior to the time period since the cumulative movement index last exceeded a fourth threshold amount; a second number of time windows after the time period until the cumulative movement index first exceeded a fifth threshold amount; an average heart rate for one of the one or more second time windows associated with the time period; a standard deviation of the average heart rate for one of the one or more second time windows associated with the time period; an average peak-to-peak value of cardiopulmonary pulse-related data for one of the one or more second time windows associated with the time period; an average trough-to-trough value of cardiopulmonary pulse-related data for one of the one or more second time windows associated with the time period; a standard deviation of peak-to-peak values of cardiopulmonary pulse-related data for one of the one or more second time windows associated with the time period; a standard deviation of trough-to-trough values of cardiopulmonary pulse-related data for one of the one or more second time windows associated with the time period; a difference between a 90th percentile peak-to-peak value and a 10th percentile trough-to-trough value of cardiopulmonary pulse-related data for one of the one or more second time windows associated with the time period; a difference between a 90th percentile trough-to-trough value and a 10th percentile trough-to-trough value of cardiopulmonary pulse-related data for one of the one or more second time windows associated with the time period; a power in a low frequency band of a heart rate variability analysis of cardiopulmonary pulse-related data for one of the one or more second time windows associated with the time period (where the low frequency band can be between 0.04 Hz and 0.15 Hz); a power in a high frequency band of a heart rate variability analysis of cardiopulmonary pulse-related data for one of the one or more second time windows associated with the time period (where the high frequency band can be between 0.15 Hz and 0.4 Hz); a ratio of a power in a low frequency band of a heart rate variability analysis of cardiopulmonary pulse-related data for one of the one or more second time windows associated with the time period and a power in a high frequency band of a heart rate variability analysis of cardiopulmonary pulse-related data for one of the one or more second time windows associated with the time period (where the low frequency band can be between 0.04 Hz and 0.15 Hz and the high frequency band can be between 0.15 Hz and 0.4 between); a percentage of total power in a low frequency band of the heart rate variability analysis of the cardiopulmonary pulse-related data in one of the one or more second time windows associated with the time period to a total power of the heart rate variability analysis of the cardiopulmonary pulse-related data in one of the one or more second time windows associated with the time period (where the high frequency band can be between 0.04 Hz to 0.15 Hz); a percentage of total power in a high frequency band of the heart rate variability analysis of the cardiopulmonary pulse-related data in one of the one or more second time windows associated with the time period to a total power of the heart rate variability analysis of the cardiopulmonary pulse-related data in one of the one or more second time windows associated with the time period (where the high frequency band can be between 0.15 Hz to 0.4 Hz); a standard deviation of an envelope of the cardiopulmonary pulse-related data in one of the one or more second time windows associated with the time period; a DC value-based estimated respiration rate of the cardiopulmonary pulse-related data in one of the one or more second time windows associated with the time period; or a variability in pulse shape in the cardiopulmonary pulse-related data in one of the one or more second time windows associated with the time period.

[0085] In some additional or alternative implementations, the one or more pulse data features for each time period can include one or more pulse data features such as: an average heart rate for one of the one or more second time windows associated with the time; a standard deviation of the average heart rate for one of the one or more second time windows associated with the time period; an average peak-to-peak value of cardiopulmonary pulse-related data for one of the one or more second time windows associated with the time period; an average trough-to-trough value of cardiopulmonary pulse-related data for one of the one or more second time windows associated with the time period; a standard deviation of the peak-to-peak value of cardiopulmonary pulse-related data for one of the one or more second time windows associated with the time period; a standard deviation of the trough-to-trough value of cardiopulmonary pulse-related data for one of the one or more second time windows associated with the time period; a difference between a 90th percentile peak-to-peak value and a 10th percentile peak-to-peak value of cardiopulmonary pulse-related data for one of the one or more second time windows associated with the time period; a difference between a 90th percentile trough-to-trough value and a 10th percentile trough-to-trough value of cardiopulmonary pulse-related data for one of the one or more second time windows associated with the time period; a power in a low frequency band (where the low frequency band can be between 0.04 Hz and 0.15 Hz) of a heart rate variability analysis of cardiopulmonary pulse-related data for one of the one or more second time windows associated with the time period; a power in a high frequency band (where the high frequency band can be between 0.15 Hz and 0.4 Hz) of a heart rate variability analysis of cardiopulmonary pulse-related data for one of the one or more second time windows associated with the time period; a ratio of a power in a low frequency band (where the low frequency band can be between 0.04 Hz and 0.15 Hz) of a heart rate variability analysis of cardiopulmonary pulse-related data for one of the one or more second time windows associated with the time period and a power in a high frequency band (where the high frequency band can be between 0.15 Hz and 0.4 Hz) of a heart rate variability analysis of cardiopulmonary pulse-related data for one of the one or more second time windows associated with the time period; a percentage of a total power in a low frequency band (where the high frequency band can be between 0.04 Hz and 0.15 Hz) of a heart rate variability analysis of cardiopulmonary pulse-related data for one of the one or more second time windows associated with the time period to a total power of a heart rate variability analysis of cardiopulmonary pulse-related data for one of the one or more second time windows associated with the time period; or a percentage of a total power in a high frequency band (where the high frequency band can be between 0.15 Hz and 0.4 Hz) of a heart rate variability analysis of cardiopulmonary pulse-related data for one of the one or more second time windows associated with the time period to a total power of a heart rate variability analysis of cardiopulmonary pulse-related data for one of the one or more second time windows associated with the time period.4Hz), wherein the cardiorespiratory pulse-related data for the time period of the sleep session can be normalized prior to extracting those one or more pulse data features, and the cardiorespiratory pulse-related data can be normalized such that the average peak-to-peak or trough-to-trough value for the total time period of the sleep session equals a predetermined value. In some additional such implementations, the predetermined value can be 1.

[0086] In some additional or alternative implementations, the one or more movement features for each time period can include one or more movement features such as: a cumulative movement index extracted from the motion data for the time period; a first elapsed time between a first time within the time period and a second time before the time period at which the cumulative movement index last exceeded a first threshold amount; a second elapsed time between a third time within the time period and a fourth time after the time period at which the cumulative movement index first exceeded a second threshold amount; a third elapsed time between a fifth time within the time period and a most recent time outside of the time period at which the cumulative movement index first exceeded a third threshold amount; a first number of time windows before the time period since the cumulative movement index last exceeded a fourth threshold amount; a second number of time windows after the time period until the cumulative movement index first exceeded a fifth threshold amount.In such implementations, the one or more pulse data features can include one or more pulse data features such as: a variability of an inter-beat interval in the cardiorespiratory pulse-related data assessed by sample entropy for one of the one or more second time windows associated with the time period; a variability of the cardiorespiratory pulse-related data assessed by sample entropy for one of the one or more second time windows associated with the time period; a root mean square deviation of the inter-beat interval for one of the one or more second time windows associated with the time period; a root mean square deviation of the cardiorespiratory pulse-related data for one of the one or more second time windows associated with the time period; a low frequency spectral power of the inter-beat interval for one of the one or more second time windows associated with the time period; a low frequency spectral power of the cardiorespiratory pulse-related data for one of the one or more second time windows associated with the time period; a high frequency spectral power of the inter-beat interval for one of the one or more second time windows associated with the time period; a high frequency spectral power of the cardiorespiratory pulse-related data for one of the one or more second time windows associated with the time period; a variability of an envelope of the cardiorespiratory pulse-related data for one of the one or more second time windows associated with the time period; a variability of an envelope of the inter-beat interval for one of the one or more second time windows associated with the time period; a variability of a detrended respiratory rate extracted from the cardiorespiratory pulse-related data for one of the one or more second time windows associated with the time period; a percentiles span of a heart rate extracted from the cardiorespiratory pulse-related data for one of the one or more second time windows associated with the time period or extracted from the inter-beat interval for one of the one or more second time windows associated with the time period; a normalized detrended heart rate extracted from the cardiorespiratory pulse-related data for one of the one or more second time windows associated with the time period or extracted from the inter-beat interval for one of the one or more second time windows associated with the time period; or a cross-correlation of each of one or more pulse shapes in the cardiorespiratory pulse-related data for one of the one or more second time windows associated with the time period with a preceding pulse shape in the cardiorespiratory pulse-related data, wherein the pulse shapes can be normalized to a common duration prior to cross-correlation.

[0087] In some additional or alternative implementations, the one or more movement features can include at least one of: a time since last movement, a time until next movement, and / or a time to the nearest movement, and the one or more pulse data features can include at least one of: a variability of an inter-beat sequence assessed using sample entropy, a variability of a detrended respiratory rate extracted from the cardiorespiratory pulse-related data, and / or a cross-correlation of a pulse rate signal.

[0088] In some additional or alternative implementations, the computer-executable instructions that cause the one or more processors to classify each time segment can include instructions that, when executed, cause the one or more processors to transmit the one or more movement features and the one or more pulse data features to a server system that executes a classifier that generates a sleep stage classification for each time segment and provide the sleep stage classification to the one or more processors.

[0089] In some additional or alternative implementations, the computer-executable instructions can further, when executed, cause the one or more processors to fill in missing data points in the cardiopulmonary pulse-related data prior to causing the one or more processors to extract the one or more pulse data features from the movement data for the one or more first time windows.

[0090] These and other implementations are further described in the detailed description that follows, with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0091] The disclosure is illustrated by way of example and not by way of limitation in the figures of the accompanying drawings in which like references indicate similar elements. The following

[0092] Figure 1 A high-level flow diagram is depicted for sleep classifier training techniques.

[0093] Figure 2 An example of a sleep monitoring platform for implementing various example embodiments is shown.

[0094] Figure 3 A more detailed block diagram of various components that can be used in a sleep monitoring platform such as Figure 2 The sleep monitoring platform shown.

[0095] Figure 4 An isometric view of a wearable device that can be used to collect pulse-related data and movement data during a sleep session for use in the techniques and systems discussed herein is depicted.

[0096] Figure 5 is a block diagram depicting various modules that can be included in a processing system in accordance with example embodiments.

[0097] Figure 6 An example photoplethysmogram (PPG) signal is depicted.

[0098] Figure 7 An example interbeat interval duration over time plot is depicted.

[0099] Figure 8 An example PPG signal plot and signal envelope is depicted.

[0100] Figure 9 An example PPG signal graph is depicted with the illustrated heart rate related component and respiration rate component.

[0101] Figure 10 A flowchart of a technique for classifying sleep stages of a time period is depicted.

[0102] Figure 11 Is a conceptual example of a random forest decision tree classifier.

[0103] Figure 12 Is a flowchart of a technique that can be used to update the sleep stage classification of a time period if needed after the sleep stage classification of the time period has initially been assigned by a classifier.

[0104] Figure 13 An example of a graphical user interface that can be used to present sleep stage data for a sleep period is depicted.

[0105] Figure 14 A flowchart of a technique associated with presenting summary data is depicted.

[0106] Figure 15 A flowchart of another technique for presenting sleep stage data is depicted.

[0107] Figures 16 to 19 Depicts the GUI of Figure 13 but modified so that the percentile breakdowns associated with different sleep stages are emphasized in each graph by breaking out or fading out the remaining percentile breakdowns.

[0108] Figure 20 Depicts a more comprehensive user interface that can be used to provide information about various aspects of a person's sleep period.

[0109] Figure 21 A flowchart of a technique for displaying a GUI that allows sleep stage data to be compared to historical sleep stage data is depicted.

[0110] Figure 22 Depicts the user interface of Figure 20 except that it has been modified to include percentile breakdown comparison data.

[0111] Figure 23 A flowchart of a technique for displaying a GUI that allows sleep stage data to be compared to peer sleep stage data is depicted.

[0112] Figure 24 Depicts the user interface of Figure 20 except that it has been further modified to include percentile breakdown comparison data.

[0113] Figure 25 A hypnogram is depicted showing double sleep stage indications for some time intervals.

[0114] Figure 26 A technique for providing an enhanced hypnogram display such as Figure 25 A flowchart of one technique for providing an enhanced hypnogram display as depicted in

[0115] Figure 27 Another technique for producing an enhanced hypnogram is depicted.

[0116] Figure 28 A flowchart of a technique for compressing inter-beat interval data is depicted.

[0117] Figure 29 A flowchart of another technique for compressing inter-beat interval data is depicted. DETAILED DESCRIPTION

[0118] SUMMARY

[0119] During sleep, a person can experience a number of different sleep states or stages, which can be categorized in a number of different ways. In practical practice, a sleep session for a person can be divided into a number of intervals, such intervals are often referred to as "epochs," and each such interval can be evaluated to determine the sleep state or stage in which the person was during that interval. For example, the American Academy of Sleep Medicine guidelines define four classes of sleep: W, N1-N2, N3, and random eye movement (REM), which can be considered to correspond to wakefulness (or wake), light sleep (which can include both N1 and N2), deep sleep, and REM sleep, respectively. Other methods can include a fewer or greater number of sleep stages. For example, a "artifact / off-wrist" sleep stage can be used to classify periods of time for which data indicates that a wearable device was not worn or data from a wearable device could not be classified as a "normal" sleep stage. Alternatively, periods of time for which data indicates that a wearable device was not worn or data from a wearable device could not be classified as a "normal" sleep stage can be handled in other ways. For example, such periods of time can simply be considered gaps in the data record, can be assigned an "interpolated" sleep stage (e.g., the average sleep stage of a number of time periods immediately preceding, immediately following, or including a time period with missing sleep stage data), or can be assigned a "wildcard" sleep stage (e.g., they can be considered equivalent to a number of different sleep stages).

[0120] Discussed herein are various techniques and systems for classifying a subject's sleep state or stage based on data from a wearable electronic device, such as a wearable biometric monitoring device having, for example, a heart rate sensor and a motion sensor, rather than on more traditional systems that utilize expert interpretation of the output of an EEG device. As used herein, a "wearable" biometric monitoring device is a device that is designed to be comfortable and unobtrusive to wear, such as a wrist-worn device like the Fitbit Charge HR, Fitbit Surge, Fitbit Blaze, etc. Devices such as EEG systems and electrocardiogram systems, which feature a web of electrodes that must be adhered to various locations on the human body and then connected to some form of external processing system, are not unobtrusive and are generally not considered to be comfortable, and thus are not considered to be "wearable" biometric monitoring devices, as that term is used herein.

[0121] In the systems discussed herein, various data-driven features for a given time interval of a sleep session can be derived from the obtained optical heart rate sensor data and / or accelerometer sensor data associated with that time interval. These features can then be provided to a classifier that classifies that time interval into one of several categories or stages of sleep based on the values of one or more of the data-driven features. The classifier can be trained based on the values of similar data-driven features collected during sleep sessions that were classified, for example, using more traditional techniques such as manual scoring by a professional sleep scorer, who can often be referred to as a "polysomnographic technician" or "polysomnographic technologist," using more sophisticated equipment. Thus, for example, a small population of sleepers can be monitored during sleep using wearable biometric monitoring devices and more complex equipment used by one or more sleep scorers to evaluate and classify intervals or epochs during a sleep session into various sleep stages. Thus, each time interval or epoch collected during such training can be associated with: a) the sleep stage that has been assigned to it by the one or more trained sleep scorers; and b) a set of data-driven features collected by the wearable biometric monitoring device during the time period associated with that time interval or epoch.

[0122] Data-driven features collected by the wearable biometric monitoring system during such training sleep sessions, along with sleep stage classifications provided by sleep scorers, can then be used to train a classifier, such that the classifier can classify intervals or epochs during sleep sessions based only on data from the wearable biometric monitoring device. Such classification can include, for example, labeling time intervals with an indicator that indicates the sleep stage classification based on how the classifier classifies the time interval. Thus, a trained classifier can be used to classify sleep for a larger population of sleepers, using only a wearable biometric monitoring device to monitor their sleep, without the need for more in-depth sleep monitoring equipment or direct involvement of individuals trained in sleep scoring. In some implementations, the classifications provided by such a classifier can be post-processed to address potential classification artifacts. For example, if a series of epochs or intervals during a sleep session exhibit a classification pattern that is rarely, if ever, seen during "scored" sleep sessions, one or more of the epochs or intervals can be reclassified according to one or more heuristic or other rules.

[0123] There can be a very large number of data-driven features that can be used in a classifier. During the training process, some or many of these data-driven features can be eliminated from the classifier, as they can contribute to the accuracy of the classifier in a trivial or lesser way. Figure 1 A high-level flowchart depicting one technique for pruning out less useful features from a set of features that can be used to train a classifier.

[0124] In block 102, a global set of features to consider for a classifier can be selected. In some cases, the number of these features can be relatively high, e.g., hundreds of features. In block 104, a classifier can be trained using a training data set, e.g., features extracted from collected data associated with known classifications. As part of the classifier training, each feature involved in the training can be associated with a weighting factor or other quantifier of the importance of the feature in classification determination. In block 106, a weighting factor (or other quantifier of importance) can be obtained for each feature in the global set of features, which can then be ordered by their weighting factors (or other quantifiers of importance) in block 108. One or more features having the lowest weighting factors or other quantifiers of importance can then be removed from the global set of features (in some cases, there can be multiple features having the lowest weighting factors, so multiple features can be removed). The ranking of the features having the lowest weighting factors or other quantifiers of importance can be retained for later reference. In block 112, a determination can be made as to whether any features remain in the global set - if so, the technique can return to block 104, and the operations of blocks 104-110 can be repeated for the reduced set of features. Eventually, all features will be removed from the global set, and assigned a ranking upon removal from the global set. In practical terms, once only one feature remains in the global set, the training of a classifier using that last feature and the formal removal of that last feature from the global set can be skipped, and the remaining feature can be assigned a first position in the ranking directly. If it is determined in block 112 that no features remain in the global set (or, alternatively, that all features have been assigned a ranking in accordance with the operations in block 110 or equivalent), the technique can proceed to block 114, in which the features can be ordered by their associated rankings (e.g., as determined in block 110). In block 116, a subset of the features can be selected based on having a ranking above a selected threshold. The subset of selected features can then be used to retrain the classifier in block 118. This technique allows for the pruning out of features that contribute to classification in an unimportant manner, allowing for high classification speed and computational efficiency without significantly impacting classification accuracy in a negative manner. Various tools for training, validating, and testing various types of classifiers can be found, e.g., in the documentation for scikit-learn, a module developed for the Python programming language (http: / / scikit-learn.sourceforge.net).

[0125] The intervals or epochs of sleep periods that have been classified can then be summarized and presented to the user in various useful and informative ways via a graphical user interface.

[0126] In some implementations, the data-driven features discussed above can be compressed and stored in a particular manner to reduce the memory or bandwidth that can be required to store or transmit such data-driven features.

[0127] These and other implementations of the concepts discussed herein are further described below in the context of example implementations.

[0128] Example sleep monitoring platform or system

[0129] Figure 2 An example of a sleep monitoring platform 200 for implementing various example implementations is shown. In some implementations, the sleep monitoring platform 200 includes a wearable device 202, a secondary device 204, a network 206, and a backend system 208. The components of the sleep monitoring platform 200 can be connected directly or through the network 206, which can be any suitable network. In various implementations, one or more portions of the network 206 can include an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless WAN (WW AN), a metropolitan area network (MAN), a portion of the Internet, a portion of the Public Switched Telephone Network (PSTN), a cellular telephone network, or any other type of network, or a combination of two or more such networks.

[0130] Although Figure 2 A particular example of an arrangement of the wearable device 202, the secondary device 204, the network 206, and the backend system 208 is shown, any suitable arrangement or configuration of the wearable device 202, the secondary device 204, the network 206, and the backend system 208 is contemplated.

[0131] The wearable device 202 can be a wearable device configured to be attached to a wearer's body, such as a wristband, a watch, a finger clip, an ear clip, a chest band, an ankle band, or any other suitable device.

[0132] The secondary device 204 can be any suitable computing device, such as a smartphone, a personal digital assistant, a mobile phone, a personal computer, a laptop computer, a computing tablet, or any other device suitable for connecting with one or more of the wearable device 202 or the backend system 208.

[0133] The wearable device 202 can access the secondary device 204 or the backend system 208 directly via the network 206 or via a third-party system. For example, the wearable device 202 can access the backend system 208 via the secondary device 204.

[0134] The backend system 208 can include a network-addressable computing system (or systems), such as one or more servers, that can host and process sensor data for one or more users. The backend system 208 can generate, store, receive, and transmit sleep-related data, such as, for example, sleep duration, bedtime, wake time, wake-up time, sleep state (e.g., sleep stage, wakefulness, etc.), or any other sleep-related statistic. The backend system 208 can be accessed directly by other components of the sleep monitoring platform 200 or via the network 206. A user can use the wearable device 202 to access, send data to, and / or receive data from the auxiliary device 204 and / or the backend system 208.

[0135] Figure 3 A more detailed block diagram of various components that can be used in a sleep monitoring platform such as the sleep monitoring platform 200 is depicted. In Figure 2 the sleep monitoring platform 200 is shown. In Figure 3 the wearable device 302, which can also be referred to herein as a "wearable biometric monitoring device," a "wearable biometric tracking device," or simply a "biometric tracking device" or a "biometric monitoring device," can include one or more processors 302a, a memory 302b, and a communication interface 302c. The communication interface 302c can include, for example, a wireless communication system, such as a Bluetooth interface, for wirelessly communicating with other components, such as the auxiliary device 304. The memory 302b can store computer-executable instructions for controlling the one or more processors 302a to perform various functions discussed herein, including, for example, instructions for causing the one or more processors 302a to obtain motion data from one or more motion sensors 310 and pulse-related data from one or more optical sensors 312.

[0136] The auxiliary device 304, which can be a cellular phone, a smartphone, a laptop, or other device as discussed herein, can similarly include one or more processors 304a, a memory 304b, and a communication interface 304c. The auxiliary device 304 can also include a display 304d (wearable device 302 as well as backend system 308 can also include displays, but the auxiliary device 304 can be the most likely device through which a user can interact with data collected by the wearable device 302, and thus, the display 304d is expressly indicated due to this avenue through which such interaction can occur). The memory 304b can store computer-executable instructions for controlling the one or more processors 304a to perform various functions discussed herein and to send and receive data or instructions, for example, to or from either or both of the wearable device 302 and the backend system 308 via the communication interface 304c.

[0137] A back-end device 308, which can for example be one or more servers or other back-end processing systems, can include one or more processors 308a, memory 308b, and a communication interface 308c. The memory 308b can store computer-executable instructions for controlling the one or more processors 308a to provide the various functionality described herein.

[0138] Figure 4 An isometric view of a wearable device that can be used to collect pulse-related data and motion data during a sleep session - in this case, a Fitbit Charge 2 TM The wearable device 402 can include a housing 418 that is connected to a band 414 that can be fastened together to attach the wearable device 402 to a person's wrist. The housing 418 can include a window 416 that allows an optical sensor 412 to measure a pulse-related physiological phenomenon of the wearer. As discussed earlier, in addition to the one or more optical sensors 412 and the one or more motion sensors 410, the wearable device 402 can also include one or more processors 402a, memory 402b, and a communication interface 402c. The memory 402b can store computer-executable instructions for controlling the one or more processors to obtain motion data from the one or more motion sensors 410, to obtain pulse-related data from the one or more optical sensors 412, and then to transmit this data to other devices in a sleep monitoring platform. The one or more optical sensors can for example include one or more light sources and one or more photodetectors that are placed adjacent to a person's skin. The light source and the photodetector are typically arranged such that light from the light source cannot directly reach the photodetector. However, when such an optical sensor is placed adjacent to a person's skin, light from the light source can diffuse into the person's flesh and then be shot back from the person's flesh such that the photodetector can detect it. The amount of such light that is shot back from the person's flesh can vary depending on the heart rate, because the amount of blood that is present in the flesh varies depending on the heart rate, and the amount of light that is shot back from the person's flesh varies depending on the amount of blood that is present. Depending on the wavelength of the light that is used, different measurements of the person's circulatory system can be obtained. For example, green light is particularly suitable for measuring heart rate, while a combination of red and infrared light is particularly suitable for measuring blood oxygen levels.

[0139] Figure 5 is a block diagram depicting various modules that can be included in a processing system 500 in accordance with example embodiments. It is to be understood that the processing system 500 can be deployed in the form of, for example, a wearable device, a server computer, a client computer, a personal computer, a laptop computer, a mobile phone, a personal digital assistant, and other processing systems, or a combination thereof. For example, in one embodiment, the processing system 500 can be embodied as a server computer that is configured to provide a sleep monitoring platformFigure 2 the backend system 208 of the sleep monitoring platform 200 depicted in FIG. 2. In alternative implementations, the processing system 500 can be embodied as the wearable device 202 of the sleep monitoring platform 200. As another example, the processing system 500 can be embodied as a combination of the backend system 208 of the sleep monitoring platform 200 and the wearable device 202 or the auxiliary device 204, or a combination of the wearable device 202, the auxiliary device 204, and the backend system 208. Reference is made to Figure 2 Figure 5 In various implementations, the processing system 500 can be used to implement a computer program, logic, application, method, process, or software to label sleep states and provide other functionality, as described in more detail below.

[0140] As shown in FIG. 5, the processing system 500 can include a set of one or more motion sensors 502, a set of one or more optical sensors 504, a feature generation module 506, a sleep stage classifier module 508, and a rules engine 510. The set of one or more motion sensors 502 can be a module configured to generate motion data that characterizes movement experienced by the wearable device 202. In some cases, the set of one or more motion sensors 502 can be one or more accelerometers, e.g., a three-axis accelerometer, and the motion data can be data that characterizes acceleration experienced by the wearable device 202, including acceleration due to gravity. Other examples of motion sensors include gyroscopes or magnetometers. In some cases, the motion data can be pre-processed or post-processed to remove noise or otherwise facilitate characterization of movement experienced by the wearable device. Figure 5

[0141] Data obtained from the motion sensors can generally take the form of one or more time series that represent acceleration or other motion-related parameters (e.g., rotation or movement relative to the Earth's magnetic field) along one or more axes. This data can be pre-processed or post-processed in order to remove artifacts prior to feature extraction, fill in missing data points (e.g., via interpolation), eliminate or reduce noise, etc. In some implementations, some post-processing can actually be part of the feature extraction process discussed in more detail below. For example, the motion data can be quantified into a movement measure or a movement index for a given time period. Thus, as used herein, a movement measure or index can refer to some quantification of movement over a time period. The movement measure can quantify movement along a single axis or multiple axes (e.g., X, Y, and / or Z axes). Examples of movement measures are described in more detail in U.S. Patent Application No. 14 / 859,192, filed September 18, 2015, which is incorporated herein by reference.

[0142] ​​The set of one or more optical sensors 504 can be a module configured to generate heart rate data, e.g., the one or more optical sensors 504 can function as a heart rate sensor of the wearable device 202. The heart rate data can be referred to herein as “pulse-related data” in that it characterizes a physiological phenomenon driven by pulsatile behavior in the circulatory system of a person.

[0143] By way of example and not limitation, the set of one or more optical sensors can be a set of one or more photoplethysmographic (PPG) sensors that produce one or more time series of pulse-related data. The pulse-related data can take the form of a waveform of detected light that varies in response to the volume of blood pulsating within the skin of a wearer of the wearable device due to the heartbeat of the wearer. Similar to the motion data generated at operation 302, the pulse-related data can be pre-processed or post-processed to remove noise or artifacts, smooth the data, or otherwise enhance the data in some manner. For example, in some cases, the pulse-related data can be pre-processed to remove noise or motion artifacts. Additionally or alternatively, in some cases, the pulse-related data can be post-processed to extract inter-pulse intervals, optical data amplitude data, or some combination thereof, e.g., as part of a feature extraction process discussed in more detail below.

[0144] Feature extraction & sleep stage classifier

[0145] The feature generation module 506 can be a module, e.g., computer executable instructions stored on a non-transitory machine readable medium such as a disk drive, configured to cause one or more processors to derive a set of one or more data driven features from motion data and pulse related data for a given one or more time windows associated with a particular time period. Examples of such features can include a cumulative movement indicator extracted from the motion data, a time elapsed since a significant movement extracted from the motion data, a time to a next significant movement extracted from the motion data, a time to a most recent significant movement extracted from the motion data, a variability of an inter-beat interval extracted from the pulse related data or a variability of an amplitude of the pulse related data assessed by sample entropy, a root mean square deviation of the pulse related data, a root mean square of the inter-beat interval, a low frequency spectral power or spectral power density of the inter-beat interval or the pulse related data extracted from the pulse related data, a high frequency spectral power or spectral power density of the inter-beat interval or the pulse related data extracted from the pulse related data, a variability of an envelope of the pulse related data, a variability of an envelope of the inter-beat interval, a variability of a detrended respiration rate extracted from the pulse related data, an inter-percentile spread of a heart rate extracted from the pulse related data or from the inter-beat interval, a normalized detrended heart rate extracted from the pulse related data or from the inter-beat interval, and a cross-correlation of each of one or more pulse shapes in the pulse related data with a preceding pulse shape in the pulse related data (in which case the pulse shapes can be normalized to a common duration prior to cross-correlation). Additionally or alternatively, similar features can also be extracted from the amplitudes of the pulse related data (or motion data) in a similar manner, e.g., instead of a series of inter-beat intervals, a series of amplitudes can be analyzed and features extracted therefrom. For example, each amplitude in the series can correspond to a measure of amplitude of an individual heartbeat pulse. For example, in some implementations, a peak amplitude in the PPG signal during a heartbeat can be used as a measure of amplitude for feature extraction purposes. Alternatively, an average amplitude in the PPG signal during a heartbeat can be used as a measure of amplitude for feature extraction purposes. Various specific examples of features that can be extracted from motion data and pulse related data are discussed in more detail below.

[0146] For example, a cumulative movement index can be obtained by integrating, over a given time window, acceleration signals or other signals indicative of movement from one or more accelerometers or other motion sensors (which can optionally be smoothed and / or corrected prior to integration) to obtain a region associated with the accelerometer signals - which can then be mapped to a motion index, e.g., the cumulative movement index can be 0 if the integrated signal is less than a first threshold; 1 if the integrated signal is greater than or equal to the first threshold but less than a second threshold; 2 if the integrated signal is greater than or equal to the second threshold but less than a third threshold, and so on. The integrated signal for a period can be referred to as an "activity count." The total number of counts produced by the motion sensor during the time period can be used to obtain a measure of the subject's level of activity. For example, the more active a person is, the more movement and acceleration will be detected, which can increase the total count produced by the motion sensor during the time period. In a sense, the total count for multiple time periods can be proportional to the integrated signal for those time periods. In another implementation, another cumulative movement index can be determined by evaluating the percentage of periods during which the amplitude of the motion data is above a pre-set threshold. There are multiple ways in which the counts produced by the motion sensor over a time period can be used. For example, an activity count for a 30 second period of an acceleration signal can be obtained by counting the number of times the acceleration signal crosses a pre-determined threshold in the positive direction. Alternatively, the count can be evaluated by counting the number of times the signal in question crosses the x-axis, i.e., zero-crossings, in a period. These are just some examples of ways in which an activity count can be generated and used to produce a cumulative movement index, and it is to be understood that the present disclosure is not limited to just these examples.

[0147] If a multi-axis accelerometer is used, the acceleration used to generate the motion feature can be the absolute combined amplitude of the multiple axes, e.g., the total acceleration amplitude, rather than the multiple component amplitudes.

[0148] The cumulative movement index can then be used, for example, as a benchmark for determining whether significant movement has occurred. For example, one feature that can be extracted from the motion data is how much time has elapsed since the last time window or time period for which the cumulative movement reached or exceeded a particular value, such time being in absolute time or in terms of the number of time windows or time periods for which the cumulative movement index was calculated. For example, if a time window or time period has a cumulative movement index of 3 or greater, that time window or time period is considered to have significant movement, and the amount of time since the last significant movement can then be assessed by determining how much time has elapsed between the time period or time window for which the feature is being extracted and the last time period or time window for which the cumulative movement index was greater than or equal to 3. The same technique can also be prospective, for example, the number of time periods or time windows between the time period or time window for which the feature is being determined or extracted and the next time window or time period for which the cumulative movement index is greater than or equal to 3. Such analysis can be performed because feature extraction can be performed well after the data from which the feature is extracted has been collected, allowing for features to be extracted for a time period or time window based on data from that time period or time window and / or based on data from time periods or time windows before and after the time window or time period for which the feature is determined.

[0149] Many features can also be extracted from pulse-related data; features extracted from pulse-related data can be referred to herein as“pulse data features.” A typical photoplethysmography (PPG) signal, such as can be output by an optical heart rate sensor such as those discussed herein, can provide a periodic signal indicative of heart rate. A PPG signal is typically a voltage measured by an optical heart rate sensor and reflects volume changes of blood through blood vessels in tissue under the sensor. As the heart beats, the volume of blood in the blood vessels changes with each beat. Thus, the peak / trough pattern seen in the raw PPG signal reflects the underlying heartbeat of the person. By detecting the peaks (or alternatively the troughs) of the PPG signal, the heart activity of the person can be determined. Figure 6Such a PPG signal is depicted. Such a signal is typically analyzed to determine how many peaks occur in the signal within a given interval, e.g., if 70 peaks occur within a 1 minute interval, then this indicates a heart rate of 70 beats per minute; this is one example of a feature extracted from pulse-related data. A peak counting algorithm can be used to determine the locations of the peaks, e.g., the data can be analyzed to find maxima or minima within a specified time window. For example, a time window can be defined between each time the PPG signal exceeds a certain level and then subsequently falls below another level - the maxima within this time window can be identified as a "peak." The identified peaks can also be checked to ensure that they are "physiologically reasonable," i.e., that they are not too far apart or too close together for them to be physiologically possible. There are many techniques for identifying peaks, and the present disclosure is not limited to any particular such technique.

[0150] Another feature that can be extracted is a series of interbeat intervals, i.e., the time between each pair of adjacent peaks; the interbeat intervals can also be referred to as "PP intervals." For context, these PP intervals derived from a photoplethysmogram can be considered equivalent to "RR intervals," which are the times between QRS complexes in an electrocardiogram. A QRS complex is a specialized term used to refer to the combination of three specific and consecutive signal behaviors seen in a typical electrocardiogram. The only slight difference between PP intervals and RR intervals is when there is a slight modulation in pulse transit time (PTT), which can occur to a small degree. However, for a good approximation, the series of PP intervals is equivalent to the series of RR intervals, and thus can be used as a proxy for electrocardiogram measurements. In Figure 6 In the figure, four interbeat intervals 620 are indicated, although each pair of adjacent peaks has its own associated interbeat interval. Thus, for example, if there are 70 heartbeats in a minute, then there can be 69 interbeat intervals within that 1 minute period. The interbeat intervals can be another source of features. For example, the variation in interbeat intervals for a given time period or time window can be assessed using any of a variety of different techniques in order to obtain various features for that time period or time window. Figure 7 A plot of interbeat interval duration versus time is depicted; it can be seen that the interbeat interval durations within this example time period vary from 0.92 seconds to 1.09 seconds.

[0151] The main influencing factors on the PP interval are the parasympathetic nervous system (whose activation tends to slow down the heart rate and thus lengthen the PPG pulse) and the sympathetic nervous system (whose activation tends to speed up the heart and shorten the PP interval), which together constitute the autonomic nervous system. The parasympathetic nervous system and the sympathetic nervous system can operate on slightly different time scales (in particular, the parasympathetic nervous system can operate on a very short time scale and have an effect on the next heartbeat after the parasympathetic nervous system is activated, while the sympathetic system is mediated by acetylcholine and requires multiple heartbeats before the heart rate changes induced by the activation of the sympathetic nervous system take effect). One way to capture this difference is to take the spectral density of the PP interval sequence using a technique called heart rate variability (HRV) analysis. The higher frequency components of the spectrum, for example 0.15 Hz to 0.4 Hz, can reflect parasympathetic activation, as they correspond to a short time scale, while the lower frequency components, for example 0.04 Hz to 0.15 Hz, can reflect both parasympathetic and sympathetic effects. Thus, the power in the HF band can be attributed to parasympathetic activation, and the power in the LF band can be attributed to a mixture of sympathetic and parasympathetic activation.

[0152] One aspect of this technique is that the sleep stage of a person is reflected in the degree of activation of their sympathetic and parasympathetic nervous systems. Thus, the HRV parameters can reflect information about the sleep stage, which can be used by the classifier to distinguish between different potential sleep stages.

[0153] As mentioned earlier, at least two general categories of information can be extracted from the pulse-related data. First, as discussed above, features related to heart rate variability (HRV) can be extracted by identifying the RR intervals (RR intervals are measured in seconds and are the time between consecutive peaks of the PPG signal). The HRV can vary depending on the sleep stage, for example, the heart rate slows down in deep sleep, and the degree of respiratory sinus arrhythmia (a natural occurring variation in heart rate over the respiratory cycle that appears in the entire respiratory cycle) will increase. These physiological effects can be measured quantitatively, for example, by measuring the heart rate and measuring the high-frequency component of the HRV parameter, for example, in the frequency range of 0.15 Hz to 0.4 Hz.

[0154] Second, the envelope and shape of the PPG AC amplitude can be extracted. As previously mentioned, the amount of light that diffusely reflects off of a person's skin and is detected by a PPG sensor varies in time with the person's pulse or heart rate; the time-varying aspect of this signal is referred to as the "AC" component, and the "constant" part of this signal is referred to as the "DC" component. The AC amplitude can be determined by subtracting the DC component from the total amplitude of the signal, and is often colloquially referred to as the "PPG amplitude." The PPG amplitude is determined by the volume of blood in the tissue under the optical sensor, which in turn is determined by the compliance of the arteries in the tissue. This arterial compliance is actively controlled by the sympathetic and parasympathetic nervous systems, which are ultimately controlled by the autonomic nervous system, so the amplitude of the PPG reflects activity of the central nervous system (CNS), which in turn is associated with sleep stages. Typically, there is no parasympathetic nervous system activation of the blood vessels (except in some special locations), but sympathetic nervous system activation of the blood vessels does occur and has the effect of causing the blood vessels to constrict. This can be seen in the photoplethysmogram signal as Figure 8 shown in FIG. 6, which has a modulation of the amplitude of the PPG signal over the approximately 60 seconds shown in the figure. The amount and variability of this modulation is associated with sleep stages, so can provide a useful feature for use with a classifier. In Figure 8 the envelope of the signal power has been extracted by computing the root-mean-square power of the signal over a window of approximately 5 seconds, and is shown as the dashed line (the underlying PPG signal is the solid black line). There are a variety of alternative ways to compute the envelope of the signal, for example, taking the Hilbert transform of the signal, or using a peak-sampling hold method followed by low-pass filtering. Regardless of the method, the original PPG signal can then produce a new signal called PPGE, where PPGE represents the PPG envelope. Features can then be extracted from the PPGE signal for each epoch, such as the standard deviation of the PPGE signal.

[0155] Various examples of features that can be extracted from pulse-related data are discussed below, however this is not an exhaustive discussion, and additional features can also be extracted for use in the classifier. In some implementations, sample entropy can be used to assess the variability of the interbeat intervals to provide another feature that can potentially be used by the classifier to classify the sleep stage for a given time period or time interval.

[0156] Another pulse data feature that can be used is the root mean square deviation of pulse-related data or interbeat intervals. A further pulse data feature that can be used for classification purposes is the spectral power of a series of interbeat intervals for a time period or time window for which the feature is being determined. The spectral power can be determined over a range of frequencies, e.g., in a low frequency range (e.g., 0.04 Hz to 0.15 Hz can be considered a "low" frequency for purposes of heart rate data analysis) or in a high frequency range (e.g., 0.15 Hz to 0.4 Hz can be considered a "high" frequency for purposes of heart rate data analysis). Thus, the high frequency spectral power of interbeat intervals can be determined by integrating the spectral power density of interbeat intervals over a range of frequencies spanning between 0.15 Hz to 0.4 Hz.

[0157] Other examples of pulse data features include features based on respiration-related phenomena. For example, a PPG signal can contain a dominant periodic element that can be attributed to both heart rate-related physiological phenomena and respiration-related phenomena. Thus, for example, if a PPG signal exhibits a large frequency component in the 60 Hz to 90 Hz domain, that frequency component can be indicative of heart rate, and the periodicity of such a component can be used to determine interbeat intervals. However, the same PPG signal can also have more large frequency components in the range of 6 Hz to 15 Hz, which can be indicative of respiration rate. Thus, a PPG signal can be used to estimate both heart rate-related parameters and respiration rate-related parameters.

[0158] For example, Figure 9 Pulse-related data from a PPG sensor is depicted over a time window of approximately sixty seconds. It can be seen that there is a higher frequency heart rate signal (solid black line) that sits on top of a lower frequency respiration rate signal (dashed line); in this example, the heart rate over the interval is 62 beats per minute, while the respiration rate is 13 breaths per minute. The respiration rate (and features extracted therefrom) can be useful discriminators for a sleep stage classifier, as both the variability and absolute value of the respiration rate can be correlated with sleep stage. In deep sleep, the respiration rate is very stable and nearly constant (e.g., can remain within 0.5 breaths per minute). In contrast, in REM sleep, the respiration rate can vary greatly. Thus, the variability of the respiration rate can prove to be a valuable feature for a sleep stage classifier.

[0159] Other features that can be determined or extracted by the feature extractor can include, for example, inter-percentile span of various parameters. For example, the span between the 10th percentile inter-beat interval duration and the 90th percentile inter-beat interval duration during a given time period or time window can be used as a feature. Another feature that can be extracted is the amount of cross-correlation between the optical pulse rate signal shapes in a series of data pulses, i.e., a quantification of how similar each pulse shape in a series of pulse shapes is compared to the previous pulse shape. In determining such a feature, the feature extractor can first normalize each pulse shape to have the same duration to allow for more accurate correlation based on shape.

[0160] Returning to the processing system 500, the classifier module 508 can be a module configured to classify or otherwise label a time period having a given sleep state based on one or more features derived from the motion data and the heart rate data. For example, the classifier module 508 can apply any of a number of classifier techniques to the features of a given time period or time window in order to determine a sleep stage classification for that time period or time window. Example classifier techniques that can be used can include, for example, nearest neighbor classifiers, random forest classifiers, support vector machines, decision trees, neural networks, and linear discriminant classifiers. For example, various classifiers identified and discussed in Jacques Wainer, “Comparison of 14 different families of classification algorithms on 115 binary datasets,” Computing Institute, University of Campinas, Campinas, SP, 13083-852, Brazil, June 6, 2016, can be used as classifiers in the context of the present disclosure. The inventors have found that, as discussed earlier, such classifiers can be trained using features extracted from motion and pulse-related data collected during sleep periods conducted under the supervision of a professional or trained sleep scorer using specialized equipment and conducting in-depth analysis of each time interval being evaluated— such data can be referred to herein as “benchmark” motion data and / or pulse-related data. Thus, for example, features extracted from motion and pulse-related data collected during a time period or time window that has been assigned by a sleep scorer to a sleep stage of “deep sleep” can be used to train a classifier to assign a “deep sleep” classification to time windows or time periods having features matching the criteria of the classifier developed during such training. The classifier is initially built and tested on the benchmark data using cross-validation techniques such as k-fold cross-validation and leave-one-out cross-validation.

[0161] Figure 10 A flowchart depicting a technique for classifying sleep stages for a time period, e.g., a 30 second or 60 second epoch (epochs can be selected to have any desired duration, although 30 second epochs are common). In block 1002, motion data for a given time period can be obtained, e.g., from an accelerometer or other motion sensor. In block 1004, movement features for the time period can be extracted from the motion data for a time window associated with the time period. For some features, the time window can be coextensive with the time period, i.e., the same as the time period, but for other features, the time window can be larger than the time period and can or can not overlap the time period. For example, in many cases, a "time until last significant movement" feature can involve motion data from a time window that spans multiple time periods. In this case, the motion data to be obtained in block 1002 can be obtained for all time periods within the time window. The extracted movement features can be determined, e.g., as set forth with respect to the movement features discussed above, or using other techniques. The technique can then proceed to block 1006, where it can be determined whether other movement features are to be determined with respect to the time period for which features are being extracted. If so, the technique can return to block 1004, where another movement feature can be extracted. If all movement features have been extracted, the technique can proceed from block 1006 to block 1014.

[0162] In block 1008, the technique can also include obtaining pulse-related data from a heart rate sensor for the time period in question. In block 1010, pulse data features for the time period can be extracted from the pulse-related data for a time window associated with the time period. As with the extraction of movement features, the time window in question can be coextensive with the time period, or for some pulse data features can span multiple time periods and can or can not overlap the time period for which features are being extracted. For example, heart rate variability (HRV) for a time period can be determined using pulse-related data for a time window that includes the time period in question and two time periods immediately preceding the time period and two time periods immediately following the time period, i.e., five time periods. Thus, if the time window in block 1010 spans multiple time periods, the pulse-related data to be obtained in block 1008 can be obtained for multiple time periods. Once the pulse data features have been obtained in block 1010, e.g., such as discussed earlier herein, the technique can proceed to block 1012, where it can be determined whether other pulse data features are needed. If it is determined in block 1012 that other pulse data features are needed, the technique can return to block 1010 for further pulse data feature extraction. If it is determined in block 1012 that no further pulse data features are needed, the technique can proceed to block 1014.

[0163] It is to be understood that operations 1002 and 1008 can be performed substantially in parallel. This can be the case where the set of one or more motion sensors and the set of one or more optical sensors are active during the same time period to produce sensor data (e.g., motion data and pulse-related data) characterizing that time period. It is to be understood that in some cases, the microprocessor can interleave or perform time-slice execution of the processing of samples generated by the set of one or more motion sensors 502 and the set of one or more optical sensors 504 (e.g., using threads if an embedded operating system is executing on the microprocessor, or a series of interrupt handlers if sensor data processing is executing on bare metal). Thus, in some cases, although the sensors can be active at the same time, the microprocessor (in single-core or multi-core implementations) can interleave execution of the processing of sensor data. Such processing of sensor data can include storing samples in corresponding data structures, cleaning up signals, converting data to different formats (e.g., encoding data into a relatively smaller data representation), or any other suitable process. It is also to be understood that other operations discussed with respect to the technology presented herein can likewise be performed in parallel with one another, e.g., feature extraction can be performed for multiple different features at the same time.

[0164] The determination in blocks 1006 and 1012 of whether other features need to be extracted can be made on a contextual basis, or based on a predetermined definition of what features are deemed necessary in order to perform classification. For example, a random forest classifier can compare certain features to specific criteria at each branch of a decision tree to determine which features to compare to criteria in the next branch. Depending on the branch employed in the classifier, certain features can not be needed to reach a final determination— only the features needed in the branch nodes of the classifier used in the classification determination can be determined for a given time period. As a result, if desired, feature extraction can be performed on an as-needed basis to avoid unnecessary feature extraction, which can save CPU resources and power. Alternatively, there can be a predetermined set of motion and pulse data features that can be determined for each time period classified by the classifier.

[0165] One challenge that can be encountered during classification by the classifier using the extracted features is that some features can exhibit significant variation from person to person, e.g., average heart rate can vary widely within a population. This can make it difficult for the classifier to provide accurate results. To address this variability, some of the extracted features can be normalized (during training and during classification). In particular, these features can include features extracted from heart rate related data such as heart rate, standard deviation of heart rate over epoch, mean of PP interval or interbeat interval over epoch, standard deviation of PP interval or interbeat interval over epoch, difference between 90th percentile value and 10th percentile value of PP interval or interbeat interval over epoch, power in low frequency and / or high frequency bands of HRV analysis performed over an epoch and some additional neighboring epochs, ratio of low frequency power to high frequency power for an epoch, percentage of low frequency power and / or high frequency power with respect to total HRV power. To normalize these features, the PP interval or interbeat interval can be normalized or scaled such that the average PP interval or interbeat interval over the entire sleep session being analyzed has the same value as the average PP interval or interbeat interval of other sleep sessions used to train the classifier (same applies to data of sleep sessions used to train the classifier). Thus, for example, the PP interval or interbeat interval can be normalized such that the average PP interval value for a given sleep session is equal to 1. By normalizing the interbeat interval sequence, other features extracted based on the normalized interbeat interval sequence will also be normalized.

[0166] Once sufficient movement and pulse data features have been extracted in blocks 1004 and 1010, these features can be passed to a classifier in block 1014. The classifier can select various features from the pool of movement features and pulse data features for a time period and compare them to various criteria in order to assign a sleep stage classification for that time period in block 1016. Figure 11 is a conceptual example of a decision tree classifier in which different features selected from a set of features {X1, X2, X3, X4,... X N-1 and X N are evaluated at different levels of the branching decision tree to determine which branches to take next and ultimately determine which sleep stage classification to assign.

[0167] As discussed previously, classifiers such as decision trees can be "trained" using empirical data. For example, a sleep stage classifier can be trained by making a set of labeled data available, labeling epochs as Wake, Light, Deep, or REM (or any sleep stage desired, e.g., there can be different labels or additional labels such as artifact and / or off-wrist labels). Each epoch can have a set of attribute values or features associated with it (labeled X1, X2, X3, etc.). A decision tree can be "trained" by splitting the source set of labels into subsets based on testing attribute values for one of the features. For example, if a person starts with 1000 labeled epochs, it can be determined that using the rule X1>0.4 divides the initial set of labeled epochs into a subset of 250 epochs in one branch and 750 epochs in another branch, but the set of 250 contains most of the deep sleep epochs. This means that the rule X1>0.4 can generally be used to identify deep sleep epochs. The selection of the threshold (0.4 in this example) can be optimized by maximizing the overall accuracy of the decision rule for classifying deep versus non-deep epochs in this example split. The subset of 250 epochs can then be further refined into more accurate classifications (e.g., X6>0.2 identifies most of the light sleep epochs in the subset of 250 epochs that contains most of the deep sleep epochs. This process can be repeated for each resulting subset in a recursive manner called recursive partitioning. The selection of which feature to use at any split in the tree can be random during training, and the selection of the threshold can be chosen to maximize the classification accuracy of the selected subset. Thus, an optimal decision tree can be reached through multiple random iterations of selected feature partitioning (which can be thought of as a logical test involving a particular feature or attribute and a threshold in this case). Recursive partitioning can be stopped when further recursion results in no improvement (or an improvement below a minimum improvement amount required) in classification accuracy. Once the feature partitioning has been identified that provides a desired degree of classification accuracy relative to the training data set (1000 labeled epochs in the above example), a decision tree employing the same feature partitioning can be used to classify the sleep stage of epochs from other data sets.

[0168] Decision trees are effective classifiers, but can suffer from "over-training" in which the rules they provide are overly dependent on the training data used to define them. A classifier technique built on decision trees is called a random forest classifier. As described above, a random forest classifier is formed by randomly generating a plurality of decision trees. These decision trees can then be combined to form an overall weighted set of decisions using majority voting or the like.

[0169] A simple decision tree classifier or a random forest classifier can be executed by the wearable device, or in some embodiments, by the secondary device or the backend system. In the latter case, the extracted features can be transmitted from the wearable device to the secondary device or the backend system, or from the secondary device to the backend system.

[0170] For a system that has trained a random forest classifier as described above, a set of example features that can provide good classification performance for use in the classifier can include: time elapsed since the last movement above a preset threshold (e.g., time since the last epoch with a movement index above a preset threshold), time until the next movement above a similar or different preset threshold (e.g., time until the next epoch with a movement index above the preset threshold), very low frequency component of the respiration-derived spectrum, autocorrelation of consecutive RR intervals (or interbeat intervals), kurtosis of consecutive differences of RR intervals (or interbeat intervals), difference between current respiratory rate and respiratory rate computed over a one-hour window, and 90th percentile of heart rate and / or RR intervals (or interbeat intervals).

[0171] For example, the very low frequency component of the respiration-derived spectrum can be derived by first filtering pulse-related data collected by the PPG sensor, e.g., using a bandpass filter configured to pass data occurring at frequencies typically associated with respiratory behavior; the respiration-derived spectrum can then be obtained from the resulting bandpass signal. The very low frequency component of such a spectrum can be in the range of 0.015 Hz to 0.04 Hz.

[0172] The autocorrelation of consecutive RR intervals (or interbeat intervals) can be obtained by taking an indexed series of interbeat intervals (or RR intervals) for the epoch in question and then taking the autocorrelation of the interval series.

[0173] The kurtosis of consecutive differences of RR intervals (or interbeat intervals) can be obtained by taking a series of kurtosis values formed by taking the difference in duration of each pair of adjacent interbeat intervals in the series.

[0174] The difference between current respiratory rate and respiratory rate computed over a one-hour window can be determined by taking the respiratory rate (e.g., as determined using a bandpass filter) for the epoch in question and then subtracting the average respiratory rate over a one-hour epoch therefrom. Time periods other than one hour can also be used, e.g., 55 minutes, 65 minutes, etc.

[0175] The 90th percentile of heart rate and / or RR intervals (or interbeat intervals) can be determined by determining the heart rate and / or interbeat interval below which 90% of the heart rate and / or interbeat intervals for the epoch in question are.

[0176] In some implementations, the classifier can output a confidence score for each sleep stage for a given interval, and the classifier can assign the sleep stage with the highest value confidence score to the given interval. For example, in the case of a linear discriminant classifier, classes are separated by hyperplanes in the feature space. These hyperplanes define class boundaries, and the distance of a data point to these boundaries determines the confidence score. For example, a data point that is very close to a boundary can be classified as less reliable compared to a data point that is far from the boundary and well into a region of a particular class.

[0177] In block 1018, it can be determined whether the sleep session being analyzed is over, i.e., whether all time segments within the sleep session have been classified. If not, the technique can proceed to block 1022, where the next time segment to be classified can be selected before returning to blocks 1002 and 1008. If all time segments of the sleep session have been classified, the technique can end in block 1020.

[0178] The rules engine 510 can be a module configured to analyze and update the sleep stage classification assigned to a time segment by the classifier according to a set of sleep stage rules, which can also be referred to herein as“update rules” or“post-classification rules.” The sleep stage rules can include data or logic that defines patterns of sleep stages that are permissible, or alternatively or additionally, are allowed or prohibited sleep stage transitions, e.g., sleep stage pattern constraints. Such rules can be useful because a classifier that classifies time periods or time segments on a per-period basis can not fully take into account the temporal structure of the data. In particular, for human sleep, certain patterns of sleep stages are more likely than others, e.g., it is uncommon to go directly from a deep sleep stage to a wakeful sleep stage — it is more common to transition from a deep sleep stage to a light sleep stage. It is also uncommon to have an isolated period associated with a REM sleep stage in the middle of a sequence of periods associated with deep sleep stages. To account for these scenarios, the rules engine 510 can implement rules that correct for sleep stage sequences that are very unlikely.

[0179] For example, the rules engine 510 can use sleep stage rules to correct a sleep stage pattern from D W D (deep-wakeful-deep) to D DD (deep-deep-deep) because the sleep stage rules can specify that sleep state W can be prohibited from transitioning to sleep state D. In another example, the rules engine can correct a pattern of a series of multiple consecutive epochs associated with deep sleep stages being interrupted by one or more epochs associated with REM sleep stages. Another example is using knowledge of physiologically normal behavior to correct patterns of behavior. As a specific example, in normal overnight sleep, it is physiologically less likely for REM to precede deep sleep, so the rules engine can penalize the probability of detecting REM in a period that precedes a detection of deep sleep. For example, if one or more epochs are classified as being associated with REM sleep stages, but there are no epochs classified as being associated with "deep sleep stages" (or less than a given number of epochs or a given consecutive number of epochs) before the epochs classified as being associated with "REM sleep stages," then the epochs in question can be changed from being associated with REM sleep stages to some other sleep stage, e.g., the sleep stage of the adjacent non-REM sleep stage epochs.

[0180] Another example of sleep stage rules is majority rule analysis. In majority rule analysis, the sleep stage of an intermediate time interval of a series of multiple time intervals can be changed to the predominant sleep stage of the series of multiple time intervals. Other physiology-based rules can be time-based. For example, REM sleep is less likely soon after sleep begins, so one sleep stage rule can be that epochs within a given time interval from the beginning of a sleep period can not be permitted to be associated with REM sleep stages. Such a time interval can be based on actual duration or on a number of epochs— e.g., if the rule is that REM sleep stages are not permitted in the first 42 minutes of sleep, then any 30-second epoch classified as a REM sleep stage in the first 84 epochs of a sleep period can have its sleep stage classification changed to, e.g., the sleep stage of the adjacent non-REM sleep stage epochs.

[0181] Figure 12is a flowchart of a technique that can be used to update sleep stage classification for a time period if desired after the sleep stage classification for the time period has been initially assigned by a classifier. The technique can begin in block 1202, where the sleep stage for a given time period can be obtained. In block 1204, the sleep stage classifications assigned to other contiguous time periods that immediately precede the given time period, immediately follow the given time period, or both immediately precede and immediately follow the given time period can be obtained. The resulting pattern of sleep stage classifications for these multiple time periods can be determined in block 1206, which is then compared to one or more criteria associated with a post-classification rule in block 1208. For example, a given post-classification rule can be that any time period that is preceded or followed by a contiguous block of time periods in which the sleep stage classification is all the same but different from the sleep stage classification of the given time period should be reclassified to have the same sleep stage classification as the sleep stage classification of the time periods in that contiguous block of neighboring time periods. If the conditions or criteria of the post-classification rule evaluated in block 1208 are met, then the technique can proceed to block 1210, where the sleep stage classification of the given time period can be updated according to the post-classification rule, after which the technique can proceed to block 1212, where it can be determined whether there are other time periods that are to undergo the post-classification rule. If it is determined in block 1208 that the post-classification rule is not met, then the technique can proceed to block 1212. If it is determined in block 1212 that there are other time periods that are to undergo the post-classification rule, then the technique can return to block 1202 to analyze additional time periods. If it is determined in block 1212 that there are no other time periods that need to be tested by the post-classification rule, then the technique can proceed to block 1214, where the technique can end.

[0182] It will be appreciated that, Figure 10 and Figure 12 The methods of the present disclosure can be implemented using the sleep monitoring platforms discussed above, for example, with reference to the sleep monitoring platforms described in Figure 2 or Figure 3 For example, a wearable device having a motion sensor and an optical heart rate sensor, such as wearable device 202, can be used to obtain the motion and pulse-related data. Wearable device 202 can also perform some or all of the feature extraction, and in some cases, also perform the classification functions discussed above, as well as provide the platform for running the post-classification rules. For example, backend system 208 can provide the parameters and classifier limits of the classifier to wearable device 202; this can include the criteria used by the classifier during classification that were learned by the classifier during training. The resulting trained classifier can be completed at the backend system and then distributed to wearable device 202 for use by wearable device 202. Alternatively, some or all of the feature extraction and / or classification can be performed in a secondary device, such as secondary device 204, or even at backend system 208.

[0183] As another example, the processing system 500 can include fewer, different, or additional modules than those shown in FIG. 5. For example, in alternative implementations, the classifier module 508 and the rules engine 510 can be combined into one module. In another implementation, the classifier module 508 can be separate from and executed or processed in parallel with the rules engine 510. Figure 5

[0184] It is to be understood that tagging time periods with sleep states based on features derived from a set of motion sensors and optical sensors provides a relatively useful technique for monitoring the sleep of a user of a wearable device. This is the case because such sensors are generally low power and provide relatively comfortable tracking through a wearable device that can be, for example, strapped to the wrist of a user (e.g., in the form of a watch or band).

[0185] User interface

[0186] Once a person's sleep period has been characterized using the sleep monitoring platform and its classifier, the resulting data set for the sleep period can be broken down into various formats or analyzed and presented to the person in order to help them better understand their sleep patterns. For example, the resulting data set output from the sleep monitoring platform can include the start time of the sleep period in question, and the end time of the sleep period in question, and the start and stop times of various intervals in which various sleep stages have been assigned by the sleep stage classifier throughout the sleep period. Alternatively, the sleep period can be broken down into a standardized format, e.g., into 30 second long intervals (which, as previously mentioned, can be referred to as "epochs"), and the data set can simply list the number of epochs in which the sleep stage changed from one value to another (and what the new value of the sleep stage was). This data can then be analyzed to produce various meta-statistics about the sleep period in question, e.g., by counting the number of deep sleep epochs, the person's total deep sleep time can be determined. A percentile breakdown of sleep stages in the sleep period is another - the total amount of time spent in each sleep stage can be divided by the total amount of time spent in all sleep stages (including the "awake" sleep stage) to produce such a percentile breakdown. In another example, by measuring the time from the start of the sleep period recording to the first epoch or period of time that is classified as having a light sleep stage, "time to sleep onset" or "sleep onset latency" meta-statistics information can be determined. These meta-statistics can then be presented to the user via a GUI (some examples of such meta-statistics presentation are included in the example GUIs herein).

[0187] Figure 13 An example of a graphical user interface that can be used to present sleep stage data for a sleep period is depicted. In this example, the GUI presents a graph of the sleep stage data for a sleep period in question, and also presents various meta-statistics about the sleep period in question. Figure 13 ​The image presents a percentile decomposition 1324 of the time spent in each sleep stage tracked during a sleep period. In this example, the classifier used categorizes each interval or period into one of four sleep stages: awake, REM, light, and deep. It is to be understood that the term "sleep stage" as used herein can be used to refer to "awake" sleep stages, as well as the sleep stage in which a person is actually asleep. The percentile decomposition 1324 provides a high-level overview or snapshot of a person's overall sleep behavior during the sleep period being reviewed.

[0188] Figure 14 A flowchart depicts the techniques associated with this aggregated data. In box 1402, sleep stage classifications for multiple time periods associated with a sleep period can be obtained. In box 1404, the total amount of time spent in each sleep stage during a sleep period can be determined, and then this total amount of time is divided by the total amount of time spent in all different sleep stages to determine the percentile decomposition of the total time spent in each sleep stage during the sleep period. In box 1406, a GUI component can be presented indicating each sleep stage classification and the percentile decomposition of the amount of time spent in each sleep stage during the sleep period. For example, such a GUI component may include textual decompositions, such as “4% awake,” “22% REM,” “56% light,” and “18% deep,” and / or graphical representations, such as vertically or horizontally oriented histograms or pie charts. The technique may conclude in box 1408.

[0189] A more detailed presentation of sleep data can be provided through a sleep graph 1326, which is a timeline in which time intervals corresponding to a specific sleep stage are indicated by horizontal lines or segments 1332 at different elevations across those time intervals (relative to the horizontal axis), each elevation corresponding to a different sleep stage into which the time interval is categorized. In sleep graph 1326, time intervals assigned to “awake” sleep stages are plotted at the highest elevation, time intervals assigned to “REM” sleep stages are plotted at the next highest elevation, time intervals assigned to “light” sleep stages are plotted at the next highest elevation, and time intervals assigned to “deep” sleep stages are plotted at the lowest elevation. While sleep graphs are known in the art, the various variations of sleep graphs discussed herein represent how sleep graphs can be utilized to improve the provision of more effective and meaningful feedback to individuals seeking to monitor their sleep.

[0190] Figure 13 The GUI also includes text content 1328, which can be selected based on the type of information being presented or based on one or more aspects of the sleep period data being presented. Figure 13The text content simply explains the development of each sleep stage during typical sleep, which can be informative for those unfamiliar with the subject of sleep monitoring. User interface features or inputs allow users to scroll or turn pages. Figure 13 There are several different versions of the GUI, which are discussed in more detail below.

[0191] Figure 15 A flowchart depicts another technique for presenting sleep stage data. In box 1502, sleep stage data for time intervals of sleep periods can be obtained. As discussed earlier in this document, each time interval can be associated with a specific sleep stage according to a classifier. In box 1504, a GUI component including a sleep graph can be presented, indicating the obtained time intervals and their associated sleep stages. This GUI component can, for example, resemble... Figure 13 Sleep diagram 1326.

[0192] In box 1506, it can be determined whether, for example, via... Figure 13 The user input area 1330 of the GUI receives input indicating the selection of a specific sleep stage among the sleep stages represented in the sleep graph. If it is determined in box 1506 that the user has selected a specific sleep stage, the technique can proceed to box 1508, where the GUI component, i.e., the sleep graph, can be modified to emphasize or highlight the portion of the sleep graph corresponding to the selected sleep stage. For example, the horizontally extending elements of the sleep graph representing the time interval associated with the selected sleep stage can be thickened to create a brightness or color impact, and / or presented with more vibrant colors than in the initial sleep graph. Alternatively or additionally, the horizontally extending elements of the sleep graph representing the time intervals associated with other sleep stages can be de-emphasized, for example, by weakening their colors, displaying them with reduced color intensity, fading them out, or even completely hiding them. Thus, the highlighted time intervals of the sleep graph can have a different graphical appearance than the other time intervals of the sleep graph.

[0193] In box 1510, the text content displayed for the modified GUI can be updated or changed to include information already customized based on the selected sleep stage during the reviewed sleep period and / or actual data associated with the selected sleep stage during the reviewed sleep period. See below for more information. Figures 16 to 19 This section discusses examples of modifications that can be performed in boxes 1508 and 1510. After the GUI has been modified, the technique can return to box 1506 to further monitor whether input indicating the selection of different specific sleep stages is received.

[0194] If a determination is made in block 1506 that a selection of a particular sleep stage of the sleep stages has not been made, the technique can proceed to block 1512, where another determination can be made as to whether an input has been received indicating a selection of all of the sleep stages. If a determination is made in block 1514 that an input has been received indicating a selection of all of the sleep stages, the technique can return to block 1504. If a determination is made in block 1514 that an input has not been received indicating a selection of all of the sleep stages, the GUI can remain in whatever state it is currently in, and the technique can return to block 1506. Such a technique can allow for the presentation of summary sleep stage data using the GUI, but can also allow a user to toggle between a summary output that focuses on any particular sleep stage data and a more specific summary output that focuses on one particular sleep stage of the sleep stages. The following are examples of GUIs that focus on a particular sleep stage.

[0195] For example, in Figure 13 at the bottom of the GUI can be seen five circles - each of which can correspond to a different version of the GUI being depicted. Figure 13 The first version depicted in

[0196] For example, Figure 16 depicts Figure 13 the GUI of Figure 16 but is modified such that the percentile breakdown 1324 associated with the "wake" sleep stage, e.g., the top-most sleep stage in this example hypnogram, is emphasized by graying out or fading out the remaining percentile breakdowns 1324 associated with the other sleep stages. Further, the hypnogram 1326 can be emphasized and / or augmented with additional indicators in the same manner to emphasize the "wake" sleep stage interval. For example, in Figure 16 the portion of the hypnogram 1326 associated with sleep stages other than the "wake" stage is grayed out or faded out, similar to the percentile breakdowns 1324 for those sleep stages. Conversely, the portion of the hypnogram 1326 associated with the "wake" sleep stage can be presented in its normal color palette, or even emphasized with a new color or format. In particular, in some embodiments, vertical bars 1334 across the width of the horizontal segment 1332 indicating the time period associated with the wake sleep stage can be added to the hypnogram 1326 to allow a person reviewing the data to more easily discern where the time interval stops and starts on the timeline at the bottom of the hypnogram 1326. These vertical bars 1334 can span between the horizontal segment 1332 and the horizontal axis of the hypnogram 1326. Thus, the area marked by the vertical bars can provide a different graphical appearance compared to the area below or above the horizontal segment for other time intervals of the hypnogram, thereby further highlighting the time interval associated with the selected sleep stage.

[0197] In addition to changes to the percentiles breakdown 1324 and hypnogram 1326, the text content 1328 can also be changed based on the sleep stage data that has been collected for the sleep session. For example, in Figure 16 , the text content is customized by summarizing the number of intervals in which the person was awake, e.g., in this case, three intervals. Alternatively, the total duration spent in wakeful sleep stages can be presented in the text content or some other quantity that is associated with or driven by the sleep stage data for the wakeful sleep stages. In addition to the quantity-based information that can be presented in the text content 1328, the text content 1328 can also include other types of customized information. For example, explanatory text can be provided that provides the person with some insight into whether their sleep stage data for their wakeful stages is "normal" based on the frequency and / or duration of time the person spent in wakeful stages. In this particular example, the person using the device is reassured that the number of intervals in which the person was "awake" during the sleep session is a normal amount of such intervals.

[0198] Figure 17 the user interface of Figure 13 is depicted, but modified to emphasize the intervals or periods that have been classified as being associated with REM sleep stages. For example, a vertical bar 1334 has been added to connect the horizontal segment 1332 associated with REM sleep stages to the horizontal axis. In addition, the horizontal bars for the remaining sleep stages have been grayed out or faded. The text content 1328 has also been changed to display information related to REM sleep stages and the person's REM sleep stage data. In this example, the text content summarizes the percentage of the sleep session that the person spent in REM sleep stages. To give context to this number, the text content 1328 can include information about the typical percentage of time that an average person spends in REM sleep stages. For example, this typical percentage can be selected so as to be particularly relevant to the user. For example, if the user's age and / or gender are known, the typical percentage of time spent in REM sleep stages presented to the user can be taken from a more relevant population of individuals, e.g., the user can be presented with a range of average REM sleep stage percentages based on typical sleep behavior of a similar population of people. In addition to such comparative information, the text content 1328 can also include physiological information related to REM sleep stages. For example, the text content 1328 can include information that informs the user of what the physiological characteristics of REM sleep are, such as elevated heart rate, vivid dreams, faster breathing, increased movement, etc.

[0199] Figure 18 the user interface of Figure 13 is depicted, but modified to emphasize the intervals or periods that have been classified as being associated with light sleep stages. As with the previous example, the text content 1328 has been changed to display information related to light sleep stages and the person's light sleep stage data. In this example, the text content summarizes the percentage of the sleep session that the person spent in light sleep stages. To give context to this number, the text content 1328 can include information about the typical percentage of time that an average person spends in light sleep stages. For example, this typical percentage can be selected so as to be particularly relevant to the user. For example, if the user's age and / or gender are known, the typical percentage of time spent in light sleep stages presented to the user can be taken from a more relevant population of individuals, e.g., the user can be presented with a range of average light sleep stage percentages based on typical sleep behavior of a similar population of people. In addition to such comparative information, the text content 1328 can also include physiological information related to light sleep stages. For example, the text content 1328 can include information that informs the user of what the physiological characteristics of light sleep are, such as slower breathing, less movement, etc.Figure 17 As shown in the modified interface, the text content 1328 has been modified to display the percentage of sleep time spent in the sleep stage of interest, in this case, the percentage of sleep time spent in the light sleep stage.

[0200] Figure 19 Depicting Figure 13 The user interface has been modified to emphasize the intervals or periods that have been categorized as being associated with deep sleep stages. Like the previously modified user interface, this interface features text content that has been tailored based on the user's actual sleep data, such as indications of the percentage of sleep time spent in deep sleep stages, and information about the physiological effects of deep sleep and how much deep sleep is typical for most people.

[0201] Users can use user input 1330 to navigate to the summary screen by swiping left or right (see [link]). Figure 13 ) and a sleep-stage-specific modified user interface (see Figures 16 to 19 Navigation between any user interface within the system. Of course, other input systems can also be used to navigate between different user interfaces or their modifications.

[0202] Figure 20 A more comprehensive user interface has been designed to provide information on various aspects of a person's sleep patterns. Figure 20 The user interface is designed for use with mobile phones or other devices with portrait-mode displays. The user interface may not be fully visible within the display and can be configured to slide up or down as needed to make different parts of the user interface visible. Of course, other layouts are also possible, including layouts where parts of the user interface may not need to be displayed.

[0203] exist Figure 20 The user interface depicted can display a sleep graph 2026 similar to the sleep graph 1326 discussed earlier. A summary 2036 of the total sleep time during sleep periods can also be provided; as in this case, this summary 2036 can exclude time spent in waking sleep stages. A user input area 2030 can be provided to allow the user to navigate between different user interfaces or modify the current user interface to display different comparative data, as will be explained with reference to the accompanying figures later. In this case, the user input area 2030 includes three user-selectable options: last night, 30-day average, and baseline. These will be explained in more detail later.

[0204] Figure 20 The user interface can also include percentile decomposition 2024, which is similar to the previous one. Figures 13 to 19Further sleep session data such as the start and end of sleep in a sleep session can be displayed in sleep session timeline portion 2038. It is to be understood that some reported characteristics of a sleep session such as the duration of the sleep session, the start of sleep, the end of sleep, etc. can be assessed based on when a person is actually asleep and when the person is thereafter persistently (at least during the day) awake, while other aspects of a sleep session can include activity that occurs prior to falling asleep. For example, a sleep session can begin when it is detected that a person is attempting to fall asleep, e.g., by assuming a prone position, which can be sensed by, e.g., a motion sensor in a wearable device, or can be inferred from the time of day or even explicitly conveyed by a user, e.g., by pressing a particular button on a wearable device or providing input to a secondary device in communication with the wearable device. The period of time between the "start" of a sleep session and when a person is actually asleep can include an interval during which the person is in a "wake" sleep stage.

[0205] The user interface can also optionally include graphical elements that depict heart rate over time, such as heart rate display 2040. In this example, heart rate display 2040 is overlaid on the rendition of sleep graph 2026 to allow a user to observe the correlation between heart rate and sleep stage (in this example, the sleep graphs shown by 2026 and 2040 are different, but it is to be understood that this is for illustrative purposes only; in actual practice, the sleep graphs can be based on the same data set and in that case would be identical). Heart rate display 2040 can optionally highlight peak heart rate 2044 and minimum heart rate 2046 experienced during a sleep session, e.g., by including indicators pointing to the location of each such maximum or minimum, such indicators can have textual information indicating the actual value of those maxima / minima.

[0206] The GUIs such as shown in Figure 21 may be modified in accordance with the techniques discussed above with respect to Figure 20 to compare sleep session sleep stage data with historical sleep stage data for a user. In Figure 21 , blocks 2102, 2104, and 2106 can correspond to blocks 1902, 1904, and 1906, respectively, discussed above with respect to Figure 14The operations in blocks 1402, 1404, and 1406 are similar to the operations discussed above. For example, block 2106 can cause display of a GUI component such as the percentile breakdown 2024 to be displayed. In block 2108, a determination can be made as to whether a user input has been received that indicates that the user wishes to modify the GUI to display a comparison of the sleep session sleep stage data to historical sleep stage data for that user. If it is determined in block 2108 that no user input has been received that indicates a desire to modify the GUI, then the technique can end at block 2118. Alternatively, block 2108 can be repeated periodically during the presentation of the GUI component of block 2106 in order to monitor for such potential user inputs over time.

[0207] If it is determined in block 2108 that a user input has been received that indicates a desire to modify the GUI, then the technique can proceed to block 2110, in which a plurality of additional sleep sessions for the user can be obtained for sleep stage classification; this information can be referred to as "representative personal sleep stage data." This can be similar to the operations performed in block 2102, except that instead of a plurality of previous sleep sessions being the focus of block 2102. For example, the number of previous sleep sessions can span a time period of 1 week prior to the time at which the GUI is presented, 4 weeks prior to the time at which the GUI is presented, 15 days prior to the time at which the GUI is presented, or 30 days prior to the time at which the GUI is presented. Alternatively, such time intervals can be assessed with respect to the time at which the sleep session in question was recorded (rather than when the GUI presenting data for that sleep session was presented).

[0208] In block 2112, a relative percentile breakdown of the total time spent in each sleep stage for each of the additional sleep sessions can be determined, for example, by dividing the total amount of time spent in each sleep stage for each sleep session by the total duration of that sleep session. In block 2114, an average of the relative percentile breakdowns for each sleep stage across all of the additional sleep sessions obtained in block 2110 can be determined to yield a representative relative percentile breakdown for the sleep stages. Alternatively, the total amount of time spent in each sleep stage across all of the additional sleep sessions obtained in block 2110 can be divided by the total duration of all of those additional sleep sessions in order to obtain the representative relative percentile breakdown. In block 2116, the GUI component, e.g., the percentile breakdown 2024, can be modified to indicate information for each sleep stage classification and the percentile breakdown for each sleep stage of the sleep session of block 2102 as well as information indicating the representative relative percentile breakdown for the sleep stages. This information can be presented such that the sleep stage percentile breakdown for the sleep session in question can be easily compared to the corresponding representative relative percentile breakdown.

[0209] For example,Figure 22 depicts Figure 20 a user interface of , modified to include comparative data in the percentile breakdown 2024 in response to the user selecting "30 day average" in the user input area 2030. In response to this selection, representative relative percentile breakdowns 2042 have been added to the percentile breakdown 2024 display. In this case, these representative relative percentile breakdowns 2042 are the average percentile breakdowns for the person's sleep sessions in the previous 30 days. This allows the user to easily see if the previous night's sleep was abnormal compared to recent sleep sessions. For example, if the person woke up feeling very tired compared to recent sleep sessions, such a comparison can allow them to quickly see that they spent more (or less) time in particular sleep stages in the recent sleep session compared to recent sleep sessions of a larger population.

[0210] GUIs such as shown in Figure 23 may also be modified according to techniques discussed below with respect to Figure 20 to compare sleep session sleep stage data to sleep stage data of a plurality of users; as discussed in more detail below, the sleep stage data of the plurality of users can be selected based on a cohort of users associated with the user. In Figure 23 , blocks 2302, 2304, and 2306 can correspond to operations similar to those in blocks 1402, 1404, and 1406 discussed above with respect to Figure 14 . For example, block 2306 can cause display of GUI components such as the percentile breakdown 2024 to be displayed. In block 2308, it can be determined whether user input has been received indicating that the user wishes to modify the GUI to display a comparison of sleep session sleep stage data to sleep stage data of a plurality of users. If it is determined in block 2308 that user input indicating a desire to modify the GUI has not been received, the technique can end in block 2320. Alternatively, block 2308 can be repeated periodically during the presentation of the GUI components of block 2306 in order to monitor for such potential user input over time.

[0211] If it is determined in block 2308 that user input has been received indicating a desire to modify the GUI, the technique can proceed to block 2310, where a plurality of users can be selected. Such users can be selected to match one or more demographic criteria that match (or are within a predetermined range of) various demographic parameters of the user whose sleep stage data is being reviewed. For example, if the user in question is a 42 year old male, a plurality of users can be selected that are male and are between 40 and 45 years of age. It is to be understood that in some embodiments, a plurality of users can be selected regardless of demographic details. It is also to be understood that a plurality of users can be selected based on any number of potential demographic parameters, including, for example, approximate location latitude, time zone, age, gender, work schedule, health level, average daily activity level, etc. Once a suitable plurality of users has been selected, the technique can proceed to block 2312, where sleep stage classifications for a plurality of time periods associated with a plurality of additional sleep sessions of the plurality of users can be obtained. This can be similar to the operation performed in block 2302, except that the plurality of previous sleep sessions of the plurality of users are the focus instead of the sleep session of block 2302.

[0212] In block 23142, a relative percentile breakdown of the total time spent in each sleep stage for each additional sleep session can be determined, for example, by dividing the total amount of time spent in each sleep stage for each sleep session by the total duration of that sleep session. In block 2316, an average of the relative percentile breakdowns for each sleep stage across all of the additional sleep sessions obtained in block 2312 can be determined to yield a representative demographic relative percentile breakdown for the sleep stage. Alternatively, the total amount of time spent in each sleep stage across all of the additional sleep sessions obtained in block 2312 can be divided by the total duration of all of those additional sleep sessions in order to obtain the demographic relative percentile breakdown. In block 2318, the GUI component, for example, the percentile breakdown 2024, can be modified to indicate each sleep stage classification and information indicating the percentile breakdown for each sleep stage from the sleep session of block 2302 as well as information indicating the demographic relative percentile breakdown for the sleep stage. This information can be presented such that the sleep stage percentile breakdown for the sleep session in question can be easily compared to the corresponding demographic relative percentile breakdown.

[0213] For example, Figure 24 depicted Figure 20The user interface has been further modified to include comparative data in percentile decomposition 2024, which indicates a representative range of peer relative percentile decompositions 2042' that may come from a larger user group. This representative peer relative percentile decomposition can be determined based on data from other users and can be filtered or down-selected to more closely resonate with the user's peer group. For example, if the user is female and in her 20s, a representative peer relative percentile decomposition can be determined based on data from other female users in the 20-29 age range. Conversely, if the user is male and in his 60s, the presented representative peer relative percentile decomposition can be used for men over 60.

[0214] It is important to understand that a representative relative percentile decomposition, or a representative relative percentile decomposition of a similar group, can be expressed as follows: Figure 22 The single value shown, or represented as such Figure 24 The range shown.

[0215] In some implementations, the sleep graph display, which is included in the portion of the GUI discussed above, may be modified to provide enhanced readability and understanding for the viewer. Figure 25 Sleep graphs are depicted that show indications of dual sleep stages for certain time intervals. Specifically, time intervals classified as associated with "awake" sleep stages, in addition to being displayed as associated with "awake" sleep stages, can also be displayed as associated with one or two adjacent non-awake sleep stages if their duration is short enough. In such an implementation, the display of time intervals in "awake" sleep stages may be toned down or otherwise unemphasized regarding the display of the same time intervals in non-awake sleep stages. This can prove useful to users as it helps prevent the sleep graph from becoming cluttered, even when presenting perfectly normal sleep data, by including a large number of very short intervals associated with "awake" sleep stages. In a sense, this is another way to implement the post-classification rules discussed earlier in this paper, in addition to retaining the original sleep stage classification of the time intervals in question even after reclassification. Figure 25 In the center, the thick black lines represent the user's sleep timeframes. Each of the first graphic elements 2550, such as horizontal line segments or other graphic elements extending along the horizontal axis, can be used to indicate specific time intervals corresponding to various sleep stages (described below regarding...). Figure 26(Discussed in more detail). The light gray second graphic element 2552 indicates time intervals associated with the wakeful sleep stage but less than the duration of the first threshold, for example, less than or equal to 3 minutes—these time intervals are still shown as associated with the "wakeful" sleep stage (due to the altitude corresponding to the "wakeful" sleep stage), but are displayed in a less emphasized manner compared to the highlighted time intervals. In some embodiments, the second graphic element 2552 may be connected to the first graphic element 2550 by a vertical extension line 2554, which may be a dashed line or a dotted line (or otherwise in a different format) to prevent them from being confused with the solid vertical lines used in the main sleep chart display that connect each pair of adjacent first graphic elements 2550.

[0216] Figure 26 A flowchart depicts a technique for providing such enhanced sleep graph display. In block 2602, sleep stage data for time intervals of sleep periods can be obtained. The sleep stage data can indicate the sleep stage associated with each such time interval. In block 2604, specific time intervals can be identified, each of which can be associated with one of the sleep stages and can span one or more time intervals of the sleep stage data. Specific time intervals associated with “awake” sleep stages can each be extended and timed with a corresponding time interval associated with the “awake” sleep stage and have a duration longer than a specified first threshold amount. Specific time intervals associated with non-awake sleep stages can each be extended and timed with one or more consecutive time intervals, each of which a) is associated with the specific time interval with its associated sleep stage or b) is associated with the “awake” sleep stage and is less than or equal to the first threshold amount. It is to be understood that, in various implementations, time intervals classified as associated with “artifacts” or “dislocated” sleep stages can be processed in such an enhanced sleep graph as being associated with “awake” sleep stages (such as those by…). Figure 25intervals associated with one of the non-wakeful sleep stages. In the latter case, at least for purposes of displaying the enhanced sleep graph, these spurious or off-wrist sleep stage intervals can be reclassified as having one of the non-wakeful sleep stages, such as the non-wakeful sleep stage of the adjacent interval. In some implementations, at least for purposes of displaying the enhanced sleep graph, the spurious or off-wrist sleep stage intervals can be reclassified as having the sleep stage (wakeful or non-wakeful) of the adjacent time interval. In still further implementations, the spurious and / or off-wrist sleep stages can have their own elevation in the enhanced sleep graph to allow them to be distinguished from other sleep stages that can be present. In this case, the spurious and / or off-wrist sleep stages can be depicted in a manner similar to that used to depict the "wakeful" sleep intervals, such as with a diminished or unemphasized indicator and with sleep intervals that categorize those displayed as contiguous in a non-emphasized manner (as done with the non-wakeful sleep stage intervals in the depicted enhanced sleep graph example). Alternatively, the spurious and / or off-wrist sleep stages can be depicted in the same manner as the non-wakeful sleep intervals.

[0217] It is to be understood that the phrase "one or more contiguous time intervals" as used herein includes instances in which there are multiple time intervals as well as instances in which there is a single time interval (and thus technically no other time interval is considered to be "contiguous" with that single time interval). In other words, the phrase "one or more contiguous time intervals" is to be understood to mean "a single time interval or two or more contiguous time intervals." Similarly, if used herein, the phrase "for each of one or more <items>" is to be understood to include both single item groups and multiple item groups, i.e., the phrase "for each of" is used in the sense used for programming languages to refer to each item in whatever group of items is involved. For example, if the group of items involved is a single item, "each" would refer to that single item (although the dictionary definition of "each" often defines the term to refer to "each of two or more things") and does not imply there must be at least two of those items.

[0218] For example, if the specified threshold is 3 minutes and a series of 10 time intervals listed in the table below are analyzed, a first salient time interval "A" can be defined to include interval 1, with a duration of 4 minutes, and can be associated with the "awake" sleep stage. A second salient time interval "B" can be defined to include intervals 2 through 6, as time intervals 2, 4, and 6 are all associated with the "light" sleep stage and are separated from each other only by time intervals 3 and 5, which are both associated with the "awake" sleep stage and have a duration less than the first threshold. A third salient time interval "C" can be defined to include time interval 7, which is associated with the "awake" sleep stage and has a duration greater than the first threshold; the third salient time interval "C" is associated with the "awake" sleep stage. Finally, a fourth salient time interval "D" can be defined to include time intervals 8 through 10. Time intervals 8 and 10 are associated with the "light" sleep stage and are separated by time interval 9, which has a duration less than the first threshold and is associated with the "awake" sleep stage.

[0219]

[0220] In block 2606, a hypnogram can be caused to be generated on a display. The hypnogram can display graphical elements that correspond to horizontally extending graphical elements of salient time intervals as Figure 25 shown in FIG. 26. In some implementations, each of the horizontally extending graphical elements can be connected to an adjacent horizontally extending graphical element by a vertical line or stave. In addition to the graphical elements corresponding to salient time intervals, the hypnogram can also display horizontally extending graphical elements corresponding to time intervals that have the "awake" sleep stage and a duration less than the first threshold. These additional graphical elements can be displayed in a less prominent manner, such as a lighter color or faded, compared to the graphical elements used to represent salient time intervals.

[0221] Figure 27 Another technique for producing an enhanced hypnogram is depicted. Blocks 2702 through 2706 correspond to blocks 2602 through 2606, and reference can be made to the discussion above for blocks 2602 through 2606 for details of these operations. However, in the technique of FIG. 27, Figure 27 the graphical elements representing time intervals that are associated with the "awake" sleep stage and have a duration less than the first threshold can have a vertically extending line that spans between the ends of these graphical elements and the ends of the graphical elements representing salient time intervals that are directly below (or above, if the hypnogram has the "awake" sleep stage below other sleep stages) the ends of the graphical elements representing the time intervals.

[0222] As will become readily apparent from the above discussion, sleep stage data for a sleep session of a person, such as each time interval when the person was in a particular sleep stage during the sleep session, can be presented to a user using a variety of graphical user interfaces. While the user interfaces discussed have been described using examples formatted for display on a secondary device like a smartphone, such user interfaces can also be arranged for display on a computer display, or in some embodiments, on a display of a wearable device if the wearable device includes a display. In some embodiments, the various graphical elements used in the user interfaces can be separate from one another and displayed on different user interface display pages, for example, if the user interface is presented on a wearable device, there can be one user interface display page displaying the percentile breakdown such as percentile breakdown 2024 in Figure 20 and another user interface display page displaying a hypnogram such as hypnogram 2026. In this case, the user can navigate between the two pages, but the user interface can not be configured to display both elements on the same user interface display page. However, as discussed earlier, in other embodiments, these elements can be presented on the same user interface display page, and the user interface display page can be scrollable so that only a portion is visible at any one time.

[0223] As discussed earlier herein, the data and processing used to customize and populate the user interfaces discussed herein can occur at a variety of different locations. Generally, the starting point for such data processing will always be the wearable device, as it is the source of the sensor data that the classifier will use to classify time intervals during a sleep session. However, once such sensor data is collected, the remaining processing can be performed at any of a variety of different locations, for example, by the wearable device, by a secondary device, by a backend system, etc. For example, the wearable device can perform some or all of the data collection and feature extraction, the secondary device or a backend system can provide the classifier functionality, and the secondary device can present the graphical user interfaces and perform the various operations associated therewith. Other permutations are possible.

[0224] Data Compression

[0225] Many of the features extracted from pulse-related data for a classifier can be based on interbeat interval durations for each time interval to be classified. In many implementations, these interbeat interval durations must be stored in the wearable device for a period of time in order for the interbeat interval durations to be analyzed later to further extract pulse data features. This later analysis can be performed by the wearable device, but alternatively can be performed by a secondary device or a backend system. Regardless, the wearable device can need to store a large amount of interbeat interval information. This can present storage and data transmission complexities in these devices as they can have limited memory and / or communication bandwidth. Furthermore, if the wearable device transmits the interbeat interval information in uncompressed form, battery life can be negatively impacted. To address these issues, some implementations can apply one or more data compression techniques to the interbeat interval data prior to storage and / or transmission, as discussed in more detail below. While the techniques discussed below are presented with respect to compressing interbeat interval information, these same techniques can generally also be applied to any time series data generated based on sensor data from a wearable device, such as motion data, respiration rate data, etc.

[0226] Figure 28 A flowchart depicting techniques for compressing interbeat interval data is depicted. In block 2802, time-varying data from a heart rate sensor can be obtained for a time period at a first sampling rate. For example, the first sampling rate can be on the order of 1 kHz, however other sampling rates are possible. In block 2804, the time-varying data can be analyzed to identify interbeat intervals, for example by identifying peaks in the time-varying data and measuring the distance between each pair of adjacent peaks. The resulting interbeat interval values can then be stored as an interbeat interval sequence. It can be assumed that interbeat intervals are limited to a range of 60 to 100 beats per minute for a typical human resting heart rate (typically observed during sleep monitoring), however in some athletes a heart rate as low as 40 beats per minute can be observed due to their high peak physical condition. Thus, when extracting the interbeat interval sequence, interbeat intervals can typically be expected to be between 1.5 seconds and 0.6 seconds for a typical subject. However, to capture outliers or atypically slow heart rates, it can be desirable to store interbeat interval durations up to 2 seconds. Under such an assumption, and using a 1 kHz sampling rate, the interbeat interval durations can be represented in their raw form as 11-bit values (2 seconds at 1 kHz = 2000 counts ~ 2048 bits = 2 11). It is to be understood that lower sampling rates can also be used, e.g., due to the band-limited nature of the heartbeat signal, a sampling rate as low as 100 Hz or even 25 Hz can be interpolated to a higher count resolution with reasonable confidence - this interpolated count value can then be the value stored, in addition to or instead of the raw signal count from the sensor. Thus, storing a sequence of raw interbeat intervals can require reserving a large block of 11-bit memory, which can consume limited memory resources (particularly if the interbeat interval sequence is stored for every time interval during one or more sleep sessions - this would require storing an 11-bit value for every heartbeat that occurs during each sleep session; an average sleep session can have approximately 30,000 such data points). Moreover, as noted above, transmitting such data to a secondary device can contribute significantly to battery charge depletion due to the size of the data transmitted.

[0227] To reduce the memory and / or transmission footprint of the interbeat interval sequence, the interbeat interval sequence data can be analyzed in block 2806 to determine a reference value associated with the interbeat interval values of the time period in question. The reference value can be determined in a variety of ways, however perhaps the most useful approach is to use the average interbeat interval value as the reference value, i.e., the arithmetic mean of the interbeat interval values of the time period. Other possibilities include other measures of central tendency (other than the arithmetic mean), such as the median interbeat interval value or the mode of the interbeat interval values, or in general, any value between the minimum and maximum interbeat interval values. However, the average value can be the easiest to calculate, and as will be seen in the example discussed later, provides some additional benefits compared to other types of reference values.

[0228] Once the reference value for the time period has been determined, the difference between each interbeat interval value in the time period and the reference value can be determined and used to create a new series of adjusted interbeat interval values. Thus, for example, if the reference value is 1 second (or 1000 counts) and the length of an example interbeat interval is 1.042 seconds (or 1042 counts), the difference between them would be 0.042 seconds (or 42 counts). An example table of a series of five interbeat intervals is provided below - it can be seen that the adjusted interbeat interval values require a much smaller bit size in order to be stored or transmitted.

[0229]

[0230] In box 2810, the adjusted sequence of jump intervals for the time period and a reference value for that time period can be stored for later reference. Additionally or alternatively, in box 2812, the same information can be transmitted to a remote device, such as an auxiliary device, for storage and / or analysis. The technique can then return to box 2802 to process additional time periods.

[0231] Figure 28 Techniques typically allow for lossless (or near-lossless) compression of inter-beat interval sequences because human heart rates generally have limited variability; for example, heart rates will generally not fluctuate beyond a certain level relative to a person's average heart rate. For instance, a person's heart rate may fluctuate by as much as ±30% of their average heart rate, but fluctuations beyond that range are likely minimal or nonexistent. Therefore, the original inter-beat interval sequence may include leading bit values, which, in practice, will hardly fluctuate due to the limited range of fluctuation in inter-beat interval values. However, for each inter-beat interval value stored, these leading bit values ​​are repeated. In effect, a reference value removes this repeated element from the stored data, allowing it to be stored separately from the adjusted inter-beat interval values. The adjusted inter-beat interval values ​​can later be transformed back to their original values ​​by adding the reference value to them. In some implementations, the adjusted inter-beat interval sequence may be stored as a representation with a predetermined bit size, such as a 9-bit representation. In this case, adjusted inter-beat intervals exceeding the available predetermined bit size can be limited to a maximum or minimum value supported by the predetermined bit size. For example, if a 9-bit representation is used, this allows adjusted inter-jump intervals to be between -256 and +255; if an adjusted inter-jump interval value of 260 is encountered, it can simply be stored as the maximum allowed value of 255 represented by 9 bits. In this case, when transforming the adjusted inter-jump interval back to the actual inter-jump interval value using a reference value, there will be some resolution loss in the remote data due to this truncation. However, this loss is likely minimal and has no significant impact on the overall usefulness of the stored inter-jump interval sequence.

[0232] It can be used Figure 28 Techniques to significantly reduce the data storage or transmission requirements associated with processing inter-jump interval sequences—for example, an 11-bit representation can be replaced by a 9-bit representation, which is nearly 20% more efficient in data storage or transmission.

[0233] refer to Figure 29 Another technique for compressing inter-jump interval data is discussed, and the figure shows a flowchart of this technique. Figure 29 In the middle, boxes 2902 to 2908 correspond to Figure 28 Boxes 2802 to 2808, and the earlier descriptions of these boxes also apply to the references.Figure 29 However, after determining the difference between the reference value (which is calculated in the art as the average of the interbeat interval values of the compressed time period) and each interbeat interval value in block 2908, the art can proceed to block 2910, where one of the adjusted interbeat intervals can be selected, and then to block 2912, where it can be determined whether the selected adjusted interbeat interval value is within a first threshold range. For example, the first threshold range can be -96 to +95 milliseconds (across 192 counts), and an adjusted interbeat interval value of -96 counts to +95 counts can be considered to be within the first threshold range. If it is determined in block 2912 that the selected adjusted interbeat interval value is within the first threshold range, the art can proceed to block 2914, where the selected adjusted interbeat interval value can be quantized according to a first quantization step, such as 4 milliseconds. In such quantization, the interbeat intervals can be mapped to "bins" according to the quantization step, in order to reduce the number of discrete values in the data set. For example, if the quantization step is 4 milliseconds (or, in this example, 4 counts), the range of data quantized according to this quantization step can be subdivided into bins each having a size commensurate with the quantization step. Thus, if values ranging from 1 to 20 are to be quantized according to a quantization step of 4, this would map values 1 to 4 to bin 1, values 5 to 8 to bin 2, values 9 to 12 to bin 3, etc. In this example, the quantization step of 4 reduces the range of electrical potential values by 75%. If it is determined in block 2912 that the selected adjusted interbeat interval value is not within the first threshold range, the art can instead proceed to block 2916, where the selected adjusted interbeat interval value can be quantized according to a second quantization step that is greater than the first quantization step, such as 20 milliseconds. Since interbeat interval values of a human heart rate typically exhibit a Gaussian distribution, this has the effect of preserving the resolution of most interbeat interval values, which will cluster around the mean value / reference value. Outlier interbeat interval values can be quantized at a coarser resolution, however there will be far fewer of these interval values than those quantized at the finer resolution.

[0234] For each time period having quantized adjusted interbeat interval values, the threshold range can be set as a fixed range, or can be dynamically adjusted based on the quantized adjusted interbeat interval values. For example, in some implementations, the same fixed range can be used as the threshold range for each time period for which adjusted interbeat interval values are quantized. However, in other implementations, the threshold range can be re-determined for each time period based on the quantized adjusted interbeat interval values. For example, the threshold range can be re-set for each time period such that a given percentage (or at least a given percentage) of the adjusted interbeat interval values, e.g., at least 80%, fall within the threshold range. In some such implementations, the quantization step can be adjusted to generally maintain the same total number of quantization bins and within / outside the threshold range.

[0235] In block 2918, it can be determined whether there are further interbeat interval values in the current time period that need to be quantized - if so, the technique returns to block 2910 and a new interbeat interval value in the series of interbeat interval values can be quantized. If not, the technique can proceed to block 2920 and / or 2922, where the quantized adjusted interbeat interval values for the time period being analyzed can be stored or transmitted in association with the reference value associated with the time period. The technique can then return to block 2902 to process another time period. Due to the quantization schemes discussed above, the bit values used to represent the interbeat interval values can be further reduced. For example, if using the first threshold range example quantization scheme of 4ms / 20ms and -96 to +95 counts discussed above, in conjunction with the conversion from 11 bit values to 9 bit values achieved by using the reference value (see blocks 2906 and 2908), the 9 bit values can be further reduced to, for example, 6 bit values. This is because there can only be a total of 64 quantization levels in the quantized data - 48 quantization levels in the case of a quantization step of 4ms and 16 quantization levels in the case of a quantization step of 20ms.

[0236] It is to be understood that additional quantization levels can be used as desired in order to provide a three, four, or N-level quantization scheme - each quantization level can be associated with a different threshold range. It is also to be understood that a single quantization level can be used at all times, however such implementations can not provide nearly as much data compression (or, if they do, the quality of the data being compressed can be significantly reduced due to the application of a large quantization step to a large amount of data).

[0237] It is also to be appreciated that while quantization in the example discussed above occurs after the adjusted inter-beat interval values are determined, quantization can also be performed on the unadjusted inter-beat interval values, for example, prior to the operation of block 2906. In such an implementation, quantization can be only single-step quantization, as the first threshold range of a multi-step quantization technique is centered around the reference value, until after block 2906.

[0238] It can be seen that the above-described techniques can result in a reduction of bit cost of nearly 50% to store the inter-beat interval value sequence, which can significantly reduce data storage and transmission resource costs.

[0239] Conclusions

[0240] As can be apparent from the above discussion, the techniques, systems, and apparatuses discussed herein can be used to facilitate sleep stage tracking using a wearable device. The techniques discussed herein can be practiced as a method, or can be embodied in the form of computer-readable instructions stored on one or more memory devices for controlling one or more processors to perform the techniques discussed herein. Such memory devices can be one or more portions of an apparatus or system, such as a portion of a wearable device, a secondary device, and / or a backend system, or can be a standalone memory device such as a disk or flash memory device that can be connected with a computing device in order to copy the instructions to the computing device or use the computing device to execute the instructions from the standalone memory device.

[0241] It is important to note that the concepts discussed herein are not limited to any single aspect or implementation of the concepts discussed herein, nor to any combination and / or permutation of these aspects and / or implementations. Each of the individual aspects and / or implementations of the present application can be used alone or in combination with one another. For simplicity, many of these combinations and permutations have not been discussed here, but are contemplated.

Claims

1. A computer-implemented method for labeling sleep stages, the computer- implemented method comprising: obtaining motion data of a user during a time window; obtaining cardiopulmonary pulse-related data of the user during the time window; processing the motion data and the cardiopulmonary pulse-related data with a classifier model to generate a label for one or more time segments within the time window, the label indicating that the user was in one of a plurality of different sleep stages during the one or more time segments; extracting first movement features from the motion data of the user; and extracting second pulse data features from the cardiopulmonary pulse-related data of the user, wherein the second pulse data features comprise: a variability of an envelope of interbeat intervals, a variability of detrended respiratory rate extracted from the cardiopulmonary pulse-related data, an inter-percentile span of heart rate extracted from the cardiopulmonary pulse- related data or from the envelope of interbeat intervals, a normalized detrended heart rate extracted from the cardiopulmonary pulse- related data or from the interbeat intervals, or a cross-correlation of each of one or more pulse shapes in the cardiopulmonary pulse-related data with a preceding pulse shape in the cardiopulmonary pulse- related data, wherein the pulse shapes are normalized to a common duration prior to the cross-correlation, wherein the classifier model is trained using sleep stage classifications determined by a human for a plurality of sleep study subjects based on motion data of the plurality of sleep study subjects and cardiopulmonary pulse-related data of the plurality of sleep study subjects, and wherein processing the motion data of the user and the cardiopulmonary pulse- related data of the user with the classifier model comprises processing the first movement features and the second pulse data features to generate the label.

2. The computer-implemented method of claim 1, wherein: obtaining motion data of the user comprises obtaining the motion data from a first sensor; and obtaining cardiopulmonary pulse-related data of the user comprises obtaining the cardiopulmonary pulse-related data from a second sensor different from the first sensor. the first sensor comprises an accelerometer and the second sensor comprises an optical sensor.

3. The computer-implemented method of claim 2, wherein, the optical sensor comprises a photoplethysmogram (PPG) sensor.

4. The computer-implemented method of claim 3, wherein, the first movement features comprise an amount of time that has elapsed since the motion data last exceeded a threshold amount of motion of the user.

5. The computer-implemented method of claim 1, wherein, the second pulse data features further comprise a minimum heart rate of the user during a threshold amount of time and a maximum heart rate of the user during the threshold amount of time.

6. The computer-implemented method of claim 1, wherein, the plurality of different sleep stages comprises a wake sleep stage, a light sleep stage, a deep sleep stage, and a rapid eye movement (REM) sleep stage.

7. The computer-implemented method of claim 1, wherein, the classifier model comprises a random forest classifier or a linear discriminant classifier model.

8. The computer-implemented method of claim 1, wherein, 9. A system for labeling sleep stages, the system comprising: a plurality of sensors; and one or more processors configured to perform operations comprising: obtaining motion data of a user during a time window via one or more of the plurality of sensors; ​ ​ acquiring cardiopulmonary pulse-related data of the user during the time window via one or more of the plurality of sensors; and processing the motion data and the cardiopulmonary pulse-related data with a classifier model to generate a label for one or more time segments within the time window, the label indicating that the user was in one of a plurality of different sleep stages during the one or more time segments; extracting first movement features from the motion data of the user; and extracting second pulse data features from the cardiopulmonary pulse-related data of the user, wherein the second pulse data features include: a variability of an envelope of interbeat intervals, a variability of detrended respiratory rate extracted from the cardiopulmonary pulse-related data, an inter-percentile span of heart rate extracted from the cardiopulmonary pulse-related data or from the envelope of interbeat intervals, a normalized detrended heart rate extracted from the cardiopulmonary pulse-related data or from the interbeat intervals, or a cross-correlation of each of one or more pulse shapes in the cardiopulmonary pulse-related data with a preceding pulse shape in the cardiopulmonary pulse-related data, wherein the pulse shapes are normalized to a common duration prior to the cross-correlation, wherein the classifier model is trained using sleep stage classifications determined by a human for a plurality of sleep study subjects based on motion data of the plurality of sleep study subjects and cardiopulmonary pulse-related data of the plurality of sleep study subjects, and wherein processing the motion data of the user and the cardiopulmonary pulse-related data of the user with the classifier model includes processing the first movement features and the second pulse data features to generate the label.

10. The system of claim 9, wherein: acquiring motion data of the user includes acquiring the motion data via a first sensor of the plurality of sensors; and acquiring cardiopulmonary pulse-related data includes acquiring the cardiopulmonary pulse-related data via a second sensor of the plurality of sensors, the second sensor being different from the first sensor.

11. The system of claim 10, wherein: the first sensor includes an accelerometer; and the second sensor includes an optical sensor.

12. The system of claim 9, wherein, the first movement features include an amount of time that has elapsed since the motion data last exceeded a threshold amount of motion of the user.

13. The system of claim 9, wherein, the second pulse data features further include a minimum heart rate of the user during a threshold amount of time and a maximum heart rate of the user during the threshold amount of time.

14. The system of claim 9, wherein, the plurality of different sleep stages includes a wake sleep stage, a light sleep stage, a deep sleep stage, and a rapid eye movement (REM) sleep stage.

15. The system of claim 9, wherein, the classifier model includes a random forest classifier or a linear discriminant classifier model.

16. The system of claim 9, wherein, processing the motion data and the cardiopulmonary pulse-related data with the classifier model includes: determining a first confidence score that the user was in a wake sleep stage during the one or more time segments; determining a second confidence score that the user was in a first stage of the plurality of different sleep stages during the one or more time segments; determining a third confidence score that the user was in a second stage of the plurality of different sleep stages during the one or more time periods; and labeling the one or more time periods with a label corresponding to a highest confidence score of the first confidence score, the second confidence score, and the third confidence score.

Images