Systems and methods for combining actigraphy data with heart rate data to enhance sleep staging

Abstract

The sleep staging platform disclosed herein enables accurate sleep stage classification by combining actigraphy data with heart rate data. The platform may use a multi-model approach that includes a first machine learning model to evaluate sleep stages using heart rate patterns from polysomnography datasets related to a wearable device. The platform may combine the outputs of the heart rate-based model with actigraphy features into a second model to further evaluate sleep stages. The platform may segment events based on timestamps to detect sleep stages and calculate clinically relevant sleep metrics such as total sleep time and wake after sleep onset. By using a multi-model approach, the platform reduces feature dimensionality, enabling practical training with small polysomnography datasets related to wearable devices. The combination of heart rate patterns and actigraphy features may enhance sleep staging accuracy, particularly when heart rate data is derived from motion-sensitive polysomnography signals.

Description

SYSTEMS AND METHODS FOR COMBINING ACTIGRAPHY DATA WITH HEART RATE DATA TO ENHANCE SLEEP STAGINGCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to US Provisional Application No. 63 / 664,127, titled “Performance Evaluation of the Verily Watch Sleep Suite for Digital Sleep Assessment Against In-Lab Polysomnography” and filed on June 25, 2024, which is incorporated by reference herein in its entirety.TECHNICAL FIELD

[0002] Various embodiments concern computer programs and associated computer- implemented techniques for combining actigraphy data with heart rate data to enhance sleep staging.BACKGROUND

[0003] Sleep staging applications enable users to monitor sleep-related indicators, measures, and events over time. For example, sleep staging applications may receive information from heart rate sensors, motion sensors, and other wearable devices. These applications can record sleep-related activities, such as periods of wakefulness, light sleep, deep sleep, and rapid eye movement (REM) sleep, along with corresponding times. In some cases, sleep staging applications enable users to log their sleep quality, duration, and other relevant metrics. Traditionally, sleep staging applications allow users to view details regarding their sleep patterns over time, as well as basic analytics regarding the nature of their sleep.

[0004] For example, sleep staging applications can include personalized sleep plans, sleep disorder detection, or heart rate alerts tailored to an individual user. A traditional sleep staging application enables a user to ensure that they are meeting their sleep goals and identifying potential sleep disorders such as insomnia or sleep apnea. As such, sleep staging applications can improve an individual’s health by enabling users to adhere to better sleep hygiene and lifestyle plans. Sleep staging applications traditionally evaluatesleep stages using heart rate data from polysomnography (PSG) datasets, which can present challenges due to susceptibility to motion artifacts and physiological variances among individuals. One solution to gaps in sleep staging using only heart rate data is integrating actigraphy data in a multimodal approach. While potentially more accurate, these approaches are often too complex and require extensive dimensionality and computational resources. Consequently, existing sleep staging applications, without simplified and efficient evaluation methods, can lose effectiveness in improving users’ sleep health outcomes.BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Figure 1 illustrates a network environment that includes a sleep staging platform that is executed by a computing device.

[0006] Figure 2 illustrates an example of a computing device that is able to implement a sleep staging platform that is designed to evaluate the engagement of a human with a computer program executing on a computing device.

[0007] Figure 3 depicts an example of a communication environment that includes a sleep staging platform that is configured to receive several types of data.

[0008] Figure 4 depicts another example of a communication environment that includes a sleep staging platform that is configured to receive several types of data.

[0009] Figure 5 includes a schematic diagram of an approach to enhancing sleep stages and predicting sleep stage metrics with a computer program executing on a computing device.

[0010] Figure 6 includes a schematic diagram of an approach to enhancing sleep stages with collected sensor data and a multiple classifier model.

[0011] Figures 7A-F include schematics of simulated sleep stage data as a function of time and reference sleep metric types for a multiple classifier model.

[0012] Figure 8 includes a summary of sleep versus wake state classification metrics for a multiple classifier model.

[0013] Figures 9A-B include summaries of performance metrics for various derived sleep metrics for a multiple classifier model.

[0014] Figure 10 includes summaries of classification metrics for various sleep stages predicted by a multiple classifier model.

[0015] Figure 11 includes a schematic of a confusion matrix for various sleep stages predicted by a multiple classifier model compared to reference sleep stages.

[0016] Figure 12 includes a schematic of a timestamp jittering analysis, in which epochs from wearable device data were deliberately shifted by various time intervals.

[0017] Figure 13 is a flow diagram of a process for enhancing sleep staging by integrating actigraphy data with heart rate data.

[0018] Figure 14 is a flow diagram of a process for enhancing sleep staging by integrating actigraphy data with heart rate data.

[0019] Figure 15 is a flow diagram of a process for training a model to integrate actigraphy data with heart rate data to improve sleep staging.

[0020] Figure 16 includes a block diagram of a processing system in which at least some operations described herein can be implemented.

[0021] Various features of the technology will become more apparent to those skilled in the art from a study of the Detailed Description in conjunction with the drawings. In the drawings, embodiments are illustrated by way of example and not limitation for the purpose of illustration. Those skilled in the art will recognize that alternative embodiments may be employed without departing from the principles of the present disclosure. Accordingly, while specific embodiments are shown in the drawings, the technology is amenable to various modifications.DETAILED DESCRIPTION

[0022] Sleep staging applications enable holistic tracking of sleep-related behaviors to improve health outcomes. For example, a sleep staging application can include features for tracking sleep quality, duration, and stages, such as light sleep, deep sleep, and REM sleep. Because sleep behaviors influence overall health, such applications enable users to improve adherence to sleep hygiene and lifestyle plans based on the information provided by the application. In some cases, the sleep staging application can leverage sensor-based information, such as wearable device data (e.g., heart rate trackers within smartwatches), motion sensors, and other such devices, to improve the effectiveness of the application in tracking, measuring, and / or improving sleep health outcomes. A human’s engagement with the sleep staging application enables effective evaluation of a human’s sleep patterns and related activities. However, in situations where a human’s engagement with the sleep staging application decreases, associated analytics can decrease in accuracy. For example, if a human fails to wear the device consistently or does not input relevant sleep- related information, the sleep staging application may ineffectively track sleep progression, thereby reducing the application’s effectiveness.

[0023] Conventional sleep staging applications traditionally rely on heart rate data from polysomnography (PSG) datasets, which can present challenges due to susceptibility to motion artifacts and physiological variances among individuals. Evaluating sleep stages using only heart rate data can be insufficient for capturing the full complexity of sleep patterns. For example, using only heart rate data can be less accurate for individuals with abnormal heart rates, who may require more precise monitoring. One solution to these gaps is the integration of actigraphy data in a multimodal approach. Actigraphy data, which includes information on body movements, can provide a more comprehensive picture of a user’s sleep patterns by offering additional context that helps distinguish between different sleep stages more accurately. For instance, periods of low movement combined with specific heart rate patterns can indicate deep sleep, while higher movement levels may correspond to lighter sleep stages or wakefulness. However, existing sleep staging applications that rely solely on heart rate data or overly complex multimodal models may not be accurate or efficient in improving users’ sleep health outcomes due to the lack ofcomprehensive data and the complexity and computational demands that hinder practical application and effectiveness.

[0024] Traditionally, healthcare professionals consider multiple physiological factors, including heart rate, to assess sleep stages accurately. For example, healthcare professionals may evaluate electrocardiogram (ECG) or electroencephalogram (EEG) data in the context of other physiological signals collected during an overnight sleep study and patient history, which allows for a more nuanced understanding of sleep stages and diagnosis. However, manual interpretation can be limited by human error and the inability to continuously monitor patients over extended periods, such as every night based on data from a wearable device. Automating the integration of actigraphy data with heart rate data allows for continuous, real-time monitoring and analysis, thereby capturing subtle variations and patterns that may be missed during manual assessments.

[0025] Furthermore, conventional sleep staging applications do not provide methods to simplify and optimize the evaluation process of complex sleep data attained, for example, from a wearable device. Without efficient evaluation methods, these applications may struggle to maintain accuracy and reliability in tracking sleep patterns. Simplified and efficient evaluation methods that integrate actigraphy data with heart rate data can improve the overall effectiveness of sleep staging applications, leading to better health outcomes for users.

[0026] The sleep staging platform and associated systems and methods disclosed herein leverage computational models - and more specifically, machine learning (ML) models - to deduce sleep stages for users wearing a wearable device. For example, the sleep staging platform can extract data from PSG datasets and use two models in tandem to integrate actigraphy data (e.g., as measured through an accelerometer of the wearable device) with heart rate data, thereby enhancing sleep staging accuracy. Integrating actigraphy information into sleep staging models, which have traditionally relied solely on ECG data from PSG datasets, can significantly improve the precision of sleep stage determination by providing additional context about the user’s physical movements, thereby enhancing the overall accuracy and reliability of sleep analysis. However, challenges such as the limited availability of usable accelerometry data and inconsistentaccelerometry signals in existing datasets exist. To address these issues, the proposed solution separates heart rate and actigraphy data into two distinct ML models.

[0027] The first ML model can serve as a primary classifier, leveraging extensive ECG data from large PSG datasets traditionally used in sleep staging models to generate initial sleep stage predictions. The second ML model can serve as a secondary classifier, training on the output of the primary classifier and incorporating actigraphy features from accelerometry signals. The secondary classifier can output refined sleep stage predictions that integrate both heart rate and actigraphy data, enhancing accuracy by including movement information typically lacking from PSG datasets. Furthermore, the secondary classifier can handle features of a lower dimensionality than the first classifier, allowing the sleep staging platform to efficiently leverage the predictive abilities of the primary classifier while utilizing actigraphy data to refine the initial classification. Such an approach enables training on smaller datasets (e.g., 50 nights of data) and ensures computational efficiency for on-device training (e.g., for wearable devices).

[0028] The input for the primary classifier can comprise comprehensive heart rate data, such as ECG-derived heart rates or interbeat intervals, and the output may include sleep stage predictions, categorizing stages into four classes such as wakefulness, light sleep, deep sleep, and REM sleep. Large PSG datasets can be gathered using a combination of publicly available and proprietary sources, producing a diverse dataset optimal for training a generalizable ML model. Pooling data from multiple PSG studies can enhance the diversity of the dataset. During training, instantaneous heart rates can be derived from the ECG signals over a specified time window and provided as input to the primary classifier, with the output being sleep stages represented as binary vectors. Alternatively, reliable PSG data can also be used to derive heart rate data for input into the primary classifier. For example, during inferencing, the heart rates can be derived from PSG signals, and the primary classifier can output four-class prediction probabilities for each time segment of sleep.

[0029] The secondary classifier can combine initial sleep stage predictions from the primary classifier with manually crafted actigraphy data features to produce refined and more accurate sleep stage predictions. Public datasets often lack usable accelerometrydata or provide only actigraphy counts, making them less useful for predicting sleep stages. Therefore, a curated smaller dataset with accelerometry data can be used to train the secondary classifier. Given the low feature dimensionality and the incorporation of heart rate information, the secondary classifier can utilize a simple ML algorithm, such as random forests or gradient-boosted trees, trained using a small PSG dataset where participants also wear the specific wearable device. In some embodiments, the secondary classifier can include additional statistics or data features such as accelerometry statistics (e.g., PPG entropy), activity classifier outputs, and other features derived from PSG data (e.g., frequency spectrum-based features). For example, actigraphy features, such as accelerometer correlation and PPG entropy, can be calculated for each time window using PSG and accelerometer signals from the wearable device. Additionally or alternatively, sensors like electrodermal activity (EDA) or skin conductance sensors can be incorporated into a wearable device and used in training the secondary classifier.

[0030] The disclosed sleep staging platform offers technical advantages over conventional systems in sleep staging applications. For example, the sleep staging platform enables the detection of sleep stages and the calculation of clinically relevant sleep metrics, such as total sleep time and wake after sleep onset, through the integration of actigraphy data. By combining accelerometry and PSG data, the sleep staging platform captures comprehensive movement information during sleep, with accelerometry being sensitive to smaller movements and PSG to larger movements. By using a multi-model approach, the sleep staging platform also reduces feature dimensionality, enabling practical training with small PSG datasets related to wearable devices. Because heart rates can be reliably derived from PPG signals collected at various anatomical locations, such as the wrist and finger of a wearable device, and such heart rates are comparable to heart rates derived from ECG signals collected at the chest area, the trained machine learning models can be generalized across different wearable devices and anatomical locations. The integration of these data sources allows the sleep staging platform to leverage combined models to accurately calculate clinically relevant sleep metrics. These metrics can help determine trends in sleep stages associated with movement, overcoming the limitations of using heart rate data alone, which may be affected by external factors likephysical activity or stress, thereby improving overall patient health outcomes by providing more accurate and comprehensive sleep stage assessments.Terminology

[0031] References in the present disclosure to “an embodiment” or “some embodiments” mean that the feature, function, structure, or characteristic being described is included in at least one embodiment. Occurrences of such phrases do not necessarily refer to the same embodiment, nor do they necessarily refer to alternative embodiments that are mutually exclusive of one another.

[0032] The term “based on” is to be construed in an inclusive sense rather than an exclusive sense. That is, in the sense of “including but not limited to.” Thus, the term “based on” is intended to mean “based at least in part on” unless otherwise noted.

[0033] The terms “connected,” “coupled,” and variants thereof are intended to include any connection or coupling between two or more elements, either direct or indirect. The connection or coupling can be physical, logical, or a combination thereof. For example, elements may be electrically or communicatively connected to one another despite not sharing a physical connection.

[0034] The term “module” may refer broadly to software, firmware, hardware, or combinations thereof. Modules are typically functional components that generate one or more outputs based on one or more inputs. A computer program may include or utilize one or more modules. For example, a computer program may utilize multiple modules that are responsible for completing different tasks, or a computer program may utilize a single module that is responsible for completing multiple tasks.

[0035] When used in reference to a list of items, the word “or” is intended to cover all of the following interpretations: any of the items in the list, all of the items in the list, and any combination of items in the list.

[0036] The term “sleep stage” can be used to refer to various phases of sleep, such as light sleep, deep sleep, and REM sleep. For example, “sleep stage” can include a quantitative “sleep metric” that includes an indication of sleep quality, duration, and / or stages. In some embodiments, “sleep stage” includes a qualitative “sleep level” thatcategorizes sleep stages by characteristics and impact on overall sleep quality. The sleep staging platform described herein enables determination of sleep stages and overall health outcomes associated with the sleep stages predicted based on the integration of actigraphy data and heart rate data. By leveraging actigraphy and heart rate data sources, the platform can provide a comprehensive analysis of sleep patterns, thereby improving the accuracy and effectiveness of sleep health assessments and interventions.

[0037] The term “heart rate data” may be used to refer to data related to the measurement of heartbeats over time. For example, “heart rate data” can include information collected from ECG signals, which are traditionally obtained from PSG studies and provide detailed insights into the electrical activity of the heart. Additionally, “heart rate data” can include measurements derived from PPG sensors in wearable devices, which detect blood volume changes in the microvascular bed of tissue to estimate heart rate. By leveraging both ECG signals from PSG datasets and PPG signals from wearable devices, the heart rate data used in the machine learning models described herein can offer a comprehensive view of cardiovascular activity, which is crucial for accurate sleep staging and overall health assessments.

[0038] The terms “actigraphy,” “actigraphy features,” “actigraphy data,” or “actigraphy signals” may be used to refer to data related to physical movements and activities, typically measured by sensors in wearable devices. For example, “actigraphy” can include activity data collected from accelerometers embedded in wearable devices, such as smartwatches or fitness trackers, which detect and record body movements. Additionally, “actigraphy” can include data derived from PPG sensors, which measure blood volume changes in the microvascular bed of tissue, providing information on heart rate and other physiological parameters. By combining accelerometry and PPG data, wearable devices can capture comprehensive information about a user’s physical activity and physiological state, which can be used to enhance the accuracy of sleep staging and overall health assessments.

[0039] The terms “ML model” and “classifier” may be used interchangeably in the context of the multi-model approach for classifying sleep stages, as the approach generally involves the application of classification models. However, those skilled in the art will recognize that the features of embodiments described in the context of “classifiers” may besimilarly applicable if another type of ML model is used. Accordingly, an “ML model” in the context of the sleep staging platform can include algorithms and statistical models that analyze and interpret complex data patterns to predict sleep stages. Meanwhile, a “classifier” in the context of the sleep staging platform can refer to a type of ML model designed to classify or categorize data into distinct sleep stages such as light sleep, deep sleep, and / or REM sleep. The multi-model approach described herein involves the use of ML models - some or all of which may be classifiers - to integrate various data sources, such as actigraphy data and heart rate data, to enhance the accuracy of sleep stage predictions.Overview of Sleep Staging Platform

[0040] Figure 1 illustrates a network environment 100 that includes a sleep staging platform 102 that is executed by a computing device 104. An individual (also referred to as a “user” or a “human”) can interact with the sleep staging platform 102 via interfaces 106. For example, a patient may be able to access an interface through which information regarding sleep patterns or previous sleep data, such as digital measurements associated with sleep stages, can be reviewed. As another example, a healthcare professional may be able to access an interface through which information regarding patients, such as sleep stage data, heart rate information, biometric information (e.g., height and weight), or activity information (e.g., fitness or therapeutic procedures carried out by the human), can be reviewed. In some implementations, the interfaces 106 enable individuals, such as patients, to communicate directly with healthcare professionals (e.g., through a peripheral, such as a keyboard or a mouse). For example, the interfaces 106 enable a user to present a question relating to a sleep disorder diagnosis (e.g., relating to normal sleep patterns associated with a person of a similar age and weight). Additionally or alternatively, the interfaces 106 enable an individual to review physiological data, examine outputs produced by the sleep staging platform 102, and manage preferences. Some interfaces may be configured to facilitate interactions between patients and healthcare professionals, while other interfaces may be configured to serve as informative dashboards for patients or healthcare professionals.

[0041] The physiological data obtained by the sleep staging platform 102 could be associated with the individual accessing the interfaces 106 or some other person. For example, the interfaces 106 may enable a person concerned about their sleep health to view their own sleep data. Alternatively, the interfaces may enable an individual to view sleep data associated with another person. In such embodiments, the individual may be a healthcare professional who is responsible for monitoring, managing, or treating the other person. Examples of healthcare professionals include physicians, nurses, sleep specialists, and the like.

[0042] As shown in Figure 1 , the sleep staging platform 102 can reside in a network environment 100. Thus, the computing device 104 on which the sleep staging platform 102 resides can be connected to one or more networks 108A-B. Depending on its nature, the computing device 104 could be connected to a personal area network (“PAN”), local area network (“LAN”), wide area network (“WAN”), metropolitan area network (“MAN”), or cellular network. For example, if the computing device 104 is a computer server, then the computing device 104 may be accessible to users via respective mobile phones that are connected to the Internet via LANs. The physiological data to be examined by the sleep staging platform 102 may be generated by the respective mobile phones (e.g., sensors included in the respective mobile phones) or acquired by the respective mobile phones. Additionally or alternatively, the physiological data can be generated by wearable devices such as smartwatches, which include sensors to monitor physiological signals associated with sleep patterns, heart rate, and other relevant metrics. The wearable devices can transmit the collected data to the computing device 104 via the connected networks for analysis by the sleep staging platform 102.

[0043] Additionally or alternatively, the computing device 104 may be connected to one or more other computing devices over a short-range wireless connectivity technology, such as Bluetooth®, Near Field Communication (“NFC”), Wi-Fi® Direct (also referred to as “Wi-Fi P2P”), and the like. For example, the sleep staging platform 102 could be embodied as a mobile application that is executed by a mobile phone. In such embodiments, the mobile phone may be communicatively connected — via a wireless communication channel — to a source from which to acquire physiological data. The source could be awatch, fitness tracker, or another wearable computing device, such as a portable electronic sleep monitor. The physiological data could include sleep patterns, heart rate, and movement data, which are essential for sleep staging prediction. Alternatively, the content (e.g., sleep-related data such as heart rate or actigraphy data) could be obtained from another computer program executing on the mobile phone. For example, the sleep-related data could be acquired from another mobile application executing on the mobile phone or the operating system of the mobile phone, which collects and processes sleep-related metrics. Additionally or alternatively, the content can be acquired from databases and / or servers that are communicably linked to the computing device 104 (e.g., through a network). For example, the computing device 104 can communicate with databases or servers storing updated healthcare information from a regulatory authority (e.g., the Centers for Disease Control).

[0044] The interfaces 106 can be accessible via a web browser, desktop application, mobile application, or another form of a computer program. For example, a patient may be able to access interfaces through which information regarding their own sleep health, such as digital sleep biomarkers, sleep stage predictions, or physiological information like heart rate and movement data, is provided by a mobile application executing on a mobile device. In some implementations, the patient can access interfaces that display insights derived from the multimodal solution for predicting sleep stages, which involves a primary classifier that processes heart rate data and a secondary classifier that processes accelerometry data. Accordingly, the interfaces 106 generated by the sleep staging platform 102 may be accessible on various computing devices, including mobile phones, tablet computers, desktop computers, and the like, providing comprehensive insights from both classifiers.

[0045] Generally, the sleep staging platform 102 is executed — at least partially — by a cloud computing service operated by, for example, Amazon Web Services®, Google Cloud Platform™, or Microsoft Azure®. Thus, the computing device 104 may be representative of a computer server that is part of a server system 1 10. Often, the server system 1 10 comprises multiple computer servers. These computer servers can include different types of data (e.g., physiological data and information regarding patients, such as name, demographic information, disease classification, etc., as well as content from a corpus ofinformation, such as ECG data from PSG datasets), algorithms for processing incoming data, and other assets. Those skilled in the art will recognize that these data could also be distributed among the server system 110 and one or more computing devices. As an example, data that is input by, or related to, patients may be stored on, and processed by, their own computing devices for security or privacy purposes.

[0046] Components of the sleep staging platform 102 could also be hosted locally. That is, part of the sleep staging platform 102 may reside on the computing device used to access one of the interfaces 106. For example, the sleep staging platform 102 may be embodied as a mobile application executing on a mobile phone, as mentioned above. Note, however, that the mobile application may be communicatively connected to the server system 110 on which other components of the sleep staging platform 102 are hosted.

[0047] Figure 2 illustrates an example of a computing device 200 that is able to implement a sleep staging platform 212 designed to analyze heart rate and actigraphy data and predict sleep stages. As shown in Figure 2, the computing device 200 can include a processor 202, memory 204, display mechanism 206, communication module 208, and sensor suite 210. Each of these components is discussed in greater detail below.

[0048] Those skilled in the art will recognize that different combinations of these components may be present depending on the nature of the computing device 200. For example, if the computing device 200 is a computer server that is part of a server system (e.g., server system 110 of Figure 1 ), then the computing device 200 may not include the display mechanism 206 or sensor suite 210. Conversely, if the computing device 200 is a mobile phone, then the computing device 200 can include the display mechanism 206 and sensor suite 210.

[0049] As further discussed below, the computing device 200 on which the sleep staging platform 212 resides may not include sensors in some embodiments. In such embodiments, the data examined by the sleep staging platform 212 could instead be generated by one or more sensors 222A-N that are external to the computing device 200. For example, the computing device 200 may be a mobile phone, and the sensors 222A-N may be included in a watch or fitness tracker that is communicatively connected to the computing device 200.

[0050] The processor 202 can have generic characteristics similar to general-purpose processors, or the processor 202 may be an application-specific integrated circuit (“ASIC”) that provides control functions to the computing device 200. As shown in Figure 2, the processor 202 can be coupled to all components of the computing device 200, either directly or indirectly, for communication purposes.

[0051] The memory 204 can comprise any suitable type of storage medium, such as static random-access memory (“SRAM”), dynamic random-access memory (“DRAM”), electrically erasable programmable read-only memory (“EEPROM”), flash memory, or registers. In addition to storing instructions that can be executed by the processor 202, the memory 204 can also store data generated by the processor 202 (e.g., when executing the modules of the sleep staging platform 212). Note that the memory 204 is merely an abstract representation of a storage environment. The memory 204 could comprise actual integrated circuits (also called “chips”).

[0052] The display mechanism 206 can be any mechanism that is operable to visually convey information to a user. For example, the display mechanism 206 can be a panel that includes light-emitting diodes (“LEDs”), organic LEDs, liquid crystal elements, or electrophoretic elements. As further discussed below, outputs produced by the sleep staging platform 212 (e.g., through execution of its modules) can be posted to the display mechanism 206 for review by a user of the computing device 200.

[0053] The communication module 208 may be responsible for managing communications external to the computing device 200. The communication module 208 can be wireless communication circuitry that is able to establish wireless communication channels with other computing devices. Examples of wireless communication circuitry include 2.4 gigahertz (“GHz”) and 5 GHz chipsets compatible with Institute of Electrical and Electronics Engineers (“IEEE”) 802.11 — also referred to as “Wi-Fi chipsets.” Alternatively, the communication module 208 may be representative of a chipset configured for Bluetooth, NFC, and the like. Some computing devices — like mobile phones, tablet computers, and the like — are able to wirelessly communicate via separate channels, while other computing devices — like watches and fitness trackers — tend to wirelessly communicate via a single channel. Accordingly, the communication module 208 may beone of multiple communication modules implemented in the computing device 200, or the communication module 208 may be the only communication module implemented in the computing device 200.

[0054] The nature, number, and type of communication channels established by the computing device 200 — and more specifically, the communication module 208 — can depend on (i) the sources from which data is received by the sleep staging platform 212 and (ii) the destinations to which data is transmitted by the sleep staging platform 212. Assume, for example, that the sleep staging platform 212 resides on a mobile phone in the form of a mobile application. In such embodiments, the communication module 208 can communicate with sensors 222A-N external to the computing device 200 from which to obtain data. Moreover, the communication module 208 may communicate with a server system (e.g., server system 110 of Figure 1 ) to which analyses of the data— or the data itself — are transmitted.

[0055] Often, various sensors are implemented in the computing device 200. Collectively, these sensors may be referred to as the “sensor suite” 210 of the computing device 200. For example, the computing device 200 may include a motion sensor whose output is indicative of motion of the computing device 200 as a whole. Examples of motion sensors include accelerometers and gyroscopes. In some embodiments, the motion sensor is implemented in an inertial measurement unit (“IMU”) that measures the force, angular rate, or orientation of the computing device 200. The IMU may accomplish this through the use of one or more accelerometers, one or more gyroscopes, one or more magnetometers, or any combination thereof. As specific examples, the IMU could be a 6- axis IMU that draws low current and therefore is suitable for “always-on” applications in battery-driven computing devices, or the IMU could be a 3-axis IMU that includes logiclevel shifting circuitry that can readily interface with a microcontroller. As another example, the computing device 200 may include an ambient light sensor whose output is indicative of the amount of light in the ambient environment.

[0056] As mentioned above, data could also be acquired from sensors 222A-N that are external to the computing device 200. These sensors 222A-N could be included in another computing device, such as a watch or fitness tracker, that is directly connected tothe computing device 200. Alternatively, these sensors 222A-N could be discrete sensing units that are directly or indirectly connected to the computing device 200. For example, sensor 222A may be a pulse oximeter that monitors the oxygen saturation of the user of the computing device 200 or another person for the purpose of creating a photoplethysmogram (“PPG”). To monitor the oxygen saturation, sensor 222A may be placed on a thin part of a living body, usually a fingertip or earlobe, and then pass two wavelengths of light through that body part toward a photodetector. The photodetector can measure the changing absorbance at each wavelength, allowing sensor 222A to determine the absorbances due to the pulsing of arterial blood through that body part.

[0057] For convenience, the sleep staging platform 212 is referred to as a “computer program” that resides within the memory 204. However, the sleep staging platform 212 could comprise software, firmware, or hardware that is implemented in, or accessible to, the computing device 200. In accordance with embodiments described herein, the sleep staging platform 212 can include a processing module 214, first classifier module 218, second classifier module 220, computation module 216, analytics module 222, and graphical user interface (“GUI”) module 224. These modules could be integral parts of the sleep staging platform 212, or these modules could be logically separate from the sleep staging platform 212 but operate “alongside” it. Together, these modules enable the sleep staging platform 212 to accurately determine sleep stages for a user based on historical heart rate data and actigraphy data from a wearable device, thereby enhancing the precision of sleep staging predictions and improving overall sleep analysis. As mentioned above, the given individual could be the user of the computing device 200 or another person.

[0058] The processing module 214 can process data obtained by the sleep staging platform 212 into a format that is suitable for the other modules. For example, the processing module 214 can apply operations to sensor data obtained from the sensor suite 210 or sensors 222A-N in preparation for analysis by the other modules of the sleep staging platform 212. Additionally or alternatively, the processing module 214 can compile, process, or transform information associated with data recorded within an associated computer program (e.g., a sleep tracking application), such as interactions with thecomputer program (e.g., logged sleep patterns or physiological activities performed by a user). For example, the processing module 214 can filter or alter the data such that the data can be more readily analyzed. As another example, the processing module 214 may parse the data in order to temporally align the dataset obtained from each source. The processing module 214 can manage the flow of data through the sleep staging platform 212, including ingestion of heart rate data from PCG datasets, ingestion from wearable devices, synchronization of heart rate data collected as PPG and actigraphy data collected as accelerometry signals, and temporary data storage for processing. Accordingly, the processing module 214 may be responsible for ensuring that the appropriate data is accessible to the other modules of the sleep staging platform 212, thereby enabling accurate sleep stage predictions.

[0059] The computation module 216 (also called a “computation system”) may be responsible for deriving different datasets based on the processed data received from the processing module 214. As further discussed below, these different measurements may depend on the nature of the insights to be surfaced by the sleep staging platform 212 (e.g., by the first classifier module 218 and / or the second classifier module 220). For example, these different measurements (e.g., datasets) may be representative of different physiological parameters, movement data, sleep stages, or types of sleep patterns, such as groupings or counts of sleep-related events of the same type. The computation module 216 can compute, infer, or otherwise establish measurements for different segments of processed data. These segments may be referred to as “windows” or “time windows” of processed data. The computation module 216 can manage the computational tasks required for running the ML models, including the first classifier module 218 and the second classifier module 220. This management may involve utilizing cloud computing or local servers, supporting parallel processing of the classifiers, and ensuring scalability based on computational needs. The computation module 216 can ensure that the sleep staging platform 212 operates efficiently and can handle the integration of heart rate and actigraphy data to provide accurate and refined sleep stage predictions.

[0060] The first classifier module 218 (also called a “heart rate-based sleep stage classifier”) may be responsible for deriving initial sleep stage predictions based on theprocessed data received from the processing module 214. In some embodiments, the first classifier module 218 receives processed heart rate data in the form of ECG data from PSG datasets that has been synchronized and aligned to accommodate variations in signal timing. For example, the processing module 214 can process the PSG datasets to clean and normalize ECG data by removing noise and artifacts and extracting relevant features such as heart rate variability, average heart rate, peak intervals, and other statistical measures that provide insights into the sleep patterns relevant for sleep staging. The extracted features can be fed into a first ML model trained on heart rate data to predict sleep stage classifications, including light sleep, deep sleep, REM sleep, and wakefulness. The first ML model may serve as a first classifier that produces initial sleep stage predictions, which are then refined with actigraphy data, as described in more detail with reference to the second classifier module 220. The output of the first classifier module 218 is a set of sleep stage predictions that are made accessible to other modules within the sleep staging platform 212 for further analysis, reporting, and providing feedback to the user.

[0061] The second classifier module 220 (also called a “heart rate- and actigraphy- based sleep stage classifier”) can receive initial sleep stage predictions from the first classifier module 218 and processed actigraphy data from the processing module 214. For example, the processing module 214 may clean and normalize accelerometer signals and / or PPG signals from actigraphy data, extracting relevant features focused on movement patterns. The extracted movement features, along with the initial sleep stage predictions, can then be fed into a second ML model. The second ML model may integrate the actigraphy data with the initial sleep stage predictions to refine and enhance the accuracy of the sleep stage classifications. The refinement process ensures accurate and smooth transitions between sleep stages by addressing gaps that the movement data can fill in. The output of the second classifier module 220 may be a refined set of sleep stage predictions that are made accessible to other modules within the sleep staging platform 212 for further analysis, reporting, and providing feedback to the user.

[0062] The analytics module 222 (also called an "analytics system") can implement an analytics engine that operates by executing two programs in sequence. As furtherdiscussed below, the first program may provide a method for programmatically determining parameters based on the processed data output by the processing module 214 and the computation module 216 and implement one or more methods for determining initial sleep metrics and predictions. Meanwhile, the second program may implement additional methods for determining enhanced sleep metrics or sleep quality levels associated with the processed data and initial sleep metrics and predictions. The first program, also referred to as the “first classifier,” is implemented by the first classifier module 218 and is responsible for making the initial sleep stage predictions based on the available data. The second program, also referred to as the “second classifier,” is implemented by the second classifier module 220 and is responsible for determining enhanced sleep metrics or sleep quality levels by incorporating additional data, such as actigraphy data, to refine and improve the accuracy of the initial sleep stage predictions. Additionally or alternatively, the analytics module 222 can analyze the classified sleep stages to provide insights and trends. This includes data analysis to identify patterns, report generation for visualizations, and trend analysis to detect long-term sleep patterns. The analytics module 222 ensures that the sleep staging platform 212 can deliver comprehensive insights into users’ sleep behaviors, facilitating a better understanding and management of sleep health.

[0063] The GUI module 224 can be responsible for generating interfaces that are viewable on the display mechanism 206. Various types of information can be presented on these interfaces. For example, sleep-related health information and / or intervention procedures (e.g., intervention messages) that are calculated, derived, or otherwise obtained by the computation module 216, the first classifier module 218, the second classifier module 220, and / or any of the other modules described herein may be presented on an interface for display to the user. The GUI module 224 can offer a user-friendly interface for users to interact with the platform, displaying sleep stage predictions and physiological data, providing visualization tools like graphs and charts, and allowing user interaction for data input and feedback. The GUI module 224 ensures that users can easily access and understand their sleep data, facilitating better engagement with the sleep staging platform 212.

[0064] Figure 3 depicts an example of a communication environment 300 that includes a sleep staging platform 302 that is configured to receive several types of data. Here, for example, the sleep staging platform 302 receives sleep data 304, first sensor data 306 that is generated by a first sensor (e.g., sensor 222A of Figure 2), and second sensor data 308 that is generated by a second sensor (e.g., sensor 222B of Figure 2). Those skilled in the art will recognize that these data have been selected for the purpose of illustration. Other types of data, such as treatment data (e.g., indicators of sleep disorders, treatment regimens, medication usage, etc.), could also be obtained by the sleep staging platform 302 to enhance the accuracy and relevance of sleep stage predictions.

[0065] These data may be obtained from multiple sources.

[0066] Consider the sleep data 304, for example. The sleep data 304 could be obtained directly on the computing device on which the sleep staging platform 302 is executing. For example, if the sleep staging platform 302 is implemented on a mobile phone in the form of a mobile application, the sleep data 304 may be representative of input manually provided by an individual through interfaces generated by the mobile application regarding their sleep schedule, such as the times the individual went to sleep and woke up. Additionally or alternatively, the input could include data from a wearable device regarding the sleep schedule of the individual, such as physiological parameters indicative of the times at which the individual went to sleep and woke up. In some embodiments, the sleep data 304 could be obtained from another computing device. Referring again to the aforementioned example in which the sleep staging platform 302 is implemented as a mobile application executing on a mobile phone, the mobile phone could obtain the sleep data 304 from a server system (e.g., server system 110 of Figure 1 ).

[0067] In some implementations, the sleep data 304 may include information relevant to predicting sleep stages, such as heart rate data, movement data, and other physiological parameters collected from wearable devices. Additionally or alternatively, the sleep data 304 may include manually logged sleep schedules, such as the times the individual went to sleep and woke up, as well as any disturbances during the night. The datasets may also encompass initial predicted sleep stages based on the collected heart rate and movement data. The comprehensive datasets can be sourced from wearable devices, externalservers, or user / healthcare provider logs (e.g., sleep study logs) and can be used to enhance the accuracy of sleep stage predictions by the sleep staging platform 302. Additionally, the sleep data 304 may include heart rate data such as ECG data from large polysomnography (PSG) datasets, actigraphy data from smaller stored datasets or data collected by a fitness watch using an accelerometer providing accelerometry signals, and / or photoplethysmography (PPG) data from the watch for heart rate monitoring. The server system may manage a datastore in which responses provided by various individuals to questionnaires are documented.

[0068] As mentioned above, sensor data could be obtained from sensors included in the computing device that is responsible for executing the sleep staging platform 302, or sensor data could be obtained from sensors that are external to the computing device that is responsible for executing the sleep staging platform 302. Thus, the first and second sensor data 306, 308 may be generated by sensors included in the computing device on which the sleep staging platform 302 is executing, or the first and second sensor data 306, 308 may be generated by sensors that are external to the computing device on which the sleep staging platform 302 is executing. As an example, the first sensor data 306 may be generated by an IMU implemented in the computing device on which the sleep staging platform 302 is executing, while the second sensor data 308 may be generated by a pulse oximeter that is communicatively connected — either directly or indirectly — to the computing device on which the sleep staging platform 302 is executing.

[0069] Wearable devices can include sensors and generate associated sensor datasets (e.g., sets of sensor data). For example, wearable devices can include any wearable technology (e.g., technology designed to be used while worn). Wearable technology can include smartwatches, smart glasses, activity trackers (e.g., pedometers), smart rings, hearing aids, or implants (medical or otherwise). Wearable devices and associated sensors can directly or indirectly collect data associated with a user’s sleep, such as heart rate, movement data including accelerometer signals, sleep stages, blood oxygen levels, respiratory rate, time spent in different sleep stages, and disturbances during sleep. For example, wearable devices can utilize sensor data to determine periods of sleep, including the onset and duration of sleep, as well as periods of wakefulness duringthe night. As such, because wearable devices can be worn, such devices can collect sleep data passively, enabling associated users (e.g., humans) to sleep naturally without interference. Consequently, wearable devices enable the collection and subsequent analysis of sleep-related data.

[0070] In some implementations, the sensors associated with the sleep staging platform 302 can include other devices capable of collecting sleep-related data. For example, sensors 222A-222N can include sensors, such as inertial measurement units or accelerometers, associated with mobile devices, such as smartphones, tablets, or laptop computers. For example, sensors 222A-222N can be housed on a smartphone placed near a human while they sleep, thereby enabling dynamic collection of sleep-related data associated with the human’s movements during sleep. In some implementations, sensor data can include environmental measurements, such as ambient light, noise levels, and temperature, which can affect sleep quality. As such, the data collected at the sleep staging platform 302 can include a variety of sensor measurements associated with sleep patterns and disturbances, for example.

[0071] Figure 4 depicts another example of a communication environment 400 that includes a sleep staging platform 402 that is configured to obtain data from one or more sources. Here, the sleep staging platform 402 may obtain data from a mobile phone 404, watch 406, laptop computer 408, or server system 410 (collectively referred to as the “networked devices”). For example, the sleep staging platform 402 may obtain response data (e.g., sleep data 304 of Figure 3) from the laptop computer 408 or server system 410. As another example, the sleep staging platform 402 may obtain sensor data (e.g., sensor data 306, 308 of Figure 3) from the mobile phone 404 or watch 406.

[0072] The networked devices can be connected to the sleep staging platform 402 via one or more networks. These networks can include PANs, LANs, WANs, MANs, cellular networks, the Internet, etc. Additionally or alternatively, the networked devices may communicate with one another over a short-range wireless connectivity technology. For example, if the sleep staging platform 402 resides on the mobile phone 404 in the form of a mobile application, data may be obtained from the watch 406 over a Bluetoothcommunication channel, or data may be obtained from the server system 410 over the Internet via a Wi-Fi communication channel.

[0073] Embodiments of the communication environment 400 may include a subset of the networked devices. For example, the communication environment 400 may include a sleep staging platform 402 that obtains, in real time, data from the mobile phone 404 and watch 406 as that data is generated over the course of a free-living scenario. Additional data could be obtained from the server system 410 on a periodic basis (e.g., daily or weekly).

[0074] From the foregoing, it will be appreciated that specific embodiments of the invention have been described herein for purposes of illustration but that various modifications may be made without deviating from the scope of the invention. Accordingly, the invention is not limited except as by the appended claims.Approaches to Enhancing Sleep Stages

[0075] Figure 5 includes a schematic diagram of an approach to enhancing sleep stages and predicting sleep stage metrics with a computer program executing on a computing device.A. Data Retrieval

[0076] As mentioned above, the sleep staging platform can include a data processing component — namely, a processing module (e.g., processing module 214, as shown in Figure 2). Initially, a sleep staging platform (e.g., sleep staging platform 102) can collect data from one or more sources (e.g., at an operation 501 , through the processing module 214). For example, the sleep staging platform can collect event data from PSG datasets and / or a wearable device worn by the human in the form of PPG or actigraphy data. Additionally, the sleep staging platform can collect data from a server system (e.g., the server system 1 10 of Figure 1 ), including physiological data associated with sleep, such as heart rate data, interbeat data, movement data, or other relevant metrics, such as sleep duration, sleep quality, and sleep disturbances, recorded by the human through a computer program. The collected data can also include clinical readings related to sleep, such assleep test results, and / or surveys completed by the human regarding their sleep habits and experiences. Such information enables the sleep staging platform to determine or evaluate a human’s sleep patterns and quality, thereby utilizing multiple classifier models to enhance sleep stage predictions based on trends in the collected data. By retrieving such information, the processing module 214 can improve the ability of the sleep staging platform to enhance sleep stage predictions for a given wearable device and / or individual based on the collected sleep data, as described below.

[0077] The sleep staging platform can receive data from sensors associated with wearable devices that collect various types of sleep-related information. For example, the sleep staging platform can obtain heart rate readings, pedometer or step-count readings, and other relevant data from sensors on the wearable device associated with the human. Additionally, the sleep staging platform can receive biometric information associated with sleep patterns, such as data derived from or obtained through wearable devices that monitor sleep stages. While two actigraphy features (accelerometer correlation and PPG entropy) are described herein, similar features can be calculated using the same sensors or other commonly used sensors in wearable devices, such as those measuring electrodermal activity and temperature. Additionally or alternatively, the sleep staging platform can be associated with one or more databases that include PSG datasets for managing and analyzing sleep patterns, which can also be used to train classifiers to identify different sleep stages, detect sleep disorders, and improve the overall accuracy of sleep staging, as described in more detail herein. By receiving such information, the sleep staging platform can combine actigraphy data with heart rate data, thereby enhancing sleep staging predictions.B. Data Examination

[0078] As mentioned above, the sleep staging platform can include a computation component (e.g., the computation module 216) that can be responsible for deriving different datasets and / or inputs based on the processed data received from the processing module 214. Said another way, the computation module 216 can be responsible for examining the data included in the data retrieval (e.g., at the operation 501 ).

[0079] The data examination process (e.g., operation 502) can include generating datasets based on sensor data processed by the processing module 214, which can be utilized by the first classifier module 218 and the second classifier module 220, as described later herein. For example, a dataset can include an indication of sensor readings (e.g., estimated instantaneous heart rate data) within a given time window. To illustrate, a dataset can include a list of values associated with a human, where each value indicates a number of estimated instantaneous heart rate readings recorded by a wearable device during a given period of time. The sleep staging platform can filter, align, and / or transform such sensor readings to generate a dataset that corresponds to a particular time window. Time windows can include successive periods such as overnight, daily, weekly, or monthly intervals. By generating datasets that include sensor readings associated with humans’ sleep patterns, as well as corresponding temporal information, the sleep staging platform can enable evaluation of human sleep behavior over time to enhance sleep staging predictions and interventions. For example, the sleep staging platform can generate datasets where movement data (e.g., accelerometer signals) is computed into feature vectors or heart rate data (e.g., PPG signals) is computed into instantaneous heart rates, heart rate variability, etc. The generated datasets can provide insights into the human’s engagement with the sleep tracking system, thereby offering information on sleep quality and patterns.

[0080] In some embodiments, the sleep staging platform can generate datasets where movement data (e.g., accelerometer signals) is computed into feature vectors or heart rate data (e.g., PPG signals) is computed into instantaneous heart rates, heart rate variability, etc. The generated datasets can provide insights into the human's sleep stages, thereby offering information on sleep quality and patterns. For example, Figure 6 includes a schematic diagram of an approach 600 to enhancing sleep stages with collected sensor data and a multiple classifier model. In some embodiments, the computation module 216 can adjust input heart rate data (e.g., PPG signals 601 ) into normalized and / or aligned data for input into the multiple classifier model. Additionally or alternatively, the computation module 216 can estimate instantaneous heart rate data 603 from the PPG signals 601 and extract actigraphy features from accelerometer signals. The estimated instantaneous heartrate data 603 can be input into one or more components of the multiple classifier model, as described in more detail below with reference to the first classifier module 218.

[0081] Although processes described herein may be categorized as done by either the processing module 214 and / or the computation module 216, either one or both the processing module 214, the computation module 216, and / or any of the modules described herein can be responsible for retrieving, processing, and examining the sensor data received to prepare inputs into one or both the first classifier module 218 and / or the second classifier module 220 as described herein.C. Initial Sleep Stage Determination

[0082] Referring again to Figure 5 and as mentioned above, the sleep staging platform can include one or more sleep staging components (e.g., the first classifier module 218 and the second classifier module 220) that can be responsible for determining sleep stage predictions based on the processed and examined data received from the processing module 214 and / or the computation module 216. Said another way, the first classifier module 218 and the second classifier module 220 can be responsible for examining the data included in the data retrieval (e.g., at the operation 501 ) and data examination (e.g., at the operation 502) to make heart rate-based sleep stage determinations (e.g., at an operation 504, through the first classifier module 218) and / or heart rate- and actigraphy- based sleep stage determinations (e.g., at an operation 506, through the second classifier module 220). At a high level, a first classifier engine 503 can be representative of the core logic of the first classifier module 218 that, in operation, takes an input (e.g., ECG-derived heart rates or interbeat intervals), performs at least one operation, and then produces an output (e.g., sleep stage predictions).

[0083] More specifically, the first classifier engine 503 can execute a four-class (wake, light sleep, deep sleep, REM) classification ML model trained on a large PSG dataset to predict sleep stages using ECG-derived heart rates (or equivalently interbeat intervals) as input. Referring again to Figure 6, the first classifier module 218 can provide input heart rate data (e.g., the estimated instantaneous heart rates 603) to a heart rate classifier 604 for the initial generation of sleep stages. The heart rate classifier 604 can be capable of both sleep stage prediction and classification. For example, the heart rate classifier 604can include an artificial neural network with one or more architectures, such as a decoder- only transformer-based architecture, a recurrent neural network, and / or other similar techniques capable of generating sleep stage predictions based on the input of the estimated instantaneous heart rates 603.

[0084] During the training phase for the first ML model (e.g., the heart rate classifier 604), instantaneous heart rates can be derived from ECG signals (e.g., from PSG datasets) or PPG signals 601 (e.g., from a wearable device) over a designated time window (THR) (e.g., 30 seconds) at a pre-specified sampling frequency (SHR) (e.g., every 1 second) by detecting R-peaks in the signal. R-peaks, which are the highest points of the QRS complex (representing the Q, R, and S waves), indicate the depolarization of the ventricles. These peaks can be used to measure the time intervals between heartbeats, known as R-R intervals, which are crucial for estimating the instantaneous heart rate 603. It is worth noting that the time window of the ECG signal (THR) can be wider than the desired time window for sleep staging (TS) to provide contextual information on surrounding heart rate patterns. In some embodiments, the computation module 216 of Figure 5 may complete one or more of these steps, such as detecting R-peaks, calculating R-R intervals, and deriving the instantaneous heart rates 603, before inputting the estimated instantaneous heart rates603 into the heart rate classifier 604.

[0085] The vector, with a length equal to THR divided by SHR, of the instantaneous heart rates 603 can be provided as input to the heart rate classifier 604. The output can be a one-hot encoded pre-labeled sleep stage (e.g., each sleep stage is represented by a binary vector with only one element set to 1 and all other elements set to 0). The prelabeled sleep stage data, labeled by sleep specialists, serves as reference data for training the heart rate classifier 604, allowing the first ML model to learn the relationship between the input heart rate patterns (e.g., estimated instantaneous heart rate data 603) and the corresponding sleep stages. By using these pre-labeled outputs, the heart rate classifier604 can be trained to accurately predict sleep stages based on new, unseen heart rate data.

[0086] During the inferencing phase (e.g., once the heart rate classifier 604 is trained), for every time window TS, instantaneous heart rates can be derived from the inputPPG signal 601 over a time window of length THR at SHR frequency by detecting pulse peaks in the signal. A vector of the PPG signals, representing the estimated instantaneous heart rates 603, can be provided as input to the heart rate classifier 604 to obtain an output of four-class prediction probabilities (e.g., Pwake, Pught, Poeep, PREM) at each epoch (e.g., a specific time interval within the time series), where N is the total number of epochs, resulting in a sequence of:

[0087] In some embodiments, a different classifier may be implemented in place of the heart rate classifier 604, such as a classifier that utilizes raw PPG signals directly and is trained using a large proprietary dataset that includes PPG signals. Additionally or alternatively, various classifiers capable of making initial heart rate-based sleep stage predictions can be implemented in place of the heart rate classifier 604, and the remainder of the approach 600 can be implemented as described below.D. Enhanced Sleep Stage Determination

[0088] Referring again to Figure 5, the sequence of sleep stage probabilities can serve as input, along with refined actigraphy data, to the second classifier module 220. Similar to the first classifier engine 503, a second classifier engine 504 represents the core logic of the second classifier module 220. The second classifier engine 504 takes inputs (e.g., initial sleep stage predictions and actigraphy features), performs one or more operations, and produces outputs (e.g., refined sleep stage predictions).

[0089] Specifically, the second classifier engine 504 executes a second ML model that combines the initial sleep stage predictions with extracted actigraphy features to generate more accurate sleep stage predictions. Referring again to Figure 6, the second classifier module 220 provides the initial sleep stage predictions (e.g., as the four-class probabilities described above) from the heart rate classifier 604 along with extracted actigraphy features 605 to the second ML model (e.g., an actigraphy and heart rate classifier 606) to produce an output of refined sleep stage predictions. Since the feature dimensionality is relatively low and the heart rate information is already incorporated as a sleep staging probabilities output from the heart rate classifier 604, a relatively simpler MLalgorithm (e.g., a random forests or gradient-boosted trees) can be trained in the manner described below.

[0090] In some embodiments, the heart rate classifier 604 and the actigraphy and heart rate classifier 606 utilize similar architectural frameworks, but several distinctions can be identified. The heart rate classifier 604 can be generally configured with a substantially greater number of parameters than the actigraphy and heart rate classifier 606. For example, the heart rate classifier 604 can comprise a neural network with tens of layers, as the task of mapping heart rate data to sleep stages is generally more complex. In contrast, the actigraphy and heart rate classifier 606 can employ a simpler architecture, such as a smaller multi-layer perceptron (MLP) or a random forest, since mapping sleep stages, as determined by the heart rate classifier 604, and actigraphy features to more accurate sleep stages is a generally less complex task. Additionally or alternatively, the heart rate classifier 604 can process a larger context window, such as a convolutional neural network with an input size spanning several hours, or can operate with a smaller window size, for example, 30 seconds, corresponding to the length of a sleep stage epoch. The actigraphy and heart rate classifier 606 typically receives as input the current epoch along with a limited number of neighboring epochs, generally covering a time span of several minutes, as described in greater detail below. In some embodiments, the actigraphy and heart rate classifier 606 is trained using a generally smaller dataset, such as data from approximately 50 nights, whereas the heart rate classifier 604 can be trained using a substantially larger dataset, for example, data from approximately 5,000 nights. Additionally or alternatively, no specific architectural constraints are imposed on either classifier and / or a variety of example architectures can be employed in both cases.

[0091] The extracted actigraphy features 605 input into the second ML model (e.g., the actigraphy and heart rate classifier 606) associated with the second classifier module 220 can come from one or more sensors associated with a wearable device, such as PPG signals 601 or Inertial Measurement Unit (IMU) signals 602, in the form of a PPG entropy feature and / or an accelerometer correlation feature, respectively. For example, for every time window (epoch) of length TS over which sleep stage prediction is desired, actigraphy features are calculated using PPG and accelerometer signals from the wearable device.While two actigraphy features (e.g., accelerometer correlation and PPG entropy) are described herein, similar features can be calculated using one or more of the same sensors on the wearable device or other commonly used sensors (e.g., electrodermal activity, temperature, etc.) in wearable devices. The process of extracting actigraphy features as described herein and below can be performed by one or more of the modules described herein (e.g., the processing module 214 and / or the computation module 216) such that actigraphy features (e.g., accelerometer correlation and PPG entropy) can be input with the sleep stage probabilities outputs of the heart rate classifier 604 into the actigraphy and heart rate classifier 606 to enhance sleep staging predictions.

[0092] In some embodiments, at least one of the extracted actigraphy features 605 is an accelerometer correlation feature (e.g., F . The accelerometer correlation feature (also referred to as “IMU correlation feature”) can be derived from sensor data from the wearable device (e.g., the IMU signals 602) and serves as a measure of the magnitude of an activity level of a human during a specific time window. To calculate the accelerometer correlation feature, the computation module 216 can calculate a Pearson’s correlation coefficient (e.g., rxy, ryz, rzx) between the accelerometry readings of the IMU signals 602 on each pair of axes X, Y, and Z. For example, the correlation coefficient between the axes X and Y, rxy, can be calculated using the following equation:

[0093] The xtand ytcan be individual accelerometer readings on the X and Y axes, respectively, and x and y can be the mean values of the accelerometer readings on the X and Y axes, respectively. The correlation coefficients (e.g., rxy,ryz,rzxj can provide insight into the relationship between movements of the human along different axes, which can be representative of the overall activity patterns of the human. The accelerometer correlation feature (e.g., F ) can be the minimum of the absolute value of the correlation coefficients, calculated using the following equation:

[0094] An accelerometer correlation feature close to 0 can be representative of any of the cross-axis correlation coefficients being close to 0, whereas an accelerometer correlation feature close to 1 can be representative of all correlation coefficients being close to 1 . When a human is at rest, the accelerometer readings associated with the IMU signals 602 can predominantly consist of noise, resulting in independent magnitudes across the axes. Conversely, when a human is in motion, the accelerometer readings associated with the IMU signals 602 can be present along all axes and exhibit high correlation. Therefore, the accelerometer correlation feature can be highly sensitive to distinguishing between periods of rest and motion, thus enabling the accelerometer correlation feature to accurately monitor and analyze the activity and / or movement of the human while sleeping. It is worth noting that the accelerometer correlation feature (e.g., F ) ranges between 0 and 1 , thereby being self-normalized by design, thus providing a consistent measure of activity and / or movement correlation.

[0095] Additionally or alternatively, at least one of the extracted actigraphy features 605 is a PPG entropy feature (e.g., F2). To calculate the PPG entropy feature, the computation module 216 can filter the PPG signal 601 (e.g., with a bandpass filter between 0.2-4 Hz). The filtered signal value, or the amplitude of the PPG signal, can then be separated into N bins between the lowest and highest values in each time window. The PPG entropy feature (e.g., F2), or a normalized entropy value, can be calculated using the following equation:

[0096] Thecan be the probability of bin i, defined as the number of occurrences of bin i divided by the total number of occurrences, and N can be the number of bins. In the presence of generally strong movement of the human, the natural heartbeat oscillations of the PPG signals 601 can diminish or disappear, and heartbeat distribution can concentrate around large, saturated values, resulting in a PPG entropy feature closer to 0. In the absence of generally strong movement of the human, the oscillating behavior of the PPG signal 601 can make heartbeat distribution more uniformly distributed, resulting in a PPG entropy feature closer to 1 . Therefore, the PPG entropy feature can accurately detectstrong movements of the human while sleeping. Similar to the accelerometer correlation feature (e.g., F±), the PPG entropy feature (e.g., F2) can range between 0 and 1 , thereby being self-normalized by design, thus providing a consistent measure of movement detection.

[0097] The four-class sleep stage probabilities output from the heart rate classifier 604 (e.g., the output sequence of Pwake, Pught, Poeep, PREM) and actigraphy features (e.g., Fi and F2) can be concatenated into a new feature vector for each time window Ts. In some embodiments, the computation module 216 can concatenate the output from the heart rate classifier 604 and the actigraphy features into the feature vector. For example, if the extracted actigraphy features 605 include two actigraphy features Frand F2for N epochs, a resulting feature sequence can be:r nN nN pN nN nN pN n’ ‘ ’ ‘ Wake’ ‘ light’ ‘ Deep’ ‘ REM J J

[0098] In some embodiments, due to factors such as missing sensor data, there are missing feature values for some of the actigraphy features (e.g., F and F2) in the feature sequence. For such embodiments, the missing feature values can be interpolated (e.g., by the computation module 216) based on existing surrounding values using processes such as linear interpolation and / or cubic interpolation. Additionally or alternatively, external actigraphy data 608 from a database or source can be incorporated into the extracted actigraphy features 605. In some embodiments, incorporating contextual information around each data point into the input for the actigraphy and heart rate classifier 606 can be advantageous. For example, in addition to each feature vector in the feature sequence, the computation module 216 can compute the averages of the surrounding feature vectors. Given M-dimensional feature vectors, for each feature vector ;(1 < j < M) located at epoch i, denoted by v7[i], multiple averages centered on window i can be calculated as:

[0099] The ai j[k] can be the kthaverage (0 < k < F) for feature j at epoch i. It is worth noting that k can start from 0, and for k = 0, the average can be equivalent to the feature value at epoch i, recited as:°] = Vj[i]

[0100] As aforementioned, the approach 600 can be beneficial because, in addition to the feature value at the current time point v7[i], sleep stage prediction can benefit from feature values surrounding the current time point. Furthermore, the approach 600 can be beneficial because features at time window i convey the most information about the sleep stage at time window i, and as the distance from the time window increases, the information decreases. The calculation of the averages reflects this assumption. Time window i is present in all [K + 1] averages, thereby having the largest weight. Time windows [i - 1] and [i + 1] are present in K averages, resulting in a generally smaller weight. The furthest time windows [t - k] and [i + k] are only present in one average, which means that they have the lowest weight in the feature vector. The calculation of averages and their respective weights can be beneficial to the approach 600 as it integrates contextual information, ensures a weighted contribution based on relevance, mitigates noise, and captures the temporal dynamics of sleep stages, thereby resulting in more accurate and reliable sleep stage predictions.

[0101] For example, for K = 3, the following four averages can be computed:

[0102] After calculating the window averages for all features, the results can be concatenated to construct a final feature vector Atat epoch i, which has a dimension of M * ( + 1), resulting in the equation:

[0103] Referring again to Figure 5, the first classifier engine 503 of the first classifier module 218 and the second classifier engine 504 of the second classifier module 220 can additionally or alternatively be a singular engine that includes two programs (e.g., the firstand second ML models) that, in operation, are performed in sequence. As described above, the first program, or the heart rate classifier 604, may provide a method for predicting initial sleep stages using ECG-derived heart rates or interbeat intervals. The second program, or the actigraphy and heart rate classifier 606, may provide methods for refining these initial sleep stage predictions by combining them with actigraphy features to produce more accurate sleep stage predictions.

[0104] During the training phase for the second ML model (e.g., the actigraphy and heart rate classifier 606 of Figure 6) associated with the second classifier module 220, the final feature vector Atcan be used to train and predict sleep stages. Referring again to Figure 6, the training of the actigraphy and heart rate classifier 606 can include using a four-class ML classification algorithm (e.g., a random forests or gradient-boosted trees algorithm) to train a four-class ML classification model to output sleep stages 607 in the form of sleep stage probabilities (e.g., Pwake, Pu ht, Poeep, PREM ) for each epoch i based on the final feature vector At. The actigraphy and heart rate classifier 606 can output the sleep stages 607 with the highest probability as the sleep stage output for each epoch i. As described above and herein, incorporating actigraphy-based feature values into the final feature vector Atprovides additional context regarding movement, which is often left out of sleep staging predictions, thereby improving the accuracy of sleep stage predictions. Additionally or alternatively, the actigraphy and heart rate classifier 606 can output an uncertainty score for the sleep stages 607. For example, the uncertainty output can be a value between 0 and 1 , with 0 being representative of complete certainty of the sleep stage output at a particular epoch i and 1 being representative of maximum uncertainty of the sleep stage output at the particular epoch i.

[0105] The actigraphy features calculated in the approach 600 are self-normalized, ensuring that the actigraphy and heart rate classifier 606 is generalizable across wearable devices worn at the same anatomical region (e.g., different smartwatches worn on the wrist). However, it is worth noting that actigraphy features can differ across various anatomical regions. Therefore, for wearable devices worn at different anatomical regions (e.g., the finger, chest, etc.), the approach 600 remains applicable, but the actigraphy andheart rate classifier 606 would need to be trained using data collected specifically from those anatomical regions.

[0106] The actigraphy and heart rate classifier 606 can integrate actigraphy information (e.g., in the form of actigraphy features F±and F2) with heart rate data to improve sleep stage classification performance. For example, the output sleep stages 607 of the actigraphy and heart rate classifier 606 can have improved specificity (i.e., accuracy in detecting wake periods) compared to sleep stage classification methods that are based solely on heart rate data (e.g., the heart rate classifier 604). Additionally, the low dimensionality of the actigraphy and heart rate classifier 606 enables the actigraphy and heart rate classifier 606 to be integrated and trained on board the wearable device, allowing for improved sleep stage classification. The approach 600 can thus be particularly advantageous since wearable devices typically only implement heart rate-based sleep stage classifiers due to the challenges associated with collecting a large sample size of PSG datasets.E. Analytics

[0107] Referring again to Figure 5, after producing the enhanced sleep stage probabilities, another data computing component — namely, an analytics module (e.g., analytics module 222 of Figure 2) — can perform sleep metric determination (e.g., at operation 507 shown in Figure 5) and sleep stage analysis (e.g., at operation 508 shown in Figure 5). As shown in Figure 5, the analytics module 222 can analyze outputs from one or both the first and second classifier modules 218, 220 (e.g., including one or both the first and second ML models) to perform sleep metric determination and sleep stage analyses. More specifically, the analytics module 222 may be responsible for determining sleep quality, total sleep time, sleep versus wake states, sleep onset latency, wake after sleep onset, the distribution of sleep stages, and / or the like.

[0108] In some embodiments, sleep metric determination (e.g., at operation 507) includes calculating various quantitative measures that provide insights into the human’s sleep patterns and quality based on the sleep stages predicted. The sleep metrics can be calculated using the output of the second classifier module 220. Additionally oralternatively, the sleep metrics can be calculated using the output of the first classifier module 218 and the second classifier module 220 such that a comparison of both outputs can be performed to update the model, improve accuracy, and / or understand the overall sleep behavior of the human over time. The sleep metrics can include total sleep time (TST), sleep efficiency (SE), sleep onset latency (SOL), wake after sleep onset (WASO), and sleep stage distribution.

[0109] Additionally or alternatively, sleep analysis (e.g., at operation 508) can include a more detailed examination of the user’s sleep patterns and behaviors. For example, the analytics module 222 can perform a sleep quality assessment, which evaluates the overall quality of sleep based on the distribution and duration of sleep stages, as well as the presence of disturbances or interruptions. Additionally or alternatively, the analytics module 222 can analyze and monitor changes in sleep patterns over time to identify trends or anomalies that may indicate underlying health issues or the effectiveness of interventions. Furthermore, the analytics module 222 can perform correlation analysis that examines the relationship between sleep metrics and other factors such as physical activity, stress levels, and lifestyle habits to provide a comprehensive understanding of the user’s sleep health. In some embodiments, sleep metrics are stored and compared over time through the sleep analysis operation 508 to track the human’s sleep patterns and quality and / or allow a healthcare professional to review the human’s sleep patterns and make recommendations via the sleep staging application discussed herein. Additionally, sleep analysis can also use the sleep metrics calculated to determine one or more classification metrics regarding the accuracy or completeness of the sleep stage predictions and / or additional performance metrics related to the sleep staging platform (e.g., one or more of the modules discussed herein).

[0110] Although not explicitly shown in Figure 5, the analytics module 222 can include an analytics engine, which can be one or both the first classifier engine 503 and / or the second classifier engine 505, or an additional engine in operable communication with the first classifier module 218 and / or the second classifier module 220. The analytics engine can be used to generate, update, and / or train parameters for the first and second ML models (e.g., the heart rate classifier 604 and / or the actigraphy and heart rate classifier606) based on the sleep metrics determined (e.g., in operation 507) and / or the sleep analysis provided (e.g., in operation 508) such that the first classifier module 218 and / or the second classifier module 220 can be continuously improved.E. Summary of Analysis Metrics

[0111] Figures 7A-F include schematics of simulated sleep stage data as a function of time and reference sleep metric types for a multiple classifier model. For example, Figures 7A-F show various Bland-Altman plots of computed sleep metrics from predicted sleep stage data and reference sleep metric types, including TST (e.g., Figure 7A), light duration (e.g., Figure 7B), WASO (e.g., Figure 7C), SOL (e.g., Figure 7D), SE (e.g., Figure 7E), and REM duration (e.g., Figure 7F).

[0112] Figure 8 includes a summary of sleep versus wake state classification metrics for a multiple classifier model including sensitivity, specificity, negative predictive value (NPV), and positive predictive value (PPV). Figures 9A-B include summaries of performance metrics for various derived sleep metrics for a multiple classifier model including derived sleep metrics TST, WASO, SE, SOL, light sleep, deep sleep, and REM sleep. Figure 10 includes summaries of classification metrics such as unweighted kappa, accuracy, PPV, sensitivity, NPV, and specificity for various sleep stages predicted by a multiple classifier model, including wake, light sleep, deep sleep, and REM sleep. Figure 1 1 includes a schematic of a confusion matrix for various sleep stages predicted by a multiple classifier model compared to reference sleep stages, including wake, light sleep, deep sleep, and REM sleep. Figure 12 includes a schematic of a timestamp jittering analysis, in which epochs from the PSG data from a wearable device were deliberately shifted by 30, 60, and 90 seconds in both directions. The timestamp jittering analysis evaluates whether data synchronization (e.g., synchronization of heart rate data collected as PPG and actigraphy data collected as accelerometry signals) could introduce bias or error into the sleep analysis described herein. As shown in Figure 12, the numbers on each plot indicate the number of PSG epochs excluded as a result of these timestamp shifts.Methodoloqies for Enhancing Sleep Staging with a Computer Program

[0113] Several approaches to enhancing sleep staging with a computer program executing on a computing device are set forth below. These approaches are best understood when read in conjunction with the disclosure corresponding to Figures 5-12.

[0114] Figure 13 is a flow diagram of a process 1300 for enhancing sleep staging by integrating actigraphy data with heart rate data.

[0115] At operation 1301 , the sleep staging platform can obtain actigraphy data and heart rate data associated with a wearable device worn by a human over a set period of time. For example, the wearable device can be a smartwatch or fitness tracker. As an illustrative example, the sleep staging platform can receive information relating to physiological data associated with sleep of a human wearing the wearable device, such as ECG data, PPG signals, IMU signals, or other user-related movement information or sleep characteristics. By receiving such information, the sleep staging platform can evaluate the heart rate and actigraphy data associated with a human wearing a wearable device over time to improve sleep staging predictions as a result of using a resulting computer program.

[0116] At operation 1302, the sleep staging platform can transform the heart rate data (e.g., ECG data, PPG data, interbeats, etc.) into instantaneous heart rate data, and each instantaneous heart rate can include a heart rate metric and a corresponding time window. The heart rate metrics can include at least one of heart rate variability, average heart rate, peak heart rate, minimum heart rate, heart rate recovery, or heart rate deceleration capacity. For example, the sleep staging platform can determine a vector of average heart rates associated with a given time window for all of the data points within the heart rate data. By segmenting the data by time period, the sleep staging platform can prepare the heart rate data for input into a sleep staging classifier as described herein. This segmentation allows the platform to determine trends in heart rate data in a sleep staging classification-specific manner, thereby improving the accuracy of sleep staging determination using a multi-classifier model as described herein.

[0117] At operation 1303, the sleep staging platform can apply, to the instantaneous heart rate data, a first classifier that has been trained to produce, as output, initial sleepstages corresponding to the time windows based on the heart rate metrics. For example, the first classifier can be trained with the instantaneous heart rate data and corresponding sleep stage labels in a supervised manner. The training can enable the first classifier to assign initial sleep stage probabilities to the time windows based on the heart rate metrics associated with the instantaneous heart rate data. The sleep stage labels can include each of the four sleep stages: a wake stage, a light sleep stage, a deep sleep stage, and a REM sleep stage. During the training phase, the classifier can learn to recognize patterns and correlations between the heart rate metrics and the different sleep stages. The process can involve feeding the classifier a large dataset of labeled heart rate data, allowing it to adjust its internal parameters to minimize the error in its predictions.

[0118] Once trained, the first classifier can process new instantaneous heart rate data by analyzing the heart rate metrics within each time window. The classifier can then calculate the probability that the data within each time window corresponds to each of the four sleep stages. For instance, if the heart rate variability and average heart rate within a specific time window match the patterns typically seen during REM sleep, the classifier can assign a high probability to the REM sleep stage for that window. The output of the first classifier can be a set of initial sleep stage probabilities for each time window, which can be represented as a probability distribution across the four sleep stages. The initial probabilities can provide a preliminary classification of the sleep stages, which can be further refined by subsequent processing steps or additional classifiers within the sleep staging platform (e.g., the second classifier described herein).

[0119] At operation 1304, the sleep staging platform can calculate at least one actigraphy feature vector from the actigraphy data, and each actigraphy feature can be representative of movement information associated with the human over a corresponding time window. For example, the actigraphy features can include at least one of an accelerometer correlation, a PPG entropy, an EDA feature, or a temperature feature collected from actigraphy data in the form of accelerometer signals, PPG signals, EDA signals, and temperature signals.

[0120] The actigraphy feature vector can be a multi-dimensional representation of various movement and physiological parameters captured by the actigraphy sensors. Each podimension of the vector can correspond to a specific feature derived from the raw actigraphy data. For example, the accelerometer correlation can measure the correlation between different axes of accelerometer data (e.g., X, Y, Z axes), providing insights into the intensity and direction of movements, which can be indicative of different sleep stages or activities. As another example, the PPG entropy can quantify the complexity and variability of the PPG signals, which are used to measure blood volume changes in the microvascular bed of tissue. Higher entropy can indicate more irregularities in the signal, which can be associated with different physiological states associated with sleep. In yet another example, the EDA feature can include metrics such as skin conductance level (SCL) and skin conductance response (SCR), reflecting the autonomic nervous system’s activity and used to infer stress levels, arousal, and other physiological responses associated with sleep. Additionally, the temperature feature can include metrics such as average skin temperature, temperature variability, and temperature trends over time, providing information about the body’s thermoregulation and indicative of different sleep stages or environmental conditions.

[0121] At operation 1305, the sleep staging platform can combine the initial sleep stages with the at least one actigraphy feature vector. For example, the sleep staging platform can align the time windows corresponding to the initial sleep stages with the time windows corresponding to at least one actigraphy feature vector to combine the initial sleep stages with at least one actigraphy feature vector. As an illustrative example, the sleep staging platform can incorporate two sleep staging vectors (e.g., a first feature vector associated with accelerometer correlation over the time windows and a second feature vector associated with PPG entropy over the time windows) with the initial sleep stage predictions. The sleep staging platform can align the time windows corresponding to the initial sleep stage predictions with the time windows corresponding to the extracted features of accelerometer correlation and PPG entropy. By combining the initial sleep stage predictions with the extracted actigraphy features, the sleep staging platform can integrate the accelerometer correlation and PPG entropy values with the initial sleep stage predictions for each aligned time window. In some embodiments, by combining at least one actigraphy feature vector with the initial sleep stage predictions, the sleep staging platform can integrate comprehensive movement and physiological data associated withsleep into the initial sleep predictions. The integrated vector can provide a more comprehensive view of the user’s sleep patterns and can be used as input into a sleep stage classifier.

[0122] At operation 1306, the sleep staging platform can apply to a combination of the initial sleep stages and the at least one actigraphy feature vector, a second classifier that has been trained to produce, as output, enhanced sleep stages by incorporating the movement information into the initial sleep stages. For example, the second classifier can be trained with the combination of the initial sleep stages and at least one actigraphy feature vector and corresponding sleep stage labels based on the actigraphy features in a supervised manner. The training can enable the second classifier to assign enhanced sleep stage probabilities to the time windows based on the movement information associated with the actigraphy features. Similar to the first classifier, the sleep stage labels can include each of the four sleep stages: a wake stage probability, a light sleep stage probability, a deep sleep stage probability, and a REM sleep stage probability. During the training phase, the second classifier can learn to recognize patterns and correlations between the combined initial sleep stages, actigraphy feature vectors, and the different sleep stages. The process can involve feeding the classifier a refined dataset including extracted actigraphy features with labeled combinations of initial sleep stages and actigraphy feature vectors, allowing the second classifier to adjust internal parameters to minimize the error in sleep stage predictions.

[0123] Once trained, the second classifier can process new combinations of initial sleep stages and actigraphy feature vectors by analyzing the movement information within each time window. The classifier can then calculate the probability that the data within each time window corresponds to each of the four sleep stages. For instance, if the accelerometer correlation and PPG entropy within a specific time window match the patterns typically seen during deep sleep, the classifier can assign a high probability to the deep sleep stage for that window. The output of the second classifier can be a set of enhanced sleep stage probabilities for each time window, which can be represented as a probability distribution across the four sleep stages. The enhanced probabilities canprovide a refined classification of the sleep stages based on improved movement information, improving the accuracy of the sleep staging platform.

[0124] In some embodiments, the first classifier (e.g., as described in operation 1303 and herein) has a first dimensionality, and the second classifier has a second dimensionality. For example, the second dimensionality can be less than the first dimensionality because the second classifier focuses on refining the initial sleep stage predictions by incorporating additional movement information, which may require fewer parameters. The reduced dimensionality can lead to a more efficient and faster classification process, as the second classifier can leverage the initial sleep stage predictions and the actigraphy feature vectors to enhance the accuracy of the sleep stage probabilities without the need for extensive computational resources.

[0125] Figure 14 is a flow diagram of a process 1400 for enhancing sleep staging by integrating actigraphy data with heart rate data. The process 1400 can be generally similar to or identical to one or more of the steps of the process 1300 described in more detail with reference to Figure 13 or any of the other processes described herein.

[0126] At operation 1401 , the sleep staging platform can extract actigraphy data from a wearable device worn by a human, wherein the actigraphy data represents movement information associated with the human over a set period of time. For example, the actigraphy data can be obtained from an IMU sensor and / or a PPG sensor integrated into the wearable device as accelerometer signals and / or PPG signals.

[0127] As described above and herein, the actigraphy data can be used to monitor and analyze the movement patterns of the human wearing the device. More specifically, the actigraphy data can include information about the frequency, duration, and intensity of movements captured through various sensors such as the IMU sensor and / or PPG sensors or, additionally or alternatively, EDA sensors and / or thermal sensors. For example, accelerometer signals measured by an IMU sensor can provide insights into the physical activities of the human over the set period of time. In some embodiments, PPG signals measured by a PPG sensor can be used to infer PPG entropy, as described in more detail below, to analyze movement patterns. Additionally or alternatively, EDA signals measured by EDA sensors can provide information about the skin conductance of the human, relatedto stress and arousal levels, which can be associated with sleep patterns, while thermal signals can offer insights into body temperature, which can also be indicative of different sleep patterns. Together, the actigraphy data from the sensors on the wearable device can provide a comprehensive view of the human’s movement and physiological state over the set period of time, which can be processed and used as input for a sleep stage classifier, as described in more detail with reference to the operation 1301 of Figure 13.

[0128] At operation 1402, the sleep staging platform can calculate a first actigraphy feature vector, including a first actigraphy feature for each of a plurality of time windows over the set period of time, the first actigraphy feature corresponding to a first movement signal collected by at least one sensor on the wearable device. For example, the first actigraphy feature can correspond to the actigraphy data collected by a first sensor on the wearable device. The first movement signal can include at least one of the signals described above, including accelerometer signals, PPG signals, EDA signals, thermal signals, and / or any additional signals associated with movement and / or sleep parameters. As an illustrative example, the first actigraphy feature can be an accelerometer correlation, representing the consistency and pattern of movements over the plurality of time windows, calculated from accelerometer signals obtained by an IMU sensor on the wearable device, as described in more detail with reference to the extracted actigraphy features 605 of Figure 6 and the operation 1304 of Figure 13.

[0129] At operation 1403, the sleep staging platform can calculate a second actigraphy feature vector, including a second actigraphy feature for each of the plurality of time windows over the set period of time, the second actigraphy feature corresponding to a second movement signal collected by the at least one sensor on the wearable device. For example, the second actigraphy feature can correspond to the actigraphy data collected by a second sensor on the wearable device. Similar to the first movement signal, the second movement signal can include at least one of the signals described herein, including accelerometer signals, PPG signals, EDA signals, thermal signals, and / or any additional signals associated with movement and / or sleep parameters. As an illustrative example, the second actigraphy feature can be PPG entropy, representing the variability and complexity of the PPG signals over the plurality of time windows. PPG entropy can berepresentative of movement of the human by reflecting changes in blood volume and flow, which are influenced by physical activity. PPG entropy can be calculated from PPG signals obtained by a PPG sensor on the wearable device, as described in more detail with reference to the extracted actigraphy features 605 of Figure 6 and the operation 1304 of Figure 13.

[0130] At operation 1404, the sleep staging platform can obtain initial sleep stage predictions corresponding to each of the plurality of time windows over the set period of time. For example, the initial sleep stage predictions can be based on heart rate data collected over the same set period of time as the actigraphy data. The heart rate data can be ECG signals or PPG signals obtained from external monitoring devices (e.g., ECG signal collectors) and / or sensors integrated into the wearable device. The initial sleep stage predictions can include a wake stage probability, a light sleep stage probability, a deep sleep stage probability, and a REM sleep stage probability. In some embodiments, the initial sleep stage predictions are obtained from a first classifier trained on heart rate data, interbeat intervals, or other physiological indicators typically used in sleep staging, as described in more detail with reference to the operation 1303 of Figure 13. The first classifier can use machine learning algorithms to analyze patterns in the heart rate data and / or other physiological signals to predict the likelihood of different sleep stages.

[0131] At operation 1405, the sleep staging platform can concatenate the initial sleep stage predictions with the first and second actigraphy feature vectors. For example, the sleep staging platform can align the time windows corresponding to the initial sleep stages with the time windows corresponding to the first and second actigraphy feature vectors to combine the initial sleep stages with the actigraphy data. As an illustrative example, the sleep staging platform can incorporate two actigraphy feature vectors: a first feature vector associated with accelerometer correlation over the time windows and a second feature vector associated with PPG entropy over the time windows. By aligning the time windows corresponding to the initial sleep stage predictions with the time windows corresponding to the extracted features of accelerometer correlation and PPG entropy, the sleep staging platform can integrate the accelerometer correlation and PPG entropy values with the initialsleep stage predictions for each aligned time window, as described in more detail with respect to the operation 1305 of Figure 13.

[0132] The actigraphy data, in the form of the first and second actigraphy feature vectors, can help fill in movement gaps that the initial sleep predictions, based on heart rate data, may not have captured. For example, periods of movement detected by actigraphy signals can indicate wakefulness or transitions between sleep stages that heart rate data alone might not reveal. In some embodiments, by combining at least one actigraphy feature vector with the initial sleep stage predictions, the sleep staging platform can integrate comprehensive movement and physiological data associated with sleep into the initial sleep predictions, which can be used as input into a sleep stage classifier.

[0133] At operation 1406, the sleep staging platform can apply to a concatenation of the initial sleep stage predictions and the first and second actigraphy feature vectors, a classifier that has been trained to produce, as output, enhanced sleep stage predictions by incorporating the first and second movement signals into the initial sleep stage predictions. For example, the enhanced sleep stage predictions can also include a wake stage probability, a light sleep stage probability, a deep sleep stage probability, and a REM sleep stage probability. The classifier can be trained with the combination of the initial sleep stage predictions and the actigraphy feature vectors, along with corresponding sleep stage labels based on the actigraphy features in a supervised manner. The training can enable the classifier to assign enhanced sleep stage probabilities to the time windows based on the movement information associated with the actigraphy features. During the training phase, the classifier can learn to recognize patterns and correlations between the combined initial sleep stage predictions, actigraphy feature vectors, and the different sleep stages, as described in more detail with reference to operation 1306 of Figure 13.

[0134] Figure 15 is a flow diagram of a process 1500 for training a model to integrate actigraphy data with heart rate data to enhance sleep staging. The process 1500 can be generally similar to or identical to one or more of the steps of the processes 1300 and 1400 described in more detail with reference to Figures 13 and 14 or any of the other processes described herein.

[0135] At operation 1501 , the sleep staging platform can obtain from a wearable device associated with a human, the heart rate data and the actigraphy data recorded over a plurality of time windows. The operation 1501 can be generally similar to the operations 1301 and 1401 , as described in more detail with respect to Figures 13 and 14, respectively. For example, the sleep staging platform can obtain the heart rate data and the actigraphy data from one or more sensors on and / or within the wearable device.

[0136] At operation 1502, the sleep staging platform can transform the heart rate data into a set of heart rate metrics, with each heart rate metric associated with a time window of the plurality of time windows. For example, the set of heart rate metrics can include at least one of heart rate variability, average heart rate, peak heart rate, minimum heart rate, heart rate recovery, or heart rate deceleration capacity. As an illustrative example, the set of heart rate metrics can be a set of heart rate data or interbeat intervals transformed into instantaneous heart rates associated with a time window of the plurality of time windows, as described in more detail with respect to the operation 1302 of Figure 13.

[0137] At operation 1503, the sleep staging platform can provide the set of heart rate metrics, as input, to a first classifier to produce, as output, an initial set of sleep stage probabilities for each time window corresponding to each of the four sleep stages: (i) a wake stage, (ii) a light sleep stage, (iii) a deep sleep stage, and (iv) a REM sleep stage. For example, the first classifier can be trained with heart rate metrics labeled with probabilities for each of the four sleep stages. In some embodiments, the first classifier uses heart rate metrics to detect movement characteristics associated with larger movements of the human over the plurality of time windows. Additionally or alternatively, the first classifier can use a first allocation of computational resources to train a classifier to predict sleep stages and / or infer sleep stages based on a set of heart rate metrics input into the first classifier, as described in more detail with reference to the operation 1303 of Figure 13.

[0138] At operation 1504, the sleep staging platform can transform the actigraphy data into at least one set of actigraphy features, with each actigraphy feature associated with a time window of the plurality of time windows. For example, the set of actigraphy features can correspond to movement characteristics of the human over the plurality oftime windows. In some embodiments, the at least one set of actigraphy features can include at least one of a set of accelerometer correlations, PPG entropies, EDA features, or temperature features collected from actigraphy data in the form of accelerometer signals, PPG signals, EDA signals, and temperature signals. As an illustrative example, the at least one set of actigraphy features can be a set of accelerometer correlations and a set of PPG entropies associated with a time window of the plurality of time windows, as described in more detail with respect to the operations 1303, 1402, and 1403 of Figures 13 and 14, respectively.

[0139] At operation 1505, the sleep staging platform can combine the initial set of sleep stage probabilities with the at least one set of actigraphy features. The operation 1505 can be generally similar to the operations 1305 and 1405, as described in more detail with respect to Figures 13 and 14, respectively. For example, the sleep staging platform can align the plurality of time windows associated with the initial set of sleep stage probabilities with the plurality of time windows associated with the at least one set of actigraphy features.

[0140] At operation 1506, the sleep staging platform provides a combination of the initial set of sleep stage probabilities and the at least one set of actigraphy features, as input, to a second classifier to produce, as output, an enhanced set of sleep stage probabilities for each time window corresponding to each of the four sleep stages. For example, the second classifier can be trained with actigraphy features labeled with probabilities for each of the four sleep stages, as described in more detail with reference to the operations 1306 and 1406 of Figures 13 and 14, respectively. In some embodiments, the second classifier uses the actigraphy features of the at least one set of actigraphy features to detect the movement characteristics associated with smaller movements of the human over the plurality of time windows, thus filling in movement information that was not detected by the first classifier using the heart rate data, which typically corresponds to larger movements. Additionally or alternatively, the second classifier can use a second allocation of computational resources to train and / or infer sleep stage probabilities. The second allocation of computational resources can be less than the first allocation of computational resources used by the first classifier to train and / or infer sleep stageprobabilities, thereby allowing the second classifier to use generally less computational power and time, enabling the second classifier to be trained and executed, for example, on board the wearable device or by less of the computational resources within or associated with the wearable device.

[0141] In some embodiments, the second allocation of computational resources can be less than the first allocation of computational resources because the second classifier is less complex, or, stated differently, includes fewer parameters for processing than the first classifier. It can be advantageous to allocate fewer computational resources to the second classifier because the second classifier can be trained significantly faster, for example, in less than an hour as opposed to several days. This enables the training of multiple instances of the second classifier for different datasets, such as for different populations, within a short period of time. Additionally or alternatively, the first classifier generally incurs a much higher cost of training, for example, typically requiring GPU resources. In contrast, the first classifier can be trained once, and subsequently, the smaller and less resource-intensive second classifier can be developed and trained for various tasks and populations.Processing System

[0142] Figure 16 is a block diagram illustrating an example of a processing system 1600 in which at least some operations described herein can be implemented. For example, components of the processing system 1600 may be hosted on a computing device that includes a sleep staging platform (e.g., sleep staging platform 102 of Figure 1 , sleep staging platform 212 of Figure 2, sleep staging platform 302 of Figure 3, and / or sleep staging platform 402 of Figure 4).

[0143] The processing system 1600 may include a processor 1602, main memory 1606, non-volatile memory 1610, network adapter 1612, video display 1618, input / output device 1620, control device 1622 (e.g., a keyboard or pointing device), drive unit 1624 including a storage medium 1626 (e.g., a non-transitory storage medium), and signal generation device 1630 that are communicatively connected to a bus 1616. The bus 1616 is illustrated as an abstraction that represents one or more physical buses or point-to-point connections that are connected by appropriate bridges, adapters, or controllers. The bus1616, therefore, can include a system bus, a Peripheral Component Interconnect (PCI) bus or PCI-Express bus, a HyperTransport or industry standard architecture (ISA) bus, a small computer system interface (SCSI) bus, a universal serial bus (USB), inter-integrated circuit (I2C) bus, or an Institute of Electrical and Electronics Engineers (IEEE) standard 1694 bus (also referred to as “Firewire”).

[0144] While the main memory 1606, non-volatile memory 1610, and storage medium 1626 are shown to be a single medium, the terms “machine-readable medium” and “storage medium” should be taken to include a single medium or multiple media (e.g., a centralized / distributed database and / or associated caches and servers) that store one or more sets of instructions 1628. The terms “machine-readable medium” and “storage medium” shall also be taken to include any medium that is capable of storing, encoding, or carrying a set of instructions for execution by the processing system 1600.

[0145] In general, the routines executed to implement the embodiments of the disclosure may be implemented as part of an operating system or a specific application, component, program, object, module, or sequence of instructions (collectively referred to as “computer programs”). The computer programs typically comprise one or more instructions (e.g., instructions 1604, 1608, 1628) set at various times in various memory and storage devices in a computing device. When read and executed by the processors 1602, the instruction(s) cause the processing system 1600 to perform operations to execute elements involving the various aspects of the present disclosure.

[0146] Further examples of machine- and computer-readable media include recordable-type media, such as volatile memory devices and non-volatile memory devices 1610, removable disks, hard disk drives, and optical disks (e.g., Compact Disk Read-Only Memory (CD-ROMS) and Digital Versatile Disks (DVDs)), and transmission-type media, such as digital and analog communication links.

[0147] The network adapter 1612 enables the processing system 1600 to mediate data in a network 1614 with an entity that is external to the processing system 1600 through any communication protocol supported by the processing system 1600 and the external entity. The network adapter 1612 can include a network adaptor card, a wireless network interface card, a router, an access point, a wireless router, a switch, a multilayer switch, aprotocol converter, a gateway, a bridge, a bridge router, a hub, a digital media receiver, a repeater, or any combination thereof.Remarks

[0148] The foregoing description of various embodiments of the claimed subject matter has been provided for the purposes of illustration and description. It is not intended to be exhaustive or to limit the claimed subject matter to the precise forms disclosed. Many modifications and variations will be apparent to one skilled in the art. Embodiments were chosen and described in order to best describe the principles of the invention and its practical applications, thereby enabling those skilled in the relevant art to understand the claimed subject matter, the various embodiments, and the various modifications that are suited to the particular uses contemplated.

[0149] Although the Detailed Description describes certain embodiments and the best mode contemplated, the technology can be practiced in many ways no matter how detailed the Detailed Description appears. Embodiments may vary considerably in their implementation details while still being encompassed by the specification. Particular terminology used when describing certain features or aspects of various embodiments should not be taken to imply that the terminology is being redefined herein to be restricted to any specific characteristics, features, or aspects of the technology with which that terminology is associated. In general, the terms used in the following claims should not be construed to limit the technology to the specific embodiments disclosed in the specification unless those terms are explicitly defined herein. Accordingly, the actual scope of the technology encompasses not only the disclosed embodiments but also all equivalent ways of practicing or implementing the embodiments.

[0150] The language used in the specification has been principally selected for readability and instructional purposes. It may not have been selected to delineate or circumscribe the subject matter. It is therefore intended that the scope of the technology be limited not by this Detailed Description but rather by any claims that issue on an application based hereon. Accordingly, the disclosure of various embodiments is intended to be illustrative, but not limiting, of the scope of the technology as set forth in the following claims.

Claims

1. CLAIMSI / We claim:1 . A method for combining actigraphy data with heart rate data to enhance sleep staging, the method comprising: obtaining the actigraphy data and the heart rate data associated with a wearable device worn by a human over a set period of time; transforming the heart rate data into instantaneous heart rate data, in which each instantaneous heart rate is associated with a corresponding one of multiple heart rate metrics and a corresponding one of multiple time windows; applying, to the instantaneous heart rate data, a first classifier that has been trained to produce, as output, initial sleep stages corresponding to the multiple time windows based on the multiple heart rate metrics; calculating at least one actigraphy feature vector from the actigraphy data, wherein each actigraphy feature is representative of movement information associated with the human over a corresponding time window; combining the initial sleep stages with the at least one actigraphy feature vector; and applying, to a combination of the initial sleep stages and the at least one actigraphy feature vector, a second classifier that has been trained to produce, as output, enhanced sleep stages by incorporating the movement information into the initial sleep stages.

2. The method of claim 1 , further comprising: training the first classifier with the instantaneous heart rate data and corresponding sleep stage labels in a supervised manner, such that the first classifier assigns initial sleep stage probabilities corresponding to the multiple time windows based on the multiple heart rate metrics, wherein the sleep stage labels include:(i) a wake stage,(ii) a light sleep stage,(iii) a deep sleep stage, and(iv) a REM sleep stage.

3. The method of claim 1 , wherein each heart rate metric includes at least one of heart rate variability, average heart rate, peak heart rate, minimum heart rate, heart rate recovery, or heart rate deceleration capacity.

4. The method of claim 1 , further comprising: training the second classifier with the combination of the initial sleep stages and the at least one actigraphy feature vector and corresponding sleep stage labels in a supervised manner, such that the second classifier assigns enhanced sleep stage probabilities corresponding to the multiple time windows based on the movement information associated with the actigraphy features, wherein the sleep stage labels include:(i) a wake stage probability,(ii) a light sleep stage probability,(iii) a deep sleep stage probability, and(iv) a REM sleep stage probability.

5. The method of claim 1 , wherein the at least one actigraphy feature includes at least one of an accelerometer correlation, a photoplethysmogram (PPG) entropy, an electrodermal activity (EDA) feature, or a temperature feature.

6. The method of claim 1 , wherein the first classifier has a first dimensionality, the second classifier has a second dimensionality, and wherein the second dimensionality is less than the first dimensionality.

7. The method of claim 1 , wherein combining the initial sleep stages with the at least one actigraphy feature vector further comprises:aligning the multiple time windows corresponding to the initial sleep stages with the multiple time windows corresponding to the at least one actigraphy feature vector.

8. A method for enhancing sleep staging with actigraphy data, the method comprising: extracting the actigraphy data from a wearable device worn by a human, wherein the actigraphy data is representative of movement information associated with the human over a set period of time; calculating a first actigraphy feature vector, including a first actigraphy feature for each of a plurality of time windows over the set period of time, wherein the first actigraphy feature corresponds to a first movement signal collected by at least one sensor on the wearable device; calculating a second actigraphy feature vector, including a second actigraphy feature for each of the plurality of time windows over the set period of time, wherein the second actigraphy feature corresponds to a second movement signal collected by the at least one sensor on the wearable device; obtaining initial sleep stage predictions corresponding to each of the plurality of time windows over the set period of time; concatenating the initial sleep stage predictions with the first and second actigraphy feature vectors; and applying, to a combination of the initial sleep stage predictions and the first and second actigraphy feature vectors, a classifier that has been trained to produce, as output, enhanced sleep stage predictions by incorporating the first and second movement signals into the initial sleep stage predictions.

9. The method of claim 8, wherein: the first actigraphy feature corresponds to the actigraphy data collected by a first sensor on the wearable device, and the second actigraphy feature corresponds to the actigraphy data collected by a second sensor on the wearable device.

10. The method of claim 8, wherein the first movement signal, the second movement signal, or the first and second movement signals are:(i) accelerometer signals,(ii) photoplethysmogram (PPG) signals,(iii) electrodermal activity (EDA) signals, or(iv) thermal signals.1 1 . The method of claim 8, wherein the actigraphy data is obtained from an Inertial Measurement Unit (IMU) sensor and / or photoplethysmography (PPG) sensor integrated into the wearable device.

12. The method of claim 8, wherein the initial sleep stage predictions are based on heart rate data collected over the set period of time.

13. The method of claim 12, wherein the heart rate data is obtained from a photoplethysmography (PPG) sensor integrated into the wearable device.

14. The method of claim 8, wherein the initial sleep stage predictions and the enhanced sleep stage predictions include:(i) a wake stage probability,(ii) a light sleep stage probability,(iii) a deep sleep stage probability, and(iv) a REM sleep stage probability.

15. A method for combining actigraphy data with heart rate data to enhance sleep staging, the method comprising: obtaining, from a wearable device associated with a human, the heart rate data and the actigraphy data recorded over a plurality of time windows; transforming the heart rate data into a set of heart rate metrics, wherein each heart rate metric is associated with a time window of the plurality of time windows;providing the set of heart rate metrics, as input, to a first classifier to produce, as output, an initial set of sleep stage probabilities for each time window corresponding to each of four sleep stages:(i) a wake stage,(ii) a light sleep stage,(iii) a deep sleep stage, and(iv) a REM sleep stage; transforming the actigraphy data into at least one set of actigraphy features, wherein each actigraphy feature is associated with a time window of the plurality of time windows; combining the initial set of sleep stage probabilities with the at least one set of actigraphy features; and providing a combination of the initial set of sleep stage probabilities and the at least one set of actigraphy features, as input, to a second classifier to produce, as output, an enhanced set of sleep stage probabilities for each time window corresponding to each of the four sleep stages.

16. The method of claim 15, wherein the first classifier is trained with heart rate metrics labeled with probabilities for each of the four sleep stages.

17. The method of claim 15, wherein the second classifier is trained with actigraphy features labeled with probabilities for each of the four sleep stages.

18. The method of claim 15, wherein the at least one set of actigraphy features corresponds to movement characteristics of the human over the plurality of time windows.

19. The method of claim 15, wherein the first classifier uses heart rate metrics to detect movement characteristics associated with larger movements of the human over the plurality of time windows, and wherein the second classifier uses actigraphy features to detect the movement characteristics associated with smaller movements of the human over the plurality of time windows.

20. The method of claim 15, wherein the first classifier uses a first allocation of computational resources, and the second classifier uses a second allocation of computational resources, and wherein the second allocation of computational resources is less than the first allocation of computational resources.

Images