Sleep-wake classification using machine learning

Abstract

Methods, systems, and apparatus comprising a computer program coded on a computer storage medium for performing a sleep-wake classification task using a sleep-wake classification neural network. In one embodiment, the method comprises receiving accelerometer data characterizing an object; processing the accelerometer data using a sleep-wake classification neural network, which includes processing the accelerometer data using an encoder subnetwork of the sleep-wake classification neural network to generate an embedding of the accelerometer data; processing the embedding of the accelerometer data using a projection subnetwork of the sleep-wake classification neural network to generate a score distribution across a set of sleep-wake classes; and classifying the sleep-wake state of the object based on the respective scores for each class in the set of sleep-wake classes, wherein the encoder subnetwork of the sleep-wake classification neural network is pre-trained to perform an auxiliary task, which is different from the sleep-wake classification task.

Description

Technical Field

[0001] This specification relates to sleep-wake classification using machine learning.

Background Art

[0002] A machine learning model receives an input and generates an output, for example a predicted output, based on the received input. Some machine learning models are parametric models and generate an output based on the received input and the values of the model's parameters.

[0003] Some machine learning models are deep models that use multiple layers of the model to generate an output for the received input. For example, a deep neural network is a deep machine learning model that includes an output layer and one or more hidden layers that each apply a non-linear transformation to the received input to generate an output.

[0004] Sleep is a complex process and can be divided into two broad categories of non-rapid eye movement (NREM) sleep and rapid eye movement (REM) sleep. NREM sleep can be further divided into stages 1-3. Stage 1 is the lightest sleep stage, in which the body begins to relax, and breathing and heart rate slow down. Hypnagogic jerks are common during the transition to stage 1. Additionally, during stage 1, alpha wave neural oscillations may decrease while theta wave neural oscillations may increase. Stage 2 is a deeper sleep level in which the body begins to prepare for a deeper level of rest. Electroencephalogram (EEG) recordings during stage 2 are characterized by high-frequency brain activity ("sleep spindles") and short bursts of K-complexes. Stage 3 is the deepest sleep stage, in which the body experiences restorative and rejuvenating sleep, self-repairs, builds new tissue, and stores new memories. REM sleep is the final stage of sleep and is characterized by increased brain activity and rapid eye movements. REM sleep is essential for memory consolidation and learning.

Summary of the Invention

Means for Solving the Problems

[0005] This specification describes a classification system implemented as a computer program on one or more computers located in one or more locations, which can process accelerometer data characterizing the movement of an object to generate a sleep-wake classification that characterizes the sleep or wakefulness state of the object.

[0006] Throughout this specification, "subject" refers to a human subject, that is, a person.

[0007] Throughout this specification, the term "subnetwork" in the context of a neural network refers to a part of a neural network.

[0008] Throughout this specification, “embedding” of an entity (e.g., an accelerometer signal) refers to a representation of the entity in a latent space generated by a neural network (or a subnetwork of a neural network). Embeddings can be represented as an ordered collection of numbers, such as a vector, matrix, or other numerical tensor.

[0009] Throughout this specification, the term "block" (for example, "convolutional block" or "recurrent block") refers to a group of neural network layers in a neural network.

[0010] Throughout this specification, the terms “accelerometer data” and “accelerometer signal” are used interchangeably.

[0011] Certain embodiments of the subject matter described herein can be implemented to achieve one or more of the following advantages:

[0012] The classification system described herein can perform sleep-wake classification tasks by processing accelerometer data characterizing the movement of an object, i.e., it can classify the sleep or wake state of the object. The classification neural network can generate sleep-wake classifications by directly processing raw accelerometer data without requiring manual or heuristic feature engineering. More specifically, the classification neural network can be trained by machine learning training techniques to identify and extract relevant feature representations from raw accelerometer data in order to accurately classify the sleep-wake state of the object.

[0013] Obtaining labeled training data for training a classification neural network to perform a sleep-wake classification task is time-consuming and costly. ("Labeled" training data refers to accelerometer signals labeled, for example, by a physician, with corresponding sleep-wake states.) For example, obtaining labeled training data for a subject may require the subject to undergo a sleep study, in which the subject sleeps in a medical facility and is monitored using sensors such as accelerometers, electroencephalography (EEG) sensors, electrooculography (EOG) sensors, electromyography (EMG) sensors, and electrocardiogram (ECG) sensors. The physician (or other specialist) can then review the array of multimodal data generated during the sleep study and painstakingly label the subject's sleep-wake states in small segments of the accelerometer data. Because obtaining labeled training data is difficult, relatively small amounts of training data are available for training classification neural networks to perform a sleep-wake classification task. However, the performance of machine learning models such as classification neural networks can be heavily dependent on the availability of large amounts of training data, and a lack of training data can limit the performance of such machine learning models.

[0014] To address this problem, the classification system described herein can utilize large amounts of readily available unlabeled accelerometer data. In particular, the classification system can train the classification neural network by a two-step process that includes first pre-training the encoder subnetwork of the classification neural network for one or more auxiliary tasks, and then fine-tuning the classification neural network to perform sleep-wake classification. The system can pre-train the encoder subnetwork using unlabeled accelerometer data, thus encouraging the encoder subnetwork to generate rich and useful features that characterize the structure of the accelerometer data. By pre-training the encoder subnetwork using unlabeled accelerometer data, the amount of labeled training data required to fine-tune the classification neural network to perform sleep-wake classification tasks can be dramatically reduced. Therefore, by pre-training the encoder subnetwork, the classification neural network can be trained to achieve acceptable predictive accuracy on sleep-wake classification tasks even when only a relatively small amount of labeled sleep training data is available. Furthermore, by pre-training the encoder subnetwork, the consumption of computational resources such as memory and computing power can be reduced during the fine-tuning of the classification neural network.

[0015] Details of one or more embodiments of the subject matter of this specification are described in the accompanying drawings and the following description. Other features, aspects, and advantages of the subject matter will become apparent from the description, drawings, and claims. [Brief explanation of the drawing]

[0016] [Figure 1] An example classification system is shown. [Figure 2] An exemplary architecture for a classification neural network is shown. [Figure 3A] This example architecture demonstrates a classification neural network implemented using one or more residual blocks. [Figure 3B] This shows an exemplary architecture of residual blocks included in a classification neural network. [Figure 4] This is an illustrative process flowchart for performing a sleep-wake classification task using a sleep-wake classification neural network. [Figure 5] An exemplary training system is shown. [Figure 6A] This is an illustrative process flowchart for pre-training an encoder subnetwork of a classification neural network to perform a controlled embedding task. [Figure 6B] This diagram shows how to pre-train an encoder subnetwork to perform a controlled embedding task. [Figure 7] This is an illustrative process flowchart for pre-training an encoder subnetwork of a classification neural network to perform a masked reconstruction task. [Figure 8] This is an illustrative process flowchart for pre-training an encoder subnetwork of a classification neural network to perform a noisy reconstruction task. [Figure 9] This is an illustrative process flowchart for pre-training an encoder subnetwork of a classification neural network to perform supervised aiding tasks. [Figure 10] This is an illustrative process flowchart for training a classification neural network to perform a sleep-wake classification task. [Figure 11] This is an illustrative process flowchart for training a classification neural network to perform a medical symptom classification task. [Figure 12A] This paper presents experimental results illustrating the effect of pre-training the encoder neural network in a classification neural network. [Figure 12B] This specification presents experimental results illustrating the sleep-wake classification accuracy of the system described herein compared to two other sleep-wake classification systems.

Best Mode for Carrying Out the Invention

[0017] Like reference numerals and designations in the various drawings indicate like elements.

[0018] FIG. 1 shows an exemplary classification system 100. The classification system 100 is an example of a system implemented as a computer program on one or more computers located at one or more positions where the systems, components, and techniques described below are implemented.

[0019] The classification system 100 is configured to receive accelerometer data 104 that characterizes the movement of the subject 102. The classification system 100 processes the accelerometer data 104 to generate a sleep-wake classification 112 that characterizes the sleep or wake state of the subject 102.

[0020] The accelerometer data 104 can characterize the movement of the subject 102 over a time interval, such as a 30-second, 60-second, or 90-second time interval. The accelerometer data 104 can be generated by an accelerometer device disposed on or near the subject 102. For example, the accelerometer data 104 can be generated by a wearable device worn by the subject 102, such as an accelerometer within a wearable device worn on the subject's wrist.

[0021] The classification system 100 can represent the accelerometer data 104 in any of a variety of formats. For example, the classification system 100 can represent the accelerometer data 104 by one or more one-dimensional (1D) time signals, such as each 1D time signal characterizing the acceleration in each of the x, y, and z directions (where the x, y, and z directions are defined with reference to an appropriate reference coordinate system). As another example, the classification system 100 can represent the accelerometer data 104 by a two-dimensional (2D) spectrogram in the time-frequency domain, i.e., a spectrogram that includes one dimension representing time and two dimensions representing frequency.

[0022] Optionally, the classification system 100 may receive additional inputs, i.e., inputs in addition to the accelerometer data 104. These additional inputs may include any appropriate data characterizing the object 102 or the environment surrounding the object 102. The classification system 100 may process these additional inputs as part of the generation of the sleep-wake classification 112. These additional inputs may be derived from any appropriate modality and can provide the classification system 100 with additional information sources to improve the accuracy of the sleep-wake classification 112. Several examples of additional inputs to the classification system 100 are described below.

[0023] In some implementations, the classification system 100 additionally receives cardiovascular data characterizing the cardiovascular state of the subject 102. Cardiovascular data can be generated, for example, within a wearable device worn by the subject 102, using one or more cardiovascular sensors (e.g., a heart rate monitor or pulse oximeter) placed on or near the subject. In some implementations, cardiovascular data may include data defining the subject's heart rate, or the variability (e.g., variance) of the subject's heart rate, or the subject's blood oxygen saturation over a time interval. In some cases, cardiovascular data may include time-series data defining the values ​​of cardiovascular parameters at each point in time within a series of points over a time interval. For example, cardiovascular data may include one or more audio waveforms characterizing the heart sounds produced by the subject's heart. Cardiovascular data can be captured at the same time interval as the accelerometer data 104.

[0024] In some implementations, the classification system 100 additionally receives data characterizing the ambient light in the vicinity of the object 102. The term "ambient light" can refer to any light, e.g., natural light (e.g., sunlight or moonlight) or artificial light (e.g., generated by one or more lighting devices). Ambient light data can be generated using one or more light sensors placed in the vicinity of the object. Ambient light data can characterize any appropriate feature of the light in the vicinity of the object, e.g., light intensity, light color, light variation, etc. Ambient light data may include aggregated data, e.g., one or more statistics summarizing the light in the vicinity of the object over a time interval, or time-series data, e.g., defining the respective values ​​of one or more light parameters at each point in a series of points in time over a time interval. Ambient light data can be captured at the same time interval as the accelerometer data 104.

[0025] In some implementations, the classification system 100 additionally receives audio data characterizing the sounds in the vicinity of the object 102. The audio data can be generated using one or more acoustic sensors placed in the vicinity of the object 102. The audio data may include aggregated data (e.g., one or more statistics summarizing the intensity or changes in sounds in the vicinity of the object over a certain time interval) or time-series data (e.g., one or more audio waveforms generated by one or more acoustic sensors placed in the vicinity of the object 102 over a certain time interval). The audio data can be captured at the same time interval as the accelerometer data 104.

[0026] In some implementations, the classification system 100 additionally receives time data characterizing the current time when the accelerometer data 104 was captured. The classification system 100 can represent the time data in any suitable way, for example, by a one-hot embedding vector. The one-hot embedding vector may contain separate entries for each hour of the day, where the entry corresponding to the current time has a value of 1 (or another predetermined value), and the entries corresponding to times other than the current time have a value of 0 (or another predetermined value).

[0027] In some implementations, the classification system 100 additionally receives electroencephalography (EEG) data characterizing the electrical activity in the brain of the subject 102. EEG data can be generated using an electroencephalograph that measures electrical activity in the subject's brain using small metal discs (electrodes) attached to the subject's scalp. EEG data may include aggregated data (e.g., one or more statistics summarizing the intensity or changes in electrical activity in the brain over a certain time interval) or time-series data (e.g., each electrical activity waveform generated by each of one or more EEG electrodes attached to the subject's scalp over a certain time interval). EEG data can be captured at the same time interval as the accelerometer data 104.

[0028] In some implementations, the classification system 100 receives additional video data, which includes video showing a part of an object (e.g., the face or torso) or the entire object (e.g., the whole body) over a certain time interval. The video data may be captured by a video recording device such as a webcam or a smartphone video recording device. The video data can be represented as a series of video frames captured at any appropriate sampling frequency, e.g., 24 frames / second. The video data may be captured at the same time interval as the accelerometer data 104.

[0029] The classification system 100 can process accelerometer data 104 and any additional inputs characterizing the object 102 using the classification neural network 200 and classification engine 110, respectively, as described below. (The classification neural network 200 will also be referred to as the “sleep-wake” classification neural network throughout this specification.)

[0030] The classification neural network 200 is configured to receive network input 106, which includes accelerometer data 104 and optionally any additional inputs that characterize the subject or the subject's environment, such as cardiovascular data, ambient light data, audio data, time data, EEG data, video data, etc. The classification neural network 200 processes the network input 106 according to the values ​​of a set of classification neural network parameters to generate a score distribution across a set of sleep-wake classes.

[0031] A score distribution 112 across a set of sleep-wake classes can determine the respective scores for each sleep-wake class within the set. Each sleep-wake class may correspond to a specific sleep or wakefulness state. The scores for each sleep-wake class can determine the likelihood that the subject is in the corresponding sleep or wakefulness state. A set of sleep-wake classes may include any appropriate number of sleep-wake classes, e.g., two, three, four, or five sleep-wake classes. In particular, a set of sleep-wake classes may include (i) an “awake” class indicating that the subject is awake, and (ii) one or more “sleep” classes indicating that the subject is in a specific sleep stage.

[0032] In some implementations, the sleep-wake class set may include separate sleep classes corresponding to "non-REM sleep" (indicating the subject is in a non-REM sleep stage) and "REM sleep" (indicating the subject is in a REM sleep stage). In some implementations, the sleep-wake class set may include separate sleep classes corresponding to "Stage 1 non-REM sleep," "Stage 2 non-REM sleep," "Stage 3 non-REM sleep," and "REM sleep." In some implementations, the sleep-wake class set may include only two sleep-wake classes, namely the "awake" class and the "sleep" class, where the "sleep" class indicates the subject is in any sleep stage.

[0033] In an implementation where the sleep-wake class set includes only two classes, for example, a "wake" class and a "sleep" class, the classification neural network 200 may be configured to produce an output that includes scores for only one of the two classes. For example, the classification neural network 200 may be configured to produce a score that determines whether the subject is likely to belong to the "wake" class, or a score that determines whether the subject is likely to belong to the "sleep" class. The score of one class (from the set of two classes) can determine the score of the other class, for example, based on the requirement that the sum of the two scores is 1 (or another predetermined value). Thus, the score of one class can implicitly determine the score distribution across the set of two classes.

[0034] The classification neural network 200 may have any suitable neural network architecture that enables it to perform its described function, for example, the generation of score distributions across a set of sleep-wake classes. In particular, the classification neural network may include any suitable number of (e.g., 5, 10, or 50) of any suitable neural network layers (e.g., convolutional layers, attention layers, fully connected layers, recurrent layers, etc.) and be connected in any suitable configuration (e.g., as a linear sequence of layers). An exemplary architecture of the classification neural network will be described in more detail with reference to Figure 2.

[0035] The classification system 100 can use a training system to train a classification neural network 200 to perform a sleep-wake classification task. Optionally, the training system can pre-train a portion of the classification neural network 200 to perform one or more unsupervised or supervised-aided tasks before training the classification neural network 200 to perform the sleep-wake classification task. An example of a training system for training a classification neural network is described in more detail with reference to Figure 5.

[0036] The classification engine 110 is configured to process a score distribution 112 across a set of sleep-wake classes in order to generate a sleep-wake classification 112. The sleep-wake classification engine 110 can, for example, generate a sleep-wake classification that classifies a subject as belonging to the sleep-wake class associated with the highest score under the score distribution 112 across the set of sleep-wake classes.

[0037] Optionally, the classification engine 110 can generate a confidence score that characterizes the confidence of the classification system 100 in the sleep-wake classification 112 generated for the subject 102. The classification engine 110 can generate the confidence score of the sleep-wake classification 112 in any suitable way. For example, the classification engine 110 can generate the confidence score by calculating the entropy of the score distribution 108 across the set of sleep-wake classes. In this example, a higher entropy of the score distribution 108 across the set of sleep-wake classes may reflect a higher uncertainty in the classification system 100 in the sleep-wake classification 112.

[0038] The classification system 100 can use the sleep-wake classification 112 generated for the subject 102 by any of the following methods. Next, some exemplary uses of the sleep-wake classification 112 will be described.

[0039] In some implementations, the classification system 100 receives accelerometer data 104 (and optionally other inputs such as video or audio data) for each time interval in a series of time intervals. The classification system 100 can process the accelerometer data 104 for each time interval to generate a corresponding sleep-wake classification 112. Thus, the classification system 100 can generate a series of sleep-wake classifications 112, each corresponding to a different time interval and characterizing the sleep-wake state of the subject 102 over that time interval. The series of sleep-wake classifications can provide detailed and real-time monitoring of the subject 102's sleep-wake state without requiring the subject 102 to undergo a sleep examination.

[0040] In some implementations, the classification system 100 can process a series of sleep-wake classifications (as described above) to predict the duration of a subject 102 being in a particular sleep-wake state over a time window (e.g., a 24-hour time window). For example, the classification system 100 can predict the duration of a subject 102 being in a particular sleep-wake class over a time window based on a combination (e.g., product) of (i) the number of time intervals (over a time window) in which the classification system 100 classified the subject as being in a sleep-wake class and (ii) the duration of those time intervals (e.g., 90 seconds). As another example, the classification system 100 can predict the duration of a subject 102 being in a specified appropriate subset of a set of sleep-wake classes over a time window based on a combination of (i) the number of time intervals (over a time window) in which the classification system 100 classified the subject as being in a sleep-wake class that belongs to an appropriate subset of the sleep-wake class and (ii) the duration of those time intervals (e.g., 90 seconds). Therefore, for example, the classification system can predict the duration of sleep that subject 102 was in, based on the number of time intervals in which the subject was classified as being in stage 1 non-REM sleep, stage 2 non-REM sleep, stage 3 non-REM sleep, or REM sleep.

[0041] The classification system 100 can generate notifications indicating the duration of time a subject is in a specific sleep-wake class (or a subset of sleep-wake classes) over a time window, and can provide these notifications to, for example, subject 102 or subject 102's healthcare provider. For example, the classification system 100 can send notifications via a data communication network, for example, as email or text message. Furthermore, the classification system 100 can automatically save the generated sleep-wake classification 112 (or data derived from the sleep-wake classification, as described above) for subject 102 to subject 102's electronic medical record. For example, the classification system 100 can interface with a database storing subject 102's electronic medical record, for example, via an application programming interface (API), and save the sleep-wake classification (or data derived from the sleep-wake classification) to the appropriate field in subject 102's electronic medical record.

[0042] In some implementations, the classification system 100 can trigger one or more actions based on its determination that the subject 102 belongs to a specific sleep-wake class (or a subset of sleep-wake classes) for a threshold duration. For example, in response to the classification system 100 determining that the subject 102 has slept for at least a threshold duration, e.g., 8 hours, it can trigger an alarm sound or turn on a light to transition the subject 102 to an awake state. As another example, in response to the classification system 100 determining that the subject 102 has been awake for at least a threshold duration, e.g., 20 hours, it can trigger a notification (e.g., a text message) indicating that the subject 102 should consider sleep.

[0043] In some implementations, the classification system 100 can generate an alarm in response to generating one or more sleep-wake classifications 112 indicating that the user has transitioned to a sleep state. The alarm can be configured to return the user to an awake state. In certain applications, the classification system 100 can be used to maintain wakefulness of a subject 102 who is in an environment where it is undesirable to be asleep or is engaged in an activity where it is undesirable to be asleep (e.g., driving a vehicle). The alarm can be, for example, an audible alarm, a visual alarm (e.g., a flashing light), or a motion alarm (e.g., vibration of the subject 102's wearable device).

[0044] Optionally, in combination with or as an alternative to generating sleep-wake classifications, the classification system 100 can generate medical symptom classifications. More specifically, the classification neural network 200 can be configured to process a network input 106 that characterizes an object 102 over a time interval to generate a score that determines the likelihood that the object will exhibit signs of a medical symptom during that time interval. The network input 106 may include accelerometer data 104 and optionally other types of data (e.g., video data, audio data, etc.). The medical symptom could be, for example, restless legs syndrome or seizures. The classification system 100 may classify an object 102 as exhibiting signs of a medical symptom over a time interval if, for example, the likelihood that the object will exhibit signs over the time interval (determined by the output of the classification neural network 200) meets (e.g., exceeds) a threshold.

[0045] The classification system 100 can generate a series of medical symptom classifications, each containing a medical symptom classification for each time interval in a series of time intervals. The classification system 100 can process the series of medical symptom classifications to generate a medical diagnosis. For example, in response to generating at least a threshold number of medical symptom classifications indicating that the subject has shown signs of a medical symptom, the classification system 100 can generate a medical diagnosis indicating that subject 102 is predicted to have a medical symptom. After generating the medical diagnosis, the classification system 100 can generate a notification indicating the diagnosis, which can be sent, for example, to subject 102 or subject 102's healthcare provider.

[0046] Figure 2 shows an exemplary architecture of a classification neural network 200 included in the classification system 100 described, for example, with reference to Figure 1. The classification neural network 200 is configured to receive a network input 106 which includes accelerometer data characterizing the motion of an object. The network input 106 may include additional data characterizing the object or its environment, such as cardiovascular data, video data, audio data, or temporal data, as described above with reference to Figure 1. The classification neural network 200 processes the network input 106 to generate a score distribution across a set of sleep-wake classes 112.

[0047] The classification neural network 200 includes an encoder subnetwork 202 and a projection subnetwork 206, each of which will be described in detail below.

[0048] The encoder subnetwork 202 is configured to process the network input 106 in order to generate an embedding 204 of the network input 106 in latent space. The encoder subnetwork 202 may have any suitable neural network architecture that enables the encoder subnetwork 202 to generate the embedding 204 of the network input 106. In particular, the encoder subnetwork may include any suitable type of neural network layer (e.g., convolutional layers, recurrent layers, attention layers, fully connected layers, etc.) in any number (e.g., 5, 10, or 50 layers) and in any suitable configuration (e.g., as a linear sequence of layers). A specific example of the architecture of the encoder subnetwork 202 will be described in more detail with reference to Figure 3.

[0049] The network input 106 may include data from multiple modalities, such as accelerometer data, video data, audio data, time data, and cardiovascular data. The architecture of the encoder subnetwork 202 can be configured in any of several possible ways to enable the encoder subnetwork 202 to process multimodal data. For example, the encoder subnetwork 202 may include a sequence of encoder neural network layers corresponding to each modality. For each modality, the encoder subnetwork 202 can use the corresponding sequence of encoder neural network layers to process the data derived from that modality and generate a modality-specific embedding of the data. The encoder subnetwork 202 can combine the modality-specific embeddings of each modality included in the network input 106 to generate an embedding 204 of the network input 106. The encoder subnetwork 202 can combine modality-specific embeddings, for example, by concatenating them or by pooling them (e.g., averaging, summing, or max pooling).

[0050] The projection subnetwork 206 is configured to process the embedding 204 of the network input 106 to generate a score distribution 108 across a set of sleep-wake classes. The projection subnetwork 206 can have any suitable neural network architecture that enables the projection subnetwork 206 to generate a score distribution across a set of sleep-wake classes. In particular, the projection subnetwork can include any suitable type of neural network layer (e.g., convolutional layers, recurrent layers, attention layers, fully connected layers, etc.) in any number (e.g., 5, 10, or 50 layers) and in any suitable configuration (e.g., as a linear sequence of layers). In one specific example, the projection subnetwork 206 may include a sequence of dense (fully connected) neural network layers. Another specific example of the projection subnetwork 206 is described with reference to Figure 3A.

[0051] Optionally, the classification neural network 200 may include a second projection subnetwork configured to process embeddings 204 of the network input 106 to generate a score that determines the likelihood that a subject is exhibiting signs of a medical condition, such as restless legs syndrome or seizures.

[0052] Figure 3A shows an exemplary architecture of a classification neural network implemented using one or more residual blocks. The classification neural network can be included in the classification system 100, as described with reference to Figure 1. The classification neural network 200 is configured to process a network input 106 containing accelerometer data (and optionally additional data such as cardiovascular data, video data, audio data, and time data) that characterizes the subject, and to produce a network output that defines a score distribution across a set of sleep-wake classes 112. The classification neural network includes an encoder subnetwork 202 and a projection subnetwork 206.

[0053] In the exemplary architecture illustrated in Figure 3A, the classification neural network includes residual blocks (e.g., 302, 308, 314, 320), pooling layers (e.g., 304, 310, 316, 322, 326), dropout layers (e.g., 306, 312, 318, 324), and one or more fully connected layers (e.g., 328).

[0054] A residual block refers to a block (i.e., a group of neural network layers) configured to process a block input with one or more neural network layers to generate intermediate outputs, and then generate a block output by combining (e.g., summing) the block input and intermediate outputs. In other words, a residual block includes shortcut connections that allow the input to the block to be combined with the output of the block. Including residual blocks in a neural network can improve the stability of the neural network and allow it to learn more efficiently, for example, by reducing the effects of vanishing gradients during training. An exemplary architecture of a residual block is illustrated with reference to Figure 3B.

[0055] The architecture of a residual block can be partially defined by the "kernel," "extension," and "filter" parameters. The kernel parameter can, for example, define the number of weights included in the filters of the residual block's convolutional neural network layer. The "extension" parameter can define the spacing between weights included in the filters of the residual block's convolutional neural network layer. The "filter" parameter can define the number of convolutional filters within the residual block's convolutional neural network layer.

[0056] A pooling layer is configured to receive a layer input containing a set of embeddings and to combine the embeddings contained in the layer input through a pooling operation to produce a layer output containing fewer embeddings. The pooling operation may be, for example, a maximization operation or an averaging operation. The architecture of the pooling layer can be partially defined by the "pool" parameter and the "stride" parameter. The "pool" parameter can define the number of embeddings that the pooling operation is configured to combine into a single embedding. The "stride" parameter can define the resolution of the pooling operation.

[0057] A dropout layer works by randomly setting some input units to zero with a certain probability during training of a neural network, thus reducing the risk of overfitting. The architecture of a dropout layer can be partially defined by a "probability" parameter that determines the likelihood that each input unit to the dropout layer will be set to zero during training.

[0058] Figure 3B shows an exemplary architecture of a residual block included in a classification neural network, for example, the classification neural network illustrated in Figure 3A. The residual block can be configured to process a set of input embeddings derived from accelerometer data characterizing the motion of an object, according to the values ​​of a set of residual block parameters, to generate a set of output embeddings. The residual block includes skip (shortcut) connections 330, one-dimensional (1-D) convolutional layers (e.g., 332, 336, 340), batch normalization (BN) layers (e.g., 334, 338, 342), and rectified linear unit (ReLU) layers (e.g., 334, 338, 344).

[0059] Figure 4 is a flowchart of an exemplary process 400 for performing a sleep-wake classification task using a sleep-wake classification neural network. For convenience, process 400 is described as being performed by a system of one or more computers located in one or more locations. For example, a classification system appropriately programmed according to this specification, e.g., classification system 100 in Figure 1, can perform process 400.

[0060] The system receives a network input containing accelerometer data generated by the target wearable device (402). Optionally, the network input may include one or more additional data, such as cardiovascular data (e.g., heart rate data, heart rate variability data, blood oxygen saturation data, etc.), EEG data, ambient light data, audio data, time data, video data, etc.

[0061] A sleep-wake classification neural network is used to process the network input and generate a network output that defines a score distribution across a set of sleep-wake classes (404). More specifically, the system processes the network input (including accelerometer data) using an encoder subnetwork of the sleep-wake classification neural network to generate an embedding of the accelerometer data in latent space. The system then processes the embedding of the accelerometer data using a projection subnetwork of the sleep-wake classification neural network to generate a score distribution across a set of sleep-wake classes.

[0062] A set of sleep-wake classes includes (i) at least one class corresponding to a wakeful state and (ii) at least one class corresponding to a sleep state. A set of sleep-wake classes may include multiple classes corresponding to each sleep state. For example, a set of sleep-wake classes may include the respective classes corresponding to Stage 1 non-REM sleep, Stage 2 non-REM sleep, Stage 3 non-REM sleep, and REM sleep.

[0063] A sleep-wake classification neural network may include one or more residual blocks. Each residual block is configured to process its block input with one or more neural network layers to produce an intermediate output, and to generate a block output by summing (i) the block input and (ii) the intermediate output.

[0064] The system can train a sleep-wake classification neural network using machine learning training techniques. In particular, the system can pre-train the encoder subnetwork of the sleep-wake classification neural network to perform an auxiliary task, which is different from the sleep-wake classification task. After pre-training the encoder subnetwork, the system can train the sleep-wake classification neural network to perform the sleep-wake classification task. An example of a training system for training a sleep-wake classification neural network is illustrated in Figure 5.

[0065] The system classifies the sleep-wake state in question based on its respective score for each class in a set of sleep-wake classes (406). For example, the system can classify the sleep-wake state in question into the class associated with the highest score in the set of sleep-wake classes.

[0066] Figure 5 shows an exemplary training system 500. The training system 500 is an example of a system implemented as a computer program on one or more computers in one or more locations where the systems, components, and techniques described below are implemented.

[0067] The training system 500 is configured to train the classification neural network 200 included in the classification system 100, as described with reference to Figure 1. The classification neural network 200 is configured to process network inputs, including accelerometer data characterizing the movement of an object, to generate a score distribution across a set of sleep-wake classes.

[0068] The classification neural network 200 includes an encoder subnetwork and a projection subnetwork, as illustrated with reference to Figure 2. The encoder subnetwork can process the network input, which includes accelerometer data, to generate an embedding of the network input. The projection subnetwork can process the embedding of the network input to generate a score distribution across a set of sleep-wake classes.

[0069] The training system 500 can train a classification neural network in a two-stage sequence. In the first stage, the training system 500 can pre-train the encoder subnetwork 202 of the classification neural network 200 to perform one or more auxiliary tasks. In the second stage, the training system 500 can train the classification neural network 200 to perform a sleep-wake classification task. Pre-training of the encoder subnetwork 202 can provide effective initialization of the parameter values ​​of the encoder neural network, thus facilitating fine-tuning of the classification neural network for performing sleep-wake classification. In particular, pre-training can prompt the encoder subnetwork 202 to generate embeddings that encode rich features characterizing the structure of accelerometer data (and optionally, other types of data, e.g., cardiovascular data, video data, audio data, etc.). By pre-training the encoder subnetwork 202, the classification neural network 200 can achieve higher predictive accuracy for the sleep-wake classification task while requiring less labeled training data than would otherwise be necessary.

[0070] In the first stage of training, the training system 500 can initialize a set of parameters for the encoder subnetwork 202 using an appropriate initialization technique, such as random initialization or glorot initialization. The training system 500 can then train the encoder subnetwork 202 to perform auxiliary tasks. Several examples of auxiliary tasks are described below.

[0071] In some implementations, the training system 500 can pre-train the encoder subnetwork 202 to perform a symmetric embedding task 502. More specifically, the training system 500 can train the encoder subnetwork to generate similar embeddings for "positive" pairs of accelerometer signals and different embeddings for "negative" pairs of accelerometer signals. A "pair" of accelerometer signals includes a first accelerometer signal and a second accelerometer signal. A "positive" pair of accelerometer signals can refer to a pair of accelerometer signals where both the first and second accelerometer signals are transformed versions of the same underlying accelerometer signal. A "negative" pair of accelerometer signals can refer to a pair of accelerometer signals where the first accelerometer signal is derived from a different underlying accelerometer signal than the second accelerometer signal. An exemplary process for training the encoder subnetwork 202 to perform the symmetric embedding task 502 is described in more detail with reference to Figures 6A and 6B.

[0072] In some implementations, the training system 500 can pre-train the encoder subnetwork 202 to perform a masked reconstruction task 504. More specifically, the training system can train the encoder subnetwork to process a masked accelerometer signal to generate an embedding that enables accurate reconstruction of the complete (unmasked) accelerometer signal. An exemplary process for training the encoder subnetwork 202 to perform the masked reconstruction task 504 is described in more detail with reference to Figure 7.

[0073] In some implementations, the training system 500 can pre-train the encoder subnetwork 202 to perform a noisy reconstruction task 506. More specifically, the training system can train the encoder subnetwork to process a noisy accelerometer signal to generate an embedding that enables accurate reconstruction of the original (de-noised) accelerometer signal. An exemplary process for training the encoder subnetwork 202 to perform the noisy reconstruction task 506 is described in more detail with reference to Figure 8.

[0074] In some implementations, the training system 500 can pre-train the encoder subnetwork 202 to perform one or more supervised aid tasks 508. More specifically, the training system can train the encoder subnetwork to process the accelerometer signal to generate an embedding that enables accurate prediction of one or more features of the accelerometer signal, such as the number of steps taken by the subject during the duration covered by the accelerometer signal, or the actions performed by the subject during the duration covered by the accelerometer signal. An exemplary process for training the encoder subnetwork to perform the supervised aid task 508 is described in more detail with reference to Figure 9.

[0075] Optionally, the training system 500 can pre-train the encoder subnetwork to perform multiple auxiliary tasks, i.e., not just a single auxiliary task. For example, the training system 500 can pre-train the encoder subnetwork to perform two auxiliary tasks, three auxiliary tasks, or four auxiliary tasks.

[0076] In the second stage of training, the training system 500 can initialize the set of parameters for the projection subnetwork using an appropriate initialization method, such as random initialization or glorot initialization. (The set of parameters for the encoder subnetwork 202 may have values ​​determined during the pre-training of the encoder subnetwork in the first stage of training.) The training system 500 can then train the classification neural network 200 to perform the sleep-wake classification task. An exemplary process for training the classification neural network to perform the sleep-wake classification task 510 will be described in more detail with reference to Figure 10.

[0077] In some implementations, the training system 500 trains a classification neural network 200 to perform both a sleep-wake classification task and a medical symptom classification task. In these implementations, the classification neural network 200 may include a first projection subnetwork that generates a score distribution across a set of sleep-wake classes and a second projection subnetwork that generates a score determining the likelihood that a subject has a medical symptom (as illustrated with reference to Figure 1). The training system 500 can leverage the synergies between the sleep-wake classification task and the medical symptom classification task to achieve higher predictive accuracy in both tasks. More specifically, as part of training the classification neural network to perform the sleep-wake classification task, the training system can backpropagate the gradient of the sleep-wake classification objective function through the first projection subnetwork to the encoder subnetwork of the classification neural network. Similarly, as part of training the classification neural network to perform the medical symptom classification task, the training system can backpropagate the gradient of the medical symptom classification objective function through the second projection subnetwork to the encoder subnetwork of the classification neural network. Therefore, the parameter values ​​of the encoder subnetwork can be jointly trained using training signals from both the sleep-wake classification task and the medical symptom classification task, thereby enabling the encoder subnetwork to learn to leverage the synergies and commonalities between the tasks. An exemplary process for training a classification neural network to perform the medical symptom classification task is illustrated with reference to Figure 11.

[0078] For convenience, the first stage of training—in particular, the pre-training of the encoder neural network—will be described as being performed before the second stage of training—in particular, the fine-tuning of the classification neural network for performing sleep-wake classification and / or medical symptom classification. However, the two stages of training can overlap, for example, so that both the pre-training of the encoder neural network and the fine-tuning of the classification neural network are performed in one or more identical training iterations.

[0079] After training the classification neural network 200, the training system can provide the classification neural network 200 for use by the classification system 100, as described with reference to Figure 1.

[0080] Figure 6A is a flowchart of an exemplary process 600 for pre-training an encoder subnetwork of a classification neural network to perform a controlled embedding task. For convenience, process 600 is described as being performed by a system of one or more computers located in one or more locations. For example, a training system appropriately programmed according to this specification, e.g., training system 500 in Figure 5, can perform process 600. The training system can iteratively perform the steps of process 600 as part of the pre-training of the encoder subnetwork.

[0081] The system acquires a set of "base" accelerometer signals (602).

[0082] The system generates one or more pairs of positive accelerometer signals (604). To generate pairs of positive accelerometer signals, the system selects base accelerometer signals from a set of base accelerometer signals (e.g., by random sampling). The system generates a first transformed version of the base accelerometer signal by, for example, randomly sampling a first transformation from the transformation space and applying the first transformation to the base accelerometer signal. The system generates a second transformed version of the base accelerometer signal by, for example, randomly sampling a second transformation from the transformation space and applying the second transformation to the base accelerometer signal. The first and second transformed versions of the base accelerometer signal jointly determine the pairs of positive accelerometer signals.

[0083] The transformation space may include, for example, flip transforms (including flipping the accelerometer signal around the time axis), inversion transforms (including reversing the direction of the accelerometer signal on the time axis), zoom transforms (including cropping and resizing the accelerometer signal), swap transforms (including swapping the order of the accelerometer signals), noise transforms (including adding random noise to the accelerometer signal), resize transforms (including changing the amplitude of the accelerometer signal), and identity transforms (which do not affect the accelerometer signal). In some cases, a positive pair of accelerometer signals may include the same accelerometer signal as the base accelerometer signal, for example, in situations where the system selects an identity transform to apply to the base accelerometer signal.

[0084] The system generates one or more positive pair embeddings (606). In particular, for each positive pair of accelerometer signals, the system processes the first accelerometer signal using an encoder subnetwork to generate an embedding for the first accelerometer signal, and the system processes the second accelerometer signal to generate an embedding for the second accelerometer signal. The embedding for the first accelerometer signal and the embedding for the second accelerometer signal jointly define the positive pair embedding.

[0085] In some implementations, the system uses both an encoder subnetwork and another neural network, conveniently called an embedded neural network, to generate embeddings of the accelerometer signal during the pre-training of the encoder subnetwork. Specifically, to generate embeddings of the accelerometer signal, the system can use the encoder subnetwork to process the accelerometer signal to generate a first embedding, and then use the embedded neural network to process the first embedding to generate a second embedding. During the pre-training of the encoder subnetwork, the system can define the embeddings generated by the embedded neural network as the embeddings of the accelerometer signal.

[0086] The system generates one or more pairs of negative accelerometer signals (608). To generate pairs of negative accelerometer signals, the system selects a pair of base accelerometer signals that include a first base accelerometer signal and a second different base accelerometer signal (e.g., by random sampling). The system generates a transformed version of the first base accelerometer signal, for example, by randomly sampling a transform from the transform space and applying the transform to the first base accelerometer signal. The system generates a transformed version of the second base accelerometer signal, for example, by randomly sampling a transform from the transform space and applying the transform to the second base accelerometer signal. The transformed versions of the first and second base accelerometer signals jointly determine the pairs of negative accelerometer signals. The system may select the identity transform of the first or second base accelerometer signal such that the pairs of negative accelerometer signals include the base accelerometer signals (rather than the modified accelerometer signals).

[0087] The system generates one or more negative pair embeddings (610). Specifically, for each negative pair of accelerometer signals, the system processes the first accelerometer signal using an encoder subnetwork to generate the first accelerometer signal embedding, and the system processes the second accelerometer signal to generate the second accelerometer signal embedding. The first and second accelerometer signal embeddings jointly define the negative pair embedding. In some implementations, the system generates accelerometer signal embeddings using both an encoder subnetwork and an embedding neural network, as described above.

[0088] The system trains an encoder subnetwork to optimize an auxiliary contrast objective function that depends on positive pair embeddings and negative pair embeddings (612). More specifically, the auxiliary contrast objective function can measure the error (e.g., Euclidean distance) between a first embedding and a second embedding for each pair embedding. In particular, for each positive pair embedding, the auxiliary contrast objective function can promote a higher similarity between the first and second embeddings. For each negative pair embedding, the auxiliary contrast objective function can promote a lower similarity between the first and second embeddings.

[0089] To train the encoder subnetwork to optimize the auxiliary counter-objective function, the system can determine the gradient of the auxiliary counter-objective function (e.g., using backpropagation) and then use the gradient to adjust the parameter values ​​of the encoder subnetwork according to, for example, a gradient descent optimization algorithm, e.g., RMSprop or Adam update rules. That is, the system can backpropagate the gradient of the auxiliary counter-objective function through the encoder subnetwork. In implementations where the system uses both the encoder subnetwork and the embedded neural network to generate an embedding of the accelerometer signal (as described above), the system can jointly train the encoder subnetwork and the embedded neural network by, for example, backpropagating the gradient to the encoder subnetwork through the embedded neural network.

[0090] Figure 6B provides a diagram illustrating the pre-training of the encoder subnetwork to perform the symmetric embedding task, as described with reference to Figure 6A. In particular, Figure 6B illustrates the application of a first transformation 616 to the base accelerometer signal 614 to generate a first transformed version 618 of the base accelerometer signal 614, and the application of a second transformation 620 to the base accelerometer signal 614 to generate a second transformed version 622 of the base accelerometer signal 614. The system process uses the encoder subnetwork 202 and the embedding neural network 624 to generate the respective embeddings of the first and second transformed versions of the base accelerometer signal. The system then jointly trains the encoder subnetwork and the embedding neural network to optimize the auxiliary symmetric objective function 628.

[0091] Figure 7 is a flowchart of an exemplary process 700 for pre-training an encoder subnetwork of a classification neural network to perform a masked reconstruction task. For convenience, process 700 is described as being performed by a system of one or more computers located in one or more locations. For example, a training system appropriately programmed according to this specification, e.g., training system 500 in Figure 5, can perform process 700. The training system can iteratively perform the steps of process 700 as part of the pre-training of the encoder subnetwork.

[0092] The system acquires the accelerometer signal (702).

[0093] The system generates a masked accelerometer signal by masking a portion of the accelerometer signal (704). The system can mask a portion of the accelerometer signal by replacing that portion with default data, such as a predetermined value (e.g., zero) or random noise (e.g., sampled from a normal distribution). The system can randomly select the portion of the accelerometer signal to mask.

[0094] The system uses an encoder subnetwork to process the masked accelerometer signals and generate an embedding of the masked accelerometer signals (706).

[0095] The system uses a decoder neural network to process the embedding of the masked accelerometer signal to generate a predictive reconstruction of the accelerometer signal (708). The decoder neural network may have any suitable neural network architecture that enables the decoder neural network to generate a predictive reconstruction of the accelerometer signal. In particular, the decoder neural network may include any suitable type of neural network layer (e.g., convolutional layers, recurrent layers, attention layers, fully connected layers, etc.) in any number (e.g., 5, 10, or 50 layers) and in any suitable configuration (e.g., as a linear sequence of layers).

[0096] The system jointly trains an encoder subnetwork and a decoder neural network to optimize an auxiliary reconstruction objective function that measures the error in predictive reconstruction of the accelerometer signal (710). The error can be measured, for example, as an L1 error, or an L2 error, or using any other appropriate error metric.

[0097] Figure 8 is a flowchart of an exemplary process 800 for pre-training an encoder subnetwork of a classification neural network to perform a noisy reconstruction task. For convenience, process 800 is described as being performed by a system of one or more computers located in one or more locations. For example, a training system appropriately programmed according to this specification, e.g., training system 500 in Figure 5, can perform process 800. The training system can iteratively perform the steps of process 800 as part of the pre-training of the encoder subnetwork.

[0098] The system acquires the accelerometer signal (802).

[0099] The system generates a noisy accelerometer signal by adding noise to the accelerometer signal (804). The system can sample noise from a predetermined probability distribution, such as a normal distribution.

[0100] The system uses an encoder subnetwork to process the noisy accelerometer signal and generate an embedding of the noisy accelerometer signal (806).

[0101] The system uses a decoder neural network to process the embedding of the noisy accelerometer signal to generate a predicted denoised accelerometer signal, i.e., a predicted reconstruction of the original (denoised) accelerometer signal (808). The decoder neural network may have any suitable neural network architecture that enables the decoder neural network to generate the predicted denoised accelerometer signal. In particular, the decoder neural network may include any suitable type of neural network layer (e.g., convolutional layers, recurrent layers, attention layers, fully connected layers, etc.) in any number (e.g., 5, 10, or 50 layers) and in any suitable configuration (e.g., as a linear sequence of layers).

[0102] The system jointly trains an encoder subnetwork and a decoder neural network to optimize an auxiliary denoising objective function that measures the error in the denoised accelerometer signal (810). More specifically, the auxiliary denoising objective function can measure the error between (i) the original accelerometer signal and (ii) the predicted denoised accelerometer signal. The error can be measured, for example, as an L1 error, or as an L2 error, or using any other appropriate error metric.

[0103] Figure 9 is a flowchart of an exemplary process 900 for pre-training an encoder subnetwork of a classification neural network to perform a supervised aid task. For convenience, process 900 is described as being performed by a system of one or more computers located in one or more locations. For example, a training system appropriately programmed according to this specification, e.g., training system 500 in Figure 5, can perform process 900. The training system can iteratively perform the steps of process 900 as part of the pre-training of the encoder subnetwork.

[0104] The system obtains (i) an accelerometer signal and (ii) a target label for the accelerometer signal (902). In some implementations, the target label for the accelerometer signal determines the number of steps taken by the subject during the duration covered by the accelerometer signal. In some implementations, the target label for the accelerometer signal determines the action performed by the subject during the duration covered by the accelerometer signal.

[0105] The system uses an encoder subnetwork to process the accelerometer signal and generate an embedding of the accelerometer signal (904).

[0106] The system uses a predictive neural network to process the embedding of the accelerometer signal and generate a predictive output that characterizes the predicted labels for the accelerometer signal (906). In some implementations, the predictive output directly defines the predicted labels. In some implementations, the predictive output defines a score distribution across a set of possible labels.

[0107] The system jointly trains an encoder subnetwork and a predictive neural network to optimize an auxiliary supervised objective function that measures the error between (i) a target label and (ii) a predicted output characterizing the predicted label (908). The auxiliary supervised objective function can measure the error, for example, as cross-entropy error or as squared error.

[0108] Figure 10 is a flowchart of an exemplary process 1000 for training a classification neural network to perform a sleep-wake classification task. For convenience, process 1000 is described as being performed by one or more computer systems located in one or more locations. For example, a training system appropriately programmed according to this specification, e.g., training system 500 in Figure 5, can perform process 1000. The training system can iteratively perform the steps of process 1000 as part of training the classification neural network.

[0109] The system acquires (i) a network input including an accelerometer signal (and optionally one or more other types of data, such as cardiovascular data, audio data, or video data), and (ii) a target sleep-wake classification corresponding to the network input. (1002)

[0110] The system uses an encoder subnetwork to process the network input and generate an embedding of the network input (1004).

[0111] The system uses a projection subnetwork to process the embedding of the network input to generate a score distribution across a set of sleep-wake classes (1006).

[0112] The system trains a projection subnetwork of a classification neural network to optimize a sleep-wake objective function that measures the error between (i) a score distribution across a set of sleep-wake classes and (ii) a target sleep-wake classification (1008). The sleep-wake objective function can be measured for error, for example, as cross-entropy error.

[0113] In particular, the system determines the gradient of the sleep-wake objective function using, for example, backpropagation, and then uses the gradient to adjust the values ​​of the set of parameters of the projection subnetwork by appropriate gradient descent optimization techniques, such as RMSprop or Adam update rules. That is, the system backpropagates the gradient of the sleep-wake objective function through the projection subnetwork. In some implementations, the system jointly trains the projection subnetwork and the encoder subnetwork by backpropagating the gradient through the projection subnetwork to the encoder subnetwork. In other implementations, the system freezes the parameter values ​​of the encoder subnetwork after pretraining it to perform one or more auxiliary tasks, and does not adjust the parameter values ​​of the encoder subnetwork to optimize the sleep-wake objective function.

[0114] Figure 11 is a flowchart of an exemplary process 1100 for training a classification neural network to perform a medical symptom classification task. For convenience, process 1100 is described as being performed by one or more computer systems located in one or more locations. For example, a training system appropriately programmed according to this specification, e.g., training system 500 in Figure 5, can perform process 1100. The training system can iteratively perform the steps of process 1100 as part of training the classification neural network.

[0115] The system acquires (i) a network input containing accelerometer signals (and optionally, one or more other types of data, such as cardiovascular data, audio data, video data, etc.) and (ii) a target medical symptom classification corresponding to the network input (1102).

[0116] The system uses an encoder subnetwork to process the network input and generate an embedding of the network input (1104).

[0117] The system uses a second projection subnetwork to process the embedding of network inputs and generate a score that determines the likelihood that a subject has a medical condition (1106).

[0118] The system trains a second projection subnetwork of the classification neural network to optimize a medical symptom objective function that measures the error between (i) a score determining the likelihood that a subject has a medical symptom and (ii) a target medical symptom classification (1108). The medical symptom objective function can be measured for error, for example, as cross-entropy error.

[0119] The system determines the gradient of the medical symptom objective function, for example, using backpropagation, and then uses the gradient to adjust the values ​​of a set of parameters in a second projection subnetwork using an appropriate gradient descent optimization technique, such as RMSprop or Adam's update rule. In other words, the system backpropagates the gradient of the medical symptom objective function through the second projection subnetwork. In some implementations, the system jointly trains the second projection subnetwork and the encoder subnetwork by backpropagating the gradient through the second projection subnetwork to the encoder subnetwork. In other implementations, the system freezes the parameter values ​​of the encoder subnetwork after pretraining it to perform one or more auxiliary tasks, and does not adjust the parameter values ​​of the encoder subnetwork to optimize the medical symptom objective function.

[0120] Figure 12A shows experimental results illustrating the effect of pre-training the encoder neural network of a classification neural network. Bar graph 1202 compares the balanced accuracy of the classification neural network on the sleep-wake classification task with and without pre-training the encoder subnetwork ("supervised"). Bar graph 1204 compares the macro-mean F1 score of the classification neural network on the sleep-wake classification task with and without pre-training the encoder subnetwork ("supervised"). Each bar graph shows experimental results when the classification neural network is trained with all labeled training data ("100% trained patients") and when the classification neural network is trained with half of the labeled training data ("50% trained patients"). Pre-training generally improves the performance of the classification neural network, and the improvement in performance is particularly pronounced in the configuration where only half of the labeled training data is used for training.

[0121] Figure 12B shows experimental results illustrating the sleep-wake classification accuracy of the system described herein ("System #3 (Our System)") compared to two other sleep-wake classification systems, "System #1" and "System #2." The system described herein is superior to both of the other systems used for comparison.

[0122] This specification uses the term “configured” in relation to system and computer program components. For a system of one or more computers, to be configured to perform a particular operation or action means that software, firmware, hardware, or a combination thereof is installed on the system that causes the system to perform the operation or action during operation. For one or more computer programs to be configured to perform a particular operation or action means that when the programs are executed by a data processing device, they contain instructions that cause the device to perform the operation or action.

[0123] The subject matter and functional operating embodiments described herein may be implemented in digital electronic circuits, tangibly implemented computer software or firmware, computer hardware including structures disclosed herein and their structural equivalents, or one or more combinations thereof. Embodiments of the subject matter described herein may be implemented as one or more modules of computer program instructions, i.e., computer program instructions executed by a data processing device or coded in a tangible non-temporary storage medium for controlling the operation of a data processing device. The computer storage medium may be a machine-readable storage device, a machine-readable storage board, a random or serial access memory device, or one or more combinations thereof. Alternatively or additionally, the program instructions may be coded in artificially generated propagating signals, such as mechanically generated electrical, optical or electromagnetic signals generated to code information for transmission to a receiving device suitable for execution by a data processing device.

[0124] The term "data processing device" refers to data processing hardware and encompasses all kinds of devices, machines, and equipment for data processing, including, for example, programmable processors, computers, or multiple processors or computers. A device may also be, or further include, dedicated logic circuits, such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits). Optionally, in addition to hardware, a device may include code that forms the execution environment for computer programs, such as processor firmware, protocol stacks, database management systems, operating systems, or one or more combinations thereof.

[0125] Computer programs, also called or written as programs, software, software applications, apps, modules, software modules, scripts, or code, may be written in any form of programming language, including compiler or interpreter languages ​​or declarative or procedural languages, and may be deployed in any form, including standalone programs or modules, components, subroutines, or other units suitable for use in a computer environment. A program may, but is not required, correspond to a file in a file system. A program may be stored in part of a file that holds other programs or data, such as one or more scripts stored in a markup language document, a single file or multiple collaborative files dedicated to the program in question, such as a file that stores one or more modules, subprograms, or parts of code. A computer program may be deployed to run on one computer or one location, or distributed across multiple locations and interconnected by a data communication network.

[0126] In this specification, the term “engine” is used broadly to refer to a software-based system, subsystem, or process programmed to perform one or more specific functions. Generally, an engine is implemented as one or more software modules or components and installed on one or more computers located in one or more locations. In some cases, one or more computers are dedicated to a particular engine, while in other cases, multiple engines can be installed and operated on one or more of the same computers.

[0127] The processes and logic flows described herein may be executed by one or more programmable computers running one or more computer programs to perform functions by acting on input data and producing outputs. The processes and logic flows may also be executed by dedicated logic circuits, such as FPGAs or ASICs, or by a combination of dedicated logic circuits and one or more programmed computers.

[0128] A computer suitable for running computer programs may be based on a general-purpose or dedicated microprocessor, or both, or other types of central processing units. Generally, the central processing unit receives instructions and data from read-only memory, random-access memory, or both. Essential elements of a computer are a central processing unit that executes or carries out instructions, and one or more memory devices that store instructions and data. The central processing unit and memory may be reinforced or incorporated by dedicated logic circuits. Generally, a computer will have one or more mass storage devices for storing data, including, for example, magnetic, magneto-optical disks, or optical disks, or will be operablely coupled to receive data from or transfer data to them, or both. However, a computer is not required to have such devices. Furthermore, a computer may be incorporated into another device, to name a few, such as a mobile phone, personal digital assistant (PDA), mobile audio or video player, game console, Global Positioning System (GPS) receiver, or portable storage device, such as a Universal Serial Bus (USB) flash drive.

[0129] Computer-readable media suitable for storing computer program instructions and data include, for example, semiconductor memory devices such as EPROMs and EEPROMs, flash memory devices, magnetic disks such as internal hard disks or removable disks, magneto-optical disks, and all forms of non-volatile memory, media, and memory devices, including CD-ROMs and DVD-ROMs.

[0130] To enable user interaction, embodiments of the subject matter described herein may be implemented in a computer having a display device that displays information to the user, such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, and an indicator device that allows the user to provide input to the computer, such as a keyboard and mouse or trackball. Other types of devices may be used to enable user interaction; for example, the feedback provided to the user may be any form of sensory feedback, such as visual, auditory, or tactile feedback, and the input from the user may be received in any form, including acoustic, voice, or tactile input. In addition, the computer may interact with the user by sending and receiving documents to and from the device used by the user, for example, by sending a web page to a web browser on the user's device in response to a request received from a web browser. The computer may also interact with the user by sending text messages or other forms of messages to a personal device, such as a smartphone running a messaging application, and then receiving response messages from the user.

[0131] Data processing devices that implement machine learning models may include, for example, dedicated hardware accelerator units that handle the general and computationally intensive parts of machine learning training or production, i.e., inference workloads.

[0132] Machine learning models can be implemented and deployed using machine learning frameworks, such as the TensorFlow framework or the Jax framework.

[0133] Multiple embodiments of the subject matter described herein may be implemented in a computing system that includes, for example, a backend component as a data server, or a middleware component, such as an application server, or a frontend component, such as a client computer equipped with a graphical user interface, a web browser, or an application that allows a user to interact with the implementation of the subject matter described herein, or any combination of one or more such backend, middleware, or frontend components. The components of the system may be interconnected by digital data communication in any form or medium, such as a communication network. Examples of communication networks include local area networks (LANs) and wide area networks (WANs), such as the Internet.

[0134] A computing system may include a client and a server. The client and server are generally located remotely from each other and typically interact via a communication network. The client-server relationship arises from computer programs running on each computer that have a client-server relationship with each other. In some embodiments, the server sends data, such as an HTML page, to a user device for the purpose of displaying data to a user interacting with a device acting as a client and receiving input from the user. Data generated on the user device, such as the results of the interaction with the user, may be received from the device on the server side.

[0135] This specification includes many specific implementation details, but these should not be construed as limiting the scope of any invention or claim, but rather as describing features specific to a particular embodiment of a particular invention. Certain features described herein in relation to separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in relation to a single embodiment may also be implemented in multiple embodiments separately or in any suitable partial combination. Furthermore, even if features are described above as acting in a particular combination and are initially claimed in that manner, one or more features from the claimed combination may, in some cases, be excluded from that combination, and the claimed combination may refer to a partial combination or a variation of a partial combination.

[0136] Similarly, while operations are depicted in a specific order in the drawings and enumerated in the claims, this should not be understood as meaning that such operations must be performed in a specific or sequential order as illustrated, or that all the illustrated operations must be performed, in order to achieve the desired result. In certain situations, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged in multiple software products.

[0137] Specific embodiments of the subject matter have been described. Other embodiments are also included in the scope of the following claims. For example, the desired results can be achieved by performing the actions enumerated in the claims in a different order. As an example, the processes shown in the accompanying drawings do not necessarily have to be performed in the specific illustrated or sequential order in order to obtain the desired results. Multitasking and parallel processing may be advantageous in some cases.

Claims

1. A method performed by one or more computers, wherein the method is This involves performing a sleep-wake classification task using a sleep-wake classification neural network, Receiving accelerometer data generated by the target wearable device, Processing the accelerometer data using a sleep-wake classification neural network, The accelerometer data is processed using the encoder subnetwork of the sleep-wake classification neural network to generate an embedding of the accelerometer data in the latent space, The process involves using the projection subnetwork of the sleep-wake classification neural network to process the embedding of the accelerometer data and generate a network output that defines a score distribution across a set of sleep-wake classes. Includes, The set of sleep-wake classes includes (i) at least one class corresponding to the wakeful state, and (ii) at least one class corresponding to the sleep state. Processing and The sleep-wake state of the subject is classified based on the respective scores for each class in the set of sleep-wake classes. Including, including, The aforementioned sleep-wake classification neural network is The pre-training of the encoder subnetwork of the sleep-wake classification neural network to perform an auxiliary task, wherein the auxiliary task is different from the sleep-wake classification task, After pre-training the encoder subnetwork, the sleep-wake classification neural network is trained to perform the sleep-wake classification task. Trained by actions that include, method.

2. Pre-training the encoder subnetwork of the sleep-wake classification neural network in order to perform the aforementioned auxiliary task is To acquire the base accelerometer signal, This involves generating positive pair embeddings in the latent space, The encoder subnetwork is used to process a first transformed version of the base accelerometer signal to generate an embedding corresponding to the latent space, The encoder subnetwork is used to process a second transformed version of the base accelerometer signal to generate an embedding corresponding to the latent space, Including generating, The encoder subnetwork is trained to optimize an auxiliary objective function that depends on (i) the embedding of the first transformed version of the base accelerometer signal, and (ii) the embedding of the second transformed version of the base accelerometer signal. The method according to claim 1, including the method described in claim 1.

3. To generate the first converted version of the base accelerometer signal, Randomly sampling the first transformation from the transformation space, Applying the first transformation to the base accelerometer signal in order to generate the first transformed version of the base accelerometer signal, The method according to claim 2, further comprising generating, including

4. To generate the second converted version of the base accelerometer signal, Randomly sampling a second transformation from the transformation space, Applying the second transformation to the base accelerometer signal in order to generate the second transformed version of the base accelerometer signal, The method according to claim 2 or 3, further comprising generating, including

5. The method according to any one of claims 2 to 4, wherein the auxiliary objective function measures the error between (i) the embedding of the first converted version of the base accelerometer signal and (ii) the embedding of the second converted version of the base accelerometer signal.

6. Pre-training the encoder subnetwork of the sleep-wake classification neural network in order to perform the aforementioned auxiliary task is Acquiring accelerometer signals, By masking a portion of the aforementioned accelerometer signal, a masked accelerometer signal is generated. The encoder subnetwork is used to process the masked accelerometer signal to generate an embedding of the masked accelerometer signal, Using a decoder neural network, the embedding of the masked accelerometer signal is processed to generate a predictive reconstruction of the accelerometer signal. The encoder subnetwork and the decoder neural network are jointly trained to optimize the auxiliary objective function for measuring the error in the prediction reconstruction of the accelerometer signal. The method according to any one of claims 2 to 5, including the method described in any one of claims 2 to 5.

7. Pre-training the encoder subnetwork of the sleep-wake classification neural network in order to perform the aforementioned auxiliary task is Acquiring accelerometer signals, By adding noise to the aforementioned accelerometer signal, a noisy accelerometer signal is generated. The encoder subnetwork is used to process the noisy accelerometer signal to generate an embedding of the noisy accelerometer signal, The decoder neural network is used to process the embedding of the noisy accelerometer signal to generate a denoised accelerometer signal, The encoder subnetwork and the decoder neural network are jointly trained to optimize the auxiliary objective function for measuring the error of the denoised accelerometer signal. The method according to any one of claims 2 to 6, including the method described in any one of claims 2 to 6.

8. Pre-training the encoder subnetwork of the sleep-wake classification neural network in order to perform the aforementioned auxiliary task is (i) to obtain an accelerometer signal and (ii) to obtain a target label for the accelerometer signal, The encoder subnetwork is used to process the accelerometer signal to generate an embedding of the accelerometer signal, The process involves using a predictive neural network to process the embedding of the accelerometer signal and generate a predicted label for the accelerometer signal. The encoder subnetwork and the decoder neural network are jointly trained to optimize an auxiliary objective function that measures the error between the target label and the predicted label. The method according to any one of claims 2 to 7, including

9. The method according to claim 8, wherein the target label of the accelerometer signal determines the number of steps taken by the object during the duration covered by the accelerometer signal.

10. The method according to claim 8, wherein the target label of the accelerometer signal defines the action performed by the target during the duration covered by the accelerometer signal.

11. Training the sleep-wake classification neural network to perform the sleep-wake classification task is (i) to obtain a training accelerometer signal, and (ii) to obtain the target sleep-wake classification of the training accelerometer signal. The encoder subnetwork is used to process the training accelerometer signal to generate an embedding of the training accelerometer signal, Using the projection subnetwork, the embedding of the training accelerometer signal is processed to generate a score distribution across the set of sleep-wake classes, The projected subnetwork of the sleep-wake classification neural network is trained to optimize a sleep-wake objective function that measures the error between (i) the score distribution across the set of sleep-wake classes and (ii) the target sleep-wake classification. The method according to any one of claims 1 to 10, including the method described in any one of claims 1 to 10.

12. The method according to claim 11, wherein the parameter values ​​of the encoder subnetwork are frozen during the training of the sleep-wake classification neural network for performing the sleep-wake classification task.

13. Training the sleep-wake classification neural network to perform the sleep-wake classification task is Applying dropout to one or more layers of the sleep-wake classification neural network during training to perform the sleep-wake classification task. A method according to any one of claims 1 to 12, including the method described in any one of claims 1 to 12.

14. The sleep-wake classification neural network includes one or more residual blocks, each residual block being: Receive block input, The block input is processed by one or more neural network layers of the residual block to generate an intermediate output. (i) The block input and (ii) the intermediate output are summed to generate a block output. The method according to any one of claims 1 to 13, configured as described above.

15. The method according to any one of claims 1 to 14, wherein the set of sleep-wake classes includes a plurality of classes corresponding to each sleep state.

16. The method according to claim 15, wherein the set of sleep-wake classes includes a class corresponding to rapid eye movement (REM) sleep.

17. The method according to claim 15 or 16, wherein the set of sleep-wake classes includes one or more classes corresponding to non-REM sleep.

18. The method according to claim 17, wherein the set of sleep-wake classes includes each class corresponding to one or more of stage 1 non-REM sleep, stage 2 non-REM sleep, or stage 3 non-REM sleep.

19. Performing the aforementioned sleep-wake classification task further includes receiving the cardiovascular data of the subject, Processing the accelerometer data using the sleep-wake classification neural network is The accelerometer data and the cardiovascular data are processed together using the sleep-wake classification neural network. The method according to any one of claims 1 to 18, including the method described in any one of claims 1 to 18.

20. Performing the sleep-wake classification task further includes receiving ambient light data around the subject, Processing the accelerometer data using the sleep-wake classification neural network is The method according to any one of claims 1 to 19, comprising jointly processing the accelerometer data and the ambient light data using the sleep-wake classification neural network.

21. Performing the sleep-wake classification task further includes receiving audio data that characterizes the sounds surrounding the subject, Processing the accelerometer data using the sleep-wake classification neural network is The accelerometer data and the audio data are processed together using the sleep-wake classification neural network. The method according to any one of claims 1 to 20, including the method described in any one of claims 1 to 20.

22. Performing the sleep-wake classification task further includes receiving time data characterizing the current time when the accelerometer data was captured, Processing the accelerometer data using the sleep-wake classification neural network is The accelerometer data and the time data are processed together using the sleep-wake classification neural network. A method according to any one of claims 1 to 21, including the method described in any one of claims 1 to 21.

23. Classifying the sleep-wake state of the subject based on the respective scores for each class in the set of sleep-wake classes is: To classify the aforementioned sleep-wake state of the subject into the class associated with the highest score from the set of sleep-wake classes, The method according to any one of claims 1 to 22, including

24. The process involves repeatedly performing the sleep-wake classification over a time window to generate a set of sleep-wake classifications, The set of sleep-wake classifications is processed to predict the duration during which the subject was in a sleep state within the time window, The method according to any one of claims 1 to 23, further comprising:

25. In response to classifying the sleep-wake state of the subject as a sleep state, To trigger an alarm and transition the subject to an awakened state. The method according to any one of claims 1 to 24, further comprising:

26. It is a system, One or more computers, One or more storage devices communicably coupled to one or more computers, the one or more storage devices storing instructions that cause the one or more computers to perform the operation of each of the methods described in any one of claims 1 to 25 when executed by the one or more computers, A system that includes these features.

27. One or more non-temporary computer storage media that, when executed by one or more computers, store instructions causing one or more computers to perform the operation of each of the methods described in any one of claims 1 to 25.

Images