Training models to combine measures for detecting disease progression

Abstract

Introduced here is an approach for training machine learning models to generate combined measures to detect disease progression. Methods and systems discussed herein generate combined measures in a supervised or unsupervised manner using single measures derived from higher-frequency, longitudinal free-living data (or simply "free-living data") or lower-frequency, controlled-setting data, such as virtual motor exam (VME) data or gait analysis data. By generating combined measures, these models can more accurately detect disease progression, capturing a more comprehensive view of the patient's condition.

Description

PATENTAtorney Docket No. 124824.8148.WO01TRAINING MODELS TO COMBINE MEASURES FOR DETECTING DISEASE PROGRESSIONCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to U.S. Provisional Application No. 63 / 697,333, titled “Generating Digital Composite Measures in a Supervised or Unsupervised Manner for Disease Progression Using Free-Living Wearable Sensor Data,” filed on September 20, 2024 and U.S. Provisional Application No. 63 / 770,296, titled “TRAINING MODELS TO COMBINE MEASURES FOR DETECTING DISEASE PROGRESSION,” filed on March 11 , 2025, which are hereby incorporated by reference in their entireties.TECHNICAL FIELD

[0002] Various embodiments concern computer programs and associated computer-implemented techniques for generating combined measures of patient data to detect disease progression.BACKGROUND

[0003] Detecting disease progression in patients is a critical aspect of managing conditions, as it informs treatment adjustments and helps in monitoring the effectiveness of interventions. However, the process of accurately detecting disease progression over time comes with challenges. One of the difficulties lies in the reliance on single measures derived from sensor data, which may not capture the full complexity of a patient’s condition. These single measures, such as step count or walking speed, can be influenced by a variety of factors unrelated to the disease itself, leading to potential inaccuracies in assessing progression.

[0004] The variability in patient behavior and environmental conditions further complicates the detection of disease progression. Patients may exhibit different activity levels and movement patterns over time based on their daily routines, which can introduce noise into the sensor data. For instance, a patient may have a lower step count on a particular day due to external factors such as weather or personal circumstances, rather than a true decline in their health status. This variability makes itPATENTAtorney Docket No. 124824.8148.WO01 challenging to distinguish between genuine changes in disease progression and mere fluctuations. Moreover, these single measures may be heterogenous across patients, causing the measures to be flawed indicators of disease progression in any one patient.

[0005] Another significant challenge is the quality and nature of the training labels used to train models. Training labels, which may include clinical assessment scores and patient-reported outcomes (PROs), are essential for teaching the algorithms how to interpret sensor data. However, these labels can be noisy and subjective. For example, clinical assessment scores may be obtained during periodic evaluations and may not reflect the patient’s condition accurately over time. Additionally, patient-reported outcomes can be influenced by the patient’s perceptions and ability to recall their experiences, introducing further subjectivity and potential bias. Addressing these challenges is essential for improving detection of disease progression in patients.BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 illustrates a network environment that includes a progression platform that is executed by a computing device.

[0007] Figure 2 illustrates an example of a computing device that is able to implement a platform designed to detect disease progression.

[0008] Figure 3 illustrates an example machine learning model.

[0009] Figure 4 illustrates an example virtual motor exam (VME).

[0010] Figure 5 illustrates a “flow” for detecting disease progression using supervised learning.

[0011] Figure 6 illustrates a “flow” for using supervised learning to train machine learning algorithms to detect disease progression.

[0012] Figure 7 includes an illustrative graph showing combined measures compared to benchmarks.

[0013] Figure 8 illustrates a “flow” for detecting disease progression using unsupervised learning.

[0014] Figure 9 includes an illustration of an unsupervised learning structure.PATENTAtorney Docket No. 124824.8148.WO01

[0015] Figure 10 includes a flow diagram of a process for detecting disease progression using supervised learning.

[0016] Figure 11 includes a flow diagram of a process for detecting disease progression using unsupervised learning.

[0017] Figure 12 is a block diagram illustrating an example of a processing system that can perform at least some operations described herein.

[0018] Various features of the technology will become more apparent to those skilled in the art from a study of the Detailed Description in conjunction with the drawings. In the drawings, embodiments are illustrated by way of example and not limitation for the purpose of illustration. Those skilled in the art will recognize that alternative embodiments may be employed without departing from the principles of the present disclosure. Accordingly, while specific embodiments are shown in the drawings, the technology is amenable to various modifications.DETAILED DESCRIPTION

[0019] Detecting disease progression in patients is crucial for managing conditions, but it presents several challenges. Reliance on single digital measures (or simply “measures”) that are derived from sensor data, such as step count, bout duration, or walking speed, may not capture the full complexity of a patient’s condition and can be influenced by various unrelated factors, leading to inaccuracies. The variability in patient behavior further complicates this process, as different activity levels and movement patterns can introduce noise into the data, making it difficult to distinguish between genuine changes in disease progression and mere fluctuations. Additionally, training labels can be noisy and subjective, introducing potential biases.

[0020] Introduced here is an approach for training machine learning models to generate combined measures to detect disease progression. Methods and systems discussed herein detect progression of a disease (e.g., Parkinson’s disease) by generating combined measures in a supervised or unsupervised manner using single measures derived from higher-frequency, longitudinal free-living data (or simply “free- living data”) or lower-frequency, controlled-setting data, such as virtual motor exam (VME) data or gait analysis data. Single measures of free-living data (e.g., step count) or controlled-setting data (e.g., VME data) may have limited statistical power to detectPATENTAtorney Docket No. 124824.8148.WO01 individual changes over time because of the possibility that a motor symptom could be heterogeneous across patients or variable overtime for individual patients. In particular, each patient may exhibit different activity levels and movement patterns over time based on their daily routines, and these single measures may also be heterogenous across patients, causing them measures to be flawed indicators of disease progression in any one patient.

[0021] Combined measures are combinations of the single measures derived from free-living data or controlled-setting data. For example, a combined measure may be a linear or non-linear combination of multiple measures, which may be weighted based on relevance to disease progression. Combined measures may be more sensitive to detect changes over time and can therefore be used as digital biomarkers for disease progression. By generating combined measures, the system may not only be able to more accurately detect disease progression, for example, in Parkinson’s disease patients, but also detect disease progression in a more timely manner as changes in a combined measure may be a stronger signal of progression than changes in a single measure - or even multiple single measures if viewed separately.

[0022] Supervised machine learning models can be employed to enhance the detection of disease progression. This process training involves a supervised machine learning model using training data and weighted training labels. This model generates combined measures that better predict the condition progression, incorporating combinations of weighted single measures from the training data. This supervised approach ensures that the models are fine-tuned to the specific nuances of the disease, leading to more precise and reliable predictions.

[0023] Unsupervised neural networks can also be trained with the available training data. The neural network can learn to generate combined measures that predict condition progression more effectively than individual single measures. This is achieved by causing the neural network to minimize the reconstruction loss of the training data at each time point, allowing the neural network to assign optimal weights to the single measures in various combinations. This approach is particularly beneficial in addressing the issue of noisy or subjective training labels, as the unsupervised model does not rely on predefined labels and can autonomously identify patterns and relationships within the data.PATENTAtorney Docket No. 124824.8148.WO01

[0024] Both supervised and unsupervised approaches offer significant advantages in overcoming the limitations of single measures. By generating combined measures, these models can capture the multifaceted nature of disease progression, providing a more comprehensive view of the patient’s condition. The supervised machine learning models benefit from the structured training process, which allows for the integration of expert knowledge and clinical insights into the model training. On the other hand, the unsupervised neural network’s ability to autonomously learn from the data without relying on potentially biased training labels makes it particularly robust in handling noisy or subjective inputs.

[0025] The implementation of these advanced machine learning techniques can lead to significant improvements in the detection of disease progression. In particular, the technique of training machine learning models to make predictions based on combinations of single input measures enhances the accuracy and robustness of predictive analytics. This training method accounts for intricate patterns and relationships that single measures alone may fail to capture and thus improves upon conventional training processes. By combining single measures, the models can achieve a more comprehensive and detailed understanding of the underlying data, resulting in more precise and reliable predictions. This improvement is particularly valuable in fields such as healthcare, where accurate predictions can catch conditions earlier and may inform better treatment decisions. By leveraging the strengths of supervised and unsupervised models, healthcare providers can obtain more accurate and timely insights into a patient’s condition. This, in turn, enables more effective treatment adjustments and better monitoring of intervention outcomes.Terminology

[0026] References in the present disclosure to “an embodiment” or “some embodiments” mean that the feature, function, structure, or characteristic being described is included in at least one embodiment. Occurrences of such phrases do not necessarily refer to the same embodiment, nor are they necessarily referring to alternative embodiments that are mutually exclusive of one another.

[0027] Unless the context clearly requires otherwise, the terms “comprise,” “comprising,” and “comprised of” are to be construed in an inclusive sense rather than an exclusive or exhaustive sense. That is, in the sense of “including but not limited to.”PATENTAtorney Docket No. 124824.8148.WO01The term “based on” is also to be construed in an inclusive sense. Thus, the term “based on” is intended to mean “based at least in part on.”

[0028] The terms “connected,” “coupled,” and variants thereof are intended to include any connection or coupling between two or more elements, either direct or indirect. The connection or coupling can be physical, logical, or a combination thereof. For example, elements may be electrically or communicatively coupled to one another despite not sharing a physical connection.

[0029] The term “module” may refer broadly to software, firmware, hardware, or combinations thereof. Modules are typically functional components that generate one or more outputs based on one or more inputs. A computer program may include or utilize one or more modules. For example, a computer program may utilize multiple modules that are responsible for completing different tasks, or a computer program may utilize a single module that is responsible for completing all tasks.

[0030] When used in reference to a list of multiple items, the word “or” is intended to cover all of the following interpretations: any of the items in the list, all of the items in the list, and any combination of items in the list.Overview of Progression Platform

[0031] Figure 1 illustrates a network environment 100 that includes a progression platform 102 (or simply “progression platform”) that is executed by a computing device 104. An individual (also referred to as a “user”) can interact with the progression platform 102 via interfaces 106. For example, a user (e.g., a medical professional) may be able to access an interface through which progression results are displayed. As another example, a user (e.g., a patient) may be able to access an interface through which the views time series analyses of various measures associated with a patient. Some interfaces 106 may be designed to allow for the review of information acquired, derived, or produced by the progression platform 102 (e.g., by medical professionals), while other interfaces 106 may be designed to facilitate information gathered from users (e.g., from patients).

[0032] As shown in Figure 1 , the progression platform 102 can reside in a network environment 100. Thus, the computing device 104 on which the progression platform 102 resides can be connected to one or more networks 108A-108B. Depending on itsPATENTAtorney Docket No. 124824.8148.WO01 nature, the computing device 104 may be connected to a personal area network (PAN), local area network (LAN), wide area network (WAN), metropolitan area network (MAN), or cellular network. For example, if the computing device 104 is a computer server, then the computing device 104 may be accessible to users via respective mobile phones that are connected to the Internet via LANs. Data to be examined by the progression platform 102 may be generated by the respective mobile phones or acquired by the respective mobile phones. Alternatively, the computing device 104 may be associated with, and accessible to, a user — in which case the computing device 104 may be connected to a server system 110 that is responsible for supporting the progression platform 102. In such embodiments, the computing device 104 may be a mobile phone, tablet computer, or wearable computing device (e.g., a fitness tracker or watch), for example.

[0033] Additionally or alternatively, the computing device 104 may be connected to one or more other computing devices over a short-range wireless connectivity technology, such as Bluetooth®, Near Field Communication (NFC), Wi-Fi® Direct (also referred to as “Wi-Fi P2P”), and the like. As an example, the progression platform 102 may be embodied as a mobile application that is executed by a mobile phone. In such embodiments, the mobile phone may be communicatively connected — via a wireless communication channel — to a source from which to acquire data. The source may be a watch, fitness tracker, or another wearable computing device, for example. The data may alternatively be obtained from another computer program executing on the mobile phone. For example, the data may instead be acquired from another mobile application executing on the mobile phone or the operating system of the mobile phone.

[0034] The interfaces 106 may be accessible via a web browser, desktop application, mobile application, or another form of computer program. For example, a user may be able to access interfaces through which information regarding their own patient information can be provided, viewed, or altered via a mobile application executing on a mobile phone. Through these interfaces, the progression platform 102 may provide queries, updates, or results to the user regarding the user’s health state.

[0035] Generally, the progression platform 102 is executed — at least partially — by a cloud computing service operated by, for example, Amazon Web Services®, Google Cloud Platform™, or Microsoft Azure®. Thus, the computing device 104 may bePATENTAtorney Docket No. 124824.8148.WO01 representative of a computer server that is part of a server system 110. Often, the server system 110 includes multiple computer servers. These computer servers can include different types of data (e.g., free-living and controlled-setting data and information regarding patients, such as name, demographic information, disease classification, etc.), algorithms for processing incoming data, models (e.g., supervised and unsupervised models) for tracking disease progression, and other assets. Those skilled in the art will recognize that these data may also be distributed among the server system 110 and one or more computing devices. For example, some data that is input by, or related to, users may be stored on, and processed by, their own computing devices for security or privacy purposes. As a specific example, free-living data that is captured by a user with their own device may remain on that computing device, though insights derived via analysis of the data may be transmitted external to that computing device (e.g., to the server system 110 for further consideration or analysis).

[0036] Components of the progression platform 102 may also be hosted locally. That is, part of the progression platform 102 may reside on the computing device used to access one of the interfaces 106. For example, the progression platform 102 may be embodied as a mobile application executing on a mobile phone as mentioned above. Note, however, that the mobile application may be communicatively connected to the server system 110 on which other components of the progression platform 102 are hosted.

[0037] Figure 2 illustrates an example of a computing device 200 that is able to implement a progression platform 212. As shown in Figure 2, the computing device 200 can include a processor 202, memory 204, display mechanism 206, and communication module 208. Each of these components is discussed in greater detail below.

[0038] Those skilled in the art will recognize that different combinations of these components may be present depending on the nature of the computing device 200. For example, if the computing device 200 is a computer server that is part of a server system (e.g., server system 110 of Figure 1 ), then the computing device 200 may not include the display mechanism 206. Conversely, if the computing device 200 is a mobile phone, then the computing device 200 can include the display mechanism 206.

[0039] The processor 202 can have generic characteristics similar to general- purpose processors, or the processor 202 may be an application-specific integratedPATENTAtorney Docket No. 124824.8148.WO01 circuit (ASIC) that provides control functions to the computing device 200. As shown in Figure 2, the processor 202 can be coupled to all components of the computing device 200, either directly or indirectly, for communication purposes.

[0040] The memory 204 can be comprised of any suitable type of storage medium, such as static random-access memory (SRAM), dynamic random-access memory (DRAM), electrically erasable programmable read-only memory (EEPROM), flash memory, or registers. In addition to storing instructions that can be executed by the processor 202, the memory 204 can also store data generated by the processor 202 (e.g., when executing the modules of the progression platform 212). Note that the memory 204 is merely an abstract representation of a storage environment. The memory 204 may be comprised of actual integrated circuits (also called “chips”).

[0041] The display mechanism 206 can be any mechanism that is operable to visually convey information to a user. For example, the display mechanism 206 can be a panel that includes light-emitting diodes (LEDs), organic LEDs, liquid crystal elements, or electrophoretic elements. As further discussed below, outputs produced by the progression platform 212 (e.g., through execution of its modules) can be posted to the display mechanism 206 for review by a user of the computing device 200. In some embodiments, the user may be an individual monitoring progression of a disease via the progression platform 212.

[0042] The communication module 208 may be responsible for managing communications external to the computing device 200. The communication module 208 can be wireless communication circuitry that is able to establish wireless communication channels with other computing devices. Examples of wireless communication circuitry include 2.4 gigahertz (GHz) and 5 GHz chipsets compatible with Institute of Electrical and Electronics Engineers (IEEE) 902.12 — also referred to as “Wi-Fi chipsets.” Alternatively, the communication module 208 may be representative of a chipset configured for Bluetooth, NFC, and the like. Some computing devices — like mobile phones, tablet computers, and the like — are able to wirelessly communicate via separate channels, while other computing devices — like watches and fitness trackers — tend to wirelessly communicate via a single channel. Accordingly, the communication module 208 may be one of multiple communication modules implemented in thePATENTAtorney Docket No. 124824.8148.WO01 computing device 200, or the communication module 208 may be the only communication module implemented in the computing device 200.

[0043] The nature, number, and type of communication channels established by the computing device 200 — and more specifically, the communication module 208 — can depend on (i) the sources from which data is received by the progression platform 212 and (ii) the destinations to which data is transmitted by the progression platform 212. Assume, for example, that the progression platform 212 resides on a server. In such embodiments, the communication module 208 can communicate with sources 211 A-211 N external to the computing device 200 from which to obtain data. This data can include patient data, training data, information about various diagnoses, health information, non-health information, nutritional information, physiological information, contextual information, or other relevant data. As mentioned above, data may be acquired by the progression platform 212 from one of the sources 211 A-211 N, derived by the progression platform 212 (e.g., via analysis of sensor data, metadata, etc.), or simply provided by users (e.g., via a survey, a test, or other mechanism).

[0044] Note that the terms “health information” and “non-health information” may also be used to refer to data that is ingested, acquired, or otherwise obtained by the progression platform 212. The terms “health information” and “health data” may be used to refer to patient data, nutritional information, physiological information, and other potential health-related information associated with an individual (e.g., regarding symptoms, diseases, allergies, etc.). The terms “non-health information” and “non- health data” may be used to refer to any information that is not related to health but may affect the patient more generally. Examples of non-health information include contextual information as well as information relegated to taste palette, religious restrictions, cultural restrictions, personal restrictions (e.g., due to geographical limitations, financial limitations, etc.), and the like.

[0045] For convenience, the progression platform 212 is referred to as a computer program that resides within the memory 204. However, the progression platform 212 may be comprised of software, firmware, or hardware that is implemented in, or accessible to, the computing device 200. In accordance with embodiments described herein, the progression platform 212 can include a processing module 214, a training module 216, a derivation module 218, a validation module 220, and a graphical userPATENTAtorney Docket No. 124824.8148.WO01 interface (GUI) module 222. These modules may be integral parts of the progression platform 212, or these modules may be logically separate from the progression platform 212 but operate “alongside” it.

[0046] The processing module 214 can process data that is obtained by the progression platform 212 into a format that is suitable for the other modules. For example, the processing module 214 can apply operations to data acquired from the sources 211 A-211 N in preparation for analysis by the other modules of the progression platform 212. For example, the processing module 214 can filter or alter the data, such that the data can be more readily analyzed. As another example, the processing module 214 may parse different types of data (e.g., patient data) or data provided via different mechanisms (e.g., data directly uploaded to progression platform 212 and data obtained by progression platform 212 from memory 204 or another computer program executing on the computing device 200) in order to temporally arrange these data, thereby ensuring that any insights gleaned through analysis of these data are temporally sensical. Such an approach may be helpful in ensuring that changes — for example, in patient data — are properly detected. In some embodiments, processing the data may include processing raw sensor signals. For example, raw sensor signals can include free-living data collected from individuals as they go about their daily lives, outside of a controlled or laboratory environment. This type of data may include various forms of behavioral, physiological, and environmental information gathered through wearable devices, mobile apps, or other monitoring technologies.

[0047] For raw sensor signals, standard signal preprocessing techniques may be performed, including data resampling, bias removal, and noise removal using a filter (e.g., band-pass filter). The processed data may then be used to derive ambulatory and non-ambulatory (e.g., walking and non-walking) bouts from the data. For example, ambulatory and non-ambulatory bouts can be identified via bout algorithms, which can exclude patient bed-time and include only patient on-wrist time. Free-living single digital measures can then be derived from ambulatory and non-ambulatory bouts on a daily basis. For example, free-living single digital measures may be associated with a corresponding individual and be based on data that is generated by a sensor included in a computing device that is associated with the corresponding individual.PATENTAtorney Docket No. 124824.8148.WO01

[0048] The progression platform 212 utilizes information stored in different knowledge bases to gain insights into the diagnoses and corresponding questions for patients. These knowledge bases may be stored in the memory 204, or these knowledge bases may be external to the computing device 200 but accessible via the communication module 208. These knowledge bases may include disease information, preference information, or other information. These knowledge bases can be constructed based on an analysis of data related to users, data provided by users, or other data.

[0049] In some embodiments, the progression platform 212 can apply a model suite to information associated with a user in order to detect progression of a disease of the user. The model suite can include neural networks, supervised and unsupervised models, and learning engines. The training module 216 may be responsible for training these models. In some embodiments, the training module 216 can train models such as those discussed in relation to Figure 3.

[0050] The derivation module 218 may derive certain measures from raw or processed data. For example, the derivation module 218 can derive free-living single digital measures from ambulatory and non-ambulatory bouts at different aggregated time points (e.g., on a daily basis). As an example, for ambulatory bouts, step counts are calculated for every 11 -second window of the preprocessed signal data. The 11- second window is further classified as ambulatory if there are more than a certain number (e.g., 6) of steps within the window. Consecutive windows classified as ambulatory are joined to form an ambulatory (e.g., walking) bout provided the joined length is greater than or equal to 30 seconds. The derivation module 218 can derive different walking measurements from the walking bouts. For example, the derivation module 218 can derive the daily number of ambulatory bouts, the mean ambulatory bout duration, or other single measures based on the data from the ambulatory bout. Non-ambulatory bouts may include periods that do not consist of walking (e.g., step counts < 6 steps in a 11-second window) and sleeping (e.g., based on an on-bed classification algorithm to determine if a participant is on-bed or off-bed). Different nonwalking measurements can be derived from the non-walking bouts. For example, the derivation module 218 can derive the daily number of non-walking bouts, the mean non-PATENTAtorney Docket No. 124824.8148.WO01 walking bout duration, or other single measures based on the data from the nonambulatory bout.

[0051] The validation module 220 may validate models once they have been trained. The validation module 220 may validate a model to determine whether combined measures identified by the model can detect changes in disease progression over time. For example, the validation module 220 may validate a model using a timeseries analysis of combined measures identified by the model. In some embodiments, the validation module 220 may determine whether a rate of change of a combined measure over time in individuals is statistically significant. In some embodiments, the validation module 220 may compare the combined measures to one or more clinical benchmarks. In some embodiments, the validation module 220 may determine, based on an analysis, that combined measures achieved a higher assessment measurement (e.g., Cohen’s d) than the clinical benchmarks, demonstrating that the combined measures can show more changes over time than the clinical benchmarks.

[0052] GUI module 222 may present a series of queries through an intuitive GUI aimed at gathering relevant information about the patient’s symptoms, medical history, and other pertinent factors. For example, the GUI module 222 may ask the patient about the duration and intensity of certain symptoms, any recent changes in their health, and any known preexisting conditions. The GUI module 222 may provide functionality to capture patient response to the queries in various formats, such as text input, audio or video recordings, or other formats. As the patient responds to these queries, the GUI module 222 may capture and record their responses in real time. In some embodiments, the GUI module 222 may collect patient feedback, which can be used as reference feedback by the training module 216 for training machine learning models.

[0053] Figure 3 illustrates an example machine learning model 302. The machine learning model 302 may be the detection model, a feature impact model, or another model. According to some examples, the machine learning model may be any model. In some embodiments, the machine learning model 302 may be trained to intake input 304, including input data received. As a result of inputting the input 304 into the machine learning model 302, the machine learning model 302 may then output an output 306. As described herein, the input data can include data such as a training dataset. In some embodiments, the output 306 may include combinations of single measures included inPATENTAtorney Docket No. 124824.8148.WO01 the training dataset that more accurately predict disease progression in a user than any of the single measures when analyzed singularly or on their own.

[0054] The output parameters may be fed back to the machine learning model 302 as input to train the machine learning model 302 (e.g., alone or in conjunction with user indications of the accuracy of outputs, labels associated with the inputs, or other reference feedback information). The machine learning model 302 may update its configurations (e.g., weights, biases, or other parameters) based on the assessment of its prediction and reference feedback information (e.g., user indication of accuracy, reference labels, or other information). Connection weights may be adjusted, for example, if the machine learning model 302 is a neural network, to reconcile differences between the neural network’s prediction and the reference feedback.

[0055] One or more neurons of a neural network may require that their respective errors are sent backward through the neural network to facilitate the update process (e.g., backpropagation of error). Updates to the connection weights may, for example, be reflective of the magnitude of error propagated backward after a forward pass has been completed. In this way, for example, the machine learning model may be trained to generate better predictions.

[0056] In some embodiments, the machine learning model 302 may include an artificial neural network. In such embodiments, the machine learning model 302 may include an input layer and one or more hidden layers. Each neural unit of the machine learning model 302 may be connected to one or more other neural units of the machine learning model 302. Such connections may be enforcing or inhibitory in their effect on the activation state of connected neural units. Each individual neural unit may have a summation function that combines the values of all of its inputs together. Each connection (or the neural unit itself) may have a threshold function that a signal must surpass before it propagates to other neural units. The machine learning model 302 may be self-learning or trained rather than explicitly programmed and may perform significantly better in certain areas of problem-solving as compared to computer programs that do not use machine learning. During training, an output layer of the machine learning model 302 may correspond to a classification of the machine learning model 302, and an input known to correspond to that classification may be input into an input layer of the machine learning model 302 during training. During testing, an inputPATENTAtorney Docket No. 124824.8148.WO01 without a known classification may be input into the input layer, and a determined classification may be output.

[0057] The machine learning model 302 may include embedding layers in which each feature of a vector is converted into a dense vector representation. These dense vector representations for each feature may be pooled at one or more subsequent layers to convert the set of embedding vectors into a single vector. The machine learning model 302 may be structured as a factorization machine model. The machine learning model 302 may be a non-linear model or supervised learning model that can perform classification or regression. For example, the machine learning model 302 may be a general-purpose supervised learning algorithm that the progression platform uses for both classification and regression tasks. Alternatively, the machine learning model 302 may include a Bayesian model configured to perform variational inference on the graph or vector.

[0058] To assist in understanding the present disclosure, some concepts relevant to neural networks and machine learning are discussed herein. Generally, a neural network includes a number of computation units (sometimes referred to as “neurons”). Each neuron receives an input value and applies a function to the input to generate an output value. The function typically includes a parameter (also referred to as a “weight”) whose value is learned through the process of training. A plurality of neurons may be organized into a neural network layer (or simply “layer”) and there may be multiple such layers in a neural network. The output of one layer may be provided as input to a subsequent layer. Thus, input to a neural network may be processed through a succession of layers until an output of the neural network is generated by a final layer. This is a simplistic discussion of neural networks and there may be more complex neural network designs that include feedback connections, skip connections, or other such possible connections between neurons or layers, which are not discussed in detail here.

[0059] A deep neural network (DNN) is a type of neural network having multiple layers or a large number of neurons. The term DNN can encompass any neural network having multiple layers, including convolutional neural networks (CNNs), recurrent neural networks (RNNs), multilayer perceptrons (MLPs), Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), and auto-regressive models, among others.PATENTAtorney Docket No. 124824.8148.WO01

[0060] DNNs are often used as machine learning-based models for modeling complex behaviors (e.g., human language, image recognition, object classification, etc.) in order to improve the accuracy of outputs (e.g., more accurate predictions) such as, for example, as compared with models with fewer layers. In the present disclosure, the term “machine learning-based model” or more simply “machine learning model” may refer to a DNN. Training a machine learning model refers to a process of learning the values of the parameters (or weights) of the neurons in the layers such that the machine learning model is able to model the target behavior to a desired degree of accuracy. Training typically requires the use of a training dataset, which is a set of data that is relevant to the target behavior of the machine learning model.

[0061] Training a machine learning model generally involves inputting into a machine learning model (e.g., an untrained machine learning model) training data to be processed by the machine learning model, processing the training data using the machine learning model, collecting the output generated by the machine learning model (e.g., based on the inputted training data), and comparing the output to a desired set of target values. If the training data is labeled, the desired target values may be, e.g., the ground truth labels of the training data. If the training data is unlabeled, the desired target value may be a reconstructed (or otherwise processed) version of the corresponding machine learning model input (e.g., in the case of an autoencoder), or can be a measure of some target observable effect on the environment (e.g., in the case of a reinforcement learning agent). The parameters of the machine learning model are updated based on a difference between the generated output value and the desired target value. For example, if the value outputted by the machine learning model is excessively high, the parameters may be adjusted so as to lower the output value in future training iterations. An objective function is a way to quantitatively represent how close the output value is to the target value. An objective function represents a quantity (or one or more quantities) to be optimized (e.g., minimize a loss or maximize a reward) in order to bring the output value as close to the target value as possible. The goal of training the machine learning model typically is to minimize a loss function, or a reconstruction loss, or maximize a reward function.

[0062] The training data can be a subset of a larger data set. For example, a data set may be split into three mutually exclusive subsets: a training set, a validation (orPATENTAtorney Docket No. 124824.8148.WO01 cross-validation) set, and a testing set. The three subsets of data may be used sequentially during machine learning model training. For example, the training set may be first used to train one or more machine learning models, each machine learning model, e.g., having a particular architecture, having a particular training procedure, being describable by a set of model hyperparameters, or otherwise being varied from the other of the one or more machine learning models. The validation (or cross- validation) set may then be used as input data into the trained machine learning models to, e.g., measure the performance of the trained machine learning models or compare performance between them. Where hyperparameters are used, a new set of hyperparameters can be determined based on the measured performance of one or more of the trained machine learning models, and the first step of training (e.g., with the training set) may begin again on a different machine learning model described by the new set of determined hyperparameters. In this way, these steps can be repeated to produce a more performant trained machine learning model. Once such a trained machine learning model is obtained (e.g., after the hyperparameters have been adjusted to achieve a desired level of performance), a third step of collecting the output generated by the trained machine learning model applied to the third subset (the testing set) may begin. The output generated from the testing set may be compared with the corresponding desired target values to give a final assessment of the trained machine learning model’s accuracy. Other segmentations of the larger data set or schemes for using the segments for training one or more machine learning models are possible.

[0063] Backpropagation is an algorithm for training a machine learning model. Backpropagation is used to adjust (e.g., update) the value of the parameters in the machine learning model with the goal of optimizing the objective function. For example, a defined loss function is calculated by forward propagation of an input to obtain an output of the machine learning model and a comparison of the output value with the target value. Backpropagation calculates a gradient of the loss function with respect to the parameters of the machine learning model, and a gradient algorithm (e.g., gradient descent) is used to update (e.g., “learn”) the parameters to reduce the loss function. Backpropagation is performed iteratively so that the loss function is converged or minimized. Other techniques for learning the parameters of the machine learning model can be used. The process of updating (or learning) the parameters over many iterations is referred to as training. Training may be carried out iteratively until a convergencePATENTAtorney Docket No. 124824.8148.WO01 condition is met (e.g., a predefined maximum number of iterations has been performed, or the value outputted by the machine learning model is sufficiently converged with the desired target value), after which the machine learning model is considered to be sufficiently trained. The values of the learned parameters can then be fixed and the machine learning model may be deployed to generate output in real-world applications (also referred to as “inference”).

[0064] In some examples, a trained machine learning model may be fine-tuned, meaning that the values of the learned parameters may be adjusted slightly in order for the machine learning model to better model a specific task. Fine-tuning of a machine learning model typically involves further training the machine learning model on a number of data samples (which may be smaller in number / cardinality than those used to train the model initially) that closely target the specific task.Overview of Methods for Detecting Disease Progression

[0065] Supervised machine learning models can be employed to enhance the detection of disease progression. This process training involves a supervised machine learning model using training data and weighted training labels. This model generates combined measures that better predict the condition progression, incorporating combinations of weighted single measures from the training data. This supervised approach ensures that the models are fine-tuned to the specific nuances of the disease, leading to more precise and reliable predictions.

[0066] In some embodiments, the progression platform may retrieve training data that is collected in relation to users having a condition (e.g., individuals known to have a disease). The training data may include single measures relating to the users. Each of the single measures may be based on data that is generated by a sensor included in a computing device that is associated with the corresponding user. In some embodiments, for example, raw sensor data is collected at a high frequency, for example, every second, with a sampling rate of >=30Hz. As previously discussed, single measures of free-living data (e.g., step count) or controlled-setting data (e.g., VME data) may have limited statistical power to detect individual changes over time because of the possibility that a motor symptom could be heterogeneous across patients or variable over time for individual patients.PATENTAtorney Docket No. 124824.8148.WO01

[0067] In particular, the training data may include single measures derived from free-living sensor data and controlled-setting data relating to the users. The progression platform may derive the single measures from the free-living sensor data and controlled- setting data using the derivation module 218, as discussed in relation to Figure 2. The free-living sensor data includes ambulatory data and non-ambulatory data, and single measures may be derived from the ambulatory data and the non-ambulatory data. The controlled-setting data may include data collected in relation to tasks performed by users. For example, the tasks may be designed to assess motor skills of the users.

[0068] Figure 4 illustrates an example VME 400. A VME may involve a number of tasks designed to assess different domains of motor symptoms, for example, of a Parkinson disease patient. Participants may be instructed to perform tasks on a weekly basis. During the execution of tasks, tri-axial accelerometer and gyroscope data may be collected, and motor exam measures may be derived from sensor data (e.g., using the derivation module 218). Examples of measures derived from different tasks for different motor symptoms may include, for example, gait: arm swing acceleration, time to arise from chair; bradykinesia: pronation supination amplitude, pronation supination rate, pronation supination energy, leg movement up and down median speed, leg movement acceleration range; and tremor: rest / postural tremor frequency, rest / postural tremor amplitude.

[0069] In some embodiments, once daily free-living measures and weekly VME measures are derived, the progression platform aggregates the single measures into a monthly level to generate monthly single measures. The progression platform may apply different aggregation operations (e.g., mean value in a month) to the measures. This aggregation process may help in identifying trends and patterns over a longer period, which can be crucial for understanding the progression of certain conditions or behaviors. Additionally, by consolidating data into monthly measures, the platform may reduce the complexity and volume of data that needs to be analyzed.

[0070] Figure 5 illustrates a “flow” 500 for detecting disease progression using supervised learning. As shown in flow 500, the progression platform may collect raw sensor data (e.g., free-living and controlled-setting data) at 502. The progression platform may preprocess this data at 504, including by using standard signal preprocessing techniques, such as data resampling, bias removal, and noise removalPATENTAtorney Docket No. 124824.8148.WO01 using a filter (e.g., band-pass filter). At 506, the progression platform may derive single measures, for example, from the free-living and controlled-setting data. As an example, the progression platform can derive different walking measurements from the walking bouts of free-living data. For example, the progression platform can derive the daily number of walking bouts, the mean walking bout duration, or other single measures based on the data from the walking bout. Different non-walking measurements can be derived from the non-walking bouts. For example, the progression platform can derive the daily number of non-walking bouts, the mean non-walking bout duration, or other single measures based on the data from the non-ambulatory bout. The progression platform may additionally derive single measures from the VME data.

[0071] The process may involve retrieving a first plurality of training labels corresponding to the training data. These training labels may enable supervised models to perform optimization and learn weights for combined measure models. The first plurality of training labels may include different types of labels including clinical assessment scores and PROs. Clinical assessment scores can be obtained through clinical assessments for diseased patients. On the other hand, PROs are based on patient-reported scores for their medical conditions in a free-living setting. By incorporating clinical assessment scores and PROs, the model can learn the weights based on various types of labels.

[0072] Figure 6 illustrates a “flow” 600 for using supervised learning to train machine learning algorithms to detect disease progression. As shown in FIG. 5, flow 600 may include training labels 602. The training labels can include clinical assessment scores, PROs, or other assessments or measures. The training labels 602, along with single measures 604, can then be fed into a machine learning algorithm 606 (e.g., the first model). The single measures 604 may include measures derived from the signal data including free-living sensor data and controlled-setting data, in some embodiments, the machine learning algorithm 606 may include a linear regression model with L1 , L2, or L1 +L2 regularization, a random forest regressor model, a gradient boosted regression model, or another type of model. These machine learning algorithms may all be capable of learning the set of weights required to combine the single digital measure inputs to form the combined measures. The learning of these weights may be achieved through the optimization of each machine learning algorithm’sPATENTAtorney Docket No. 124824.8148.WO01 intrinsic cost functions. The progression platform may determine the optimal machine learning algorithm to be used for a particular disease, for example, based on cross- validation.

[0073] A first model may determine the weights that are used to train a second model for determining combined measures. In particular, the progression platform may input, into a first machine learning model, the training data and the first plurality of training labels. This may cause the first machine learning model to output a second plurality of training labels. In some embodiments, the progression platform may apply a first neural network to the training dataset and the first plurality of labels, which causes the first neural network to output a second plurality of labels. Each label in this second plurality may be representative of a combination of one of the first plurality of labels with one of a plurality of weights. The first model may be trained to identify combinations of labels among the first plurality of labels that more accurately predict the progression of the disease than single measures. In particular, the first model may be trained to identify combinations of training labels that predict condition progression more accurately than any of the plurality of single measures when analyzed singularly. The first model may be able to capture complex relationships and interactions between different types of data, such as clinical assessment scores, PROs, and VME scores.

[0074] Each training label included in the second plurality of training labels may be representative of a combination of the first plurality of training labels with respective weights. For example, the second training labels may include different combinations of the first training labels. In some embodiments, the second training labels may include weighted labels from the first training labels, where the first labels are assigned specific weights based on importance or relevance to the prediction task. As an illustrative example, a training label in the second plurality of training labels may be a gait-related score which includes: MDS-UPDRS Part 2 gait subscore, MDS-UPDRS Part 3 gait subscore, and PDQ-310 mobility subscore. This approach may leverage the combined information from multiple training labels to enhance the predictive power of the second, combined model. In particular, the use of weighted combinations allows the model to prioritize certain aspects of the data that are more indicative of disease progression, thereby refining its predictive capabilities and providing more nuanced insights into patient health.PATENTAtorney Docket No. 124824.8148.WO01

[0075] Returning to Figure 6, the machine learning algorithm 606 may receive, as inputs, the training labels 602 and the single measures 604. Based on these inputs, the machine learning algorithm 606 may output updated training labels to a supervised combined measure model 608 (e.g., the second model). The updated training labels may be used to train the second machine learning model to identify combinations of the single measures 604.

[0076] In particular, the progression platform may train a second machine learning model to identify combinations of the single measures that better predict disease progression than the single measures alone. For example, the progression platform may train a second machine learning model, using the training data and the second plurality of training labels, to identify combinations of the plurality of single measures included in the training data that better predict the condition progression. In some embodiments, the progression platform may train a second neural network with the training dataset and the second plurality of labels, such that the second neural network learns to identify combinations of the plurality of single measures that more accurately predict progression of the disease. Each of the combinations may be representative of a combined measure that is generated by the second model. The combined measures may capture complex relationships and interactions between single measures, leading to more accurate and reliable predictions of condition progression, ultimately improving the ability to monitor and manage diseases. As an illustrative example, a combined measure may be a formula such as: Combined measure = wi * (bout duration) + W2 * (step count) + bo, where wi and W2 are weights, bout duration and step count are single measures, and bo is the intercept, which accounts for the value of the combined measure when all the features are set to zero.

[0077] Returning to Figure 5, the progression platform trains the second model (e.g., the combined measure model) in a supervised manner at 508. For example, the progression platform uses the single measures derived, at 506, from the preprocessed raw sensor data. The progression platform also uses the second training labels, which have been generated by the first model, to train the second model. The progression platform may use any of the training methods discussed in relation to FIG. 3 to train the second model. In some embodiments, the training module 216, as shown in FIG. 2, may perform the training.PATENTAtorney Docket No. 124824.8148.WO01

[0078] Once a set of combined measures are identified by the second model, the progression platform uses criteria to determine if a combined measure can detect changes overtime. As one criterion, the progression platform may assess the test-retest reliability of the monthly (e.g., 30-day) aggregated metric. For instance, the intraclass correlation (ICC) between adjacent aggregation periods (e.g., monthly) may be calculated to quantify the test-retest reliability of the metric at different day-levels. The progression platform may check whether the test-retest reliability is sufficient in a specific daily aggregation period, for example, 30 days. This process ensures that the metric remains consistent and reliable over time, providing confidence in the stability and accuracy of the measurements across different periods. By evaluating the ICC, the system may verify that the aggregated metric maintains high reliability, which is important for longitudinal studies and consistent monitoring.

[0079] As another criterion, the progression platform may validate the second model by assessing whether a combined measure identified by the second model can detect changes over time. The progression platform may use a time-series analysis of the combinations identified by the second model to validate the second model. Returning to Figure 5, the progression platform may perform the time-series analysis at 510. For example, to validate the second model using the time-series analysis of the combinations, the progression platform may generate a time-series analysis of the one or more combinations and compare the time-series analysis to a benchmark to assess the capability of the one or more combinations to detect progression of the condition over time.

[0080] A time-series analysis is a statistical technique that involves analyzing a sequence of data points collected or recorded at specific time intervals. This type of analysis is used to identify patterns, trends, and other significant characteristics within the data over time. In the context of validating a machine learning model, time-series analysis can help determine how well the model’s predictions align with the actual progression of a condition. By plotting the combined measures over time, the progression platform can observe how these measures change and whether they accurately reflect the progression of the disease.

[0081] A clinical benchmark, on the other hand, is a standard or reference point derived from clinical data, expert assessments, or established progression metrics forPATENTAtorney Docket No. 124824.8148.WO01 a specific condition. It serves as a comparison point to evaluate the performance of the model’s predictions. For example, in the case of Parkinson’s Disease, a clinical benchmark may include established scales such as the Unified Parkinson’s Disease Rating Scale (UPDRS) or the Hoehn and Yahr scale, which are used to assess the severity and progression of the disease. These benchmarks provide a reliable reference to determine whether the model’s predictions are accurate and meaningful.

[0082] The progression platform may assess the performance of the supervised combined measures using Cohen’s d, which is a statistical measure used to indicate the standardized difference between two means. In some embodiments, the supervised combined measures achieve higher Cohen’s d than the clinical benchmarks, demonstrating that the combined measures can more accurately detect changes over time. The comparison of supervised (reference-based) combined measures and clinical benchmarks over time may demonstrate that the supervised combined measure demonstrated statistically significant (p<0.05) monotonic change over time. Additionally, the supervised combined measure may show less variance than the clinical benchmark over time, leading to larger Cohen’s d.

[0083] Based on the comparison, the progression platform may determine whether the combinations detect progression of the condition over time. For example, the comparison may indicate that the supervised combined measures are more sensitive to changes over time and provide a more reliable assessment of disease progression. By validating the second model through time-series analysis and comparison with clinical benchmarks, the progression platform can ensure that the model’s predictions can more accurately predict disease progression over time than other measures.

[0084] Figure 7 includes an illustrative graph 700 showing combined measures compared to benchmarks. As shown by the graph 700, the composite measures may achieve higher Cohen’s d than the clinical benchmarks, demonstrating that these composite measures can more accurately detect changes overtime. This indicates that the composite measures are more sensitive to temporal variations and can capture significant differences more effectively than traditional clinical benchmarks. By achieving a higher Cohen’s d, the composite measures may provide a more robust and nuanced understanding of the changes occurring over time, thereby enhancing the ability to monitor and assess the progression of conditions with greater precision. ThisPATENTAtorney Docket No. 124824.8148.WO01 superior performance underscores the potential of composite measures in delivering more insightful and actionable data to patients and healthcare professionals.

[0085] Once the progression platform has trained the second model, the progression platform may use the trained second model to detect disease progression in a particular individual. For example, the progression platform may receive user data from an individual who has a condition. This user data may include single measures derived from, for example, free-living sensor data and controlled-setting data related to the user. Free-living sensor data may encompass daily activity metrics such as step count and bout duration, while controlled-setting data may include clinical assessments and test results obtained in a controlled environment.

[0086] The next step may involve inputting this user data into the second machine learning model. The model may then generate a combination of the single measures of the user data using the respective weights assigned during the training phase. These weights may help to integrate various aspects of the single measures, creating a combined measure that represents the overall health status of the individual.

[0087] Following this, the progression platform may perform a monitored timeseries analysis of the combination of the single measures. This analysis tracks the combined measure overtime, allowing the platform to detect any changes or trends that indicate the progression of the condition in the user. By continuously monitoring these changes, the platform can provide a dynamic and real-time assessment of the individual’s health status. Based on the monitored time-series analysis indicating the progression of the condition, the progression platform may output an indication of the progression to the user. This output may be in the form of a report, alert, or visual representation, providing the user with insights into their disease progression. The progression platform can thus offer a personalized and accurate assessment of disease progression, enabling better management and intervention strategies for the individual.

[0088] Unsupervised neural networks can also be trained with the training data. The neural network can learn to generate combined measures that predict condition progression more effectively than individual single measures. This is achieved by causing the neural network to minimize the reconstruction loss of the training data at each time point, allowing the neural network to assign optimal weights to the single measures in various combinations. This approach is particularly beneficial in addressingPATENTAtorney Docket No. 124824.8148.WO01 the issue of noisy or subjective training labels, as the unsupervised model does not rely on predefined labels and can autonomously identify patterns and relationships within the data.

[0089] As discussed above in relation to supervised models, the progression platform may retrieve training data for unsupervised models, as well. In some embodiments, the progression platform may retrieve training data that is collected in relation to users having a condition (e.g., individuals known to have a disease). The training data may include single measures relating to the users. Each of the single measures may be based on data that is generated by a sensor included in a computing device that is associated with the corresponding user. As previously discussed, single measures of free-living data (e.g., step count) or controlled-setting data (e.g., VME data) may have limited statistical power to detect individual changes over time because of the possibility that a motor symptom could be heterogeneous across patients or variable over time for individual patients.

[0090] In particular, the training data may include single measures derived from free-living sensor data and controlled-setting data relating to the users. The progression platform may derive the single measures from the free-living sensor data and controlled- setting data using the derivation module 218, as discussed in relation to Figure 2. The free-living sensor data includes ambulatory data and non-ambulatory data, and single measures may be derived from the ambulatory data and the non-ambulatory data.

[0091] The controlled-setting data may include data collected in relation to tasks performed by users. For example, the tasks may be designed to assess motor skills of the users. A VME may involve a number of tasks designed to assess different domains of motor symptoms, for example, of a Parkinson disease patient. Participants may be instructed to perform tasks on a weekly basis. During the execution of tasks, tri-axial accelerometer and gyroscope data may be collected, and motor exam measures may be derived from sensor data (e.g., using the derivation module 218). Examples of measures derived from different tasks for different motor symptoms may include, for example, gait: arm swing acceleration, time to arise from chair; bradykinesia: pronation supination amplitude, pronation supination rate, pronation supination energy, leg movement up and down median speed, leg movement acceleration range; and tremor: rest / postural tremor frequency, rest / postural tremor amplitude.PATENTAtorney Docket No. 124824.8148.WO01

[0092] In some embodiments, once daily free-living measures and weekly VME measures are derived, the progression platform aggregates the single measures into a monthly level to generate monthly single measures. The progression platform may apply different aggregation operations (e.g., mean value in a month) to the measures. This aggregation process may help in identifying trends and patterns over a longer period, which can be crucial for understanding the progression of certain conditions or behaviors. Additionally, by consolidating data into monthly measures, the platform may reduce the complexity and volume of data that needs to be analyzed.

[0093] Figure 8 illustrates a “flow” 800 for detecting disease progression using unsupervised learning. 802, 804, 806. As shown in flow 800, the progression platform may collect raw sensor data (e.g., free-living and controlled-setting data) at 802. The progression platform may preprocess this data at 804, including by using standard signal preprocessing techniques, such as data resampling, bias removal, and noise removal using a filter (e.g., band-pass filter). At 806, the progression platform may derive single measures, for example, from the free-living and controlled-setting data. As an example, the progression platform can derive different walking measurements from the walking bouts of free-living data. For example, the progression platform can derive the daily number of walking bouts, the mean walking bout duration, or other single measures based on the data from the walking bout. Different non-walking measurements can be derived from the non-walking bouts. For example, the progression platform can derive the daily number of non-walking bouts, the mean nonwalking bout duration, or other single measures based on the data from the nonambulatory bout. The progression platform may additionally derive single measures from the VME data. These measures can then be used for training.

[0094] The progression platform can then train a neural network with this training data in an unsupervised manner. As shown in Figure 8, the progression platform trains the second model (e.g., the combined measure model) in an unsupervised manner at 808. For example, the progression platform uses the single measures derived, at 806, from the preprocessed raw sensor data. The progression platform may use any of the training methods discussed in relation to FIG. 3 to train the second model. In some embodiments, the training module 216, as shown in FIG. 2, may perform the training. In some embodiments, the progression platform can train the neural network using thePATENTAtorney Docket No. 124824.8148.WO01 training data such that the neural network learns how the plurality of single measures correspond to condition progression. The neural network learns to identify combinations of the single measures included in the training data that more accurately predict the condition progression than any of the plurality of single measures when analyzed singularly. In some embodiments, the progression platform uses training methods discussed in relation to Figure 2 to train the neural network.

[0095] In some embodiments, as a result of said training, the neural network learns weights to assign to single measures included in the combinations by minimizing a reconstruction loss that represents how well the neural network is able to recreate the training data at different time points. The process of minimizing the reconstruction loss may involve iterative optimization techniques such as gradient descent. During each iteration, the neural network adjusts the weights assigned to the single measures based on the gradient of the reconstruction loss with respect to the weights. This adjustment process continues until the reconstruction loss converges to a minimum value, indicating that the neural network has learned the optimal weights for accurately recreating the training data. In some embodiments, minimizing the reconstruction loss of the training data at each time point may involve minimizing a Euclidean distance difference, at each time point, between a data point of the training data and a corresponding output.

[0096] As the neural network learns to minimize the reconstruction loss, it becomes more adept at generating combinations of single measures that accurately reflect the progression of the condition over time. These combinations can then be used to monitor and assess the health status of individuals, providing valuable insights into disease progression. By leveraging the trained neural network and its ability to minimize reconstruction loss, the progression platform can offer a robust and reliable tool for detecting and managing conditions such as Parkinson’s Disease.

[0097] This approach focuses on minimizing the reconstruction loss aims to reduce the discrepancy between the input single measure vector and the reconstructed output at a single time point. This approach ensures that the neural network can accurately recreate the training data. However, this method has limitations in learning middle latent representations, or new combined measures, that are sensitive to changes over time. Consequently, the combined measures identified by the model may notPATENTAtorney Docket No. 124824.8148.WO01 exhibit significant temporal variation, leading to potential failures in achieving statistical significance during time-series analysis. Additionally, this approach may result in large Cohen’s D values, indicating substantial standardized differences between mean values at different time points (e.g., Month 0 and Month 23).

[0098] Figure 9 includes an illustration of an unsupervised learning structure 900. In some embodiments, the unsupervised learning structure 900 is representative of the method described above for training a neural network by minimizing a reconstruction loss. The single neural network is designed to combine different single measures through learned weights to generate combined measures at each time point. The single neural network may include an encoder, a middle latent representation, and a decoder. The encoder includes ( k ) layers, with each layer containing a different number of hidden units. These hidden units and the number of layers are model hyperparameters that can be tuned using a validation set. The middle latent representation consists of ( i ) hidden units (e.g., ( i ) = 2 in Figure 9), which are also tunable. The decoder mirrors the structure of the encoder but in reverse. All input, output, and hidden units in the neural network models are fully connected via activation functions. Each connection is associated with a weight, and each hidden unit value in a hidden layer is a linear combination of the nodes from the previous layer with the activation function. In some embodiments, the activation function introduces non-linearity. The neural network model is trained by minimizing the reconstruction loss. This ensures that the neural network can accurately recreate the input data, capturing the essential features and patterns within the single measures.

[0099] Once the neural network model is trained, it can generate the combined measures based on input data. At each time point for each subject sample, the input to the model is an array of single measures. The neural network model compresses this array of single measures into only ( i ) representations in the middle latent representation. These middle latent representations are the new combined measures at a given time point for a subject sample. This compression process involves a linear combination of the single measures, resulting in a more compact and informative representation of the data. In summary, the single neural network model learns to combine different single measures through a structured process involving an encoder, a middle latent representation, and a decoder. By minimizing the reconstruction loss,PATENTAtorney Docket No. 124824.8148.WO01 the model accurately recreates the input data and generates combined measures that reflect the essential features of the single measures. These combined measures provide a valuable tool for monitoring and assessing the progression of conditions over time.

[0100] In some embodiments, the neural network may be trained by minimizing not only the reconstruction loss but also another distance metric known as triplet ranking loss. In this approach, the objective function is modified to minimize a combined loss function: Loss = reconstruction loss + triplet ranking loss. This dual optimization process enhances the sensitivity of the middle latent representations to temporal changes, resulting in combined measures that better reflect the progression of the condition over time.

[0101] In particular, the reconstruction loss focuses on how well the neural network can recreate the training data at different time points by minimizing the Euclidean distance between the actual data points and the corresponding outputs, and incorporating triplet ranking loss adds an additional layer of refinement to the training process. Triplet ranking loss is a metric used to ensure that the neural network learns to distinguish between similar and dissimilar data points effectively. It involves training the neural network with triplets of data points: an anchor, a positive example (similar to the anchor), and a negative example (dissimilar to the anchor). The goal is to minimize the distance between the anchor and the positive example while maximizing the distance between the anchor and the negative example. This helps the neural network to learn a more nuanced representation of the data, capturing temporal changes more accurately.

[0102] For example, this method may involve iteratively adjusting the weights of the neural network based on the gradients of both the reconstruction loss and the triplet ranking loss. The combined loss function guides the optimization process, ensuring that the neural network learns to balance the accuracy of data reconstruction with the ability to distinguish between different data points. As a result, the trained neural network becomes more adept at detecting subtle changes in the condition over time, providing valuable insights for disease management and intervention.

[0103] In some embodiments, many single neural networks, each trained with both reconstruction loss and triplet ranking loss, are utilized. These individual neuralPATENTAtorney Docket No. 124824.8148.WO01 networks may be trained using different random seeds. Random seeds are initial values used to initialize the random number generators in algorithms, ensuring reproducibility and variability in the training process of models. The use of different random seeds ensures that each neural network learns slightly different representations of the data, capturing a diverse range of patterns and nuances. This diversity is crucial for creating a more comprehensive and generalized model. Once these single neural networks are trained, they are combined to form a regressor. The regressor leverages the strengths of each individual neural network to generate the combined measures. By aggregating the outputs of multiple neural networks, the regressor can produce more accurate and reliable predictions. This ensemble approach mitigates the risk of overfitting and enhances the model’s ability to generalize to new, unseen data.

[0104] In particular, the progression platform can begin by training each neural network independently, minimizing both the reconstruction loss and the triplet ranking loss. The reconstruction loss ensures that each neural network can accurately recreate the training data, while the triplet ranking loss ensures that the neural networks can effectively distinguish between similar and dissimilar data points. By using different random seeds, each neural network captures unique aspects of the data, contributing to the overall diversity of the ensemble. After training, the outputs of the individual neural networks are combined to form the regressor. The regressor integrates the diverse representations learned by each neural network, generating a combined measure that reflects the progression of the condition. This combined measure is more robust and reliable than the output of any single neural network, as it benefits from the collective knowledge of the ensemble. In practice, this method involves aggregating the predictions of the individual neural networks, for example, through techniques such as averaging or weighted averaging. The combined measure generated by the regressor provides a comprehensive assessment of the condition, capturing subtle changes and trends that may be indicative of disease progression.

[0105] Once a set of combined measures are identified by the second model, the progression platform uses criteria to determine if a combined measure can detect changes overtime. As one criterion, the progression platform may assess the test-retest reliability of the monthly (e.g., 30-day) aggregated metric. For instance, the intraclass correlation (ICC) between adjacent aggregation periods (e.g., monthly) may bePATENTAtorney Docket No. 124824.8148.WO01 calculated to quantify the test-retest reliability of the metric at different day-levels. The progression platform may check whether the test-retest reliability is sufficient in a specific daily aggregation period, for example, 30 days. This process ensures that the metric remains consistent and reliable over time, providing confidence in the stability and accuracy of the measurements across different periods. By evaluating the ICC, the system may verify that the aggregated metric maintains high reliability, which is important for longitudinal studies and consistent monitoring.

[0106] As another criterion, the progression platform may validate the neural network by assessing whether a combined measure identified by the neural network can detect changes over time. The progression platform may use a time-series analysis of the combinations identified by the neural network to validate the neural network. Returning to Figure 8, the progression platform may perform the time-series analysis at 810. For example, to validate the neural network using the time-series analysis of the combinations, the progression platform may generate a time-series analysis of the one or more combinations and compare the time-series analysis to a benchmark to assess the capability of the one or more combinations to detect progression of the condition over time.

[0107] The progression platform may assess the performance of the supervised combined measures using Cohen’s d. In some embodiments, the supervised combined measures achieve higher Cohen’s d than the clinical benchmarks, demonstrating that the combined measures can more accurately detect changes over time. The comparison of supervised (reference-based) combined measures and clinical benchmarks over time may demonstrate that the supervised combined measure demonstrated statistically significant (p<0.05) monotonic change over time. Additionally, the supervised combined measure may show less variance than the clinical benchmark over time, leading to larger Cohen’s d.

[0108] Based on the comparison, the progression platform may determine whether the combinations detect progression of the condition over time. For example, the comparison may indicate that the supervised combined measures are more sensitive to changes over time and provide a more reliable assessment of disease progression. By validating the neural network through time-series analysis and comparison withPATENTAtorney Docket No. 124824.8148.WO01 clinical benchmarks, the progression platform can ensure that the model’s predictions can more accurately predict disease progression over time than other measures.Methodologies for Detecting Disease Progression Using Combined Measures

[0109] Figure 10 includes a flow diagram 1000 of a process for detecting disease progression using supervised learning. Initially, the progression platform may acquire (i) a training dataset associated with individuals known to have a condition, the training dataset including single measures and (ii) first labels (step 1001). For example, the training dataset includes a plurality of single measures, each of which is associated with a corresponding one of the plurality of individuals. In some embodiments, the training labels correspond to the training data. Thereafter, the progression platform may apply, to the training dataset and the first labels, a first model that outputs second labels, each of which is representative of a combination of the first plurality of labels with one of a plurality of weights (step 1002). In some embodiments, the first model is trained to identify, among the first labels, combinations of labels that more accurately predict progression of the condition than single measures. Then, the progression platform may train a second model with the training dataset and the second labels, such that the second model learns to identify combinations of the single measures (step 1003). For example, the combinations may more accurately predict progression of the condition than the single measures.

[0110] Figure 11 includes a flow diagram 1100 of a process for detecting disease progression using unsupervised learning. Initially, the progression platform may acquire a training dataset associated with individuals known to have a condition, the training dataset including single measures (step 1101 ). For example, the training dataset may include a plurality of single measures, each of which is associated with a corresponding one of the plurality of individuals. Thereafter, the progression platform may train a neural network with the training dataset in an unsupervised manner, such that the neural network learns how the single measures correspond to condition progression and identifies combinations of the single measures included in the training dataset (step 1102). For example, the combinations of the single measures included in the training dataset may more accurately predict the condition progression than any of the single measures when analyzed singularly. In some embodiments, as a result of said training,PATENTAtorney Docket No. 124824.8148.WO01 the neural network may learn weights to assign to single measures included in the combinations.Processing System

[0111] Figure 12 is a block diagram illustrating an example of a processing system 1200 that can perform at least some operations described herein. For example, some components of the processing system 1200 may be hosted on a computing device that includes a progression platform (e.g., progression platform 102 of Figure 1 or progression platform 212 of Figure 2).

[0112] The processing system 1200 may include a processor 1202, a main memory 1206, a non-volatile memory 1210, a network adapter 1212, a display mechanism 1218, an input / output device 1220, a control device 1222 (e.g., a keyboard or pointing device), a drive unit 1224 including a storage medium 1226, and a signal generation device 1230 that are communicatively connected to a bus 1216. The bus 1216 is illustrated as an abstraction that represents one or more physical buses or point- to-point connections that are connected by appropriate bridges, adapters, or controllers. The bus 1216, therefore, can include a system bus, a Peripheral Component Interconnect (PCI) bus or a PCI-Express bus, a HyperTransport or an industry standard architecture (ISA) bus, a small computer system interface (SCSI) bus, a universal serial bus (USB), an inter-integrated circuit (l2C) bus, or an IEEE standard 13124 bus (also referred to as “Firewire”).

[0113] While the main memory 1206, non-volatile memory 1210, and storage medium 1226 are shown to be a single medium, the terms “machine-readable medium” and “storage medium” should be taken to include a single medium or multiple media (e.g., a centralized / distributed database and / or associated caches and servers) that store one or more sets of instructions 1228. The terms “machine-readable medium” and “storage medium” shall also be taken to include any medium that is capable of storing, encoding, or carrying a set of instructions for execution by the processing system 1200.

[0114] In general, the routines executed to implement the embodiments of the disclosure may be implemented as part of an operating system or a specific application, component, program, object, module, or sequence of instructions (collectively referred to as “computer programs”). The computer programs typically comprise one or morePATENTAtorney Docket No. 124824.8148.WO01 instructions (e.g. , instructions 1204, 1208, 1228) set at various times in various memory and storage devices in a computing device. When read and executed by the processor 1202, the instruction(s) cause the processing system 1200 to perform operations to execute elements involving the various aspects of the present disclosure.

[0115] Further examples of machine- and computer-readable media include recordable-type media, such as volatile memory devices and non-volatile memory 1210, removable disks, hard disk drives, and optical disks (e.g., Compact Disc Read- Only Memory (CD-ROMs) and Digital Versatile Discs (DVDs)), and transmission-type media, such as digital and analog communication links.

[0116] The network adapter 1212 enables the processing system 1200 to mediate data in a network 1214 with an entity that is external to the processing system 1200 through any communication protocol supported by the processing system 1200 and the external entity. The network adapter 1212 can include a network adapter card, a wireless network interface card, a router, an access point, a wireless router, a switch, a multilayer switch, a protocol converter, a gateway, a bridge, a bridge router, a hub, a digital media receiver, a repeater, or any combination thereof.Remarks

[0117] The foregoing description of various embodiments of the claimed subject matter has been provided for the purposes of illustration and description. It is not intended to be exhaustive or to limit the claimed subject matter to the precise forms disclosed. Many modifications and variations will be apparent to one skilled in the art. Embodiments were chosen and described in order to best describe the principles of the invention and its practical applications, thereby enabling those skilled in the relevant art to understand the claimed subject matter, the various embodiments, and the various modifications that are suited to the particular uses contemplated.

[0118] Although the Detailed Description describes certain embodiments and the best mode contemplated, the technology can be practiced in many ways no matter how detailed the Detailed Description appears. Embodiments may vary considerably in their implementation details while still being encompassed by the specification. Particular terminology used when describing certain features or aspects of various embodiments should not be taken to imply that the terminology is being redefined herein to bePATENTAtorney Docket No. 124824.8148.WO01 restricted to any specific characteristics, features, or aspects of the technology with which that terminology is associated. In general, the terms used in the following claims should not be construed to limit the technology to the specific embodiments disclosed in the specification, unless those terms are explicitly defined herein. Accordingly, the actual scope of the technology encompasses not only the disclosed embodiments but also all equivalent ways of practicing or implementing the embodiments.

[0119] The language used in the specification has been principally selected for readability and instructional purposes. It may not have been selected to delineate or circumscribe the subject matter. It is therefore intended that the scope of the technology be limited not by this Detailed Description but rather by any claims that issue on an application based hereon. Accordingly, the disclosure of various embodiments is intended to be illustrative, but not limiting, of the scope of the technology as set forth in the following claims.

Claims

PATENTAtorney Docket No. 124824.8148.WO01CLAIMSWhat is claimed is:

1. A method for training supervised machine learning models to combine measures for detecting condition progression, the method comprising: retrieving training data that is collected in relation to a plurality of users having a condition, the training data including a plurality of single measures relating to the plurality of users; retrieving a first plurality of training labels corresponding to the training data; inputting, into a first machine learning model, the training data and the first plurality of training labels to cause the first machine learning model to output a second plurality of training labels, wherein each training label included in the second plurality of training labels is representative of a combination of two or more of the first plurality of training labels with respective weights, and wherein the first machine learning model is trained to identify combinations of training labels that predict condition progression more accurately than any of the plurality of single measures when analyzed singularly; training a second machine learning model, using the training data and the second plurality of training labels, to identify one or more combinations of the plurality of single measures included in the training data that better predict the condition progression, wherein each of the one or more combinations is representative of a combined measure that is generated by the second machine learning model; and validating the second machine learning model using a time-series analysis of the one or more combinations identified by the second machine learning model.PATENTAtorney Docket No. 124824.8148.WO012. The method of claim 1 , further comprising: receiving user data of a user having the condition, the user data including single measures derived from free-living sensor data and controlled-setting data relating to the user; inputting, into the second machine learning model, the user data to cause the second machine learning model to generate a combination of the single measures of the user data using the respective weights; performing a monitored time-series analysis of the combination of the single measures to detect progression of the condition in the user over time; and based on the monitored time-series analysis indicating the progression of the condition, outputting an indication of the progression of the condition to the user.

3. The method of claim 1 , wherein validating the second machine learning model using the time-series analysis of the one or more combinations comprises: generating the time-series analysis of the one or more combinations; comparing the time-series analysis to a benchmark to assess capability of the one or more combinations to detect progression of the condition over time; and based on the comparing, determining that the one or more combinations detect the progression of the condition over time.

4. The method of claim 1 , wherein the training data includes free-living sensor data and controlled-setting data relating to the plurality of users.

5. The method of claim 4, wherein the free-living sensor data includes ambulatory data and non-ambulatory data, and wherein the plurality of single measures is derived based on the ambulatory data and the non-ambulatory data.

6. The method of claim 4, wherein the controlled-setting data includes data collected in relation to a plurality of tasks performed by the plurality of users, wherein the plurality of tasks is designed to assess motor skills of the plurality of users.PATENTAtorney Docket No. 124824.8148.WO017. The method of claim 1 , wherein the first plurality of training labels includes a plurality of assessment results and a plurality of patient results.

8. A method for training neural networks in a supervised manner to identify combinations of measures that are predictive of progression of a disease, the method comprising: acquiring:(i) a training dataset that is associated with a plurality of individuals known to have a disease, wherein the training dataset includes a plurality of single measures, each of which is associated with a corresponding one of the plurality of individuals and is based on data that is generated by a sensor included in a computing device that is associated with the corresponding individual, and(ii) a first plurality of labels that includes a plurality of assessment results and a plurality of patient results; applying, to the training dataset and the first plurality of labels, a first neural network that outputs a second plurality of labels, each of which is representative of a combination of labels of the first plurality of labels with a plurality of weights, wherein the first neural network is trained to identify, among the first plurality of labels, combinations of labels that more accurately predict progression of the disease than single measures; and training a second neural network with the training dataset and the second plurality of labels, such that the second neural network learns to identify combinations of the plurality of single measures that more accurately predict progression of the disease.

9. The method of claim 8, further comprising: receiving user data of a user having the disease, the user data including single measures derived from free-living sensor data and controlled-setting data relating to the user;PATENTAtorney Docket No. 124824.8148.WO01 inputting, into the second neural network, the user data to cause the second neural network to generate a combination of the single measures of the user data using the plurality of weights; performing a monitored time-series analysis of the combination of the single measures to detect progression of the disease in the user over time; and based on the monitored time-series analysis indicating the progression of the disease, outputting an indication of the progression of the disease to the user.

10. The method of claim 8, further comprising validating the second neural network using a time-series analysis of the combinations identified by the second neural network.

11. The method of claim 10, wherein validating the second neural network using the time-series analysis of the combinations comprises: generating the time-series analysis of the combinations; comparing the time-series analysis to a benchmark to assess capability of the combinations to detect progression of the disease over time; and based on the comparing, determining that the combinations detect the progression of the disease over time.

12. The method of claim 8, wherein the data that is generated by the sensor includes ambulatory data and non-ambulatory data, and wherein the plurality of single measures is derived based on the ambulatory data and the non-ambulatory data.

13. The method of claim 8, wherein the training dataset further includes controlled-setting data relating to the plurality of individuals.

14. The method of claim 13, wherein the controlled-setting data includes data collected in relation to a plurality of tasks performed by the plurality of individuals, wherein the plurality of tasks is designed to assess motor skills of the plurality of individuals.PATENTAtorney Docket No. 124824.8148.WO0115. A method comprising: acquiring:(i) a training dataset that is associated with a plurality of individuals known to have a condition, wherein the training dataset includes a plurality of single measures, each of which is associated with a corresponding one of the plurality of individuals, and(ii) a first plurality of labels; applying, to the training dataset and the first plurality of labels, a first model that outputs a second plurality of labels, each of which is representative of a combination of labels of the first plurality of labels with one of a plurality of weights, wherein the first model is trained to identify, among the first plurality of labels, combinations of labels that more accurately predict progression of the condition than single measures; and training a second model with the training dataset and the second plurality of labels, such that the second model learns to identify combinations of the plurality of single measures that more accurately predict progression of the condition.

16. The method of claim 15, further comprising: receiving user data of a user having the condition, the user data including single measures derived from free-living sensor data and controlled-setting data relating to the user; inputting, into the second model, the user data to cause the second model to generate a combination of the single measures of the user data using the plurality of weights; performing a monitored time-series analysis of the combination of the single measures to detect progression of the condition in the user over time; and based on the monitored time-series analysis indicating the progression of the condition, outputting an indication of the progression of the condition to the user.PATENTAtorney Docket No. 124824.8148.WO0117. The method of claim 15, further comprising validating the second model using a time-series analysis of the combinations identified by the second model.

18. The method of claim 17, wherein validating the second model using the time-series analysis of the combinations comprises: generating the time-series analysis of the combinations; comparing the time-series analysis to a benchmark to assess capability of the combinations to detect progression of the condition over time; and based on the comparing, determining that the combinations detect the progression of the condition over time.

19. The method of claim 15, wherein each of the plurality of single measures is associated with a corresponding one of the plurality of individuals and is based on either (i) data that is generated by a sensor included in a computing device that is associated with the corresponding individual or (ii) controlled-setting data includes data collected in relation to a plurality of tasks performed by the plurality of individuals, wherein the plurality of tasks is designed to assess motor skills of the plurality of individuals.

20. The method of claim 19, wherein the data that is generated by the sensor includes ambulatory data and non-ambulatory data, and wherein at least a portion of the plurality of single measures is derived based on the ambulatory data and the nonambulatory data.21 . A method for training neural networks to combine measures for detecting condition progression, the method comprising: retrieving training data that is collected in relation to a plurality of users having a condition, the training data including a plurality of single measures relating to the plurality of users; training a neural network with the training data in an unsupervised manner, such that the neural network learns how the plurality of single measures correspond to condition progression and identifies one or more combinations of the plurality of single measures included in the trainingPATENTAtorney Docket No. 124824.8148.WO01 data that more accurately predict the condition progression than any of the plurality of single measures when analyzed singularly, wherein as a result of said training, the neural network learns weights to assign to single measures included in the one or more combinations by minimizing a reconstruction loss that represents how well the neural network is able to recreate the training data at different time points; and validating the neural network using a time-series analysis of the one or more combinations identified by the neural network.

22. The method of claim 21 , further comprising: receiving user data of a user having the condition, the user data including single measures derived from free-living sensor data and controlled-setting data relating to the user; inputting, into the neural network, the user data to cause the neural network to generate a combination of the single measures of the user data using the weights; performing a monitored time-series analysis of the combination of the single measures to detect progression of the condition in the user over time; and based on the monitored time-series analysis indicating the progression of the condition, outputting an indication of the progression of the condition to the user.

23. The method of claim 21 , wherein validating the neural network using the time-series analysis of the one or more combinations identified by the neural network comprises: generating the time-series analysis of the one or more combinations; comparing the time-series analysis to a benchmark to assess capability of the one or more combinations to detect progression of the condition over time; and based on the comparing, determining that the one or more combinations detect the progression of the condition over time.PATENTAtorney Docket No. 124824.8148.WO0124. The method of claim 21 , wherein the training data includes free-living sensor data and controlled-setting data relating to the plurality of users.

25. The method of claim 24, wherein the free-living sensor data includes ambulatory data and non-ambulatory data, and wherein a set of the plurality of single measures is derived based on the ambulatory data and the non-ambulatory data.

26. The method of claim 24, wherein the controlled-setting data includes data collected in relation to a plurality of tasks performed by the plurality of users, wherein the plurality of tasks is designed to assess motor skills of the plurality of users.

27. The method of claim 21 , wherein minimizing the reconstruction loss of the training data at each time point comprises minimizing a Euclidean distance difference, at each time point, between a data point of the training data and a corresponding output.

28. A method for training neural networks in an unsupervised manner to identify combinations of measures that are predictive of progression of a disease, the method comprising: acquiring a training dataset that is associated with a plurality of individuals known to have a disease, wherein the training dataset includes a plurality of single measures, each of which is associated with a corresponding one of the plurality of individuals and is based on data that is generated by a sensor included in a computing device that is associated with the corresponding individual, and training a neural network with the training dataset in an unsupervised manner, such that the neural network learns how the plurality of single measures correspond to disease progression and identifies one or more combinations of the plurality of single measures included in the training dataset that more accurately predict the disease progression than any of the plurality of single measures when analyzed singularly, wherein as a result of said training, the neural network learns weights to assign to single measures included in the one or morePATENTAtorney Docket No. 124824.8148.WO01 combinations by minimizing a reconstruction loss that represents how well the neural network is able to recreate the training dataset at different time points.

29. The method of claim 28, further comprising: receiving user data of a user having the disease, the user data including single measures derived from free-living sensor data and controlled-setting data relating to the user; inputting, into the neural network, the user data to cause the neural network to generate a combination of the single measures of the user data using the weights; performing a monitored time-series analysis of the combination of the single measures to detect progression of the disease in the user over time; and based on the monitored time-series analysis indicating the progression of the disease, outputting an indication of the progression of the disease to the user.

30. The method of claim 28, further comprising validating the neural network using a time-series analysis of the combinations identified by the neural network.31 . The method of claim 30, wherein validating the neural network using the time-series analysis of the combinations comprises: generating the time-series analysis of the combinations; comparing the time-series analysis to a benchmark to assess capability of the combinations to detect progression of the disease over time; and based on the comparing, determining that the combinations detect the progression of the disease over time.

32. The method of claim 28, wherein the data that is generated by the sensor includes ambulatory data and non-ambulatory data, and wherein the plurality of single measures is derived based on the ambulatory data and the non-ambulatory data.PATENTAtorney Docket No. 124824.8148.WO0133. The method of claim 28, wherein the training dataset further includes controlled-setting data relating to the plurality of individuals, and wherein the controlled- setting data includes data collected in relation to a plurality of tasks performed by the plurality of individuals, wherein the plurality of tasks is designed to assess motor skills of the plurality of individuals.

34. The method of claim 28, wherein minimizing the reconstruction loss of the training dataset at each time point comprises minimizing a Euclidean distance difference, at each time point, between a data point of the training dataset and a corresponding output.

35. A method comprising: acquiring a training dataset that is associated with a plurality of individuals known to have a condition, wherein the training dataset includes a plurality of single measures, each of which is associated with a corresponding one of the plurality of individuals; and training a neural network with the training dataset in an unsupervised manner, such that the neural network learns how the plurality of single measures correspond to condition progression and identifies combinations of the plurality of single measures included in the training dataset that more accurately predict the condition progression than any of the plurality of single measures when analyzed singularly, wherein as a result of said training, the neural network learns weights to assign to single measures included in the combinations.

36. The method of claim 35, further comprising: receiving user data of a user having the condition, the user data including single measures derived from free-living sensor data and controlled-setting data relating to the user; inputting, into the neural network, the user data to cause the neural network to generate a combination of the single measures of the user data using the weights;PATENTAtorney Docket No. 124824.8148.WO01 performing a monitored time-series analysis of the combination of the single measures to detect progression of the condition in the user over time; and based on the monitored time-series analysis indicating the progression of the condition, outputting an indication of the progression of the condition to the user.

37. The method of claim 35, further comprising validating the neural network by: generating a time-series analysis of the combinations; comparing the time-series analysis to a benchmark to assess capability of the combinations to detect progression of the condition over time; and based on the comparing, determining that the combinations detect the progression of the condition over time.

38. The method of claim 35, wherein each of the plurality of single measures is associated with a corresponding one of the plurality of individuals and is based on either (i) data that is generated by a sensor included in a computing device that is associated with the corresponding individual or (ii) controlled-setting data includes data collected in relation to a plurality of tasks performed by the plurality of individuals, wherein the plurality of tasks is designed to assess motor skills of the plurality of individuals.

39. The method of claim 38, wherein the data that is generated by the sensor includes ambulatory data and non-ambulatory data, and wherein at least a portion of the plurality of single measures is derived based on the ambulatory data and the nonambulatory data.

40. The method of claim 39, wherein the neural network learns the weights to assign to the single measures by minimizing a reconstruction loss that represents how well the neural network is able to recreate the training dataset at different time points, and wherein minimizing the reconstruction loss of the training dataset at each time point comprises minimizing a Euclidean distance difference, at each time point, between a data point of the training dataset and a corresponding output.

Images