Systems and methods for determining clinical outcomes by signal-based feature analysis - Patents.com

Abstract

The present disclosure provides a method for receiving individual electrical signals generated based on a body part, generating a plurality of extracted features based on the individual electrical signals, and identifying clinically relevant features from the plurality of extracted features, the clinically relevant features meeting a threshold determined based on a clinical outcome.

Description

[Technical field]

[0001] The embodiments disclosed herein are directed to systems and methods for profiling features derived from a signal (e.g., a signal based on a subject's biometric cues using a biometric device, including but not limited to a wearable device) for use in clinical outcomes. Clinical outcomes may include early detection and / or treatment of potential diseases or disorders experienced by a patient. Aspects of an exemplary wearable biometric device are also disclosed. This application claims priority to U.S. Provisional Application No. 63 / 129,357, filed December 22, 2020, which is incorporated herein by reference in its entirety. [Background technology]

[0002] Identifying statistical data to provide clinical outcomes (e.g., for clinical trials, for disease or disorder identification, for treatment planning, etc.) can be difficult due to the type, amount, and / or depth of data available.

[0003] For example, various neuromuscular diseases affect the nerves that transmit the electrical signals that control voluntary muscles. The disorder can damage and slowly weaken the nerves, causing the muscles to atrophy and die over time. One example of a neuromuscular disease is myasthenia gravis (MG), which causes facial effects such as drooping eyelids (ptosis), double vision (diplopia), and / or difficulty expressing facial expressions. Myasthenia gravis (MG) can also cause difficulty speaking, breathing, chewing, and / or swallowing. Traditionally, medical professionals screen patients for neuromuscular diseases via observation and self-assessment (e.g., patient-reported outcomes). For example, the evaluation may involve the clinician administering a questionnaire in which the patient ranks the observed physical effects (e.g., ptosis and staring) and the ability to perform certain activities (e.g., closing the eyes, speaking, and chewing). Such methods are inaccurate because observations are subjective and patients may adapt their behavior over time to compensate for problem symptoms. During clinical visits, patients often under-report their chewing and swallowing symptoms and severity, especially if they have had symptoms for a long time, as they adapt to softer or liquid eating habits. As a result, current assessment methods used in clinics can lead to inaccurate and missed diagnoses and treatments. Thus, there is an unmet medical need for objective, quantitative, and accurate assessment of symptoms and severity in patients.

[0004] The use of statistical data to generate diagnoses and treatments may provide results based on objective data. However, the use of statistical data is difficult due to the type, amount, and / or depth of data available for a given test. For example, data applicable to identify one clinical outcome may not be applicable to identify another clinical outcome. Also, a given data parameter, signal collection mechanism, and / or action during signal collection may be optimal for a first clinical output, but not optimal for a second clinical output.

[0005] Thus, there is a need for improved techniques for evaluating, determining diagnoses, and allocating treatments for patients with neuromuscular diseases. Summary of the Invention

[0006] Aspects of the present disclosure relate to signal-based feature analysis. In one aspect, the present disclosure is directed to a method that includes receiving individual electrical signals generated based on a body part, generating a plurality of extracted features based on the individual electrical signals, and identifying clinically relevant features from the plurality of extracted features, where the clinically relevant features meet a threshold determined based on a clinical outcome.

[0007] The method may also include determining a clinical outcome result by applying the clinically relevant features. The clinical outcome result is one of a diagnosis or a treatment plan. The individual electrical signal may be generated based on a body electrical signal generated by the body part. The individual electrical signal may be generated based on a movement of the body part. The individual electrical signal may be generated based on a characteristic of the body part. The plurality of extracted features may be based on one or more of an amplitude feature, a zero crossing rate, a standard deviation, a variance, a root mean square, a kurtosis, a frequency, a band power, or a skew. The individual electrical signal may be generated by a wearable device comprising a sensor. In this case, the wearable device may be configured to output a mixed signal, and / or a signal separation module extracts the extracted features from the mixed signal.

[0008] For example, the signal separation module may extract extracted features from the mixed signals by applying one or more of blind signal separation, blind source separation, discrete transform, Fourier transform, integral transform, two-sided Laplace transform, Mellin transform, Hartley transform, short-time Fourier transform (STFT), rectangular mask short-time Fourier transform, chirplet transform, fractional Fourier transform (FRFT), Hankel transform, Fourier-Bros-Iagolnitzer transform, or linear canonical transform. A random forest algorithm may be used to score the extracted features. The threshold may be a random forest threshold, and extracted features having a random forest score equal to or greater than the random forest threshold may be identified as clinically relevant features. The threshold may be a confidence threshold, and extracted features having a confidence score equal to or greater than the confidence threshold may be identified as clinically relevant features. The confidence score may be based on one or more of Spearman correlation, intraclass correlation (ICC), covariance (CV), area under the curve (AUC), clustering, or Z-score.

[0009] In another aspect, the present disclosure is directed to a system including a wearable device including a plurality of sensors, a processor, and a computer-readable data storage device storing instructions that, when executed by the processor, cause the system to obtain from the wearable device electrical activity information of a subject detected by the plurality of sensors, and identify a clinically relevant characteristic based on the electrical activity information.

[0010] The system may be further configured to classify the clinically relevant features as one or more diseases, determine a disease of the subject based on the one or more diseases, determine an extent of the disease, and / or determine a treatment plan based on the extent of the disease. The plurality of sensors may include an electroencephalography (EEG) sensor, an electrooculography (EOG) sensor, an electromyography (EMG) sensor, an image sensor, and / or an eye tracking sensor. The clinically relevant features may be identified using a machine learning algorithm. [Brief description of the drawings]

[0011] [Figure 1A] FIG. 1A is a relationship diagram illustrating feature extraction and selection according to an embodiment of the present disclosure. [Figure 1B] FIG. 1B is a system block diagram illustrating an exemplary headgear according to an embodiment of the disclosure. [Diagram 2] FIG. 2 is a system block diagram illustrating an example environment for implementing systems and processes according to aspects of the disclosure. [Diagram 3] FIG. 3 is a block diagram illustrating an example of a controller according to an aspect of the disclosure. [Figure 4A] FIG. 4A is a flow chart illustrating an example of a method performed by a system according to an embodiment of the present disclosure. [Figure 4B] FIG. 4B is a flow chart illustrating an example of a method performed by a system according to an aspect of the disclosure. [Figure 4C] FIG. 4C is a flow chart illustrating identification and application of clinically relevant features according to an embodiment of the present disclosure. [Diagram 5] FIG. 5 illustrates the limitations of existing clinical outcome measures, some of which are subject to recall bias. [Figure 6] FIG. 6 is a diagram illustrating exemplary output measurements according to one embodiment of the disclosure. [Figure 7] FIG. 7 illustrates additional exemplary output measurements according to one embodiment of the disclosure. [Figure 8] FIG. 8 is a heat map of the correlation between the four tasks and Z-scores, according to one embodiment of the present disclosure. [Figure 8-1] FIG. 8-1 is a heat map of the correlation between the four tasks and Z-scores according to one embodiment of the present disclosure. [Figure 8-2] FIG. 8-2 is a heat map of the correlation between the four tasks and Z-scores according to one embodiment of the present disclosure. [Figure 9]FIG. 9 shows heat maps of variables and activities showing qualitative differences between subjects and activities according to an embodiment of the present disclosure. [Figure 9-1] FIG. 9-1 shows heat maps of variables and activities showing qualitative differences between subjects and activities according to an embodiment of the present disclosure. [Figure 9-2] FIG. 9-2 shows heat maps of variables and activities showing qualitative differences between subjects and activities according to an embodiment of the present disclosure. [Figure 9-3] FIG. 9-3 shows heat maps of variables and activities showing qualitative differences between subjects and activities according to an embodiment of the present disclosure. [Figure 9-4] FIG. 9-4 shows heat maps of variables and activities showing qualitative differences between subjects and activities according to an embodiment of the present disclosure. [Figure 9-5] FIG. 9-5 shows heat maps of variables and activities showing qualitative differences between subjects and activities according to an embodiment of the present disclosure. [Figure 9-6] 9-6 show heatmaps of variables and activities showing qualitative differences between subjects and activities according to an embodiment of the present disclosure. [Figure 9-7] 9-7 show heatmaps of variables and activities showing qualitative differences between subjects and activities according to an embodiment of the present disclosure. [Figure 9-8] 9-8 show heatmaps of variables and activities showing qualitative differences between subjects and activities according to embodiments of the present disclosure. [Figure 10] FIG. 10 shows a Spearman plot of the data from FIG. 9, demonstrating that highly correlated parameters are likely measuring similar aspects of facial biology, in accordance with an embodiment of the present disclosure. [Figure 10-1] FIG. 10-1 shows a Spearman plot of the data from FIG. 9, demonstrating that highly correlated parameters are likely measuring similar aspects of facial biology, in accordance with an embodiment of the present disclosure. [Figure 10-2]FIG. 10-2 shows a Spearman plot of the data from FIG. 9, demonstrating that highly correlated parameters are likely measuring similar aspects of facial biology, in accordance with an embodiment of the present disclosure. [Figure 10-3] FIG. 10-3 shows a Spearman plot of the data from FIG. 9 in accordance with an embodiment of the present disclosure, demonstrating that highly correlated parameters are likely measuring similar aspects of facial biology. [Figure 10-4] FIG. 10-4 shows a Spearman plot of the data from FIG. 9 in accordance with an embodiment of the present disclosure, demonstrating that highly correlated parameters are likely measuring similar aspects of facial biology. [Figure 10-5] FIG. 10-5 shows a Spearman plot of the data from FIG. 9 in accordance with an embodiment of the present disclosure, demonstrating that highly correlated parameters are likely measuring similar aspects of facial biology. [Figure 11] FIG. 11 illustrates an intraclass correlation (ICC) measure for testing test-retest reliability of parameters to infer clinical significance, according to an embodiment of the present disclosure. [Figure 12] FIG. 12 illustrates an ICC measure that is useful for testing test-retest reliability of parameters to infer clinical significance, according to an embodiment of the present disclosure. [Figure 12-1] FIG. 12-1 illustrates an ICC measure that is useful for testing test-retest reliability of parameters to infer clinical significance, according to an embodiment of the present disclosure. [Figure 12-2] FIG. 12-2 illustrates an ICC measure, useful for testing the test-retest reliability of parameters to infer clinical significance, in accordance with an embodiment of the present disclosure. [Figure 13] FIG. 13 is a diagram illustrating a schema of a random forest approach, according to an embodiment of the present disclosure. [Figure 14] FIG. 14 illustrates an F1 score for measuring how well a model classifies a particular activity (e.g., swallowing) according to an embodiment of the present disclosure. [Figure 15]FIG. 15 illustrates improved F1 scores using parameters from two rounds of feature manipulation, resulting in improved F1 scores for some activities, in accordance with an embodiment of the present disclosure. [Figure 16] FIG. 16 illustrates an improved F1 score presented in a histogram format according to one embodiment of the present disclosure. [Figure 17] FIG. 17 illustrates morning and evening F1 scores according to one embodiment of the present disclosure. [Figure 18] FIG. 18 illustrates a chart of algorithm options for selection according to one embodiment of the present disclosure. [Figure 19] FIG. 19 is a diagram illustrating swallow values ​​for different bandwidths, according to one embodiment of the present disclosure. [Figure 20] FIG. 20 is a diagram illustrating clusters and corresponding channels according to an embodiment of the present disclosure. [Figure 21] FIG. 21 is a diagram illustrating amplitude, frequency, and band power channels along with other factors according to one embodiment of the present disclosure. [Figure 22] FIG. 22 illustrates the covariance (CV) of a mixed effects model based on morning measurements, according to one embodiment of the present disclosure. [Figure 23] FIG. 23 illustrates measurements of various components collected during multiple measurement times, according to one embodiment of the present disclosure. [Figure 24] FIG. 24 is a diagram illustrating Z-scores for various combinations of individuals, times, and tasks, according to an embodiment of the present disclosure. [Figure 24-1] FIG. 24-1 is a diagram illustrating Z-scores for various combinations of individuals, times, and tasks according to an embodiment of the present disclosure. [Diagram 25] FIG. 25 is a diagram illustrating a task plotted on a Uniform Manifold Approximation and Projection (UMAP) chart, according to an embodiment of the present disclosure. [Figure 26]FIG. 26 illustrates individual data plotted on a Uniform Manifold Approximation and Projection (UMAP) chart, according to one embodiment of the present disclosure. [Figure 27] FIG. 27 illustrates time data plotted on a Uniform Manifold Approximation and Projection (UMAP) chart, according to one embodiment of the present disclosure. [Figure 28] FIG. 28 illustrates swallowing values ​​across multiple channels according to one embodiment of the present disclosure. [Figure 28-1] FIG. 28-1 illustrates swallowing values ​​across multiple channels according to one embodiment of the present disclosure. [Figure 28-2] FIG. 28-2 illustrates swallowing values ​​across multiple channels according to one embodiment of the present disclosure. [Figure 28-3] FIG. 28-3 illustrates swallowing values ​​across multiple channels according to one embodiment of the present disclosure. [Figure 29] FIG. 29 is a diagram illustrating random forest importance values ​​across multiple channels, according to one embodiment of the present disclosure. [Diagram 30] FIG. 30 is a diagram illustrating cluster statistics according to an embodiment of the present disclosure. [Diagram 31] FIG. 31 illustrates Z-scores between tasks during morning collection according to one embodiment of the present disclosure. [Diagram 32] FIG. 32 illustrates Z-scores across tasks during an evening collection according to an embodiment of the present disclosure. [Diagram 33] FIG. 33 shows Z-scores for leave-one-out cross validation (LOOCV) for a task according to an embodiment of the present disclosure. [Diagram 34] FIG. 34 illustrates Z-scores of standard deviation (SD) during morning collection according to one embodiment of the present disclosure. [Diagram 35] FIG. 35 illustrates Z-scores across tasks during an evening collection according to one embodiment of the present disclosure. [Diagram 36] FIG. 36 illustrates the CV of the mixed effects model during the evening collection, according to one embodiment of the present disclosure. [Figure 37]FIG. 37 illustrates a diagram of spread of band power measurements for smiley face collection according to an embodiment of the present disclosure. [Figure 37-1] FIG. 37-1 illustrates a diagram of spread band power measurement for smiley face collection according to an embodiment of the present disclosure. [Figure 38] FIG. 38 illustrates a clustered ICC heatmap for evening measurements according to an embodiment of the present disclosure. [Figure 38-1] FIG. 38-1 illustrates a clustered ICC heatmap for evening measurements according to an embodiment of the present disclosure. [Figure 39] FIG. 39 illustrates a clustered ICC heatmap of a mixed-effects model using evening measures according to one embodiment of the present disclosure. [Diagram 40] FIG. 40 illustrates a t-distributed stochastic neighbor embedding (t-SNE) chart for individuals, tasks, and time, according to an embodiment of the present disclosure. [Diagram 41] FIG. 41 illustrates various charts of collection classes according to an embodiment of the present disclosure. [Diagram 42] FIG. 42 illustrates a UMAP chart with respect to individuals, tasks, and time, according to one embodiment of the present disclosure. [Diagram 43] FIG. 43 illustrates verification results for different tasks across various channels, according to one embodiment of the present disclosure. [Diagram 44] FIG. 44 illustrates verification results for different tasks across various channels, according to one embodiment of the present disclosure. [Diagram 45] FIG. 45 is a flowchart of a feature manipulation process according to an embodiment of the present disclosure. [Figure 46] FIG. 46 is a diagram illustrating feature representation in frequency and time domains according to one embodiment of the present disclosure. [Figure 47] FIG. 47 illustrates qualitative differences observed from representative signals, according to an embodiment of the present disclosure. [Figure 47-1]FIG. 47-1 illustrates qualitative differences observed from representative signals, according to an embodiment of the present disclosure. [Figure 48] FIG. 48 illustrates a Spearman correlation of features according to an embodiment of the present disclosure. [Figure 49] FIG. 49 is a diagram illustrating the qualitative differences between 16 simulated PerfO activities, according to an embodiment of the present disclosure. [Figure 50] FIG. 50 is a diagram illustrating a heat map of features according to one embodiment of the present disclosure. [Figure 50-1] FIG. 50-1 is a diagram illustrating a heat map of features according to one embodiment of the present disclosure. [Figure 50-2] FIG. 50-2 is a diagram illustrating a heat map of features according to one embodiment of the present disclosure. [Figure 51] FIG. 51 illustrates a CNN model constructed using raw biosignal data from a biometric sensor device, according to an embodiment of the present disclosure. [Figure 52] FIG. 52 illustrates an activity chart for activities with level classification F1 scores for biometric sensor device features in accordance with an embodiment of the present disclosure. [Figure 53] FIG. 53 illustrates a heat map of feature attribute analysis according to one embodiment of the present disclosure. [Figure 54] FIG. 54 is a diagram of a data flow for training a machine learning model, according to one or more embodiments. [Figure 55] FIG. 55 is an exemplary diagram of a computing device, according to one or more embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate various examples and, together with the description, serve to explain the principles of the disclosed examples and embodiments. Aspects of the present disclosure may be implemented in conjunction with the embodiments illustrated in the accompanying drawings, which depict different aspects of the disclosure, and where appropriate, reference numerals designating similar structures, components, materials, and / or elements in different drawings are similarly labeled. Various combinations of structures, components, and / or elements other than those specifically depicted are also contemplated and are within the scope of the present disclosure.

[0013] Also, many embodiments are described and illustrated herein. The present disclosure is not limited to any single aspect or embodiment thereof, nor to any combination and / or permutation of such aspects and / or embodiments. Also, each of the aspects and / or embodiments of the present disclosure may be used alone or in combination with one or more of the other aspects and / or embodiments of the present disclosure. For the sake of brevity, specific permutations and combinations are not separately described and / or illustrated herein. It should be noted that an embodiment or implementation described herein as "exemplary" should not be construed as being preferred or advantageous over other embodiments or implementations, for example, but rather is intended to reflect or indicate that one or more embodiments are "examples" of one or more embodiments.

[0014] As used herein, the terms "comprising," "including," or any other variants thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements does not include only those elements, but may include other elements not expressly listed or other elements inherent to such process, method, article, or apparatus. The term "exemplary" is used in the sense of "example" rather than "ideal." Additionally, the terms "first," "second," and the like, used herein do not denote an order, quantity, or importance, but rather are used to distinguish one element or structure from another. Additionally, the term "a" or "an" used herein denotes the presence of one or more of the referenced item, rather than a limitation of quantity.

[0015] In particular, certain aspects of the figures show the general structure and / or method of construction of various embodiments for simplicity and clarity of description. Descriptions and details of well-known features and techniques may be omitted to avoid unnecessarily obscuring other features. Elements in the figures are not necessarily drawn to scale, and dimensions of some features may be exaggerated relative to other elements to improve understanding of the illustrative embodiments. For example, those skilled in the art will appreciate that side views are not drawn to scale and should not be viewed as depicting proportional relationships between different components. Side views are provided to help illustrate the various components of the illustrated assemblies and to show their positioning relative to one another.

[0016] Reference will now be made in detail to the embodiments of the present disclosure that are illustrated in the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts. The term "distal" refers to the part of the device that is furthest from a user when introducing the device into a subject. In contrast, the term "proximal" refers to the part of the device that is closest to a user when placing the device into a subject. In the following description, relative terms such as "about," "substantially," and "approximately" are used to indicate a possible variation of ±10% in stated numerical values.

[0017] Aspects of the subject matter of the present disclosure are generally directed to receiving a signal generated based on an individual's body components. The signal may be or may be generated based on electrical activity, physical activity, biometric data, movement data, or any attribute of the individual's body, an action associated with the individual's body, a response of the individual's body, and the like. The signal may be generated by a signal capture device that may capture the signal using one or more sensors. For example, aspects of the subject matter of the present disclosure are directed to a method for profiling a subject's biometric cues using a wearable biometric device. Computer-mediated profiling for early detection of potential diseases or disorders in a patient is also described so that a diagnosis can be obtained and treatment implemented early. Aspects of an exemplary wearable biometric device are also disclosed.

[0018] Implementations of the subject matter of the present disclosure include wearable systems for identifying biometric cues of subjects. The systems and methods disclosed herein can be used to address unacceptable detection and treatment gaps in patients exhibiting neurological diseases or disorders. In particular, non-invasive wearable biometric devices (e.g., ear-hanging devices) for detecting patient movements, particularly facial movements such as speaking, chewing, swallowing, neck movements, and / or eye movements, are disclosed.

[0019] Implementations of the subject matter of the present disclosure provide a method for uploading large amounts of data for analysis. For example, the analysis can be performed using advanced statistical analysis and machine-based learning (or artificial intelligence), which ensures reliable results that can be retested and understood. The systems and methods disclosed herein enable patient comfort and compliance, a large array of input / output channels for large-scale data collection, machine-assisted statistical analysis with high reliability, early detection of disorders or diseases and early intervention therefor, and improved clinical outcomes.

[0020] Forms of the presently disclosed subject matter may be used to detect diseases and / or disorders based on the collection of objective statistical data. For example, implementations of the presently disclosed subject matter may be suitable for neurological diseases, such as myasthenia gravis (MG), where subclinical cues may go undetected.

[0021] Improving the pipeline for development and analysis of wearable sensor data, as well as the framework for how to use these data in a clinical environment, is important to improve accurate patient diagnosis and monitor treatment response at all stages of clinical drug development. Challenges exist in both the development of wearable devices, as well as the way in which wearable data is processed and analyzed in a clinical environment. The methods disclosed herein address these challenges, including an exemplary proof-of-concept of assessing facial and / or eye muscle movements in a study of healthy control groups using a signal capture device 10 (e.g., a sleep aid / biometric sensor wearable measuring electromyography (EMG), electroencephalography (EEG), and electrooculography (EOG)), as shown in FIG. 1B. The utility of an unbiased feature manipulation approach for classifying activities intended to represent true performance outcome assessments (PerfOs) is disclosed. An approach for analyzing and ranking the utility of features generated from a biometric sensor device is disclosed, as well as how this data can be used to classify activities performed in this research setting. The limitations of time series analysis on biosignal data collected over short periods of time, as compared to feature-based analysis approaches, are also disclosed.

[0022] The data generated by the biometric sensor device can be used to classify an individual's physical information (e.g., certain types of cranial muscles and eye movements). The data disclosed herein suggests that the biometric wearable device can be used to objectively monitor certain physical information (e.g., cranial movements such as eye blink rate). For example, such physical information can include movements that are increased in some neuromuscular diseases such as ocular myasthenia gravis and decreased in parkinsonian diseases. Also, given the demonstrated utility of these waveforms for measuring diseases in clinical settings, there is an advantage to measuring multiple types of waveforms simultaneously from a single device.

[0023] As disclosed herein, feature importance analysis may indicate that EOG is associated with a large contribution to gaze or eye movement activity (up, left, and right) when analyzing which features are most important in classifying activity using RF models. EOG, EMG, and other signals play important roles in such indications. The presence of signal artifacts may confound waveform contribution analysis. For example, when performing a chewing activity, residual EMG activity that overlaps with typical EEG frequencies persists in the EEG signal after signal separation, resulting in an overestimation of EEG waveform contribution. As described herein, there are numerous neuromuscular and / or neurodegenerative conditions that could benefit from improved use of wearable sensor technology.

[0024] The methods disclosed herein include several feature manipulation and evaluation considerations. The classification accuracy (F1 score) of the model built from both processed sensor data is compared to the model built from the raw biosignal data. Regardless of the data augmentation, regularization, and other methods used to combat overfitting, it was observed that the training data set used in the examples provided herein was too small to train a generalizable convolutional neural network (CNN) model. However, the level or amount of data collected in the examples disclosed herein may be representative of data collected in a clinical laboratory environment. Therefore, as further disclosed herein, it is important to understand the most appropriate analysis method (e.g., clinically relevant features) for a particular clinical outcome. The analysis method (e.g., clinically relevant features) used for a given clinical outcome may differ from another clinical outcome.

[0025] The term "algorithm" refers to a defined sequence of computer-implementable instructions, typically for solving a class of problems or for performing a computation. Figures 1-3, 4A, and 4B provide components for implementations of algorithms and / or implementations of example algorithms.

[0026] The term "AUC," as understood in the art, refers to the Area Under the Curve associated with statistical analysis. The term "BCI" refers to a Brain Computer Interface system that measures activity of the central nervous system (CNS) and converts that activity into artificial and / or digital outputs for analysis.

[0027] The term "BMI" refers to the Body Mass Index value, which is derived from a person's weight and height. BMI is recognized as an index for broadly classifying people as underweight, normal weight, and overweight. BMI is frequently measured as a factor in entry into clinical trials.

[0028] The term "CNN" refers to a convolutional neural network algorithm that can receive an input and assign importance (learnable weights and biases) to various aspects / objects in that input, as well as distinguish between various aspects / objects. CNNs can perform both generative and descriptive tasks using deep learning, and are often used in machine vision, including image and video recognition, along with recommendation systems and natural language processing (NLP). The layers of a CNN may include an input layer, an output layer, and a hidden layer that includes multiple convolutional layers, pooling layers, fully connected layers, and normalization layers. Removing the limitations and improving processing efficiency results in a much more effective and easier to train system.

[0029] The term "CV" refers to covariance in statistics, which outputs a positive number if the variables being measured are positively related and a negative number if the variables being measured are negatively related. A high covariance may indicate a strong relationship between the variables. A low value may indicate a weak relationship exists between the variables.

[0030] The term "EEG" refers to electroencephalography, which is a biometric assessment of brain activity. EEG can detect abnormalities in brain waves, or abnormalities in the brain's electrical activity. EEG can be collected by attaching electrodes, which have small metal disks with thin wires, to the scalp. The electrodes can detect electrical charges resulting from brain cell activity. As disclosed herein, EEG data can be detected using non-invasive devices (e.g., ear-mounted devices).

[0031] The term "EMG" refers to electromyography, which is a biometric assessment of facial muscle weakness. EMG data can be collected by using electrodes attached to the skin or inserted into the muscles to record or receive the electrical activity of muscle tissue or its representation as a visual display or an audible signal. As disclosed herein, EMG data can be detected using a non-invasive device (e.g., an ear-hanging device).

[0032] The term "EOG" refers to electrooculography, which is an assessment of eye movement activity. EOG data can be collected by measuring electrical potentials between locations close to the eyes, which are used to investigate eye movements, particularly in physiological studies. As disclosed herein, EOG data can be detected using non-invasive devices (e.g., ear-mounted devices).

[0033] The term "F1 score" refers to a measure of the accuracy of a model on a dataset as a binary classification, where a score of 0 is poor and a score of 1 is best. The F1 score may be calculated from the precision and recall of a test, where precision is the number of true positive results divided by the number of all positive results, including those that were not correctly identified, and recall is the number of true positive results divided by the total number of samples that should have been identified as positive. Precision may be the positive predictive value. Recall may be the sensitivity in diagnostic binary classification. The F1 score may be the harmonic mean of precision and recall.

[0034] The term "false positive" refers to an outcome in which the model incorrectly predicts the positive class. The term "false negative" refers to the result where the model incorrectly predicts the negative class. The term "ICC" refers to the intraclass correlation coefficient, which can be used when quantitative measurements are made on units organized into groups. It can be used to assess the consistency or reproducibility of quantitative measurements made by different observers measuring the same quantity.

[0035] The term "ISO" refers to an isometric measure relating to or indicating muscle activity in which tension occurs without muscle contraction. The term "LOOCV" refers to Leave One Out Cross Validation analysis, a procedure used to estimate the performance of machine learning algorithms. In LOOCV, the number of folds can be the same as the number of instances in the dataset. Thus, the learning algorithm can be applied once per instance, using all other instances as a training set and a selected instance as a single-item test set.

[0036] The term "MG" refers to Myasthenia Gravis, a neurodegenerative disease that is assessable by application of the presently disclosed subject matter. The term "PSG" refers to polysomnography, a type of sleep study that uses multiparameter testing as a diagnostic tool in sleep medicine. During a PSG analysis, brain waves, oxygen levels in the blood, heart rate, breathing, and eye and leg movements may be recorded and / or analyzed.

[0037] The term "random forest" refers to the combination of many decision trees into a single model. Individually, the predictions made by a decision tree (or a person) may not be accurate, but the combination of such predictions may increase their overall accuracy. Random forests or random decision forests are ensemble learning methods for classification, regression, and other tasks that work by building a large number of decision trees at training time. For classification tasks, the output of a random forest may be the class selected by most of the trees.

[0038] The term "RMS" is the root mean square and refers to a statistical measure of the magnitude of a fluctuating quantity. RMS can be calculated for a series of discrete values ​​or for a continuously varying function.

[0039] The term "SD" refers to standard deviation. A lower standard deviation indicates that the values ​​tend to be closer to the mean (also called the expected value) of the set, while a higher standard deviation indicates that the values ​​are spread over a wider range.

[0040] The term "spectrogram" refers to a visual representation of the spectrum of frequencies of a signal as they vary over time. The term "true positive" refers to an outcome in which the model correctly predicts the positive class.

[0041] The term "true negative" refers to an outcome where the model correctly predicts the negative class. The term "Z-score" refers to the value of how many standard deviations a given piece of data is from the mean. If the Z-score is equal to 0, the data is at the average. A positive Z-score indicates that the raw score is higher than the mean. A negative Z-score indicates that the raw score is below the mean.

[0042] A review of classification techniques of EMG signals during isotonic and isometric contractions; Sensors. 2016 Aug 17;16(8): 1304, Real-Time Surface EMG Pattern Recognition for Hand Gestures Based on an Artificial Neural Network. Sensors. 2019 Jul; 19(14): 3170, and Techniques of EMG signal analysis: detection, processing, classification, and applications. Biol. Proceedings. Online. 2006; 8: 11-35 are each incorporated by reference and relate to EMG and / or EOG.

[0043] Time and frequency domain analysis of EEG signals for seizure detection (review) (Harpale, VK and Vinayak K. Bairagi. "Time and frequency domain analysis of EEG signals for seizure detection: A review" 2016 International Conference on Microelectronics, Computing and Communications (MicroCom) (2016): 23-25 ​​Jan. 2016: 1-6) is incorporated by reference and is relevant to the algorithms described herein.

[0044] Challenges in Detecting Physiological Changes Using Wearable Sensor Data (SciPy 2019) is incorporated by reference and relates to interpreting time series data.

[0045] WO 2016 / 110804 A1 describes an exemplary mobile wearable monitoring system including a headgear device for use in connection with the principles of the present disclosure, and is incorporated herein by reference.

[0046] According to an implementation of the subject matter of the present disclosure, as shown in FIG. 1A, a signal capture device 10 may be used to capture a signal associated with an individual's body. The signal may be based on the body's electrical activity, physical movement, biometric information, temperature information, action, reaction, or any attribute that can be captured as a signal (e.g., as an electrical signal). The signal capture device 10 may include one or more sensors, electrodes, cameras, or other components used to capture signals based on the individual's body. The signals captured by the signal capture device 10 may include one or more individual signals 30 (e.g., individual signal A 32, individual signal B 34, and individual signal C 36).

[0047] Alternatively, the signals captured by the signal capture device 10 may be processed via the signal manipulation module 20. The signal manipulation module 20 may analyze the individual signals 30 from the raw data received from the signal capture device 10 by applying signal filtering techniques. For example, the signal manipulation module 20 may identify the individual signals 30 from the signals provided by the signal capture device 10 by applying one or more of blind signal separation, blind source separation, discrete transform, Fourier transform, integral transform, two-sided Laplace transform, Mellin transform, Hartley transform, Short-time Fourier transform (or short-term Fourier transform) (STFT), rectangular mask short-time Fourier transform, Chirplet transform, Fractional Fourier transform (FRFT), Hankel transform, Fourier-Bros-Iagolnitzer transform, linear canonical transform, etc.

[0048] The individual signals 30 may be used to generate a number of extracted features 40 (e.g., extracted feature A 42, extracted feature B 44, ..., extracted feature N 46). The extracted features 40 may be generated based on characteristics of the extracted features 40, alone or in combination with each other. For example, the extracted features 40 may be based on one or more of signal frequency band power (e.g., for each of the individual signals 30 across a number of frequencies), spectral entropy, peak frequency shrinkage, peak frequency, mean amplitude, absolute amplitude percentage, standard deviation, absolute amplitude standard deviation, root mean square, initial deflection maximum amplitude, initial deflection polarity, detrended fluctuation Hurst parameters, Petrosian fractal dimension, approximate entropy, zero crossing rate, amplitude kurtosis, amplitude skew, perceptible onset time, amplitude variance, and / or any applicable signal-based attribute.

[0049] Implementations of the present disclosure may be used to identify clinically relevant features 50 (e.g., clinically relevant features 52 and / or clinically relevant features 54) for a given clinical output based on the extracted features 40. For example, a first set of extracted features may be optimal for identifying a first clinical output (e.g., a diagnosis or treatment of a disease or disorder), while a second set of extracted features may not be optimal for identifying the first clinical output, but may be optimal for identifying a second clinical output. Thus, methods of the present disclosure may be used to identify clinically relevant features 50 for a given clinical output based on one or more of the extracted features 40, the individual signal 30, the signal manipulation module 20, the signal capture device 10, the clinical output, the individual, etc. For example, the signal capture device 10 may have sensitivity qualities to generate a first feature well, but may not be sensitive enough to generate a second feature. Thus, when using the signal capture device 10, the first feature may be identified as a clinically relevant feature, but the second feature may not be identified as a clinically relevant feature.

[0050] Similarly, a given clinical outcome (e.g., a diagnosis of Parkinson's disease) may be identified using a first feature that has a lower standard deviation in people with Parkinson's disease when compared to a second feature that has a higher standard deviation in people with Parkinson's disease. Thus, the first feature may be identified as a clinically relevant feature because it may be more consistent in predicting, for example, the presence of Parkinson's disease when compared to the second feature.

[0051] The clinically relevant features 50 may be identified based on the signal data collected and analyzed for a test user or a group of test users. A group of test users may be any group of test users or a group of test users having overlapping attributes (e.g., demographic factors) with one or more individuals. For example, one or more signal capture devices 10 may be used to generate individual signals 30 for one or more test users. Extracted features 40 may be generated from these individual signals 30. Clinically relevant features 50 may be identified for each individual and a given clinical output (e.g., detection of a disorder). As disclosed herein, these clinically relevant features 50 may meet or exceed one or more reliability thresholds such that the clinically relevant features 50 may be relied upon to generate a clinical output with a degree of confidence. Clinically relevant features 50 identified based on data from one of the test users or a group of test users may be approved for use in clinical trials based on the clinical output confidence. As one or more individuals participate in such a clinical trial, data corresponding to the clinically relevant features 50 for those one or more individuals may be compared with reference data (e.g., data from one of the test users or a group of test users).

[0052] Additionally or alternatively, the data obtained for a given individual and / or the combined data for multiple individuals may be used as an endpoint in a given study. For example, the identification of a clinically relevant factor 50 may be an endpoint in a clinical trial. The endpoint may provide an indication of the quality and / or capabilities of the signal capture device 10 being tested in the clinical trial. Alternatively, the endpoint may provide an indication of a clinically relevant factor 50 that reliably provides a clinical output given the patient population, type of disease or disorder, and / or signal data.

[0053] Additionally or alternatively, data from an individual (e.g., signal data, discrete signals 30, extracted features 40, and / or clinically relevant features 50) may be compared to corresponding data from one or more other individuals. For example, such data may be collected from each of multiple users in a clinical trial. In this example, data for one or more individuals receiving a treatment (e.g., a drug, therapy, etc.) may be compared to data for one or more individuals receiving an alternative treatment (e.g., a different dose, duration, or type of drug or therapy), data for one or more individuals not receiving the treatment (e.g., a placebo group), and / or a reference dataset (e.g., control data). According to one implementation, data associated with a given individual at a first time may be compared to data from that individual at a second time.

[0054] Comparison of data (e.g., signal data, discrete signals 30, extracted features 40, and / or clinically relevant features 50) from different individuals (e.g., receiving different treatments, placebos, control groups, etc.), or data from the same individual over time, can be used to identify clinically relevant factors. Clinically relevant factors can be, for example, the effect of a given treatment, the effect of dosage or time amount of treatment, differences in the presentation of a given disease or disorder in different individuals (e.g., for optimal treatment regimens), identification of clusters or groupings, etc.

[0055] According to example implementations disclosed herein, electrical information of the individual's brain, nervous system, and / or muscles may be obtained. Additionally or alternatively, sensory (e.g., visual) information (e.g., video, image, infrared image, thermal image, vibration, etc.) of the individual's body (e.g., the individual's face or a portion thereof) may be obtained. According to implementations, a headgear or wearable device (e.g., signal capture device 10) may be used to collect signals from the individual's body. The headgear or wearable device may include one or more sensors for capturing electrical and / or sensory information. It is noted that although the terms headgear and / or wearable device are used herein, any equipment configured to capture electrical or sensory information at a body part (e.g., the individual's face or a portion thereof) may be used in accordance with the methods disclosed herein. The headgear and / or wearable device may include any apparatus configured to be stationary and / or placed on or around the individual's head or body part. The headgear and / or wearable device may or may not be fixed to a portion of the individual's body.

[0056] According to an exemplary implementation, the headgear and / or wearable device may include sensors for capturing electrical signals. Such electrical signals may include electroencephalography (EEG) data, electrooculography (EOG) data, and / or electromyography (EMG) data. Also, the exemplary headgear and / or wearable device may include sensory information sensors (e.g., image sensors, video sensors, infrared sensors, thermal sensors, vibration sensors, etc.) for capturing individual input data, such as face data (e.g., face recognition data), eye tracking data, movement data, environmental data (e.g., thermal data), etc. Furthermore, the controller may receive signal data (e.g., EEG, EOG, EMG, and individual input data).

[0057] The controller may be configured to classify the signal data (e.g., EEG, EOG, EMG, and individual input data), and the signal data may be used to identify a clinical outcome. For example, the signal data may be used to identify whether an individual has one or more diseases. Alternatively or additionally, the signal data may be used to determine characteristics of the individual's illness to provide an individual's treatment plan. According to implementations disclosed herein, the classification may be performed using machine learning techniques. Also, in some implementations, the systems and methods may further combine the signal data (e.g., EEG, EOG, EMG data, and individual input data) into a classification of a potential condition.

[0058] The techniques disclosed herein may be used to determine a range of conditions and / or treatment plans corresponding to that range. In accordance with implementations of the subject matter of the present disclosure, a wearable biometric device (e.g., headgear and / or wearable device) may be used to detect and / or treat a disease or disorder in an individual. The subject matter of the present disclosure may be used for early identification of a disease or disorder that may be pre-clinical, subclinical, and / or undiagnosed. The techniques disclosed herein have broad applications for quantifying a range of neurological and muscular diseases and disorders.

[0059] According to the introduction of the subject matter of the present disclosure, techniques in the fields of patient intake, statistical analysis, use of wearable electronics, artificial intelligence (AI), algorithms, machine-based learning, statistical analysis, and wearable devices and / or electronic modes of measuring subjects may be implemented to ensure unbiased objective metrics related to an individual and / or associated with a disease or disorder. Using objective data-driven analysis, inaccurate patient self-reports and potential statistical noise can be eliminated to achieve reliable metrics for proactively identifying a person's disease or disorder. While such information may be supplemented with subjective questions during intake and / or to determine a baseline, objective analysis may be used to more accurately identify a disease or disorder and / or provide a treatment plan.

[0060] According to implementations of the subject matter of the present disclosure, reliable measurements of an individual's biometric cues can be used to detect and / or treat a disease or disorder. One or more sensors (e.g., sensors disposed on or around headgear) can be used to collect biometric measurements of an individual at a given time and / or over a range of time. FIG. 1B shows a block diagram illustrating an example of an environment 100 for implementing systems and methods according to aspects of the present disclosure. In some implementations, the environment 100 can include an individual 101, a headgear 103, and a controller 105. The headgear 103 can correspond to the signal capture device 10 of FIG. 1A.

[0061] The individual 101 may be any person. In some implementations, the individual 101 may be a baseline individual for collection of control data. In some implementations, the individual 101 may be a medical patient. For example, the individual 101 may be a patient who may have a neuromuscular disease, such as myasthenia gravis. Although "headgear" and "wearable devices" are generally referenced herein, the devices for collection of electrical data and / or personal input data may be positioned in any applicable location on, under, around, partially around, or near the individual's head or other body part. For example, the headgear 103 may refer to glasses that may rest on the individual's ears. As another example, the headgear 103 may refer to ear pieces that are inserted in or around the individual's ears.

[0062] The headgear 103 may be a device including one or more sensors that capture information of the individual 101 representative of voluntary muscle movements. The sensors may collect electrical information and / or individual input information (e.g., facial data (e.g., facial recognition data), eye-tracking data, movement data, environmental data (e.g., thermal data), etc.). In some implementations, the headgear 103 may include electrical sensors 111, sensory information sensors 113, and / or a device controller 115. The electrical sensors 111 may be configured to capture EEG data, EOG data, and EMG data. The sensory information sensors 113 may include facial recognition sensors, eye-tracking sensors, image sensors, video sensors, infrared sensors, thermal sensors, and vibration sensors.

[0063] The device controller 115 may be a computing device connected to the controller 105 via one or more wired or wireless communication channels 121. The communication channels 121 may use various serial, parallel, and / or transmission (e.g., video transmission) protocols. The device controller 115 may include hardware, software, firmware, or combinations thereof for performing operations according to the present disclosure. The operations may include receiving personal input data, such as EEG, EOG, EMG, facial data (e.g., facial recognition data), eye tracking data, movement data, environmental data (e.g., thermal data), from the electrical sensors 111 and / or the sensory information sensors 113, and transmitting the data to the controller 105 using the communication channels 121 with one or more transmission protocols.

[0064] The controller 105 may include hardware, software, or a combination thereof for performing operations according to the present disclosure. Operations performed by the controller 105 may include receiving, filtering, and normalizing data transmitted by the device controller 115 of the headgear 103. Operations may also include classifying the EEG, EOG, EMG, and / or individual input data individually by determining one or more descriptive categories and their respective severity. The categories may be species or genera of symptoms or disorders. In some implementations, the classification may be performed using machine learning techniques. For example, an elaborate array of advanced statistical data may be interpreted using a random forest schema, as further described herein.

[0065] Implementations of the subject matter of the present disclosure include classifying the combination of the EEG, EOG, EMG, and individual input information by determining one or more descriptive categories or disorders. In some implementations, the classification may be performed using machine learning techniques to classify the symptoms or disorders determined using the individual classifications of the data. In some implementations, the operations further include determining a treatment plan corresponding to the one or more disorders and their respective severities.

[0066] For example, FIG. 1B shows a system diagram illustrating an example of headgear 103 according to an embodiment of the present disclosure. The headgear 103 may be the same as or similar to the headgear and / or wearable devices described above. In some implementations, the headgear 103 may include a device controller 115, an electrical sensor 111, and a sensory information sensor 113. These may be the same as or similar to those described above.

[0067] 2 is a system block diagram illustrating an example of an environment for implementing the systems and processes disclosed herein. The electrical sensors 111 may include any applicable sensors, such as, for example, an EEG sensor 205, an EOG sensor 207, and an EMG sensor 209. These sensors generate EEG, EOG, and EMG data, which may be the same or similar to those described above. Although the EEG sensor 205, the EOG sensor 207, and the EMG sensor 209 are shown as separate sensors, one or more of the EEG, EOG, and EMG sensors may be combined.

[0068] The sensory information sensor 113 may include an image sensor 211 and an eye tracking sensor 213 that generate facial recognition data and eye tracking data. The facial recognition data and eye tracking data may be the same or similar to those described above. Although the image sensor 211 and the eye tracking sensor 213 are shown as separate sensors, the image sensor 211 and the eye tracking sensor 213 may be combined.

[0069] The device controller 201 may be the device controller 115 of FIG. 1B or a part thereof, and may be one or more devices that process data generated by the EEG sensor 205, the EOG sensor 207, the EMG sensor 209, the image sensor 211, and the eye tracking sensor 213. In some implementations, the device controller 201 may include a processor 225, a memory device 227, a storage device 229, a communication interface 231, an input / output (I / O) processor 233, and a data bus 235. The device controller 201 may include a signal manipulation module 20. Alternatively, the signal manipulation module 20 may be independent of the device controller 201 and / or the signal capture device 10 in general. For example, the signal manipulation module 20 may be part of a separate analysis component located remotely from the signal capture device 10.

[0070] In some implementations, the processor 225 may include one or more microprocessors, microchips, or application specific integrated circuits. The memory device 227 may include one or more types of random access memory (RAM), read only memory (ROM), and cache memory used during the execution of program instructions. The processor 225 may communicate with the memory device 227, the storage device 229, the communication interface 231, the image processor, and / or the spatial sensor using a data bus 235. The storage device 229 may include a computer-readable non-volatile hardware storage device that stores information and program instructions.

[0071] For example, the storage device 229 may be one or more of a flash drive and / or a hard disk drive. The transmitter / receiver may be one or more devices that may be used to communicate signals and that encode / decode data into wireless signals, such as ranging signals.

[0072] The processor 225 may execute program instructions (e.g., an operating system and / or application programs) that may be stored in the memory device 227 and / or the storage device 229. The processor may also execute program instructions of the sensor module 251. The sensor module 251 may include program instructions to process data generated by the EEG sensor 205, the EOG sensor 207, the EMG sensor 209, the image sensor 211, and the eye tracking sensor 213. The processing may include, for example, filtering, amplifying, and normalizing the data to remove noise and other artifacts. It should be noted that the device controller 201 is merely representative of various possible equivalent computing devices that may perform the processes and functions described herein. In this regard, in some implementations, the functionality provided by the device controller 201 may be any combination of general-purpose and / or special-purpose hardware and / or program instructions. In each implementation, the program instructions and hardware may be created using standard programming and engineering techniques.

[0073] 3 shows a functional block diagram illustrating a controller 105 according to an embodiment of the present disclosure. The controller 105 may be the same or similar to those described herein above. The controller 105 may include a computing device 306 with a processor 305, a memory device 307, a storage device 309, an I / O processor 325, and a data bus 331. The controller 105 may also include input connections (e.g., image input connections) and / or output connections (e.g., image output connections) for receiving and / or transmitting image signals from the image processor. Additionally, the controller 105 may include input / output connections for receiving / transmitting data signals from the I / O processor 325.

[0074] In some implementations, the controller 105 may include one or more microprocessors, microchips, or application specific integrated circuits. The memory device 307 may include one or more types of random access memory (RAM), read only memory (ROM), and cache memory used during the execution of program instructions. The controller 105 may also include one or more data buses 331 that communicate with the memory device 307, the storage device 309, and the I / O processor 325.

[0075] The storage device 309 may include a computer-readable non-volatile hardware storage device that stores information and program instructions. For example, the storage device 309 may be one or more flash drives and / or hard disk drives. The storage device 309 may include reference data 310 for access via the communication data bus 331.

[0076] An I / O processor 325 may be coupled to the processor 305. The I / O processor 325 may include any device that allows an individual to interact with the processor 305 (e.g., a user interface) and / or any device that allows the processor 305 to communicate with one or more other computing devices using any type of communication link. The I / O processor 325 may generate and receive digital and analog input / output (e.g., electronic signals), for example, according to a variety of data transmission protocols.

[0077] The processor 305 executes program instructions (e.g., an operating system and / or application programs) that may be stored in the memory device 307 and / or the storage device 309. The processor 305 may also execute program instructions of modules 351.

[0078] The controller 105 may include the signal manipulation module 20. Alternatively, the signal manipulation module 20 may be independent of the controller 105. For example, the signal manipulation module 20 may be located remotely from the signal capture device 10 or may be part of a separate analysis component. The controller 105 may include a disorder classification module 355 and / or a sensor classification module 359. The disorder classification module 355 and / or the sensor classification module 359 may apply statistical analysis techniques disclosed herein to identify and / or apply clinically relevant features for classification of disorders and / or sensor signals.

[0079] The controller 105 may include a communication interface 311 to facilitate inter-controller or intra-controller communication (e.g., via a data bus 331). The controller 105 may also include one or more I / O devices 333 for communicating with the I / O processor 325. According to one implementation, the I / O devices may include the device controller 201 and / or the device controller 115.

[0080] It should be noted that the controller 105 is merely representative of various possible equivalent computing devices capable of performing the processes and functions described herein. In this regard, in some implementations, the functionality provided by the controller 105 may be any combination of general-purpose and / or special-purpose hardware and / or program instructions. In each implementation, the program instructions and hardware can be created using standard programming and engineering techniques.

[0081] 4A and 4B include a flowchart 400 illustrating an example of an implementation disclosed herein. As described herein, implementations of the subject matter of the present disclosure may be used to identify clinically relevant features for a given clinical output based on extracted features from signal data. FIG. 4A and 4B illustrate an example related to using a wearable device to generate signal data based on a sensor on the wearable device. FIG. 4A and 4B illustrate an example of the functionality and operation of a possible implementation of a system, method, and computer program product according to various implementations consistent with the present disclosure.

[0082] Each block in the flowchart of Figure 4A or 4B may represent a module, segment, or portion of program instructions, which includes one or more computer-executable instructions for achieving the functions and operations shown. In some alternative implementations, the functions and / or operations shown in a particular block of the flowchart may occur in an order different from the order shown in Figure 4A or 4B.

[0083] For example, two blocks shown in succession may be executed substantially simultaneously, or the blocks may be executed in reverse order depending on the functionality involved. Note that each block of the flowchart and combinations of blocks within the block may also be implemented by a dedicated hardware-based system that performs the specified functions or operations, or a combination of dedicated hardware and computer instructions.

[0084] At 401 of the flowchart 400, the subject may perform a facial movement. The facial movement may be performed based on a request, based on the subject's natural state, etc. Although facial movements are specifically disclosed herein, it may be understood that any body component, action, or characteristic may be observed as part of the subject matter of the present disclosure. At 405, EEG information may be acquired from one or more EEG sensors of the wearable device. The EEG information may be collected based on contact or non-contact reception of signals via electrodes. At 409, EOG information may be acquired from one or more EOG sensors of the wearable device. The EOG information may be collected based on contact or non-contact reception of signals via electrodes.

[0085] At 413, face information may be obtained from a sensory information sensor, such as an image sensor. The face information may include movement, attribute information (e.g., temperature, length, elasticity, angle, etc.), etc. The face information may be captured using the sensory information sensor based on a trigger (e.g., a request for face information, a facial action, or a change, etc.), or may be captured on a continuous basis. At 418, eye position information may be obtained from an eye tracking sensor. The eye position information may include movement, attribute information (e.g., degree of movement, direction of movement, extension, angle, etc.), etc. The eye position information may be captured using the sensory information sensor based on a trigger (e.g., a request for eye position information, an action, or a change, etc.), or may be captured continuously.

[0086] At 421, one or more of the EEG information, EOG information, face information, and / or eye position information may be filtered and / or normalized. The information may be filtered and / or normalized based on any applicable technique disclosed herein. The information may be filtered and / or normalized to remove noise, extract features, etc. At 425, the EEG information may be classified. At 429, the EOG information may be classified. At 433, the face information may be classified. At 437, the eye position information may be classified.

[0087] At 441 (shown in FIG. 4B), EOG information, face information, and eye tracking information may be combined. At 445, EEG information and face information may be combined. It may be understood that information may be combined based on clinical outcomes. For example, 441, 445 in FIG. 4B may be different for different clinical outcomes, such that different information may be combined than that presented in FIG. 4A and FIG. 4B. It may be understood that any signal information related to an individual may be statistically evaluated based on, for example, the flows shown in FIG. 1A, FIG. 4A, and FIG. 4B.

[0088] At 449, the combined EOG information, face information, eye tracking information, and / or other combined information may be compared to reference information. As shown in FIG. 1A, the comparison may be based on extracted features from the EOG, EEG, face information, eye position information, or any other applicable information. The comparison may be based on extracted features determined to be clinically relevant features as shown in FIG. 1A.

[0089] At 453, the clinically relevant features based on the sensed information may be used to determine the subject's state. For example, the state may be determined based on the combined EOG, face, and eye tracking information, the combined EEG and face information, and the reference information. At 457, a determination may be made whether the state has been determined. If the state has not been determined, the steps described herein beginning with 401 may be repeated (e.g., as indicated by "B" in FIG. 4A and FIG. 4B). If the state has been determined at 457, a determination may be made of a range of the state, for example, based on the second reference information, at 461. At 465, a treatment plan may be determined (e.g., further based on the range of the state) based on the third reference information and / or the second reference information. FIG. 4C shows a flow chart 470 for identifying and applying clinically relevant features. At 472, a plurality of extracted features may be received. As described herein, the plurality of extracted features may be based on signals collected from or about the individual's body. The signals may be collected when an action is being taken or when a particular body part is being observed (e.g., angry impression, chewing, eye movement, eye Iso, medial Iso, jaw, left gaze-left, left gaze-right, right gaze-left, right gaze-right, lateral Iso, sad impression, smiling Iso, surprised impression, swallowing, talking, upward gaze, wrinkle Iso, etc., as further described in Table 7). Thus, a given extracted feature may be based on a given type of signal (e.g., ECG, EEG, etc.), the action during which the signal is being collected, the time the signal is being collected, the individual, etc. At 472, multiple available extracted features may be received.

[0090] At 473, a statistical filter technique may be identified to apply to the multiple extracted features. The statistical filter technique may be a single technique or multiple techniques applied at once. The statistical filter technique may be identified to output clinically relevant features that can be used to best determine a clinical outcome. As described herein, the clinical outcome may be an identification of a disease or disorder (e.g., 453 in FIG. 4B) and / or a treatment plan for the identified disease or disorder (e.g., 465 in FIG. 4B). The clinical outcome may be based on objective data (e.g., based on sensor-collected signals).

[0091] At 474, one or more statistical filter techniques identified at 473 may be applied to the plurality of extracted features received at 472. The statistical filter techniques may include, but are not limited to, Spearman correlation 474A, ICC 474B, random forest algorithm 474C, CV 474D, AUC 474E, clustering 474F, Z-score 474G, etc., or combinations thereof. These statistical filter techniques are further described herein.

[0092] At 476, clinically relevant features may be identified based on the statistical filter technique applied at 474. In identifying clinically relevant features from the plurality of features received at 472, a clinical trial or other program may be performed using only the clinically relevant features based on, for example, a given clinical outcome. The identifying clinically relevant features may be features that can be used to test the clinical outcome in a reliable manner such that the identified clinically relevant features can provide reliable data for the given clinical outcome. The reliability may meet a given reliability threshold for the given clinical outcome. The given reliability threshold may be a numerical value determined based on one or more of Spearman correlation, ICC, random forest results, CV, AUC, clustering, and / or Z-score associated with the data. For example, the given reliability threshold may be based on a minimum or maximum value associated with one or more of Spearman correlation, ICC, random forest results, CV, AUC, clustering, and / or Z-score for a given set of raw data or features. The raw data may correspond to the features received at 427. Thus, the confidence threshold can be a single value (e.g., a binary score, a ratio, a percentage, etc.) or a set of values ​​(e.g., one for each of Spearman correlation, ICC, random forest results, CV, AUC, clustering, and / or Z-score) that indicates that a given set of clinically relevant features provides relevant data for a given clinical outcome. At 478, the clinically relevant features can be applied to determine a clinically relevant outcome.

[0093] It should be noted that while Figure 4C presents a set of exemplary statistical filter techniques (Spearman correlation, ICC, Random Forest results, CV, AUC, clustering, and / or Z-score), one or more additional techniques may be applied, and the set of statistical filter techniques is not limited to those disclosed herein. Any statistical filter technique that indicates the reliability of one or more features may be used to identify clinically relevant features.

[0094] According to implementations of the subject matter of the present disclosure, the identification of clinically relevant features at 476 of FIG. 4C may be used to determine the limitations and / or quality of a sensor-based device (e.g., signal capture device 10 of FIG. 1A) used to collect one or more signals (e.g., individual signals 30 or signals provided to signal manipulation module 20) used to determine the extracted features received at 472. By applying the techniques (e.g., procedures) disclosed herein, at 477, clinically relevant factors for a given clinical outcome (e.g., prediction of a particular disorder and / or treatment of a particular disorder) may indicate the limitations and / or quality of the sensor-based device. For example, clinically relevant features based on a wearable device and a given clinical outcome may be identified. Based on the clinical features (e.g., one or more frequencies, one or more types of electrical signal channels, one or more spectral entropies, one or more peak frequencies, one or more amplitudes, one or more standard deviations, one or more root mean squares, one or more deflections, one or more fluctuations, one or more fractal dimensions, one or more crossing rates, one or more amplitude kurtosis, one or more skews, one or more onset times, one or more variances, etc.), a determination may be made whether a corresponding sensor-based device can reliably output a clinical outcome (e.g., whether a threshold number of extracted features meet a confidence threshold, whether a clinical relevance threshold exceeds a confidence threshold, etc.).

[0095] [Example] Implementations of the subject matter of the present disclosure are disclosed herein with reference to a number of examples, although it can be understood that the implementations disclosed herein are not limited solely to the data, sequences, or specifications disclosed in the examples.

[0096] [Example 1] FIG. 5 includes a chart 500 illustrating limitations of existing clinical outcome measures, in part due to their susceptibility to recall bias. For example, the chart 500 may be used by a clinical professional to receive a subjective physician score 502 based on condition 504. As an example, a healthcare provider may test for ptosis (ease of looking up) by observing the number of seconds the gaze direction is held. A subjective physical score may be determined by the healthcare provider. However, such physical scores and corresponding assessments are susceptible to errors, recall bias, and missing information, which may lead to misdiagnosis.

[0097] A study was conducted to perfect a reliable biometric data acquisition activity. Objective data was collected and the study was designed with the following objectives: 1) Understand data quality and omissions from biometric devices. 2) Understand the test-retest reliability of biometric devices. 3) Understand the ability of biometric devices to quantify various facial muscle activities and differentiate between them.

[0098] As disclosed herein, the above objectives were determined based on the statistical analysis disclosed, for example, in FIG. 4C. A total of 16 facial and eye movement tasks were selected for the study (swallowing, chewing, speaking, facial expressions, eye closure, and gaze in different directions). A control group with a total of N=10 participated in the study. During the study, data were received via a biometric device with 31 variables. An initial data exploration was performed on the measurements.

[0099] According to implementations of the subject matter of the present disclosure, ICC and CV analysis document the test-retest performance between variables (e.g., 31 variables tested). Using ICC and / or CV, the predictability of all variables based on the task (activity) was evaluated using a multinomial logistic regression model. For example, the clinical focus for the analysis of the "swallowing" task (preferred activity) was addressed.

[0100] Additional measurements were received from an EOG sensor. The sensor was placed on a biometric device in close proximity to the individual. A total of 60 summary variables and the ICC analysis included several additional variables with high test-retest power. Model selection was performed (e.g., at 473 in FIG. 4C). The results of the model selection identified the best model (e.g., based on signals and / or biometric devices) for predicting activities and subjects as the Random Forest model.

[0101] FIG. 6 illustrates exemplary output readings collected using one or more sensors of a biometric device. As shown in chart 602, alpha waves (0.3-35 Hz) were collected from various sensors while an eyes-closed task behavior was being performed. As shown in chart 604, vertical EOG data (0.3-10 Hz) was collected from multiple sensors during a blink task. As shown in chart 606, horizontal EOG data (0.3-10 Hz) was collected during a left gaze task and a right gaze task. As shown in FIG. 6, facial EMG (10-100 Hz) was collected using multiple sensors while a bruxism task was being performed. Electrodermal activity (EDA) (galvanic skin response) (0.1-1.5 Hz) was collected while an arousal task was being performed, as shown in chart 610.

[0102] FIG. 7 illustrates exemplary output readings collected using one or more sensors of a biometric device. The output readings shown in FIG. 7 are collected using a device (e.g., signal capture device 10) that can reliably measure various aspects of facial biology using waveform signals for brain activity, eye movements, facial muscles, etc., verified against polysomnography (PSG). Amplitudes (μV) collected during a smiling task are collected at 702, amplitudes collected while puffing out cheeks are collected at 704, amplitudes collected while closing eyes tightly are collected at 706, and amplitudes collected while biting into an apple are collected at 708. The amplitudes shown in charts 702, 704, 706, and 708 are shown over time (seconds).

[0103] The signals shown in FIG. 6 and / or FIG. 7 may be the individual signals 30 of FIG. 1A and / or may be provided to the signal manipulation module 20 to generate the individual signals 30. Predictability based on tasks and individual subjects was analyzed using a random forest model with F1 scores. Z-scores were also obtained. The Z-scores are presented herein as heat maps. Heat maps of variables and activities (tasks) show qualitative inter-subject and inter-activity differences as shown in FIG. 8 and FIG. 9.

[0104] FIG. 8 (and FIG. 8-1, FIG. 8-2) are heat maps of four tasks and Z-score correlations. FIG. 8 includes a chart 802 that provides example feature descriptions and comments. Feature descriptions (e.g., fractal dimension, sample entropy, peak frequency shrinkage, spectral entropy, and band power) may be extracted from the signals (e.g., extracted features 40) shown in FIG. 6 and FIG. 7, for example. Comments associated with the features in chart 802 provide explanations for the respective figures. Chart 804 shows Z-scores 804B for each of a number of features 804C, calculated based on normalized raw data separated by task (e.g., smiling, breathing in, closing eyes, chewing), individual (person), and time (e.g., morning / evening). Z-scores are distributed using values ​​ranging from -3 to +3, and each value is assigned a color according to legend 804A.

[0105] Similarly, FIG. 9 (and FIGS. 9-1 through 9-8) show heat maps of variables and activities showing qualitative differences between subjects and activities. Chart 904 shows Z-scores 904B for each of a number of features 904C calculated based on normalized raw data split by task (e.g., anger, chewing, eyes, eyes Iso, medial Iso, jaw, left gaze-left (L gaze L), left gaze-right (L gaze R), lateral Iso, sadness, smile Iso, surprise, swallowing, talking, up gaze, wrinkle Iso, as further described in Table 7), individual (person), and time (e.g., morning / evening). Z-scores are distributed using values ​​ranging from -3 to +3, with each value assigned a color according to legend 904A.

[0106] FIG. 10 (and FIGS. 10-1 through 10-5) shows a Spearman correlation chart 1000 for each of the features 1000B, showing the correlation of the feature 1000B to itself. The Spearman correlation chart 1000 may be used to identify relationships between features. Highly correlated features are likely to measure similar aspects (e.g., similar aspects of the individual signals 30) of the facial biology and / or other signals (e.g., electrical activity) collected by the signal capture device 10. The Spearman correlation chart 1000 may be used to identify clusters 1000A, such that similar features within the same cluster may be omitted or otherwise reduced to reduce redundancy in the analysis. For example, cluster-based reduction may be applied to identify clinically relevant features 50.

[0107] FIG. 11 shows an ICC measurement 1100 for a feature 1200A that tests the test-retest reliability of a parameter and infers clinical significance, and FIG. 12 (and FIG. 12-1, FIG. 12-2) shows the ICC measurement 1200. The ICC measurement 1100 is shown as a heat map, which corresponds to the morning measurement-based feature 1100A with fixed age, BMI, and gender. The ICC measurement can be scaled from low clinical significance (e.g., 0) to high clinical significance (e.g., 1), as shown in legends 1100B, 1200B, respectively.

[0108] The ICC measurements may be per subject (individual). A high clinical significance (e.g., 1) may indicate that similar data should be expected if a given measurement is repeated for a subject. A low clinical significance (e.g., 0) may indicate that dissimilar data should be expected if a given measurement is repeated for a subject, per subject. For a given clinical outcome, a higher clinical significance (e.g., above a clinical significance threshold) may result in a given number being designated as clinically relevant (e.g., as a clinically relevant feature 50). A higher clinical significance (e.g., in the range of 0.6 to 1) may be required for use in clinical trials. The ICC measurements may be used to cluster features, as shown at 1100C and 1200C, respectively. Clusters of features with higher clinical significance may be designated as clinically relevant (e.g., as a clinically relevant feature 50).

[0109] FIG. 13 illustrates an exemplary predictive model 1300 (e.g., a random forest model) according to an implementation of the subject matter of this disclosure. As illustrated in FIG. 13, bootstrap sampling may be used to build a model (e.g., a decision tree or network). For bootstrap sampling, r (a percentage) of examples may be selected (e.g., 0.63 in a standard implementation) and divided into random subsamples. As illustrated, a source sample 1300A may be divided into subsamples 1300B. For each subsample 1300B, a decision tree may be built based on a random set of m features (covariances) in 1300C, and the results may be classified into leaves. In 1300D, bootstrap aggregation may be performed with the results from all the built trees collected and averaged. In 1300E, a final prediction may be derived from each of the predictions in 1300D.

[0110] Several machine learning approaches were tried to optimize the F1 score, and Random Forest outperformed other models. The data was split into 80% training to fit the model and 20% testing. Random Forest built a large number of decision trees to predict individual activities / objects using the training dataset, outputting a weighted sum prediction of the individual activities / objects.

[0111] F1 score was used to quantitatively evaluate the prediction accuracy from the random forest. F1 score measures how well the model classifies a particular activity, such as swallowing, as shown by the results and calculations in Figures 14 and 15. Using parameters from both rounds of feature manipulation, an improvement in F1 score was observed for some activities, as shown in Figures 15 and 16. As shown in Figure 14, via the calculations and results 1400, a recall of 0.987 and a precision of 0.975 were output based on the true positive, false negative, and false positive test results 1400A. An F1 score of 0.98 was output in 1400B. 1400C shows the model used to output the F1 score criteria.

[0112] Chart 1500 in FIG. 15 shows F1 score results split by morning, evening, and overall. As shown, based on the signal capture device 10 used, swallowing has the highest overall F1 score at 0.98, and eye Iso has the lowest overall F1 score at 0.39. FIG. 16 shows a graph 1600 with results from the random forest model results using the CNN model, the first distribution model, and the second distribution model. As shown, certain features (parameters) show improvement when compared to other features based on the model used. FIG. 17 shows a chart 1700 with results of a study that used all variables from a particular task (activity) to predict each subject, or used all given activities to predict subjects, split by morning and evening scores. As shown, chewing has the highest morning-evening combined score (0.79 and 0.90, respectively), and sad emotion has the lowest morning-evening combined score (0.55 and 0.75, respectively). Chewing, wrinkle Iso, and speaking activities were among the top activities in predicting individual subjects (F1 score >0.85). Sadness, eye, and anger activity tracking had poor reliability in predicting individuals. Small differences in predicting individuals from morning to evening were recorded. Commonly measured facial movements were those that changed across morning and evening time points of the day. As a result of the higher F1 score-based reliability, we considered the application of the device to be used in clinical settings, focusing on measuring chewing, speaking, and swallowing.

[0113] FIG. 18 shows a chart 1800 of algorithm options for selection (e.g., for application to a random forest). One or more algorithms (rational discrete short-time Fourier transform (DSTFT), Fourier transform, discrete wavelet transform with linear classifier, Gabor wavelet transform, Hjorth parameters, Hilbert-huang transform (HHT), smoothed Wigner-Ville distribution (SWVD)) may be applied to data collected by a signal capture device. The chart 1800 provides methods, uses, advantages, and limitations of these exemplary algorithms.

[0114] FIG. 19 shows a chart 1900 of band power characteristics of ten individuals for swallowing activity. Four different band powers are shown in the four respective charts. As shown, for each individual, multiple amplitudes are collected for each individual during both morning and evening testing times. According to one implementation, each of the four different band powers in the four respective charts can be an endpoint in a clinical trial. For example, the data represented in the four respective charts can be a clinical output that is sought as a result of a clinical trial.

[0115] 20 shows the clusters and corresponding channels in table 2000. Each feature extracted from the individual signals 30 may be placed into a cluster based on, for example, CV clustering, ICC clustering, random forest clustering, etc., as disclosed herein. Features with similar results may be grouped together, as shown in the example provided in table 2000. The clusters may be used to trim the total number of features to those that may be most critically related to clinical outcomes.

[0116] 21 shows a graph 2100 with features clustered based on type (e.g., amplitude, frequency, band power channel, and / or other factors). The clusters can be used to trim the total number of features to those that may be most critically related to clinical outcomes.

[0117] FIG. 22 shows a heatmap 2200 of CV based on a mixed effects model using morning measurements with fixed effects of age, BMI, and sex. The CV heatmap 2200 shows CV values ​​for features 2200A for task 2200C based on legend 2200B, ranging from 0 to 1.2. The results of the CV heatmap 2200 may be used to identify which features are reliable (e.g., low variance) to determine clinical outcomes (e.g., disease designation). For example, a lower CV for a given feature and activity may indicate that a given feature can be reliably repeated (e.g., meets a CV reliability threshold) for multiple tests. Clinical trials may require that features used for testing meet such a CV reliability threshold.

[0118] FIG. 23 shows a chart 2300 illustrating how reliably a given task can be used to classify individuals. Band power measurements measuring AUC for various tasks are shown in table 2300A. Results of such measurements for smile Iso are shown in chart 2300B, and results of such measurements for sad emotion are shown in chart 2300C. A higher band power AUC measurement may indicate that a given task (e.g., smile Iso) meets the AUC threshold for classifying individuals (e.g., distinguishing one individual from the next), while a lower band power AUC measurement may indicate that a given task (e.g., sad emotion) does not meet the AUC threshold. Thus, the signal capture device 10 used to generate the measurements shown in FIG. 23 may be more reliable in distinguishing individuals when a smile activity is performed compared to when a sad emotion is experienced.

[0119] FIG. 24 (and FIG. 24-1) shows a heatmap 2400 (including heatmaps 2400A, 2400B) similar to chart 904 of FIG. 9 for parameter 2400C. Heatmap 2400 shows a heatmap of variables and activities showing qualitative differences between subjects and activities. Heatmap 2400 shows Z-scores for each of a number of features calculated based on normalized raw data separated by task (e.g., anger, chewing, eye, eye Iso, medial Iso, jaw, left gaze-left (L gaze L), left gaze-right (L gaze R), lateral Iso, sadness, smile Iso, surprise, swallowing, talking, up gaze, wrinkle Iso, as further described in Table 7), individual (person), and time (e.g., morning / evening). Z-scores are distributed using values ​​ranging from -3 to +3, with each value assigned a color as indicated in the legend.

[0120] FIG. 25 shows a task 2500A plotted on a UMAP chart 2500. The UMAP chart 2500 may be generated based on a visual depiction generated by reducing each of multiple parameters (e.g., parameter 2400C from FIG. 24) to two values. Thus, each trial with multiple iterations (e.g., rows) is reduced to two iterations (e.g., rows) and the results are plotted on a UMAP. The UMAP chart 2500 shows the resulting data separated by the task 2500A. For example, the UMAP chart 2500 may be used to cluster based on each of the tasks 2500A. Similarly, FIG. 26 is a UMAP chart 2600 generated using the same data used to generate the UMAP chart 2500. The UMAP chart 2600 shows the resulting data separated by the individual 2600A. For example, the UMAP chart 2600 may be used to cluster based on each of the individuals 2600A. 27 is a UMAP chart 2700 generated using the same data used to generate UMAP chart 2500 and UMAP chart 2600. UMAP chart 2700 shows the result data separated by collection time 2700A. For example, UMAP chart 2700 can be used to cluster based on each of collection times 2700A.

[0121] FIG. 28 (and FIGS. 28-1-28-3) are similar to FIG. 19 and show charts 2800 of various characteristics for 10 individuals for swallowing activity. Thirty-one different data plots are shown in each chart. As shown, for each individual, multiple amplitudes are collected for each individual during both morning and evening testing times. The data represented in chart 2800 can be used to generate, for example, the Z-score-based heat map shown in FIG. 24. According to one implementation, the 31 different data plots, each or as a combined data set, can be an endpoint in a clinical trial. For example, the data represented in the individual data plots or a combined representation of the data can be a clinical output sought as a result of the clinical trial.

[0122] FIG. 29 shows random forest importance values ​​across multiple features 2900B in a chart 2900 based on the final decision shown in legend 2900A. The chart 2900 shows the distribution of each feature 2900B for a given task. The chart 2900 may be generated using the Boruta feature selection algorithm for feature selection from a dataset. The applied Boruta algorithm may act as a wrapper algorithm around the random forest. The Boruta algorithm adds randomness to a parameter-based given dataset by creating shuffled copies of all features (e.g., shadow features). A random forest classifier may then be trained on the augmented dataset. A feature importance measure (e.g., mean reduction accuracy) may be applied to assess the importance of each feature, where higher means more important. At each iteration, a check may be made as to whether the actual feature has a higher importance than the best of its shadow features (e.g., whether the feature has a Z-score higher than the maximum Z-score of its shadow features). Features that are deemed highly unimportant (e.g., above the Volta threshold) may then be discarded to identify clinical outcomes. Clinical trials may require the use of features that meet or exceed the Volta threshold. The algorithm may complete a cycle when all features have been confirmed or eliminated, or when a specified limit for a random forest run has been reached.

[0123] The random forest importance values ​​across multiple features shown in FIG. 29 may be used to determine how important each feature is for classifying a particular activity. Features with high importance (e.g., above the Volta threshold) may be designated as clinically relevant features 50. As an example, the results shown in FIG. 29 may be used to reduce the number of features based on comparing the actual value of a given feature to a shuffled value (generated based on replication). A given feature that adds more value to the classification (e.g., determined based on removing a feature and determining how much information was lost) may receive a higher score. Machine learning may be used for permutation-based score discrimination. A given feature may be determined to be more clinically relevant for a given task than another feature. Additionally or alternatively, an endpoint in a clinical trial may be determined when removal of any remaining features reduces the ability to classify information by a given threshold. The random forest score may be based on the results of the Volta model with the extracted features being given a higher score due to the results of the Volta model. Extracted features with a random forest score equal to or greater than the random forest threshold are identified as clinically relevant features.

[0124] FIG. 30 shows a chart 3000 including a gap statistic. The gap statistic (e.g., k value) can be used to determine the optimal number of clusters for a given data set. The chart 3000 can be related to the Spearman correlation plot of FIG. 10, for example. This Spearman correlation can be used to identify the number of clusters based on the available signal-based data. As shown, the optimal number of clusters 3000A based on the data presented in the chart 3000 is four, and having four clusters given the respective data provides a threshold balance between the difference between features within the clusters and the number of clusters.

[0125] FIG. 31 illustrates Z-scores from a heatmap 3100 across tasks during morning collection. Z-scores are for each feature 3100A per task 3100B per legend 3100C, with colors assigned in the range of values ​​from -3 to +3. FIG. 32 illustrates Z-scores from a heatmap 3200 across tasks during evening collection. Z-scores are for each feature 3200A per task 3200B per legend 3200C, with colors assigned in the range of values ​​from -3 to +3. As disclosed herein, the Z-scores shown on a heatmap (e.g., heatmap 3100 and / or 3200) are normalized to their respective ranges such that higher values ​​within the range of possible values ​​correspond to higher Z-scores and lower values ​​within the range of possible values ​​correspond to lower Z-scores.

[0126] FIG. 33 shows the Z-scores of LOOCV across tasks on a heatmap 3300. The heatmap 3300 of FIG. 33 is generated by a procedure used to estimate the performance of a machine learning algorithm. Data ranges in value from -3 to +3 and is provided to individuals 3300A, tasks 3300B based on a legend 3300C with assigned colors. In LOOCV, the number of folds may be the same as the number of instances in the dataset. Thus, the learning algorithm may be applied once per instance using all other instances as a training set and selected instances as a single-item test set. For example, to generate the heatmap 3300, the individual's data may be removed from the training set of a predictive machine learning algorithm. The remaining data may be used to train the algorithm. The algorithm may then be used to predict the data of the removed individual. The Z-scores shown in the heatmap 3300 may be generated based on how the predictions compare to actual data and / or how well the predictions identify a given task for the individual.

[0127] FIG. 34 shows Z-scores in a heatmap 3400 for standard deviation (SD) during morning collection. Heatmap 3400 shows Z-scores of standard deviation for feature 3400A in task 3400B based on legend 3400C, with colors assigned with value ranges from -3 to +3. FIG. 35 shows Z-scores in a heatmap 3500 for SD during evening collection. Heatmap 3500 shows Z-scores of standard deviation for feature 3500A in task 3500B based on legend 3500C, with colors assigned with value ranges from -3 to +3. The standard deviation indicates the amount of variability in the feature data, such that a high standard deviation may indicate less confidence, while a low standard deviation may indicate more confidence.

[0128] FIG. 36 shows CV with heatmap 3600 for a mixed effects model in evening collection. FIG. 36 is similar to FIG. 22. FIG. 36 shows CV based on a mixed effects model using morning measurements with fixed effects of age, BMI, and sex. CV heatmap 3600 shows CV values ​​for features 3600A for task 3600C based on legend 3600B, with colors assigned with values ​​ranging from 0 to 1.2. The results of CV heatmap 3600 may be used to identify which features are reliable (e.g., low variance) to determine clinical outcomes (e.g., disease designation). For example, a lower CV for a given feature and activity may indicate that a given feature can be reliably repeated (e.g., meets a CV confidence threshold) for multiple tests. Clinical trials may require that features used for testing meet such a CV confidence threshold.

[0129] FIG. 37 (and FIG. 37-1) shows a spread 3700 of band power measurements for a collection of smiles. The spread 3700 is similar to that calculated for swallowing in FIG. 28. The spread 3700 shows various characteristics of the smile activity of ten individuals. Ten different data plots are shown in each spread. As shown, for each individual, multiple amplitudes were collected for each individual during both morning and evening testing times. The data represented in spread 3700 can be used to generate, for example, the Z-score based heat map shown in FIG. 24.

[0130] FIG. 38 (and FIG. 38-1) shows a clustered ICC heatmap 3800 for evening measurements with fixed effects of age, BMI, and gender. The heatmap 3800 is based on the features 3800A of the task 3800C based on the legend 3800B, with colors assigned with values ​​ranging from 0 to 1. The ICC measurements shown on the heatmap 3800 base indicate how similar the results are for a given person when the same measurements are calculated for that person across multiple collections. The results identify correlations with data from the same individual. Thus, the heatmap 3800 indicates test-retest reliability. A higher ICC value (e.g., 1) indicates low differences across multiple tests for a given individual, and therefore data for that individual correlates with itself. A lower ICC value (e.g., 0) indicates high differences across multiple tests for a given individual. The heatmap 3800 also indicates which features meet the ICC threshold, so that features associated with higher ICC values ​​(e.g., 1) across subjects may be used in clinical trials as reliably providing data about an individual. As shown in 3800D, the various ICC correlation values ​​may be clustered (e.g., into six clusters in this example).

[0131] FIG. 39 shows another clustered ICC heatmap 3900 for evening measurements with fixed effects of age, BMI, and gender. The heatmap 3900 is based on features 3900A of task 3900C based on legend 3900B, with colors assigned with values ​​ranging from 0 to 1. The ICC measurements shown on the heatmap 3900 base indicate how similar the results are for a given person when the same measurements are calculated for that person across multiple collections. The results identify correlations with data from the same individual. Thus, the heatmap 3900 indicates test-retest reliability. A higher ICC value (e.g., 1) indicates low differences across multiple tests for a given individual, and therefore data for that individual correlates with itself. A lower ICC value (e.g., 0) indicates high differences across multiple tests for a given individual. The heatmap 3900 also indicates which features meet the ICC threshold, so that features associated with higher ICC values ​​(e.g., 1) across subjects may be used in clinical trials as reliably providing data about an individual. As shown in 3900D, the various ICC correlation values ​​may be clustered (e.g., into four clusters in this example).

[0132] FIG. 40 shows t-distributed stochastic neighborhood embedding (t-SNE) charts 4000A, 4000B, 4000C for individuals, tasks, and time. Chart 4000A corresponds to reduced data based on individuals 4000D, chart 4000B corresponds to reduced data based on tasks 4000E, and chart 4000C corresponds to reduced collection time 4000F. Charts 4000A, 4000B, 4000C may be generated based on an algorithm that reduces the number of data sets to two dimensions. Charts 4000A, 4000B, 4000C may be used to compare how similar or dissimilar the respective data sets are (e.g., how variable the data associated with each individual in 4000D is, as shown in 4000A). Comparing the charts 4000A, 4000B, 4000C may show contributions from variance based on a given parameter (e.g., across individuals 4000D, across tasks 4000E, and / or across collection times 4000F).

[0133] FIG. 41 shows various charts 4100 for visualization of data reduced to two dimensions based on task 4100A. Chart 4100B is based on Principal Component Analysis (PCA), where the principal components of a set of points in real coordinate space are a sequence of p unit vectors, where the i-th vector is the direction of the line that best fits the data while being orthogonal to the first i-1 vectors. Chart 4100C is based on both PCA and t-SNE reduction. Chart 4100D is based on t-SNE reduction. Chart 4100E is based on UMAP reduction.

[0134] FIG. 42 shows a UMAP chart 4200 for individuals in chart 4200A, tasks in chart 4200B, and time in chart 4200C. Charts 4200A, 4200B, 4200C may be similar to those provided in FIGS. 25-27. Chart 4200A may show tasks plotted using UMAP reduction. Chart 4200A shows a visual depiction generated by reducing each of multiple parameters (e.g., parameter 2400C from FIG. 24) to two values. Thus, each test with multiple iterations (e.g., rows) is reduced to two iterations (e.g., rows) and the results are plotted on the UMAP chart. Chart 4200A shows data of results separated by tasks. For example, chart 4200A may be used to cluster based on each of the tasks. Similarly, chart 4200B is generated using the same data used to generate chart 4200A. Chart 4200B shows data of results separated by individuals. For example, chart 4200B may be used to cluster based on each individual. Similarly, chart 4200C is generated using the same data used to generate chart 4200A or chart 4200B. Chart 4200C shows the resulting data separated by collection time. For example, chart 4200C may be used to cluster based on each time.

[0135] 43 and 44 show the validation results in charts 4300, 4400 for different tasks across various channels. The data provided in charts 4300, 4400 may be generated based on the Volta analysis (e.g., feature importance) described in FIG. 29. Charts 4300, 4400 show validation results 4300B, 4400B for tasks 4300A, 4400A for parameters 4300C, 4400C. For example, FIG. 29 shows the analysis results for a single task, while FIG. 43 and FIG. 44 show the analysis results for multiple tasks. Charts 4300, 4400 show whether given data (e.g., results 4300B, 4400B for tasks 4300A, 4400A for parameters 4300C, 4400C) are clinically relevant in identifying clinical outcomes. Relevant data is shown as confirmed, and non-relevant data is rejected. Data that do not meet the relevance or non-relevance thresholds are designated as provisional. Clinical trials may require that parameters used in the trial meet relevance thresholds for confirmation, for use in the clinical trial to determine clinical outcomes.

[0136] [Example 2] This example provides a procedure used to evaluate the capabilities of the signal capture device 10 to facilitate the determination of clinically relevant characteristics based on the capabilities and clinical outcomes of the signal capture device 10. While this example provides a particular application of the methods disclosed herein, it can be understood that additional applications of the methods can be implemented.

[0137] According to the technology disclosed herein, as a first step in developing a digital assessment for neuromuscular diseases, a study was conducted to determine whether a biometric sensor device could be utilized to objectively measure facial muscle and eye movements intended to be representative of PerfO using a task designed to model a clinical performance outcome assessment (PerfO). This is referred to as simulated PerfO activity. The specific objectives of this study were to determine whether the raw EMG, EOG, and EEG signals of the biometric sensor device could be processed to extract features that describe these waveforms, to determine the quality, test-retest reliability, and statistical properties of the biometric sensor device feature data, to determine whether features derived from the biometric sensor device can be used to determine differences between various facial muscle and eye movement activities, and to determine which features and types of features are important for simulated PerfO activity level classification.

[0138] It can be seen that the clinical outcome tested in this example is to identify, based on the biometric sensor device, which features and types of features are important for classifying the simulated PerfO activity level.

[0139] The biometric sensor device used in this embodiment is an ear-mounted wearable originally developed to measure cognitive function. The biometric sensor device measures, for example, electroencephalography (EEG), electromyography (EMG), and electrooculography (EOG) data, and may therefore also have the potential to objectively quantify facial muscle and eye movement activity relevant to the evaluation of neuromuscular diseases.

[0140] A total of N=10 healthy volunteers participated in the study. Each study participant performed 16 simulated PerfO activities, including talking, chewing, swallowing, closing eyes, looking in different directions, puffing out cheeks, biting an apple, and making various facial expressions. Each activity was repeated four times in the morning and four times in the evening. A total of 161 summary features were extracted from the EEG, EMG, and EOG biosensor data. The feature vectors were used as input to machine learning models to classify the simulated PerfO activities, and model performance was evaluated on a presented test set. A convolutional neural network (CNN) was also used to classify low-level representations of the raw biosensor data for each task, whereby model performance was evaluated and directly compared with the feature classification performance.

[0141] The predictive accuracy of the model against the classification ability of the biometric sensor device was quantitatively evaluated. The study results showed that the tested biometric sensor device can potentially quantify different aspects of face and eye movements and can be used to distinguish simulated PerfO activities. In particular, the identified clinically relevant features showed that the tested biometric sensor device distinguishes speaking, chewing, and swallowing tasks from other tasks with an observed F1 score above 0.9. While EMG features contribute to the classification accuracy of all tasks, EOG features are important for classifying gaze tasks. The analysis using summary features was found to outperform CNN for activity classification.

[0142] As further described herein, the biometric sensor device was determined to meet clinical thresholds such that it may be used to measure cranial muscle activity associated with neuromuscular disease assessment. Classification performance of simulated PerfO activity using summary features allows for strategies to detect disease-specific signals relative to control groups, as well as monitoring of within-subject treatment responses.

[0143] Facial / cranial and ocular motor dysfunction is a key feature of several neurological disorders that affect multiple levels of the neuraxis. Examples include complete facial weakness due to facial nerve palsy or stroke, diplopia, ptosis, and dysphagia caused by neuromuscular diseases such as myasthenia gravis, dystonia, complex extraocular motor deficits, asthenia, and dysphagia caused by Parkinson's (and other neurodegenerative) conditions.

[0144] As discussed herein, clinical assessment of these symptoms remains a challenge in medicine and clinical research. Existing clinical assessments, such as Clinician-Reported Outcomes (ClinROs) or Patient-Reported Outcomes (PROs), may require patients to visit the facility frequently, rely primarily on subjective measures, and may not necessarily reflect the patient's condition in the real world. Importantly, patients' symptoms may be intermittent and vary throughout the day, making reliable assessment difficult. Finally, they may vary from patient to patient as they adapt to increasing muscle weakness. For example, chart 500 in FIG. 5 highlights the subjective nature of physician scores 502, which are based on observations. Such subjective scoring may cause variability from patient to patient and from provider to provider.

[0145] Although tools exist for performing quantitative analysis of cranial muscle function, these tools have significant limitations. For example, facial movements can be measured with video-based techniques that use either still images or video capture. Surface EMG, which records the electrical movement of facial muscles, can also be used, alone or in combination with video-based methods. Small studies suggest that EOG, which measures the electrical potential from the front to the back of the eye, can detect differences between Parkinson's disease patients and control groups. Screen-based trackers and wearable glasses have been used to monitor extraocular movements and epicranial activity (e.g., blinking). In their current applications, these approaches are cumbersome, difficult to implement, and most importantly, are unable to capture brief periods of facial movements in an artificial setting.

[0146] Thus, the methods disclosed herein are advantageous in providing an opportunity to identify and / or develop novel non-invasive approaches to measure individual attributes (e.g., cranial manifestations of neuromuscular and neurodegenerative diseases) to address problems in important patient populations. The techniques disclosed herein serve to assist clinicians in diagnosis and disease progression assessment, as well as outcome assessment in clinical studies. If such approaches can leverage wearable sensing technology, they may be able to address the challenges of existing clinical sensors that are limited to use in highly controlled environments as opposed to more natural environments (e.g., at home).

[0147] The biometric sensor device being tested is an ear-hanging device developed to measure neural and physiological processes. Electrophysiological signals are acquired at 250 Hz via four reusable electrodes fabricated from conductive silicon material. The electrodes of the device are placed at scalp locations directly above the left and right ears and over the left and right mastoid processes, resulting in raw biosignal data similar to that which may be acquired at electroencephalography (EEG) reference locations T3, T4, M1, M2 of 10-20 electrode placement locations. This EEG is a measure of surface electroencephalographic function. This electrode configuration also allows for high-fidelity acquisition of EMG activity from activation of the temporalis and surrounding muscle groups, as well as EOG signals resulting from eye deflection.

[0148] While traditional clinical evaluations using biophysiological data can be invasive, expensive, and time-consuming, the biometric sensor device tested aims to provide high-fidelity data acquisition and processing to the general population. EMG, EEG, and EOG signals monitored with the tested biometric sensor device were used for detection and evaluation of a wide variety of physiological phenomena, such as sleep monitoring, microsleep detection, and acute postoperative pain quantification. Based on the identification of facial muscle-related features 50 using the biometric sensor device (signal capture device 10), it was determined that the device has the potential to support outcome evaluation of neuromuscular diseases by objectively quantifying clinical and eye-movement tasks through capturing and analyzing biosignal data. The objective quantification quality was evaluated by the extracted features 40 generated from the individual signals 30 collected via the biometric sensor device. The individual signals 30 were generated using a signal manipulation module 20 that received signals from the biometric sensor device. The techniques disclosed herein were applied to identify clinically relevant features 50 from the extracted features 40. Fifty clinically relevant features met the threshold for clinical outcomes including diagnosis and / or treatment of neuromuscular disease.

[0149] A key challenge addressed by the methods disclosed herein is that raw biosignal data is inherently noisy due to several factors, such as participant movement during clinical assessment, possible perturbations in electrode-skin contact, and / or the presence of artifacts from cardiac activity, etc. In addition, similar factors naturally induce artifacts in the acquired signal data, and EEG, EMG, and EOG signals overlap in typical frequency ranges, making direct separation and analysis of the waveform data challenging.

[0150] Therefore, to develop a digital assessment for neuromuscular diseases, this exemplary study was conducted to determine whether a biometric sensor device can measure facial muscle and eye movements. Some specific objectives of this study are: how the biometric sensor device EMG / EOG / EEG signals can be processed to extract features; determine the quality, test-retest reliability, and statistical properties of the biometric sensor device feature data; determine whether the features derived from the biometric sensor device can quantify various facial and eye activities; and determine which features are important for activity level classification (e.g., clinically relevant features 50) compared to raw biosignal data classification approaches.

[0151] In this study, 16 simulated performance outcome assessments (Simulated PerfO) were designed to assess face and eye movements using a biometric sensor device on N=10 control volunteer participants. A fit-for-purpose feature manipulation pipeline is then implemented. In this implementation, features from EMG, EOG, and EEG waveforms are derived, feature relationships are evaluated relative to each other, and a qualitative evaluation of how the features classify different simulated PerfO activities is performed. The steps of conducting this exemplary study support the usefulness of the framework for the development of digital assessments and the analytical validation steps. Taken together, the results from this exemplary study highlight the usefulness of the biometric sensor device as a potential measurement tool in a clinical trial environment to evaluate face and eye movement tasks, enabling further clinical development with this and similar devices. The usefulness of the biometric sensor device is determined by the identification of sufficient clinically relevant features 50, as described herein.

[0152] To evaluate how well the biometric sensor device could classify (e.g., distinguish between) facial muscle tasks and eye movement tasks, a study was conducted with N=10 participants who performed 16 facial muscle task exercises (Simulated PerfO) four times in the morning and four times in the evening. Table 1 shows the demographic characteristics of the study participants.

[0153] [Table 1] The raw biosignal data of the biometric sensor device was processed into 161 summary features, most of which represent EMG, EOG, and EEG waveforms. As shown in FIG. 1A, the biometric sensor device (signal capture device 10) provided raw signals to a signal manipulation module 20. The signal manipulation module 20 generated individual signals 30 (e.g., EMG, EOG, and EEG waveforms) from the raw signals. The 161 summary features were generated based on the characteristics of the individual signals 30.

[0154] The process for summarizing the raw biometric sensor device biosignal data into features is described in detail herein. Briefly, features are computed from EMG, EOG, and EEG waveform components that are separated from the raw mixed waveform biosignal via specialized signal combining and filtering mechanisms (e.g., by the signal manipulation module 20). A high-level overview of the feature manipulation process 4500 is also summarized in FIG.

[0155] As shown in FIG. 45, a mixed signal waveform 4502 is received from a biometric sensor device. A signal separation module 4504 can extract separate virtual EEG signals 4506A, virtual EOG signals 4506B, and virtual EMG signals 4506C from the mixed signal waveform 4502. An event-based segmentation algorithm can be applied in an event-based segmentation algorithm 4508, and feature computation in 4510 can result in feature extraction via a feature vector representation 4512. As shown, the signal separation module 4504 is applied to the mixed signal derived from the biometric sensor device to separate the EEG, EMG, and EOG waveforms into their component parts. These signals are then applied to the event-based segmentation algorithm 4508 and features are extracted.

[0156] As shown in Fig. 46, the features may represent both the frequency and time domain of the biometric device signal via amplitude in time 4600A and frequency 4600B collected of a subject drinking water. Fig. 46 shows a time and frequency representation of EMG activity resulting from a participant drinking water. Plot 4600 shows approximately 6.5 seconds of EMG data in both the time domain 4600A and frequency domain 4600B.

[0157] Representative mixed signal waveforms 4502 were collected for each of the 16 simulated PerfO activities. For example, FIG. 47 shows the qualitative differences observed from representative signals 4700 from each of the 16 simulated PerfO activities. As shown, each activity has a qualitatively different waveform. Representative signals 4700 show EMG activity visualized in the time domain across the 16 activities.

[0158] As an example, multiple features extracted from the representative signal 4700 from each of the 16 simulated PerfOs can be used to generate a Z-score heatmap, as described further herein and also shown in Figures 8 and 9.

[0159] Tables 2-1 to 2-3 show a list of extracted features in this example, where the features are described in the described categories. Amplitude features, zero crossing rate, standard deviation, variance, root mean square, kurtosis, frequency, band power, skew, as well as other standard waveform features were processed from the biometric sensor data. The features were selected according to a standard feature processing pipeline. Amplitude features describe the amplitude or maximum distance from the baseline of each wave in the associated component space. Band power features describe the average power of the waveform in a particular frequency range (where there are multiple frequency ranges specific to each signal type for the biometric sensor device). Other specified features mathematically describe the shape, variance, or complexity of the EMG, EOG, or EEG waveform.

[0160] [Table 2-1]

[0161] [Table 2-2]

[0162] [Table 2-3] The biometric sensor device-based features shown in Tables 2-1 to 2-3 above can measure unique aspects of face and eye movements. The relationship between the parameters of all 16 simulated PerfO activities was analyzed by performing Spearman correlation of all parameters with each other (e.g., by the method described with respect to FIG. 4C and FIG. 10). To determine the optimal number of parameter groups that describe the overall signal variability, k-means clustering of the Spearman correlation of all features with each other for all activities has been performed, as shown in FIG. 48. Six unique clusters of parameters were determined from the Spearman correlation (based on k-means clustering). k-means clustering is a method of vector quantization that divides n observations (e.g., features based on Spearman correlation) into k clusters, with each observation belonging to the cluster with the closest mean and serving as a prototype for the cluster.

[0163] As described herein, the Spearman correlation chart 4800 is used to identify relationships between features. Highly correlated features are likely measuring similar aspects (e.g., similar aspects of the individual signals 30) of facial biology and / or other signals (e.g., electrical activity) collected by the biometric sensor device. The Spearman correlation chart 4800 is used to identify six clusters such that similar features in the same cluster are omitted or otherwise reduced to reduce duplication in the analysis. For example, cluster-based reduction may be applied to identify clinically relevant features 50. FIG. 47 shows the Spearman correlation of all 161 Earable features with each other, represented as a heat map. k=6 clusters (optimal number) from k-means clustering are shown. All 16 simulated PerfOs are pooled for the correlation analysis shown in FIG. 47.

[0164] In this example, the amplitude and band power parameters tended to cluster together in two of the six clusters, while other parameters, such as those from the frequency domain, clustered separately.

[0165] To investigate the qualitative differences between the 16 simulated PerfO activities (across each participant and time point, as well as all 161 biometric sensor device parameters), UMAP dimensionality reduction was performed. The qualitative differences between the 16 simulated PerfO activities 4900A are shown in chart 4900 of FIG. 49. Although there was overlap between some of the activities 4900A, activities such as swallowing were clearly separated from the rest using this approach. The UMAP dimensionality reduction shown in chart 4900 can be used to identify clusters of data and / or to separate data by dimension (e.g., by task). The UMAP dimensionality reduction of all 161 biometric sensor device features is shown in FIG. 49. Each individual activity iteration is a point on chart 4900. The visual features of a given point represent the activity performed during that activity.

[0166] FIG. 50 includes a chart 5000 that provides a heat map 5000A (EEG-based features), 5000B (EMG-based features), 5000C (EOG-based features), 5000D (other features) of feature Z-scores across data showing differences between tasks 5004 for different classes 5002 of features (amplitude, band power, frequency, kurtosis, other, skew, time, variance). Data for each individual 5006 is collected over time 5008 and represented as Z-scores based on legend 5010 and assigned colors ranging in value from -3 to +3. Chart 5000 may be generated in a manner and include information similar to FIG. 8, FIG. 9, and FIG. 24 described herein. Taken together, the results shown in chart 5000 demonstrate the utility of a biometric sensor device for generating parameters that may describe unique simulated PerfO activity.

[0167] Figure 50 contains a heatmap of all 161 biometric sensor device features (rows) for all activity repetitions (columns). The columns are first sorted by the 16 activities in the study, by participant within each activity, and then by the time of day the activity was performed, as described above.

[0168] To evaluate the test-retest reliability of the biometric sensor device features, linear mixed-effects modeling is used with participants as random effects to evaluate feature properties. Table 3 shows the variance components analysis of the biometric sensor device features. First, for each of the 161 biometric sensor device features, the ICC for participants to evaluate the variance associated with each person for each feature is determined based on the flow in Table 4. As described herein, the ICC is a measure of how similar, and therefore how reliable, the same data from the same participant is for the same activity, and ranges from 0 to 1 (e.g., an ICC below 0.5 is unreliable, an ICC between 0.5 and 0.7 is moderately reliable, and an ICC above 0.7 can be interpreted as a reliable metric). The observed ICC values ​​ranged from 0 to 0.92, with the average ICC value of all parameters across the 16 activities being 0.31. Then, the CV of each parameter within participants across each time point (morning and evening) is calculated according to the methods disclosed herein. The variance of each feature for each activity associated with the time of activity (morning or evening), the individual participant themselves, and the individual test repetitions, as well as the unexplained variance, are calculated. The ICC calculations, CV, and / or variability are used to identify clinically relevant features 50 from the extracted features 40, for example, as shown in FIG. 1A. In this example, the results support that many biometric sensor device features reliably measure within-participant variability, providing a metric by which candidate features can be ranked for further downstream analysis. Such features may therefore be designated clinically relevant features 50.

[0169] [Table 3]

[0170] [Table 4] It is determined that the tested biometric sensor device can accurately classify the motor activities of some facial muscles. To investigate whether the biometric sensor device data can classify a given one of the 16 simulated PerfO activities, a random forest classification model as described herein is constructed to detect each activity (one-versus-all classification) from the other 15 activities (e.g., as described with reference to FIG. 29, FIG. 43, and FIG. 44). The activity detection F1 score is used as the primary metric to evaluate the model performance, as described further herein.

[0171] Following the development evaluation on the test dataset for all 161 features, a second model is constructed for activity level classification. This second model uses an optimized set of biometric sensor device features with the goal of eliminating noisy features that do not contribute to the overall classification performance. For the determination of the optimized set of biometric sensor device features, feature reduction using the Boruta package is performed. Such feature reduction is described in this disclosure with reference to Figures 29, 43, and 44. For example, the features of Figures 29, 43, and 44 from all 161 are copied (e.g., designated as shadow features) and their classification labels are randomly shuffled. Each shadow feature is compared with the actual value for 1,000 iterations of classification, and only features that perform better than a given threshold (e.g., 50%) are designated as confirmed. This analysis showed 101 confirmed sets for the features of Figures 29, 43, and 44 used for the second classification model.

[0172] To evaluate how well the biometric sensor device features perform versus using a low-level representation of the biometric sensor device waveform data, a CNN model is built using the biometric sensor device's raw biosignal data to classify 16 simulated PerfO activities, as shown in FIG. 51. FIG. 51 shows a model 5100 that implements an architecture diagram of the final CNN implemented for activity classification. A single channel spectrogram calculated from the segmented waveform is input to the model during classification. A probability distribution over each of the 16 activities is output. The activity associated with the highest output likelihood estimate is inferred. In this modeling, a fixed-size spectrogram that quantifies how the power at a given frequency changes as a function of time is calculated from the simulated PerfO signal segment and used as the model input.

[0173] FIG. 52 shows an activity chart 5200 for an activity 5200A with level classification F1 scores for all biometric sensor device features (161 features) 5200B, Boruta selected biometric sensor device features (101 features) 5200C, and raw waveform data (CNN) 5200C. The F1 score ranges from 0 to 1, with 1 indicating perfect classification. The Boruta selected biometric sensor device features (101 features) 5200C may be clinically relevant features 50 extracted from the biometric sensor device features (161 features) 5200B (e.g., as shown in FIG. 29, FIG. 43, and FIG. 44). The raw waveform data (CNN) 5200C may be generated using the model 5100 of FIG. 51. Figure 53 includes a heatmap 5300 showing feature attribute analysis using SHapley Additive exPlanations (SHAP) values ​​for each feature (rows) 5300A for each activity (columns) 5300B determined on the model from the full set of 161 features. SHAP values ​​are Z-scored across all activities and assigned colors ranging in value from -3 to +3, as shown in legend 5300C.

[0174] As shown in FIG. 52, F1 scores are compared for classification accuracy between the complete set of 161 features 5200B, the optimized set of 101 features 5200C, and the predictions from CNN 5200D. To determine the most important underlying features for the complete RF model with 161 features (e.g., clinically relevant features 50), feature attribute analysis with SHAP is applied in FIG. 54. For a particular activity prediction, the SHAP value of a biometric sensor device feature is calculated as the change in the expected value of the model output when this feature is observed compared to when it is missing for the test set prediction. The impact of each feature when added to the model is summed and averaged over all 161 biometric sensor device features used. In FIG. 53, the features are represented as average SHAP values ​​and shown in a heatmap (log10) 5300. The percent contribution of each waveform group of features to each activity is determined and shown in Table 5. The normalized sum of absolute SHAP values ​​for each activity is compared to the sums within the EMG, EEG, and EOG features and normalized by the number of features in that group to calculate the percent contribution of each waveform to the classification accuracy.

[0175] [Table 5] Table 5 includes 16 simulated PerfO activities and shows how EMG, EEG, and EOG feature groups contribute to classification accuracy. Table 5 shows the relative EMG, EEG, and EOG percent contributions to classification importance along with the normalized sum of absolute SHAP values ​​from the RF model. Feature importance is normalized based on the total number of features in each EMG, EEG, or EOG group compared to the total number of features in all three categories. Features not related to any waveform are excluded from this analysis.

[0176] As disclosed herein, a total of 10 healthy volunteers (5 males and 5 females) contributed to this exemplary study. All participants completed two 45-minute sessions. During each session, each participant was asked to complete a series of tasks listed in Table 6 below. These tasks were selected to represent tasks that MG patients may have difficulty completing. Participants were asked to take a 1-minute break between each task.

[0177] [Table 6] Each study participant attended two study sessions, one in the morning and one in the evening. The training session was conducted one-on-one by the study moderator. During the morning session, the study moderator administered an informed consent form (ICF) with the participant to ensure that he / she understood the form and agreed to participate. Participants had time to ask any questions before signing the ICF.

[0178] The study moderator read the study script, which provided an overview of the study and a description of the various study activities. The study moderator then collected baseline (background) information on the participants.

[0179] Next, in each study session, the study moderator had participants do the following: Smile widely and show as many teeth as possible 1 minute break Wrinkle your forehead as tightly as possible 1 minute break Close your eyes as tightly as possible.

[0180] 1 minute break Puff out your cheeks as much as possible 1 minute break Suck your cheeks in as far as possible 1 minute break Chew for 30 seconds 1 minute break Swallowing 1 minute break Close your eyes normally for 5 seconds 1 minute break Talk for 30 seconds 1 minute break Stare upwards for 45 seconds 1 minute break Gaze to the left for 45 seconds 1 minute break Gaze to the right for 45 seconds 1 minute break Open and close your jaw as wide as possible 1 minute break Make a surprised expression 1 minute break Have a sad expression 1 minute break Make an angry face The label annotations disclosed herein correspond to the following tasks, shown in Table 7:

[0181] [Table 7] Raw biometric sensor device data was collected continuously during each activity in this exemplary study. To ensure reliable ground truth data annotation, data from each activity was manually labeled by an expert technician. For each activity, onset and extinction endpoints for each performed activity were annotated accordingly. Time-synchronized video recordings of participants were utilized as a reference source in this annotation procedure. These activity annotations were then used to segment the signal according to the annotated onset and extinction timestamps. Note that raw data collection may be performed automatically by using sensors that transmit raw data to one or more receivers or controllers (e.g., as shown in Figures 1-3) according to the methods disclosed herein.

[0182] After completion of the activity, the resulting signals from each channel were scaled to counteract the effects of amplification performed in the device hardware for noise suppression purposes and filtered offline using a second-order infinite impulse response (IIR) notch filter to remove 60 Hz power line noise. Each signal contained a mixture of EEG, EMG, and EOG data (e.g., mixed signal waveform 4502). A signal separation algorithm was applied (e.g., by signal separation module 4504) to better separate each component, resulting in a total of six channels in this example (two each for EEG, EMG, and EEG).

[0183] Following signal scaling, filtering, and separation, the signal for each of the six separated channels was segmented based on the presence or absence of facial movement activity, as shown in FIG. 45. For further downstream analysis (e.g., 4510 in FIG. 45), a comprehensive approach to feature extraction was taken. General features of each waveform were summarized apart from the subset of features specific to EMG, EOG, or EEG activity. Features that clearly identify the simulated PerfO activity performed within the data collection process but do not generalize to the performance of the activity outside of a laboratory context were omitted (e.g., the duration of the activity each participant was instructed to perform over a specified period of time).

[0184] An event-based segmentation algorithm 4508 and feature computation 4510 were performed. Statistical measures from each isolated signal segment were computed to summarize the signal behavior in the time domain (see, e.g., FIG. 46). Such measurements allow depiction of information such as time-varying amplitude behavior, amplitude distribution, and signal trends observable in raw form. Since the frequency and time-frequency domains also contain a vast amount of information within the biosignal data, a digital signal processing (DSP) analysis was performed to decompose each isolated signal segment into frequency components and evaluate patterns within this alternative representation, as shown in FIG. 45. To better represent such activities in the summary feature vector, features related to theoretical EMG, EOG, and EEG behavior during specific simulated PerfO activities were computed.

[0185] As described herein, the steps described above (also shown in FIG. 45) resulted in a 161-dimensional feature vector representation for each simulated PerfO activity performed, as shown in Tables 2-1 to 2-3. These features correspond to the extracted features 40 in FIG. 1A. Feature reduction using the Boruta algorithm was implemented to remove features that are potentially irrelevant to the activity-based classification for a given clinical outcome studied. As shown in FIG. 52, a total of 161 features were trimmed to obtain a lower dimensional feature vector representation of each simulated PerfO activity. As shown, 60 features estimated to be "unimportant" were removed from each feature vector, resulting in a 101-dimensional feature vector. To perform the feature reduction, a Python implementation of the Boruta package (BorutaPy, version 0.3) was used.

[0186] Correlations between biometric sensor device parameters and parameter differences between activities were observed. Spearman correlations between all parameters and all activities were calculated, as shown in Figure 48. Silhouette techniques were used to determine the optimal number of clusters with the factoextra package in R using the function fviz_nbclust with 100 bootstrap samples. For each of the 16 activities, the number of tasks analyzed (n), minimum (min), maximum (max), median (median), mean (mean), mean standard deviation (sd), and mean standard error (se) are shown in Table 4 for all 161 b parameters calculated.

[0187] Relationships between biometric sensor device features and activities or between biometric sensor device features and demographic information were determined. For data from the exemplary study, ICCs for participants as a group were calculated in the following manner using linear mixed-effects modeling with the lmer package in R (one participant). ICCs were calculated for each of the 161 biometric sensor device features separately for each of the 16 activities according to Table 4. Coefficients of variation were also calculated comparing within each activity according to Table 4.

[0188] Within- and between-study variability attributable to repeated measures, time of day, and participant was calculated. Variance not explained by these three factors was also calculated according to Table 4. A nested linear mixed-effects model to derive the variance explained by each component was used: approximately 1 + (one time) + (one participant) + (one repetition / time), where the time component indicates the time of day (morning or evening), the participant component indicates the subject, and the repetition component indicates repetitions of the same activity nested within the same time.

[0189] As shown in Figures 25, 26, 27, 40, 41, 42, and 49, dimensionality reduction of the biometric device features was performed using umap-learn or one or more applicable dimensionality reduction methods in Python. Reduction was performed using the effective minimum distance between the embedding points and default parameters. For example, as shown in Figure 49, the UMAP coordinates were plotted with ggplot2 in R. As shown in Figure 50 (and Figures 50-1, 50-2), heat maps of the biometric sensor device features are displayed with individual activities as columns and biometric sensor device features as rows. The heat maps of the biometric sensor device data (e.g., Figures 8-12, 22, 24, 31-36, 38, 39, 48, 50, and 53) show the z-scored feature rows, computed across all activities. The heat maps were constructed using the ComplexHeatmap package in R.

[0190] Study activities and participant-level predictions were quantified. A multi-class classification model using the Python sklearn module was implemented to determine how biometric sensor device features could be used to classify each of the 16 activities. A random forest classifier (e.g., using sklearnRandomForestClassifier classifier) ​​with 500 decision trees was implemented for model construction. For each classification setting, 80% of the dataset was used to train and validate the model, and the remaining 20% ​​of the dataset was withheld for testing. Data samples were randomly assigned to one of two subsets to reduce bias in the evaluation results. The F1 score was calculated to evaluate model performance on the test set, as shown in Figure 52. The F1 score is the harmonic mean of precision and recall, and accounts for the number of predictions that were accurate by the model, balancing both false negatives and false positives.

[0191] We determined a CNN model for activity level prediction. Deep learning models have been used to achieve high performance in many tasks related to classification of biosignal data. Among the many popular deep learning architectures leveraged in such tasks, CNNs are widely used due to their ability to learn patterns in structured multidimensional data (e.g., time-frequency signal representations). In applying such methodology to the task of simulated PerfO activity level classification, 16-class CNN classification models were developed and analyzed. These CNN models were constructed to map 2-dimensional spectrogram representations of simulated PerfO activity signal segments to probability distributions across 16 classes.

[0192] Since deep learning models often require large datasets to learn generalizable functions, data augmentation was used to maximize diversity in the training set. Each time a signal segment was read into the training dataset, multiple random crops of this segment were also added to the training set. To some extent, this allowed for an increase in the size of the training dataset without collecting additional samples and helped to combat overfitting. To maintain a constant length of input signal among simulated PerfO activities of varying duration, activity segments with a duration shorter than a fixed input data duration (e.g., 30 seconds) were repeated after shifting the segment according to a randomized cropping scheme, and segments with longer durations were truncated to the fixed input data duration via randomized cropping. Data augmentation was not performed on the testing set, as it would bias the model performance estimates obtained. Additional methods applied to reduce model variance included the use of L2 kernel regularization in the convolutional and fully connected model layers, as well as the inclusion of dropout layers throughout the network. Following development and evaluation on the training and validation datasets, a shallow CNN as shown in Figure 51 was trained and used for testing purposes.

[0193] Data from this exemplary study suggests that the tested biometric sensor device, as well as similar wearable devices, may be used for objective quantification of cranial and ocular muscle movements. The techniques disclosed herein (e.g., for identifying clinically relevant features 50) may be used to identify the capabilities and limitations of a given device based on clinical outcomes. The methods disclosed herein may be used to test the utility of wearable devices in disease populations, to more accurately measure disease progression within participants, to test how wearable device features or data relate to existing PROs, and / or to more accurately measure treatment effects within disease populations. Use of biometric sensor devices in longitudinal studies where disease progression may be measured, such as ongoing natural history studies, may help elucidate which features are most important for quantifying disease effects. Exploratory use of these devices in clinical trials as part of a wearable clinical development strategy may enable more sensitive detection of treatment responses within disease populations. These clinical validation steps may further support strategies to use devices such as the tested biometric sensor devices for passive monitoring purposes. Such monitoring may be performed by obtaining a signal from the signal capture device 10, identifying clinically relevant features 50 based on data collected by the signal capture device 10, and / or using the clinically relevant features 50 to provide a clinical outcome on an ongoing (e.g., continuous) basis (e.g., identification of a disease or disorder and / or a treatment plan based thereon).

[0194] One or more implementations disclosed herein include a machine learning model. The machine learning model disclosed herein may be trained using the data flow 5410 of FIG. 54. As shown in FIG. 54, the training data 5412 may include one or more of a stage input 5414 and a known result 5418 associated with the machine learning model to be trained. The stage input 5414 may be from any applicable source, including data input or output from the components, steps, or modules shown in FIG. 1A, FIG. 1B, FIG. 2, FIG. 3, FIG. 4A, and / or FIG. 4B. The known result 5418 may be included for machine learning models generated based on supervised or semi-supervised training. An unsupervised machine learning model may not be trained using the known result 5418. The known result 5418 may include a known or desired output for future inputs similar to or in the same category as the stage input 5414 that does not have a corresponding known output.

[0195] The training data 5412 and the training algorithm 5420 may be provided to a training component 5430, which may apply the training data 5412 to the training algorithm 5420 to generate a machine learning model. According to one implementation, the training component 5430 may be provided with a comparison result 5416 that compares a previous output of a corresponding machine learning model to apply the previous result to retrain the machine learning model. The comparison result 5416 may be used by the training component 5430 to update the corresponding machine learning model. The training algorithm 5420 may utilize machine learning networks and / or models, including, but not limited to, deep learning networks such as deep neural networks (DNN), convolutional neural networks (CNN), fully convolutional networks (FCN) and recurrent neural networks (RCN), probabilistic models such as Bayesian networks and graphical models, and / or discriminative models such as decision forests and maximum margin methods.

[0196] FIG. 55 is a simplified functional block diagram of a computer system 5500 that may be configured as a device for performing the methods disclosed herein, according to an exemplary embodiment of the present disclosure. FIG. 55 is a simplified functional block diagram of a computer system that may generate features, statistics, analytics, and / or other systems, according to an exemplary embodiment of the present disclosure. In various embodiments, any of the systems disclosed herein (e.g., computer system 5500) may be an assembly of hardware including, for example, a data communication interface 5520 for packet data communication. Also, computer system 5500 may include a central processing unit ("CPU") 5502 in the form of one or more processors for executing program instructions 5524. Computer system 5500 may include an internal communication bus 5508 and a storage unit 5506 (ROM, HDD, SDD, etc.) that may store data on a computer-readable medium 5522, while computer system 5500 may receive programming and data via network communication (e.g., over network 110). The computer system 5500 may also have a memory 5504 (such as a RAM) that stores instructions 5524 for performing the methods presented herein, although the instructions 5524 may be stored temporarily or permanently in other modules of the computer system 5500 (e.g., the processor 5502 and / or the computer-readable medium 5522). The computer system 5500 may also include input / output ports 5512 and / or a display 5510 for connecting with input / output devices such as a keyboard, mouse, touch screen, monitor, display, etc. Various system functions may be implemented in a distributed manner on several similar platforms to distribute the processing load. Alternatively, the system may be implemented by suitable programming of one computer hardware platform.

[0197] Program aspects of the technology can be considered as a "product" or "article of manufacture" typically in the form of executable code and / or associated data carried on or embodied in some type of machine-readable medium. "Storage" type media includes any or all of the tangible memory of a computer, processor, etc., various semiconductor memories, tape drives, disk drives, etc., or associated modules thereof, which can provide non-transitory storage at any time for software programming. All or part of the software may possibly be communicated over the Internet or various other telecommunications networks. Such communication may, for example, enable loading of the software from one computer or processor to another, for example, from a management server or host computer of a mobile communications network to a server's computer platform, and / or from a server to a mobile device. Thus, another type of media that can carry software elements includes optical, electrical, and electromagnetic waves, such as those used over various air links, through wired and optical landline networks, across physical interfaces between local devices. Physical elements that carry such waves, such as wired or wireless links, optical links, etc., can also be considered as media that carry software. As used herein, unless limited to non-transitory tangible "storage" media, terms such as computer or machine "readable medium" refer to any medium that participates in providing instructions to a processor for execution.

[0198] Although the methods, devices, and systems of the present disclosure are described with exemplary reference to transmitting data, embodiments of the present disclosure may be applicable to any environment, such as desktop or laptop computers, mobile devices, wearable devices, applications, etc. Also, the embodiments disclosed herein may be applicable to any type of Internet protocol.

[0199] It will be apparent to those skilled in the art that various modifications and variations can be made in the disclosed devices and methods without departing from the scope of the disclosure. Also, other aspects of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the features disclosed herein. It is intended that the specification and examples be considered as illustrative only.

[0200] Aspects of the present disclosure relate to signal-based feature analysis. In one aspect, the present disclosure is directed to a method that includes receiving individual electrical signals generated based on a body part, generating a plurality of extracted features based on the individual electrical signals, and identifying clinically relevant features from the plurality of extracted features, where the clinically relevant features meet a threshold determined based on a clinical outcome.

[0201] The method may also include determining a clinical outcome result by applying the clinically relevant features. The clinical outcome result is one of a diagnosis or a treatment plan. The individual electrical signal may be generated based on a body electrical signal generated by the body part. The individual electrical signal may be generated based on a movement of the body part. The individual electrical signal may be generated based on a characteristic of the body part. The plurality of extracted features may be based on one or more of an amplitude feature, a zero crossing rate, a standard deviation, a variance, a root mean square, a kurtosis, a frequency, a band power, or a skew. The individual electrical signal may be generated by a wearable device comprising a sensor. In this case, the wearable device may be configured to output a mixed signal, and / or a signal separation module extracts the extracted features from the mixed signal.

[0202] For example, the signal separation module may extract extracted features from the mixed signals by applying one or more of blind signal separation, blind source separation, discrete transform, Fourier transform, integral transform, two-sided Laplace transform, Mellin transform, Hartley transform, short-time Fourier transform (STFT), rectangular mask short-time Fourier transform, chirplet transform, fractional Fourier transform (FRFT), Hankel transform, Fourier-Bros-Iagolnitzer transform, or linear canonical transform. A random forest algorithm may be used to score the extracted features. The threshold may be a random forest threshold, and extracted features having a random forest score equal to or greater than the random forest threshold may be identified as clinically relevant features. The threshold may be a confidence threshold, and extracted features having a confidence score equal to or greater than the confidence threshold may be identified as clinically relevant features. The confidence score may be based on one or more of Spearman correlation, intraclass correlation (ICC), covariance (CV), area under the curve (AUC), clustering, or Z-score.

[0203] In another aspect, the present disclosure is directed to a system including a wearable device including a plurality of sensors, a processor, and a computer-readable data storage device storing instructions that, when executed by the processor, cause the system to obtain from the wearable device electrical activity information of a subject detected by the plurality of sensors, and identify a clinically relevant characteristic based on the electrical activity information.

[0204] The system may be further configured to classify the clinically relevant features as one or more diseases, determine a disease of the subject based on the one or more diseases, determine an extent of the disease, and / or determine a treatment plan based on the extent of the disease. The plurality of sensors may include an electroencephalography (EEG) sensor, an electrooculography (EOG) sensor, an electromyography (EMG) sensor, an image sensor, and / or an eye tracking sensor. The clinically relevant features may be identified using a machine learning algorithm.

[0205]

Claims

1. A method for determining a clinical outcome, comprising: activating the wearable device to generate an electrical signal based on the detection of the body part; receiving the electrical signal generated based on the detection of the body part; applying statistical filtering techniques to generate a plurality of extracted features based on the received electrical signals; identifying clinically relevant features from the plurality of extracted features, the clinically relevant features satisfying a threshold determined based on a clinical outcome; determining that a number of the identified clinically relevant features that satisfy the threshold number exceeds a confidence threshold number associated with the clinical outcome; determining a clinical outcome based on a number of the identified clinically relevant features that satisfy the threshold exceeding the confidence threshold number; A method for providing the above.

2. The method of claim 1 , wherein the clinical outcome result is one of a diagnosis or a treatment plan.

3. The method of claim 1, wherein the electrical signal is generated based on a body electrical signal generated by the body part.

4. The method of claim 1, wherein the electrical signal is generated based on movement of the body part.

5. The method of claim 1, wherein the electrical signal is generated based on characteristics of the body part.

6. The method of claim 1 , wherein the plurality of extracted features are based on one or more of the following features: amplitude features, zero-crossing rate, standard deviation, variance, root-mean-square, kurtosis, frequency, band power, or skew.

7. The method of claim 1, wherein the electrical signal is generated by a sensor associated with the wearable device.

8. The method of claim 1 , wherein the wearable device is configured to output a mixed signal.

9. The method of claim 8, wherein signal separation is performed to extract the plurality of extracted features from the mixed signal.

10. The method of claim 9, wherein the signal separation for extracting the plurality of extracted features from the mixed signal comprises one or more of blind signal separation, blind source separation, discrete transform, Fourier transform, integral transform, two-sided Laplace transform, Mellin transform, Hartley transform, short-time Fourier transform (or short-term Fourier transform) (STFT), rectangular mask short-time Fourier transform, chirplet transform, fractional Fourier transform (FRFT), Hankel transform, Fourier-Bros-Iagolnitzer transform, or linear canonical transform.

11. The method of claim 1 , wherein a random forest algorithm is used to score the plurality of extracted features.

12. 12. The method of claim 11, wherein the threshold is a random forest threshold, and the plurality of extracted features having a random forest score equal to or greater than the random forest threshold are identified as clinically relevant features.

13. The method of claim 1 , wherein the threshold is a confidence threshold, and the plurality of extracted features having a confidence score equal to or greater than the confidence threshold are identified as clinically relevant features.

14. 14. The method of claim 13, wherein the reliability score is based on one or more of Spearman correlation, intraclass correlation (ICC), covariance (CV), area under the curve (AUC), clustering, or Z-score.

15. determining a treatment plan based on the determined clinical outcome; displaying the treatment plan on a display; The method of claim 1 further comprising:

16. The method described in claim 15, wherein the electrical signals are generated while a subject is performing facial movements, and the treatment plan for treating an identified neuromuscular disorder is determined based on the facial movements. determining a Z-score for the plurality of extracted features; and displaying a heat map of the Z-scores ordered based on the Z-scores; and The method of claim 1 further comprising:

18. 1. A system comprising: A wearable device including a sensor; A processor; A computer readable data storage device storing instructions that, when executed by the processor, activating the wearable device to generate electrical activity information based on detection of the subject's body part; acquiring, from the wearable device, information of the electrical activity of the subject detected by the sensor; applying a statistical filtering technique to generate a plurality of extracted features based on the electrical activity information; identifying a clinically relevant feature based on the electrical activity information, the clinically relevant feature meeting a threshold determined based on a clinical outcome; determining that a number of the identified clinically relevant features that satisfy the threshold exceed a confidence threshold number associated with the clinical outcome; determining a clinical outcome based on a number of the identified clinically relevant features that satisfy the threshold exceeding the confidence threshold number; the computer readable data storage device storing the instructions for causing the system to execute the A system comprising:

19. The system of claim 18, further configured to classify the clinically relevant features as one or more diseases.

20. The system of claim 19, further configured to determine a disease of the subject based on the one or more diseases.

21. The system comprises: determining the extent of the disease; Determine a treatment plan based on the extent of the disease 21. The system of claim 20, further configured to:

22. 20. The system of claim 18, wherein the sensor is at least one of an electroencephalography (EEG) sensor, an electrooculography (EOG) sensor, an electromyography (EMG) sensor, an image sensor, or an eye-tracking sensor.

23. The system of claim 18 , wherein the clinically relevant features are identified using a machine learning algorithm.

Images